NAME

Plucene::Analysis::Tokenizer - base class for tokenizers

SYNOPSIS

        my $tokenizer = Plucene::Analysis::Tokenizer::Subclass
                ->new({ reader => $reader });

This is an abstract base class for tokenizers.

A Tokenizer is a TokenStream whose input is a Reader.

        my $tokenizer = Plucene::Analysis::Tokenizer::Subclass
                ->new({ reader => $reader });

This will create a new tokenizer.

This must be defined in a subclass

Close the input reader.

2005-09-05

perl v5.40.2

Visit the GSP FreeBSD Man Page Interface.
Output converted with ManDoc.