GSP
Quick Navigator

Search Site

Unix VPS
A - Starter
B - Basic
C - Preferred
D - Commercial
MPS - Dedicated
Previous VPSs
* Sign Up! *

Support
Contact Us
Online Help
Handbooks
Domain Status
Man Pages

FAQ
Virtual Servers
Pricing
Billing
Technical

Network
Facilities
Connectivity
Topology Map

Miscellaneous
Server Agreement
Year 2038
Credits
 

USA Flag

 

 

Man Pages


Manual Reference Pages  -  PLUCENE::ANALYSIS::TOKEN (3)

.ds Aq ’

NAME

Plucene::Analysis::Token - A term in a field

CONTENTS

SYNOPSIS

DESCRIPTION

A Token is an occurence of a term from the text of a field. It consists of a term’s text, the start and end offset of the term in the text of the field, and a type string.

The start and end offsets permit applications to re-associate a token with its source text, e.g., to display highlighted query terms in a document browser, or to show matching text fragments in a KWIC (KeyWord In Context) display, etc.

The type is an interned string, assigned by a lexical analyzer (a.k.a. tokenizer), naming the lexical or syntactic class that the token belongs to. For example an end of sentence marker token might be implemented with type eos. The default token type is word.

METHODS

    new



        my $token = Plucene::Analysis::Token->new({
                type  => $type,
                text  => $text,
                start => $start,
                end   => $end });



This will create a new Plucene::Analysis::Token object.

Search for    or go to Top of page |  Section 3 |  Main Index


perl v5.20.3 PLUCENE::ANALYSIS::TOKEN (3) 2005-09-05

Powered by GSP Visit the GSP FreeBSD Man Page Interface.
Output converted with manServer 1.07.