GSP
Quick Navigator

Search Site

Unix VPS
A - Starter
B - Basic
C - Preferred
D - Commercial
MPS - Dedicated
Previous VPSs
* Sign Up! *

Support
Contact Us
Online Help
Handbooks
Domain Status
Man Pages

FAQ
Virtual Servers
Pricing
Billing
Technical

Network
Facilities
Connectivity
Topology Map

Miscellaneous
Server Agreement
Year 2038
Credits
 

USA Flag

 

 

Man Pages


Manual Reference Pages  -  LINGUA::EN::SUMMARIZE (3)

.ds Aq ’

NAME

Lingua::EN::Summarize - A simple tool for summarizing bodies of English text.

CONTENTS

SYNOPSIS



  use Lingua::EN::Summarize;
  my $summary = summarize( $text );                    # Easy, no? :-)
  my $summary = summarize( $text, maxlength => 500 );  # 500-byte summary
  my $summary = summarize( $text, filter => html );  # Strip HTML formatting
  my $summary = summarize( $text, wrap => 75 );        # Wrap output to 75 col.



DESCRIPTION

This is a simple module which makes an unscientific effort at summarizing English text. It recognizes simple patterns which look like statements, abridges them, and concatenates them into something vaguely resembling a summary. It needs more work on large bodies of text, but it seems to have a decent effect on small inputs at the moment.

Lingua::EN::Summarize exports one function, summarize(), which takes the text to summarize as its first argument, and any number of optional directives in name => value form. The options it’ll take are:
maxlength Specifies the maximum length, in bytes, of the generated summary.
wrap Prettyprints the summary output by wrapping it to the number of columns which you specify.
filter Passes the text through a filter before handing it to the summarizer. Currently, only two filters are implemented: "html", which uses HTML::TreeBuilder and HTML::FormatText to strip all HTML formatting from a document, and "easyhtml", which quickly (and less accurately) strips all HTML from a document using a simple regular expression, if you don’t have the abovementioned modules. An "email" filter, for converting mail and news messages to easily-summarizable text, is in the works for the next version.
Unlike the HTML::Summarize module (which is quite interesting, and worth a look), this module considers its input to be plain English text, and doesn’t try to gather any information from the formatting. Thus, without any cues from the document’s format, the scheme that HTML::Summarize uses isn’t applicable here. The current scheme goes something like this:

"Filter the text according to the user’s filter option. Split the text into discrete sentences with the Text::Sentence module, then further split them into clauses on commas and semicolons. Keep only the ones that have a (subject very-simple-verb object) structure. Construct the summary out of the first sentences in the list, staying within the maxlength limit, or under 30% of the size of the original text, whichever is smaller."

Needless to say, this is a very simple and not terribly universally effective scheme, but it’s good enough for a first draft, and I’ll bang on it more later. Like I said, it’s not a scientific approach to the problem, but it’s better than nothing (and often better than HTML::Summarize!), and I don’t really need A.I. quality output from it.

AUTHOR

Dennis Taylor, <dennis@funkplanet.com>

SEE ALSO

HTML::Summarize, Text::Sentence, http://www.vancouvertoday.com/city_guide/dining/reviews/barbers_modern_club.html
Search for    or go to Top of page |  Section 3 |  Main Index


perl v5.20.3 SUMMARIZE (3) 2001-02-20

Powered by GSP Visit the GSP FreeBSD Man Page Interface.
Output converted with manServer 1.07.