NAME

DecodeShortURLs - Expand shortened URLs

SYNOPSIS

  loadplugin    Mail::SpamAssassin::Plugin::DecodeShortURLs

  url_shortener bit.ly
  url_shortener go.to
  ...
  body HAS_SHORT_URL          eval:short_url()
  describe HAS_SHORT_URL      Message contains one or more shortened URLs

  body SHORT_URL_CHAINED      eval:short_url_chained()
  describe SHORT_URL_CHAINED  Message has shortened URL chained to other shorteners

DESCRIPTION

This plugin looks for URLs shortened by a list of URL shortening services and upon finding a matching URL will connect using to the shortening service and do an HTTP HEAD lookup and retrieve the location header which points to the actual shortened URL, it then adds this URL to the list of URIs extracted by SpamAssassin which can then be accessed by other plug-ins, such as URIDNSBL.

This plugin also sets the rule HAS_SHORT_URL if any matching short URLs are found.

This plug-in will follow 'chained' shorteners e.g. from short URL to short URL to short URL and finally to the real URL

If this form of chaining is found, then the rule 'SHORT_URL_CHAINED' will be fired. If a loop is detected then 'SHORT_URL_LOOP' will be fired. This plug-in limits the number of chained shorteners to a maximim of 10 at which point it will fire the rule 'SHORT_URL_MAXCHAIN' and go no further.

If a shortener returns a '404 Not Found' result for the short URL then the rule 'SHORT_URL_404' will be fired.

If a shortener returns a '200 OK' result for the short URL then the rule 'SHORT_URL_200' will be fired.

This can cover the case when an abuse page is displayed.

NOTES

This plugin runs the check_dnsbl hook with a priority of -10 so that it may modify the parsed URI list prior to the URIDNSBL plugin which runs as priority 0.

Currently the plugin queries a maximum of 10 distinct shortened URLs with a maximum timeout of 5 seconds per lookup.

ACKNOWLEDGEMENTS

A lot of this plugin has been hacked together by using other plugins as examples. The author would particularly like to tip his hat to Karsten Bräckelmann for his work on GUDO.pm, the original version of this plugin could not have been developed without his code.

PRIVILEGED SETTINGS

url_shortener_cache_type (default: none)

The specific type of cache type that is being utilized. Currently only sqlite is configured however plans to support redis cache is planned.

Example: url_shortener_cache_type sqlite

url_shortener_cache_dsn (default: none)

The dsn to a database file to write cache entries to. The database will be created automatically if is does not already exist but the supplied path and file must be read/writable by the user running spamassassin or spamd.

Note: You will need to have the proper DBI version of the cache type installed.

Example:

url_shortener_cache_dsn dbi:SQLite:dbname=/tmp/DecodeShortURLs.sq3

url_shortener_cache_username (default: none)

The username that should be used to connect to the database.

url_shortener_cache_password (default: none)

The password that should be used to connect to the database.

url_shortener_cache_ttl (default: 86400)

The length of time a cache entry will be valid for in seconds. Default is 86400 (1 day).

NOTE: you will also need to run the following via cron to actually remove the records from the database:

echo "DELETE FROM short_url_cache WHERE modified < NOW() - "ttl"; | sqlite3 /path/to/database"

NOTE: replace "ttl" above with the same value you use for this option

url_shortener_loginfo (default: 0 (off))

If this option is enabled (set to 1), then short URLs and the decoded URLs will be logged with info priority.

max_short_urls (default: 10)

The max depth of short urls that will be chained until it stops looking further.