Last-Modified and ETag support
If you use a local cache (see the Cache parameter to fetch), URI::Fetch will keep track of the Last-Modified and ETag headers from the server, allowing you to only download pages that have been modified since the last time you checked.
Proper understanding of HTTP error codes
Certain HTTP error codes are special, particularly when fetching syndication feeds, and well-written clients should pay special attention to them. URI::Fetch can only do so much for you in this regard, but it gives you the tools to be a well-written client.
The response from fetch gives you the raw HTTP response code, along with special handling of 4 codes:
If you make a request using a cache and get back a 304 response code (Not Modified), then if the content was returned from the cache, then is_success() will return true, and $response->content will contain the cached content.
URI::Fetch->fetch($uri, CW%param)Fetches a page identified by the URI $uri.
On success, returns a URI::Fetch::Response object; on failure, returns undef.
%param can contain:
o LastModified o ETag
LastModified and ETag can be supplied to force the server to only return the full page if its changed since the last request. If youre writing your own feed client, this is recommended practice, because it limits both your bandwidth use and the servers.
If youd rather not have to store the LastModified time and ETag yourself, see the Cache parameter below (and the SYNOPSIS above).
If youd like URI::Fetch to cache responses between requests, provide the Cache parameter with an object supporting the Cache API (e.g. Cache::File, Cache::Memory). Specifically, an object that supports $cache->get($key) and $cache->set($key, $value, $expires).
If supplied, URI::Fetch will store the page content, ETag, and last-modified time of the response in the cache, and will pull the content from the cache on subsequent requests if the page returns a Not-Modified response.
o UserAgent o NoNetwork
Optional. Controls the interaction between the cache and HTTP requests with If-Modified-Since/If-None-Match headers. Possible behaviors are:
false (default) If a page is in the cache, the origin HTTP server is always checked for a fresher copy with an If-Modified-Since and/or If-None-Match header. 1 If set to 1, the origin HTTP is never contacted, regardless of the page being in cache or not. If the page is missing from cache, the fetch method will return undef. If the page is in cache, that page will be returned, no matter how old it is. Note that setting this option means the URI::Fetch::Response object will never have the http_response member set. N, where N > 1 The origin HTTP server is not contacted <B>ifB> the page is in cache <B>andB> the cached page was inserted in the last N seconds. If the cached copy is older than N seconds, a normal HTTP request (full or cache check) is done. o ContentAlterHook
Optional. A subref that gets called with a scalar reference to your content so you can modify the content before its returned and before its put in cache.
For instance, you may want to only cache the <head> section of an HTML document, or you may want to take a feed URL and cache only a pre-parsed version of it. If you modify the scalarref given to your hook and change it into a hashref, scalarref, or some blessed object, that same value will be returned to you later on not-modified responses.
Optional. A subref that gets called with the URI::Fetch::Response object about to be cached (with the contents already possibly transformed by your ContentAlterHook). If your subref returns true, the page goes into the cache. If false, it doesnt.
o Freeze o Thaw
Optional. Subrefs that get called to serialize and deserialize, respectively, the data that will be cached. The cached data should be assumed to be an arbitrary Perl data structure, containing (potentially) references to arrays, hashes, etc.
Freeze should serialize the structure into a scalar; Thaw should deserialize the scalar into a data structure.
By default, Storable will be used for freezing and thawing the cached data structure.
Optional. A boolean that indicates a URI::Fetch::Response should be returned regardless of the HTTP status. By default undef is returned when a response is not a success (200 codes) or one of the recognized HTTP status codes listed above. The HTTP status message can then be retreived using the errstr method on the class.
URI::Fetch is free software; you may redistribute it and/or modify it under the same terms as Perl itself.
Except where otherwise noted, URI::Fetch is Copyright 2004 Benjamin Trott, email@example.com. All rights reserved.
|perl v5.20.3||URI::FETCH (3)||2015-07-04|