http_get - get the contents of an http URL
] [-r referrer
] [-a username:password
] [-a header value
fetches an http URL and dumps the contents to stdout. It does
not do gopher, ftp, file, news, or any other type of URL, only http. It can be
configured to do https fetches as well.
The -t flag specifies a timeout in seconds. If that much time passes with no
activity, the fetch is aborted. The default is 60 seconds.
The -r flag specifies a referrer header to send. Some tightly-clenched web
admins like to jigger their server to only return files if the proper referrer
The -u flag specifies a User-Agent header to send. Some *really*
tightly-clenched web admins like to jigger their server to only return files
if a well-known User-Agent is given.
The -a flag lets you do Basic Authentication.
The -c flag lets you pass in a cookie.
The -h flag lets you pass in one extra header.
The -v flag is for debugging; it tells http_get to show the HTTP headers as well
as the rest of the response.
If the fetch is successful (HTTP status 200), the command's exit status is 0.
Otherwise the exit status is the HTTP status code. However, since a Unix
command exit status is only one byte, the HTTP codes show up mod 256. Adding
to the confusion, csh interprets the status byte as signed, while in other
shells it is unsigned. Anyway, here is a table of some common status values:
HTTP sh csh
200 0 0
302 46 46
304 48 48
400 144 -112
401 145 -111
403 147 -109
404 148 -108
408 152 -104
500 244 -12
501 245 -11
503 247 -9
Originally based on a simple version by Al Globus <email@example.com>.
Debugged, prettified, and extended by Jef Poskanzer