|As detailed below.|
|The name of the outgoing spool queue. If not specified, the value of the NNTPSERVER environment variable will be used.|
|The hostname of the remote NNTP server to connect to. If not specified, the name of the spool will be used.|
The port used for connecting to
hostname. Will usually not need to be specified, and port "nntp" will be assumed.
The port name "telnet" will receive special treatment.
-f folder, --posted folder Posted articles will be appended to the standard mail folder indicated. Note that the article will be added to the folder the first time posting of an article is attempted. The actual status of the posting will appear in the log file. -l logfile, --log logfile Log information about successfully posted articles to the file indicated. --stat statfile A summary of article exchange statistics will be logged to the file indicated. --scanlogs Obey locks set by the INN scanlogs utility. This is currently only meaningfull for use with a modified scanlogs that understands newsx logs.
-a authfile, --auth authfile If specified, the username and password combination found in authfile will be sent to the server as part of the simple authorization protocol when the connection is first made. The file should consist of a line with two blank-separated words, the first being the username, the second the password. Do ensure that this file only has read access only for those users that need to know. --authgeneric Authentication with the server is performed using AUTHINFO GENERIC. This works by spawning a child process via /bin/sh string, where string is set in the environment variable NNTPAUTH. This process is expected to handle the authentication portion of the connection. -r, --reader A MODE READER command is sent to the remote server. This is used when connecting to remote INN hosts that needs it, and is the default state when using --ihave. With ihave, the command is issued between the ihave and fetch phase, otherwise the command is issued initially. --no-reader No MODE READER command is sent to the remote server. This is the default state when not using --ihave. --readbeforeauth Send the MODE READER command before attempting to authenticate via an AUTHINFO command. Otherwise, it is sent after the authentication. This option will obviously have no effect unless you also specify both --reader and one of --authgeneric or --auth authfile. -w chatfile, --chat chatfile Use chatfile to control any special connect sequence that may be required before the NNTP protocol is connected. Only implemented for the "telnet" protocol. -x command, --exec command If specified, the command will be executed after the host port has been opened. This option is used in rare cases when special tricks has to be employed to get an NNTP connection activated. -y program, --connect program If specified, connect via a bidirectional pipe to program instead of to a socket. The hostname and port arguments will be ignored.
-p, --no-post Do not attempt to do any posting. --ihave Use NNTP IHAVE mechanism for submitting news messages, instead of the default POST. If this is for forwarding news in general, and not necessarily news generated locally, the setting of the --fail and --bounce option should be carefully reviewed. -m, --no-msgid Remove any message-ID header field from the posted message. This should cause the receiving end to generate a fresh message-ID. It should never been used together with --ihave. -k, ----keep-path Keep the Path when posting. In posting mode, the Path is by default stripped together with the NNTP-Posting-Host and Xref header items, so that the path of the news article as posted will appear to start at the actual posting host. With --ihave, the path is always included. --continue Ignore (but report) errors about non-existing outgoing spools, and continue with news fetching instead. --max-path len
All outgoing articles are checked to see if the length of
their paths are not too long. This is an extra precaution against inadvertently submitting foreign articles for posting. This option sets the maximum path length allowed. The default value is 1, consistent with a simple, local, newsspool. To disable this feature, set it to an impossibly large value, like 999.
It can also be useful to specify that only a subset of groups should be fetched for the occasion, e.g:
-g, --no-fetch Do not attempt to fetch any news articles. -e end_tag, --end end_tag Specify a tag in the incoming host state file, see in.hosts(5). News fetching will end when a corresponding tag line has been reached. The tag line consists of a colon followed by the tag itself. This is a useful tool for dividing the newsgroups into several categories. Only a few newsgroups may be fetched during prime time, whereas the full list can be fetched once every night, for instance. -W winsize, --window winsize Specify the window size with respect to how many outstanding NNTP requests will be allowed at any one time. The default setting is 10. A value of 0 will disable windowing (aka. streaming) completely, making debugging easier. Increasing the window can significantly improve the article transfer speed. The effect is particulary pronounced over slow links and when many of the articles are already present in the local spool, where an improvement in speed of over 10 times has been observed compared to running with windowing disabled. All this at the penalty of an increased host newsserver load, obviously. -b minspool, --size minspool Specify the minimum incoming spool size. After every new group, if the spool file has reached the byte size specified, a new spool batch will be made. Setting this value to 1 will cause every newsgroup to be placed in a separate spool. --rnews Pipe incoming messages directly to rnews instead of placing them into the in-coming spool. --pipe-to program Pipe incoming messages to program instead of placing them into the in-coming spool. A separate program and pipe is initiated for each incoming message. --rnews-to program Like --pipe-to, but the pipe is continuous for each session, with article seperation is in rnews style. -z, --sync Omit actual pulling of news articles. Instead, simply update the host incoming state file to synchronize with the currently highest available article numbers from the server in question. This option can be useful when a fresh local spool is initiated. No article posting will be attempted. See also in.hosts(5). -Z, --syncnew Omit actual pulling of news articles in previously unseen groups only. The net effect is that newly added groups will start fetching from now on, instead of from the first article available at the host server. --reset Ignore the latest article information in the host incoming state file and attempt fetching all articles available from the server, subject to the normal history lookup constraints. This option is useful if the remote spool has been reconfigured. See also in.hosts(5). --filter filterprog Enable message filtering via the program filterprog. Uses the highwind interface model, where each article is given to the filter in NNTP-fashion, and the filter responds with a 335 to accept, and a 435 to reject a message. --mfilter filterprog Enable filtering by message ID. The filterprog will be invoked once for each unseen message, with the message ID as a command line argument. A non-zero return status will prevent the message from being transferred. This function is obsolete, and will probably be removed in later versions. -X num, --maxnew num Fetch at most num articles from each new and previously unseen newsgroup. --maxart num Fetch at most num articles from each newsgroup. Note that this might cause articles to be silently lost. --no-path Unless the exclusion pattern is set to newsx, all incoming articles are checked to see if their path is consistent with the exclusions given in /usr/local/news/etc/newsfeeds. This option omits this check. --forget-inactive Remove information in the incoming hosts state file about newsgroups that are removed from the /usr/local/news/db/active file. See also in.hosts(5). --minfree N Dont fetch news if there are less than N kbytes free space in the news spool. --group list Specify pattern for groups to be fetched that will apply in addition to the ones in /usr/local/news/etc/newsfeeds. The syntax is basically the same as for INN newsfeeds(5). It is recommended to put list in single quotes, since shells may otherwise treat the ! character in strange ways. For a list that specifies only negations, an initial * clause will be assumed. This options is particularily useful to specify unsymmetric behaviour, i.e. groups where there will be outgoing traffic, but no incoming articles. For instance, to explicitly avoid fetching of control groups, use:
It is recommended to put the argument for --groups in single quotes.
-c, --cnews Set to C News mode as opposed to INN. Controls details regarding handling of spool and lock files. -i, --inn Set to INN mode as opposed to C News. Controls details regarding handling of spool and lock files. --newline Uses single newline character as line terminator instead of the carriage return, line feed sequence. -v, --version Display program version. Include this when reporting bugs to http://www.kvaleberg.com/bug/ --help Give a very brief usage summary. --no-ps Do not update the process status display.
Note: Unless otherwise specified, these options are in addition to the usual actions and do not imply --no-post or --no-fetch. Specify those switches if desired.
--list listname Obtain a list of newsgroups from the newsserver to listname. The list consists of one line per newsgroup, containing the newsgroup name, a blank, and a letter showing the group status (usually y for an active group, m for a moderated group). If the first character is a |, listname is assumed to be a program path which will receive the list as standard input (you will need to put the entire argument in quotes to escape the usual shell interpretation of |). If it is a single -, the list will appear on the standard output. Otherwise it will be assumed to be a filename. --newlist listname Obtain a list of newsgroups not currently present on the local server from the newsserver to listname. The list is also subject to the limitations of the sys or newsfeeds file. The operation is otherwise as for the --list option. --desc descname Obtain a list of newsgroups descriptions from the newsserver to descname. Only descriptions for locally active newsgroups relevant to the newsfeed in question will be fetched. The list consists of one line per newsgroup, containing the newsgroup name, a tab, and a short textual description. A pipe may be specified in descname kist as for the --list option. --alldesc descname Obtain a list of all newsgroups descriptions available from the newsserver to descname. The operation is otherwise as for the --desc option.
-t timeout, --timeout timeout Specify timeout for TCP/IP and lockfile operations, in seconds. The default value is 600. --fail time Specify the maximum age of an article. Whenever an attempt of posting an article fails, and the specified time limit is exceeded, the article will be declared as failed, and a bounce message will be generated. The default unit is hours; the suffixes s for seconds, m for minutes, h for hours, d for days and w for weeks would change this. They can also be combined, e.g. "4h30m". In absence of this option, no fail by age will be performed. Setting the time to zero will bounce messages after the first failed attempt. --bounce addr Specify the destination address for messages bounced due to the fail time being exceeded. If addr is specified as none, no bounce will be generated. The default value, poster, causes messages to be returned to the sender. This setting should definitely only be used in situations where newsx is utilized for posting locally generated news. --attach how Specifies the way the original message is handled for bounced messages: mime Send the original as a Mime attachement. This is the default behaviour. yes Append the original to the mail message itself. no Do not attach the original message.
The following options are not for ordinary use: They would only be used for special configurations where newsx is used as a sort of gateway to inject news messages from other sources.
--inews Pipe incoming messages directly to inews, inject them into the local spool via the same interface as is used for local new messages. --inews-options options Specify command line options for --inews. The leading - must be included. The default inews options are "-hOS". --add-header header Add a specific header to incoming messages. The header will be added exactly as specified, adding a trailing newline. The exception is for the special 4 letter string "Path". This will be replaced by a header line that says "Path: HOSTNAME!not-for-mail" if the incoming message contains no Path:. This can be quite essential for news gateways to prevent injected messages to be retured back to the source.
--profile file Read a newsx profile from file . This file can contain command line options to specify default newsx behaviour. See newsx.conf(5) for an overview of the format. Any option may be used in profile files. Some options are really only meaningfull in profile files: --spoolname spoolname Specify a default spoolname. --hostname hostname Specify a default hostname. --port port Specify a default port for connecting to hostname.
Configuration to the local news system peculiarities is traditionally done at build-time, but it is possible to specify at runtime in various ways.
--config Read and obey the standard news configuration file in /usr/local/news/lib/innshellvars. This allows newsx to adapt to changes in the local newsserver configuration dynamically. --config-is configfile Like --config, except that the configuration filename is specified. A list of files may also be specified, seperated by colons. In this case, the first file on this list that exists will be used. --home newshome Use newshome as the news home directory instead of the default /usr/local/news. A full path must be specified. -s spooldir, --spool spooldir Use spooldir as the news article spool directory instead of the default /usr/local/news/spool/articles. A full path must be specified. --togo togofile Specify the name of the C News togo file for outgoing news. No effect in INN mode. Required only if different from the default value /usr/local/news/spool/outgoing/spoolname/togo. A leading / specifices a full path, a leading . is relative from the current directory, otherwise the name is relative to /usr/local/news/spool/outgoing/spoolname. --batch batchdir Use batchdir as the outgoing news article spool directory instead of the default /usr/local/news/bin. A leading / specifices a full path, a leading . is relative from the current directory, otherwise the name is relative to /usr/local/news/spool/articles. -h historydb, --history historydb Specify the name of the history database file, used for checking if news articles are already present in the spool. Required only when using a name different from the default /usr/local/news/db/history. An empty argument "" will prevent any referrence to the news history database. --active activefile Specify the name of the active newsgroup file for incoming news. Required only if different from the default value /usr/local/news/db/active. A leading / specifices a full path, a leading . specifies a path relative to the current directory, otherwise the name is relative to /usr/local/news. --newsfeeds newsfeedsfile Specify the name of the newsfeed specification file. This will override the default both in C News (sys) and INN (newsfeeds) modes. File name conventions as for --active. --incoming incomingdir Use incomingdir as the incoming news article spool directory instead of the default /usr/local/news/spool/incoming. Fine name conventions as for --active, except that the default directory is /usr/local/news/spool/articles. --inhosts inhostsdir Use inhostsdir as the news host active file directory instead of the default /usr/local/news/spool/inhosts. Name conventions as for --incoming. See also in.hosts(5). --hostconfig hostconfigdir Use hostconfigdir as the directory for the host specific profile. instead of the default /usr/local/news/spool/articles/in.hosts. Name conventions as for --incoming. See also newin.hosts(5) and newsx.conf(5).
--debug level Set debug level. Any level different from zero diverts reporting to stderr instead of syslog, and turns on extra debugging output. Level 2 will output a . for every new article. Level 3 will log all NNTP commands. Level 4 will produce the same state information that is available via the process status display. Going to level 5 will produce additional debug information, level 6 will include history database debug, while level 7 also will show all communications over the NNTP socket, article content included. -d, --verbose Enable debugging, and sets the debug level according to the number of times it is specified. -n, --dry-run No-action flag, will "fake" an NNTP connection, and leave the outgoing batch untouched. Probably useful only with --debug for debugging and dry-testing. --verify Used together with --configor --config-is, causes the configuration to be verified, but not adapted to. -o, --keepold Keep the previous outgoing spool in an ".old" file. This might seem like a useful option for ordinary use, but the problem is that a non-empty file might cause a news watch program to believe there is a stale outgoing spool. -u, --no-force After the timeout of lockfile operations, just give up and do not attempt to unlock the lockfiles and remove the stale process. --no-queue Do not queue up for access to a news host. If a lock already exists for the specified host, just give up and dont queue up for it. --locks locksdir Use locksdir as the lock file directory instead of the default /usr/local/news/run. Fine name conventions as for --active, except that the default directory is /usr/local/news. If the string none is supplied, no locking will occur. All this should of course be used only if you really, really know what you are doing. -q "msgid", --enquire "msgid" Enquires whether the msgid is present in the local history database. The msgid should include the angle brackets. Implies --no-post and --no-fetch. --no-hostlock Do not implement the newsx host access lock. Should only be used if you really know what you are doing. --no-next Do not use the NNTP NEXT command. This causes newsx not to attempt to use NEXT for filling out gaps in the article sequence. --missing "num" Tunes the number of missing articles in a row before a NEXT command is issued, instead of a sequential STAT. A value of zero will cause NEXT to be used extensively. The default value is 0 if no window, 2 if a window is specified. --keep-fake Some remote news servers generate faked news articles. Normally, you would want newsx to skip them, but with this option you can actually fetch them.
The -w option specifies that a special connect sequence is required, and that a script for this sequence can be found in the chatfile. This file consists of lines that contains pattern receive and send pairs, separated by blanks. The patterns may be enclosed in quotes. The script is currently only available for the "telnet" protocol.
A typical invocation line would be:
newsx -w chat.acme acme login.acme.net telnet
The file chat.acme could look like this:
# login for acme.net NNTP
$ "exec telnet news.acme.net nntp"
The script will have to be adapted for local conditions, of course.
The special tag ECHO specifies that line echo should be suppressed, required since the host telnet operates in line mode.
Since this file usually contains passwords and other sensitive information, ensure that read access to it is limited.
The ps process status will also show the current newsx status. During the actual news transfer phase, it will show which group and article number that is being fetched. A continuous status can be obtained by;
This feature can be disabled via the --no-ps option.
For INN, the following configuration items will be used:
$INND Signals an INN configuration. $NEWSBIN Directory for INN binaries. $NEWSHOME News home directory. $PATHETC Directory for news configuration files. Defaults to $NEWSHOME if not available. $SPOOL If $SPOOL contains the string "/articles", the incoming host directory $INHOSTS will be formed by replacing it with "/inhosts". Otherwise, "$SPOOL/in.hosts" will be used. $BATCH Outgoing spool. $INCOMING Incoming spool. $ACTIVE Active file. $HISTORY History database. $LOCKS DIrectory for lockfiles. $NEWSFEEDS Newsfeeds file. $EXTENDEDDBZ True is history database in extended format. Only relevant for early INN 2.x where there is no libinn, otherwise ignored. For C News: $NEWSBIN $NEWSCTL Equivalent to INN $NEWSHOME. $ACTIVE will be "$NEWSCTL/active". $HISTORY will be "$NEWSCTL/history". $NEWSFEEDS will be "$NEWSCTL/sys". $LOCKS and $PATHETC will be "$NEWSCTL". $NEWSARTS Equivalent to INN $SPOOL. $BATCH will be "$NEWSARTS/out.going". $INCOMING will be "$NEWSARTS/in.coming". $INHOSTS will be "$NEWSARTS/in.hosts".
The -l log file of posted articles will contain one line for each article. To be compatible with the INN format, each field is separated by a tabstop:
Month Standard 3 letter abbreviation. Date 2 digits HH:MM:SS Local time. Year 4 digits. Spool Name of outgoing spool. <Msgid> Within angle brackets. Filename As it appears in the spool file. Sender The value of the Sender-field, or From-field if no Sender is available. Status "OK" for successfully posted items, "Duplicate" for items already posted. Lines Number of lines in the posting. Status Message status. "OK" if everything went well, otherwise an error message appears.
The incoming spool consists of files with the file name ending by ".t". Each file can contain many articles. Each article is prefixed by the following header:
#! rnews N
The number N is the number of bytes of the article proper, not counting the header line. The article follows after the header line. It appears exactly as received from the news server except that CR/LF sequences are replaced by proper newlines, and the terminating full stop is not included.
Incoming message headers will pass through without modification, with the exception that the text "newsx" will be added if not there already when the exclusion pattern for the newshost is set to "newsx".
Most outgoing message header items will also pass through, altough some header items are handled specially:
From: Will be used for indentifying the sender if everything else fails. Message-ID: Will be used for logging purposes. The --no-msgid will remove this item for messages being posted. NNTP-Posting-Date: NNTP-Posting-Host: Will always be removed for messages being posted. Path: Will be removed for messages being posted, unless the ----keep-path option is given. Reply-to: Will be used for indentifying the sender. Sender: Will be used for indentifying the sender. Xref: Will always be removed from outgoing messages. X-Server-Date: X-Trace: X-Complaints-To: Will always be removed for messages being posted.
newsx returns error codes as follows:0 - Successful completion
1 - General system error
2 - Incorrect arguments supplied.
3 - Error connecting to remote host
4 - NNTP Protocol error
5 - Errors accessing outgoing spool file.
6 - Errors accessing host newsrc file.
7 - Errors accessing incoming spool file.
8 - Errors accessing local active and sys files.
9 - Socket access error (e.g. connection timeout)
10 - Errors accessing authinfo file.
11 - Connect script failure.
12 - Option -q did not find the message ID.
13 - Wait for lock timed out.
Distribution fields are ignored by newsx, and the handling of them left to the news server.
Report any bugs to http://www.kvaleberg.com/bug/
See also in.hosts(5).
/usr/local/news/etc/newsx.conf Newsx standard profile. If it exists, this profile file will be read before the command line arguments are parsed. See newsx.conf(5). /usr/local/news/lib/innshellvars The standard news configuration file. For C News: /usr/local/news/spool/outgoing/spoolname/togo Spool file /usr/local/news/spool/outgoing/spoolname/togo.old Previous spool file /usr/local/news/spool/outgoing/spoolname/LOCKb Lock file /usr/local/news/spool/incoming/.tmp.$$ Incoming spool, temporary file. $$ is the process ID. /usr/local/news/spool/incoming/T.$$.S.t Incoming spool, when finished and ready for newsrun. T is the current Unix time, $$ is the process ID, and, S is a sequence number. /usr/local/news/db/active Local active file, contains a list over all currently active newsgroups in the local spool. This is used as a basis for constructing a list over which newsgroups will actually be requested. /usr/local/news/sys Newsserver configuration file. This is used as a filter against the local active list, deciding which newsgroups which will actually be pulled. The basic format of each specification is:
If the exclusions field is set to newsx, newsx will modify the path of incoming news to include the "newsx" pattern. The site field identifies the remote news source. The grouplist is a comma separated list of group pattern names. A !-prefix signifies exclusions, and the name all matches anything. /usr/local/news/db/history /usr/local/news/db/history.pag /usr/local/news/db/history.dir Local news spool database. For INN: /usr/local/news/spool/outgoing/spoolname Spool file /usr/local/news/spool/outgoing/spoolname.old Previous spool file /usr/local/news/run/LOCK.spoolname Lock file /usr/local/news/db/active Local active file, contains a list over all currently active newsgroups in the local spool. This is used as a basis for constructing a list over which newsgroups will actually be requested. /usr/local/news/etc/newsfeeds Newsserver configuration file. This is used as a filter against the local active list, deciding which newsgroups which will actually be pulled. The basic format of each specification is:
If the exclusions field is set to newsx, newsx will modify the path of incoming news to include the "newsx" pattern. The site field identifies the remote news source. The grouplist is a comma separated list of group pattern names. A !-prefix signifies groups not exchanged, and the name * matches anything. /usr/local/news/db/history /usr/local/news/db/history.pag /usr/local/news/db/history.dir Local news spool article database.
Egil Kvaleberg <firstname.lastname@example.org>
RFC-977 - Network News Transfer Protocol (NNTP)
RFC-1036 - Usenet Article Format