			What is XC_NEWS?
			================

XC_NEWS.EXE is a network news client program, inspired by X_NEWS from
Karl-Heinz Weiss (khweis@mvmpc9.ciw.uni-karlsruhe.de) as used in the
SLIP_IT! suite of programs.

It provides the following:

1)  A 'getfile' option so that complete news articles can be retrieved
from a known <Message-ID>.

2)  A history of retrieved article IDs is maintained so that duplicate
or cross-posted articles are fetched only once.

3)  An overview of all new news is fetched prior to any articles, so that
it can be processed automatically against a 'killfile', so that only articles
of interest will be fetched. This overview is itself saved as a pseudo-news-
item that can be viewed by your offline newsreader.

4)  Multiple killfiles are supported, up to one for each group. This uses
public domain code by Rich Saltz (rs@missor.TMC.COM in 1986) as used in KA9Q,
but always processes *ALL* killfile lines in sequence to arrive at a
fetch/nofetch decision.

5)  A 'newgroups' option which maintains a file of new newsgroups created
since the last login.


Note: XC_NEWS requires the TCP/IP, scripting and packet-driver support
provided by SLIP_IT! and will not run stand-alone.


Configuration
-------------

The options are controlled from within XC_NEWS's configuration file X_NEWS.RC.
Here is a quick tutorial:


Suppose you are interested in the following newsgroups -

alt.binaries.pictures.gibbons
alt.politics.speciesism
rec.trees.climbing
comp.os.chimpdos.announce
alt.binaries.sounds.whoop.whoop.whoop
sci.energy.brachiation
soc.religion.grooming
soc.zoos

Then simply create the file X_NEWS.RC with the newsgroups listed,
like this:

# X_NEWS.RC
# All comment lines begin with a '#' and will be ignored
#
alt.binaries.pictures.gibbons
alt.politics.speciesism
rec.trees.climbing
comp.os.chimpdos.announce
alt.binaries.sounds.whoop.whoop.whoop
sci.energy.brachiation
soc.religion.grooming
soc.zoos
#
# end

When SLIP_IT! runs and invokes XC_NEWS, the file will be processed,
and a new version of it written back to disk (the old one is saved
as XC_NEWS.BAK) and might look like this:

# X_NEWS.RC
# All comment lines begin with a '#' and will be ignored
#
alt.binaries.pictures.gibbons 9244
alt.politics.speciesism 588431
rec.trees.climbing 550
comp.os.chimpdos.announce 196
alt.binaries.sounds.whoop.whoop.whoop 11217
sci.energy.brachiation 318493
soc.religion.grooming 380912
soc.zoos 645001
#
# end

Each group has its last-encountered article number put next to it. Since no
number was there in the original file, the most recent 'new_max' articles
are fetched from each group. New_max is set within X_TCP.CFG and
is 100 by default. So up to 100 articles are fetched from each group.

This may not be a good idea. Some of the articles in these groups are
very big. So to begin with (and with any new group added subsequently)
it might be wise to limit the number by using a negative value as
the last-encountered article number. Thus (for example) -25 would
mean 'fetch the most recent 25 articles in this group', as follows:

# X_NEWS.RC
#
alt.binaries.pictures.gibbons -10
alt.politics.speciesism -20
rec.trees.climbing -5
comp.os.chimpdos.announce -30
alt.binaries.sounds.whoop.whoop.whoop -10
sci.energy.brachiation -100
soc.religion.grooming -100
soc.zoos -30
#
# end

This will work just once, then the numbers will be overwritten as before
to show the most recent article number encountered. For groups that are not
very busy or typically have small articles, this wouldn't matter.
But some groups are very busy, and have enormous articles (typically
binary broups) so some other method needs to be used to be more selective


Killfiles
---------

A killfile is a file that contains one or more patterns which are used
by XC_NEWS as a filter to ignore undesirable postings in certain groups,
and to grab more interesting postings.

When XC_NEWS looks at what articles are available in a newsgroup, it
always gets a quick summary of each article (from the current article
number listed in X_NEWS.RC up to the most recent one posted), by means of
the XOVER nntp command. This summary (or *overview*) is always saved and is
readable as a news-article itself with your news reader. The summary
contains details of who each article is from, what the subject is,
how big it is (lines and bytes) and its Message-ID (which is used
with a history file to avoid duplicate fetchings of cross-posted
articles). Each overview line has the following structure:

<#> <Subject> <From> <Date> <Message-ID> <XRefs> <Bytes> <Lines>

Each field is separated by a TAB character. An example (with line breaks
between some fields for clarity):

36441	Re: Repost of eating habits FAQ
mighty@joe.young.org (Joe Young)	23 Mar 1995 22:40:55 GMT
<4kp0tu$22x0@joe.young.org>		562	3


The killfile has these special characters: '*' matches any length of text,
[0-9] matches a numeric character, [a-z] matches a lowercase alpha
character, and so on. A '!' in the first column of a line means 'not',
and a '#' in the first column means the line is ignored. Also, if the
first two characters are '%L' or '%B' then special actions are performed
(see below). All other characters (except newline) are used for
pattern-matching, even whitespace.

Thus if you did not like any articles posted on the subject of
'Fruit series', in alt.binaries.pictures.gibbons then you could
create a killfile called (say) ABPG.KIL and put in it the following:

# ABPG.KIL
#
*Fruit series*
#
# end

Then you need to tell XC_NEWS about it by means of a '%' directive
in X_NEWS.RC:

# X_NEWS.RC
#
# Here's the killfile:
#
%killfile abpg.kil
alt.binaries.pictures.gibbons 9244
alt.politics.speciesism 588431
...

All groups following %killfile will use ABPG.KIL to filter out unwanted
articles, until either another %killfile is encountered, or until
the directive %nokillfile is found (which discards any killfile settings
present).

A more sophisticated killfile might look like this:

# KILLIT.
#
# The next '*' matches everything, meaning no articles are fetched:
*
# But some articles on the subject 'Tea Party' are interesting:
!*Tea Party*
# but not 'repost', or 'get-a-life'
*repost*
*get-a*
# But... missed some vital stuff from 'chestthumper@bower.org' and 'K. Kong'
# so we want these:
!*chestthumper@bower*
!*K. Kong*
#
# You get the idea.



Killing Files that are Too Big
------------------------------

To kill articles that are simply too big, in X_NEWS.RC use:

%killlinesover 100	# This will not fetch any article over 100 lines long
alt.big-articles -10	# fetch the last 10 from here, provided not so big
%nokilllinesover	# turn off %killlinesover directive

%killbytesover 2048	# This will not fetch any article over 2048 bytes long
rec.grin 55263		# 
%nokilllinesover	# turn off %killbytesover directive
#
# Note that X_NEWS.RC will be rewritten to disk with article numbers etc
# updated to show the last valid values, and that trailing comments after
# such fields (as in the above) may be lost.
#
# End.


An alternative would be to put these size limitation specifiers actually
within a killfile. This has the advantage of refining the kill/nokill
decision in combination with other patterns:

# Another Killfile
#
# Kill articles with over 100 lines:
%L 100
# Unless it's this:
!*Flint-impact fabrication technology - Latest*
# But kill any article so far with over 20000 bytes anyway:
%B 20000
# etc...


Retrieving Killed Articles
--------------------------

If you find from looking at the news summary that there is an article
that you want to see but it has been killed, then it can be fetched
explicitly the next time XC_NEWS runs if its Message-ID is put into
a 'getfile'. The name of the getfile can be specified in X_TCP.CFG
but by default is 'GET'. If the file does not exist or contains no
recognisable Message-IDs then no action will be taken. However, any
line beginning with '<' or 'Message-ID: <' and having a '>' somewhere
will define a Message-ID to be fetched. The article is not tested
against any killfile or the history file - it is assumed to be wanted.
However, your newsreader may reject the article as a duplicate if
an article with that same Message-ID has previously been imported
into your newsreader (say a corrupt article from a different news
server).

The getfile is modified so that successfully-fetched articles are
not fetched again on subsequent runs (a '#' is put in the first
column to effect this), and the old getfile is renamed with a
'.BAK' extension.



Other Commands in X_NEWS.RC
---------------------------

The 'newgroups' command will generate a list of the latest groups:

# Fetch a list of new groups created since 10 March 1995 6:10:15pm and add it
# to the end of file 'newlist.txt':
#
%newgroups 950310 181015 newlist.txt
#


NNTP Commands
-------------

These are marked by the character '$' in the first column of lines
in X_NEWS.RC. These send commands direct to the newsserver (see
RFC 977). A ' >' character sequence in the line will send the response
from the server into a named file. Eg:

$LIST > active.lst

This will get a list of all newsgroups from the server and put it in
'active.lst'. The command will be commented-out with a '#' when
X_NEWS.RC is updated so that these commands are executed once only
(unless you re-edit X_NEWS.RC of course). All responses from the
server from '$' commands are assumed to terminate with a '.'


History and Log Files
---------------------

These will grow indefinitely if left to themselves, and performance
of XC_NEWS will suffer if the history file becomes large. The files
can be 'tailed' or deleted from time to time without serious consequence.


List of Configurable Parameters
-------------------------------

This is a list of keywords which can be put in X_TCP.CFG and their default
values. Case is unimportant; space, TAB or '=' can be used to separate
parameters from their values:

logfile debug.log	# filename used for logging XC_NEWS session
timeout 60		# news server response timeout in seconds
newsrc x_news.rc	# XC_NEWS newsgroups configuration file
getfile get		# 'getfile' name
newgroupsfile newgrps	# Newgroups file name if undefined
historyfile history	# Message-ID list of retrieved articles
t_max 3600		# Max time in seconds before quitting
new_max 100		# Number of new articles fetched if undefined
msg_max 5000		# Max number of articles fetched before quitting
cmd_stack 4		# Number of commands sent to server in advance
keepbuf 0		# Whether posted articles are deleted or not


Epilogue
--------

So there we are. XC_NEWS was written for DOS, but may one day find itself
ported to Linux, or even Windows...

--
Ian Hall  ian@station.airtime.co.uk
