The Easy Way to Set Up a Local News Server

Introduction

A few months ago, I decided to set up a local news server on my Linux Box,
in order to read off-line the articles. Before then I read
the news directly from the Usenet server of my Internet Provider, at the
detriment of my phone bill...(Here, in France, even the local communications
are rather expensive)

This package (leafnode-0.9.tar.gz, 29 Kb) contains three little programs,
very easy to install, and to use:

"Fetch" can feed a local news server from a remote
Usenet server (university, ISP...).
It also posts the outgoing articles, using classical NNTP requests.

"Leafnode" is a USENET server, run by the inetd
daemon, when a connection in required on the NNTP port, by a newsreader.

"Texpire" is generally run daily from the crontab
to erase the oldest articles from the news spool.

The main advantage of this system, is the transparent way it is
inserted between the remote news server and the local newsreader. The
remote Usenet server sees
Leafnode exactly like a classical newsreader (like tin, trn,
netscape,...) and the local
news reader sees Leafnode just like a USENET server.

I have been happily using Leafnode for several months, and I would like to
describe here the (very simple) steps to install and
configure it. Then I will explain some
hints to use it in a multi-users environment.

I've been using leafnode 0.8 for a few months, but I've recently upgraded
to version 0.9. The process for installing it is the same as the previous
version, but I've had a little problem, maybe due to my version of
make, and I'll describe how I have fixed it.

Compiling the source code

Installing leafnode

Make sure there is a "news" user and a "news" group
on your system.

Then you can type:

# make install

If "make install" complains when making directories in
/var/spool/news/message.id/,
you may have the same problem as I've had. To fix it, I've modified the
Makefile, to insert a part of the leafnode-0.8 Makefile:

First you have to copy the file /usr/lib/leafnode/config.example to
/usr/lib/leafnode/config and edit him, to put the name of your
remote NNTP server in place of:

server = news.hiof.no

Edit the file /etc/inetd.conf, and look for a line like:

nntp stream tcp nowait root /usr/sbin/tcpd in.nntpd

then modify it like this:

nntp stream tcp nowait news /usr/sbin/tcpd /usr/local/sbin/leafnode

Depending on your configuration, there may be no TCP wrapper installed,
so you could have to remove the "/usr/sbin/tcpd" part of this line.

and do

# killall -HUP inetd

First run

Connect to your Usenet provider, and, while being
"root" or "news", run:

$ fetch

You will have to wait for a moment, because Leafnode is asking the
NNTP server the list of all the active groups. Once fetch ends,
run a newsreader, as normal user, and ask him to contact the localhost.
for example, with tin do:

$ export NNTPSERVER=localhost
$ tin -r

(With Netscape you have to put localhost in "Options/Mail
and News/Servers/NNTP server")

You will get the list of all available newsgroups, then choose interesting
ones, and read them. They will appear empty at this time. It's normal.

As root run again fetch. It will download all the content of the
previously read newsgroups. The first downloading will take a while, but
the next will obviously be very much quicker.

There are problems to avoid, especially if there are other users
on your system, or if the leafnode server is on a local network. First
you must make sure of the validity of the headers
in the outgoing posts, but you also need to limit the list of the accessible
newsgroups. Don't
forget that Fetch will download the whole content of a newsgroup if someone
tries to read it. So, be very careful with newsgroups like
alt.binaries.pictures...

With some newsreaders, the "From:" field of the outgoing articles will
be set to myname@my.station.on.my.lan and not
myname@my.internet.provider.com.
With some of them you can configure the "From:" and "Reply-to:" fields,
while the others need you to recompile them.

This problem can be worse if you have a Linux box with several users.
Some of them can have misconfigured newsreader (sometimes on purpose...)
and it may be safer to check the headers of the outgoing articles before
posting them.

Here's a small awk filter which allows a kind of masquerading of the
"From:" line of an article. It will change the line
"From: username@my.station.on.my.lan (user real name)"
to a line
"From: username@my.internet.provider.com (user real name)".

You may also ensure that username is correct (i.e. in a list
of allowed users). The same Perl script will help us to determine the
correct articles. Otherwise it will add a line "*** Wrong From field -
This article must be deleted ***" to the message.

#! /usr/bin/gawk -f
#
# /usr/local/sbin/change_article_from_domain
#
# awk script to change the domain name on the "From:"
# line of outgoing articles.
# If the username is not valid a message will be added
# at the bottom o the file, allowing a 'grep' to delete
# him.
BEGIN {
# replace with the correct domains
local_domain="my.station.on.my.lan"
real_domain ="my.internet.provider.com"
# insert here the name of your users allowed to post articles
# (may be just one)
valid_usernames["user1"]
valid_usernames["user2"]
must_be_deleted=0
}
/^From:/ {
gsub(local_domain, real_domain)
username=substr($2,1,index ($2, "@") - 1)
if (! (username in valid_usernames)) {
must_be_deleted=1
# you can also add a system command
# example : mail to newsmaster with
# the username of the wrong article
}
}
END {
if (must_be_deleted != 0) {
print "*** Wrong From field - This article must be deleted ***"
}
}
{
print
}

This script can be useful if you have up to, say, ten users, otherwise
you'll need to improve it in order to read the list of allowed users in
an otherfile for example.

Now we will execute the above script on all the outgoing articles,
sitting in /var/spool/news/out.going, then delete (or move to another
directory) those with bad usernames.

(The backquote is used to catch the result of grep) Don't forget to create
a /tmp/modified_articles/ directory.
This script cannot prevent the fake "From:" lines, when an authorized user is
hidden behind an other
correct username. This can not be easily done, and if you really don't trust
your users, you'll have to use another Usenet package like Inn or Cnews.

The second important point to check out is the list of fetched newsgroups.
If you haven't got a huge disk
space, it would be better to avoid downloading alt.binaries groups
or alt.2600. for example...
The problem is that fetch will download the content of each newsgroup
corresponding to a file in /var/spool/news/interesting.groups, for
example /var/spool/news/interesting.groups/comp.os.linux.announce

A file in this directory is touched by leafnode every time a user tries
to read the content of the group. Are you
sure that none of your users will try to have a look at
alt.binaries.pictures.erotica...?
So there are two possible solutions:

You can edit the /usr/lib/leafnode/groupinfo file, in order to suppress
the forbidden newsgroups. But this is not a very good solution, because
fetch will re-create it during the next connection.

You can suppress the forbidden groups from
/var/spool/news/interesting.groups.
This can be done by shell script, in many different ways, for example:

CONCLUSION

I think that Leafnode is a very interesting package for those
(most of us) who are running
Linux on a standalone box with intermitent connection to a Usenet server,
or on a small local network
with few users. It's powerful and much simpler to install and to configure
than Inn or Cnews,
designed for bigger sites. Moreover it does not require any maintenance.