Netscape Smart Browsing Explained

Netscape has added a new feature to their Communicator product called "Smart
Browsing". The purpose of this feature is to assist surfers in their
quest for information. One aspect of the smart browsing is called What's
Related. The ability to enable or disable this feature is provided
in the Communicator preferences dialog under
Edit->Preferences->Navigator->Smart Browsing. To enable, select
the toggle box to the left of the "Enable 'What's Related'" option. Make
sure you select "After first use" for the load mode. Once
enabled, a new menu appears to the right of the URL entry box labeled originally
enough "What's Related". When this menu is selected, a list of sites
related to the current entry in the URL box are displayed. For example,
navigate to
http://www.ibm.com. Now
select the What's Related menu. At the time this article was written,
this is what was displayed:

The first section gives a list of supposedly related sites that you may be
interested in surfing to. In reality, this is a list maintained by Netscape
that is based on priority ordering. Speculation would have this priority
based on monetary gains, however, this has not been verified by Netscape.

Now for an explanation on how Netscape pulls this off. It really isn't
that tricky. Every time you select the What's New option, a hidden
http request is made to a Netscape webserver. This request includes
the exact contents of the URL text box. The webserver then does a lookup
of associated entries related to that domain. In our case, it is retrieving
the URL's related to .ibm.com. Since this is simply an http request,
we can actually send our own requests to this server and mimic this functionality
in a manner we can analyze. The hidden http request actually looks like:
http://www-rl.netscape.com/wtgn?url (where
url is the text in the URL text box). In
our case, this is the actual http request:
http://www-rl.netscape.com/wtgn?www.ibm.com.
The actual results for this query are: (again, at the time this article
was written)

As you can see, this is the exact contents of the What's Related menu with
extra HTML tags surrounding it. The What's Related functionality knows
how to parse this information and display it in a convenient menu. You
can, of course, mimic any other request by simply changing the string after
the '?' mark in the http request.

If you are sitting behind a firewall, this may be a small security risk.
Not from an immediate risk of intrusion,
but from a standpoint of information mining. As this article points
out, you are actually sending the URL you are viewing down to Netscape.
Since this is a webserver, their logs will contain at least two
important pieces of information. The site doing the request (in
our case, the firewall) and the URL you surfed to. Whip a quick perl
script to parse the logs and you can now distill out every site that your
employees surf to - both internal to your network and external. Granted,
this is only sites your employees have surfed to and selected the
What's Related option. (Unless "always" was selected as the load option).
In a worst case scenario consider a sudden large number of IBM hits to a
particular site such as Wallop or to a new technology like Java. If interpreted
correctly, this could imply a possible buy out (i.e. Wallop) or a new endeavor (i.e. Java).
Before you consider joining Conspiracy
Theorists 'R' Us, keep in mind these are long shots and only mentioned from
a data awareness standpoint. At a minimal, you should add your companies suffix
to the suffixes area of the smart
browsing preferences dialog. This will prevent Netscape from logging
your internal nodes.

I will point out that the "site info" option is slightly interesting since
it gives some statistics about size and ownership. Here is the information
returned about IBM. Note, as in the first example, Netscape appears to have a problem
generating the proper list items - evident by the ?'s:

The 500 line is suppose to say "in the top" websites. My guess is Netscape will have
this problem resolved when you try this yourself.

Keep in mind, this process applies to all smart surfing including your home use as well.