Re: searchable index of the web (Ray Stell)
From: (Ray Stell)
Message-id: <>
Subject: Re: searchable index of the web
Date: Wed, 30 Jun 93 15:59:37 EDT
In-reply-to: <9306301905.AA25385@uranus.MIT.EDU>; from "" at Jun 30, 93 3:05 pm
X-Mailer: ELM [version 2.3 PL4]
> I have written a perl script that wanders the WWW collecting URLs, keeping
> tracking of where it's been and new hosts that it finds.  Eventually,
> after hacking up the code to return some slightly more useful information
> (currently it just returns URLs), I will produce a searchabe index of this.
> There is a complete list of all the sites it has found at
> <a href="">
> A complete list of sites found by the W4 (World Wide Web Wanderer)
> </a>
> I'll announce here when we get this index properly running, however it probably
> won't be until sometime in August, as I am going on vacation.  Until then...
> 					Matthew Gray
> Visit the SIPB WWW Plexus Server.  URL:

This sounds good, better than depending on my memory.  Will w4
crack open an log the <Title> or is this directory tree oriented?
If I'm running a server, how will you know to wander to it?  
Should there be a web of wanderers (w5?) so w4e in Europe can
feed from w4u in the US, etc?  Why is this topic such a quiet one? 
Ray Stell		(703) 231-4109