Searchable Indexes: LISTEN NOW!

• Mail folder: WWW Talk Jul-Oct 1993
• Next message: ts: "Re: httpd on a SUN"
• Previous message: joe@athena.mit.edu: "Re: searchable index of the web"
• In-reply-to: joe@athena.mit.edu: "Searchable Indexes: LISTEN NOW!"

Date: Thu, 1 Jul 1993 15:07:48 +1200
From: Nathan Torkington <Nathan.Torkington@vuw.ac.nz>
Message-id: <199307010307.AA18019@kauri.vuw.ac.nz>
To: www-talk@nxoc01.cern.ch
In-reply-to: joe@athena.mit.edu's message <9307010252.AA03535@theodore-sturgeon>
Subject: Searchable Indexes: LISTEN NOW!
Status: RO

joe@athena.mit.edu writes:

> >1) Sites should be able to just say ``No''.  If I don't want my ftp
> >site to appear on archie, I can say so -- the web should be the same.
> 
> The way I've dealt with the problem is to list only sites that seem to
> be public.  The list of sites that I have are gotten from public
> sources and I try and cull sites with any sort of restriction on them.

I'm suggesting a different method -- perhaps the URL
	http:/config.http
should have something like

<AutoSearch>false</AutoSearch>
<ArchieList>http://site/a/url/with/prebuilt/list/in</ArchieList>

> The trouble with ls-lR, is that it doesn't contain keyword and breadth
> information.  Without such information, an index is very difficult to
> deal with.  

Which is why the format and contents of the archie-like files should
be agreed upon before anything is done.  That way the HTTP daemon can
be tweaked to provide these automagically, if need be -- local access
will (99.995%) always be (a) quicker, (b) smarter, and (c) cheaper.

Nat.