Re: Quality problems - AGREE!

"Roy T. Fielding" <fielding@simplon.ICS.UCI.EDU>
Errors-To: listmaster@www0.cern.ch
Date: Sat, 16 Apr 1994 02:39:55 --100
Message-id: <9404151736.aa15474@paris.ics.uci.edu>
Errors-To: listmaster@www0.cern.ch
Reply-To: fielding@simplon.ICS.UCI.EDU
Originator: www-talk@info.cern.ch
Sender: www-talk@www0.cern.ch
Precedence: bulk
From: "Roy T. Fielding" <fielding@simplon.ICS.UCI.EDU>
To: Multiple recipients of list <www-talk@www0.cern.ch>
Subject: Re: Quality problems - AGREE! 
X-Listprocessor-Version: 6.0c -- ListProcessor by Anastasios Kotsikonas
Content-Length: 2239

Becky Burnard writes:

> Jacob and Dennis ARE RIGHT. It is annoying to stumble around and get stuck
> on so many broken links. But it's very time consuming to maintain large
> webs, particularly when they are so volatile. We are all experiencing
> phenomenal growth in use of the web, and one of the beauties of the web is
> that it is SO EASY to get a few pages going. It's not surprising there are
> tons of broken links out there. Not only are many of the people browsing
> the web NOT webmasters, but many are quite new to computers as well! I say
> all this interest can only help the cause, and we should welcome them. WE
> provide a service and ought to be helping these people, not slapping them
> around for making mistakes!

And then Brian Behlendorf adds:

> Howbout a program which, given a local html file, recurses through
> links on that file to other files, and the base case on the recursion
> is a link to a file on another machine.  When it reaches this, it
> sends out a HEAD command, and if the file exists it moves on, but if
> it gets sent back a REDIRECT header, it finds this new document, and
> reports back to the web maintainer on the local machine that his link
> to "x" is wrong and needs to be changed to "x'".  If the script were
> robust enough, this could be set up to do interactively, or even
> automatically.  Thus the conundrum of moving pages or whole sites
> could work.


This thread is just too nice a coincidence.  I will be presenting a paper
at the WWW94 conference on exactly that type of tool, although with a much
more robust design and efficient operation.  The program is called the
Multi-Owner Maintenance spider (MOMspider).  

The paper (or at least most of it) can now be read at:

    http://www.ics.uci.edu/WebSoft/MOMspider/WWW94/paper.html

I would very much appreciate any comments.  The missing bits of the paper
(the one figure and appendices) will be added tomorrow, so there's no need
to tell me about that -- I'd do it now, but I can barely see straight after
working all night.


...Roy Fielding   ICS Grad Student, University of California, Irvine  USA
                   (fielding@ics.uci.edu)
    <A HREF="http://www.ics.uci.edu/dir/grad/Software/fielding">About Roy</A>