HTTP batch file fetcher?

altis@ibeam.jf.intel.com (Kevin Altis)
Message-id: <m0p3B0s-0003WDC@ibeam.intel.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Date: Fri, 26 Nov 1993 13:49:08 -0800
To: www-talk@nxoc01.cern.ch
From: altis@ibeam.jf.intel.com (Kevin Altis)
Subject: HTTP batch file fetcher?
Has anyone written an application (perhaps a Perl script) to automatically
fetch a series of files from an HTTP server, storing them on a local disk?
Most Web clients have the ability to download to disk, but they can't be
used yet to get a series of files automatically.

The other problem is that when downloading to disk, the user is prompted
for a filename, even though the last part of the URL could have been used
for the name; at least Lynx (thanks Lou) uses the last part of the URL for
files it can't display, but alas Lynx has no download to disk "mode" and no
remote control options.

As more and more HTTP and Gopher servers pop up, the ability to get files
automatically via FTP, including batch download and recursive download of
whole directory trees seems to have gone away. Pointers to tools for Mac,
Windows, Unix that allow me to do batch downlods via HTTP... would be
greatly appreciated.

ka