November 19, 2007

Tools to Pull a Whole Web Site Down Into your Hard Drive

Just random notes.

Goal: Save a set of web pages to the local drive and interlink them intelligently leaving some links to the internet for known dynamic content.

So far it looks like the Firefox web page complete format is the best and most flexible local copy format.

Here are some things to investigate:

FireFox - Save as Web Complete

Can this be controlled by an extension? Can we script this?

I like this because the HTML and files are very obvious and accessible.

http://developer.mozilla.org/en/docs/Extensions

I cannot find the API to figure out whether the "save as web page complete" functionality is available to plugins or extensions.

An Application that Pulls down Sites:

Offiline Explorer - This was recommended as the best site downloader - the key is whether or not we can get at the data once the download is finished to adjust the files. This needs investigating.

http://www.metaproducts.com/

Other possibilities - need investigation

IE - Make available offline

http://support.microsoft.com/kb/196646

Web Site Downloader for Windows

http://winwsd.uw.hu/

Local Web Site Archive

http://www.aignes.com/lwa.htm

Web Site Downloaders

http://www.snapfiles.com/Freeware/downloader/fwoffline.html

Posted by csev at November 19, 2007 12:31 PM