wget example

The following script
convert windows accept filenames
convert links to local link
convert cgi/jsp link to html file name
recursive download 3 levels

wget  --restrict-file-names=windows -E -r  -p  -l 3  -k   -Ahtml -Ahtm -Ajpg -np
 -nv -nd -nH http://www.fda.gov/Safety/Recalls/ArchiveRecalls/2007/default.htm

The following example want to download only html file from
one website and put it all the files in one certain directory
and convert link relationship to make it cross-referable.

#!/usr/bin/env bash

MONTH=("jan" "feb" "mar" "apr" "may" "jun" "jul" "aug" "sep" "oct" "nov" "dec")
YEAR=("00" "01" "02" "03" "04" "05" "06" "07" "08" "09" "90" "91" "92" "93" "94"
 "95" "96" "97" "98" "99")


for mon in ${MONTH[*]}
do
    for year in ${YEAR[*]}
    do
        x=$mon$year
 mkdir ~/fay/$x
        cd ~/fay/
 wget  -p -r -l 2  -k -nc -Ahtml  -P$x  -np -nv -nd -nH  http://www.cpsc.
gov/cpscpub/prerel/prerel$x.html
    done
done

echo "Wj Script done"