[Ilugc] [OT] need of libraries for exploring websites (Python/PERL)

kish realmailer at gmail.com
Mon Dec 29 17:34:53 IST 2008


hi

I wish to download a bunch of files from a site which allows directory
listing
with links like

http://38.106.111.111/folder/sub%20folder/sub%20folderl%201/
zYWIxNDQ2ODliZDhhMTc4NzRjMGQwMzY/
file%201.zip
.......
......
http://38.106.111.111/folder/sub%20folder/sub%20folderl%205/
afkafkajfkljlkfjiejjflkjlFJLKJFLAY/
file%205.zip

As you can see, the second line of the link changes with every link
The file name is a combination of sub folder name and .zip

Could anybody point  me to a library or modules to explore a given
website to gather all the links in the site?

After which I can apply a filter to extract only those I need and pass
the list to wget or the like

Am looking for python or perl libraries

Thnx

-- kish
---------------
Esse Quam Videri
krisk.wordpress.com


More information about the ilugc mailing list