web crawling in perl
david at cantrell.org.uk
Mon May 22 20:36:57 BST 2006
On Mon, May 22, 2006 at 07:37:55PM +0100, Sam Smith wrote:
> What do people think is the "best" perl (or possibly
> otherwise if it's much better) module/script for crawling
> remote websites?
I've been using WWW::Robot for some time and it Just Works. I do
vaguely recall having to patch it to cope with a change in some other
module that it depends on, but the patch was an obvious one-liner.
David Cantrell | Hero of the Information Age
Eye have a spelling chequer / It came with my pea sea
It planely marques four my revue / Miss Steaks eye kin knot sea.
Eye strike a quay and type a word / And weight for it to say
Weather eye am wrong oar write / It shows me strait a weigh.
More information about the london.pm