Web scraping frameworks?

[Dave Hodgkinson]

I've tended to use Parallel::Process where remote sites have been able to
keep up and haven't been throttled, otherwise just let it run.


On Tue, Mar 4, 2014 at 11:49 PM, Kieren Diment <diment at gmail.com> wrote:

> Gearman's fine until you need a reliable queue.  It's certainly less of a
> pain to set up than rabbitmq, but if you start with gearman and find you
> need reliability after a while there's substantial pain to be experienced
> (unless you already know all about your reliable job queue implementation
> of choice).
>
> On 05/03/2014, at 10:35 AM, Jérôme Étévé wrote:
>
> > - For queuing jobs, I'm a big fan of Gearman. It's light, very stable
> > and very simple.
>
>
>