Web scraping frameworks?

[Kieren Diment]

Gearman's fine until you need a reliable queue.  It's certainly less of a pain to set up than rabbitmq, but if you start with gearman and find you need reliability after a while there's substantial pain to be experienced (unless you already know all about your reliable job queue implementation of choice).

On 05/03/2014, at 10:35 AM, Jérôme Étévé wrote:

> - For queuing jobs, I'm a big fan of Gearman. It's light, very stable
> and very simple.