Perl-friendly message queue-like system

[Tomas Doran]

On 22 Sep 2011, at 12:33, Jason Clifford wrote:

> On Thu, 2011-09-22 at 09:15 +0100, Martin A. Brooks wrote:
>> I have a bunch of servers doing mail ilftering.  I would like them  
>> to send tiny messages about the results of said filtering to a  
>> central point.  I would then like something at the central point to  
>> pop messages off the queue and update a database.  Think of it as  
>> sort of a centralised collection point for statistical data.  If  
>> the odd message gets eaten, it really doesn't matter.
>
> Would a simple syslog based solution do the job? I still have a simple
> system in place that uses syslog to distribute messages as necessary
> with the target server throwing them into a pipe and a very simple  
> perl
> script taking messages from the pipe and updating an authentication
> database.
>
> It is very simple and works very well.


I almost suggested syslog also. :)

My impression is that the only real requirement is avoiding calling  
fsync() on the database server once per log message (i.e. aggregating  
several seconds worth and doing a bulk insert), so I can imagine  
something like this being perfect..

There are other advantages of having a 'real' message queue, and if  
you're likely to later want one for other things then that's entirely  
fair. But if this is the only problem like this you're likely to have  
to want to solve, I'd be very tempted to just steal Jason's code. :)

Cheers
t0m