isolating thread-unsafe modules

[Bob MacCallum]

Parallel::ForkManager seems to do the trick.



On Sun, Jul 28, 2013 at 10:02 PM, Bob MacCallum <uncoolbob at gmail.com> wrote:

> Good question - I have always hacked something with fork in the past but
> it just seemed so simple with Thread::Queue and threads.  I've never needed
> much IPC in the past.
>
> It looks like this could be a better option:
>
>
> http://search.cpan.org/~aristotle/Proc-Fork-0.802/lib/Proc/Fork.pm#Multi-child_example
>
> (thanks also to Dominic for the same warning re: threads)
>
>
>
> On Sun, Jul 28, 2013 at 9:13 PM, Dave Hodgkinson <davehodg at gmail.com>wrote:
>
>> Threads? Now you have two problems. Why not processes?
>>
>> Sent from my iPhone
>>
>> On 28 Jul 2013, at 20:07, Bob MacCallum <uncoolbob at gmail.com> wrote:
>>
>> > Sorry for the Perl question, but I'm a bit out of my depth...
>> >
>> > I have a Dancer app providing a web service, and I have another script
>> > using the same Dancer config file and db schema (e.g. "use Dancer
>> > qw/:script/").  This script watches the database (for changes made by
>> the
>> > web app) and does the "heavy lifting".  Now I want to parallelise the
>> heavy
>> > lifting with threads and it seems from the core dump and googling that
>> > DBD::Pg is not thread safe (see the few lines of debugger output below).
>> >
>> > That's fine - I am happy to run only non-database code in my "worker
>> > threads".  The trouble is that even if I do this, I think the
>> > threads->create is threading the Dancer globals which include postgres
>> > schema connections.
>> >
>> > I tried a few "no My::Schema; no DBD::Pg; no Dancer ..." declarations in
>> > the block with the threads->create - but that didn't seem to help.
>> >
>> > Does anyone know how I can use thread-safe and thread-unsafe code in the
>> > same script?  I think I just need to make a block where none of the
>> > Dancer/database stuff is in scope?
>> >
>> > I thought Thread::Isolate might help, but it hangs on the first test
>> during
>> > the install.  It's dated 2005.
>> >
>> > thanks!
>> > Bob.
>> >
>> >
>> >
>> > Reading symbols from
>> > .../perl5/x86_64-linux-thread-multi/auto/DBD/Pg/Pg.so...done.
>> > Loaded symbols for .../perl5/x86_64-linux-thread-multi/auto/DBD/Pg/Pg.so
>> > Reading symbols from
>> > /usr/local/pgsql_dir/postgresql-8.4.7/lib/libpq.so.5...(no debugging
>> > symbols found)...done.
>> > Loaded symbols for /usr/local/pgsql_dir/postgresql-8.4.7/lib/libpq.so.5
>> > Core was generated by `perl bin/maintain_populations-threaded.pl'.
>> > Program terminated with signal 11, Segmentation fault.
>> > #0  0x00007f9de39e923e in PerlIOEncode_getarg () from
>> >
>> /usr/lib/perl5/5.12.3/x86_64-linux-thread-multi/auto/PerlIO/encoding/encoding.so
>>
>>
>