Perl threads and libwww wierdness

[Toby Wintermute]

Hi,
I'm hitting some really odd behaviour, infrequently, with libwww and
mechanize under a highly-threaded Perl.
Can I get a quick check to see if what I'm doing is known to work
reliably for you?


I have encountered a situation where I see unusual 404 errors - in
between 0.03% to 0.10% of requests.
Errors are randomly spaced on random pages, but over time the average
amounts are quite consistent.
Error rates initially increase with the number of simultaneous
threads, but seem to top off at .1%. (ie. One in a thousand requests)

The 404 errors are reported on the distant webserver as well, for URLs
that are definitely not 404. (as the identical URL is being requested
successfully many times in the same period).

Scale: This is typically running around 40 threads, all going flat-out
on an 8-core system; issues show up whenever you get over ~6 threads
though.

The only reason I don't think this is a problem with the network or
webserver is that the problems don't show up if I use fork() instead
of threads. (On otherwise identical code; and the same overall
throughput rates are reached. However the fork() version is just for
that bit of code for testing this; it misses some functionality.)

It's also being a pain to try and replicate the threaded issue with a
standalone server away from our code though, which isn't a good sign.

Hence, looking for some confirmation of whether this might just be a
known-bug with Perl or libwww before I go chasing down this rabbit
hole for miles. :(

This was running on Perl 5.14.1 and current versions of the above
modules. (I don't think there are any bugfixes listed in 5.14.2 that
would affect this issue? [1])

Any thoughts?

Thanks!
Toby

[1 http://search.cpan.org/~flora/perl-5.14.2/pod/perldelta.pod ]

-- 
Turning and turning in the widening gyre
The falcon cannot hear the falconer
Things fall apart; the center cannot hold
Mere anarchy is loosed upon the world