Should exist, but probably doesn't (2)

[Tim Sweetman]

(My previous claime)
 >>>the application code doesn't need to know it's running an async job
 >>>at all.

> Andy Armstrong:
>>So does the main thread / process / whatever get an async  
>>notification when one of the workers completes? Like a signal perhaps?

It rather depends. The idea was that by saying

   do_all_these ( @files, sub { $_->[0] ... convert_to_mp3(...) } });

the logic which controls the worker processes is effectively segregated 
from the application logic. Signals would be problematic, because of the 
possibility that the signal will do something unintended with the 
application logic, rather than being handled by do_all_these.

Simon Cozens wrote:
> Sounds like a job for continuations. Until then, there's POE.

What I'm proposing would work by attempting to process each of the 
elements of @files. Each one would throw an exception. At some point, 
do_all_these finds it has a queue of work to do in the background, 
spawns it, and waits. When work returns from background processing, it 
can restart processing for the @files where conversion has occurred.

The problems are: overhead (of marshalling all the exceptions); 
counterintuitiveness for the maintenance programmer; and whether the 
process within sub { ... } will correctly handle repeated runs following 
the throwing of an exception.

Continuations, as I understand them, would mean that, for each element 
of @files, the system would store the entire state of the app, then 
continue. As background processing returned results, those 
whole-app-images could be restored, and restarted. Is that going to be a 
better solution, in general, than the exception-handling scheme I'm 
proposing?

The other context in which do_all_these would be nice is 
object-relational mappers. If your application requires you to load 
(say) several thousand objects of various classes from the database, 
each of those objects may have distinct ways of querying. If you can put 
the logic inside a do_all_these loop, each CLASS, rather than each 
INSTANCE, can do one database query. This enormously reduces the number 
of (colosally expensive) app-to-database round trips, without all that 
dangerous mucking about with write-through distributed caches...

ti'