[OT] benchmarking "typical" programs

[Paulo Edgar Castro]

How open to craziness are you ????

A program, any program on any language can only do so many things right ?

Write/Read to memory
Write/Read to disk
Write/Read to the network
Write/Read to a given port ( Serial / Parallel / USB )
Write/Read to another program

( I might be missing something, though but my current in cerebrum 
visualisation of the Von Newman architecture doesn't leave room for more )


This should be relatively simple to write or scavenge from somewhere 
amongst the many test suites there are.
If you REALLY REALLY REALLY want to make sure your cache, wherever it 
might be located doesn't play tricks on you, you can always go berserk 
mode and use the most used feature in the Windows world. Reboot.

These tests that you might have or might have written could read 
randomish patterns of data big/small enough to fit your needs from 
specific test data files.
And if they're all random between tests I would say that that's even 
better, but that's up to the architect to decide ;)

So the recipe could be:

Get a cheap box you don't mind thrashing with reboots ( there's nothing 
wrong with reboots btw )
Make it boot into single mode with network on ( So you you can upload 
things or trigger another Perl build )
Make sure cron/anacron and friends ( whomever might want to run havoc 
whilst the test is running ) is disabled
Get a trigger in place ( ssh my_super_evil_boz "./build_perl 5.42 
--with-evil-test-suite" || via trigger it via post-commit or similar )
Make sure that in the end the results are kept in the box and or are 
mailed to you or posted to some other interface
reboot if (end_of_test)
SYSTEM READY 4 ANOTHER RIDE ...

I suppose that amongst the exact same build you should expect some 
fluctuations for different test runs much like the ones that happen in 
the BogoMips calculation.
But never to a degree of magnitude that might indicate a regression.

Regards,



                 PECastro