Devel::Cover recommendations... or maybe not?

[Paul Golds]

On 3/15/07, Tom Hukins <tom at eborcom.com> wrote:
> Nik Clayton gave an excellent talk to Milton Keynes Perl Mongers last
> year about testing legacy code that included  using overloading to
> change the behaviour of things like open().  You could use a similar
> approach to simulate, if not manufacture, edge cases.

I have happily overloaded such things for test cases, but sometimes
the pain and other tradeoffs involved just aren't worth it.

For example a module I was writing not a million moons ago needed
potentially to read the same file twice inside one public API call,
and being a good little paranoid programmer I happily added checks to
both open(...) calls to make sure they worked.

I then checked the first version by deleting the file, and all was
happy.  I then tried to test the second one, overloading open(..) with
something which returned the normal value the first time and a failure
the second time.

It then struck me that the reason I was opening the file twice was
implementation dependant and this was turning a respectable black-box
test into a grubby little white-box thing.

Catching myself doing this kind of hackery I thought "This is not
productive, and it is making me feel dirty" and reverted these
changes.

This didn't cause me a loss of sleep.  I'd rather feel I could happily
gut the entire implementation of my module and replace it with an
equivilant without touching unit tests than I was getting the absolute
100% coverage.  Since then I've used coverage as a guide of what to
test, in cases like this I just walk away and leave a little untested.

Paul