Devel::Cover recommendations... or maybe not?

[Paul Johnson]

On Thu, Mar 15, 2007 at 09:17:59AM +0000, Matt Wilson wrote:
> On 3/14/07, Paul Johnson <paul at pjcj.net> wrote:
> >Here are some reasons why perfectly good code might not have 100%
> >coverage from my cpancover output:
> >
> > - My specific combination of hardware, operating system, configuration,
> >   installed software or some other variable, doesn't allow some of the
> >   code to be exercised.
> 
> I'll agree with that - sometimes OS-specific code can't be completely 
> covered.

Well, not by one coverage run.  You can run your tests on different
combinations of setups and merge the coverage though.

> > - Some configuration might be needed to run all the tests.
> 
> Surely that's the point, though? Write your unit tests explicitly so
> that your configurations are going to achieve maximum coverage.

My point was more that some tests might not be run by default.  For
example, I might need to set $PLEASE_DO_REALLY_LONG_AND_EXPENSIVE_TESTS
to get the most coverage.

> > - The author performed a manual test, or in some other way convinced
> >   themselves that the code was good, and considered their precious time
> >   might be better used somewhere else.
> 
> Heh, and that just flies in the face of unit testing. "Works for me"
> is rarely a useful reply for somebody who can't get your code to work
> for them.

Of course.  But I was talking about (theoretical) code that actually
does work here.  Broken code was covered further down.

But my main point was that since few of us have unlimited resources, we
need to set our priorities correctly, and sometimes 100% code coverage
won't be at the top.

Having said that, on one project that I was on, literally the last test
to get 100% coverage (as far as Devel::Cover can measure it) uncovered
an error in a DB schema caused by incorrect assumptions leading to
inconsistencies in the specification.

> > - Devel::Cover might have bugs.
> 
> It does, definitely. We've used it extensively - all our internal
> modules have it running against them daily to ensure we have high
> coverage - and there are definitely a few internal gotchas. However,
> all in all it's pretty good.

Do I already have bug reports for those gotchas?

-- 
Paul Johnson - paul at pjcj.net
http://www.pjcj.net