Best practice for unit tests that rely on internet access?

[Toby Wintermute]

Hmm, so using LWP::Online, and putting a sensible minimum version of
Perl in my Makefile (5.8.8) have removed most of the failures.. but
I'm still getting occasional automated test failure reports that
confuse me.. The tests have passed some API calls, so must be online,
but then fail on a specific call.
Always the same bit fails, when it fails - but mostly it passes for
other people.

Annoying - I suspect the keys used for the unit tests might be getting
throttled at the service or something, so if enough automated tests
are run close together, it just fails some things? Damn :/

On 3 May 2011 23:09, David Cantrell <david at cantrell.org.uk> wrote:
> On Fri, Apr 29, 2011 at 11:15:58AM +1000, Toby Wintermute wrote:
>> What is the best practice for writing unit tests that rely on internet access?
>>
>> ie. You have a CPAN module which is all about talking to a web
>> service.. So you have unit tests that do just that.
>> What if the system testing it doesn't allow HTTP connections out? The
>> tests will fail..
>
> Skip 'em if you can't even connect - or if the user doesn't provide an
> API key or whatever that the tests need.
>
>> a) Prompt the user for whether to skip the tests? (I hate this
>> option.. I dislike interactivity in installers)
>
> Interactivity is fine, provided it has sensible defaults.
>
>> b) Only run the tests if specifically set by AUTHOR_TEST or something,
>> otherwise mock the server? (I dislike this.. means few good tests run
>> for users)
>> c) Try to test if there's a working connection, and silently skip the
>> tests if not? (Risks skipping tests if the connect fails for other
>> reasons than no outbound HTTP allowed)
>
> You could always check whether outbound HTTP is allowed by connecting to
> somewhere entirely different.  Try penthouse or thepiratebay -
> somewhere which is highly unlikely to have been explicitly whitelisted,
> so is a good test of whether Generic Web Stuff can be expected to work.
>
> --
> David Cantrell | A machine for turning tea into grumpiness
>
> It wouldn't hurt to think like a serial killer every so often.
> Purely for purposes of prevention, of course.
>



-- 
Turning and turning in the widening gyre
The falcon cannot hear the falconer
Things fall apart; the center cannot hold
Mere anarchy is loosed upon the world