On 1 May 2013 20:59, John Carter <john.carter@...> wrote:
I guess this could a "per language" thing...
In perl, unless you run with -w (which you should), you never even get told
about using uninitialized variables, and _every_ variable is initialized to
null.
In C/C++ the uninitialized stuff sometimes "accidentally" works if there is
left over correct values from the previous test lying in memory / on stack
etc.
Well - the same thing kind-of applies in perl. Default initialisation and
left over correct values can lead to the wrong behaviour in Perl too.
I'm sure there are per-language issues - but those weren't the class of
bugs that were being surfaced.
The problems that were showing up were related to global state / singletons
that were being left in a "bad" state, or code that was expecting the
"default" state - but was getting a valid non-default state after another
test had run.
For example - I remember there was a serious problem with one test suite
with the logging code that switching to xUnit surfaced. The tests worked
fine in a separate process - but in the shared environment it failed.
The reason was that the logging code failed to use the in-app pool of
database connections properly and always spun out a new connection. This
worked fine when it was isolated in a separate process - since nothing else
had touched the pool. In the shared-process model it failed.
This bug exhibited itself in the live system by the silent loss of some
error/info logs under situations of high load. Ouch!
I've not shifted from per-process tests to shared-process tests in C/C++ -
so I can't be sure. But after my experiences with those Perl test suites
I'd be surprised if you didn't discover new bugs that were being hidden in
addition to having problems with tests succeeding when they should fail.
Maybe the ratios would be different - I don't know.
Cheers,
Adrian
PS Considering the group I should mention that the test suites I'm
discussing were produced test-last not test-first. Whether
that affects things in relation to this discussion I'm unsure ;-)
--
adrianh@... twitter.com/adrianh
t. +44 (0)7752 419080 skype adrianjohnhoward pinboard.in/u:adrianh
