?

Hello,

To those who have experienced test suites that run long, say ~25 minutes, what are some techniques you have used to mitigate it?? We still would like to be able to check in often and run the test suites prior to checking our code in to our repository but these long running tests makes us stretch what we mean with "often".

Joen

On May 22, 2016, at 11:50 PM, Adam Sroka adam.sroka@... [testdrivendevelopment] <testdrivendevelopment@...> wrote:

25 minutes is ridiculously long. The old XP rule was 10 minutes and that was with 90s hardware. I start to get annoyed when it takes more than a few seconds.?

Likely the problem is too many of the wrong kinds of tests. What you want are what our friend Mike Hill calls microtests--small tests that exercise one behavior of one interface in isolation. What you don't want is things that emulate users poking at a real UI. Anything in between is diminishing returns. A couple integration tests will give you some confidence. A bunch of them will get so annoying that you stop paying attention ("just run it again...")?

On May 22, 2016, at 20:25, "Joselito D Moreno?joenmoreno@...?[testdrivendevelopment]" <testdrivendevelopment@...> wrote:

Hello,

To those who have experienced test suites that run long, say ~25 minutes, what are some techniques you have used to mitigate it?? We still would like to be able to check in often and run the test suites prior to checking our code in to our repository but these long running tests makes us stretch what we mean with "often".

Joen

?

It might well depend on the size and complexity of the system. I agree that ten minutes is long, but… on a system with several million lines of code and and tends of thousands of unit tests, it could take quite a while, especially if your build tool runs each test class separately to prevent tests from affecting one another.

There are a few things you could look into. First, as Adam notes, you may have a lot of tests which are too slow. Unit tests should be very very fast - if they are taking more than a few milliseconds each, they are too slow and need to be rewritten. Second, your system could be too large. There is no reason that a single releasable item should be millions of lines long. Ideally, you should be able to break it into smaller projects which integrate.

But, a bit of perspective. The ideal world isn’t always the real one. I have worked on enormously large systems that had very poor unit test coverage, and used functional tests for everything. The full check in suite on one ran over 8 hours, which was mitigated by building once (which took over an hour by itself) and then spawning each test to a separate computer, leading to checkin times of 2-3 hours. Very very painful.

-----------------

Author,?Getting Started with Apache Maven <>

Author, HttpUnit <> and SimpleStub <>

Come read my webnovel,?Take a Lemon?<>,?

and listen to the Misfile radio play <>!

On May 22, 2016, at 11:50 PM, Adam Sroka adam.sroka@... [testdrivendevelopment] <testdrivendevelopment@...> wrote:

25 minutes is ridiculously long. The old XP rule was 10 minutes and that was with 90s hardware. I start to get annoyed when it takes more than a few seconds.?

Likely the problem is too many of the wrong kinds of tests. What you want are what our friend Mike Hill calls microtests--small tests that exercise one behavior of one interface in isolation. What you don't want is things that emulate users poking at a real UI. Anything in between is diminishing returns. A couple integration tests will give you some confidence. A bunch of them will get so annoying that you stop paying attention ("just run it again...")?

On May 22, 2016, at 20:25, "Joselito D Moreno?joenmoreno@...?[testdrivendevelopment]" <testdrivendevelopment@...> wrote:

Hello,

To those who have experienced test suites that run long, say ~25 minutes, what are some techniques you have used to mitigate it?? We still would like to be able to check in often and run the test suites prior to checking our code in to our repository but these long running tests makes us stretch what we mean with "often".

Joen

Hi,

I'd say you have a good opportunity for refacoring! If your tests cover your functionality; go a head and refactor into smaller functional units.

good luck!

/niklasb

<-----Ursprungligt Meddelande----->

?

?

From: Joselito D Moreno joenmoreno@... [testdrivendevelopment] [testdrivendevelopment@...] Sent: 23/5/2016 5:25:11 AM To: testdrivendevelopment@... Subject: [TDD] Long running test suite?

? Hello, To those who have experienced test suites that run long, say ~25 minutes, what are some techniques you have used to mitigate it?? We still would like to be able to check in often and run the test suites prior to checking our code in to our repository but these long running tests makes us stretch what we mean with "often". Joen