¿ªÔÆÌåÓý

ctrl + shift + ? for shortcuts
© 2025 Groups.io
  1. Testdrivendevelopment
  2. Topics

paper: Test-Driven Development Benefits Beyond Design Quality: Flow State and Developer Experience

 

Hey everyone,

Some months ago I asked for help in the group and some of you gently answered my survey; now the research paper is out!

You can check the preprint here


And I will be glad to discuss future work and evolutions of what is discussed here.

Merry Christmas,
Pedro Calais
 

TL;DR:

TDD is good not only in terms of how it helps the design, but it also favor our mental conditions to be more productive.
 

I'll add what I always say about such papers:

We all agree with the motivations - it is preaching to the choir.

The surveys mean nothing because of the bias in the sample being surveyed.

To me, such work is not worthy?of publication unless it contains objective?measures that strongly indicate?the code is better (less bugs?, more maintainable?, faster to production?) than the other professional alternatives.

Steven Gordon, PhD

On Sat, Dec 24, 2022 at 7:32 AM Pedro Calais <pedro.calais@...> wrote:
TL;DR:

TDD is good not only in terms of how it helps the design, but it also favor our mental conditions to be more productive.

 

I am all open to criticism to improve the work BUT you need to read the paper first.

The survey is just one evidence I've used, there is a strong theoretical component in the paper. The survey is almost a bonus part.
I am also not looking at code quality in the paper, but at other dimension -- developer experience.

Best,
Pedro
 

I read it.

The theoretical portion was not new to me, having done?TDD and Agile,?taught it in University and customer sites, and wrestled with how to actually prove it is better for several years.? The survey added nothing because it is exactly what should be expected from a survey of interested practitioners of any technique.? I am exactly as convinced that Agile TDD is a great way to develop software after?reading it as I was before.?

If the code quality is better, present some new concrete evidence.? Many have tried, but the code compared is always too trivial to be convincing that it is a better technique in the real world in general.? There is a reason for that - it would be hugely expensive to develop the same system using TDD and using something else in order to have code bases to compare.? Furthermore, if the TDD code was not developed using Agile methods, then TDD would not be fully exploited.? But if the comparison system was not also developed using the same Agile methods, then it could be argued the better code was partially due to Agile practices other than TDD.? If you take that approach, then for all your efforts, all you proved was that Agile is better with TDD.? ?It is a tricky problem that I eventually decided was not worth the cost and effort to pursue.

If the developer experience is better, present some concrete evidence, not just that it feels better.? Maybe, developer retention in the real world?? (Do developers stay longer at companies that practice TDD?)

If it was a master's thesis, I would give it an A (although I would have tried to convince the student to tackle a more concrete topic).? I do not see how the paper advanced the very difficult academic topic of whether TDD is better than the alternatives and why.

If you are still reading, Software Development is more a people problem than a technical problem.? Researching it using social science research methodologies will bear more fruit than using engineering research methodologies.? Any serious social scientist would scoff at the survey designs and experiment designs that have been used in Software Development research.

On Sun, Dec 25, 2022 at 8:09 AM Pedro Calais <pedro.calais@...> wrote:
I am all open to criticism to improve the work BUT you need to read the paper first.

The survey is just one evidence I've used, there is a strong theoretical component in the paper. The survey is almost a bonus part.
I am also not looking at code quality in the paper, but at other dimension -- developer experience.

Best,
Pedro

[email protected]
 

I fully?agree with Steve's comments, and I'd like to add that even if you did pursue the avenue of developing the same system using the non-Agile methods, and without TDD, and then developing the same system using Agile methods, and using TDD, you'd still be short of reaching a proper scientifically sound conclusion, as you'd also need a clean slate/mind before venturing to develop the mentioned systems.?

In other words, the issues you'd run into while developing system A, would have an impact on the process of developing system B, and vice-versa.

There's a famous philosopher, Paul Ree, who explored the issues of and for his mental experiment he imagined a donkey standing between two piles of hay, and to cut the story short, he posited that given the same donkey, in the same exact situation, with same environment and same piles of hay, he'd always make the same decision and always make the same choice. This mental experiment was the basis for his argument for determinism... the main problem was that you could never, ever perform this experiment, as you could never, ever have the same donkey, in the same environment, with two same piles of hay, twice.

Best regards,
Petar


On Mon, Dec 26, 2022 at 9:45 AM Steve Gordon <sgordonphd@...> wrote:
I read it.

The theoretical portion was not new to me, having done?TDD and Agile,?taught it in University and customer sites, and wrestled with how to actually prove it is better for several years.? The survey added nothing because it is exactly what should be expected from a survey of interested practitioners of any technique.? I am exactly as convinced that Agile TDD is a great way to develop software after?reading it as I was before.?

If the code quality is better, present some new concrete evidence.? Many have tried, but the code compared is always too trivial to be convincing that it is a better technique in the real world in general.? There is a reason for that - it would be hugely expensive to develop the same system using TDD and using something else in order to have code bases to compare.? Furthermore, if the TDD code was not developed using Agile methods, then TDD would not be fully exploited.? But if the comparison system was not also developed using the same Agile methods, then it could be argued the better code was partially due to Agile practices other than TDD.? If you take that approach, then for all your efforts, all you proved was that Agile is better with TDD.? ?It is a tricky problem that I eventually decided was not worth the cost and effort to pursue.

If the developer experience is better, present some concrete evidence, not just that it feels better.? Maybe, developer retention in the real world?? (Do developers stay longer at companies that practice TDD?)

If it was a master's thesis, I would give it an A (although I would have tried to convince the student to tackle a more concrete topic).? I do not see how the paper advanced the very difficult academic topic of whether TDD is better than the alternatives and why.

If you are still reading, Software Development is more a people problem than a technical problem.? Researching it using social science research methodologies will bear more fruit than using engineering research methodologies.? Any serious social scientist would scoff at the survey designs and experiment designs that have been used in Software Development research.

On Sun, Dec 25, 2022 at 8:09 AM Pedro Calais <pedro.calais@...> wrote:
I am all open to criticism to improve the work BUT you need to read the paper first.

The survey is just one evidence I've used, there is a strong theoretical component in the paper. The survey is almost a bonus part.
I am also not looking at code quality in the paper, but at other dimension -- developer experience.

Best,
Pedro

 

Perhaps some pointers to examples of research you approve of would have helped here?

If the developer experience is better, present some concrete evidence, not just that it feels better.? Maybe, developer retention in the real world?? (Do developers stay longer at companies that practice TDD?)
I would have thought that when talking about developer experience, "feeling better" is potentially a valid data point.

Totally agree on using social research approaches for this. Helen Sharp (Open University) did some interesting observational studies, including one of my team at the time. She had a certain amount of opposition from the more "hard-core engineering" colleagues in her department, which was short sighted on their side.
 

I would not advocate engineering faculty attempting to apply sophisticated social science research techniques, partly because it takes years of work to learn how to do it right and partly because it is detrimental to your career.? What I would advocate is interdisciplinary research involving both people who understand software development and people who understand how groups of people work.? There are still the problems of where to publish such research that both silos would accept and who would actually be qualified to evaluate such research.

I was part of a group trying to do that in 2002-3, but we found no journal or conference who considered it to be in their purview.? Some pure social science research was successfully published.

On Tue, Dec 27, 2022 at 6:21 AM <groups.io@...> wrote:
Perhaps some pointers to examples of research you approve of would have helped here?

If the developer experience is better, present some concrete evidence, not just that it feels better.? Maybe, developer retention in the real world?? (Do developers stay longer at companies that practice TDD?)
I would have thought that when talking about developer experience, "feeling better" is potentially a valid data point.

Totally agree on using social research approaches for this. Helen Sharp (Open University) did some interesting observational studies, including one of my team at the time. She had a certain amount of opposition from the more "hard-core engineering" colleagues in her department, which was short sighted on their side.

 

In the previous post, by "understand", I mean are specialists in both the topic and how to research that topic.


On Tue, Dec 27, 2022 at 11:50 AM Steven Gordon <sgordonphd@...> wrote:
I would not advocate engineering faculty attempting to apply sophisticated social science research techniques, partly because it takes years of work to learn how to do it right and partly because it is detrimental to your career.? What I would advocate is interdisciplinary research involving both people who understand software development and people who understand how groups of people work.? There are still the problems of where to publish such research that both silos would accept and who would actually be qualified to evaluate such research.

I was part of a group trying to do that in 2002-3, but we found no journal or conference who considered it to be in their purview.? Some pure social science research was successfully published.

On Tue, Dec 27, 2022 at 6:21 AM <groups.io@...> wrote:
Perhaps some pointers to examples of research you approve of would have helped here?

If the developer experience is better, present some concrete evidence, not just that it feels better.? Maybe, developer retention in the real world?? (Do developers stay longer at companies that practice TDD?)
I would have thought that when talking about developer experience, "feeling better" is potentially a valid data point.

Totally agree on using social research approaches for this. Helen Sharp (Open University) did some interesting observational studies, including one of my team at the time. She had a certain amount of opposition from the more "hard-core engineering" colleagues in her department, which was short sighted on their side.

1 - 9 of 9