¿ªÔÆÌåÓý

Locked Re: Testing (was JMRI 4.7.6 and apologies


 

I'm not Bob but I made a post about 2 years ago under the title The Power of Open Source, quoted below:

...
I got curious today and decided to take a look at how 'big' JMRI is, in the sense of how much work
has gone into it over the years. Now, there are a lot of different ways to quantify this but I
thought I would use a simple count of the Source Lines of Code (SLOC) as a rough measure. There is
a commonly used utility in Linux called 'sloccount' by David A. Wheeler that counts the lines of
code (not including blank lines or comments) and does an estimate of the commercial cost of development.

I took a download of the project, applied the utility using defaults and was somewhat surprised at
the results (all results rounded):

1.4 million lines of code, which it estimated would take 76 developers 5 years and cost $53,000,000.

Looking at the breakdowns, it can be argued that those results are overstated since about 60% of
that is in XML files for the decoder and programmer definitions, and those are generally easier to
create since they can be done using a lot of cut and paste and many are done by non-programmers.
Mind you, the cost estimates also used industry salary figures from 2001, so they could be
understated also.

Applying the utility to just the 'java' directory, which is where the actual program code resides,
still gave some impressive results:

Over 500 thousand lines of code, estimated as taking 44 developers 4 years and costing $22,000,000.

Cost to users: $0

This also doesn't count the effort in all the help files, manuals, web site, clinics/tutorials and
the efforts of the various people that help each other on this mailing list. I counted over 240
individuals and organizations explicitly identified on the acknowledgement page.

Not too shabby for a bunch of model railroaders.

Dennis Miller

...

On 6/24/2017 3:36 PM, Donald Perla donperla@... [jmriusers] wrote:
Bob,
To give people some idea of how large the JMRI program is could you tell us how many lines of code there are ?
Don

On Jun 23, 2017, at 7:01 PM, Bob Jacobsen rgj1927@... [jmriusers] <jmriusers@...> wrote:


On Jun 22, 2017, at 2:55 AM, leo pesce lpescester@... <mailto:lpescester@...> [jmriusers] <jmriusers@... <mailto:jmriusers@...>> wrote:

First apologies for my tirade about the new version. I could have been more
tactful.
Thanks.

But I¡¯d like to address, again, a misapprehension you have, and perhaps others have:

On Jun 20, 2017, at 9:39 AM, leo pesce lpescester@... <mailto:lpescester@...> [jmriusers] <jmriusers@... <mailto:jmriusers@...>> wrote:
I wish there was a better attention to detail when it comes to "whatever"
releases, ie. releasing versions that "break" a lot of stuff, in the
attempt to fix or improve something else.
We run _tens_ _of_ _thousands_ of tests on every release. The most recent release, for example, ran over 28,000 tests. Those were repeated on macOS, Windows and Linux; with and without attached screens; they include reading and checking over 150 different XML files from different JMRI versions with different panel configurations. We run these _every_ _time_ somebody proposes a change to the code. We also run tests on the results after each update to the code, not just on test releases. Many people have put huge effort into making making this automatic, and making sure that any problems found are fixed.

But clearly problems do sneak through. In large part, that¡¯s why we have test releases.

Clearly, you think that ¡°the developers¡± should do better so that test releases are perfect. But for large parts of JMRI, ¡°the developers¡± is a misnomer: large parts of JMRI have _one_ person working on them. (Some have nobody working on them anymore, but they tend to not change hence not break)

Sometimes that person is really interested in code testing. Dan Boudreau, who¡¯s written greater than 99% of the Operations code, is that way. There are hundreds of tests in that area to make sure that problems don¡¯t get into test releases. The update I¡¯m currently working on in the CTC area has more new lines in the tests than in the actual code. Several people who work on JMRI even write some of their tests before they write the code itself!

But not everybody is that way. Some features, like the NX Warrants you mentioned, have just one person working on it, and that person¡¯s style is to spend most of their time on code changes. Sometimes, those changes break things away from the part that¡¯s being worked on. With automated pre-written tests you can see that and fix them before the code goes out; without those, problems away from where the work is being done might not get noticed until users encounter them later in a release.

Note: it¡¯s _absolutely_ _OK_ for people to work on JMRI code that way, because without encouraging & enabling that person to work in the style they find most efficient, you would never have had the feature in the first place.

But this means that we¡¯ve only got about 1/3 of the code tested automatically.

So what to do? Test releases are created so people can contribute to JMRI by _testing_ them. And, when those tests find things, often/usually somebody will dig in and fix them.

So I agree with you that we need to pay ¡°better attention to detail¡±. But the ¡°we¡± in that sentence _specifically_ _includes_ _you_. When a test release comes out, _test_ the features you consider important. Do it in a safe environment, so you don¡¯t have to "restage 35 trains manually¡±. Then, if you find something wrong, make a positive comment and help anybody who _volunteers_ to fix it.

Or even better: Provide some automated tests of the features that you think aren¡¯t being tested enough before release.

It¡¯s up to you. If you want to be able to use new features in JMRI, get involved in testing related changes as JMRI evolves so that production releases are clean. Or expect that you¡¯ll be sticking with something that doesn¡¯t have new features. Either approach should work pretty well.

Or you could stick with the misapprehension that you should never see a problem in a test release because ¡°this here is cupcakes¡±. To me, that approach seems destined to make you repeatedly unhappy, but I guess everybody gets to pick their own hobby...

Bob

--
Bob Jacobsen
rgj1927@... <mailto:rgj1927@...>

------------------------------------
Posted by: Donald Perla <donperla@...>
------------------------------------
------------------------------------
Yahoo Groups Links

Join [email protected] to automatically receive all group messages.