开云体育

Variable length data records in CSV files are somewhat common, if not universally so. I am not a Java programmer but I have been involved with computers and software for longer than I would like to admit.

It is likely that the major changes involving the change to the JAVA CSV library now distributed with JMRI, effective with version 4.19.3 (as of 2020-01-28) has caused some problems in transition.

In a script that I wrote for Sam that use the previously available library, variable length data records were allowed. Non-trivial changes had to be made to accommodate the new library.

On Sun, May 10, 2020 at 02:38 PM, Dan Boudreau wrote:

The code has been modified by one of the other JMRI programmers, and is a bit broken.? I found that I could import his files if I added 10 extra commas to each line.? The original code accepted variable length input fields, but now the code is hardwired to search and expect 18 fields.

But it took me several hours to find and verify that any legacy data files with variable length data records, could still be read with the modified script.

In part, some of the original script looked like:

import com.csvreader        # Allow us to read CSV formatted text files
# http://javacsv.sourceforge.net/com/csvreader/CsvReader.html 
~ ~ ~
class ManageTurnoutListeners(jmri.jmrit.automat.AbstractAutomaton):
~ ~ ~
    def ReadTurnoutFile ( 
                self,
                turnoutFileName
            ) :
        # https://docs.oracle.com/javase/7/docs/api/java/io/File.html
        tempInputFile = java.io.File(turnoutFileName)
~ ~ ~
        # Check if file exists
        if tempInputFile.exists() :
            # It does, so load it
            tempCsvFileReader = com.csvreader.CsvReader( turnoutFileName )

            # Read the headers
            tempCsvFileReader.readHeaders()
~ ~ ~
            while (tempCsvFileReader.readRecord()):

                # Read the record details
                ColumnCount = tempCsvFileReader.getColumnCount()

But the similar snippet in the revised script indicates what eventuality passed for the no longer available getColumnCount() function.

import org.apache.commons.csv   # Read CSV formatted text files  Updated for 4.19.3ish
###!    http://commons.apache.org/proper/commons-csv/user-guide.html
~ ~ ~
class ManageTurnoutListeners(jmri.jmrit.automat.AbstractAutomaton):
~ ~ ~
    def ReadTurnoutFile ( 
                self,
                turnoutFileName
            ) :
        # https://docs.oracle.com/javase/7/docs/api/java/io/File.html
        tempInputFile = java.io.File(turnoutFileName)
~ ~ ~
        # Check if file exists
        if tempInputFile.exists() :
            # It does, so load it
            ###!   https://commons.apache.org/proper/commons-csv/apidocs/org/apache/commons/csv/CSVParser.html 
            tempCSVParser = org.apache.commons.csv.CSVParser.parse(
                        tempInputFile, 
                        java.nio.charset.StandardCharsets.UTF_8, 
                        org.apache.commons.csv.CSVFormat.DEFAULT.withFirstRecordAsHeader().withCommentMarker('#')
                    )
~ ~ ~
            ###! tempCsvFileReader.readHeaders()
            ###! https://commons.apache.org/proper/commons-csv/apidocs/org/apache/commons/csv/CSVRecord.html
            for record in tempCSVParser.getRecords():
                # Type ( record ) == List
~ ~ ~
                # Read the record details
                ###! ColumnCount = tempCsvFileReader.getColumnCount()
                ColumnCount = record.size()

As is evident, with the org.apache.commons.csv library, the script code that eventually did provide the record column count was a bit more convoluted than I would have preferred.

Perhaps some similar revision to the Java code that reads CSV files could also be considered.

Both versions of the script were uploaded to either the Files/ProblemsBeingWorkedOn or else the Files/Script Examples folder at one time but are not there now. If requested, a new copy could be uploaded. The revised script contains a lot of comments as to what was changed for the new library.

This note got too long.

Cliff in Baja SoCal