From Wikipedia:?MQTT is an ISO standard publish-subscribe-based messaging protocol. It works on top of the TCP/IP protocol. It is designed for connections with remote locations where a "small code footprint" is required or the network bandwidth is limited.

I guess that answers the question for Wifi connected devices. Below that, e.g. in a BLE network, you could convert the MQTT messages to topic strings and sending via UART or? use 6LoWPAN. That would be a gateway from your BLE device network to MQTT.

Hans

开云体育

Thanks Bob for providing the boundary conditions. That's critical information, better than I feared.

I'd like to thank whoever wrote the MQTT code already in JMRI.? That's a big first step and opened the door for a giant leap.

Here are some thoughts, for whatever they're worth.


I'd rather leave JMRI "username" to be plain text descriptors that users see (eg with Engine Driver) to identify the turnout. I would rather not use this for MQTT coding, use the system name instead.
The JMRI system name can have "MS" for sensors, followed by "." for easier reading, followed by the topic. Whilst perhaps not best practice, omit the "." if we retain the leading / in the topic.
?
If "JMRI has specific control metaphors: Turnout (i.e. one-bit digital output), Sensor (i.e. one-bit digital input)" implies a payload must be binary - 1 or 0 or perhaps words such as CLOSED and THROWN that are effectively binary, then other details need to be in the topic. Otherwise, more information can by put into a JSON payload if so desired. Other stuff could be put into payloads but for turnouts and sensors, it may only be the binary bit that is processed - CLOSED or THROWN, ACITVE or INACTIVE, 1 or 0.

Turnout code is already there so we should retain compatibility with it.

I would prefer to use signal masts though some people may prefer signal heads so they probably both need to be done.
However using masts means the device needs to know how to display "Clear", etc. I think JMRI already has that info so perhaps it can be included in the payload.


Getting a bit more specific, here are some simple suggestions. I'm not using JSON here but open minded on that.


Topic structure:?? RailroadIdentifier/NodeID/ObjectType/ID

Where
RailroadIdentifier is a user definable root component (currently fixed as /trains)
NodeID can be a board name or number or concentrator reference of some sort (currently "track" is used)
ObjectType is turnout, sensor or whatever
ID could be pin number or some other identifier for the specific item, whatever is meaningful to the node.


Sample JMRI system names:

MS.TxNamib/board.0003/sensor/5???????? [sensor on pin 5 on node called board.0003]
MT.TxNamib/board.0003/turnout/3.servo? [turnout controlled by a servo connected to pin 3 though it may be preferable for the node to have 'pin 3 is a servo' defined in its config (EEPROM) or maybe only does servos so it wouldn't need to be specified]
MT.TxNamib/board.0003/turnout/4??????? [turnout controlled by setting pin 4 to high or low]
Expansion boards may make the ID more complex, perhaps to include the expansion board address.

Perhaps having too much detail in a system name may create difficulties if they need changing.


Sample messages:

TxNamib/board.0003/sensor/5 1???????? [published by the node to report that the sensor on pin 5 has become active]
TxNamib/board.0003/turnout/3.servo 1? [published by JMRI to set the servo on pin 3 to thrown position]


JMRI will subscribe to TxNamib/+/sensor/# to get notified of changes detected by sensors.
JMRI can perhaps publish to a sensor a payload like "Status?" to ask for the status of that sensor though if retention is enabled, JMRI will get the last status when it starts a subscribe session.

Nodes subscribe to TXNamib/their NodeID/# except for anything they publish. With retention enabled, when a node starts a subscribe session, it will be reminded what it should have last done, perhaps requested while it was asleep.

A configuration utility can be used to publish something like "TxNamib/board.0003/cfg" with a SHOW payload for the node to report status and config details from its EEPROM or equivalent, or SET payload with configuration details the node should use to update its config.


Regards, Dave Mc.

I don’t get why you are putting the board id as part of the topic string. MQTT doesn’t care about this all for general control of the layout excluding DCC. You the user tell what boards or “nodes” what topics subscribe and publish too. This multiple boards can tell JMRI to change a state of a turnout ?or whatever.?

This is is a great conversation and I agree with what most are saying. I do however think we need to apply the KISS (Keep It Simple Stupid) method and that is flexible for layout control. MQTT is lightweight, flexible, powerful, and is easy to use. It works across multiple different systems at the same time without problems. This is why it is used in home automation.

Maybe the easy way is get rid of the MQTT connect ion in JMRI and those of use that want to use it use NodeRed to get or post to the socket for MQTT to create a gateway. This way people can do whatever they want.

We in model railroading need to pay attention to things like this because it will allow people better control of their layouts at a fraction of the cost. Buy some ESP8266 boards or whatever to control the layout turnouts, signals, blocks, lights, etc and use mqtt to control with JMRI.?

Chris

Just to reiterate, I’m happy to (eventually, not this week; maybe when the semester is over) work on coding up something. From listening, it seems there are two issues that we need to address when specifying the MQTT handling for Turnouts, Sensors, etc.

1) The naming hierarchy that JMRI knows about

2) How to handle the message payload, i.e. mapping JMRI’s state to the payload and vice versa

I don’t have a lot of experience with this, don’t know about community standards, but I think (1) can be handled by making JMRI agnostic to hierarchy choices through system names: Whatever your MQTT setup needs, just put that in the system name. So one person might have “MT/turnouts/EastBranchWestSwindonSiding” while another might have “MT/turnouts/2”. The setup _outside_ of JMRI, done however the modeler wants, will drive the names and JMRI will just accept.

But I don’t know how to handle (2). What kinds of payloads are there? We can certainly append something to the end of the system name that provides (a bit of) instruction about decoding the payload, but we shouldn’t turn this in to an incredible pain in the neck to configure.

Bob
--
Bob Jacobsen
rgj1927@...

If we’re going to allow flexible specification of items in the MQTT interface (my recommendation, but not decided), are there items that need to be included?

For example, does JMRI need to be told about use or non-use of retention? (I’ve quoted a bit below about that) We could build that in several ways: A column in the i.e. Turnout Table or Sensor Table, or as part of the system name (i.e. a “(R=Y)” suffix) or some other way. Might be good to have the right default, though, to save a lot of setup work and annoying errors.

Are there others?

Bob

On Dec 7, 2018, at 8:06 AM, Speed <gertmul@...> wrote:

- Do we use retention? I found it convenient in my ESP implementation to save the state of the turnout in /TxNamib/servo.0003/01 as a '0' or a '1' with retention on, so when the ESP boots up, it knows where to move the servo without keeping the state saved in local EEPROM. I know some don't like physical addressing boards, but I found it convenient to simply subscribe to /TxNamib/servo.0003/# for all 16 turnouts connected to the ESP. So if retention is used, JMRI is going to turn on and get hundreds of messages (that it needs to keep up with) and also might have the "INCONSISTENT" state if it determines the turnout to be otherwise.

- JMRI typically (or used to) only generate a message to hardware when there is a state change, so without retention in MQTT, the servo won't know where to go until there is a change in JMRI.

--
Bob Jacobsen
rgj1927@...

Maybe the key is to add a row to the system table to place the mqtt topic there. This way the user can use what they want and the system can associate that topic to that turnout for example

Bob J,

From watching the conversation (I've not directly worked with MQTT), would
the conventions of MERG or LCC work where the systemName entry is at
'something_on;something_off' pattern entry. So like the events, here is the
thing I will match and consider as the on/off trigger. The difference (or
maybe not) is the MQTT would require the leading part of both to match and
only differ in the trailing item? Granted this concept is simple enough for
binary items like sensors or turnouts. I don't know how well it would map to
complex objects like a signal mast for example.

-Ken Cameron, Member JMRI Dev Team
www.jmri.org
www.fingerlakeslivesteamers.org
www.cnymod.com
www.syracusemodelrr.org

MQTT payloads (states) can be an integer, string, or even JSON. I think though if it can be done we should be using language that is easy ?to understand. Such as on, off, closed, thrown, etc., instead of 0 or 1. Even for Records the payload can be whatever the user wants.

Chris

How much control does the layout owner have over the payload?

I.e. can sensor data from layout to JMRI (almost always) be configured to match something specific? Or does JMRI need to be configured to match what’s sent, because some device sends 1 vs 0, another On vs Off, another ON vs OFF, and another Yes vs No?

The existing JMRI turnout sends (unless very low-level configuration is done) “THROWN” and “CLOSED”. Does only specific hardware understand that, or is there a way in the broker to transform it?

Bob

On Dec 9, 2018, at 5:37 PM, Chris Rood <crood58@...> wrote:

MQTT payloads (states) can be an integer, string, or even JSON. I think though if it can be done we should be using language that is easy to understand. Such as on, off, closed, thrown, etc., instead of 0 or 1. Even for Records the payload can be whatever the user wants.

--
Bob Jacobsen
rgj1927@...

Bob,
Answering some of your questions, though I'm not an expert on this so hopefully others will add or correct as need be..

The existing JMRI turnout sends (unless very low-level configuration is done) “THROWN” and “CLOSED”. Does only specific hardware understand that, or is there a way in the broker to transform it?
It's up to the user to write the software to process the message published in MQTT so, for example, an arduino can be programmed with a MQTT client to get the message (topic+payload) from the MQTT broker and then do as instructed to throw or close the turnout. The topic and payload it sees is exactly the payload that was published. The current MQTT turnout topic + payload shows "/trains/track/turnout/88 THROWN" so the arduino sketch needs to know whether turnout 88 is one under its control, which pin it is attached to and to do whatever it has to do to throw it.

Likewise, it's up to the user to write the software to process the sensor activity and publish it into MQTT in the format we require so if we say, for example, that the topic is to be /RailroadID/sensor/{JMRI sensor system name} with payload ON or OFF (or perhaps ACTIVE or INACTIVE to use JMRI terminology), all you need to do is subscribe to topic /RailroadID/sensor/#, you will get a message for each sensor change so set the JMRI sensor nominated to the relevant status.
?
Retention (last entry only for that topic so that gives last known state) is set by the MQTT client publishing the message so JRMI will set most of those but not for sensors. Retention is enabled in the current JMRI MQTT turnout code. When a client subscribes to a MQTT topic, it has the option whether or not to receive the retained message. It seems to me that retention is a good thing but Speed has warned that on startup, JMRI may receive a lot of messages in a rush though can choose not to see them. Finding out the status but that is perhaps a good thing, JMRI sensors can be set to the last known status instead of Unknown.
As for making that user definable, I have no opinion on the granularity that may be desired, I'll leave that for someone else to respond.

If the system name in JMRI is effectively free text for each turnout or whatever, then we're free to use whatever we like in JMRI to match however we code the processing nodes. Everyone can do their own thing but the big gain we get is a fairly simple way of integrating our service nodes with JMRI.

Regards, Dave Mc

Thank you! That detail is helpful.

I see how people writing their own code for e.g. Arduinos can handle JMRI standardizing on particular text to publish. Certainly, we can have JMRI use reasonable defaults.

Are there pre-existing devices out there that don’t have the flexibility of an Arduino? I.e. a power switch that only knows ON and OFF, and doesn’t have a way to handle any other text in the payload?

Bob

On Dec 10, 2018, at 5:11 AM, Dave <davemc@...> wrote:

Bob,
Answering some of your questions, though I'm not an expert on this so hopefully others will add or correct as need be..

The existing JMRI turnout sends (unless very low-level configuration is done) “THROWN” and “CLOSED”. Does only specific hardware understand that, or is there a way in the broker to transform it?
It's up to the user to write the software to process the message published in MQTT so, for example, an arduino can be programmed with a MQTT client to get the message (topic+payload) from the MQTT broker and then do as instructed to throw or close the turnout. The topic and payload it sees is exactly the payload that was published. The current MQTT turnout topic + payload shows "/trains/track/turnout/88 THROWN" so the arduino sketch needs to know whether turnout 88 is one under its control, which pin it is attached to and to do whatever it has to do to throw it.

Likewise, it's up to the user to write the software to process the sensor activity and publish it into MQTT in the format we require so if we say, for example, that the topic is to be /RailroadID/sensor/{JMRI sensor system name} with payload ON or OFF (or perhaps ACTIVE or INACTIVE to use JMRI terminology), all you need to do is subscribe to topic /RailroadID/sensor/#, you will get a message for each sensor change so set the JMRI sensor nominated to the relevant status.

Retention (last entry only for that topic so that gives last known state) is set by the MQTT client publishing the message so JRMI will set most of those but not for sensors. Retention is enabled in the current JMRI MQTT turnout code. When a client subscribes to a MQTT topic, it has the option whether or not to receive the retained message. It seems to me that retention is a good thing but Speed has warned that on startup, JMRI may receive a lot of messages in a rush though can choose not to see them. Finding out the status but that is perhaps a good thing, JMRI sensors can be set to the last known status instead of Unknown.
As for making that user definable, I have no opinion on the granularity that may be desired, I'll leave that for someone else to respond.

If the system name in JMRI is effectively free text for each turnout or whatever, then we're free to use whatever we like in JMRI to match however we code the processing nodes. Everyone can do their own thing but the big gain we get is a fairly simple way of integrating our service nodes with JMRI.

Regards, Dave Mc

--
Bob Jacobsen
rgj1927@...

Bob,

The broker doesn’t care what the payload is at all. The only thing the broker is doing is handling the topics received and sending them out to “nodes” that subscribe. All it does is broker the communication.

Lets say I have a couple ESP8266 boards on my layout (These Arduino type boards with Wi-Fi in case you are not fimilar) and using a specific library they can publish and subscribe to an mqtt broker. Let’s also say these ESP8266 boards are control turnouts. When they are programmed the user would make the code do something based on a closed or thrown state received from the broker. The same board could read a button and publish the state of the turnout back so JMRI knows it was closed or thrown.?

So to answer your question MQTT can handle whatever JMRI gives it in terms of payload. The user just needs to program or configure the boards or whatever publishing and subscribing to topics to match the payloads from JMRI. I am just opposed to 0 and 1. I rather see on or off, or thrown or closed.?

Chris

I’m sorry I haven’t been clear.

I understand the case when you can change the programming in the nodes.

Is there a case where you _cannot_ change the programming in the nodes? For example, you bought a piece of equipment with its MQTT tech built in and already expecting certain strings.

Or is that case not important?

Bob

On Dec 10, 2018, at 5:52 AM, Chris Rood <crood58@...> wrote:

Lets say I have a couple ESP8266 boards on my layout (These Arduino type boards with Wi-Fi in case you are not fimilar) and using a specific library they can publish and subscribe to an mqtt broker. Let’s also say these ESP8266 boards are control turnouts. When they are programmed the user would make the code do something based on a closed or thrown state received from the broker. The same board could read a button and publish the state of the turnout back so JMRI knows it was closed or thrown.

So to answer your question MQTT can handle whatever JMRI gives it in terms of payload. The user just needs to program or configure the boards or whatever publishing and subscribing to topics to match the payloads from JMRI. I am just opposed to 0 and 1. I rather see on or off, or thrown or closed.

--
Bob Jacobsen
rgj1927@...

Yes the topic can be changed for the nodes. There is really no off the shelf equipment for layout control using MQTT that I know of. I would expect anyone using MQTT for layout control would be programming their own devices such as arduino and such.?

Chris

Hi all;

I'm the one who did this very simple mqtt addition to JMRI, quite happy to see such enthousiasm, yes topic is hardcoded, really that was for fun.

I just uploaded my Python turnout mqtt client there : . it subscribes to?/trains/track/turnout/# and

Regards;
Lionel

开云体育

Trying to look at the big picture here. Putting JMRI aside for the moment Surely this is a much better way of sending commands to trains via the rails instead of the current DCC protocol?

Andy


---
This email has been checked for viruses by AVG.

Andy:? As I understand it, DCC provides the fundamental data transfer along the track while MQTT is a higher level system that manages the information to be put in or read from the DCC data stream. So MQTT cannot replace DCC but make it easier to interface with other data systems such as JMRI, wifi, USB etc.

Jan

I was thinking of all the non obvious but necessary setting of bits in a huge and growing range of configuration registers.

Eg? Loco/4234/lights/ditch /on

Andy



On 12/10/2018 12:16 PM, Jan Carr wrote:

Andy:? As I understand it, DCC provides the fundamental data transfer along the track while MQTT is a higher level system that manages the information to be put in or read from the DCC data stream. So MQTT cannot replace DCC but make it easier to interface with other data systems such as JMRI, wifi, USB etc.

Jan

---
This email has been checked for viruses by AVG.