Keyboard Shortcuts
Likes
Search
Locked MQTT Connection in JMRI
|
I just noticed last night that there is now a MQTT connection type in JMRI. This is exciting. Especially for me, since I was planning on using MQTT connection in JMRI through a custom script. IT seems though that this connection type is still in development even though it is in a production release. From what I read it is only available in for turnouts right now. When I added the connection and went to turnouts it seems that it created another internal connection type for the MQTT connection.?
I was hoping if someone could share if this development is going to actively grow for this connection type to include sensors, lights, records, etc.? I think that in the connection type a username and password field should be included in case someone has username and password required on the MQTT server and also the ability to choice what the master topic will be.? Again, I think that this is great for JMRI. Especially since, model railroads are using Arduino based boards such as the ESP8266 and ESP32 which are commonly used with MQTT.? Chris? |
|
Has anyone been able to get this to work? I have MQTT broker set up on localhost:1883 and I can pub.sub from all sorts of devices, so I know my infrastructure works. But I try to pub to /trains/MT1 either closed or thrown and I cannot get JMRI to respond.
I'm reading through the JMRI code to see if there is something I am missing, but thought I would ask to see if there is a trick I am missing. Nathan |
|
Got it...
the MQTT topic is like this: /trains/track/turnout/1 where 1 is the address of my turnout. Now what I really need is for mqtt to be supported for sensors/block detection. Anyone got this to work using python, seems like some people have come close when I hunt around. My goal is to use all these adafruit?feathers I have sitting around to send mqtt messages about block detection with a goal being a track powered wireless block detection system. Look, mom, no wires! |
|
Like I said in my previous post I am also going to do third, but you were the only to respond from August Nathan. I think most people in model railroading don¡¯t know about MQTT. I personally think it is the future for turnouts, signals, and more.?
I am sure you can write a script to using the existing mqtt connection in JMRI to get it to work with what you are looking for. I haven¡¯t done it yet but have researched it. chris |
|
emmanuel ALLAUD
Hi, I know that there are some python libraries to handle mqtt connection. This might make it easier to write scripts to pub/sub in reaction from sensors or to control turnouts. Just my 2 cents... Manu Le mer. 5 d¨¦c. 2018 18:40, Chris Rood <crood58@...> a ¨¦crit?: Like I said in my previous post I am also going to do third, but you were the only to respond from August Nathan. I think most people in model railroading don¡¯t know about MQTT. I personally think it is the future for turnouts, signals, and more.? |
¿ªÔÆÌåÓýI agree, higher level protocols in a real network are (hopefully) the future of layout control. And I think the communication to the network should come directly from the layout control bus. That¡¯s why I did the LocoNet to MQTT gateway () and programs like JMRI should get the information from there, not from a slow point to point interface. In my opinion this is also the correct layer for multi system integration. Why develop just another hardware and network layer like OpenLCB (with all the difficulties to get manufacturers to accept and support it) ?when integration simply can be done on the presentation and application layer level? Hans ? ? |
|
Note: I didn¡¯t write the MQTT code in JMRI, so might not know what I¡¯m talking about there.
toggle quoted message
Show quoted text
I think it would be good to add JMRI Sensor code to go alongside the JMRI Turnout code. Some parts of it seem to be there, but large parts are missing. In particular, there¡¯s nothing about mapping MQTT traffic to particular sensors: How to figure out names, states, etc. Is there somebody with MQTT knowledge interested in working with me on that? I can do the JMRI part, but don¡¯t know enough about how people would want it to work. Bob On Dec 5, 2018, at 5:53 AM, Nathan Tableman <nathan@...> wrote: |
|
Bob,
I would be interested in helping. MQTT protocol is very light and powerful. It also allows multiple programming languages to get data or pass data to each other.? To to get a little of educated on MQTT. You can check this out -? JMRI currently offers MQTT connection and it makes the root topic /trains/ ?and wants a connection to MQTT broker (server) which passes the data to all the connections that subscribe to a topic. I think JMRI should allow to change the ¡°root¡± topic, because it currently does not.? You asked about how will JMRI know the system name or username. I invasion that the topic for a turnout will be something of the following. I use the root topic of trains for example. trains/turnouts/username/? MQTT uses payloads, which can be user friendly, so for a turnout it would be thru or thrown. if a sensor: trains/sensors/username/ Payload: Active or Unactive This could be done for block detectors, signals, lights, etc. I also think it is a good idea to have the fast clock info published to MQTT. Also, would like to see the ability for people to expand on MQTT in JMRI with scripts for addition ¡°play¡± value like jynstruments. Chris |
|
Bob,?
I'd be happy to help getting MQTT subscriptions on the inside of JMRI. We already have it working in the Python side, but built-in would be better.? I have about 25 ESP-8266 ESP-01 modules up and running (with Paho MQTT in JavaScript (mqttws31.js) )? already, so testing it is also easy. |
|
- I would suggest you make the topic user configurable. With fixed topics it starts to be real fun if more then one user uploads to a public server like HIveMQ ;-)
- In order to make everything simple I would suggest to make the topic structure as simple as possible, i.e. as few topics as possible. In general I find it easier to use one topic and embed the message e.g. in a JSON string that can be interpreted by the application - Adding Fast Clock should be a no brainer, I think. Maybe you have seen my video on implementing the Fast Clock in Node RED and feeding it back to LocoNet via MQTT (https://www.youtube.com/watch?v=6RjzpHJu-vU). The advantage in case of the Digitrax system is that you now have an FC that also can support date, not just time. Hans |
¿ªÔÆÌåÓýSo I have been pondering this: I use MQTT for everything, hence my interest with JMRI - I have a large saltwater aquarium and all my control equipment and monitoring is home made and sends signals around using MQTT, as does all my home automation stuff.But as I thought about this¡I am not sure it really is needed for JMRI - I am trying to think where MQTT would be superior to WebSockets. Im using WebSockets in my iOS app, my physical throttle device (rotary encoder), and a host of other things. Then I thought, ok, so I have a rasp pi zero for my MQTT broker and it also has node red on it and I can use the pins for input and output in node red too, so I made a flow that takes in JMRI type messages in either MQTT topics or JSON and changes both to JSON to send to the WebSockets on my JMRI computer. The reason I wanted to put node red in there is I want an abstraction layer, I can take in GPIO from the zero, MQTT, and JSON and more¡and all end up in JMRI. I can even create a test harnesses to fake train data to experiment with my automation. The real goal is to use either the macOS or AWS object tracker ML to send in positional data for all the open areas of the layout and then maybe (if I need them) ¡°normal¡± sensors for the hidden areas. A good camera above the layout it a hell of a lot fewer wires. Plus it turns out the tops of my locomotives are different enough I can even ID them from above. Working on stitching this all together now from prototype on a small track to a larger layout. So yeah MQTT support: seems like a good thing to expand, as maybe people dont also want to fiddle with node red! My vote is to keep the topics simple and use JSON payloads - they are already there in the WebSocket, so why not use the same ones? Nathan
|
|
Folks,
I'm also keen to see the MQTT implementation expanded to include sensors and signals (signal masts). MQTT is a quite simple technology, easy to set up and use, no rocket science there - it's new to me but the learning curve is easy. As has been mentioned, having JMRI able to interface to the MQTT broker opens the way for people to easily expand on JMRI capabilities with different hardware and ¡°play¡±/explore new ideas. The main issue at the moment seems to me to be agreement on how to structure the MQTT topics and payload, allowing flexibility for the downstream apps to perform the tasks required while remaining within the capabilities of a JMRI connection. That's where you need to tap into people's experiences to get a better idea what is preferred so you can begin with the end or multiple ends in mind. Obviously there has already been some consideration to come up with the /trains/track/turnout/1 structure. Regards, Dave Mc. |
¿ªÔÆÌåÓýI am going to guess that the reason the path/topic is used normally is that it gives a hierarchy. So you can fo the # and + thing to segment what you are interested in. That said, almost every other MQTT device I have use JSON in the payload¡and node red loves a good JSON.Standardizing all messages on JSON would allow a large degree of interoperability. Perhaps /trains/blocks/ then the normal JSON for blocks and so forth. But the thing I keep coming back to is how do I make this as simple as I can on the sensor device so I can crank them out without writing a lot of code. I am exploring sending in the MAC+sub# and then using node red to map this to a JMRI sensor. So if you wanted to make this work without node red you¡¯d need to take the hardware field (or some UUID) on JMRI and map it to the third level in the topic: /trains/sensors/001422012345+1 with JSON of?{"type":"sensor","data":{"name":"001422012345+1¡±,"state":4}} or payload of 1 or 0 if people dislike the JSON. I hope my thinking makes sense - the goal being I do not want to custom config my sensors, I just want to deploy them and virtually wire them up. I want the?Mac+sub because I wire up?multiple sensors to the same ESP and this?allows me to?separate out the input and output?really easily for bidirectional communications over MQTT. Nathan
|
|
I agree with what everyone is saying. I do home automation also including node red. The key is to offer people flexibility, so they aren¡¯t forced to use other software if they choose not to. I guess like Nathan said you can use the WebSockets and create a WebSocket to MQTT gateway using nodered on a computer, or other custom methods such as an ESP8266/ESP32, or Raspberry Pi.?
I also do agree with the JSON too. JSON allows you to pass multiple pieces of information at once.? I also agree that the Fast Clock then use a date also ?like Hans. The key is to get information and data in and out in the shortest amount of time. I think the key is to make JMRI flexible and powerful with IOT. This will allow a lot more control of not just the layout but the whole area where the layout is, such as room lighting, etc. |
|
it would be great if the community could converge on how JMRI should work with MQTT devices.
Some boundary conditions: 1) JMRI has specific control metaphors: Turnout (i.e. one-bit digital output), Sensor (i.e. one-bit digital input), Light, SignalMast/SignalHead, Reporter, etc. They each function a bit differently. _How_ they work with MQTT is something that needs to be defined: What does it mean when a MQTT-based light wants to set intensity to 75%? What should a Sensor look for to decide whether it should go to ACTIVE, INACTIVE, UNKNOWN or INCONSISTENT, and if something else happens, what should it do? 2) JMRI hardware flexibility comes from the system name metaphor. A Turnout (T) in the MQTT system (M) can have a system name like MT/stuff/morestuff/123/$5/dancing-bear-emoji for all JMRI cares. So you can put a lot of information into that system name to define the operation, so long as it can be unambiguously parsed. It¡¯s even possible to have different formats for the same underlying hardware devices, c.f. CT3001 vs CT3b1 and all the weird and wonderful MERGE CBUS formats. So it would be _great_ if the MQTT community could come up with an addressing approach (which can have alternatives, i.e. with and without JSON) and an example of how a few cases, i.e. Sensor and Light. should work with it. We can then get that coded up for testing during the semester break that¡¯s coming. Bob -- Bob Jacobsen rgj1927@... |
On Dec 7, 2018, at 09:35, Bob Jacobsen <rgj1927@...> wrote:What do you mean by look for? Ideally it looks like this. /trains/hardwareID/item Hardware ID = Mac address Item = sub item on that device JRMI has a hardware id of MAC/item JSON in the payload = to the JSON used in WebSockets. This would make it THE most flexible, but the LEASE human readable. Not sure who the MQTT community is¡but I can write use cases, if that is what you are asking.
|
|
A few more things to think about...
- will JMRI subscribe to every topic that contains a sensor/turnout/mast? I have found there are limits (in performance and memory) for subscribing to too many things. About a year ago I created this Jython script () to publish and subscribe to MQTT for "Virtual Masts" from within JMRI. I simply used the "Username" to keep the topic for each virtual mast, which was simple at the time: "mast.0003". Today I would go with /TxNamib/mast.0001/03, where the mast.0001 is the board where the masts are connected to.? - Do we use retention? I found it convenient in my ESP implementation to save the state of the turnout in /TxNamib/servo.0003/01 as a '0' or a '1' with retention on, so when the ESP boots up, it knows where to move the servo without keeping the state saved in local EEPROM. I know some don't like physical addressing boards, but I found it convenient to simply subscribe to /TxNamib/servo.0003/# for all 16 turnouts connected to the ESP. So if retention is used, JMRI is going to turn on and get hundreds of messages (that it needs to keep up with) and also might have the "INCONSISTENT" state if it determines the turnout to be otherwise.? - JMRI typically (or used to) only generate a message to hardware when there is a state change, so without retention in MQTT, the servo won't know where to go until there is a change in JMRI. - Like Bob said, somethings work different than others. A block detector sensor is by nature controlled by the block detection circuit, but a global "I need the Dispatcher's attention" sensor could be turned on by more than one. (And who turns it off?). Masts should also be single sourced, in this case I would say JMRI determines the aspect. - Will JMRI allow us to pick what messages to send? For example, "Active" and "Inactive" versus '1' and '0' versus 'True' or 'False'? Might not sound important here, but who wrote the code on the other side, it matters over there. - And Masts having 'Clear', 'Approach' and Signal Heads using 'Green', 'Flashing Red', 'Dark', ...? Or do we start with only Heads? - JMRI also need to allow, not only the username and password as mentioned earlier, but also the QOS (quality of service), and other (or future), settings. [ Disclaimer about my Jython script: I can't promise the "threading" part is working well enough, sometimes you need to restart JMRI. The JButton on the Desktop was a simple way to remove the Listeners when you needed to rename a mast, instead of restarting JMRI. ] |
¿ªÔÆÌåÓýHi Everyone, ? I am very interested in this conversation.? As a DIYer and C/C++ and Java hobbyist programmer, QA Test Specialist and Project Manager in my day job, I like where this is going.? And yes there is a but¡ ? But, I think that two if not three areas of communications are being combined in to this single topic: ¡°What will be the JMRI specification¡±.? These are JMRI, MQTT and Hardware and from what I read these are be talked about as if JMRI will set the spec for all three. ? Again my opinion so shout me down if you want but JMRI can only deal with how the MQTT Broker talks to JMRI and that may be the other way around as with DCC Systems.? (Each DCC Systems specs how JMRI will talk to it.)? How the MQTT Broker talks to the Hardware, I believe is already defined. ? JMRI has defined how scripts talk about Turnouts, Sensors, Signal Heads and Signal Mast as well as other items so putting this on a communications line to the MQTT is basically done. All that needs to be decided is the format to send to/read from ?MQTT, again what I read, that is mostly done as well /Trains/Turnout/Board ID/Item ID/Command or /Trains/Sensor/Board ID/Item ID/Data. ? How the MQTT Broker passes this information to the Hardware is up to the MQTT Broker designer/coder (the MQTT Specification).? As is how the Hardware gets the information from MQTT is already defined and how the Hardware deals with that information it gets it is up to the Hardware designer. ? There may well be two or more flavors of MQTT Brokers, as there are several flavors of DCC Systems. ? Just my two cents, feel free to disregard or flame as desired. ? Victor ? ? Victor Gilbert Supertendent Sugarwood Railroad ????????????? ___???????????? __ _????????????????????????????????????????????? _???????????? ___????????????? _?????? _??????????????????????????????? _?? ??????o O O? / __|?? _? _??? / _` |? __ _????? _ _? __ __ __? ___???? ___??? __| |?????????? | _ \?? __ _???? (_)???? | |????? _ _??? ___??? __ _??? __| |? ?????o?????? \__ \? | +| |?? \__, | / _` |??? | '_| \ V? V / / _ \?? / _ \? / _` |?????????? |?? /? / _` |??? | |???? | |???? | '_|? / _ \? / _` |? / _` |? ?????o?????? |___/?? \_,_|?? |___/? \__,_|?? _|_|_?? \_/\_/? \___/?? \___/? \__,_|?????????? |_|_\? \__,_|?? _|_|_?? _|_|_?? _|_|_?? \___/? \__,_|? \__,_|? ????TS__[O]? |"""""| |"""""| |"""""| |"""""| |"""""| |"""""| |"""""| |"""""| |"""""| |"""""| |"""""| |"""""| |"""""| |"""""| |"""""| |"""""| |"""""| |"""""| ??? {======|_|???? |_|???? |_|???? |_|???? |_|???? |_|???? |_|???? |_|???? |_|???? |_|???? |_|???? |_|???? |_|???? |_|???? |_|???? |_|???? |_|???? |_|???? | ? ./o--000'"`-0-0-'"`-0-0-'"`-0-0-'"`-0-0-'"`-0-0-'"`-0-0-'"`-0-0-'"`-0-0-'"`-0-0-'"`-0-0-'"`-0-0-'"`-0-0-'"`-0-0-'"`-0-0-'"`-0-0-'"`-0-0-'"`-0-0-'"`-0-0-'" ? ? From: [email protected] [mailto:[email protected]] On Behalf Of Speed ? A few more things to think about... |
Bob Jacobsen