More d-rats public fork progress
Hello all,
The python3 d-rats fork is progressing slowly and there is still a lot that needs to be done.
The fork now should have the various e-mail gateways and APIs working.
The pluginsrv module is now working and there is a test program in the repositories.
Winlink and Mac support is now implemented.
All is not roses though.
I have a report that file transfers are not working between Mac systems, but is working with Mac to or from Windows.
The Mac user that reported this is trying to get more data to determine why this failure is occurring.
We are still seeing random segfault events.
We are also seeing random GTK Critical events logged to the console.
There are two main known causes of both those events:
1. Using deprecated GTK code. There is still a lot of this to be removed.
2. Calling GTK code improperly from threads.
Unfortunately neither the segfaults or the GTK Critical events messages usually give useful information as to the failure.
Sometimes we can find the cause of the GTK Critical event by noticing the operation that preceded it, other times by adding additional logging.
The segfaults really need learning the current program flow of the threads and making sure that it is done properly, and that also involves learning more about how GTK programming on python works,
My current course of action is going to be to start documenting the program flow module by module in my wiki, and at the same time doing code cleanups for those modules.
73,
-John, wb8tyw
|
Re: Running a D-Rats Ratflector on a dynamic IP system
Yes ...as Glen says NO-IP works great I have been using it since 2014 on my BPQ32 System and DX Cluster Node.
Lee WB4QOJ --------
On Apr 17, 2022 12:34 PM, Glen Strecker <gstrecker55@...> wrote: For anyone wanting to run a D-Rats ratflector off of a residential or commercial internet service that provides only dynamic IP addresses (in other words, your IP changes every time their network resets) you need to set up some method that monitors your IP address to continually link it to your ratflector name on the Internet so people can connect to it.? I use NoIP.com.? They can assign a name to your ratflector with a ddns.net extension that you can then share with your users so they can link to the system.? Here is a link to get a $5 coupon code to get started with your service .
|
Running a D-Rats Ratflector on a dynamic IP system
For anyone wanting to run a D-Rats ratflector off of a residential or commercial internet service that provides only dynamic IP addresses (in other words, your IP changes every time their network resets) you need to set up some method that monitors your IP address to continually link it to your ratflector name on the Internet so people can connect to it.? I use NoIP.com.? They can assign a name to your ratflector with a ddns.net extension that you can then share with your users so they can link to the system.? Here is a link to get a $5 coupon code to get started with your service .
|
Re: subst.py and gps.py module
John,
As a fellow software engineer, I read through your thorough and thoughtful description of subst.py and support your decision to toss the buggy module out.
73,
On Sat, Apr 16, 2022 at 11:08 AM John E. Malmberg < wb8tyw@...> wrote: Hello,
Started converting / testing the subst.py module.
It does not work in the existing code as near as I can determine, and
has an infinite loop in it once other bugs in it are fixed.
Just about 1/2 of the functions in the modules have bugs in it, and
fixing those are done basically done by removing most of those because
they are effectively doing almost nothing.
The only caller of this module is the gps.py.
It appears to be intended to do a replacement of the text between two
slash characters in the comment section of the GPS message, if that text
is found in a configuration file.
If not, the text is supposed to be passed through unchanged.? It
actually just removes the text in the current implementation.
The caller in gps.py discards the result of the conversion, so all this
code is doing is wasting space in memory and in the repository.
I am thinking of just yanking this module out, other than trying to get
it implemented correctly.
As it is currently implemented, it is creating a new copy of the
conversion substitution object for every GPSPosition object in d-rats,
as as these are used in the map point functions, there could be lots of
them.
It would have to be re-written to be what is known as a Singleton object
if it is to be retained to avoid the needless memory overhead.
73,
-John
|
subst.py and gps.py module
Hello,
Started converting / testing the subst.py module.
It does not work in the existing code as near as I can determine, and has an infinite loop in it once other bugs in it are fixed.
Just about 1/2 of the functions in the modules have bugs in it, and fixing those are done basically done by removing most of those because they are effectively doing almost nothing.
The only caller of this module is the gps.py.
It appears to be intended to do a replacement of the text between two slash characters in the comment section of the GPS message, if that text is found in a configuration file.
If not, the text is supposed to be passed through unchanged. It actually just removes the text in the current implementation.
The caller in gps.py discards the result of the conversion, so all this code is doing is wasting space in memory and in the repository.
I am thinking of just yanking this module out, other than trying to get it implemented correctly.
As it is currently implemented, it is creating a new copy of the conversion substitution object for every GPSPosition object in d-rats, as as these are used in the map point functions, there could be lots of them.
It would have to be re-written to be what is known as a Singleton object if it is to be retained to avoid the needless memory overhead.
73,
-John
|
First that has been updated is about GitHub workflow guidelines.
Not trying to scare anyone away from assisting development or testing, but common guidelines are needed for group coordination.
Certain things just do not automatically just work as expected with Microsoft Windows with GitHub or with cross-platform projects.
And this type of documentation will be needed for when a D-Rats GitHub Organization project is set up to make the project run smoothly.
Next is a growing list of things to watch for in Code Reviews or writing code to prevent bugs from creeping into the program.
And some documentation on how to use the Mailserv module and its current limitations.
There is still a PR in progress for fixing the Mailserv module, as it was recently discovered that if the POP3 server does not completely shutdown when the configuration is updated.
73,
-John
|
Re: Python3 fork progress
John,
Stellar progress! Thanks for giving so much of your time to this project. The D-Rats community admires and is very grateful for what you are doing.
As soon as the "spare time" arrives, I plan to look into how to update the Python packages I have and make a copy to test. As previously stated. AGW interface is high on my list.
Jack Spitznagel – KD4IZ
Science River LLC
Biomedical Consulting Services
toggle quoted message
Show quoted text
-----Original Message----- From: [email protected] < [email protected]> On Behalf Of John E. Malmberg Sent: Sunday, April 3, 2022 18:04 To: [email protected]Subject: [d-rats] Python3 fork progress Hello all, Some progress this weekend. The agwpe and wl2k modules are now both converted to python3. I have not done any testing with d-rats actually connecting to either as I do not have any way to test agwpe, and still do not know much about Winlink. I added mock servers for testing the agw and w2lk modules. I have added the code to use the python-sphinx module to generate some d-rats internals documentation from the DocStrings that I have been adding. Of course now that means that the generated documentation will need to be proofread and enhanced. I have not had a chance to learn the new python packaging, other than documenting how to setup a venv for building the packages. There are still a lot that I have not yet looked into, especially things like transfers between d-rats and internet e-mail, and other features like communication with GPS servers. 73, -John wb8tyw@.../wb8tyw@...
|
Hello all,
Some progress this weekend.
The agwpe and wl2k modules are now both converted to python3.
I have not done any testing with d-rats actually connecting to either as I do not have any way to test agwpe, and still do not know much about Winlink.
I added mock servers for testing the agw and w2lk modules.
I have added the code to use the python-sphinx module to generate some d-rats internals documentation from the DocStrings that I have been adding. Of course now that means that the generated documentation will need to be proofread and enhanced.
I have not had a chance to learn the new python packaging, other than documenting how to setup a venv for building the packages.
There are still a lot that I have not yet looked into, especially things like transfers between d-rats and internet e-mail, and other features like communication with GPS servers.
73,
-John
wb8tyw@.../wb8tyw@...
|
Re: Using pip installed modules with D-rats more safely
Thanks for posting that John and it's interesting how Python can run
in this form of a chroot jail.? I'm curious if it would be worth it
to consider developing a flatpak version similar to what the Chirp
team has done?? It might be especially helpful during this
transition time until all of the legacy stuff is modernized.
--David
KI6ZHD
On 03/28/2022 04:49 PM, John E.
Malmberg wrote:
toggle quoted message
Show quoted text
I just wrote a wiki article on running my D-rats fork
in a VENV.
There are a lot of ways to have python on systems, Particularly on
Windows and Mac, so this probably needs some help from users of
those python packages to flesh it out.
If you are currently running the "setup.py" in d-rats to install
it, this article is important for an upcoming change.
But it will also be useful in general, I hope.
I am using msys2 mingw64 python on Microsoft Windows as met the
following requirements for me.
1. It had a free license, I can use it on professional projects
with out a license conflict.
2. It had all of the packages that were required to run d-rats
available from a signed repository.
3. It was reasonably simple to setup and maintain.
And as a bonus, it originally had a bug that made some windows
features on d-rats not work, and the msys2 maintainers actually
quickly fixed that from my bug report.
If there are alternatives that meet those criteria for python on
Microsoft Windows, I would be interested in knowing them to try
them out.
Also, this year may be the last year for support of python3 on
Windows 7 or earlier.?? The maintainers of the windows specific
packages are dropping support for anything older than Windows 8.
Unfortunate, because while us Hams like the shiny new stuff, we
also do like to get the most life we can out of equipment and do
not see tossing out a perfectly good laptop just because it can
not run a newer OS.
73,
-John
|
Using pip installed modules with D-rats more safely
I just wrote a wiki article on running my D-rats fork in a VENV.
There are a lot of ways to have python on systems, Particularly on Windows and Mac, so this probably needs some help from users of those python packages to flesh it out.
If you are currently running the "setup.py" in d-rats to install it, this article is important for an upcoming change.
But it will also be useful in general, I hope.
I am using msys2 mingw64 python on Microsoft Windows as met the following requirements for me.
1. It had a free license, I can use it on professional projects with out a license conflict.
2. It had all of the packages that were required to run d-rats available from a signed repository.
3. It was reasonably simple to setup and maintain.
And as a bonus, it originally had a bug that made some windows features on d-rats not work, and the msys2 maintainers actually quickly fixed that from my bug report.
If there are alternatives that meet those criteria for python on Microsoft Windows, I would be interested in knowing them to try them out.
Also, this year may be the last year for support of python3 on Windows 7 or earlier. The maintainers of the windows specific packages are dropping support for anything older than Windows 8.
Unfortunate, because while us Hams like the shiny new stuff, we also do like to get the most life we can out of equipment and do not see tossing out a perfectly good laptop just because it can not run a newer OS.
73,
-John
|
Re: Where is the config file hidden?
Maurizio Bonjourno! We are using D-RATS over VHF, via IC 2820, IC 880, and ID 5100, it appears that it can sometimes be intermittent also, (Rx some stations not others). So this indicates to me some interference, although to be honest even that is not making complete sense as we have tried both simplex and repeater. We will continue to pursue this and see what we come up with. We did find eveything just fine, did a complete wipe and reinstall, difficult to say if any improvement.
If we find an answer we will report it here.
Grazie
--
AL M
KF5SMH
|
Re: Where is the config file hidden?
Ciao Al
config file:?in my case, on windows, it is here:
C:/Users/mauri/AppData/Roaming/D-RATS-EV/ and the filename is C:/Users/mauri/AppData/Roaming/D-RATS-EV/d-rats.config
in his case he should substitute "mauri" with his userfolder
if you can transmit and not receive it could be needed to reconnect to the ratflector. option 1: close? and reopen d-rats? option 2: open preferences panel and click save (this will force a reconnect to the same ratflector)
hope this helps, maurizio
Il giorno lun 28 mar 2022 alle ore 01:19 Al Massaro KF5SMH < almassaro1@...> ha scritto: Have one of the locals trying to delete all the D-RATS files so we can reinstall, He can Tx but not Rx, can't remember where that sill file is, he removed everything he could find in a search, still retains callsign. Help!
--
AL M
KF5SMH
|
Re: Where is the config file hidden?
We now have 3 stations that will not receive, transmit ok. All stations running V 0.3.9, not beta.
--
AL M
KF5SMH
|
Where is the config file hidden?
Have one of the locals trying to delete all the D-RATS files so we can reinstall, He can Tx but not Rx, can't remember where that sill file is, he removed everything he could find in a search, still retains callsign. Help!
--
AL M
KF5SMH
|
Re: Is anyone using d-rats with agwpe?
On 3/27/2022 9:51 AM, Jack Spitznagel (KD4IZ) wrote: John,
I would use AGWPE hook if it was available. I have been using the SV2AGW packet suite for several years. He maintains it well and I have found a number of programs that take advantage of it or provide it (like in Direwolf). I did not know that it was in the plan for the original D-Rats. Hello Jack, The AGWPE option is in the original D-Rats, you can find it in as one of the radio types in the configuration menu. I have no idea if the option works, as I have no way to test it. If it did work, I do not think I did anything to break it in my fork, but, I also do not have the resources to debug it, if I did accidentally break it. It really looks like the module was a work in progress. There are some unit tests, that need to be edited to be enabled in the agw.py module that someone with the agwpe driver can use to test it and attempt to fix it. Right now, it is coded to enable a "simulator" for the driver, and that of course will not work if a real agwpe driver is present on the same port: 8000. 73, -John
|
Re: Is anyone using d-rats with agwpe?
John,
I would use AGWPE hook if it was available. I have been using the SV2AGW packet suite for several years. He maintains it well and I have found a number of programs that take advantage of it or provide it (like in Direwolf). I did not know that it was in the plan for the original D-Rats.
Jack Spitznagel
KD4IZ
toggle quoted message
Show quoted text
-----Original Message----- From: [email protected] < [email protected]> On Behalf Of John E. Malmberg Sent: Sunday, March 27, 2022 09:19 To: D-Rats Mailing list < [email protected]> Subject: Re: [d-rats] Is anyone using d-rats with agwpe? On 3/25/2022 6:28 PM, John E. Malmberg wrote: Hello All,
Just curious if anyone is using agwpe connections with D-Rats? Based on what I found in testing the source code, and the lack of responses here, the answer is no. The existing source has a lot of debugging enabled and was incomplete in that it was not receiving the payload for any packets received from AGWPE when I tried to test it. So while I can not fully test it, a fixed agw.py module has now been merged into the python3_tyw branch of my fork. 73, -John, wb8tyw@.../wb8tyw@...
|
Re: Is anyone using d-rats with agwpe?
On 3/25/2022 6:28 PM, John E. Malmberg wrote: Hello All,
Just curious if anyone is using agwpe connections with D-Rats? Based on what I found in testing the source code, and the lack of responses here, the answer is no. The existing source has a lot of debugging enabled and was incomplete in that it was not receiving the payload for any packets received from AGWPE when I tried to test it. So while I can not fully test it, a fixed agw.py module has now been merged into the python3_tyw branch of my fork. 73, -John, wb8tyw@.../wb8tyw@...
|
Is anyone using d-rats with agwpe?
Hello All,
Just curious if anyone is using agwpe connections with D-Rats?
AGWPE is SV2AGW Protocol Engine, the part that d-rats uses is a TCP/IP server that runs on Windows and converts TCP/IP data to a software TNC using the sound card.
I can not fully test the fixes needed for that module, so I am using a simulator to simulate an agwpe port enough to validate the logic.
Not only do I need to make the python3 changes, I also need to fix some existing issues to be compatible with the official documentation at
.
73,
-John
|
Update to python3_tyw test fork - 2 bugs fixed.
Hello all,
If you are using the python3_tyw test fork, I just pushed an update to it that fixes outgoing file transfers.
Apparently at some point after testing the file transfers, I accidentally inserted a typo into the file, and this was missed before that change got merged in.
My guess is that I had a focus in a different window than I was typing in, and did not notice I had modified the wrong content.
However so far no one has notified me that they tripped on that bug.
I found that bug only because I am trying to get the agw.py module converted to python3, and discovered an anomaly in the trying to run test the existing code.
That made me have to look up the the AGW protocol from:
.
This showed me that d-rats was not using the correct decoding for the packets. If it works, it only works for a subset of the AGW protocol.
When I went to make the correction, I first did a search in the d-rats code to see if I could find similar bugs. Unfortunately I did.
This second bug is harmless to windows on x86 users and most systems likely to be running d-rats, but will prevent file transfers to and from systems that use big-endian storage. Testing the fix exposed the first bug above.
This is now fixed, but only in the python3_tyw fork.
There are also fixes to file and form transfers in this fork that are not in the upstream version, mainly in handling the recovery of partial transfers.
73,
-John
|
Contributing to my Python3 fork of d-rats
Hello all,
I am trying to maintain some standards on my fork and python3_tyw branch that are close to my professional programming work, so please look over this link before submitting PRs to my repository.
And please read before submitting tickets for my fork:
If you want to assist with reviewing coded, I just need to send an invite to your github username.
Thank you for your help and feedback.
This is also to help work out the kinks of project collaboration before creating a "GitHub Organization Project" for D-Rats with Maurizio as a new home for the D-Rats project.
73,
-John
|
Glen Strecker
Dave Slotter, W3DJS
Jack Spitznagel (KD4IZ)
Al Massaro KF5SMH