? I'm going to be away from the console of these test Pis for a few days, so it's a bit pointless to test a crash that only takes a few hours to happen. ?Instead, I'm going to remove rmsgw from ax25d and let the test cycle run while I'm away. ?

? Just a quick update on this, I checked on the Pis I had left in this state (with rmsgw disabled) and found that both were healthy after 8 days. ?To close the loop on this and demonstrate that rmsgw was causing the crash, I re-enabled rmsgw in ax25d.conf and restarted the daemon. ?There were no other changes and PAT was able to connect to check mail in the next connection cycle. ?About 12 hours later, I checked and found RMS had a faulted netstat output:

Active AX.25 sockets

Dest ? ? ? Source ? ? Device ?State ? ? ? ?Vr/Vs ? ?Send-Q ?Recv-Q

* ? ? ? ? ?MYCALL-10 ?(null) ?LISTENING ? ?000/000 ?0 ? ? ? 0 ? ??

I found it interesting that netstat printed "(null)" this time, instead of just an empty column. ?Not relevant, just interesting. ?After finding netstat like this, a quick axcall command crashed the Pi. ?I think this confirms that rmsgw is the culprit here.

Still, this might not be specific to rmsgw. ?I think I'm seeing this crop up with rmsgw because it is the only process that is actively accepting AX.25 connections on my Pi. ?I might see if I can use node to generate some traffic on the test box.

? Cheers
? Mike

I got the Geekworm Rpi 5 case. They have the Active Cooler and NVMe hat for a SSD. All fit and work good together. They also have the 5A power supplies.

Greg
KE5DXA

On Feb 19, 2024, at 9:52?AM, Kelly K7MHI via groups.io <kellykeeton@...> wrote:

There was a new beta released 8.6 which I loaded successfully on a pi5 latest yesterday after the wine environment is setup with all the VB OCX

just install varAC with the installer. I just loaded a new pi5 over the weekend and wine9 box 32/64 and the program ran successfully, I saw more stability possibly the font issue is now handled for example ..

Is the rtladr package required for an sdrplay rsp1a ?
--
Jay

WB2QQJ

Good afternoon,

I have been trying to get sdrtrunk to work with my rsp1a(sdrplay). The program runs but does not detect the rsp1a.. Am I missing something? It say there is no tuner.

Any help would be greatly appreciated,

Hello Everyone,

As I continue to learn Raspberry Pi OS Bookworm (Debian 12), I'm finding a strange inconsistency and I think it either depends on:

?? a) the hardware I'm using or the use of Raspberry Pi OS Lite (No GUI) vs Raspberry Pi OS Desktop (GUI)
?? b) different versions of Rpi-OS images where they are making changes mid-release


I'm curious if anyone can confirm this on your setups:


?? Raspberry Pi 4 with 4GB of RAM running Bookworm Lite (No GUI)
????? --
????? cat /etc/os-release
????? --
????? PRETTY_NAME="Debian GNU/Linux 12 (bookworm)"
????? NAME="Debian GNU/Linux"
????? VERSION_ID="12"
????? VERSION="12 (bookworm)"
????? VERSION_CODENAME=bookworm
????? ID=debian
????? HOME_URL=
????? SUPPORT_URL=
????? BUG_REPORT_URL=

????? --
????? Shows all config files in /boot are symlinks to /boot/firmware/.? That makes things backwards compatible:
??????? --
? ?? ??? ls -la /boot/*.txt
??????? lrwxrwxrwx 1 root root 20 Oct? 9 20:39 /boot/cmdline.txt -> firmware/cmdline.txt
???? ? ? lrwxrwxrwx 1 root root 19 Oct? 9 20:39 /boot/config.txt -> firmware/config.txt
??????? --

????? Here is what is in the new /boot/firmware directory for config files
??????? --
??????? ls -la /boot/firmware/*.txt
??????? -rwxr-xr-x 1 root root? 132 Oct? 9 21:00 /boot/firmware/cmdline.txt
??????? -rwxr-xr-x 1 root root 1364 Oct 27 17:39 /boot/firmware/config.txt
??????? -rwxr-xr-x 1 root root? 145 Oct? 9 20:57 /boot/firmware/issue.txt
??????? --


Now compare that to a Raspberry Pi 5 with 8GB RAM running Bookworm Desktop (GUI enabled):

?? $ cat /etc/os-release
?? --
?? PRETTY_NAME="Debian GNU/Linux 12 (bookworm)"
?? NAME="Debian GNU/Linux"
?? VERSION_ID="12"
?? VERSION="12 (bookworm)"
?? VERSION_CODENAME=bookworm
?? ID=debian
?? HOME_URL=
?? SUPPORT_URL=
?? BUG_REPORT_URL=
?? --

?? $ ls -la /boot/*.txt
?? -rw-r--r-- 1 root root 92 Feb? 5 18:49 /boot/cmdline.txt
?? -rw-r--r-- 1 root root 91 Feb? 5 18:49 /boot/config.txt
?? lrwxrwxrwx 1 root root 18 Dec? 4 21:04 /boot/issue.txt -> firmware/issue.txt

????? These configuration files only contain text like:
? ? ???? --
? ? ? ?? DO NOT EDIT THIS FILE

???? ? ? The file you are looking for has moved to /boot/firmware/config.txt
? ? ???? --

?? The new REAL location of these files is now ONLY in /boot/firmware..
?? --
?? $ ls -la /boot/firmware/*.txt
?? -rwxr-xr-x 1 root root? 132 Feb? 5 18:57 /boot/firmware/cmdline.txt
?? -rwxr-xr-x 1 root root 1315 Feb 14 12:45 /boot/firmware/config.txt
?? -rwxr-xr-x 1 root root? 145 Dec? 4 21:04 /boot/firmware/issue.txt
?? --


By NOT making /boot/cmdline.txt a symlink to /boot/firmware/cmdline.txt, this BREAKS backwards compatibility with various tools, documentation, etc.? I'm curious if anyone knows WHERE and WHY this is happening?

--David
KI6ZHD

Hello Everyone,

As I continue to learn Raspberry Pi OS Bookworm (Debian 12), I'm finding a strange inconsistency and I think it either depends on:

?? a) the hardware I'm using or the use of Raspberry Pi OS Lite (No GUI) vs Raspberry Pi OS Desktop (GUI)
?? b) different versions of Rpi-OS images where they are making changes mid-release


I'm curious if anyone can confirm this on your setups:


?? Raspberry Pi 4 with 4GB of RAM running Bookworm Lite (No GUI)
????? --
????? cat /etc/os-release
????? --
????? PRETTY_NAME="Debian GNU/Linux 12 (bookworm)"
????? NAME="Debian GNU/Linux"
????? VERSION_ID="12"
????? VERSION="12 (bookworm)"
????? VERSION_CODENAME=bookworm
????? ID=debian
????? HOME_URL=
????? SUPPORT_URL=
????? BUG_REPORT_URL=

????? --
????? Shows all config files in /boot are symlinks to /boot/firmware/.? That makes things backwards compatible:
??????? --
? ?? ??? ls -la /boot/*.txt
??????? lrwxrwxrwx 1 root root 20 Oct? 9 20:39 /boot/cmdline.txt -> firmware/cmdline.txt
???? ? ? lrwxrwxrwx 1 root root 19 Oct? 9 20:39 /boot/config.txt -> firmware/config.txt
??????? --

????? Here is what is in the new /boot/firmware directory for config files
??????? --
??????? ls -la /boot/firmware/*.txt
??????? -rwxr-xr-x 1 root root? 132 Oct? 9 21:00 /boot/firmware/cmdline.txt
??????? -rwxr-xr-x 1 root root 1364 Oct 27 17:39 /boot/firmware/config.txt
??????? -rwxr-xr-x 1 root root? 145 Oct? 9 20:57 /boot/firmware/issue.txt
??????? --


Now compare that to a Raspberry Pi 5 with 8GB RAM running Bookworm Desktop (GUI enabled):

?? $ cat /etc/os-release
?? --
?? PRETTY_NAME="Debian GNU/Linux 12 (bookworm)"
?? NAME="Debian GNU/Linux"
?? VERSION_ID="12"
?? VERSION="12 (bookworm)"
?? VERSION_CODENAME=bookworm
?? ID=debian
?? HOME_URL=
?? SUPPORT_URL=
?? BUG_REPORT_URL=
?? --

?? $ ls -la /boot/*.txt
?? -rw-r--r-- 1 root root 92 Feb? 5 18:49 /boot/cmdline.txt
?? -rw-r--r-- 1 root root 91 Feb? 5 18:49 /boot/config.txt
?? lrwxrwxrwx 1 root root 18 Dec? 4 21:04 /boot/issue.txt -> firmware/issue.txt

????? These configuration files only contain text like:
? ? ???? --
? ? ? ?? DO NOT EDIT THIS FILE

???? ? ? The file you are looking for has moved to /boot/firmware/config.txt
? ? ???? --

?? The new REAL location of these files is now ONLY in /boot/firmware..
?? --
?? $ ls -la /boot/firmware/*.txt
?? -rwxr-xr-x 1 root root? 132 Feb? 5 18:57 /boot/firmware/cmdline.txt
?? -rwxr-xr-x 1 root root 1315 Feb 14 12:45 /boot/firmware/config.txt
?? -rwxr-xr-x 1 root root? 145 Dec? 4 21:04 /boot/firmware/issue.txt
?? --


By NOT making /boot/cmdline.txt a symlink to /boot/firmware/cmdline.txt, this BREAKS backwards compatibility with various tools, documentation, etc.? I'm curious if anyone knows WHERE and WHY this is happening?

--David
KI6ZHD

Hmmm..this shows ax25 version 0.11.8

newer? changes?

Just sorta stumbled upon this..haven't tried anything tho...

On 2024-02-12 12:23 a.m., David Ranch wrote:

Hello Mike,

Again, thank you for the detailed email and I think this all helps in tracking down the real issue here.? I've been discussing this on the side with Bernard F6BVP who maintains FPAC (node) and FBB (BBS) and uses the ROSE protocol heavily.? He reported that he's "running three ROSE/FPAC nodes on a local network and I haven't observed any connections issues with Raspbian OS 64bit for a long time nor with Ubuntu (20.04)".? He showed months of uptime with LOTS of connections without either any panics or any orphaned AX.25 connections.? One key point he mentioned is that he does NOT have any RF connections, it's all via AXUDP and he also noted he's NOT using mkiss for linking the AXUDP to the kernel with kissattach.

--David

On 02/10/2024 12:27 PM, Michael Dunn wrote:

? Hi everyone,

Been quiet on this topic recently as I haven't had much to report. ?I've had a lack of any crashes for over 12 days now, which seems to be related to disabling Pat in my environment. ?Please don't jump to conclusions here; this is a complex issue. ?As I've said in the past, Pat doesn't appear to be the cause of the crash, just the process that trips over the kernel garbage to trigger it.

Since I've had some time to think about this problem, here's what I've noticed:

? - Jon and I both run rmsgw. ?We both have crashes on the system running rmsgw.
? - The process that triggers the crash is mobile (beacon, pat, netstat), but is never rmsgw.
? - Jon and I both have few outside connections to rmsgw. ?My last outside connection was 10 days ago.
? - Jon has crashes in just a few hours; my crashes take days to weeks.
? - Jon frequently self checks his mail (possibly hourly? but I don't think he stated). ?I self check my mail infrequently (daily).

Based on these facts, my theory is this. ?rmsgw puts the kernel in some sort of bad state. ?This state is tripped over later by some unsuspecting process, causing the kernel crash.

To prove this out, I need to separate my rms server from my rms client. ?Unfortunately, I have only one radio, so I decided to take this test off-air. ?In my test setup, I built 2 Pi's, let's call them RMS and PAT to distinguish their roles. ?The Pis were built by imaging the SD card from PROD in the manner I've described previously.

Instead of connecting to a radio, I simply connected sound card to sound card using a pair of TRS cables (headphone to microphone in both directions). ?Direwolf was configured to match and PTT was disabled. ?RMS ran rmsgw from ax25d and a shell loop on PAT checked my mail every 30 minutes using pat -s (send only, as not to eat my actual inbox).

I let this setup run over night. ?In the morning, neither Pi had crashed, but I did notice that PAT was no longer able to connect to RMS. ?Tracking through the logs, it looks like about 7 hours after I setup the test, connections started failing. ?Digging into the RMS Pi, I found the netstat condition that Jon first reported. ?Note ax0 missing from the device column:

Dest ? ? ? Source ? ? Device ?State ? ? ? ?Vr/Vs ? ?Send-Q ?Recv-Q

* ? ? ? ? ?MYCALL-10 ? ? ? ? ?LISTENING ? ?000/000 ?0 ? ? ? 0

?

The kernel on RMS had been trashed, but the Pi was still operating. ?Checking the PAT Pi, netstat output looked normal. ?Realizing I still needed an AX25 event to trigger the crash, I used axcall on RMS ?to generate some traffic. ?The RMS Pi immediately crashed, blaming axcall as it went down:

[61160.353159] CPU: 1 PID: 130380 Comm: axcall Tainted: G ? ? ? WC ? ? ?6.1.0-rpi7-rpi-vB #1 Debian 1:6.1.63-1+rpt1

?

For me, this is great news. ?I have an off-air way to quickly show the problem. ?This also continues to show that the crash is mobile between processes and demonstrates an unrelated trigger event. ?Next steps are to reproduce the crash to ensure it is reliable. ?I'm also going to move RMS and PAT out of the RF environment (e.g. the other end of the house) to ensure there is no RFI element.

? Cheers
? Mike

Hello Mike,

Again, thank you for the detailed email and I think this all helps in tracking down the real issue here.? I've been discussing this on the side with Bernard F6BVP who maintains FPAC (node) and FBB (BBS) and uses the ROSE protocol heavily.? He reported that he's "running three ROSE/FPAC nodes on a local network and I haven't observed any connections issues with Raspbian OS 64bit for a long time nor with Ubuntu (20.04)".? He showed months of uptime with LOTS of connections without either any panics or any orphaned AX.25 connections.? One key point he mentioned is that he does NOT have any RF connections, it's all via AXUDP and he also noted he's NOT using mkiss for linking the AXUDP to the kernel with kissattach.

--David

On 02/10/2024 12:27 PM, Michael Dunn wrote:

? Hi everyone,

Been quiet on this topic recently as I haven't had much to report. ?I've had a lack of any crashes for over 12 days now, which seems to be related to disabling Pat in my environment. ?Please don't jump to conclusions here; this is a complex issue. ?As I've said in the past, Pat doesn't appear to be the cause of the crash, just the process that trips over the kernel garbage to trigger it.

Since I've had some time to think about this problem, here's what I've noticed:

? - Jon and I both run rmsgw. ?We both have crashes on the system running rmsgw.
? - The process that triggers the crash is mobile (beacon, pat, netstat), but is never rmsgw.
? - Jon and I both have few outside connections to rmsgw. ?My last outside connection was 10 days ago.
? - Jon has crashes in just a few hours; my crashes take days to weeks.
? - Jon frequently self checks his mail (possibly hourly? but I don't think he stated). ?I self check my mail infrequently (daily).

Based on these facts, my theory is this. ?rmsgw puts the kernel in some sort of bad state. ?This state is tripped over later by some unsuspecting process, causing the kernel crash.

To prove this out, I need to separate my rms server from my rms client. ?Unfortunately, I have only one radio, so I decided to take this test off-air. ?In my test setup, I built 2 Pi's, let's call them RMS and PAT to distinguish their roles. ?The Pis were built by imaging the SD card from PROD in the manner I've described previously.

Instead of connecting to a radio, I simply connected sound card to sound card using a pair of TRS cables (headphone to microphone in both directions). ?Direwolf was configured to match and PTT was disabled. ?RMS ran rmsgw from ax25d and a shell loop on PAT checked my mail every 30 minutes using pat -s (send only, as not to eat my actual inbox).

I let this setup run over night. ?In the morning, neither Pi had crashed, but I did notice that PAT was no longer able to connect to RMS. ?Tracking through the logs, it looks like about 7 hours after I setup the test, connections started failing. ?Digging into the RMS Pi, I found the netstat condition that Jon first reported. ?Note ax0 missing from the device column:

Dest ? ? ? Source ? ? Device ?State ? ? ? ?Vr/Vs ? ?Send-Q ?Recv-Q

* ? ? ? ? ?MYCALL-10 ? ? ? ? ?LISTENING ? ?000/000 ?0 ? ? ? 0

?

The kernel on RMS had been trashed, but the Pi was still operating. ?Checking the PAT Pi, netstat output looked normal. ?Realizing I still needed an AX25 event to trigger the crash, I used axcall on RMS ?to generate some traffic. ?The RMS Pi immediately crashed, blaming axcall as it went down:

[61160.353159] CPU: 1 PID: 130380 Comm: axcall Tainted: G ? ? ? WC ? ? ?6.1.0-rpi7-rpi-vB #1 Debian 1:6.1.63-1+rpt1

?

For me, this is great news. ?I have an off-air way to quickly show the problem. ?This also continues to show that the crash is mobile between processes and demonstrates an unrelated trigger event. ?Next steps are to reproduce the crash to ensure it is reliable. ?I'm also going to move RMS and PAT out of the RF environment (e.g. the other end of the house) to ensure there is no RFI element.

? Cheers
? Mike