Keyboard Shortcuts
Likes
Search
Gemini-2 stops tracking & losses comms to PC and HC during NINA sessions
? I¡¯m experiencing a new and troubling issue with my Losmandy mount and seeking suggestions/guidance. After roughly 100 successful sessions since receiving the mount (GM811G w/level 6) in 2023 I¡¯ve had multiple instances this month where in the middle of taking exposures using NINA the mount stops tracking (evidence: hours of image files showing nothing but star trails), the virtual Gemini hand-controller cannot communicate with the mount, and the physical hand control responds to key presses until I have it send commands to the Gemini controller at which point it stops responding. The only way to get the system to function is to cycle the power at which point it will work for the rest of the night, exhibiting the standard very good guiding (0.5 to 0.8 RMS) typically for my system. I have no issues with meridian flips, using PHD2 to calibrate the mount/guide camera + run guiding assistance, initial slew/centering, plate solving, etc. mount limits are set properly. No issues with the windows 11 PC communicating via USB to the cameras (ZWO & QHY), EFW, AEF, Pegasus rotator & power box. In the first instance the issue happened in the middle of the night, well after the (successful) meridian flip and well away from the safety limits. After this initial occurrence I changed the gemini-2 coin battery since it was the original battery from August 2023. The old battery was showing 3.06 vdc, new battery initially measures as 3.3vdc (now 3.107 Vdc when read via the web interface). I had three successful overnight runs after changing the battery so I thought I¡¯d solved the issue ¡ until last night when the same problem occurred. Fortunately, it happened ten minutes into NINA¡¯s sequence so I was able to intervene. Cycling the power, cold starting at the CWD position, followed by un-pausing NINA¡¯s sequencer and the system took good images for the remainder of the night. NINA parked the mount but, oddly, upon checking the system in the morning the NINA sequence was still waiting for confirmation that the mount had parked (even though it had). I found again that the virtual hand control could not communicate with the mount and the physical hand controller seems to freeze when asked it to send a command to the gemini-2 controller. ? The ASCOM Gemini log files have the following lines starting at the time when the Gemini controller no longer respond: Various RetransmitUDP commands followed by: 02:09:17.697 Exception???????????????? TID:11 [Error]: System.NullReferenceException: Object reference not set to an instance of an object.[0D][0A]?? at ASCOM.GeminiTelescope.GeminiHardwareBase.UDPSendNACK() in C:\Users\ypa\Documents\Ascom.Gemini.Telescope\GeminiTelescope\Connection.cs:line 222[0D][0A]?? at ASCOM.GeminiTelescope.GeminiHardwareBase.RetransmitUDP() in C:\Users\ypa\Documents\Ascom.Gemini.Telescope\GeminiTelescope\Connection.cs:line 108[0D][0A]?? at ASCOM.GeminiTelescope.GeminiHardwareBase.getEthernetCommandResult(Int32 timeout) in C:\Users\ypa\Documents\Ascom.Gemini.Telescope\GeminiTelescope\Connection.cs:line 361[0D][0A]?? at ASCOM.GeminiTelescope.GeminiHardwareBase.GetCommandResultEthernet(CommandItem command, Boolean bResyncOnError, Boolean& bReturn) in C:\Users\ypa\Documents\Ascom.Gemini.Telescope\GeminiTelescope\Connection.cs:line 508 (this error is repeated endlessly until the mount is powered down) These same lines appear in each of the ASCOM Gemini log files at the moment when the issue occurs >> I¡¯ve uploaded the ASCOM Gemini log file from last night that shows these comm errors starting at 21:45:43 << I haven¡¯t tried changing the ethernet cable, although if this is a cable issue I would think that the mount should continue to track and the hand controller should be able to communicate with the mount. It seems as the though the controller has entered into a brain-dead state until being power-cycled. System info: HC firmware: v1.64 Mainboard firmware v 6.12 Servos v3.0, v3.0 Connection type: ethernet New battery installed (3.107 Vdc) Losmandy power brick: 15.134 Vdc running windows 11, ASCOM 7 with update 2 ? guidance or suggestions would be greatly appreciated - thank you for your help! ? Mike Hundley ? |
Hi Mike,
?
There have now been a few reports of similar behavior. Unfortunately, the logs don't capture what actually happens at the time of lock-up.
?
A few questions that may or may not be related:
1.? Did you update the bootloader on this Gemini? And?
2. What size and type of SD card is installed in the main unit?
3. Do you have PEC programmed and enabled when this happens?
?
Regards,
?
? ?-Paul
?
On Sun, May 25, 2025 at 04:48 PM, Michael Hundley wrote:
|
Been plagued by this for a while also, have developed some workarounds, and am able to image through the night now.
?
Example scenario:
? ?Gemini-2 unresponsive, then/and
? ?Exposures show bad star trails, then/and related errors: Phd2 complains guide pulses not reaching mount.
? ?System hangs, reboots, and other bad things...
?
? ? ?1. After too many comm errors, the Gemini-2 driver will display a modal dialog under N.I.N.A. (for example) on a mount restart, when the driver is instructed to "set Gemini xxx on connect". The modal dialog can't be seen, and halts all mount communications until responded to, killing the session.
? ? 2. On my system, use of the ASCOM 7 framework seems to slow the comms traffic down sufficiently to cause many more retransmit failures, thus causing #1 above to recur many times in an evening.
?
Fix: 1. Remove modal dialog box from driver behavior, or disable the "Set Gemini [site/time] on connect" for the user session.
? ? ? ?2. Revert to ASCOM 6.6 to lessen repeated failures behavior from slower communications.
?
I found that the comms problems also occurred with USB/Serial ports. New cables, cleaned sockets, etc. not effective.
I am using a MELE3c, which is a relatively slow PC, with Win11 and ASCOM 7, and NINA over RDP, can be causing these issues, especially if generating log files at detailed levels, etc.
?
?
? ??
Here are a few snippets from ASCII.Gemini Telescope.NET
? ??
21:23:00.475 Ethernet received: ? ? ? ?TID:11 [0]: <237:, 18000;18000
21:23:00.508 TransmitUDP packet#, cmd ?TID:11 [0]: 745, <150:r# 21:23:00.508 Ethernet wait for respons TID:11 [0]: <150: 21:23:10.527 ReTransmitUDP packet#, cm TID:11 [0]: 746 21:23:13.566 ReTransmitUDP packet#, cm TID:11 [0]: 747 21:23:15.614 ReTransmitUDP packet#, cm TID:11 [0]: 748 21:23:17.661 ReTransmitUDP packet#, cm TID:11 [0]: 749 21:23:19.704 ReTransmitUDP packet#, cm TID:11 [0]: 750 21:23:19.707 Exception ? ? ? ? ? ? ? ? TID:11 [Error]: System.Exception: Ethernet connection error[0D][0A] ? at ASCOM.GeminiTelescope.GeminiHardwareBase.GetCommandResultEthernet(CommandItem command, Boolean bResyncOnError, Boolean& bReturn) in C:\Users\ypa\Documents\Ascom.Gemini.Telescope\GeminiTelescope\Connection.cs:line 517 21:23:21.796 TransmitUDP packet#, cmd ?TID:11 [0]: 751, 21:23:21.796 Ethernet wait for respons TID:11 [0]: 21:23:21.796 Ethernet Received ? ? ? ? TID:11 [0]: 21:23:21.796 Ethernet received: ? ? ? ?TID:11 [0]: , G ? ? ?
This occurs 3 times, which causes this on the third:
0:49:52.510 Ethernet wait for respons TID:11 [0]: <150:
00:50:02.523 ReTransmitUDP packet#, cm TID:11 [0]: 107943 00:50:04.558 ReTransmitUDP packet#, cm TID:11 [0]: 107944 00:50:06.591 ReTransmitUDP packet#, cm TID:11 [0]: 107945 00:50:08.624 ReTransmitUDP packet#, cm TID:11 [0]: 107946 00:50:10.651 ReTransmitUDP packet#, cm TID:11 [0]: 107947 00:50:10.655 Exception ? ? ? ? ? ? ? ? TID:11 [Error]: System.Exception: Ethernet connection error[0D][0A] ? at ASCOM.GeminiTelescope.GeminiHardwareBase.GetCommandResultEthernet(CommandItem command, Boolean bResyncOnError, Boolean& bReturn) in C:\Users\ypa\Documents\Ascom.Gemini.Telescope\GeminiTelescope\Connection.cs:line 517 00:50:10.656 Too many errors ? ? ? ? ? TID:11 [Error]:? 00:50:10.659 Resetting communications ?TID:11 [0]:? 00:50:12.690 TransmitUDP packet#, cmd ?TID:11 [0]: 107948, ?
?
Hope this helps,
Dale
|
Hi Paul,
?
1. I haven¡¯t applied that recent bootloader update.
2. The SD card is what was supplied by Losmandy circa 2023. Looking at the SD card directory via the Gemini web interface, it looks like there is 4 GB total storage space. The card has a white sticker (¡°L6¡±) which covers most of the details other than ¡°micro SD¡± and ¡°class 4¡±.
3. funny you should ask about PEC as one of the issues I¡¯ve noticed lately is that PEC turns on automatically when the controller is tracking and I cannot turn it off (unchecking the PEC box removes the check mark very briefly then it pops back on; note that am using the latest version of the Gemini firmware that was perhaps should have addressed this issue????). The ¡°enable PEC at start of tracking¡± is definitely un-checked. I fiddled with creating a PEC curve last year using the trial mode of permpro (30 minute limit) and didn¡¯t get very clean unguided data (likely poor seeing + so-so PA + me still working along the Gemini learning curve) and hence when running with PEC on I found guiding not much different from using PPEC in Phd2. Hence, presently I prefer no PEC + phd2¡¯s PPEC set to 239.4 sec period.
?
Mike Hundley? |
Hi Mike,
?
For clearing PEC permanently (or until you want to program it again), you'll need to remove CurrPEC.pec file from the PEC folder on the Gemini SD card. You can do this from the ASCOM driver: ? /g/Gemini-II/message/28718
?
On Sun, May 25, 2025 at 08:02 PM, Michael Hundley wrote:
|
Thanks, Dale. It appears the actual error is Gemini not responding, resulting in a number of attempts to retransmit the request, and then a failure. Why Gemini stops responding is something to be determined. Strange that setting time and location would cause anything like this, especially since that's done at the very beginning of the session and should have no effect on Gemini later on.
?
But I'd be curious to hear if others who are experiencing a similar lock-up have the ASCOM driver set to update those settings on connect?
?
Regards,
?
? -Paul
?
On Sun, May 25, 2025 at 07:30 PM, DaleN wrote:
|
Paul,
?
In my Gemini-2 lock-up case, both "Set Gemini Site on Connect" and "Set Gemini Time on Connect" are unchecked.? Actually I've never checked these settings since I began to use the Gemini Telescope.NET.
Unlike Dale's log, there were a lots of Bad Checksum errors when the lock-up occurred for my Gemini-2s.
As there have been no clear nights for almost three weeks or so, I don't have any chance to test the cross-over cable connection between Gemini-2 and MeLE Quieter 4C so far.
I will report back once I finish the test.
?
Regards,
masami
P.S.
My ASCOM platform is 7.0.2 and Gemini-2 FW version is same as Dale's. |
Hi masami,
?
That's good information - thank you! Bad checksums are nearly always a problem with the network or serial communications between PC and Gemini. The timeout errors that Dale and Mike have been reporting appear without checksum errors, but seem to be the result of an occasional lock-up of Gemini where even the physical HC stops responding. I believe you've seen this behavior before, as well, no?
?
Let us know what you find once you're able to test!
?
Regards,
?
? -Paul
?
On Tue, May 27, 2025 at 08:38 AM, YAMADA masami wrote:
|
I am also having these issues.
?
My system is as follows:
?
G2 L6.12
HC 1.64
Servos 3.0
GeminiTelescope 1.1.28
Ethernet connected to a WAP
Windows 11, ASCOM 7
14.065 vdc/3.045vdc
"Set Gemini Site/Time on connect" are both checked.
?
Lately, it seems like shutting down/restarting GeminiTelescope (ASCOM) has resolved the problem for that night.
?
Art
?
?
? |
Can one of you gentlemen, who experienced the lock-up Gemini problem, try to downgrade to a previous version of Gemini firmware?
?
Using GFU you can select L6.11 or even L6.1 and update your Gemini to that version. I'd like to see if this lock-up is something that's caused by the latest firmware version. I don't think you need to downgrade HC firmware or any other HC files, just flash the main firmware version.
?
Regards,
?
? -Paul |
I will downgrade my gemini's firmware to either 6.11 or 6.1 before my next session. Astrospheric currently is showing plenty of clouds here in the South Carolina low country so my next imaging night may not be until May 31st.
?
question for Paul: is there a way to tell on what date I upgraded to the 6.12 firmware, perhaps in a log file hiding somewhere? While I take lots of notes during imaging sessions, I haven't been in the habit of noting when I made a firmware ?upgrade and am wondering when I made the change to 6.12 and also if I skipped 6.11. Since the beginning of May I've had three problem-free sessions (14th, 15th, 16th) and two problematic ones (5th & 24th) and am fairly (but not 100%) certainly that these are all when using 6.12. I also had four problem-free sessions in April but don't know if these were with 6.12.
?
regards,
Mike Hundley |
Paul, I will downgrade the FW and retry tonight.
?
FYI, what I discovered was that the Gemini-2 lockups (not the causes of the comm failures) were occurring because of the "Set time" and "Set xxx" options on the Gemini-2 driver. By setting any of these options, when the driver detects 3 "Ethernet Communication Errors", it throws the "Too many errors" exception and resets/restarts the Gemini-2 controller. It is this action that then causes the lockup, as it displays a modal dialog box under NINA where it can't be seen. The driver hangs here.
?
My suspicion is that the underlying "Comm Error" is actually a low level timing issue, where some combination of Win11, Mele3C, ASCOM 7, other non-ASCOM drivers are causing the Gemini driver to miss interrupts.?
?
For me, this behavior began when I installed ASCOM 7. I have since downgraded to ASCOM 6.6, where the errors still occur, but not as many in a row, such that the Gemini restart is never issued.?
?
Also, in my case, I have turned off the "Set" options in the driver, eliminating the modal dialog blockage.
?
I am inspecting my log files over the last month or so to determine if there were issues before ASCOM 7.?
?
Thanks!
Dale
?
? |
I can downgrade as well. No clear skies in sight for a while, though. I suppose I could do some ¡°blind¡± testing. But, the PHD2 pulse related ?problem will obviously not be tested.
toggle quoted message
Show quoted text
Art Trail attrail1@... On Tuesday, May 27, 2025, 3:56 PM, DaleN via groups.io <aleday.elsonnay@...> wrote:
|
I can answer my own question regarding a log file that shows what Gemini firmware version was in use for a given session.?
?
The POINTING.DAT file in the SD card "Logs" directory includes the firmware version in use + the UTC date every time the Gemini is powered up. So, since installing v. 6.12 on March 27th I've had seven trouble free sessions and two nights when the Gemini locked-up.
?
I'll switch back to 6.11 and report back when I get some clear nights ... |
Looking forward to seeing your results, folks. I don't recall hearing of this problem being reported with earlier versions of the L6 firmware, so trying to narrow down as to what may have caused it. If the problem goes away with firmware downgrade, at least Rene will have some indication as to where to start looking. |
Hi Paul,
?
I reported this lock up issue earlier this year, and the lock up happened from v6.10. I upgraded to v6.12 and still experienced the lock up. I tried changing the power supply and it still locked up. I replaced the coin cell battery and did SRAM reset 4 days ago. On the second night, I loaded new PEC curve. Three nights were okay, but it locked up again last night. I uploaded my log in the files section.
?
Thank you,
Seokman |
Hi Seokman,
?
Did you try to downgrade to an older version? 6.02 would be sufficient if you want to use PEC (6.0 didn't perform well with PEC, so if you try, I'd turn it off).
The other part that may be worth trying is to turn off PEC with newer firmware versions to see if that's related. Many of the changes starting with 6.02 and later included some new PEC features, so I'm curious to see if that's at all related to the issue. ?
Regards,
?
? ?-Paul
?
On Wed, May 28, 2025 at 04:16 PM, Seokman Han wrote:
|
Hi Paul,
?
Enclosed Gemini Logs.
?
In no cases over the last 2 sessions (since unchecking the Gemini-2 "set" options) have I had a complete Gemini-2 freeze up/lockout.
In my case, that problem seems solved. (@Art: this may be your issue as well)
?
"Too many errors" "reset communications" occurred 23x over 2 hours.?
May 24 session had 2x over 6 hours (FW 6.12 ASCOM 6.6.2, Driver 1.0.28)
?
FW 6.02 seems to encounter many more communication errors. The telescope log shows many more ReTransmits vs May24 and a few bad checksums, but I suspect some of this is due to using log level (n-1) instead of earlier "some detail".
?
There were several points in time where I noticed PHD2 was not showing guiding pulses for 10,20,30 seconds. I did not record the event times where this occurred.
?
Actual imaging/guiding began about 22:00, Polar Alignment and PHD2 calibration was prior.
?
What is a typical Gemini-2 UDP error rate? Given the general unreliability of UDP, I can't assess the error rate I see here.?
?
?
Downgraded FW to 6.02 (from 6.12).
Driver 1.0.28
NINA 3.2.
ASCOM 6.6.2 downgraded from ASCOM 7.0.2
Ethernet via GL.iNet router
?
Thanks!
Dale Nelson
?
?
? |