Hi all
Hoping for some guidance on this issue that has been plaguing me for months. Thanks in advance!
As topic name, almost every trading day, the gateway will disconnect from the IBKR servers 3-4 times around 2:05pm every day. Since this first started occurring I have moved houses and put my software on AWS to no avail.
For example today, whilst running on AWS Sydney, while it was able to reconnect each time, it disconnected 3 times and gave me error 2105 at (AEST/UTC+10):
14:03:33
14:04:17
14:05:23
and the respective logs:
2022-06-29 14:03:31.314 [BB] INFO? [JTS-EServerSocketNotifier-104] - [1:157:157:1:0:90:-1:INFO] Historical data update sent [requestId: 3].
2022-06-29 14:03:33.122 [BB] INFO? [JTS-ushmdsListenerS7-109] - Socket for connection ndc1.ibllc.com:4000 was closed by peer.
2022-06-29 14:03:33.125 [BB] INFO? [JTS-ushmdsListenerS7-109] - Disconnecting ndc1.ibllc.com:4000 [disconnectDetails=DisconnectDetails[sessionID=7,endPoint=ndc1.ibllc.com:4000,reason=DISCONNECT_ON_BROKEN_SOCKET,cause=null,systemMessage=null,keepSocketOpen=false]]...
2022-06-29 14:03:33.125 [BB] INFO? [JTS-ushmdsListenerS7-109] - Socket closed.
2022-06-29 14:03:33.125 [BB] INFO? [JTS-ushmdsListenerS7-109] - Interrupting dispatcher [sessionID=7]...
2022-06-29 14:03:33.126 [BB] INFO? [JTS-ushmdsDispatcherS7-110S7-111] - Dispatcher thread terminating [sessionID=7,interrupted=true]...
2022-06-29 14:03:33.130 [BB] INFO? [JTS-ushmdsListenerS7-109] - Interrupting listener [sessionID=7,disconnectSocket=true]...
2022-06-29 14:03:33.130 [BB] INFO? [JTS-ushmdsListenerS7-109] - AuthTimeoutMonitor-ushmds: deactivate
2022-06-29 14:03:33.130 [BB] INFO? [JTS-ushmdsListenerS7-109] - Listener thread terminating [sessionID=7] [seen=62643,totalShifted=62643,moreAvailable=0]
2022-06-29 14:03:33.131 [BB] INFO? [JTS-DisconnectedS7-147] - Farm ushmds/NATIVE: Lost active connection with disconnect status DISCONNECT_ON_BROKEN_SOCKET
2022-06-29 14:03:33.132 [BB] INFO? [JTS-DisconnectedS7-147] - Farm ushmds/NATIVE: Resetting
2022-06-29 14:03:33.132 [BB] INFO? [JTS-DisconnectedS7-147] - CONN CALC: Last connection attempt time is cleared!
2022-06-29 14:03:33.132 [BB] INFO? [JTS-DisconnectedS7-147] - CONN CALC: Last connection set as now:14:03:33:132
2022-06-29 14:03:33.132 [BB] INFO? [JTS-DisconnectedS7-147] - HTBP Disconnected from live peer!::0
2022-06-29 14:03:33.132 [BB] INFO? [JTS-DisconnectedS7-147] - CONN CALC: Last disconnect time set as now:14:03:33:132
2022-06-29 14:03:33.132 [BB] INFO? [JTS-ushmdsServicePingS7-114] - Terminating ping thread
2022-06-29 14:03:33.137 [BB] INFO? [JTS-DisconnectedS7-147] - Starting fuse [name=ushmds-nativepause,timeout=0]...
2022-06-29 14:03:33.141 [BB] INFO? [JTS-Fuse-ushmds-nativepause-148] - Farm ushmds/DISCONNECTED: Connecting natively (ushmds)...
2022-06-29 14:03:33.143 [BB] INFO? [JTS-Fuse-ushmds-nativepause-148] - Reconnection time cleared: argClearBackoff=false base=0 range=5000 next=0 disconnected=0
2022-06-29 14:03:33.148 [BB] INFO? [JTS-AsyncNonLocked-44] - Keep-alive scheduled for:ushmds
2022-06-29 14:03:33.149 [BB] INFO? [JTS-AsyncNonLocked-44] - [1:157:157:1:0:4:2:DET] Sending error.
2022-06-29 14:03:33.149 [BB] INFO? [JTS-AsyncNonLocked-44] - [1:157:157:1:0:4:2:DET] [4;2;-1;2105;HMDS data farm connection is broken:ushmds]
2022-06-29 14:03:33.149 [BB] INFO? [JTS-AsyncNonLocked-44] - [1:157:157:1:0:4:2:DET] Error sent.
2022-06-29 14:03:33.149 [BB] INFO? [JTS-AsyncNonLocked-44] - cdebug: ROUTING | connTracker:ushmds@5cd4e9d | Disconnected | 1656475412135 | true
2022-06-29 14:03:33.146 [BB] INFO? [JTS-Fuse-ushmds-nativepause-148] - AsyncDisconnectNotifiers: 1
2022-06-29 14:03:33.157 [BB] INFO? [JTS-Fuse-ushmds-nativepause-148] - Connecting ndc1.ibllc.com:4000 timeout=0...
2022-06-29 14:03:33.159 [BB] INFO? [JTS-Fuse-ushmds-nativepause-148] - CONN CALC: Last connection set as now:14:03:33:159
--------------------------------------------------------------------------------------------------------------------
2022-06-29 14:04:16.273 [BB] INFO? [JTS-EServerSocketNotifier-104] - [1:157:157:1:0:90:-1:INFO] Historical data update sent [requestId: 3].
2022-06-29 14:04:16.518 [BB] INFO? [JTS-CCPDispatcherS3-49] - Setting time offset to -517 diff -252
2022-06-29 14:04:16.518 [BB] INFO? [JTS-CCPDispatcherS3-49] - CCP reported time: 2022-06-29 04:04:16.000 (GMT) [1656475456000], local time: 2022-06-29 04:04:16.518 (GMT), local time: 2022-06-29 14:04:16.518 (AEST), difference in seconds: -0.518, handling delta:1 [Msg1]
2022-06-29 14:04:16.537 [BB] INFO? [JTS-ushmdsNativeUpgrade-162] - Farm ushmds/CCP: Trying native connection, attempt
2022-06-29 14:04:16.539 [BB] INFO? [JTS-ushmdsNativeUpgrade-162] - Trying connecting ndc1.ibllc.com:4000 timeout=0...
2022-06-29 14:04:16.943 [BB] INFO? [JTS-ushmdsNativeUpgrade-162] - Farm ushmds/CCP: Trying native connection
2022-06-29 14:04:16.943 [BB] INFO? [JTS-ushmdsNativeUpgrade-162] - AsyncDisconnectNotifiers: 1
2022-06-29 14:04:16.943 [BB] INFO? [JTS-ushmdsNativeUpgrade-162] - Connecting ndc1.ibllc.com:4000 timeout=0...
2022-06-29 14:04:17.158 [BB] INFO? [JTS-ushmdsNativeUpgrade-162] - Connected to ndc1.ibllc.com:4000 on local port 0 socket local addrs:port /172.31.39.211:55610
2022-06-29 14:04:17.158 [BB] INFO? [JTS-ushmdsNativeUpgrade-162] - Farm ushmds/CCP: Connected, sending secure connect and/or auth
2022-06-29 14:04:17.159 [BB] INFO? [JTS-ushmdsNativeUpgrade-162] - InnerConnection.onConnected() [sessionID=14,secureConnectEnabled=true,logonReplySigned=true].
2022-06-29 14:04:17.222 [BB] INFO? [JTS-ushmdsNativeUpgrade-162] - Not exhausted because ndc1.ibllc.com was connected
2022-06-29 14:04:17.222 [BB] INFO? [JTS-ushmdsDispatcherS14-164S14-165] - Starting dispatcher thread [sessionID=14]...
2022-06-29 14:04:17.222 [BB] INFO? [JTS-ushmdsListenerS14-163] - Starting listener thread [sessionID=14]...
2022-06-29 14:04:17.223 [BB] INFO? [JTS-ushmdsListenerS14-163] - Initial listener buffer size is 131072.
2022-06-29 14:04:17.223 [BB] INFO? [JTS-ushmdsListenerS14-163] - Capping listener buffer size at 65536.
2022-06-29 14:04:17.449 [BB] INFO? [JTS-ushmdsDispatcherS14-164S14-165] - cert #0 subject:CN=tws.ibllc.com,OU=tws.ibllc.com,O=IBLLC,C=US??? issuer:CN=prod.ckg.ibllc.com,O=Interactive Brokers LLC,L=Greenwich,ST=Connecticut,C=US??? notBefore:20220628-13:36:17 Europe/London (1656419777000)??? notAfter:20220630-21:36:17 Europe/London (1656621377000)
2022-06-29 14:04:17.449 [BB] INFO? [JTS-ushmdsDispatcherS14-164S14-165] - cert #1 subject:CN=prod.ckg.ibllc.com,O=Interactive Brokers LLC,L=Greenwich,ST=Connecticut,C=US??? issuer:CN=DigiCert SHA2 High Assurance Server CA,OU=www.digicert.com,O=DigiCert Inc,C=US??? notBefore:20200812-01:00:00 Europe/London (1597190400000)??? notAfter:20220817-13:00:00 Europe/London (1660737600000)
2022-06-29 14:04:17.450 [BB] INFO? [JTS-ushmdsDispatcherS14-164S14-165] - cert #2 subject:CN=DigiCert SHA2 High Assurance Server CA,OU=www.digicert.com,O=DigiCert Inc,C=US??? issuer:CN=DigiCert High Assurance EV Root CA,OU=www.digicert.com,O=DigiCert Inc,C=US??? notBefore:20131022-13:00:00 Europe/London (1382443200000)??? notAfter:20281022-13:00:00 Europe/London (1855828800000)
2022-06-29 14:04:17.455 [BB] INFO? [JTS-ushmdsDispatcherS14-164S14-165] - Current time is: 20220629-05:04:17 Europe/London
2022-06-29 14:04:17.459 [BB] INFO? [JTS-ushmdsDispatcherS14-164S14-165] - CipherContext initialization successed.
2022-06-29 14:04:17.459 [BB] INFO? [JTS-ushmdsDispatcherS14-164S14-165] - Cipher context created: CipherContext[id=19,encrypt=true].
2022-06-29 14:04:17.459 [BB] INFO? [JTS-ushmdsDispatcherS14-164S14-165] - Cipher context created: CipherContext[id=20,encrypt=false].
2022-06-29 14:04:17.459 [BB] INFO? [JTS-ushmdsDispatcherS14-164S14-165] - set nsMsgVersion / cryptors: farm=HISTORICAL_DATA-ushmds; nsMsgVersion=44; encryptor=true; decryptor=true
2022-06-29 14:04:17.459 [BB] INFO? [JTS-ushmdsDispatcherS14-164S14-165] - AuthTimeoutMonitor-ushmds: deactivate
2022-06-29 14:04:17.459 [BB] INFO? [JTS-ushmdsDispatcherS14-164S14-165] - AuthTimeoutMonitor-ushmds: timeout=20000
2022-06-29 14:04:17.459 [BB] INFO? [JTS-ushmdsDispatcherS14-164S14-165] - AuthTimeoutMonitor-ushmds: activate
2022-06-29 14:04:17.544 [BB] INFO? [JTS-ushmdsDispatcherS14-164S14-165] - Start ping with 5000 for JTS-ushmdsServicePreAuthPingS14-167
2022-06-29 14:04:17.544 [BB] INFO? [JTS-AuthTimeoutMonitorThread-ushmds-166] - Starting auth timeout monitor thread for ushmds
2022-06-29 14:04:17.562 [BB] INFO? [JTS-ushmdsServicePreAuthPingS14-167] - Starting ping thread
------------------------------------------------------------------------
2022-06-29 14:05:23.418 [BB] INFO? [JTS-ushmdsDispatcherS18-176S18-177] - Received MD auth msg classid = 1
2022-06-29 14:05:23.419 [BB] INFO? [JTS-ushmdsDispatcherS18-176S18-177] - Receiver: 772
2022-06-29 14:05:23.419 [BB] INFO? [JTS-ushmdsDispatcherS18-176S18-177] - Received Result
2022-06-29 14:05:23.419 [BB] INFO? [JTS-ushmdsDispatcherS18-176S18-177] - Passed session token authentication.
2022-06-29 14:05:23.662 [BB] INFO? [JTS-ushmdsDispatcherS18-176S18-177] - Logon reply received [signed=true].
2022-06-29 14:05:23.662 [BB] INFO? [JTS-ushmdsDispatcherS18-176S18-177] - Reconnection time cleared: argClearBackoff=true base=0 range=5000 next=0 disconnected=0
2022-06-29 14:05:23.662 [BB] INFO? [JTS-ushmdsDispatcherS18-176S18-177] - AuthTimeoutMonitor-ushmds: deactivate
2022-06-29 14:05:23.663 [BB] INFO? [JTS-ushmdsDispatcherS18-176S18-177] - Farm ushmds/CCP: Upgrading to native connection
2022-06-29 14:05:23.663 [BB] INFO? [JTS-ushmdsDispatcherS18-176S18-177] - Disconnecting hdc1.ibllc.com:4000 [disconnectDetails=DisconnectDetails[sessionID=12,endPoint=hdc1.ibllc.com:4000,reason=DISCONNECT_NATIVE_CONNECTION_RESTORED,cause=null,systemMessage=null,keepSocketOpen=false]]...
2022-06-29 14:05:23.663 [BB] INFO? [JTS-ushmdsListenerS12-156] - Socket or stream for connection hdc1.ibllc.com:4000 was closed by another thread.
2022-06-29 14:05:23.663 [BB] INFO? [JTS-ushmdsDispatcherS18-176S18-177] - Socket closed.
2022-06-29 14:05:23.663 [BB] INFO? [JTS-ushmdsDispatcherS18-176S18-177] - Interrupting dispatcher [sessionID=12]...
2022-06-29 14:05:23.663 [BB] INFO? [JTS-ushmdsDispatcherS18-176S18-177] - Interrupting listener [sessionID=12,disconnectSocket=true]...
2022-06-29 14:05:23.663 [BB] INFO? [JTS-ushmdsDispatcherS18-176S18-177] - AuthTimeoutMonitor-ushmds: deactivate
2022-06-29 14:05:23.664 [BB] INFO? [JTS-ushmdsListenerS12-156] - Listener thread terminating [sessionID=12] [seen=6881,totalShifted=6881,moreAvailable=0]
2022-06-29 14:05:23.664 [BB] INFO? [JTS-ushmdsDispatcherS18-176S18-177] - Start ping with 60000 for JTS-ushmdsServicePingS18-181
2022-06-29 14:05:23.665 [BB] INFO? [JTS-DisconnectedS12-180] - Farm ushmds/NATIVE: Old CCP connection hdc1.ibllc.com:4000 UNCONNECTED was disconnected after native upgrade
2022-06-29 14:05:23.665 [BB] INFO? [JTS-AsyncNonLocked-44] - Keep-alive scheduled for:ushmds
2022-06-29 14:05:23.665 [BB] INFO? [JTS-AsyncNonLocked-44] - [1:157:157:1:0:4:2:DET] Sending error.
2022-06-29 14:05:23.665 [BB] INFO? [JTS-AsyncNonLocked-44] - [1:157:157:1:0:4:2:DET] [4;2;-1;2105;HMDS data farm connection is broken:ushmds]
2022-06-29 14:05:23.665 [BB] INFO? [JTS-AsyncNonLocked-44] - [1:157:157:1:0:4:2:DET] Error sent.
2022-06-29 14:05:23.666 [BB] INFO? [JTS-AsyncNonLocked-44] - cdebug: ROUTING | connTracker:ushmds@5cd4e9d | Disconnected | 1656475522897 | true
2022-06-29 14:05:23.666 [BB] INFO? [JTS-AsyncNonLocked-44] - [1:157:157:1:0:4:2:DET] Sending error.
2022-06-29 14:05:23.666 [BB] INFO? [JTS-AsyncNonLocked-44] - [1:157:157:1:0:4:2:DET] [4;2;-1;2106;HMDS data farm connection is OK:ushmds]
2022-06-29 14:05:23.666 [BB] INFO? [JTS-AsyncNonLocked-44] - [1:157:157:1:0:4:2:DET] Error sent.
2022-06-29 14:05:23.666 [BB] INFO? [JTS-AsyncNonLocked-44] - Keep-alive applied to:ushmds
2022-06-29 14:05:23.666 [BB] INFO? [JTS-AsyncNonLocked-44] - cdebug: ROUTING | connTracker:ushmds@5cd4e9d | Connected | 1656475522897
2022-06-29 14:05:23.666 [BB] INFO? [JTS-ushmdsNativeUpgrade-162] - Farm ushmds/NATIVE: Terminating native background upgrade
2022-06-29 14:05:23.666 [BB] INFO? [JTS-ushmdsServicePreAuthPingS18-179] - Terminating ping thread
2022-06-29 14:05:23.666 [BB] INFO? [JTS-ushmdsDispatcherS12-157S12-158] - Dispatcher thread terminating [sessionID=12,interrupted=true]...
2022-06-29 14:05:23.667 [BB] INFO? [JTS-ushmdsServicePingS12-161] - Terminating ping thread
2022-06-29 14:05:23.667 [BB] INFO? [JTS-AuthTimeoutMonitorThread-ushmds-178] - Terminating auth timeout monitor thread for ushmds
2022-06-29 14:05:23.673 [BB] INFO? [JTS-ushmdsServicePingS18-181] - Starting ping thread
