Another option is to request the next ID and wrap that into a try....except clause

Thank you for your response. I have used the error() method, specifically the error codes, to monitor the connection status between my C# App and TWS. In the case where the connection is established between my App and TWS, but the NextValidID callback is not called, I am unsure whether I should proceed with calling clientSocket.reqIds(-1) or attempt to reconnect? Note that my App connects to IB without any issues most of the time, there are occasional problems with the connection, such as the NextValidID callback not being triggered.

Note: I am using a paper trading account, and I run My App and IB as Administrator

A couple scenarios come to mind when you get no callback during a client connection to TWS/API. Please make sure you implement/override all three? methods so that you get all possible error messages.

The first scenario is triggered in case your client sends API requests too early after the call to connect. Take a look at the chapter in the where it says:

Important: The callback is commonly used to indicate that the connection is completed and other messages can be sent from the API client to TWS. There is the possibility that function calls made prior to this time could be dropped by TWS.

• From a quick glance at the C# implementation it looks like you would get an callback via the function with signature "void error(Exception e)"
• The Java implementation, for example, will send a (misleading) error 502 via the "void error(int id, int errorCode, ...)" callback

So make sure you wait (under no circumstance sleep !) until is called before you send any API requests.

There is a second scenario where TWS/IBGW generally would simply close the connection instead of sending an error code or . That is when your client attempts to connect with the same clientId another client already uses. You would get the same error callback as in the first scenario (different ones in C# and Java).

This scenario? can happen even if the client that originally used that clientId has disconnected or is terminated. Depending on your network setup, the first client's socket might be "stuck" or lingering and TWS/IBGW will refuse connection requests with that clientId until the operating system completely closes the first client's socket. In TWS, you can click the green DATA button in the upper right corner and it shows you a list of all currently used clientIds.

In both cases, TWS/IBG has given up on the connection, the socket is closed, and calling clientSocket.reqIds(-1) will have no effect (and should actually fail). You would have to reconnect at that point.

As a matter of practice, our framework blocks the client during connect until both and callback have arrived. The arrival order of these callbacks is random, so that often comes well after and, at least when talking to TWS, after a series of message code 21xx informational error messages about the connection status of the various data farms. This approach has led to rock stable connection logic forus for? years.

Hope this helps,

闯ü谤驳别苍

Thank you for your response. I have used the error() method, specifically the error codes, to monitor the connection status between my C# App and TWS. In the case where the connection is established between my App and TWS, but the NextValidID callback is not called, I am unsure whether I should proceed with calling clientSocket.reqIds(-1) or attempt to reconnect? Note that my App connects to IB without any issues most of the time, there are occasional problems with the connection, such as the NextValidID callback not being triggered.

Note: I am using a paper trading account, and I run My App and IB as Administrator

• the client immediately appears in the TWS/IBG API client list
• Market data connection callbacks (2104, 2106, 2158) are shown in the IBG API log (but not received by the client)
• the IBG API logs show the version handshake:
  
• tracing through connect() shows conn_time and server_version correctly obtained, and IBApi.EClient.connState set to 2 (Connected)
• API logs show no other errors, and the client receives no errors
• but....no nextValidId callback is recieved, at least within the 120 seconds I wait for it

import threading
from ibapi.client import EClient
from ibapi.wrapper import EWrapper

class IB_API_Client(EWrapper, EClient):
??? def __init__(self, port, client_id):
??????? EClient.__init__(self, self)
??????? EWrapper.__init__(self)
??????? self.port = port
??????? self.client_id = client_id
??????? self.event_nextValidId_callback = threading.Event()

??? def nextValidId(self, request_ID):
??????? self.event_nextValidId_callback.set()
??????? super().nextValidId(request_ID)

??? def open_connection(self):
??????? self.connection_error = False
??????? self.event_nextValidId_callback.clear()
??????? self.connect("127.0.0.1", self.port, self.client_id)
? ? ? ??# wait for nextValidId

? ? ? ??if not self.event_nextValidId_callback.wait(120) or self.connection_error:
? ? ? ? ? ? raise ConnectionError((1, f'Could not connect on IB account port {self.port}'))
??????? else:
??????????? print('successful connection')

??? def error(self, reqId, errorCode, errorString, advancedOrderRejectJson=''):
??????? if errorCode in [502, 504]:
??????????? self.connection_error = True
??????? else:
??????????? print(f'Error {errorCode} - {errorString}')

??? def winError(self, text, lastError):
??????? raise WindowsError(text)


if __name__ == '__main__':
??? print('Seeking connection')
??? api = IB_API_Client(4002, 0)
??? api.open_connection()

Unfortunately I am not Python literate enough to give you concrete help, Dave. But on the surface, your approach looks fine.

I am wondering, though, whether there is some adverse interaction between the call to? self.event_nextValidId_callback.wait() and the ability of your client to process incoming TWS API messages. You say "Market data connection callbacks (2104, 2106, 2158) are shown in the IBG API log (but not received by the client)" which makes me wonder whether messages from TWS/IBGW get processed.

Not sure whether that happens behind the scene, but wouldn't you have to start the message reader in a separate thread as it is described in the chapter?

闯ü谤驳别苍

On Wed, Jun 14, 2023 at 04:42 PM, 闯ü谤驳别苍 Reinold wrote:

you have to start the message reader in a separate thread

That's already happening via the base classes. Unfortunately, by calling the main thread itself (which should receive the callback is blocked) and then immediately throws a ConnectionError as soon as possible.

Dave, it's not clear what you're trying to accomplish by using the threading event. Threading events are a mechanism for IPC (inter-process communication) and meant to communicate between threads, but you don't start any more threads of your own.

Why not try getting rid of that stuff and just call app.run(), or in your case api.run(), as your final step in main like the example provided w/ the API?

Would Event objects provided bz the asyncio package work better here (IBKR suggests asyncio and ib_insync is built upon it) ?

• The socket connection with TWS/IBGW is established
• The protocol is started (client)

• And TWS/IBGW has sent the NextValidID message

So the connect() call needs to create some kind of event object that can be used to block the main thread until the NextValidID can release the wait block.

As a matter of practice, our (Java) based connect logic waits for the nextValidId() and managedAccounts() callbacks, since our clients cannot do any meaningful work until both have arrived.

闯ü谤驳别苍

On Wed, Jun 14, 2023 at 06:12 PM, 闯ü谤驳别苍 Reinold wrote:

Would Event objects provided bz the asyncio package work better here (IBKR suggests asyncio and ib_insync is built upon it) ?

Hard to say since I think it depends on the overall context and mostly on what the program is meant to accomplish. For example, is it meant to be long lived or just do some processing and exit? What are the real-time requirements?

I could also envision simply setting a member to None and changing it to the next valid id once it arrives... then checking that member in busy loop; assuming concurrency isn't of any interest at all. Are we trying to conserve electricity? Insure simplicity and readability?

Not sure there are simple answers in this respect.

On Wed, Jun 14, 2023 at 06:12 PM, 闯ü谤驳别苍 Reinold wrote:

So the connect() call needs to create some kind of event object that can be used to block the main thread until the NextValidID can release the wait block.

If you mean between the client and wrapper then I'd say yes.

On Wed, Jun 14, 2023 at 07:24 PM, buddy wrote:

between the client and wrapper then I'd say yes

Sorry, this should be between the client and reader. But, the set and wait are called from the same thread of Dave's example and so all that's accomplished is deadlock.

• The typical TWS API Python client is a two-thread solution.
• EReader runs in its own thread, reads TWS/IBGW messages from the network socket connection, puts them into msg_queue, but does not perform any callbacks that relate TWS messages.
• EClient and EWrapper run in the second thread. That thread not only processes your application logic but must also read messages from msg_queue and process the API callbacks.

• When your code calls wait(120) in open_connection, all API callback processing stops, you don't get any of the error messages, nextValidId is never called, and your program is in dead lock

Your approach would work just fine in a three-thread approach where EWrapper and EClient run in two separate threads (plus a third thread for EReader). That is the typical TWS API Java setup and has the advantage that callback processing continues, even if the application thread is very busy or preoccupied with a long blocking operation (say spends time reading or writing a file, or god-forbid, does a sleep()).

• If you want to stay with the two-thread approach, take a look at what the IBKR Testbed/Program.py does. You'd basically define a "start" function as an entry point for your application code and call it at the end of the nextValidId function. No further synchronization is required since there is really only one thing going on at a time.

• If you want to go with the three-thread approach, just start self.run() in a separate thread (see below). The consequence is, however, that there is more parallelism in your application (callbacks can now happen all the time) and you may need more synchronization amount the pieces.

I simply added this to your code and it worked right away:

开云体育

On Wed, Jun 14, 2023 at 11:39 PM, David Walker wrote:

Here’s the full updated code which works fine:

I couldn't get your new version to work until I moved the reader_thread.start() call to after the self.connect("127.0.0.1", self.port, self.client_id) call; like 闯ü谤驳别苍's version.

But yeah... I'm glad the problem is resolved.

On Wed, Jun 14, 2023 at 11:00 PM, 闯ü谤驳别苍 Reinold wrote:

Hope this helps. I learned something new.

Me too... that was fun.