Hi Brian,

It is secure on localhost. Using such self-signed certs are at least as secure as the CA signed ones, since it uses the same encryption technology. The self-signed certs are just not publicly trusted by EVERYONE during a??You may add your to the browser's root certificate store, if that red sign bothers you. Maybe you could use rather tool like mkcert to create one and you may use it with the Client gateway app. Look for the jks file in app and replace it with one created by mkcert.

We use such self-signed cert mechanism for our app (quantdroid.com) in the localhost communication. Even the communication is only on localhost, and such self-signed certs are mostly used nowadays, because there are browsers like Chrome (maybe chromium based, so the Edge) that by default enforces https communication (and CORS mechanism) and it would be an extra effort for the users (~mostly developers who uses them) to start the browser in insecure mode or hack it with settings.

hope it helps,
David

Yeah, you are right, apologies for pushing further confusion about this topic, I should have been more careful commenting on a feature that I only played around with too little to fully understand.


--
Best,
DS

I'm making the transition from TWS API to Client Portal API. When I login to the Edge browser says Client Portal API is not secure. Why? Does this even matter? If it does matter, how can I fix this?

Other details: OpenJDK 11 from Microsoft (not Oracle). Paper trading account, for the time being (while the new app is developed). Windows 11.

That shouldn't be too hard. Something like this:

• Pick a good time for TWS/IBKR restart based upon the trading schedule for the instruments you are working with. For example some time between 16:00 and 17:00 US/Central for Chicago traded futures (say 16:30) or after 20:00 US/Eastern for NYC traded stocks (say 20:30). Keep an eye on the IBKR daily reset/maintenance window for your region and stay away from that.
• Start your client a little before trading starts for the instruments you are interested in.
• At client start, make calls to for all instruments your client works with. That way, you get fresh Contract objects that you can use for all requests that take a Contract parameter and you know the actual trade schedule for that day (and the next few days). That takes care of any last minute exchange hour changes or holiday hours.
• Now you know the time when trading ends and you can stop your client shortly after hat time (but before your scheduled TWS/IBGW restart). Your client can save enough state so that it can start again before the next session.
• Any error 502 received before markets close is unscheduled should probably trigger some kind of alert and possible needs some kind of manual interaction. There are ways to automate TWS/IBGW and client restart (see IBC for example) should this really happen, but you may need to provide at least 2FA login authorization (that can be with IBKR Key on a mobile device).

闯ü谤驳别苍

Hey there Jurgen, thanks a lot for your input, it has really helped me see this problem in a different way.?

What's the best way I could approach a scheduled full disconnect right before TWS restarts and the make a fresh new connection after TWS has restarted ?
I imagine this is what most people do, instead of having to deal with a reconnection because of all the issues that arise from it.

Thanks Jurgen!

Thanks Gordon!

I will take @btw advice as you say.?

Your opinion is exactly why I am in this problem:

?BTW there are no reason your connection broke unless TWS restart which is a different issue.

The fact that I need my program to go through the required TWS restart is what's got me struggling with this issue, from what you and Jurgen told me I think?
the best solution is to have my program disconnect right before TWS restarts and then proceeds to make a fresh new connection after TWS has restarted.?
Any advice on what the best approach on this would be ??

I have followed this post for a while now. Gordon and btw provided a lot of great and relevant advice already, but what you (makeitrain512) are trying to do is pretty advanced. I am wondering whether there are simpler ways to achieve your goal.

You will find many more corner cases that you have not even through about before your reconnect logic will work flawlessly. What if 502 was caused by the daily TWS/IBGW restart? So there may be quite some time before TWS/IBGW can even accept a new socket connection, especially if you run the "latest" TWS/IBGW version that occasionally updates itself during restarts. Or what if TWS/IBGW do not restart properly so a reconnect will not be possible until you manually intervene?

And when you are done with reconnects working properly, you have to reestablish all data feeds and subscriptions your client had active before the 502 disconnect. Depending on how you have structured your code, that might mean that you even have to use the exact same requestIds the client had used originally so that TWS/IBGW responses through the new connection get handled correctly by your client. So you will need a solid and very layered design such as an API Controller ontop of an API Adapter. The adapter would handle all the low-level stuff such as the network socket, the reader thread, and reconnects while the controller would hide all API details from the client (such as requestIds) know about all active subscriptions, and would transparently reestablish the feeds once the reconnect was successful.

And you might have to inform your client or the trading logic about the fact that a chunk of market data was missing during the time it took to reconnect or that possibly some orders got filled or cancelled while that happened. That can all be done, but there will still be a lot of work ahead of you.

@btw made a great point that you might want to consider. Other than the daily TWS/IBGW restarts (that you can completely control when they happen), TWS/IBGW never really disconnect your client as long as your compute platform is stable and your network connection solid. Maybe an approach like this is much easier and meets you requirements as well:

• you configure TWS/IBGW such that they restart shortly after the market for the instruments you are trading closes (regular or extended hours). Just make sure that you stay away from the daily IBKR maintenance and reset window.
• client connects to TWS/IBGW just before the market (or pre-market) opens and stays connected throughout the trading session
• after the end of the session, client either exits well before TWS/IBGW restart or takes the 502 disconnect caused by the TWS/IBGW restart as a sign that it is time to wrap up
• if your client experiences an 502 during the trading session (extremely unlikely), send an alert and handle the case manually.

You could even design your client such that it always keeps enough state on permanent storage that it can restart after a 502 disconnect and pickup right where it left off. All of these will probably take a fraction of the effort than getting the transparent reconnect work flawlessly.

Just a thought,

闯ü谤驳别苍

On Tue, Nov 21, 2023 at 04:14 PM, Gordon Eldest wrote:

@btw give the best path:
you should start from a working example and increase complexity gradually. Splitting issues to do small studies set, one topic at a time,? then assemble what mastered from each study, minded with the principle "divide an conquer".

Opinion:
- your reconnect() method is either too complex or not enough, so make it simpler or remove the concept of reconnect().
If your connection is broken prematurely you cannot even assume that received messages are consistent, better throw away all of them.? BTW there are no reason your connection broke unless TWS restart which is a different issue.
- what you want to do need that you study the code in and around EReader, and see how to .join() it to be sure you never have more than 1 thread running it at same time.
disconnect() should be enough to exit the EReader thread by exiting the while loop, as long as it's not the case, you probably have a racing issue. and/or need to study isConnected()

Not surprising, hence the recommendation from @Hilmar,

We don't know your testing condition and benchmark. Keep in mind that even a few instruments are traded only once a day. (or even not at all!), and also that the time to assemble and process your order is something to factor in too.
So 4 aggregations/second won't change that much. In my limited experience there generally are only about 1000 US Stocks really can go beyond 4 tick/sec
Try APPL on friday at 15:45, or on Forex pair like USD/EUR 10 seconds after a speech from Powell or publication of CPI that didn't follow predicted values.

reqMktData with 4 aggregations second (stocks) or faster on other classes had most probably be designed thinking that it's enough to take a decent decisions and keep IBKR IO pace under control, and it most probably do both efficiently.
Rogers

I did a test by running my python program with TWS offline and the results are as follows:?

1 - From line 1 to 89 TWS was offline so it was not able to connect with anything.?
2- Proceed to login into TWS
3 - From line 90 to 98 the program is able to connect to TWS and the Ereader thread is started
4- On line 109 I get an error saying the socket is either closed or broken
5 - at line 113 the Ereader thread is finished

and this process goes on and on forever, so the readers thread is starting and finishing.?

I have not been able to figure out what the number? after setConnState 1810740610832 is, everytime there is a new connection this number is different, I have a theory that if im able to refresh this number after a new connection attempt I will be able to succeed in reconnecting.

Thanks Gordon and bespalex.?

The value of profit_order_ID was assigned outside of the wrapper but after following bespalex's suggestion to print its value within orderStatus, I found?that the assignment was not accessible from within the wrapper, probably because of some Pythonic error I am making.? So I ended up creating a workaround with openOrders instead and I'm able to get the order info I need...?

Chris??

I would blend your code with the working code.? Add yours block by block until something doesn't work.
Or do nothing.? I can't remember a problem in 20+ years unless there's been a power failure.
Better to host at AWS or something if you need extreme connection/power reliability.? Also, use logic in your code that is robust to interruptions in data or order/position/portfolio callbacks.

You are right Juergen. great find (for me at least)!

I had been receiving daily flex reports since ancient times with OrderRefs filled without any additional settings until they stopped filling it a few months ago or so, so I was unaware that this would now need additional configuration and thought it went the way of the respective field not getting filled for any period exceeding the last trading date. This new configuration certainly sounds quite redundant to me since my explicit ordering of OrderRefs in the report column settings signals I want it filled, but well, I guess IB tried to further minimize the chance that this potentially dense data field inadvertently tax their system.

--
Best,
DS

开云体育

Thank you Richard as well.

Thank you 闯ü谤驳别苍 and Robby for your quick response. I followed your instructions and it worked.?

Why don't you print profit_order_ID right before 'if' to find out? Also they might be of different type, I'd check that too.

Thank you for clarifying this, strangely I do not see much more incoming data messages when using TickByTick, so I am not sure how it saves the bandwidth for them, but lesson's learned.

Fran,

Try my way removing localSymbol. It works for me though I am using C#.

When you specify localSymbol you don't have to provide lastTradeDateOrContractMonth as well. But if you do, they have to match. You set lastTradeDateOrContractMonth to "202315", which is not even a real date in the first place.

I suggest you leave that field alone (that's what I do) or you'd have to provide the correct date of "20231215"

闯ü谤驳别苍

On Mon, Nov 20, 2023 at 08:48 PM, fran wrote: