#2126

Hi,

I run two instances of TWS on the same machine. Sometimes TWS autorestart works, and sometimes it doesn't. Sometimes it appears to work, but then (I suspect) TWS is restarted in read-only mode. I run IBC 3.16.2, and TWS 10.19.2a
It's a bit contrived, hopefully I can explain it properly.

Here is what happened this week so far.

I started the two instances of TWS on Sunday morning. Everything worked fine.
Autorestart is set to happen at 8pm for one TWS instance, and 8:05pm for the other.
Autorestart worked fine on Sunday evening and Monday evening.
On Tuesday evening autorestart failed. On the computer it looks like the old instance is still running, and a new instance has been started (it says something to the effect that 'session is already in use').
I shut down all TWS running (either by closing the TWS window or by shutting down IBC's console window), and I restarted both instances manually.
On Wednesday evening, TWS restart appears to have happened (I wasn't there when it happened); however, looking at IBC log I don't see any evidence that autorestart happened.
The next morning, it seems like TWS was in read-only mode (is there a way to confirm this? I'm only saying this because my API client was able to connect to TWS, but then couldn't cycle through accounts to get positions).
I attach two sets of logs for each instance (one log that started on Sunday and the other on Tuesday when I manually restarted IBC). There were no other logs until I restarted on Thursday morning.
I did delete 32k lines of "JTS-EServerSocket-296: Couldn't write to log file - java.io.IOException: write beyond end of stream" from the instance 1 logs just to make them more readable. I have an ongoing issue with the API logs aborting shortly after my api client connects (only to instance 1), and that seems to output a lot of TWS error messages to the console, which ends up in the IBC log.

#2127

开云体育

Well, I’ve been puzzling over this for a couple of hours now, and I can’t see that IBC is doing anything it shouldn’t (or not doing something it should).

?

TWS, on the other hand, seems to be behaving badly.

?

First off there’s the issue with the TWS error messages about writing to its logfile. That in itself is strange, but what’s stranger is that it only affects instance 1. Perhaps both instances are trying to write to the same logfile? (Note that these lines end up in IBC’s log because TWS presumably just writes them to stderr, as it’s not managing to write to its Log4J logfile, and IBC always records anything written to stderr.)

?

Secondly I notice that both instances are configured to run in live mode rather than paper trading mode, and yet only instance 1 seems to invoke 2^nd factor authentication. Does the instance 2 account perhaps do 2FA via other means than IBKR Mobile?

?

Next, as you say TWS sometimes seem to sail though the auto-restart time without doing it. There’s nothing IBC can do about this: it doesn’t even know whether or when TWS should auto-restart.

?

Then there’s the couple of cases where there is an existing session. Perhaps you somehow started these manually and forgot about it? The only other way it can happen is at auto-restart if the TWS executable still has its original filename: on Windows this is tws.exe. The start scripts rename this to tws1.exe, otherwise when TWS auto-restarts it loads a new instance of tws.exe directly which then runs without IBC (and the IBC script itself also starts another one running under IBC).

?

Overall, I’m not sure what to suggest here. I note that your configuration certainly looks as though there should be no interaction at all between the two instances, but it might help to just turn one off for the moment and see if the other one behaves itself.

?

And I think the following are worth considering:

?

Make sure you have the latest version of TWS 1019 installed, which is 10.19.2a
It wouldn’t do any harm to upgrade to the latest IBC, ie 3.17.0 released a few days ago
If you use a Scheduled Task to periodically run IBC (for example to ensure it’s restarted after unexpected exit), make sure the task is configured not to run if an existing instance is already running
Make sure you never rename tws1.exe back to tws.exe while IBC is running
See what happens if you leave an instance running after it fails to initiate auto-restart. Does it do it the next day? Does it keep running forever?
Check whether there are any modal dialogs displayed at auto-restart time. It’s just possible that TWS would not honour auto-restart in this case (but I’ve no idea whether that’s actually true).

?

Finally you say that TWS appeared to be in ‘read-only API’ mode. If that’s the case, then it’s TWS misbehaving, not IBC. IBC can set this at startup, but only if ReadOnlyApi=yes in config.ini.

?

I hope something in all this helps…

?

Richard

?

Show quoted text

From: [email protected] <[email protected]> On Behalf Of Jimmy
Sent: Thursday, June 29, 2023 7:34 PM
To: [email protected]
Subject: [ibc] Autorestart issue

?

Hi,

I run two instances of TWS on the same machine. Sometimes TWS autorestart works, and sometimes it doesn't. Sometimes it appears to work, but then (I suspect) TWS is restarted in read-only mode. I run IBC 3.16.2, and TWS 10.19.2a
It's a bit contrived, hopefully I can explain it properly.

Here is what happened this week so far.

I started the two instances of TWS on Sunday morning. Everything worked fine.
Autorestart is set to happen at 8pm for one TWS instance, and 8:05pm for the other.
Autorestart worked fine on Sunday evening and Monday evening.
On Tuesday evening autorestart failed. On the computer it looks like the old instance is still running, and a new instance has been started (it says something to the effect that 'session is already in use').
I shut down all TWS running (either by closing the TWS window or by shutting down IBC's console window), and I restarted both instances manually.
On Wednesday evening, TWS restart appears to have happened (I wasn't there when it happened); however, looking at IBC log I don't see any evidence that autorestart happened.
The next morning, it seems like TWS was in read-only mode (is there a way to confirm this? I'm only saying this because my API client was able to connect to TWS, but then couldn't cycle through accounts to get positions).
I attach two sets of logs for each instance (one log that started on Sunday and the other on Tuesday when I manually restarted IBC). There were no other logs until I restarted on Thursday morning.
I did delete 32k lines of "JTS-EServerSocket-296: Couldn't write to log file - java.io.IOException: write beyond end of stream" from the instance 1 logs just to make them more readable. I have an ongoing issue with the API logs aborting shortly after my api client connects (only to instance 1), and that seems to output a lot of TWS error messages to the console, which ends up in the IBC log.

#2128

Thanks for giving this some thoughts Richard.

Based on your questions, I'm going to try three experiments. Please let me know if you have ideas on what else I should try.

1. This week I'm running only instance 1, directly (without IBC).?

I can already confirm that the issue?with the API log aborting (TWS log was always fine) is still occurring. So this part clearly has nothing to do with IBC, or with running multiple instances of TWS. I already have a ticket with IB about this. Sure enough, they tried to blame it on IBC, but that has now been debunked.

So far TWS restarted twice (Sunday & Monday) without issues.

2. Assuming the restarts run fine all week, next week I will run only instance 1 still, but this time *with* IBC.

3. The week after that I will run both instances without IBC.

And to answer your questions (in case this might trigger some additional ideas):

1. Yes, both instances are in live mode, and instance 2 really doesn't use 2FA (it uses an account domiciled in a jurisdiction that doesn't require 2FA).

2. Both instances are running TWS 10.19.2a?

3. There are no scheduled tasks to run IBC. If there were no restart issues I would just start IBC manually once a week on Sundays.

4. I never renamed tws1.exe to tws.exe. I do notice just now (while I'm not using IBC) that in Jts/1019 there is a tws1.exe dated 2022-11-15 (file version 1.19.1.7), while tws.exe is dated 2023-06-13 9file version 1.19.2.1). Does that mean that when IBC restarts TWS it uses an older version of TWS? Is that suspicious? Should I delete tws1.exe??

5. Unfortunately I can't leave instance?1 running if it is in read-only?mode (especially instance 1),?as I need non read-only for trading. Perhaps I can try that with instance 2. Is there a way to tell by looking at TWS if it is in read-only?mode (other than submitting an order)?

6. I'm not usually around when TWS restarts, but I'll try to be around. Does IBC log all TWS windows that come up, even if it is not a window it is waiting for or even knows about?

Thanks

Jimmy

Show quoted text

On Fri, Jun 30, 2023 at 6:58?AM Richard L King <rlking@...> wrote:

Well, I’ve been puzzling over this for a couple of hours now, and I can’t see that IBC is doing anything it shouldn’t (or not doing something it should).
?
TWS, on the other hand, seems to be behaving badly.
?
First off there’s the issue with the TWS error messages about writing to its logfile. That in itself is strange, but what’s stranger is that it only affects instance 1. Perhaps both instances are trying to write to the same logfile? (Note that these lines end up in IBC’s log because TWS presumably just writes them to stderr, as it’s not managing to write to its Log4J logfile, and IBC always records anything written to stderr.)
?
Secondly I notice that both instances are configured to run in live mode rather than paper trading mode, and yet only instance 1 seems to invoke 2^nd factor authentication. Does the instance 2 account perhaps do 2FA via other means than IBKR Mobile?
?
Next, as you say TWS sometimes seem to sail though the auto-restart time without doing it. There’s nothing IBC can do about this: it doesn’t even know whether or when TWS should auto-restart.
?
Then there’s the couple of cases where there is an existing session. Perhaps you somehow started these manually and forgot about it? The only other way it can happen is at auto-restart if the TWS executable still has its original filename: on Windows this is tws.exe. The start scripts rename this to tws1.exe, otherwise when TWS auto-restarts it loads a new instance of tws.exe directly which then runs without IBC (and the IBC script itself also starts another one running under IBC).
?
Overall, I’m not sure what to suggest here. I note that your configuration certainly looks as though there should be no interaction at all between the two instances, but it might help to just turn one off for the moment and see if the other one behaves itself.
?
And I think the following are worth considering:
?
Make sure you have the latest version of TWS 1019 installed, which is 10.19.2a
It wouldn’t do any harm to upgrade to the latest IBC, ie 3.17.0 released a few days ago
If you use a Scheduled Task to periodically run IBC (for example to ensure it’s restarted after unexpected exit), make sure the task is configured not to run if an existing instance is already running
Make sure you never rename tws1.exe back to tws.exe while IBC is running
See what happens if you leave an instance running after it fails to initiate auto-restart. Does it do it the next day? Does it keep running forever?
Check whether there are any modal dialogs displayed at auto-restart time. It’s just possible that TWS would not honour auto-restart in this case (but I’ve no idea whether that’s actually true).
?
Finally you say that TWS appeared to be in ‘read-only API’ mode. If that’s the case, then it’s TWS misbehaving, not IBC. IBC can set this at startup, but only if ReadOnlyApi=yes in config.ini.
?
I hope something in all this helps…
?
Richard
?

#2129

开云体育

Jimmy

?

I was just about to go to bed when I noticed your reply, so just a quick response for now.

?

The fact that you have both a tws.exe and a tws1.exe is almost certainly the cause of the problem. You should definitely delete the one that is the older version.

?

This means I need to look more closely at the way the renaming works. The problem probably arises if a new file version of the particular TWS version is installed (eg 1.19.1.7 -> 1.19.2.1), so that the updated tws.exe is put into place while the previous tws1.exe.is still there from previously running IBC. Then when IBC next starts, it tries to rename tws.exe to tws1.exe and I suspect that fails silently (or perhaps with an error that I’ve failed to notice), so that the new tws.exe remains in place. Then when restart occurs, a new tws instance is created from tws.exe but not running under IBC, and IBC also creates a new instance from tws1.exe. Hence the session already existing scenario.

?

I’ll have to check this all out properly tomorrow – not in a fit mental state to do it now!

?

And yes, IBC always records the fact that a window has been opened, regardless of whether it’s interested in it.

?

R.