|
been using a standard call to historical data for years but suddenly (9/21/22) I am getting
Error 366, reqId 4: No historical data query found for ticker id:4, contract: Stock(symbol='AAPL', exchange='SMART', currency='USD')
Did I miss a change in the API?
I am subscribed to all the relevant data and the code hasn't changed. I was working on 9/20/22 but not since then.
|
On Wed, Sep 27, 2023 at 06:43 AM, comicpilsen wrote:
I am subscribed to all the relevant data and the code hasn't changed. I was working on 9/20/22 but not since then.
What do you mean ? "I" or "it" ? You should submit herewith your request in full. You have an issue which is pretty unique and unusual. It maybe that it was luck that make it working before 9/20/22 Specify the "SecType" of the contract (I guess you look for STK) , it's more important than currency (if you specify explicitly the Exchange) Then try "ISLAND" instead of "SMART", if work be ready to use NASDAQ (IB is in the process of getting ride of ISLAND) Also surprising is the very small number for reqId, do you catch the IBApi.EWrapper.nextValidID? ?
|
fair point about the I / It. The statement should have been. The program has been working for several years without modification and it suddenly stopped working on 9/20/22. I am not asking for anyone to help me fix it. I just wanted to know if there had been any changes in IB's python API post 9/20/22. I couldn't see anything in the docs or the revisions. The code is fine so the problem is not in the logic.
|
Gordon ? I don¡¯t see anything in the least surprising about the ¡®very small number for reqId¡¯. ? nextValidId is only really relevant for order ids, because of the requirement for them to be increasing. ? Other ids, for example for market or historical data requests, can be anything you like in any order. In particular you can use the same id concurrently for different request types (ie each request type has its own id space). And you can use the same id for the same request type immediately after you¡¯ve cancelled a request or the request has completed. For example if you request contract details using id 0, then as soon as you get the end contract details callback you can make another contract details request using id 0. ? Note that you need to be a little careful when re-using ids. For example, if you use id 0 for a market data request, then cancel the request, and then immediately re-use id 0 for a different market data request, you might still get some market data callbacks from the original request because that data might have been in transit at the time you cancelled the original request. ? Richard ?
|
I agree with your summary of the requirements for orderIds and requestIds and that they are independent of each other, Richard.
From a practical point of view it makes sense to avoid situations where an active requestId (say for a market data subscription) and an active orderId have the same numerical value. While all callbacks will work perfectly fine and are unambiguous in such a situation, error handling gets more tricky since the id parameter returned by is now ambiguous and may relate to the active order or the active request.
We follow the lead of the ApiController class in the Java TWSAPI and separate the numerical values for orderIds and requestIds by a large value upon the? ? callback.
m_reqId = m_orderId + 10_000_000; // let order id's not collide with other request id's
Our framework then uses a sequence of requestIds starting with m_reqId and error messages can automatically be routed to the correct handler order or request handlers.
´³¨¹°ù²µ±ð²Ô
On Thu, Sep 28, 2023 at 05:16 AM, Richard L King wrote:
Gordon
?
I don¡¯t see anything in the least surprising about the ¡®very small number for reqId¡¯.
?
nextValidId is only really relevant for order ids, because of the requirement for them to be increasing.
?
Other ids, for example for market or historical data requests, can be anything you like in any order. In particular you can use the same id concurrently for different request types (ie each request type has its own id space). And you can use the same id for the same request type immediately after you¡¯ve cancelled a request or the request has completed. For example if you request contract details using id 0, then as soon as you get the end contract details callback you can make another contract details request using id 0.
?
Note that you need to be a little careful when re-using ids. For example, if you use id 0 for a market data request, then cancel the request, and then immediately re-use id 0 for a different market data request, you might still get some market data callbacks from the original request because that data might have been in transit at the time you cancelled the original request.
?
Richard
?
|
Requesting historical data for a contract with symbol='AAPL', exchange='SMART', currency='USD' and and undefined primaryExchange still work fine for us (tried 1_day bars for the last 100 days).
say "Historical market data request with this ticker id has either been cancelled or is not found." The historical data in your request does may indeed not exist for the bar size and time frame you are asking for, or your request got cancelled for some reason (such as a historical data pacing violation).
TWS nowadays allows you to enable logging of all Historical Market Data queries (HMDS). The log is very verbose but would probably give you the exact reason for error 366.
´³¨¹°ù²µ±ð²Ô
On Wed, Sep 27, 2023 at 08:43 AM, comicpilsen wrote:
been using a standard call to historical data for years but suddenly (9/21/22) I am getting
Error 366, reqId 4: No historical data query found for ticker id:4, contract: Stock(symbol='AAPL', exchange='SMART', currency='USD')
Did I miss a change in the API?
I am subscribed to all the relevant data and the code hasn't changed. I was working on 9/20/22 but not since then.
|
thank you Jurgen
I just wanted to see if there was any change in the API or Ib's
pacing rules. The code wants 1y 1d for about 100 symbols and has
been working for about 5 years, all the symbols are have valid
credentials.? I'm sure it's something my end and it's going to
give me an opportunity to improve my jupyter lab skills now they
have incorporated a debugger.
thanks again for everything you do and for running the 1d 100day
test. Most kind
I didn't know that there was enhanced logging for HMDS and that's
good to know. Thanks again.
On 9/28/23 10:01, ´³¨¹°ù²µ±ð²Ô Reinold via
groups.io wrote:
toggle quoted message
Show quoted text
Requesting historical data for a contract with symbol='AAPL',
exchange='SMART', currency='USD' and and undefined
primaryExchange still work fine for us (tried 1_day bars for the
last 100 days).
say "Historical market
data request with this ticker id has either been cancelled or is
not found." The historical data in your request does may indeed
not exist for the bar size and time frame you are asking for, or
your request got cancelled for some reason (such as a historical
data pacing violation).
TWS nowadays allows you to enable logging of all Historical
Market Data queries (HMDS). The log is very verbose but would
probably give you the exact reason for error 366.
´³¨¹°ù²µ±ð²Ô
On Wed, Sep 27, 2023 at 08:43 AM, comicpilsen wrote:
been using a standard call to historical data for years but
suddenly (9/21/22) I am getting
Error 366, reqId 4: No historical data query found for ticker id:4, contract: Stock(symbol='AAPL', exchange='SMART', currency='USD')
Did I miss a change in the API?
I am subscribed to all the relevant data and the code hasn't
changed. I was working on 9/20/22 but not since then.
|
comicpilsen, I see you consistently mentioning the date to be 9/20/22. Are you sure that you stopped getting data one year ago? Or am I misunderstanding what the issue is?
|
WOW that's my bad sorry. the last date the program worked was Thu
21 Sep 2023 02:11:09 AM then the next day Fri 22 Sep 2023
02:11:09 AM CDT? it failed to get any data.? Excellent work Holmes
:-)?
On 9/28/23 10:40, J G via groups.io
wrote:
toggle quoted message
Show quoted text
comicpilsen, I see you consistently mentioning the date to be
9/20/22. Are you sure that you stopped getting data one year ago?
Or am I misunderstanding what the issue is?
|
´³¨¹°ù²µ±ð²Ô ? I wasn¡¯t trying to suggest that one should use the same order ids for different request types, merely that the rules are far more relaxed than Gordon¡¯s post implied. ? Indeed, what I¡¯ve been doing in my own platform since 10 March 2006 (according to my source control repository) is to use different ranges of ids for different classes of requests (not simply distinguishing between order ids and all other ids). Apart from simplifying error handling (which was indeed the initial impetus for doing this), it is also helpful in the next layer up from the component that implements the TWSAPI. ? I think that somewhat predates IB¡¯s ApiController class! The latter is from May 2013 in TWS API v9.70.. ? Richard
|
You are welcome.
Generally, IBKR has relaxed historical data pacing rules over the years, but in the spirit of "did something change", here a couple of observations. This is "thin ice territory" in the sense that I'll describe some observations but I have no more insight into or documentation for what is really going on.
During the transition from V9 to V10 TWS/IBGW we detected that Historical Market Data Servers (HMDS farms) had become more prominent and that they must have taken on functions other than simply providing historical market data. They may had been involved before, but as part of our V10 testing and qualification they stood out. Also, TWS/IBGW suddenly had the HMDS logging feature that gives great insight into the kinds of XML database queries TWS/IBGW send to the various HMDS farms.
HMDS and market data farms in each region seem to be responsible for contract related details, real time data, and historical data for the instruments traded in that region. Below a log overview for an attempt to get real-time market data for a Frankfurt FWB traded instrument that we do not have a market data subscription for. You see that our TWS was connected to usfuture.nj, usfuture, usfarm, ushmds, and secdefnj at the moment our client started and that euhmds and eufarm were temporarily connected once we made the call. They were automatically disconnected after 3 minutes of inactivity.
This may be relevant to your problem since most (all ?) HMDS queries trugered by your clients' requests (not just calls) seem to count somehow towards historical market data pacing. At least for TWS around V10.10? to V10.12 when we did our tests.
The funny thing was that we got unexplained TWSAPI errors (similar to your 366) that did not say "pacing violation" in the error message but the corresponding TWS HMDS internal logs clearly did. We restructured the tool that caused the errors, removed calls to that we really did not need (they caused HMDS queries) and the errors when away.
So I guess this was the long way of saying that the combined requests your client sends to TWS/IBGW may trigger a silent pacing violation that leads to error 366 "ticker id was cancelled" situations. After all, you said you make requests for 100 instruments.
Just a thought,
´³¨¹°ù²µ±ð²Ô === 12:31.13 ===Connect to TWSreqError code=2104 msg=Market data farm connection is OK:usfuture.njreqError code=2104 msg=Market data farm connection is OK:usfuturereqError code=2104 msg=Market data farm connection is OK:usfarmreqError code=2106 msg=HMDS data farm connection is OK:ushmdsreqError code=2158 msg=Sec-def data farm connection is OK:secdefnjreqContractDetails for ADS:STK:FWBreqMktData( ADS:STK:FWB )reqError code=2106 msg=HMDS data farm connection is OK:euhmdsreqError code=2104 msg=Market data farm connection is OK:eufarmtickReqParams[ADS] minTick 0.0 bboExchange 250001 snapshotPermissions 4reqError code=354 msg=Requested market data is not subscribed.Delayed market data is not available.ADS IBIS/TOP/ALL=== 12:31:16.488 ===Conection stays active by no more API requests or responses=== 12:34:52.270 ===reqError code=2108 msg=Market data farm connection is inactive but should be available upon demand.eufarm=== 12:34:52.346 ===reqError code=2108 msg=Market data farm connection is inactive but should be available upon demand.eufarm
On Thu, Sep 28, 2023 at 10:21 AM, dent wrote:
thank you Jurgen
I just wanted to see if there was any change in the API or Ib's pacing rules. The code wants 1y 1d for about 100 symbols and has been working for about 5 years, all the symbols are have valid credentials.? I'm sure it's something my end and it's going to give me an opportunity to improve my jupyter lab skills now they have incorporated a debugger.
thanks again for everything you do and for running the 1d 100day test. Most kind
?
I didn't know that there was enhanced logging for HMDS and that's good to know. Thanks again.
?
On 9/28/23 10:01, ´³¨¹°ù²µ±ð²Ô Reinold via groups.io wrote:
Requesting historical data for a contract with symbol='AAPL', exchange='SMART', currency='USD' and and undefined primaryExchange still work fine for us (tried 1_day bars for the last 100 days).
say "Historical market data request with this ticker id has either been cancelled or is not found." The historical data in your request does may indeed not exist for the bar size and time frame you are asking for, or your request got cancelled for some reason (such as a historical data pacing violation).
TWS nowadays allows you to enable logging of all Historical Market Data queries (HMDS). The log is very verbose but would probably give you the exact reason for error 366.
´³¨¹°ù²µ±ð²Ô
On Wed, Sep 27, 2023 at 08:43 AM, comicpilsen wrote:
been using a standard call to historical data for years but suddenly (9/21/22) I am getting
Error 366, reqId 4: No historical data query found for ticker id:4, contract: Stock(symbol='AAPL', exchange='SMART', currency='USD')
Did I miss a change in the API?
I am subscribed to all the relevant data and the code hasn't changed. I was working on 9/20/22 but not since then.
|
thank you AGAIN for such a detailed analysis.? I have turned on
the HMDS logging feature for extra detail, thanks again for the
heads up.? I am always a little nervous when there is a hint of
data flow consolidation. I can see why they would want to do it
but...
the issue I have with these logs is that I would have expected
the log to show the unique identifiers for the request. Your
request reqMktData( ADS:STK:FWB )? has no, that I can see,
identifier NOR does the log give me a CLEAR indication of which
dataserver got the request. I understand that they have load
balancers but it would be nice to be able to read the logs more
clearly.
I had not thought to look at the internal TWS logs partly because
I don't like having to decrypt them all the time. I hoped that I
could turn encryption off but I suppose that it's necessary in
this day and age.
Based on your wonderful ( as always) overview I am going through
all my linux boxes ( scattered over the house) and check the
crontab for TWS scripts that might be running that add to my data
pacing. I'm decrypting an api log now so I'll try to figure out
what's going on. I am not a fan of the TWS log encryption
preferring straightforward text so I can ssh into the tws machine
and work that way.
thanks again for EVERYTHING you do.
On 9/28/23 13:52, ´³¨¹°ù²µ±ð²Ô Reinold via
groups.io wrote:
toggle quoted message
Show quoted text
You are welcome.
Generally, IBKR has relaxed historical data pacing rules over
the years, but in the spirit of "did something change", here a
couple of observations. This is "thin ice territory" in the
sense that I'll describe some observations but I have no more
insight into or documentation for what is really going on.
During the transition from V9 to V10 TWS/IBGW we detected that
Historical Market Data Servers (HMDS farms) had become more
prominent and that they must have taken on functions other than
simply providing historical market data. They may had been
involved before, but as part of our V10 testing and
qualification they stood out. Also, TWS/IBGW suddenly had the
HMDS logging feature that gives great insight into the kinds of
XML database queries TWS/IBGW send to the various HMDS farms.
HMDS and market data farms in each region seem to be
responsible for contract related details, real time data, and
historical data for the instruments traded in that region. Below
a log overview for an attempt to get real-time market data for a
Frankfurt FWB traded instrument that we do not have a market
data subscription for. You see that our TWS was connected to
usfuture.nj, usfuture, usfarm, ushmds, and secdefnj at the
moment our client started and that euhmds and eufarm were
temporarily connected once we made the call. They
were automatically disconnected after 3 minutes of inactivity.
This may be relevant to your problem since most (all ?) HMDS
queries trugered by your clients' requests (not just
calls) seem to count somehow towards historical market data
pacing. At least for TWS around V10.10? to V10.12 when we did
our tests.
The funny thing was that we got unexplained TWSAPI errors
(similar to your 366) that did not say "pacing violation" in the
error message but the corresponding TWS HMDS internal logs
clearly did. We restructured the tool that caused the errors,
removed calls to that
we really did not need (they caused HMDS queries) and the errors
when away.
So I guess this was the long way of saying that the combined
requests your client sends to TWS/IBGW may trigger a silent
pacing violation that leads to error 366 "ticker id was
cancelled" situations. After all, you said you make requests for
100 instruments.
Just a thought,
´³¨¹°ù²µ±ð²Ô
===
12:31.13 ===
Connect to
TWS
reqError
code=2104 msg=Market data farm connection is OK:usfuture.nj
reqError
code=2104 msg=Market data farm connection is OK:usfuture
reqError
code=2104 msg=Market data farm connection is OK:usfarm
reqError
code=2106 msg=HMDS data farm connection is OK:ushmds
reqError
code=2158 msg=Sec-def data farm connection is OK:secdefnj
reqContractDetails
for ADS:STK:FWB
reqMktData(
ADS:STK:FWB )
reqError
code=2106 msg=HMDS data farm connection is OK:euhmds
reqError
code=2104 msg=Market data farm connection is OK:eufarm
tickReqParams[ADS]
minTick 0.0 bboExchange 250001 snapshotPermissions 4
reqError
code=354 msg=Requested market data is not subscribed.Delayed
market data is not available.ADS IBIS/TOP/ALL
===
12:31:16.488 ===
Conection
stays active by no more API requests or responses
===
12:34:52.270 ===
reqError
code=2108 msg=Market data farm connection is inactive but should
be available upon demand.eufarm
===
12:34:52.346 ===
reqError
code=2108 msg=Market data farm connection is inactive but should
be available upon demand.eufarm
On Thu, Sep 28, 2023 at 10:21 AM, dent wrote:
thank you Jurgen
I just wanted to see if there was any change in the API or
Ib's pacing rules. The code wants 1y 1d for about 100 symbols
and has been working for about 5 years, all the symbols are
have valid credentials.? I'm sure it's something my end and
it's going to give me an opportunity to improve my jupyter lab
skills now they have incorporated a debugger.
thanks again for everything you do and for running the 1d
100day test. Most kind
?
I didn't know that there was enhanced logging for HMDS and
that's good to know. Thanks again.
?
On 9/28/23 10:01, ´³¨¹°ù²µ±ð²Ô Reinold
via groups.io wrote:
Requesting historical data for a contract with
symbol='AAPL', exchange='SMART', currency='USD' and and
undefined primaryExchange still work fine for us (tried
1_day bars for the last 100 days).
say "Historical
market data request with this ticker id has either been
cancelled or is not found." The historical data in your
request does may indeed not exist for the bar size and time
frame you are asking for, or your request got cancelled for
some reason (such as a historical data pacing violation).
TWS nowadays allows you to enable logging of all Historical
Market Data queries (HMDS). The log is very verbose but
would probably give you the exact reason for error 366.
´³¨¹°ù²µ±ð²Ô
On Wed, Sep 27, 2023 at 08:43 AM, comicpilsen wrote:
been using a standard call to historical data for years
but suddenly (9/21/22) I am getting
Error 366, reqId 4: No historical data query found for ticker id:4, contract: Stock(symbol='AAPL', exchange='SMART', currency='USD')
Did I miss a change in the API?
I am subscribed to all the relevant data and the code
hasn't changed. I was working on 9/20/22 but not since
then.
|
Good luck with log reading. I hope it helps you find the root cause.
I removed lots of log detail in the hope that would make things clearer, but apparently it did not. To summarize the log one more time:
- The real-time market data request for a Frankfurt based instrument is handled by eufarm in Switzerland
- My TWS is in the US, did not have an active connection to eufarm but automatically established one when needed, No surprise here.
- The somewhat surprising observation is that the Historical Market Data Farm euhmds is also needed, although my client did not request any historical data. In fact TWS always connects the corresponding HMDS farm before the actual data farm since it provides certain instrument related information it needs for the real-time market data subscription.
Since you push the envelope a little by requesting historical data for 100 instruments I thought it might be good to make sure the client is careful with other requests that inadvertently cause HMDS queries.
´³¨¹°ù²µ±ð²Ô
On Thu, Sep 28, 2023 at 02:16 PM, dent wrote:
thank you AGAIN for such a detailed analysis.? I have turned on the HMDS logging feature for extra detail, thanks again for the heads up.? I am always a little nervous when there is a hint of data flow consolidation. I can see why they would want to do it but...
the issue I have with these logs is that I would have expected the log to show the unique identifiers for the request. Your request reqMktData( ADS:STK:FWB )? has no, that I can see, identifier NOR does the log give me a CLEAR indication of which dataserver got the request. I understand that they have load balancers but it would be nice to be able to read the logs more clearly.
I had not thought to look at the internal TWS logs partly because I don't like having to decrypt them all the time. I hoped that I could turn encryption off but I suppose that it's necessary in this day and age.
Based on your wonderful ( as always) overview I am going through all my linux boxes ( scattered over the house) and check the crontab for TWS scripts that might be running that add to my data pacing. I'm decrypting an api log now so I'll try to figure out what's going on. I am not a fan of the TWS log encryption preferring straightforward text so I can ssh into the tws machine and work that way.
thanks again for EVERYTHING you do.
?
|
thank you for the clarification. Gives me a better handle on
what's going on. I'm going to rationalize my data approach in 2024
as I have MANY experimental scripts ( python, julia, and now mojo)
which are run by cron jobs. I just shut one down that's been
running for 2 years and there's a 20gb csv with all the data.?
Thanks again for all you do and taking the time.
On 9/28/23 14:33, ´³¨¹°ù²µ±ð²Ô Reinold via
groups.io wrote:
toggle quoted message
Show quoted text
Good luck with log reading. I hope it helps you find the root
cause.
I removed lots of log detail in the hope that would make things
clearer, but apparently it did not. To summarize the log one more
time:
- The real-time market data request for a Frankfurt based
instrument is handled by eufarm in Switzerland
- My TWS is in the US, did not have an active connection to
eufarm but automatically established one when needed, No
surprise here.
- The somewhat surprising observation is that the Historical
Market Data Farm euhmds is also needed, although my client did
not request any historical data. In fact TWS always connects
the corresponding HMDS farm before the actual data farm since
it provides certain instrument related information it needs
for the real-time market data subscription.
Since you push the envelope a little by requesting historical
data for 100 instruments I thought it might be good to make sure
the client is careful with other requests that inadvertently
cause HMDS queries.
´³¨¹°ù²µ±ð²Ô
On Thu, Sep 28, 2023 at 02:16 PM, dent wrote:
thank you AGAIN for such a detailed analysis.? I have turned
on the HMDS logging feature for extra detail, thanks again for
the heads up.? I am always a little nervous when there is a
hint of data flow consolidation. I can see why they would want
to do it but...
the issue I have with these logs is that I would have
expected the log to show the unique identifiers for the
request. Your request reqMktData( ADS:STK:FWB )? has no, that
I can see, identifier NOR does the log give me a CLEAR
indication of which dataserver got the request. I understand
that they have load balancers but it would be nice to be able
to read the logs more clearly.
I had not thought to look at the internal TWS logs partly
because I don't like having to decrypt them all the time. I
hoped that I could turn encryption off but I suppose that it's
necessary in this day and age.
Based on your wonderful ( as always) overview I am going
through all my linux boxes ( scattered over the house) and
check the crontab for TWS scripts that might be running that
add to my data pacing. I'm decrypting an api log now so I'll
try to figure out what's going on. I am not a fan of the TWS
log encryption preferring straightforward text so I can ssh
into the tws machine and work that way.
thanks again for EVERYTHING you do.
?
|
I never got a first nextValidID? < 100.?
Got things like "117", why ? never look at the why, I don't know of any zombie process and it works, so ...
Your absolutely right, rules are more relax, and I am a bad guy, I don't abide all rules that I may recommend. However I was burned being too relax.
You have more experience than I on handling IB and have the practical knowledge of the limits in use. As your approach work and fulfill your need, and your right, classifying request is not a must.
To be a better guy ... hereafter sharing what I do. (IMHO, just a use case) not the right thread for that but the djinn is out of the bottle
1- I use a single central requestId 'pump', thread safe,
and another one, separate, for OrderId.
I used to use the same pump for both, but it's not pretty. (very subjective criteria);
2- I always track nextValidId and use it as seed for it.
Taking the highest of next one or IB suggestion.
Handling of nextValidID must be within the protected critical section of the TS pump.
But it's very rare, happens only on connect.
3- I don't call reqIds , not that it's bad, just that I am not ready to wait for the answer.
4- I prefer "overpumping" (wasting ids) than taking the risk to re-use an existing id, even if serviced (even if it may work, I feel it's like playing with fire)
5- I try to keep continuity in OrderId, (I do in fact, but the code is ready for any discontinuity)
But I don't care at all about continuity on 'request'.
Assuming that the higher is the reqid the latest the request was built, was enough for diag.
The side effect is that OrderId grow much slower than reqid, so rapidly you can diagnose what you are looking at just by the range of the number.
@Jurgen, de facto it does something like what you suggest with +100000, (which make more sense, using a fixed value, I agree, as its very unlikely you ever do 100000 orders without a reboot)
However again human-wise: it's difficult to read a huge number.
Most of the time, tests are conducted at beginning of code start, hence small numbers which are easier to memorize or even simply read.
6- I did try to make it bit field on LSB,
In fact in was not so handy as sorting pending/send/received request id manually during debug can become cumbersome.
It help tracking the class of request but you loose the ease to track the sequence.
I dislike ranges, because I always wonder if or when they may overlap. (or require code to handle it as an exception) Not enough experiences with that, probably too dogmatic
This is just a use case.
|
@Richard
I know that you were not trying to suggest one should use the same numerical ids for different request types. I just wanted to add another reason why one should not do it.
Looks like we "late comers" have a little advantage since the TWS API source code contains more useful bits than what you had to work with in the early years.
@Gordon
We are meandering away from the original topic, but a couple thoughts on why you get "117" or something similar for nextValidId.
TWS/IBGW actually memorizes the highest orderId for each clientId and each account. I did not know this either for the longest time, but I think it was JG who made a comment related to this in a post a couple months ago. So I went through the logs and looked for the oldest clientId I could find for a client that placed orders (that clientId had not been used in at least three years), used that clientId to connect to the account, and for sure, automagically, nextOrderId upon connection returned a number that was one higher than the last order that client had placed three years ago.
So if you are reusing the same clientId and that client occasionally places orders, nextOrderId will be going up slightly over time. I guess the clientId that gave you a nextValidId of "117" has placed 115 or so orders over time.
You can break that cycle and reset all orderIds for all clientIds for an account back to, I believe, "1" with the "Reset API order ID sequence" button at the bottom of "Global Configuration -> Configuration -> API -> Settings".
We also never call reqIds while a client is running. The nextValidId we receive as part of the connection protocol is sufficient for a sequence of (thread safe) incrementing orderIds.
We recently added a little code that potentially nudges the clients' internal nextValidOrderId counter while they are running. That code assures that the nextValidOrderId counter is always higher than any orderId the client is exposed to by openOrder and orderStatus callbacks. This is most important for the master client and client 0 but we just added it to the framework for all clients based on this comment in :
"However if there are multiple client applications connected to one account, it is necessary to use an order ID with new orders which is greater than all previous order IDs returned to the client application in openOrder or orderStatus callbacks."
´³¨¹°ù²µ±ð²Ô
On Thu, Sep 28, 2023 at 03:00 PM, Gordon Eldest wrote:
I never got a first nextValidID? < 100.?
Got things like "117", why ? never look at the why, I don't know of any zombie process and it works, so ...
|
´³¨¹°ù²µ±ð²Ô wrote: "TWS/IBGW actually memorizes the highest orderId for each clientId and each account. I did not know this either for the longest time, but I think it was JG who made a comment related to this in a post a couple months ago."
Your memory has not failed you: it was indeed me who wrote on a few occasions that TWS/IBGW memorizes the highest orderId for each clientId.
|
