Rarely is lack of "last hop" bandwidth the root cause for network issues. In my experience it is pretty much always latency. That is latency from processing, end-to-end "flight time", congestion along the route, traffic shaping games played by the ISP, or packet loss and retransmission. But since you can reproduce what you are seeing with a t3.2xlarge AWS instance, I am pretty sure that network issues are not the root cause of what you are seeing. Having said that, latency (or changes of latency) could very well make your experience better or worse.
I am still not convinced that there is a "bug" here and I was wondering, whether you have tried (or could try) some runs without timeouts. At least the Python code currently gives up and cancels the subscription after 20 seconds, right? Maybe it just takes that long occasionally when you send massive numbers of subscriptions and cancellations.
There are a few reasons why that may be the case:
- There is no interface contract or service level agreement that specifies how long it will take for the first market data ticks to arrive after a successful subscription. It may just be that it sometimes takes a while. Especially since in many cases a set of historical tick (last close, open, ...) is sent before the real-time data starts.
- The API documentation for market data snapshot says, that the tickSnapshotEnd() callback "... will indicate the snapshot reception is finished. Expected to occur 11 seconds after beginning of request." It does not guarantee that any data has been sent before the end of the 11 second period.
- I checked my records for the SPY ETF how long it takes from the market data subscription to the first data arriving. This comparison is not perfect, since my client subscribes between 16:00 and 17:00 US/Central, keeps the subscriptions active for 24 hours, and only makes few subscriptions compared to your 1,000++ runs. For about 2,500 subscriptions over the last few years, max time for the arrival of the first data tick was 6.3s with an average of 510ms and stdev of 794ms. That does not include quite a few cases where the first tick did not arrive for more than a minute and was a "LastTimeStamp" callback and not a data tick.
Just curious whether the data eventually arrives if you give it enough time.
闯ü谤驳别苍
PS. Just as an FYI attached recent latency and packet loss measurements for the IBKR entry points as experienced by a server outside of Chicago.
?
?
On Fri, Dec 20, 2024 at 10:10 AM, ajn wrote:
Uh, my sanity is restored :)
?
As for your comment about the network, yes my dev setup is in EU (yet its 250Mbit up/down fiber, typically no issues here with Internet otherwise, but of course anything can happen :) and again switching back and forth between 10.19 and 10.30 all the time, where 10.19 does't have issues) However, I can reproduce it from my test site in US just as well. Test site is in AWS with pretty biffy setup on the East Coast? (us-east-1, i..e supposed to be closest to IB servers, ?t3.2xlarge instance aws claims 5gbit network, ok it is probably less in reality but itis not likelly a bottleneck even it is "just" 500Mbit to the Internet ).?
