I agree that the old post is still relevant for reqMktData, Richard. I think I even said so. And price sampling at a rate of a few samples per second is also highly relevant and useful.
But I cannot confirm your observation that TickByTickLast data is badly behind sampled LastPrice ticks, though. At least not in our scenario:
- Very liquid instrument ESZ2 (ES futures)
- Very low flight time between IBKR and us (<4ms) and exchange and us (<10ms)
- Very high speed internet connection with huge bandwidth
- Java API 9.85 and TWS 10.18.1c
I don't have the time right now to do an in-depth analysis with accurate time measurements (though it is over due and I should be able to do something more formal next week) but attached a couple quick snapshots from ESZ2 at market open yesterday.
One snapshot is for the first ~30 seconds:
- There were 1,972 trades reported via individual TickByTickLast callbacks (the little blue dots)
- That corresponded to 79 LastPrice samples (red lines and larger red dots)
The second snapshot zooms in a little and shows the first ~10 seconds:
- In that time frame, 1,027 trades were reported by individual TickByTickLast callbacks
- With 36 corresponding LastPrice samples
In both charts, time is not to scale. In other words, ticks are listed left-to-right with equal spacing in the order they were received. Distance between dots in the charts does not represent the time between dots.
Obviously, LastPrice samples at a rate of 3 to 4 per second adequately represent the price movement, though the details and specifically fast price changes can only be learned from TickByTick since there are 25x more of them. And I am not saying that there are no instances, where TickByTickLast trades arrive after a sampled LastPrice tick, but in our case, at least empirically, TickByTickLast are not consistently behind LastPrice samples by 100++ ms.
And I agree with you that reqTickByTick data does not suddenly allow you to run HFT schemes. But for us they are invaluable for measuring changes in price movements:
- Are prices pretty much stagnant and are just jumping between Ask and Bid?
- Can we see a clear direction where every few trades the price moves up or down by a tick?
- Or are we seeing erratic jumps within a window of a six to ten ticks?
- And was that fast and huge price change just a spike from one or a few orders, or does it look like price has moved permanently and will stay there
TickByTick data helps us in risk assessment even if trading has a horizon of many seconds to many minutes.or even hours. It's one more gauge for fly-by-instrumens.
More on this, hopefully, next week.
´³¨¹°ù²µ±ð²Ô
PS: For completeness, here a histogram of all ESZ2 related ticks in the first 15 minutes after market open yesterday.
Count of ticks received from 08:30 to 08:45 US/Central on 20220915
| Count |
TickType |
API source |
| ? |
? |
? |
| 250,265 |
MarketDepth |
reqMarketDepth |
| ? |
? |
? |
| 228,906 |
TickByTickMidPoint |
reqTickByTick |
| 228,906 |
TickByTickBidAsk |
reqTickByTick |
| 37,238 |
TickByTickLast |
reqTickByTick |
| ? |
? |
? |
| 7,447 |
AskSize |
reqMktData |
| 7,442 |
BidSize |
reqMktData |
| 5,144 |
LastSize |
reqMktData |
| 3,354 |
Volume |
reqMktData |
| 2,361 |
AskPrice |
reqMktData |
| 2,349 |
BidPrice |
reqMktData |
| 2,091 |
OptionHistoricalVolatility |
reqMktData |
| 2,087 |
LastPrice |
reqMktData |
| 900 |
LastTimestamp |
reqMktData |
| 753 |
MarkPrice |
reqMktData |
| 357 |
OptionPutVolume |
reqMktData |
| 330 |
OptionCallVolume |
reqMktData |
| 180 |
RtDataBar |
reqMktData |
| 30 |
VolumeRate |
reqMktData |
| 30 |
TradeRate |
reqMktData |
| 30 |
TradeCount |
reqMktData |
| 15 |
ShortTermVolume5Minutes |
reqMktData |
| 15 |
ShortTermVolume3Minutes |
reqMktData |
| 15 |
ShortTermVolume10Minutes |
reqMktData |
| 11 |
Low |
reqMktData |
| 8 |
OptionImpliedVolatility |
reqMktData |
| ? |
? |
? |
| 780,264 |
Total recorded ESZ2 ticks |
? |
| 867 |
Ticks per second for ESZ2 |
? |
| ? |
? |
? |
| 2,968,365 |
Totoal recorded ticks all instruments |
? |
| 3,298 |
Ticks per second recorded |
? |
On Thu, Sep 15, 2022 at 06:09 PM, Richard L King wrote:
Jurgen
?
Yes it is an old post, but it¡¯s still accurate in its description of the basic mechanism. The sampling intervals are shorter now, especially for Forex, but it still works the same way.
?
Also changed is that when I wrote it, each reqMktData call knocked 1 off your count of available market data lines, and was actually a slightly different data stream because the alignment of the sampling periods wasn¡¯t the same . But at some point IB improved it that so that no matter how many requests you make for a particular contract, and whether in the same or different programs or even live or paper-trading, it only counts as one line.
?
It¡¯s quite easy to check out my description simply by making simultaneous requests for ¡®realtime¡¯ data and ¡®tick-by-tick¡¯ data and writing the received ticks to a text file. What you find is that the ¡®realtime¡¯ data ticks always arrive before the corresponding ¡®tick-by-tick¡¯ ticks, and actually I was amazed to discover how badly behind the ¡®tick-by-tick¡¯ data is: it usually arrives several hundred milliseconds behind the ¡®realtime¡¯ data.
?
So while you might get every tick with ¡®tick-by-tick¡¯ data, it would be useless for making snappy trading decisions because it¡¯s so out-of-date by the time it arrives.
?
Richard
?
?
The post you pointed to is 15 years old, and, at that point, only reqMktData existed. The API Reference Guide chapter ""? describes how sampling/aggregation is handled today. So the post is still relevant for reqMktData (even if sampling is a little faster and smarter now).
was introduced only a few years back and is not sampled or aggregated. Consequently, data volume is huge and the limits of how many simulations subscriptions can exist are much tighter, Every trade (ask/bid) is reported to you as it happens.
If you subscribe, for example, via reqMktData for an instrument, you will never get more that 4 to 6 updates per second even if that instrument is traded heavily. If you subscribe to TickByTiclLast for the same instrument, you might get hundreds if not thousands of ticks in very busy seconds. And you will see every price change including the "102" the post in your link refers to.
´³¨¹°ù²µ±ð²Ô
