Re: Collecting tick-by-tick price data
¿ªÔÆÌåÓý´³¨¹°ù²µ±ð²Ô ? Thanks. Very informative. Clearly my statement about ¡°how badly behind the ¡®tick-by-tick¡¯ data is¡± in my post of 16 September is incorrect, and there is something in the path between IB¡¯s servers and my network that is causing the individual tick messages to be amalgamated into larger transmission lumps. ? There are 14 hops between my network and either IB¡¯s US or European servers (87ms ping to the US, 35ms to Europe). I guess any of those items could be contributing to this, but it seems pretty unlikely to me that massive, general-purpose internet routers would do anything other than send every packet on its way as soon as possible. ? So the most likely culprits are my ISP, their broadband router, and something in my own network. ? As it happens, I¡¯m switching to a new ISP early next week, with a different router, so it¡¯ll be interesting to see if that makes any difference. This will be higher bandwidth, but latency will be much the same. ? But before that, I intend to utilise the second (currently unused) network adapter on my virtualisation server to create a direct route just for IB market data to the broadband router ¨C at the moment there¡¯s about 40Mbps of other data flowing through the same network adapter as the market data. Given that it¡¯s a 1Gbps connection, I don¡¯t really see that this should cause any real congestion. But separating the market data out via a dedicated network adapter can only help. We shall see¡ ? I might also try relocating TWS to another virtual machine that has less of a workload, perhaps to my Linux VM (which does nothing at all most of the time, in fact not even powered on as I don¡¯t really use it except when I have to). ? Anyway, I¡¯ll let you know if I discover anything of interest. ? Richard ? ? ? From: [email protected] <[email protected]> On Behalf Of ´³¨¹°ù²µ±ð²Ô Reinold via groups.io
Sent: 23 September 2022 07:11 To: [email protected] Subject: Re: [TWS API] Collecting tick-by-tick price data ? Richard, thank you for your explanation and the data. I am still mostly consumed with travel but have made progress on my analysis. I hope I can wrap that up and post soon. In the meantime, I found an hour (donated by early rising due to jet lag) and I lined up your logged data with ours. XLSX and CVS versions attached. The TickByTick data records match perfectly (we subscribe to the TickByTickLastAll though) but the LastTimeStamp, LastPrice, LastSize, and Volume data is similar but arrival times vary. We receive all data from the same feed. For the TickByTick data I calculated the time difference of arrival between when we received them and when you did. Assuming that neither of our servers made major time adjustments during the few seconds for this log I had expected some fluctuation but an approximately constant difference. It shows a much more complex pattern, though, and it looks like TickByTick data is delivered to you in large chunks (back to back data arrives in the same or next millisecond) while we get a stream, where only few back-to-back items arrive at essentially the same time and the time between data varies greatly. I guess there is the answer. I agree with your overall description (saves me a bunch of paragraphs for for when I post my analysis) but I would like to offer a small addition that explains occasional chunking of TickByTick data (but not the kind you experience). ESZ2 is very liquid but a lot of trades have only few contracts (less than 10, most of them just 1 or 2). Larger orders (say 20 contracts and more) will quickly be filled, but that will result in a burst of smaller trades to do so. When you sum up the sizes for TickByTickLast trades in our log with sub-millisecond back-to-back arrival, you frequently get sums such as 20, 30, 50, or similar numbers. Anyway. A more comprehensive Last vs TickByTick analysis soon. ´³¨¹°ù²µ±ð²Ô On Wed, Sep 21, 2022 at 07:18 PM, Richard L King wrote:
|