开云体育

Not receiving some (many) historical ticks from TWS through API

ds-avatar All Messages By This Member #46383 I'm having a problem getting a great deal of historical trade ticks from TWS that are available in Time & Sales and this issue appears pretty frequent for the more liquid stocks. For certain days in certain stocks, all I receive is a bunch of ticks until some intraday time, and nothing beyond that. If this issue is reproducible I will have to abandon my attempts to work with historical ticks through API. Can someone with good knowledge and easy access to API operations please check if you can get any content from the API with the following request: Request: reqHistoricalTicks() Args: contract: STK MU SMART USD startDateTime = "20210107 11:04:00" @America/New_York numberOfTicks = 1000 whatToShow = "TRADES" (data callback:?EWrapper.historicalTicksLast) useRTH = 0 ? This above is a request for a few trades in the Micron stock at a particular time. I can't get anything beyond 11:03:21 on 7 Jan 2021 despite heavy trading which is fully reflected in TWS Time and Sales and so apparently available from IBKR servers. This issue appears to be pretty frequent for this stock as well as BABA and some others that I checked. I'm not getting historical trades for MU also on January 25 and 27, and for BABA the situation is pretty horrible altogether - most of the days are incomplete. The issue seems to correlate with stocks' trading density as it's rarely seen in the less liquid stocks. Many thanks in advance.
Richard L King Member Profile All Messages By This Member #46388 开云体育 When I run your specified query I get 1000 ticks back. I've attached a file containing the results. ? Note that the timestamps are in my local timezone (UK), which is how the API returns the data, but they correspond to the range you specified. ? So I can't confirm your problem. ? ? ? toggle quoted message Show quoted text From: [email protected] <[email protected]> On Behalf Of ds-avatar Sent: 05 February 2021 11:17 To: [email protected] Subject: [TWS API] Not receiving some (many) historical ticks from TWS through API ? I'm having a problem getting a great deal of historical trade ticks from TWS that are available in Time & Sales and this issue appears pretty frequent for the more liquid stocks. For certain days in certain stocks, all I receive is a bunch of ticks until some intraday time, and nothing beyond that. If this issue is reproducible I will have to abandon my attempts to work with historical ticks through API. Can someone with good knowledge and easy access to API operations please check if you can get any content from the API with the following request: Request: reqHistoricalTicks() Args: contract: STK MU SMART USD startDateTime = "20210107 11:04:00" @America/New_York numberOfTicks = 1000 whatToShow = "TRADES" (data callback:?EWrapper.historicalTicksLast) useRTH = 0 ? This above is a request for a few trades in the Micron stock at a particular time. I can't get anything beyond 11:03:21 on 7 Jan 2021 despite heavy trading which is fully reflected in TWS Time and Sales and so apparently available from IBKR servers. This issue appears to be pretty frequent for this stock as well as BABA and some others that I checked. I'm not getting historical trades for MU also on January 25 and 27, and for BABA the situation is pretty horrible altogether - most of the days are incomplete. The issue seems to correlate with stocks' trading density as it's rarely seen in the less liquid stocks. Many thanks in advance. Historical ticks - MU.txt Historical ticks - MU.txt
Jürgen Reinold All Messages By This Member #46392 My historical TickByTick harvester also works fine for the time and instrument. Is your code prepared for more than 1000 ticks arriving. My initial request returned 1695 ticks, My time stamps are in US/Central time @Richard If I read your data correctly, it was off by a year. Brexit related? ============================ 14:28:48.798 INFO - Head time stamps for MU: ?? ?RTH Only:??? 1262163600 -> 2009-12-30 03:00:00.000 ?? ?No RTH Only: 1262163600 -> 2009-12-30 03:00:00.000 14:28:48.813 INFO - Parameters: ?? ?rootFolder = L:\tmp ?? ?symbol = MU ?? ?whatToShow = TRADES ?? ?startDateTime = 20091230 03:00:00 ?? ?endDateTime = 20210107 10:04:00 [1610035440] 14:28:48.813 INFO - Requesting TRADES data for MU/9939 ending 20210107 10:04:00 [1610035440000] 14:28:49.701 INFO - historicalTickLast(id=10000004, ticks=1695) -> [1610035401, 1610035439] -> [2021-01-07 10:03:21.000, 2021-01-07 10:03:59.000] 14:28:49.733 INFO - Requesting TRADES data for MU/9939 ending 20210107 10:03:20 [1610035400000] 14:28:50.176 INFO - historicalTickLast(id=10000005, ticks=1123) -> [1610035261, 1610035399] -> [2021-01-07 10:01:01.000, 2021-01-07 10:03:19.000] 14:28:50.181 INFO - Requesting TRADES data for MU/9939 ending 20210107 10:01:00 [1610035260000] 14:28:50.572 INFO - historicalTickLast(id=10000006, ticks=1001) -> [1610035146, 1610035259] -> [2021-01-07 09:59:06.000, 2021-01-07 10:00:59.000] 14:28:50.574 INFO - Requesting TRADES data for MU/9939 ending 20210107 09:59:05 [1610035145000] 14:28:50.982 INFO - historicalTickLast(id=10000007, ticks=1008) -> [1610035020, 1610035144] -> [2021-01-07 09:57:00.000, 2021-01-07 09:59:04.000] 14:28:50.982 INFO - Requesting TRADES data for MU/9939 ending 20210107 09:56:59 [1610035019000] 14:28:51.392 INFO - historicalTickLast(id=10000008, ticks=1008) -> [1610034877, 1610035018] -> [2021-01-07 09:54:37.000, 2021-01-07 09:56:58.000] 14:28:51.409 INFO - Requesting TRADES data for MU/9939 ending 20210107 09:54:36 [1610034876000] 14:28:51.792 INFO - historicalTickLast(id=10000009, ticks=1049) -> [1610034705, 1610034875] -> [2021-01-07 09:51:45.000, 2021-01-07 09:54:35.000] 14:28:51.792 INFO - Requesting TRADES data for MU/9939 ending 20210107 09:51:44 [1610034704000] 14:28:52.156 INFO - historicalTickLast(id=10000010, ticks=1011) -> [1610034588, 1610034703] -> [2021-01-07 09:49:48.000, 2021-01-07 09:51:43.000] 14:28:52.156 INFO - Requesting TRADES data for MU/9939 ending 20210107 09:49:47 [1610034587000] 14:28:52.494 INFO - historicalTickLast(id=10000011, ticks=1004) -> [1610034471, 1610034586] -> [2021-01-07 09:47:51.000, 2021-01-07 09:49:46.000] .... 14:30:55.933 INFO - Requesting TRADES data for MU/9939 ending 20210107 06:31:16 [1610022676000] 14:30:56.111 INFO - historicalTickLast(id=10000065, ticks=540) -> [1610010000, 1610022666] -> [2021-01-07 03:00:00.000, 2021-01-07 06:31:06.000] 14:30:56.122 INFO - Requesting TRADES data for MU/9939 ending 20210107 02:59:59 [1610009999000] 14:31:01.499 INFO - historicalTickLast(id=10000066, ticks=1000) -> [1609967629, 1609981177] -> [2021-01-06 15:13:49.000, 2021-01-06 18:59:37.000] 14:31:02.662 INFO - Data file written: L:\tmp\TRADES\MU-1609967629.TickByTickLast.json.gz 14:31:02.663 INFO - Requesting TRADES data for MU/9939 ending 20210106 15:13:48 [1609967628000] 14:31:03.074 INFO - historicalTickLast(id=10000067, ticks=1007) -> [1609966773, 1609967626] -> [2021-01-06 14:59:33.000, 2021-01-06 15:13:46.000] ... MU-1609967629.TickByTickLast.json.gz MU-1609967629.TickByTickLast.json.gz
Richard L King Member Profile All Messages By This Member #46394 开云体育 Oh dear me, you're dead right. I'm getting much too careless. Not so much Brexit as senescence… ? The stupid thing is that after making the (incorrect query), I thought I'd try it up to the latest point, and I realised then I was a year out, so I tried again with the correct date and all was well. But even though I hadn't sent the post at that point, it didn't occur to me to do the first query again… ? So I've attached the file again with the correct data this time. I also incorrectly said that it give me 1000 ticks, but in fact it was 1047, and this time it's 1695, the same as you. ? Where do I apply for a brain transplant?... ? ? ? toggle quoted message Show quoted text From: [email protected] <[email protected]> On Behalf Of JR Sent: 05 February 2021 20:46 To: [email protected] Subject: Re: [TWS API] Not receiving some (many) historical ticks from TWS through API ? @Richard If I read your data correctly, it was off by a year. Brexit related? ============================ Historical ticks - MU.txt Historical ticks - MU.txt
Jürgen Reinold All Messages By This Member #46395 senescence ... so that's the word for when I don't recognize the man in the mirror in the bathroom in the morning but decide to wash him anyway .. No worries, Richard. From what I read on this end, not appointment for brain transplant needed.
ds-avatar All Messages By This Member #46429 Richard, JR, thank you for offering your help. The responses from the API you posted made it obvious the problem is in my code.? ? After looking carefully and debugging I found the culprit in my tick response sequencing routine which always cut off the ticks at the last second of any response to fetch more data right off the discarded timestamp in order to avoid incomplete seconds (which is the finest tick data resolution the API offers). Apparently in some very liquid stocks there may be spikes in trading frequency resulting in more than 1000 ticks in a second so the entire API response comes with the same timestamps, leaving nothing to cut off this way, so I ended up discarding the whole thing. ? I think there is a kind of assumption in the API that the historical tick responses should complete a second full of data but it's not stated obviously in the docs. I rewrote my routine to simply advance 1 second off the last response' timestamp to harvest more data without cutting anything off and it seems to work alright now, though without an explicit commitment from IBKR it feels less reliable, doesn't it?
Nick #46430 It is documented, but not obviously: From "To complete a full second, more ticks may be returned than requested" toggle quoted message Show quoted text On 2/9/2021 7:53 AM, ds-avatar wrote: I rewrote my routine to simply advance 1 second off the last response' timestamp to harvest more data without cutting anything off and it seems to work alright now, though without an explicit commitment from IBKR it feels less reliable, doesn't it?

Previous Topic Next Topic