Keyboard Shortcuts
Likes
- Twsapi
- Messages
Search
|
Re: ib market data. is it clean and any good alterantive sources?
This really needs to be more specific as to which market data you are particularly interested in. Historical/real-time? Prices, quotes, earnings, ratings, news? For prices - every tick or OHLC bars such as 1-second or EOD? In my own projects, I'm now phasing out any IBKR API market data as a single source since for my purposes (perhaps quite specific ones, admittedly) they time and again proved to be somewhat unreliable, but even then it would be hard for me to give Polygon, which I also use quite a bit, a blanket recommendation because while their data quality seems mostly beyond reproach, some of their services require a lot of incremental labor in my project to be made use of, all the while coming at a pretty steep price. But then I guess compared to Bloomberg Terminal everything must be kind of cheap.
?
--
Best, DS |
|
Re: API documentation location?
I believe that is all we have now and my only hope is that we will not need one day to pass references to the Wayback machine between each other in order to understand the workings of API beyond those few dumbed-down cheat-sheets and object property dumps? on the "campus".
?
BTW if I may add, I'd appreciate if the group could help find a way to download the original TWS API web site to a local Windows computer for personal use (I'm sure this won't violate any licenses), because I'm really worried this unmaintained piece of precious of knowledge will go down some day still without proper substitute, let alone meaningful improvements. Neither web browsers nor WinHTTack managed to download it completely and with working navigation for me so far.
?
--
Best, DS |
|
API documentation location?
Hi all, I've been forced to update my code (thanks IB!) but I can't find an up-to-date version of? which is what I used the first time. Any ideas where the newer functionality is documented?? appears to not have a search function and also appears to not define any functions anywhere. Thanks in advance :) M +44 (0) 7528 551604? |
|
Re: ib market data. is it clean and any good alterantive sources?
toggle quoted message
Show quoted text
|
|
Re: requesting midpoint data
开云体育Why don’t you just ask IBKR? If anybody is likely to be able to provide you with chapter and verse, it’s them. ? Richard ? ? From: [email protected] <[email protected]> On Behalf Of tbrown122387 via groups.io
Sent: 07 February 2025 22:52 To: [email protected] Subject: [TWS API] requesting midpoint data ? I'm trying to request midpoint data (for symbols such as SPY, QQQ, etc.) via reqTickByTickData() and says "For some securities, getting tick-by-tick data requires Level 2 data bundles." ? 1/3: Did this change? If so, was that recent? I thought I remembered being able to do this while only subscribing to level 1. ? 2/3: Does that mean I need: ?US Equity and Options Add-On Streaming Bundle (P)Includes streaming real-time quotes for NYSE (CTA/Network A), AMEX (CTA/Network B), NASDAQ (UTP/Network C), and OPRA (US Options). In order to subscribe to US Equity and Options Add-On Streaming Bundle (P), the user must already be subscribed to US Securities Snapshot Bundle (P). ? Can I get rid of what I bought earlier this week? ? NYSE American, BATS, ARCA, IEX, and Regional Exchanges (Network B)(P,L1)Provides real-time quotes for stocks listed outside of NYSE and NASDAQ. Exchange listings include (but not limited to) those on NYSE American (formerly known as AMEX), ARCA, BATS, and IEX. Examples include IBKR, SPY, and VXX. ? 3/3: If I end up having to subscribe to the fancier one, I assume I'm also allowed to use reqMarketDepth() to subscribe to the data in an alternative format. Is that right? |
|
Re: ib market data. is it clean and any good alterantive sources?
how long have you been using it? appreciate if you can share real experience, like telling me oh i have been extracting them daily, hourly for some years and found the data quality high and without issue when extracting them.
?
when in production, i want something consistent (without format change, data change or delay).?
?
don't know how reliable polygon.io is but i would prefer something with long history |
|
Re: ib market data. is it clean and any good alterantive sources?
What would be timely for you? I found to be a good source, especially as they provide flat file data downloads on a daily basis:? On Sat, 8 Feb 2025 at 07:54, alan chau via <curiosityhk2019=[email protected]> wrote:
|
|
ib market data. is it clean and any good alterantive sources?
i have been using IB API for pacing order only. can anyone tell if the US equities/ETF data from IB clean and up-to-date?
?
Recently i am looking for data source which provide data timely and clean. my focus is US equities/ETF data, can anyone suggest some free (or low cost) alternative for bloomberg terminal? better have python API to extract them.
?
especially for production purpose, they must update and provide the data timely with less frequent format/changes in their raw sources.
?
understand that this question may not be that related to IB API here. but i do think the people here can address my question efficiently. appreciate if anyone can share the experience to me here.
?
thanks! |
|
requesting midpoint data
I'm trying to request midpoint data (for symbols such as SPY, QQQ, etc.) via reqTickByTickData() and says "For some securities, getting tick-by-tick data requires Level 2 data bundles."
?
1/3: Did this change? If so, was that recent? I thought I remembered being able to do this while only subscribing to level 1.
?
2/3: Does that mean I need:
?US Equity and Options Add-On Streaming Bundle (P)Includes streaming real-time quotes for NYSE (CTA/Network A), AMEX (CTA/Network B), NASDAQ (UTP/Network C), and OPRA (US Options). In order to subscribe to US Equity and Options Add-On Streaming Bundle (P), the user must already be subscribed to US Securities Snapshot Bundle (P). ? Can I get rid of what I bought earlier this week? ? NYSE American, BATS, ARCA, IEX, and Regional Exchanges (Network B)(P,L1)Provides real-time quotes for stocks listed outside of NYSE and NASDAQ. Exchange listings include (but not limited to) those on NYSE American (formerly known as AMEX), ARCA, BATS, and IEX. Examples include IBKR, SPY, and VXX. ? 3/3: If I end up having to subscribe to the fancier one, I assume I'm also allowed to use reqMarketDepth() to subscribe to the data in an alternative format. Is that right? |
|
Re: Decimal remaining value retrival
The short answer is no. When you look at the data model, one Order is related to (or filled by) one or more Trades (executions). "filled" and "remaining" are attributes of Orders and not attributes of Trades. As such they are reported ithough orderStatus() callbacks, where they belong. When you manage orders and trades take place, you will receive orderStatus() callbacks right before or after execDetails() callbacks so your client will have that information. 闯ü谤驳别苍 ?
?
On Thu, Feb 6, 2025 at 09:13 AM, <tarun.joshi@...> wrote:
|
|
Decimal remaining value retrival
Hello Folks,
Is is possible to get Decimal remaining value? in the below callback //! [execdetails] void execDetails(int reqId, const Contract& contract, const Execution& execution) { ? printf("ExecDetails. ReqId: %d - %s, %s, %s - %s, %s, %s, %s, %s\n", reqId, contract.symbol.c_str(), ? ? ? ? ?contract.secType.c_str(), contract.currency.c_str(), execution.execId.c_str(), ? ? ? ? ?longMaxString(execution.orderId).c_str(), decimalStringToDisplay(execution.shares).c_str(), ? ? ? ? ?decimalStringToDisplay(execution.cumQty).c_str(), intMaxString(execution.lastLiquidity).c_str()); like we receive in Order Status callback void orderStatus(OrderId orderId, const std::string& status, Decimal filled, Decimal remaining, ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?double avgFillPrice, int permId, int parentId, double lastFillPrice, int clientId, ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?const std::string& whyHeld, double mktCapPrice) { ? printf( ? ? ? "OrderStatus. Id: %ld, Status: %s, Filled: %s, Remaining: %s, AvgFillPrice: %s, PermId: %s, LastFillPrice: %s, " ? ? ? "ClientId: %s, WhyHeld: %s, MktCapPrice: %s\n", ? ? ? orderId, status.c_str(), decimalStringToDisplay(filled).c_str(), decimalStringToDisplay(remaining).c_str(), ? ? ? doubleMaxString(avgFillPrice).c_str(), intMaxString(permId).c_str(), doubleMaxString(lastFillPrice).c_str(), ? ? ? intMaxString(clientId).c_str(), whyHeld.c_str(), doubleMaxString(mktCapPrice).c_str()); I wan't to get? decimalStringToDisplay(remaining).c_str() in ?execdetails callback. Thanks. |
|
Re: QQQ 1 Minute Data >6years?
I can't believe that the date/time format has any impact. I went with the "Date/Time in the instrument's time zone" approach a long time ago. The only major difference between our approaches is the direction of time for a sequence of requests. I don't use historical data download that much, but I don't recall a case where a request would hang forever (in other words no historicalDataEnd() or error() callback). My client is entirely event driven and basically sends the next request upon the receipt of the previous request's historicalDataEnd() callback. Though the framework between the client and TWS API handles collecting bars and returning all bars for a request as a list of bars. 闯ü谤驳别苍 ? ? On Wed, Feb 5, 2025 at 11:03 AM, David Armour wrote:
|
|
Re: QQQ 1 Minute Data >6years?
Hi Jurgen,
?
How does that work when you get no data and no historicalDataEnd() callback? If you have no timeout then your code would just hang wouldn't it?
?
I used to download backwards, but it gave me other problems so I reversed it. I always check once I have received the historicalDataEnd() that I received the expected number of bars. So far, I have never had an error for missing bars. (I have a function to manage holidays and short trading days.)
?
I must say your delays seem long to me, but I have never checked mine other than they are all less than 15secs (based on my start latch timeout). I only ever got a timeout after 15secs with problems like the one I am facing now with QQQ. I have downloaded hundreds of symbols with long histories of 1min and 5min bars so this is not stated without some experience.
?
This is just one of those problems that IB throws at us now and again with data.
?
Going to sleep on it. May try using a different timestamp format tomorrow and see if that has any impact. Cannot really think of anything else to try.
?
? |
|
Re: QQQ 1 Minute Data >6years?
Hi Dale,
?
Perhaps you missed it in the long discussion below, but that is exactly what was done. I am unable to get data before 5th June 2018.
?
Somehow, Jurgen is able to though.
?
I suspect there is some difference in the call to reqHistoricalData() somewhere. For example, I am using the UTC format YYYYMMDD-HH:mm:ss but I believe that Jurgen is using the format like "YYYYMMDD HH:mm:ss US/EASTERN" as he mentioned it below. Not sure why that would make a difference.
?
? |
|
Re: QQQ 1 Minute Data >6years?
Hmm. I use no set timeouts and I do not cancel nor restart requests. But I perform historical data downloads in reverse time order (and I know several other members do that, too). When you make a request you specify the "end date and time" plus a duration, but you may or may not get data for that entire duration. So once you receive data, simply start the next request based upon the earliest time stamp you received. I am wondering whether your delays are caused because you make requests in ascending time order. That might go against the IBKRs ability of prefetching and pipelining the data for you. For a download of 1min data with 1day duration that started at 2018-06-03 and ended at 2018-05-01 (in reverse time order) I get the following delays between request and data arrival (in seconds): 1, 1, 6, 1, 7, 6, 7, 18, 19, 1, 1, 6, 6, 6, 7, 12, 24, 1, 1, 6, 6, 1, 6 That is the kind of pattern I see for longer downloads, too. 闯ü谤驳别苍 ? On Wed, Feb 5, 2025 at 10:16 AM, David Armour wrote:
|
|
Re: QQQ 1 Minute Data >6years?
maybe get 1 day of data, 3 years ago, then 4 years ago...etc. to see where and how the timing starts to change? On Wed, Feb 5, 2025 at 11:26?AM David Armour via <dave.armour=[email protected]> wrote:
|
|
Re: QQQ 1 Minute Data >6years?
I tried setting the wait up to 10mins when you suggested it before. It just timed out after 10mins. No data is being sent and frankly waiting 10mins is ridiculous. The first set of data is always sent in less than 1 second in my experience.
?
Also, I am following the historical data downloading pacing guidelines and 1 day is normally what is acceptable for 1 minute data. In the past I have asked for more only to see my request get slower and slower until it ground to a halt. This was due to soft pacing being enforced by IB. (It may not be the case now, but it certainly was in the past and I found 1 day at? a time to be a sweetspot.)
?
I get all other stock data just fine the way I do it. It is only QQQ that is having an issue.
?
? |
|
Re: QQQ 1 Minute Data >6years?
15 seconds isn't near long enough to wait for a reach that far back. for 2019 data, i believe it'd take several minutes (as in > 4) to start getting any reply messages. I'd also suggest you request more than 1 day at a time and storing the response on? your side since it takes so long? On Wed, Feb 5, 2025 at 11:16?AM David Armour via <dave.armour=[email protected]> wrote:
|
|
Re: QQQ 1 Minute Data >6years?
I get no data. Right now I am running a job starting from 2015-01-01 and working my way forward day by day. Up to now I get nothing returned and I am at 2015-06-01.
My code waits 15secs for the data download to start so it is a little slow when no data is returned. It is a latch using a conditional_variable. ?
Here is a snapshot of the log:
?
?
You can see it making the request, then 15secs later cancelling it and issuing a new request for the following day. |