Re: conditional order conditioned on execution of buy order only?
?
DS,
?
From looking through relevant posts, and doing some experimenting, I can see only two ways to "pause" a limit order via the TWS API:
?
- cancel the order, the submit it again to "resume" the order, or
- modify the limit price to prevent the order from filling, then modify it again to "resume" the order.
?
It looks like the only parameters of a standing limit order that can be changed may be the quantity, the price, and the time in force.
?
In particular, modifying or removing a time condition (whether attached to the order when created, or added later) doesn't seem to be supported. ?Nor does modifying or removing a goodAfterTime.
?
At least, this is the best explanation given what I've read and the results of things I've tried. ?I'd be happy to find out this is not the case.
?
-Neal
|
Are you trying to use Ross Hemminway's library? If so I bought a license and could not make it work for me either. But I run solely on Linux these days and tried to implement it with Lazarus so I never tried it on Delphi. If youre working on implementing your own socket connection with the wire protocol I might be able to get you started.
?
|
I am not a professional programmer. I write code for my own use, which is usually, because of my age, in pascal. That said I obviously needed to understand the wire protocol and was surprised to learn support was almost non-existent. I studied the C++ API and used wire shark to learn the necessary message format to right the code I needed to support the type of trading I do. It works quite well and is fast.
?
The warnings I've seen on the board about getting out of sync and hopelessly unable to recover without a restart are unfounded. Why would anyone suggest that,? when the API itself is simply converting you requests to the wire protocol and decoding the incoming wire protocol messages to a human readable format. The ability to recover is what you make of it in your code.
?
While studying the C++ code I was amazed at how poorly the code is maintained. Its like a Swiss army knife trying to do everything under the sun until it becomes buggy. Look at EClient::placeOrder. Its amazing how many if else statements are being executed based upon what protocol version you might be using. And if you just happen to be using a certain version the bug raises its ugly head. This API should be paired with the TWS workstation release and your code should refuse to connect if you update the workstation and try to connect with client software written with an older version.
?
I wanted to share my knowledge of using the wire protocol directly but Ive seen posts that this isn't the forum to do that, and since I have been unable to find a forum on the net that handles this I'm left with the decision of creating one myself.
?
If there are others who have interest in that please respond. Thanks and I hope that I haven't offended anyone with this post.
?
|
Re: reqContractDetails() does not get a callback if secType = "OPT"
Working fine here on TWS 10.30, including on non-RTH.?
Try not doing this ¡ª I definitely?don't:
optContract.multiplier = "" optContract.conId = 0
On Mon, Jan 13, 2025 at 8:19?AM ´³¨¹°ù²µ±ð²Ô Reinold via <TwsApiOnGroupsIo= [email protected]> wrote: Worked for me just now, so your problem is not related to trading hours. I received 2,404 contracts within 10 to 12 seconds.
Maybe you need to check your setup or get a "second opinion" by using a different client. Richard's screen shot was from his very handy "Contract Inspector" tool that you can . Maybe start with that.
´³¨¹°ù²µ±ð²Ô
?
On Mon, Jan 13, 2025 at 12:18 AM, Stephan Gueorguiev wrote:
OK, I think the issue is trying this outside of trading hours like on a Sunday - this weekday morning it seems to work as expected.? Weird as I'd have thought looking up option contracts should not need live market data but it appears that's not the case.? It just so happened I tried the smaller chain first this AM which is why it worked rather than the chain size.
-- Daniel
|
Re: reqContractDetails() does not get a callback if secType = "OPT"
Worked for me just now, so your problem is not related to trading hours. I received 2,404 contracts within 10 to 12 seconds.
Maybe you need to check your setup or get a "second opinion" by using a different client. Richard's screen shot was from his very handy "Contract Inspector" tool that you can . Maybe start with that.
´³¨¹°ù²µ±ð²Ô
?
On Mon, Jan 13, 2025 at 12:18 AM, Stephan Gueorguiev wrote:
OK, I think the issue is trying this outside of trading hours like on a Sunday - this weekday morning it seems to work as expected.? Weird as I'd have thought looking up option contracts should not need live market data but it appears that's not the case.? It just so happened I tried the smaller chain first this AM which is why it worked rather than the chain size.
|
Re: reqContractDetails() does not get a callback if secType = "OPT"
OK, I think the issue is trying this outside of trading hours like on a Sunday - this weekday morning it seems to work as expected.? Weird as I'd have thought looking up option contracts should not need live market data but it appears that's not the case.? It just so happened I tried the smaller chain first this AM which is why it worked rather than the chain size.
|
Re: reqContractDetails() does not get a callback if secType = "OPT"
Thanks Richard, appreciate you looking.
From your screenshot, it doesn't look like this is running via the gateway, or is it?
?
The API messages for the first of my calls are:
?
Whereas for the second call they are (and then it hangs):
?
?
?
So it's definitely not getting a call back rather than something in my code.
?
Actually, I've just tried another ticker given the v. large option chain on AAPL you report.? If the ticker is a smaller chain, say LPRO, then it does work as expected.? Surely this is a bug though?? Just hanging without a call back for 60s if the chain is too big?? Should I be reporting it or is it considered normal operation?
?
?
|
Re: reqContractDetails() does not get a callback if secType = "OPT"
Your request for the AAPL options actually returns 2482 contract details. The whole lot is returned within a second or two. You should be able to easily verify this from the API log. ? See attached screenshot. ? I can¡¯t speculate as to why your code doesn¡¯t receive them, but the fault is certainly not in the IBKR API code. ? Richard
|
Re: reqHistoricalTicks() & batching vs. reqTickByTick() vs. reqHistoricalData(), Backtesting vs. Realtime execution
Yes agreed it does appear that the sequence they arrive is correct as long as it's maintained. Good idea tagging them with a serial number.... I'm gonna add that for good measure.
?
?
-----
?
Side note (just because I'm looking at it) in case helpful for anyone, here's a chart of NQ with the prices from each source. Similar but not identical, where bento seems to have higher price resolution and ibkr sorta averages ticks.
?
|
Re: reqHistoricalTicks() & batching vs. reqTickByTick() vs. reqHistoricalData(), Backtesting vs. Realtime execution
Just a note that the price I mentioned was for trade ticks only. The bid/ask and book ticks are more.
|
reqContractDetails() does not get a callback if secType = "OPT"
I had some old code I haven't run for a few years, which used to successfully retrieve an option chain.? I've been trying to update it but for the life of me I can't get hold of an option chain through reqContractDetails() with code which used to work.
?
I've updated to IB Gateway 10.33 and also a python client implementing API 10.30.01.
?
I can call reqContractDetails
?
contract = Contract()
contract.symbol = "AAPL"
contract.secType = "STK"
contract.currency = "USD"
contract.exchange = "SMART"
reqContractDetails(1, contract)
?
This works perfectly and I retrieve the details just fine.
?
If however I then do:
optContract = Contract()
optContract.symbol = "AAPL"
optContract.currency = "USD"
optContract.exchange = "SMART"
optContract.secType = "OPT"
optContract.multiplier = ""
optContract.conId = 0
reqContractDetails(2, optContract)
?
Then the contractDetails() callback never gets called, or at least within my 60 second timeout.? I have tried with "" for exchange and multiplier - no dice.
?
I can see how if I'm not properly defining a request for the option then it won't find it - but then I'd expect a "security not found" type message - whereas this is more like as soon as the contract type is set to OPT the callback never takes place.
?
Any ideas?? Is it possible that I'm doing this on a Sunday and it will work during regular trading hours?
?
Thanks in advance,
S.?
|
Re: reqHistoricalTicks() & batching vs. reqTickByTick() vs. reqHistoricalData(), Backtesting vs. Realtime execution
My apologies I did not see your quoted text. This is right from the documentation:
Pacing Violations for Small Bars (30 secs or less)
Although Interactive Brokers offers our clients high quality market data, IB is not a specialised market data provider and as such it is forced to put in place restrictions to limit traffic which is not directly associated to trading. A Pacing Violation occurs whenever one or more of the following restrictions is not observed:
- Making identical historical data requests within 15 seconds.
- Making six or more historical data requests for the same Contract, Exchange and Tick Type within two seconds.
- Making more than 60 requests within any ten minute period.
- Note that when BID_ASK historical data is requested, each request is counted twice. In a nutshell, the information above can simply be put as ¡°do not request too much data too quick¡±
Important: these limitations apply to all our clients and it is not possible to overcome them. If your trading strategy¡¯s market data requirements are not met by our market data services please consider contacting a specialized provider.
?
This is what I was referring too, not exactly saying it's unreliable, but they say they're a broker not a data provider.
?
?
?
?
|
Re: reqHistoricalTicks() & batching vs. reqTickByTick() vs. reqHistoricalData(), Backtesting vs. Realtime execution
I was in talks with Portara CQG in November of 2024 about purchasing a large amount of tick data for /NQ and /ES. They may not have live tick feeds, as I am looking over their website again right now if that's what you are asking about. Reopened the thread with them though to see if they have some sort of software solution there. They seemed to work to your needs and were quite accommodating in our last discussion.
|
Re: reqHistoricalTicks() & batching vs. reqTickByTick() vs. reqHistoricalData(), Backtesting vs. Realtime execution
Where have you seen that?
?
On Sun, Jan 12, 2025 at 01:10 PM, Brendan Lydon wrote:
.... I mean IBKR states that they're data is not reliable and they do recommend other data providers for tick data ...
?
?
|
Re: reqHistoricalTicks() & batching vs. reqTickByTick() vs. reqHistoricalData(), Backtesting vs. Realtime execution
I am not disputing your results, but you are now comparing reqMktData() tick #48 "RtVolume" with the tick-by-tick feed from Databento.? But our discussion here is about reqTickByTick() which is entirely different and independent from reqMktData( RtVolume ). Have you compared IBKR TickByTick data (historical or realtime) with Databento tick feeds?
The few times I had access to high-res tick-by-tick feed samples from trustworthy sources, IBKR reqTickByTick(LAST) showed the same price/size ticks in the same order as the comparable feeds. However, these were spot checks and only for CME/CBOT futures so I cannot guarantee that IBKR tickByTick feeds for all symbols and all exchanges are complete.
The 1-second resolution for time stamps in historical and real-time tickByTickData does not pose an issue for many applications since the order of tick arrival seems to be consistent and correct within the second:
- If you record live feeds, you can tag each data arrival with a serial number and a high resolution arrival time stamp (as I do) so that you remember not only the temporal order of ticks for each symbol but also the order across all symbols you record. My "session tape" contains all available reqMktData() ticks and 5 second reqRealTimeBars() for many instruments, reqTickByTick(Lastt and BidAsk) for some instruments, and L2 market depth for few instruments. That "tape" had 404 million ticks last week but there are weeks with much more than that. Depending on the task at hand, the arrival tags allow me to replay the entire tape or filtered sub-streams at "real time", much faster, or in "slomo".
- If you are only interested in TickByTick trades, filtered reqHistoricalTicks can be used as an equivalent replacement for reqTickByTick live data.
- But you will not be able to combine different reqHistoricalTick downloads into a consistent feed at the sub-second level (say Trades and BidAsk for a single instrument, or Trades from multiple instruments). But you would probably need a nanosecond resolution and precision time stamp to be able to do that accurately. Milliseconds won't do it, but that moves you into an entirely different class of specialized data providers. Keep in mind that IBKR is a brokerage, some exchanges do not event provide that resolution, few data servicers do, and you mush hope that they all have perfectly synchronized clocks.
´³¨¹°ù²µ±ð²Ô
?
?
?
On Sun, Jan 12, 2025 at 12:08 PM, wordd wrote:
Yea in realtime I'm just getting the ticks from ibkr via reqTickByTick. Databento does provide live data but I have not subscribed to it.
?
Apologies, it is relevant that the data is not consistent between the two providers... and that data bento has more trades. I haven't sorted out why this is aside from reading in a few places that ibkr shouldn't really be trusted to reliably provide tick data.?
?
Here's an example, these are tick_type=48 from ibkr vs trade ticks from dbento.
?
| dt | ticks_ibkr | ticks_ibkr_size | ticks_dbento | ticks_dbento_size |
|------------|------------|----------------|--------------|------------------|
| 2024-12-22 | 12744 | 30344 | 23458 | 30344 |
| 2024-12-23 | 136348 | 486266 | 340921 | 486486 |
| 2024-12-25 | 6701 | 16119 | 11993 | 16119 |
Note that the reported sum of volume (size columns) are the same, but the count of rows differs.?
?
Couple notes:
- for the actual contract traded under continuous futures, ibkr chooses the front running contract with a fuzzy transition period by highest volume during the transition period. I have not found a way to find with certainty which contract they used exactly on which date.. so I've had to make a best guess by trade volume
- while the prices are very close, they are not identical and it appears that ibkr seems to average ticks a little bit (not completely sure here)
- Since I'm using these ticks for order execution simulation I figured that higher resolution would be better regardless of what I get in realtime from ibkr -- this may not be the same for you since you're using the ticks for predictions
?
The tick data for NQ from databento was about $30 for the last 3 months, I suggest checking it out and comparing. Hope that helps.
|
Re: reqHistoricalTicks() & batching vs. reqTickByTick() vs. reqHistoricalData(), Backtesting vs. Realtime execution
Okay thank you for clarifying. $30 for 3 months is not bad at all. I checked out Portara CQG a couple months back for /NQ & /ES and I believe it was $895 per instrument going back to 2014 for level 1 tick data, that is matched order info & what NBBO was @ time of execution. That is also obviously terabytes of data, yet I still might go with them, but then I still run into the real-time problem although I think they may have real-time tick streams. Continuing discussion with them. I mean IBKR states that they're data is not reliable and they do recommend other data providers for tick data. Might just have to use them for what they are, a broker, and just submit my orders through them. Definitely means more work though to pull all these things together in a live, real-time system. Hesitant to hack away until I have a solid, actionable plan with this tick data problem though.
?
Thank you,
Brendan
|
Re: reqHistoricalTicks() & batching vs. reqTickByTick() vs. reqHistoricalData(), Backtesting vs. Realtime execution
Shoot i was hoping markdown formatted tables worked. :/
|
Re: reqHistoricalTicks() & batching vs. reqTickByTick() vs. reqHistoricalData(), Backtesting vs. Realtime execution
Yea in realtime I'm just getting the ticks from ibkr via reqTickByTick. Databento does provide live data but I have not subscribed to it.
?
Apologies, it is relevant that the data is not consistent between the two providers... and that data bento has more trades. I haven't sorted out why this is aside from reading in a few places that ibkr shouldn't really be trusted to reliably provide tick data.?
?
Here's an example, these are tick_type=48 from ibkr vs trade ticks from dbento.
?
| dt | ticks_ibkr | ticks_ibkr_size | ticks_dbento | ticks_dbento_size |
|------------|------------|----------------|--------------|------------------|
| 2024-12-22 | 12744 | 30344 | 23458 | 30344 |
| 2024-12-23 | 136348 | 486266 | 340921 | 486486 |
| 2024-12-25 | 6701 | 16119 | 11993 | 16119 |
Note that the reported sum of volume (size columns) are the same, but the count of rows differs.?
?
Couple notes:
- for the actual contract traded under continuous futures, ibkr chooses the front running contract with a fuzzy transition period by highest volume during the transition period. I have not found a way to find with certainty which contract they used exactly on which date.. so I've had to make a best guess by trade volume
- while the prices are very close, they are not identical and it appears that ibkr seems to average ticks a little bit (not completely sure here)
- Since I'm using these ticks for order execution simulation I figured that higher resolution would be better regardless of what I get in realtime from ibkr -- this may not be the same for you since you're using the ticks for predictions
?
The tick data for NQ from databento was about $30 for the last 3 months, I suggest checking it out and comparing. Hope that helps.
|
Re: reqHistoricalTicks() & batching vs. reqTickByTick() vs. reqHistoricalData(), Backtesting vs. Realtime execution
So, in real-time though, assuming your backtest went well and you are now running in realtime, you are using reqTickByTick()? Or are you using a live datastream from databento, if it's even offered, and just submitted orders through IBKR?
|
Re: reqHistoricalTicks() & batching vs. reqTickByTick() vs. reqHistoricalData(), Backtesting vs. Realtime execution
I also found the same issue as you where the historical ticks are rounded to the 1 second, and after much searching couldn't find any way to get higher resolution from ibkr.
?
I wasn't sure that I could trust that the order of the ticks would be the same order that they would have come in via the live tickbytick method.. so I just went and bought historical ticks from databento for the backtest time range.
?
It's a bit of a pain to use another data source, map it all, etc.. but at least now I'm confident that the data is accurate. Good luck.
|
wordd