new primary location of tws api documentation
today i received an email from ib with regard to my recent contribution to tws api documentation. here's a quote from the email related to the new documentation location:
It's also critical to note that our hosted documentation has changed locations. We are now listed under??for our primary TWS API documentation. This change took place in late October, so the shift was relatively recent. If you have any questions or concerns on this topic, please let us know.
i have completely missed this change and i'm probably not the only one so i thought it might be good to share the info.
|
Direct orders to TGATE are routed SMART, status stay dark blue
Good evening,
since years I am sending orders via API (Excel) direct with PrimExchange = TGATE and Exchange = TGATE (no smart routing). I never faced any problems. Since yesterday I see these orders int the TWS application with a " dark blue" status (instead of green) and although I used direct routing they are not sendet to Tradegate, but have a "smart" label. I have aboslutely no idea what causes this behaviour. Did IB change something in the smart routing? Is it a bug in there system?
Any help or ideas are aprpeciated.
Regards,
Daniel
|
Re: No or delayed response on placed orders using IB Python API TWS 10.19 on Linux EC2
Possibly facing disconnection/reconnection? Explore a finite state machine to detect lost orders, resend when logic conditions are met.
toggle quoted message
Show quoted text
El 6 dic. 2023, a la(s) 13:54, yonatan@... escribi¨®:
?Hello,
I've been using the TWS Python API for almost two years for automatically trading futures. I run my code on an amazon Linux EC2 machine, Python 3.9.5 , ibapi version 9.81.1.post1, TWS version 10.19.
Over the past few weeks I've noticed that some of my orders have been getting "lost". By lost I mean no response in the orderStatus() callback. Normally I get a response almost immediately (my system checks every 2 seconds), sometimes it takes 8-10 seconds, sometimes up to 3-4 minutes, and on one occasion I didn't get a response at all. I checked by the TWS logs, the API logs, and my own system logs, and nothing.
Anyone have any idea what might cause this???
Thank you,
Yonatan
|
Re: Python Datetime Processing (US/Eastern)
Hi there,
I came across a very similar problem myself and found the following discussion that solved my problem. Hope it helps.
/g/twsapi/topic/adding_timezone_to_request/93271054
|
No or delayed response on placed orders using IB Python API TWS 10.19 on Linux EC2
Hello,
I've been using the TWS Python API for almost two years for automatically trading futures. I run my code on an amazon Linux EC2 machine, Python 3.9.5 , ibapi version 9.81.1.post1, TWS version 10.19.
Over the past few weeks I've noticed that some of my orders have been getting "lost". By lost I mean no response in the orderStatus() callback. Normally I get a response almost immediately (my system checks every 2 seconds), sometimes it takes 8-10 seconds, sometimes up to 3-4 minutes, and on one occasion I didn't get a response at all. I checked by the TWS logs, the API logs, and my own system logs, and nothing.
Anyone have any idea what might cause this???
Thank you,
Yonatan
|
Re: Anyone have any success modifying the trigger method?
with regard to oca groups, i recall from my experience several years back (when trading options) that when overfill protection is used, only one order is sent to the exchange (it's also clearly visible in tws looking at the color mark at the orders). those orders were plain stop and limit orders. if overfill protection is turned off, both (all) orders in oca group are sent to the exchange. this makes sense to me as to stay on the safe side, they simply submit just one order and monitor the state of the other order(s) to submit it to the exchange once it's fillable.
with regard to the quality of data provided, i never did comparison of tick data vs market data, but i recall an issue in tws when watching forex 5 sec (and similar) realtime charts... i don't know how they build the bars, but after refreshing the chart, some of the bars changed, so it looked like the bars are being constructed in tws from sampled data. simply said, those charts were not very reliable.
|
Re: Anyone have any success modifying the trigger method?
I had started on replying a few times but was preempted by something claiming higher priority each time. I'll have to look into this some more, but here at least a quick response.
I wrote my reply "from memory" and thought I had seen a note about OCA types "with block" causing orders to become simulated (especially type 1 "Cancel all remaining orders with block."). But I cannot find that reference any longer. I checked the logs and the framework code (that I had not visited in years) and found this:
Again, I had never looked into this in any detail, but I am not interested to do so.
I used the term "aggregated" for reqMktData since that is the characterization IBKR uses in the documentation as you pointed out. While sizes and volumes are accumulated, "sampled" might indeed be a more accurate term.
And even sizes seem to be somewhat sampled according to a quick measurement I made on ESZ3 trades Monday 08:30 to 11:00 (US/Central). That period was not extremely busy (we have seen much busier times) but the sum of all LastSize values is 27% smaller than the sum of all TickByTick size fields for the same time period. This assumes, obviously, that the TickByTick values are "more correct".
But as you said, reqMktData will be perfect for many trading systems. The TickByTick higher resolution comes with a price and may not give you more insight. In fact, many of the LastPrice and LastSize values arrived at about the same time of the corresponding TickByTick events, occasionally even slightly earlier. "Sampled" does not mean slower, just less often.
´³¨¹°ù²µ±ð²Ô
On Mon, Dec 4, 2023 at 01:23 PM, Richard L King wrote:
´³¨¹°ù²µ±ð²Ô
?
Please can you explain what you mean by ¡®You should be able to force any order type to be simulated by IBKR when you place the order into a One Cancels All group¡¯.
?
I don¡¯t believe placing an order in an OCA group affects the order in any way. The order is submitted to the exchange or managed by IBKR in exactly the same way as if it were not part of an OCA group. What would it even mean to simulate a Limit order? If IBKR were monitoring price to see when the limit price is hit, and only then actually submitted the limit order, the order would not be filled at least some of the time, because that delay could mean the markets may have moved by the time the limit order arrives at the exchange.
?
Perhaps I¡¯m misunderstanding what you might mean?
?
I also disagree with the statement that reqMktData data is ¡®aggregated¡¯. I know the documentation does say this somewhere, but I¡¯ve never been able to assign any meaning to it. What I would say is that the data is sampled, not aggregated ¨C and they are very different things.
?
To me, aggregation would have to mean something like creating a VWAP for prices/sizes during the aggregation interval, and I¡¯ve never seen the slightest evidence of that.
?
Sampling means that each price and size reported are genuine, but you just don¡¯t get all of them. In particular, you can miss important prices that turn out to be the high or low of whatever bar size you¡¯re constructing. ?I¡¯ve described my understanding of IBKR¡¯s data sampling algorithm in the past, and I¡¯ve never seen any reason to revise it (I¡¯ll have to see if I can find the relevant posts again in case anyone is interested).
?
?
Richard
?
?
?
Looks like your triggers should work since, according to the IBKR documents, the order types you use are no native exchange order types. Maybe the next step would be to reach out to IBKR for clarification.
What kind of tick data did you use for your analysis? ? data is aggregated and can have a delay relative to your order executions. Or did you use that should be much closer to the order fill events.
And one last thought if you are up for another test. You should be able to force any order type to be simulated by IBKR when you place the order into a group. I am not sure whether a group with just one order would work, but you could add a dummy order (maybe with a time condition so that it can never execute) into a group with the order you are really interested in. Just for testing purpose to rule out that the order types are actually native (and that the documentation is incorrect).
´³¨¹°ù²µ±ð²Ô
|
Python Datetime Processing (US/Eastern)
Hi,
Does anyone know the directive for converting a string into a datetime in python?
e.g.
DateOne = "20231205 08:30:00 US/Eastern"
datetime.strptime(DateOne,"%Y%m%d %H:%M:%S %???")
where %??? is the unknown directive. Without this directive I receive this message: "ValueError: unconverted data remains: :00 US/Eastern" and I'm not finding great documentation online. I know I could switch to an older TWS API version where EST might be implied but I would rather not do that. I could also see using a parser but that seems like it should be an unnecessary step.
Thanks
|
Looks like nobody have a straight answer. So I share my opinion:
I don't see any part of the API aimed at delivering any data close to analysis, or indicators, information, (any outcome of a massage of data in general)
I guess you have to implement your own,you may get slightly different results than IB , my limited experience with IB make me wonder that IB may have access to higher precision data than what they deliver.
You most assuredly find this page ? and this one for tick 104,106, etc ...
|
Re: Anyone have any success modifying the trigger method?
Yes, it¡¯s not completely accurate, but you have to decide whether it¡¯s accurate enough for your purposes. If you compare a chart generated with this data against one using the full data, you¡¯ll be hard pressed to find many significant differences. You might find some bars whose high or low is one, or sometimes two ticks out. Note, that volume figures are pretty accurate because it is absolute volume that is reported (though there are frequent questions about why IBKR¡¯s volume figures differ from Yahoo or whatever, and there is no easy answer to that). ? [By the way, this reality was one of the reasons why IBKR introduced realtime bars: it was (at least partly) an attempt to enable users to correct their collected data, or to use only the realtime bars to build their own bars: I never found it to be very useful because of the latency, and the fact that IBKR¡¯s server clocks were (and still seem to be) not well synchronised ¨C for example there is currently a difference of one second between my live and paper-trading TWSs, and neither of them are the same as my computers which are well-synchronised with an internet time server.] ? The inaccuracy also varies greatly from contract to contract. A contract that, on average, only has a couple of trades per second is likely to be very accurate. Whereas something like the ES futures, which is very heavily traded, may be less so: I say ¡®may be¡¯, because although the frequency of trades is much higher, so are the bid and ask sizes, so arguably the prices don¡¯t necessarily move any more quickly, because it takes more trades to exhaust the bids or asks at each price level. ? And bear in mind that the inaccuracy only really matters where price has moved up and then back down again (or down and then up) during a sampling interval, so that one or more prices are missed, and such a price just happens to be the high or low of the interval that¡¯s of interest to your trading strategy. ? I¡¯ve never found this to be a real problem. Of course it depends very much on the trading strategies you use: if time-based bars are not relevant, then occasional minor inaccuracies in highs and lows probably don¡¯t matter. But if your strategy depends critically on tick volume (ie number of trades at each price) then this data is possibly wholly inappropriate, since you may be missing many trades ¨C and probably missing more at the more heavily traded prices. ? The bottom line is that if you need ultimate accuracy, then you have no option but to use tick-by-tick data, if you can live with its limited number of streams (and your code is sufficiently performant not to get bogged down in very busy periods). ? My advice regarding use of tick-by-tick data rather than normal market data would be similar to the well-known rules of premature optimisation: ? Rule 1: don¡¯t do it! If you do, you¡¯ll probably make a lot of effort for very little benefit. ? Rule 2 : don¡¯t do it yet! Prove that you have a real need, and have a way of measuring that the outcome is what you hoped for. ? Richard ? ? ?
toggle quoted message
Show quoted text
From: [email protected] <[email protected]> On Behalf Of kevinjiahongliang@...
Sent: Tuesday, December 5, 2023 2:27 AM
To: [email protected]
Subject: Re: [TWS API] Anyone have any success modifying the trigger method?? So If I'm understanding right, the tick data from??is inaccurate, and I should be using? ? Kevin
|
Re: Anyone have any success modifying the trigger method?
So If I'm understanding right, the tick data from??is inaccurate, and I should be using?
?
Kevin
|
Re: Anyone have any success modifying the trigger method?
Thanks for the link, i'll take a look.
|
Re: Anyone have any success modifying the trigger method?
My post describing my understanding of IBKR¡¯s market data sampling mechanism is here: ? /g/twsapi/message/8821 ? This post is very old, but I¡¯ve not seen anything that leads me to believe it is significantly inaccurate (though the sampling intervals are shorter these days, especially for Forex). ? Note that of course this only relates the market data returned from reqMktData. Tick-by-tick data is a completely different matter. ? As far as the subject of this current thread is concerned, the fact that prices during a sampling period are not all sent means that you can¡¯t reliably use data from reqMktData to ascertain whether the triggers are working correctly. Tick-by-tick data would be a much better bet. ? Richard ? ?
toggle quoted message
Show quoted text
From: [email protected] <[email protected]> On Behalf Of Richard L King
Sent: Monday, December 4, 2023 7:24 PM
To: [email protected]
Subject: Re: [TWS API] Anyone have any success modifying the trigger method?? ´³¨¹°ù²µ±ð²Ô ? Please can you explain what you mean by ¡®You should be able to force any order type to be simulated by IBKR when you place the order into a One Cancels All group¡¯. ? I don¡¯t believe placing an order in an OCA group affects the order in any way. The order is submitted to the exchange or managed by IBKR in exactly the same way as if it were not part of an OCA group. What would it even mean to simulate a Limit order? If IBKR were monitoring price to see when the limit price is hit, and only then actually submitted the limit order, the order would not be filled at least some of the time, because that delay could mean the markets may have moved by the time the limit order arrives at the exchange. ? Perhaps I¡¯m misunderstanding what you might mean? ? I also disagree with the statement that reqMktData data is ¡®aggregated¡¯. I know the documentation does say this somewhere, but I¡¯ve never been able to assign any meaning to it. What I would say is that the data is sampled, not aggregated ¨C and they are very different things. ? To me, aggregation would have to mean something like creating a VWAP for prices/sizes during the aggregation interval, and I¡¯ve never seen the slightest evidence of that. ? Sampling means that each price and size reported are genuine, but you just don¡¯t get all of them. In particular, you can miss important prices that turn out to be the high or low of whatever bar size you¡¯re constructing. ?I¡¯ve described my understanding of IBKR¡¯s data sampling algorithm in the past, and I¡¯ve never seen any reason to revise it (I¡¯ll have to see if I can find the relevant posts again in case anyone is interested). ? ? Richard ? ? ? Looks like your triggers should work since, according to the IBKR documents, the order types you use are no native exchange order types. Maybe the next step would be to reach out to IBKR for clarification. What kind of tick data did you use for your analysis? ? data is aggregated and can have a delay relative to your order executions. Or did you use that should be much closer to the order fill events. And one last thought if you are up for another test. You should be able to force any order type to be simulated by IBKR when you place the order into a group. I am not sure whether a group with just one order would work, but you could add a dummy order (maybe with a time condition so that it can never execute) into a group with the order you are really interested in. Just for testing purpose to rule out that the order types are actually native (and that the documentation is incorrect). ´³¨¹°ù²µ±ð²Ô
|
Re: Anyone have any success modifying the trigger method?
Cool, I will contact IBKR and see if I can get some clarification.
I'm using reqMktData.
|
Re: Anyone have any success modifying the trigger method?
´³¨¹°ù²µ±ð²Ô ? Please can you explain what you mean by ¡®You should be able to force any order type to be simulated by IBKR when you place the order into a One Cancels All group¡¯. ? I don¡¯t believe placing an order in an OCA group affects the order in any way. The order is submitted to the exchange or managed by IBKR in exactly the same way as if it were not part of an OCA group. What would it even mean to simulate a Limit order? If IBKR were monitoring price to see when the limit price is hit, and only then actually submitted the limit order, the order would not be filled at least some of the time, because that delay could mean the markets may have moved by the time the limit order arrives at the exchange. ? Perhaps I¡¯m misunderstanding what you might mean? ? I also disagree with the statement that reqMktData data is ¡®aggregated¡¯. I know the documentation does say this somewhere, but I¡¯ve never been able to assign any meaning to it. What I would say is that the data is sampled, not aggregated ¨C and they are very different things. ? To me, aggregation would have to mean something like creating a VWAP for prices/sizes during the aggregation interval, and I¡¯ve never seen the slightest evidence of that. ? Sampling means that each price and size reported are genuine, but you just don¡¯t get all of them. In particular, you can miss important prices that turn out to be the high or low of whatever bar size you¡¯re constructing. ?I¡¯ve described my understanding of IBKR¡¯s data sampling algorithm in the past, and I¡¯ve never seen any reason to revise it (I¡¯ll have to see if I can find the relevant posts again in case anyone is interested). ? ? Richard ? ?
toggle quoted message
Show quoted text
From: [email protected] <[email protected]> On Behalf Of ´³¨¹°ù²µ±ð²Ô Reinold via groups.io
Sent: Monday, December 4, 2023 5:55 PM
To: [email protected]
Subject: Re: [TWS API] Anyone have any success modifying the trigger method?? Looks like your triggers should work since, according to the IBKR documents, the order types you use are no native exchange order types. Maybe the next step would be to reach out to IBKR for clarification. What kind of tick data did you use for your analysis? ? data is aggregated and can have a delay relative to your order executions. Or did you use that should be much closer to the order fill events. And one last thought if you are up for another test. You should be able to force any order type to be simulated by IBKR when you place the order into a group. I am not sure whether a group with just one order would work, but you could add a dummy order (maybe with a time condition so that it can never execute) into a group with the order you are really interested in. Just for testing purpose to rule out that the order types are actually native (and that the documentation is incorrect). ´³¨¹°ù²µ±ð²Ô
|
Re: Anyone have any success modifying the trigger method?
Looks like your triggers should work since, according to the IBKR documents, the order types you use are no native exchange order types. Maybe the next step would be to reach out to IBKR for clarification.
What kind of tick data did you use for your analysis? ? data is aggregated and can have a delay relative to your order executions. Or did you use that should be much closer to the order fill events.
And one last thought if you are up for another test. You should be able to force any order type to be simulated by IBKR when you place the order into a group. I am not sure whether a group with just one order would work, but you could add a dummy order (maybe with a time condition so that it can never execute) into a group with the order you are really interested in. Just for testing purpose to rule out that the order types are actually native (and that the documentation is incorrect).
´³¨¹°ù²µ±ð²Ô
On Mon, Dec 4, 2023 at 12:55 AM, @Kevin111 wrote:
Hello ´³¨¹°ù²µ±ð²Ô,
Thanks for the reply. What I mean by "they all default to what appears to be bid/ask for buy/sell", this is based off on logging the tick data of the price, and comparing when the order was executed. The Order object after I place the order confirms that the trigger method is the one I had selected, as well as the info displayed on the trader workstation terminal.
Based on the documentation for??it seems the trigger method should be working as I am dealing with Stop Limits?andTrailing stop limits,?which are both simulated on by IB.
Some more relevant info:
I have confirmed that the Order object reflects the trigger method I chose, as well as the information provided to me from the Trader Workstation Client.
I have tested this on both paper and real accounts.
I came to this conclusion based on logging tick data sent from the API and comparing it to when my order gets executed.
?
Appreciate your help on this.
|
I track my own open positions, so checking the max orderId before the session start seems to work just fine.
|
Re: Anyone have any success modifying the trigger method?
Hello ´³¨¹°ù²µ±ð²Ô,
Thanks for the reply. What I mean by "they all default to what appears to be bid/ask for buy/sell", this is based off on logging the tick data of the price, and comparing when the order was executed. The Order object after I place the order confirms that the trigger method is the one I had selected, as well as the info displayed on the trader workstation terminal.
Based on the documentation for??it seems the trigger method should be working as I am dealing with Stop Limits?andTrailing stop limits,?which are both simulated on by IB.
Some more relevant info:
I have confirmed that the Order object reflects the trigger method I chose, as well as the information provided to me from the Trader Workstation Client.
I have tested this on both paper and real accounts.
I came to this conclusion based on logging tick data sent from the API and comparing it to when my order gets executed.
?
Appreciate your help on this.
|
Re: Anyone have any success modifying the trigger method?
Couple quick questions for clarification.
When you say "they all default to what appears to be bid/ask for buy/sell", does that mean the Order object you receive in the first callback after you place the order has a trigger method that is different from the one you had selected, or does that mean the actual triggers you experience look like they take place at ask/bid trigger points?
Also, have you verified that, for the instrument and exchange you are trading, the stop orders are indeed simulated by IBKR? The very last line in the documentation you point to says that trigger methods are ignored for stop-variant orders that execute natively at an exchange. You can determine whether your stop order is simulated through the "" documentation.
´³¨¹°ù²µ±ð²Ô
On Sun, Dec 3, 2023 at 05:57 PM, @Kevin111 wrote:
Hey all,
Been trying to modify the trigger method (),
however it doesn't seem to be working. No matter which value I set, they all default to what appears to be bid/ask for buy/sell.
Appreciate your help on this.
|
Anyone have any success modifying the trigger method?
Hey all,
Been trying to modify the trigger method (),
however it doesn't seem to be working. No matter which value I set, they all default to what appears to be bid/ask for buy/sell.
Appreciate your help on this.
|
