|
Re: No definition for SPX option using LocalSymbol
For SPX OPT you have to set up `Contract.TradingClass` as `SPX` for monthlies?and `SPXW` for daily/weekly?expiration. Do not define `LocalSymbol`. For naked SPX options, just do `Symbol`, `SecType`, `Exchange`, `Currency`, `TradingClass`, `Strike`, `Right` and `LastTradeDateOrContractMonth`. Good luck, Daniel. On Thu, Dec 19, 2024 at 6:16?AM Andy Sanders via <arteinvolo=[email protected]> wrote:
--
Daniel |
|
No definition for SPX option using LocalSymbol
Could somebody advise why I'm getting "No security definition" error for a contract with this local symbol : C SPX 20241218 5945 W
?
Security: SPX index
Strike: 5945
Side: Call?
Expiration: 2024-12-18?
Type: Weekly
?
Trying to send order to buy and sell SPX index options. When I set separate contract properties such as strike, symbol, expiration, currency, and exchange, place order request returns error "ambiguous definition" because API returns the same expiration date for monthly and weekly options.?
?
?
There are several ways to make contract definition for option unique, e.g. using TradingClass or LocalSymbol properties. When I try to use LocalSymbol I get error "No security definition found"
?
Example of local symbol that I used : C SPX 20241218 5945 W
Format mentioned in the docs : ?
?
|
|
Re: reqMktData callbacks are missing data with gateway 10.30 and 10.31 sporadically, works fine with 10.19 and older gateways/TWS
I created a second version where I removed all code which is not absolutely necessary It is roughly 150 line of code which one easily sees comparing Program.py from samples to Program_get_bid.py. As per first post: The file is part of \samples\Python\Testbed?extract provided python.zip in that folder Then, run (or even better run from debugger and step)
Program_get_bid.py -f symbols.csv -m REALTIME --data-lines 75 --loops 5 --port 4001 ?
There is 3 new functions left. Main entry , runs the test in the loop
?
Actual code subscribing to data. making sure the datalines limit is not exceeded. And knowing that Subscription is expensive I typically run 65% of line limit. I.e. for test accounts where I have only 100 lines I would run 65 max. So we never run into "max data lines limit reached" error.?
?
monitor_not_done is called after each subscription and stops new subscribtions once we reach the limit or (finalize = True) simply waits until all subscriptions returned the data or Timed out.
?
The wrappers are easier described by comparison to Program.py the original test app. Really all wrappers do is saving incoming data
?
and there is unsubscribing bit which is driven by wrappers (here on example of error), if all data is present the cancel is called and sets "sentCancelMktData" which gives signal to test loop that one received all the data
?
That is pretty much it.?
?
|
|
Re: reqMktData callbacks are missing data with gateway 10.30 and 10.31 sporadically, works fine with 10.19 and older gateways/TWS
I have implemented reqContractDetails and start constructing contract using conId I received.
so instead of this Like this ?
It made no difference, unfortunately. still not receiving some data
|
|
Re: reqMktData callbacks are missing data with gateway 10.30 and 10.31 sporadically, works fine with 10.19 and older gateways/TWS
On Wed, Dec 18, 2024 at 12:08 AM, ajn wrote:
This would be greatly appreciated. If you can make the test case as small and simple as humanly possible... this will help people (esp. IBKR internal developers, who need extra help because they're mostly inept) to id and possibly fix the problem. |
|
Re: Entry that triggers OCO with TP and SL
On Tue, Dec 17, 2024 at 09:32 PM, wordd wrote:
Just fyi, you can use for code... wrap it in single backticks for code co-mingled within a line of text, and triple backticks for blocks of code. Attachments should work too (with the obvious side-effect of being separated from the message). |
|
Re: reqMktData callbacks are missing data with gateway 10.30 and 10.31 sporadically, works fine with 10.19 and older gateways/TWS
Thanks 闯ü谤驳别苍,
very valid points. I was really afraid that complexity of this bug will cause problems. So I didn't write code from scratch but did least possible modifications the the official sample app IB provides as a common ground (Program.py in case of python) precisely hoping that maybe it will be bit more familiar to people and will save time and effort for everyone (at least to IB developers as I sent it first to them) and the way to look at that code is to run a comparison to the original IB code to really see the difference = the actual new code which is executed. And the code I have added there is less than 200 lines, the rest is the "full" test app, which can be ignored (or at least I was hoping people who would look at it would have more trust to IB's code :) and everyone has the original app to compare. I didn't delete that IB boiler code to make the comparison obvious, if I do the app would be tiny. I can do that easily = keep only code which gets executed if that will make everyone more comfortable)
In the nutshell reqMktData_test (set a break point here and easy walk) ? ?calls reqMktData_test_work in a loop ? ? ? all contracts are sent to reqMktData ? ? ? the process is slowed down and finished by monitor_not_done so we don't have more than x data-lines used at the same time, that function monitors that all data is received and we can cancel the subscription? ? ? ? the results are printed using process_and_export_data the following wrapper callbacks take the incoming data (unfortunately reqMktData triggers all those)
tickPrice
tickSize tickGeneric tickString error ?
Actually, running against a paper account is perfectly fine (this is how I run it all day). I believe running it against a paper account with no subscription will also work (need to set DELAYED data).?
Totally. In my live application I have several hundreds of data lines to work with and slightly less number of contracts I am interested in trading "now". and about 1500 - 2500 "full universe". So I go through such cycle to get latest on the market and maintain the subscription for those several hundreds I am interested in for a while (and there I run into issues that some data is not showing up for 20+ seconds (which I consider never)). Of course as I wanted the "simplest" app showing the issues. As you rightly say, it is a large chunk of code already :) I didn't want to make it larger by having another 200 lines "ok here we subscribed and waiting, and 3 min latter suddenly some of those lines lack data"?
Thanks, I double checked (happy to double check). The code which I added is only to not have the busy waiting while waiting for data delivered by a different thread. There is one place I added a slowdown (but it default to zero) in tickPrice as I was experimenting with hypothesis "what if the problems are caused by me processing the data too long", I didn't find correlation. I should have removed that before posting the code to reduce mental strain of readers. Sorry about that (yet again, that line does nothing). But indeed there are several cases in IB original code (which is not used in this test) which relies on sleep before cancelling. The test code data path doesn't rely on IB sending data within specific time as far as I can see.??
?
Another interesting bit. Thanks for sharing. As in this test we subscribe in the middle of the trading AND in case of TWS contracts from the watchlist definitely overrepresented in the list of failed symbols. Maybe something got broken in IB here. That is very tough to debug, as you get one shot per day :)?
The worrying bit that even if I do it for 5-20 data lines at a time (as a result slowly crawling). I still see the data missing. I see that it less to do with frequency of the requests?
Interesting, I didn't know about this possible issue. Will try. Although since we run several loops on the same "home brew" contracts, I would expected that after the first loop all will be good. But I will try. At this point I would try anything which has any chance of solving the issue before IB retires the 10.19 gateway and would force move to 10.30
?
?
|
|
Re: Entry that triggers OCO with TP and SL
Here's one of mine. Can't figure out how to fix the indents though.
?
def place_trailing_oca_bracket_order(self, order_details):
"""
Place a bracket-type OCA order with a trailing stop loss and an adjustable take profit order.
The stop loss is a trailing stop placed at a certain percentage away from the entry price.
The take profit starts as a stop order and converts to a trailing stop upon reaching a profit trigger.
"""
symbol = order_details["symbol"]
entry_quantity = order_details["quantity"]
time_bound = order_details.get("minutes_until_market_close", self.max_allowed_position_minutes)
# Max allowed time in mins
time_bound = min(time_bound, self.max_allowed_position_minutes)
# Determine position direction
action = order_details.get("side", "BUY").upper()
is_long = action == "BUY"
reverse_action = "SELL" if is_long else "BUY"
# Get minimum tick size
min_tick = self.get_min_tick(symbol)
strategy_params = order_details.get("strategy_params", {})
# Extract standard strategy parameters
stop_loss_percent = strategy_params.get("stop_loss_percent")
trailing_stop_percent = strategy_params.get("trailing_stop_percent")
profit_trigger_percent = strategy_params.get("profit_trigger_percent")
profit_lock_in_percent = strategy_params.get("profit_lock_in_percent")
trailing_profit_percent = strategy_params.get("trailing_profit_percent")
# Create entry order with adjusted price based on direction
if is_long:
entry_price = self.get_ask_price(symbol)
else:
entry_price = self.get_bid_price(symbol)
# Calculate price levels based on position direction
if is_long:
# Price adjustments for long positions
# Initial stop loss below entry price
initial_stop_loss_price = self.adjust_price_to_tick_size(entry_price * (1 - stop_loss_percent), min_tick)
initial_stop_loss_protection_price = (
self.adjust_price_to_tick_size(entry_price * (1 - stop_loss_percent), min_tick) - 1
)
# Profit trigger above entry price
profit_trigger_price = self.adjust_price_to_tick_size(entry_price * (1 + profit_trigger_percent), min_tick)
# Lock in price slightly below trigger price
profit_lock_in_price = self.adjust_price_to_tick_size(
entry_price * (1 + profit_lock_in_percent), min_tick
) # Must be lower than trigger for longs to protect profits
# Amount to trail by
trailing_stop_amount = self.adjust_price_to_tick_size(entry_price * trailing_profit_percent, min_tick)
else:
# Price adjustments for short positions
# Initial stop loss above entry price
initial_stop_loss_price = self.adjust_price_to_tick_size(entry_price * (1 + stop_loss_percent), min_tick)
initial_stop_loss_protection_price = (
self.adjust_price_to_tick_size(entry_price * (1 + stop_loss_percent), min_tick) + 1
)
# Profit trigger below entry price
profit_trigger_price = self.adjust_price_to_tick_size(entry_price * (1 - profit_trigger_percent), min_tick)
# Lock in price slightly above trigger price
profit_lock_in_price = self.adjust_price_to_tick_size(
entry_price * (1 - profit_lock_in_percent), min_tick
) # Must be higher than trigger for shorts to protect profits
# Amount to trail by
trailing_stop_amount = self.adjust_price_to_tick_size(entry_price * trailing_stop_percent, min_tick)
contract = self.get_contract(symbol)
prediction_id = order_details.get("prediction_id")
bracket_order = []
# Create entry order
entry_order = LimitOrder(
action=action,
totalQuantity=entry_quantity,
lmtPrice=entry_price,
orderId=self.ib.client.getReqId(),
transmit=False,
orderRef=f"{prediction_id}_entry",
tif="IOC",
)
bracket_order.append(entry_order)
# Create initial trailing stop loss order
stop_loss_order = Order(
action=reverse_action,
orderType="TRAIL",
totalQuantity=entry_quantity,
orderId=self.ib.client.getReqId(),
transmit=False,
parentId=entry_order.orderId,
orderRef=f"{prediction_id}_stop_loss",
ocaType=2,
outsideRth=True,
ocaGroup=prediction_id,
auxPrice=trailing_stop_amount, # Trailing amount using trailing_stop_percent
trailStopPrice=initial_stop_loss_price, # Initial stop price using stop_loss_percent
)
bracket_order.append(stop_loss_order)
# Create first take profit order (triggers at profit_trigger_percent and stops at profit_lock_in_percent)
take_profit_order = Order(
action=reverse_action,
orderType="STP",
totalQuantity=entry_quantity,
orderId=self.ib.client.getReqId(),
transmit=False,
parentId=entry_order.orderId,
orderRef=f"{prediction_id}_take_profit_stop",
ocaType=2,
outsideRth=True,
ocaGroup=prediction_id,
auxPrice=initial_stop_loss_protection_price,
triggerPrice=profit_trigger_price,
adjustedOrderType="STP",
adjustedStopPrice=profit_lock_in_price,
)
bracket_order.append(take_profit_order)
# Create time bound order (same for both market conditions)
time_bound_order = MarketOrder(
action=reverse_action,
totalQuantity=entry_quantity,
orderId=self.ib.client.getReqId(),
transmit=False,
parentId=entry_order.orderId,
outsideRth=True,
orderRef=f"{prediction_id}_time_bound",
ocaType=2,
ocaGroup=prediction_id,
)
# Add time condition to the time_bound_order
time_condition_after = TimeCondition()
time_condition_after.time = format_date(add_minutes_to_date(get_utc_now(), time_bound), "%Y%m%d-%H:%M:%S")
time_condition_after.isMore = True
time_bound_order.conditions.append(time_condition_after)
bracket_order.append(time_bound_order)
# Set the last order to transmit
bracket_order[-1].transmit = True
# Place all orders
trades = []
for order in bracket_order:
log.info(f"Placing {order.orderRef} for {'long' if is_long else 'short'} position")
trade = self.ib.placeOrder(contract, order)
trades.append(trade)
log.info(f"Order placed: {order.orderRef} with ID {order.orderId}")
return trades
?
? |
|
reqMktData callbacks are missing data with gateway 10.30 and 10.31 sporadically, works fine with 10.19 and older gateways/TWS
Dear Twsapi experts,
Since 10.30 gateway/tws looks like reqMktData cannot be relied upon and I am often not getting the data I am asking for. And I am talking about basic bid/ask/last etc data for popular contracts with high volume of trading on Nasdaq/NYSE (like MSFT, GOOG etc) nothing exotic. It happens so often that it makes it impossible to use reqMktData in practice (up to 10% call failure rate). Yet I've been using reqMktData for 5+ years both in test and live trading, it was pretty reliable for me, no complains so far (it was slow, sure, but reliable :) ). Moreover for gateway/tws 10.19 it works as expected still today just 10.30+ drops data. Making me believe that something got broken in 10.30 and newer software. The issue, as I will describe below seems to be related to resource usage/management of 10.30+ software, which makes me wonder what else tws is dropping once it gets somehow pushed. I could imagine that IB is loosing some data here which used to be perfect/fine it might as well drop other api call/callback (like drooping sending an order or dropping some order status data would be really bad). Thus making it more urgent to fix it.? ?
I of course reported the issue to IB with logs clearly showing that the gateway didn't provide the data it was asked. Moreover, I shipped them this test app which I will share below which (for me) 100% repeats the issue. And IB went silent for 10+ days. I have no idea atm if they confirm the issue and busy fixing or it just ended up at the bottom of the backlog. Thus really appreciate the help of the community here. Maybe I am not alone experiencing this issue.
?
My ask for community is to run the test app provided (I did one in python and one in java. both languages experience the issue more strongly pointing towards IB software issue) if more can repeat the issue (and there is no issues in the my test code :) ) maybe it will help IB to rise the urgency for fixing the issue.?
I am using IBAPI 1030.01. It works for connecting to any gateway/TWS versions. I have not tested it with other IBAPI SDK version.? in this api there is folder \samples\Python\Testbed extract provided python.zip in that folder (details for Java are later). Then, run
Program_get_bid.py -f symbols.csv -m REALTIME --data-lines 75 --loops 5 --port 4001 this will read symbols (about 1200 stocks from big, liquid companies on Nasdaq/NYSE), connect to your gateway (maybe you use a different port). will ask for REALTIME data (if you don't have subscription to US stocks + realtime addon, FROZEN, DELAYED seem to be experience the same issue). Test also will make sure that we don't ask for more that 75 contract simultaneously and run this operation 5 loops? The trick with this bug is that it is doesn't happen for a specific contract or specific order of API calls. In my experience it happens "randomly" after the gateway was used for some time. So this test app asks for data using self.reqMktData(self.nextValidOrderId, contract, "", False, False, []) and monitors callbacks if it receives the following data ["bid", "ask", "last", "volume", "last_timestamp", "high", "low"]. it waits for 20 seconds for this data to appear and if doesn't appear (this is the bug I am talking about) it declares this call as failed and prints out as TIMEOUT. Some runs will experience more issues than others. Some will have zero issues. The important bit that in my setup 5 loops of 1200 contracts always causes some failures if connected 10.30+ gateway/TWS and zero issues on 10.19. It is true for both Linux and Windows.? A fragment of output?
?
All failed symbols are printed during the run and summarized with info received at the end. Also reqId is printed, so one can go into gateway log and search for this ID. All the searches I did in the gateway's log it was clear that gateway really didn't send the info the app is missing (i.e. it is not the test app which missed it somehow but the gateway didn't even send it)
Notes
Regards,
AJ
|
buddy
wordd