|
Hi, My first post here and I hope I'm posting in the right way and place. Context: - I have a Docker container running an instance of the Interactive Brokers Gateway, paper trading account, on a remote Linux server - The IB Gateway is set to auto-restart every day at 8:30 AM GMT - I use Xvfb, a virtual display, so I have no physical graphic interface whilst running - I am currently solely testing the daily auto-restart, thus the app has no clients connected and no requests whatsoever submitted My problem is that so far, it never made it to a second restart: I may start the container today, that will successfully (reading the logs and experiencing the "another session is running..." warnings both on my phone or local instance of IB Gateway/TWS) launch and run IB Gateway, it will make it to the following day restart and succede (judging again from logs and trying to login from somewhere else); however, the second day auto-restart always fails. Worth to clarify that the failing restart is not on a Saturday/Sunday, which I'm aware is expected to require authentication again. What I can see in the launcher.log of a succesfull restart as opposed to the one of the failing restart is that the latter, at some point after the message "SPLASH Authenticating" reads "COMPETE: session kicked out" and few milliseconds after "Disconnect all farms due to competing session [control=false,context=]." and then after a few (just number after SPLASH ending differently) "SPLASH 20001 step 2 AuthDispatcher.processDataItr" "Received NS_TEST_REQUEST" .... i get a "Socket for connection cdc1.ibllc.com:4001 (SSL) was closed by peer." "Listener thread terminating [sessionID=2] [seen=399,totalShifted=399,moreAvailable=0]" "Disconnecting cdc1.ibllc.com:4001 (SSL) [disconnectDetails=DisconnectDetails[sessionID=2,endPoint=cdc1.ibllc.com:4001 (SSL),reason=DISCONNECT_ON_BROKEN_SOCKET,cause=null,systemMessage=recv: EOF,keepSocketOpen=false]]..." "Socket closed." then "SPLASH Attempt 1: server error, will retry in seconds" And after this the whole log, for hours will repeat everything again and again with just increasing the attempt number in the following message "SPLASH Attempt 2: connecting to server" In a successful restart log, right after the first message listed above, things would have taken a different fold, the authentication would succede, and some routing tasks before rotating to the operational log would be printed. To be frank, I do have a X server option when for when I test locally on my machine where I built the Doicker image, but I never tested for days the behaviour to see what might pop up. I did test few restarts in a row by just changing the restart time on the go, and they all succeded, but I understand many things change between the two setups, so this might be of little help. Anyone care to help with what's going on here? I surely can provide the logs if needed. Thank you, Renato
|
|
duplicated order id
3
Hi All, I have an issue with orderId assignment. I have a python code where the client is a class and assign orderId to different strategies running in parallel (each strategy implemented in a separate class). When a strategy has to generate an order, it request the order id calling client member function nextOrderId(). In order to avoid assigning the same id do different strategies I implemented this function as below: # return next available order id def nextOrderId(self,stratId): #locking the assignment and increase of order id to avoid same order id assigned to different strategies with self.lockOrderIdIncrease: self.strategyList[stratId].validOrderId = self.order_id self.order_id += 1 return self.strategyList[stratId].validOrderId where self.lockOrderIdIncrease is a threading Lock object, initialized in the constructor class: self.lockOrderIdIncrease = Lock() and self.strategyList is a list that for each strategy stores the current used order id, while self.order_id contains the last available order id iI have a couple of strategies that can be activated in the same instant and in paper trading in past days I observed 2 times the error “Error 103: Duplicate order id”. Can someone help me? I do not understand why this is happening as the fragment accessing to the current order Id seems to me allowing access to one thread per time, blocking the others thank you in advance Marco
|
|
Simplest Way to know if Individual Candle Data is Pre-market, Market or After-hours.
3
if I request Market data with `outsideRth=true`, I get the data for the whole pre-market, market, and after-hours market hours which is great. However, when I inspect each candle in the data, I get the candle's time (t) but, at this point I have no way of knowing whether that one candle sits in pre-market, market or after-hours market hour. So my question is, what would be the simplest way to establish whether a single candle sits in pre-market, market or after-hours market hour? Is executing date/time calculations for each candle's time (t) the only way of doing this? This could be quite complex as would need to take into account different time zones, time of year (daylight savings time), etc. when carrying out each calculation. One option in my head is to request the market data twice; once with `outsideRth=true` and a second time with `outsideRth=false` where I would then take the difference between one set and the other which would leave me with the outsideRth candles separated from the RTH candles but, the outsideRth candles would still include both pre-market and after-hours candles and therefore, I would still need to make calculations for each candle to know exactly which period any one candle belongs in (e.g. pre-market, market, after-hours). Thanks in advance for any answers to this question. PS: This is my first time posting here and I have tried searching to see if this has been asked before but I did not get any results back so, apologies if I am not using the search tool properly or not posting the question properly either.
|
|
can't seem to find IB information on PCUSEQTR
3
Hi all not too familiar with IB's financial instrument lookup web approach. Normally I'd just use TWS desktop enter the ticker into a watch list ( done and working) and look at the tearsheet. For PCUSEQTR there doesn't seem to be one. Can someone give me the link on IB's site where I can research what IB has to say about this? thanks
|
|
10326 - OCA group revision is not allowed
23
Has anyone ever experienced the following error when trying to modify bracket orders ? 10326 - OCA group revision is not allowed It is not documented in the API documentation and IB requires detailed TWS logging in order to provide assistance. -- https://www.tradingsoftwarelab.com
|
|
in a hedge pair, is it possible to have IBKR set the limit price of the child as a multiple of the execution price of the parent?
2
Is it possible to submit a hedge pair of stock limit orders where, when the parent executes, IBKR sets the limit price of the child (which is a limit order) to a given multiple of the execution price of the parent? That is, when I set up the hedge pair, can I somehow preconfigure some ratio X, so that if the parent executes at some price P, the child then becomes active with a limit price of X * P ? At the moment I am attempting this as follows. When I set up the hedge pair, I set the limit price of the parent to some price P, and I set the limit price of the (not yet active) child to X * P. Then, whenever I update the price of the parent order to some price P', I update the price of the child order to P' * X. But this approach has two shortcomings. First, the price updates often don't reach the market at the same time, and when that happens the two prices are momentarily out of sync. Second, the parent order may execute at a price other than its limit price. When that happens the limit price of the child is X * (the limit price of the parent). I want it to be X * (the execution price of the parent). (Possibly relatedly, one of the hedge-pair parameters is "dontUseAutoPriceForHedge". Documentation says only to set this to "not use auto pricing for the hedge". But "auto pricing" seems to be mean something else, such as automatically raising the limit price at a configured rate when the order is active and not filling.) Thanks, -Neal
|
|
Dear Twsapi experts, Since 10.30 gateway/tws looks like reqMktData cannot be relied upon and I am often not getting the data I am asking for. And I am talking about basic bid/ask/last etc data for popular contracts with high volume of trading on Nasdaq/NYSE (like MSFT, GOOG etc) nothing exotic. It happens so often that it makes it impossible to use reqMktData in practice (up to 10% call failure rate). Yet I've been using reqMktData for 5+ years both in test and live trading, it was pretty reliable for me, no complains so far (it was slow, sure, but reliable :) ). Moreover for gateway/tws 10.19 it works as expected still today just 10.30+ drops data. Making me believe that something got broken in 10.30 and newer software. The issue, as I will describe below seems to be related to resource usage/management of 10.30+ software, which makes me wonder what else tws is dropping once it gets somehow pushed. I could imagine that IB is loosing some data here which used to be perfect/fine it might as well drop other api call/callback (like drooping sending an order or dropping some order status data would be really bad). Thus making it more urgent to fix it. I of course reported the issue to IB with logs clearly showing that the gateway didn't provide the data it was asked. Moreover, I shipped them this test app which I will share below which (for me) 100% repeats the issue. And IB went silent for 10+ days. I have no idea atm if they confirm the issue and busy fixing or it just ended up at the bottom of the backlog. Thus really appreciate the help of the community here. Maybe I am not alone experiencing this issue. My ask for community is to run the test app provided (I did one in python and one in java. both languages experience the issue more strongly pointing towards IB software issue) if more can repeat the issue (and there is no issues in the my test code :) ) maybe it will help IB to rise the urgency for fixing the issue. I am using IBAPI 1030.01. It works for connecting to any gateway/TWS versions. I have not tested it with other IBAPI SDK version. in this api there is folder \samples\Python\Testbed extract provided python.zip in that folder (details for Java are later). Then, run Program_get_bid.py -f symbols.csv -m REALTIME --data-lines 75 --loops 5 --port 4001 this will read symbols (about 1200 stocks from big, liquid companies on Nasdaq/NYSE), connect to your gateway (maybe you use a different port). will ask for REALTIME data (if you don't have subscription to US stocks + realtime addon, FROZEN, DELAYED seem to be experience the same issue). Test also will make sure that we don't ask for more that 75 contract simultaneously and run this operation 5 loops The trick with this bug is that it is doesn't happen for a specific contract or specific order of API calls. In my experience it happens "randomly" after the gateway was used for some time. So this test app asks for data using self.reqMktData(self.nextValidOrderId, contract, "", False, False, []) and monitors callbacks if it receives the following data ["bid", "ask", "last", "volume", "last_timestamp", "high", "low"]. it waits for 20 seconds for this data to appear and if doesn't appear (this is the bug I am talking about) it declares this call as failed and prints out as TIMEOUT. Some runs will experience more issues than others. Some will have zero issues. The important bit that in my setup 5 loops of 1200 contracts always causes some failures if connected 10.30+ gateway/TWS and zero issues on 10.19. It is true for both Linux and Windows. A fragment of output All failed symbols are printed during the run and summarized with info received at the end. Also reqId is printed, so one can go into gateway log and search for this ID. All the searches I did in the gateway's log it was clear that gateway really didn't send the info the app is missing (i.e. it is not the test app which missed it somehow but the gateway didn't even send it) Notes Program_get_bid.py is based on Program.py. One can easily compare the differences to the "official" sample. It is really minimal differ
|
|
ALERT LONG MESSAGE. Hello Folks, I am working on implementing a smooth failover mechanism to ensure the integrity of the code whenever there is a connection drop between the TWS application and the IBKR server. Request you to provide valuable insights on below scenarios. Case 1: An order modification request was placed, increasing the order size beyond the previously placed quantity. The request was successfully sent to the IBKR server, but before we could receive an Order Status Callback(from that we can extract updated Order TotalQuantity), the connection between TWS and IBKR was lost. Despite the connection drop, the order was successfully modified at the exchange. Scenario During Connection Loss If the internet was down during this period, it is possible that the order was fully executed at the exchange. Once the connection is restored and we request OpenOrders, we will not receive any update related to this order because it is no longer active at the exchange. However, when we request Executions, we will receive all partial or full executions for the order. Problem with Execution Callback Data The Executions Callback does not provide the remaining order size, unlike the Order Status Callback. As a result, we cannot determine the actual remaining order size that was present at the exchange. This forces us to rely on our internal maps, which in this case would not reflect the correct remaining size of the order. Consequently, we may process extra executions, leading to discrepancies. Example to Illustrate the Issue Initial Order Size → 10 Modified Order Size → 15 (Modification was sent but confirmation was not received due to connection loss) Upon reconnection: We do not receive an Order Status update for the modified order. We receive a cumulative execution size of 15, but we have no way of verifying the actual remaining order size at the time of execution. This results in an inconsistency in tracking the order status and execution details. Currently, each time the TWS API receives an execution update, we calculate the remaining order size as follows: remaining_order_size = remaining_order_size - current_execution_size; if (remaining_order_size <= 0) { std::cout << "Order is fully executed"; // Remove order from metadata maps } if(remaining_order_size<0) remaining_order_size = 0; To ensure proper synchronization, we need to pass the remaining order size to the Execution listeners. However, in the above logic, as soon as remaining_order_size reaches zero or a negative value, the order is immediately removed from the metadata maps. To prevent premature removal and ensure consistency, I have implemented a safeguard by updating the order size at the time of order submission. This adjustment helps maintain synchronization and prevents potential discrepancies. Issues in using above mechanism :( I have identified a potential issue with updating the order size at the time of order submission. Consider the following scenario: A modification request is placed while the internet is down. At this point, we update the order size in our system to 15 (UPDATED AT THE TIME OF SENDING THE ORDER). Meanwhile, during the disconnection, the order is executed at the exchange for 10 units. Once the connection is restored, we receive only the cumulative execution size of 10 from TWS (WE ASSUMED ORDER SIZE TO BE UPDATED TO 15). However, since we do not receive all individual execution details, the internal system will not recognize that the order is fully executed. As a result, the order remains in our internal maps, and we continue requesting execution details for the remaining 5 units that no longer exist. To address this issue, we can retrieve CompletedOrders from TWS. Any other method to handle above scenarios. Thanks for you patience :)
|
|
Restoring combo in position update
When I place a combo order via the API, TWS shows the options spread as I sent via the API: However, position callbacks give only the individual option leg positions. Is there a way to restore the combo composition via the API? (I can store the composition info in DB, but I'm interested in whether there is an API way.) Thanks, Zsolt
|
|
TBill Maturity date
5
I have scoured the API (C++) and am trying to figure out how to find the maturity date of a TBILL. I have stepped through the fields of ContractDetails in my Visual Studio IDE, but any fields such as expiration , maturity etc are empty. For bonds, I can look at the name and parse the string, but for T BILLS, it seems like I am out of luck. I just have the contract ID. Has anybody solved this issue? Thx, Dave Linenberg
|
|
reqIds hangs when recereating IBClient - EWrapper
3
I have IBClient that implements EWrapper that I copied from C# samples from official TWS repository. When some event happens, I would like to destroy existing client and create new instance. This is the method I use to create this client. void Create() { var signal = new EReaderMonitorSignal(); _client = new IBClient(signal); _client.ClientSocket.SetConnectOptions("+PACEAPI"); _client.ClientSocket.eConnect(Host, Port, 0); var reader = new EReader(_client.ClientSocket, signal); var process = new Thread(() => { while (_client.ClientSocket.IsConnected()) { signal.waitForSignal(); reader.processMsgs(); } }); process.Start(); reader.Start(); } After that, I'm waiting for the next valid ID. async Task Register() { var source = new TaskCompletionSource(); void subscribe(int o) { _client.NextValidId -= subscribe; source.TrySetResult(); } _client.NextValidId += subscribe; _client.ClientSocket.reqIds(-1); await source.Task; } Then use method to destroy client. void Destroy() { _client?.ClientSocket?.eDisconnect(); } The first time everything works fine. When I try to call these 3 methods next time, "reqIds" hangs and I never receive response for "NextValidId". Is it some known limitation that only one client can connect to TWS only once or am I missing something?
|
|
Long on the stock, want to exit but: Order rejected The contract is not available for short sale
8
Hi, My algo took a long position on a stock and when target was reached and wanted to exit I got this message: 201 Order rejected - reason:The contract is not available for short sale. I'm aware of this message if I want to go short on a stock. But if I bought and own the shares why does this happen? The algo runs in paper trading account so is it related? Can this really happen live? Thanks A
|
|
Regarding the 50 msg/sec TWS rate limitation, is IBKR checking it by calculating a continuously sliding window, or are they doing a nonoverlapping time window? The first is stricter. If it's the second one, then theoretically you can send up to 100 messages in a short burst. As long as the first fifty were near the end of the first window, and the next 50 were at the beginning of the next window. If this was an option, then there would be the challenge of trying to figure out where TWS' clock was. Does anyone know if the calculation is made in TWS before orders are sent out over the wire, or are the calculations made upon receipt at the data center?
|
|
Dear Twsapi experts, I am calling reqMktData with snapshot flag set to True reqMktData(orderId, contract, "", True, False, []) The trouble is that randomly this call delivers not "latest" available data but hours old data (6 hours or more old timestamps). Note, the data gets delivered fast, that is not the problem, but the content of data is VERY old. I.e. looking at the timestamp it is very old (not seconds/minutes but hours old). And I believe even data on last/bid/ask etc is old as well. Those are harder to debug but some which I checked manually really show that "last" is not recent but indeed 5hours old. Wonder if anyone experienced this and has any workarounds? And of course it happens randomly only for some of the calls. Moreover, you call same function for a contract which just got delivered very old data and correct data is delivered. So not so easy to debug :). However the problem (for me) is 100% consistent. I.e. If I run snapshot for 1000 different contracts then 5-15-100 (random) will return very old data. If I call snapshot for very same symbols again most will return correct data, yet some other may fail (or may not). Making this function really not usable I am using: IB API Version: 10.32.1 (but same was happening with 10.30.1 API as well) I am using 10.30.1u tws (but I have seen same issue with 10.19) happens on both Windows10 and Linux Example of what I am talking about is presented in this table history: the "latest" timestamp of a snapshot which I have seen in the previous runs time1, time2 is the 1st, 2nd, 3rd call for snapshot for this this contract and corresponding timestamp so for symbol WYNN I have already seen data with timestamp 15:49:37 (I am in CET timezone +6h from NewYork). Yet on the first call with WYNN I receive data with timestamp 09:56:40. Obviously wrong since just several minutes ago calling same snapshot was giving me much later timestamp. Yet, if I call snapshot for WYNN again 2nd, 3rd time I get "correct" timestamps (correct == they are same or later) . One could argue that the workaround is just call snapshot several times. The problem is that sometimes it start failing again (in this test run I couldn't repeat it, but I see that often) and sometimes it takes 3-5 calls before I get the data. For completeness I attach the python code which can reproduce the problem, where I just call snapshot in a loop for 1000 symbols and I run it like this Program_get_bid_small.py' '-f' 'symbols.csv' '-m' 'REALTIME' '--data-lines' '75' '--loops' '3' '--port' '4001' Note, first time one needs to run the code two times to generate that "history" file, so the next tests can have that "latest" (=history) timestamp to compare against and output error in case they are not in order. I of course reported it to IB first, but the report is sitting there for 3 weeks... with no real progress. Thus wonder if community have seen it and what could be a workaround. AJ PS. Wonder if this issue is related to the one I reported earlier where reqMktData but without snapshot flag fails on TWS 10.30 and later. So I started experimenting with snapshot and run into this issue :)
|
|
Error using TWS scanner to filter stocks
Hi, I've created a script that uses the TWS scanner to filter stocks for a strategy but I'm receiving this error: "Error 162, reqId 5: Historical Market Data Service error message:API scanner subscription cancelled: 5" Has anyone experienced this/knows how to deal with this or what I'm missing?
|
|
VSTOXX daily settlement prices
7
Can I somehow retrieve the VSTOXX daily settlement prices via the API? An alternative source where I candownload a CSV would be also great.
|
|
IB Gateway 10.30.1t
Does anyone have available the installation file(s) for IB Gateway 10.30.1t ? If possible for all operating systems. Thank you. -- https://www.tradingsoftwarelab.com
|
|
Web API no Info
Hi together, I'm not quite sure if this is the right group, because there are two API options (via TWS and via Client Portal API). I am looking for https://www.interactivebrokers.com/campus/trading-lessons/requesting-market-data/ and can also retrieve conid's for given symbols. But if I now use /iserver/secdef/info/ to retrieve data for a symbol I only get Error 404 - File not found Does this perhaps only work if you have paid for market data? Otherwise there is only cash on my securities account... Best regards joh
|
|
TWS 10.33 and 10.34 crashing on Windows and Mac OS
2
I am seeing crashes with TWS versions 10.33 and 10.34 on Windows and Mac OS. TWS versions 10.33 and 10.34 are crashing on Windows 7 when exiting TWS. Windows displays a pop-up with "Trader Workstation stopped working" error. TWS version 10.34 crashes on Mac OS X 10.14.6 right when it is starting. Identifier: com.install4j.5556-0173-2810-3400.22; Crashed Thread: 4 Java: JTS-Main; Exception Type: EXC_BAD_ACCESS (SIGABRT); Exception Codes: EXC_I386_GPFLT; Exception Note: EXC_CORPSE_NOTIFY Is anyone else experiencing crashes and/or instability with TWS versions 10.33 and up on Windows and Mac OS ? -- https://www.tradingsoftwarelab.com
|
|
Closing SELL order being treated as SHORT order in the api
Has anyone else had this occur to them for stock trades?: I take a long position, and then try to close out the position (as a in a stop-loss situation), and the closing SELL order is treated as a SHORT order, occasionally resulting in a security being 'hard to borrow' and not being able to close out my position! This is a little unnerving... I don't seem to have this problem doing things 'by hand', just with orders via the API. I'm looking through the documentation to see if there's some 'closing trade' flag I can stick on an order, but it looks like 'BUY' and 'SELL' are my only options as far as side or 'action' as it's called in the docs. I DID see this, but it's only for institutional clients: string OpenClose [get, set] For institutional customers only. Valid values are O (open) and C (close). Available for institutional clients to determine if this order is to open or close a position. When Action = "BUY" and OpenClose = "O" this will open a new position. When Action = "BUY" and OpenClose = "C" this will close and existing short position. any help would be appreciated! thanks https://interactivebrokers.github.io/tws-api/classIBApi_1_1Order.html
|
Image Name
Sat 8:39am
Sat 8:39am