Yes indeed, the issue that Juergen touched here would make OrderId's pretty much useless?even if IB kept them because reqCompletedOrders returns indiscriminately, and inescapably, all today's completed orders in TWS, at least for all API clients and if asked properly, all manual orders too. So there would be no way to guarantee unique identification for completed orders using OrderIds anyway. (If there were an intention to maintain some symmetry between different order managing request branches, this call should perhaps have been called reqAllCompletedOrders or reqGlobalCompletedOrders in accord with reqAllOpenOrders or reqGlobalCancel, but apparently there's none such, so we have the most simple name here irrespective of the function it serves, simply because there are no other request methods for completed orders to differentiate).

any guide to run tws on arm tablet / smartphone??

Hi,

?

I would like to reopen a thread on auto login, and auto restart for IB Gateway on Linux. I have been using TWS for my entire algorithmic trading journey only building strategies within the confines of the stock market's cash open (9:30am to 4pm) to keep systems running all day without needing many restart solutions and auto login credentials. But, now I want to run strategies all night long and for long periods of time. That is how I came across the benefits of using IB Gateway. I have built watchdogs for my applications to handle crashes from disconnecting from TWS, but TWS itself never crashed, only my application. But now I want to build algorithms to trade all through the night without much supervision with IB Gateway. I plan to continue using bash scripting within a linux machine for process monitoring & watchdogs. So to start, how can I enable auto login on IB gateway? What are the limitations you guys have come across? I already have set up a 5:15pm auto restart, as I have a futures trading algorithm backtested and ready to go throughout the night and non cash market hours, but I do not see a place for auto login, etc?

?

Thanks,
Brendan

As @ds-avatar indicated, orderIDs have a limited shelf life. They are only meaningful as long as the order is active and only for the client (as in clientID) that created the order. Keep in mind that each clientID has its own (permanent) sequence of orderIDs, that orderIDs must strictly increase for each new order placed, and that orderIDs might have to be adjusted ("bound") when one client gets in contact with an active order of a different client.

?

That is why each order also has a permID. IBKR assigns it when the order is accepted and returns properly adjusted Order objects with the openOrder() callbacks. That is where you'd get the permID from )or anz other aspect of the order that was changed by IBKR or the exchange)

?

The permID is also used to relate execution and commission reports to an order.

?

闯ü谤驳别苍

?

?

On Mon, Feb 17, 2025 at 12:29 AM, <joshi.tarun3107@...> wrote:

I don't think IB keeps OrderId's for completed orders.

Hello Folks,

I am encountering an issue where order.orderId is consistently returning 0 when making the following request:

I'm currently using Paper Trading account.

This occurs specifically in the completedOrder(const Contract& contract, const Order& order, const OrderState& orderState) callback response.

Has anyone else experienced a similar issue or found a solution for this behavior?

Looking forward to your insights.

I sincerely thank everyone for their valuable guidance on the design patterns I should follow to prevent discrepancies.

Since there was some interest in running TWS/IBGW on ARM (such as Raspberry Pi), here a quick update since the latest TWS/IBGW 1034 make this much more feasible.

Here a copy of my post in the IBC group:

==============================================

As Richard mentioned in this post, the Linux distributions of TWS/IBGW 10.34 now ship with a Java JRE17. That means that IBKR has made the appropriate changes in the source code so that TWS/IBGW now support the Java language changes introduced by Java 11 and Java 17.

• 900MHz quad-core ARM Cortex-A7 32bit CPU
• 1GB RAM
• Raspbian/Debian 12.9 (bookworm)

Before we go on, one quick disclaimer. TWS/IBGW have not been designed for the ARM architecture and their extensive dependence on Java 8 (until now) and Java FX2 features places significant (undocumented) requirements on the system, the Java JRE, and JavaFX . While running TWS/IBGW on ARM is an interesting exercise, seems to become quite stable, and will be useful in many scenarios, using them to trade in a live account is still a really bad idea.

With that out of the way, here the simple steps that I went through to bring up TWS. No modification of TWS scripts are required:

• Install the BellSoft Full Java JRE 17.0.14 (in my case in /opt/Java/jre-17.0.14-full). JRE 21 will not work.

• Install libgtk2.0 since TWS/IBGW need libgdk-x11-2.0.so.0. Most Linux versions now ship with GTK3 or GTK4

• Download TWS from:
  

• Install TWS with:

  app_java_home="/opt/Java/jre-17.0.14-full" sh tws-latest-standalone-linux-x64.sh

• Run TWS with:
  JAVA_HOME="/opt/Java/jre-17.0.14-full" \
  ? ? INSTALL4J_JAVA_HOME="${JAVA_HOME}" \
  ? ? ? ? app_java_home="${JAVA_HOME}" \
  ? ? ? ? ? ? ? ? sh ~/Jts/1034/tws

That's it. It takes a while to start up (all spent in "Reading layout settings ..") but here it is ...

闯ü谤驳别苍

?

?

In Software Engineering (and also in Automation) there is a concept called a Controller pattern. The best analogy is the clutch when you drive stick.

A controller has a:

- Resource - the object it controls

- Desired State: what it should be

- Actual State: what it currently is

The controller simply tries to issue actions to bring the actual state closer to the desired state.

?

In this case you need a double clutch as you will be driving a race car :)

?

- Position Controller

Resource: position (indexed by security)

Desired State: total # of contracts in a certain stock/security that you want to have

Actual State: sum(# of executions) + last valid position update (usually from startup/reconnect)

Action: your trading model changes the desired state only - it does not issue any orders; the position controller checks how much total qty is active in the Order Controller and taking the difference (desired vs actual - call it position error) it will issue new requests (to the Order Controller) to bring the number of active quantity in the market equal to the desired state. If it mistakenly overfills by "k" contracts (e.g. due to a bad algo in Order Controller), it will request a -k to the Order Controller. So it's self-healing to a certain degree.

?

- Order Controller

Resource: orders (indexed by security and an internal order request UUID from position controller)

Desired State: # total == # filled; you can go on a splurge with additional "desires": desired time (T), desired risk (max 4 ticks), active/passive, iceberg, VWAP etc..

Actual State: each order has # total quantity, # filled = 0 and # unfilled = # total as received from the initial request (coming from Position Controller) and then updated by subsequent API responses by adding up executions

Action: the Position Controller will ask for more exposure in the market, or will ask to reduce its exposure in the market by issuing specific requests to the Order Controller. In turn, the order controller will then issue new order(s) based on that. It could issue none initially if a desired start time is not there, etc.. Or it could issue less than the requested quantity in case of VWAP etc.. In the simple case it will just issue a single new order with total=request, filled=0, unfilled=request. A slightly more complex scenario it will modify an existing order.

As executions fill up, you update back the filled/unfilled parameters for that order id.

?

Each controller runs in it's own thread (or async routine) in parallel.

?

Now, addressing your use-case:

You request 10 desired state and you have 0 actual state in Position Controller.

Order Controller issues a new order with total qty=10 (still unfilled) associated with an internal id=0xdeadbeef.

You modify your request to 15 desired state and immediately disconnect.

Order Controller won't issue new orders or modify existing orders while disconnected.

You will have 15 in your desired state and a 0 in your actual state in the Position Controller.

You reconnect.

Once you're back up (regardless what happened in the market), you will synchronize positions and orders during startup / reconnect.

Now your Position Controller will suddenly be 15 desired state and 10 actual state.

Your Order Controller gets the list of active orders (some it knows based on internal UUID correspondence, some are maybe new, some are maybe missing).

There is 0 unfilled so initially Order Controller won't do anything. It will just remove the reference to the filled order under it's UUID 0xdeadbeef.

Next, Position Controller will issue a new request for 15-10=5 to Order Controller with an in internal id=0xaaaabbbb.

Order Controller will issue an order in the market with total qty 5.

To check my internet connection I simply ping google.com. Very crude but does the trick for me.

It looks like there's no way of deciding whether the order was actually modified without placing a req[All]OpenOrders call. I don't usually modify working orders but if I had to approach this problem I would likely use a circular awaitable future structure that would wait for two events to complete together: an OpenOrderEnd callback and an ExecutionEnd callback, and make a new req[All]OpenOrders call every time these two events have happened. In C# structurally this can be done either with regular events or two TaskCompletionSource objects and an await Task.WhenAll (or similar) call in a cycle. Doing this would ensure optimally timed calls to req[All]OpenOrders that would neither overlap each other nor fire needlessly in the absense of new Executions. Overall while this would inevitably lead to quite many req[All]OpenOrders calls, these will actually not require much processing power or connection hopping since these data are already in TWS so getting and processing it through the socket is going to be cheap even with some redundancy, which seems to be inevitable here

--

Best,
DS

Have a look at hedginglab.com.

?

?

?

?

On Fri, Feb 14, 2025 at 01:01 PM, Richard L King wrote:

Rather than comment on the questions you’ve raised, what I’ll do here is just lay out the basic principles that underly the order management component of my trading platform, which I have honed over the past 22 years. I believe it is immune to all the situations you describe.

?

1. Maintain a model in the client containing information about all current orders. This is the client’s view of the current state of affairs, which may not be the same as the reality at any moment because of the unavoidable asynchronicity of multiple cooperating (very) loosely--coupled components.

?

2. Record the state of the model in a persistent data store that can be used to reconstitute the model after a restart of the client (for any reason). As a matter of implementation, I use a text file containing JSON-encoded records that reflect every order-related operation or event sent to or received from the API. This ‘transaction log’ can be replayed to reconstruct the model as it was at the time the last record was written.

?

3. Update the model whenever an order is placed, modified, or cancelled.

?

4. Update the model whenever an open order, order status, execution or error notification is received from the API. Note that it is execution notifications that are really in control of order state; open order and order status notifications merely provide confirmation, and an inconsistency would be a definite fault (see 6 below).

?

5. As a practical matter, the representation of orders in the model is the actual order objects used in the place order call, updated on receipt with data from open order notifications.

?

6. If at any point an irreconcilable inconsistency emerges (for example receipt of an open order notification for an order that is no longer in the model), this is a situation that needs to be investigated (typically a programming error): execution of the client must be immediately stopped since it is not safe to proceed from an indeterminate state. (This might surprise some, but where real money is at stake it is unwise to assume that the situation will somehow correct itself.)

?

7. Provide an optional capability to ensure that every order always has a protective stop-loss. For a manual trading client, this may not be needed, because there is a least a strong possibility that there will be a human present to assess unexpected conditions and take corrective action if need be. For an unattended auto-trading system, the situation may be very different.

?

I think that more or less covers it, though there is a lot of detail I’ve glossed over.

?

Sounds like a lot of work? It is!...

?

?

Richard

?

ALERT LONG MESSAGE.

Hello Folks,

Request you to provide valuable insights on below scenarios.

Case 1:

An order modification request was placed, increasing the order size beyond the previously placed quantity. The request was successfully sent to the IBKR server, but before we could receive an Order Status Callback(from that we can extract updated Order TotalQuantity), the connection between TWS and IBKR was lost.

Despite the connection drop, the order was successfully modified at the exchange.

Scenario During Connection Loss

• If the internet was down during this period, it is possible that the order was fully executed at the exchange.

• Once the connection is restored and we request OpenOrders, we will not receive any update related to this order because it is no longer active at the exchange.

• However, when we request Executions, we will receive all partial or full executions for the order.

Problem with Execution Callback Data

• The Executions Callback does not provide the remaining order size, unlike the Order Status Callback.

• As a result, we cannot determine the actual remaining order size that was present at the exchange.

• This forces us to rely on our internal maps, which in this case would not reflect the correct remaining size of the order.

• Consequently, we may process extra executions, leading to discrepancies.

Example to Illustrate the Issue

1. Initial Order Size → 10

2. Modified Order Size → 15 (Modification was sent but confirmation was not received due to connection loss)

3. Upon reconnection:

• We do not receive an Order Status update for the modified order.

• We receive a cumulative execution size of 15, but we have no way of verifying the actual remaining order size at the time of execution.

This results in an inconsistency in tracking the order status and execution details.

Currently, each time the TWS API receives an execution update, we calculate the remaining order size as follows:

remaining_order_size = remaining_order_size - current_execution_size;

if (remaining_order_size <= 0) {

????std::cout << "Order is fully executed";

????// Remove order from metadata maps

}

if(remaining_order_size<0) remaining_order_size = 0;

To ensure proper synchronization, we need to pass the remaining order size to the Execution listeners. However, in the above logic, as soon as remaining_order_size reaches zero or a negative value, the order is immediately removed from the metadata maps.

To prevent premature removal and ensure consistency, I have implemented a safeguard by updating the order size at the time of order submission. This adjustment helps maintain synchronization and prevents potential discrepancies.

Issues in using above mechanism :(

I have identified a potential issue with updating the order size at the time of order submission. Consider the following scenario:

• A modification request is placed while the internet is down.

• At this point, we update the order size in our system to 15 (UPDATED AT THE TIME OF SENDING THE ORDER).

• Meanwhile, during the disconnection, the order is executed at the exchange for 10 units.

• Once the connection is restored, we receive only the cumulative execution size of 10 from TWS (WE ASSUMED ORDER SIZE TO BE UPDATED TO 15).

• However, since we do not receive all individual execution details, the internal system will not recognize that the order is fully executed.

As a result, the order remains in our internal maps, and we continue requesting execution details for the remaining 5 units that no longer exist.


To address this issue, we can retrieve CompletedOrders from TWS.


Any other method to handle above scenarios.?

Thanks for you patience :)

First of all, thank you for the detailed information.

To answer your question—if such a mechanism existed, I would use it to prevent any order requests (new, modify, or cancel) from being sent while the connection is down. During the 45-second interval before the disconnect callback is received, multiple requests could be placed. Once the connection is restored, I want TWS to discard all order requests made during the disconnection period, ensuring they are not sent to the IBKR server.

However, any pending orders that were already active on the exchange should remain unaffected. In essence, TWS should flush only the unsent requests while maintaining all existing orders.

Additionally, I have another question regarding the reliability of the connection loss/recovery callbacks.

Is it guaranteed that the following callbacks will always be triggered in the event of a connection loss and subsequent restoration?

• Code: 1100 – "Connectivity between IBKR and Trader Workstation has been lost."
• Code: 1102 – "Connectivity between IBKR and Trader Workstation has been restored."

I ask because, in some cases, I have manually disconnected the internet, yet I did not receive these callbacks. Could you clarify if there are scenarios where these messages might not be triggered?

Looking forward to your insights.

Hello Folks,

?