¿ªÔÆÌåÓý

ctrl + shift + ? for shortcuts
© 2025 Groups.io
  1. Twsapi
  2. Messages
Expanded Topics Messages Polls
Date
Re: Downloading historical SPY options data

 

The following arguments to the Contract structure worked for me:

Contract contract;
contract.secType                        = "OPT";
contract.currency                       = "USD";
contract.lastTradeDateOrContractMonth   = "20231117";
contract.strike                         = 430;
contract.right                          = "C";
contract.symbol                         = "SPY";
contract.exchange                       = "SMART";

So "symbol" is the underlying and exchange had to be "SMART" although other might work, "ARCA" did not.

On Friday, November 3, 2023 at 08:07:59 AM EDT, Marcus via groups.io <haupt_marcus@...> wrote:



When I request historical options data from IB using the following structure I get a?
"Code: 200, Msg: No security definition has been found for the request" error.

The symbol is the one that TWS displays. What am I doing wrong?

Contract contract;
contract.secType                        = "OPT";
contract.currency                       = "USD";
contract.lastTradeDateOrContractMonth   = "20231117";
contract.strike                         = 425;
contract.right                          = "C";
contract.exchange                       = "SMART";
contract.symbol                         = "SPY Nov17'23 425 CALL";

On Thursday, November 2, 2023 at 11:01:29 AM EDT, Gordon Eldest via groups.io <hymagik@...> wrote:


TWS search SPY, give "ARCA" for primaryExchange

When in doubt use reqMatchingSymbols (work like the search of TWS plus extra like all possible venues with trading hours and liquid hours) worth the effort

symbol "SPY" , try with "SMART" for exchange unless you want a specific exchange (or set the primary).
In general primaryExchange can stay empty unless you feel there could be a synonym for the pair symbol+currency (rare)
Re: Downloading historical SPY options data

 

The symbol is simply 'SPY'
Re: Downloading historical SPY options data

 


When I request historical options data from IB using the following structure I get a?
"Code: 200, Msg: No security definition has been found for the request" error.

The symbol is the one that TWS displays. What am I doing wrong?

Contract contract;
contract.secType                        = "OPT";
contract.currency                       = "USD";
contract.lastTradeDateOrContractMonth   = "20231117";
contract.strike                         = 425;
contract.right                          = "C";
contract.exchange                       = "SMART";
contract.symbol                         = "SPY Nov17'23 425 CALL";

On Thursday, November 2, 2023 at 11:01:29 AM EDT, Gordon Eldest via groups.io <hymagik@...> wrote:


TWS search SPY, give "ARCA" for primaryExchange

When in doubt use reqMatchingSymbols (work like the search of TWS plus extra like all possible venues with trading hours and liquid hours) worth the effort

symbol "SPY" , try with "SMART" for exchange unless you want a specific exchange (or set the primary).
In general primaryExchange can stay empty unless you feel there could be a synonym for the pair symbol+currency (rare)
Re: Stable dockerized TWS version, anyone?

 

¿ªÔÆÌåÓý

Hi,

It might not be exactly what you are looking for, but I have used without any problem linuxserver/rdesktop image with TWS



It¡¯s a desktop environment accessible through Remote Desktop, with a persistent ¡°config¡± volume. TWS is installed in home folder by default so it stays within the ¡°config¡± volume. It even upgrade once a new release is available.

For a fully automated docker image I use?https://github.com/gnzsnz/ib-gateway-docker, which I¡¯m maintaining as well. It¡¯s not TWS, it ib gateway.

Regards


On 24 Oct 2023, at 23:07, barum@... wrote:

I know this is a long shot, but did you resolve this issue by any chance? Facing the same issue with TWS freezing when any audio is triggered (or going to Sound Manager in Global Settings.)
On another machine, it's running fine, so I think it has to do with Docker host configuration....

Re: Order Id in error messages

 

Thank you for lettings us know that there is no issue. Keep and eye on it and try to get us the relevant TWS API log in case it ever happens again.

You found us all a little hyper since it would have been a pretty big deal if the order rejection error message would indeed have indicated the request Id of an unrelated request. Sort of like an exception to Newton's Third Law or a new quantum particle.

´³¨¹°ù²µ±ð²Ô




On Thu, Nov 2, 2023 at 06:10 PM, bespalex wrote:
I have tried to reproduce the anomaly today, but I couldn't. Everything is back to normal, with or without reqPnLSingle. So it's either a bug in my code that somehow went away, or some temporary glitch in my communication with TWS. Weird, but that's it.

Re: Order Id in error messages

 

I have tried to reproduce the anomaly today, but I couldn't. Everything is back to normal, with or without reqPnLSingle. So it's either a bug in my code that somehow went away, or some temporary glitch in my communication with TWS. Weird, but that's it.

Re: Getting the historical marketdata only works in LIVE and not in DEMO

 

What do you call DEMO and LIVE ?

Did you properly interpret last part of Jurgen reply /g/twsapi/message/51914

Personally I won't try anything called demo from IB,
They have/had some offers but I really can understand that IB don't want to clutter there servers with real "demo", so any demo labeled account or product will unlikely be a full feature account. Paper is the thing to use for try

Re: Downloading historical SPY options data

 

TWS search SPY, give "ARCA" for primaryExchange

When in doubt use reqMatchingSymbols (work like the search of TWS plus extra like all possible venues with trading hours and liquid hours) worth the effort

symbol "SPY" , try with "SMART" for exchange unless you want a specific exchange (or set the primary).
In general primaryExchange can stay empty unless you feel there could be a synonym for the pair symbol+currency (rare)

Downloading historical SPY options data

 

Does anyone know the fields for downloading historical SPY option data from IB??
Here's what I think I need for the Nov-17 420 Call

contract.secType                        = "OPT";
contract.currency                       = "USD";
contract.lastTradeDateOrContractMonth   = "20231117";
contract.strike                         = 420;
contract.right                          = "C";

contract.exchange           =  ?
contract.primaryExchange    =  ?
contract.symbol             =  ?

Re: Order Id in error messages

 

On Wed, Nov 1, 2023 at 12:49 PM, Gordon Eldest wrote:
If I rephrase it for sake of my understanding:
In your experience, some request and at least reqPnL does act as an "On" switch about what is send back as "id" when an error is reported.
When this condition is triggered (trough for example a first successful? reqPnL) then any request like reqOrder that failed will report it's reqId instead of the Orderid.
Of course this impact only request that use an orderid, but allegedly impact every of them.
Yes, pretty much so.?
This is python client, api 10.19.1, cliendId 123
1- This may happen on errors generated before IB server is even able to interpret the order ID in message sent. (this kind of error may contains the wording?"....Java Parsing Error ....")Then IB report what is the next easiest id at hand, which is the reqid.
This makes sense, maybe.
I don't use reqPnLSingle much, so I took the easiest path, just got rid of it. But I again, I can't rule out some stupid mistake on my part, and the support just added to confusion.?


Getting the historical marketdata only works in LIVE and not in DEMO

 



Above endpoint is the /hmds/history endpoint. I have done the following to find out that this endpoint doesn't work in demo.

1. Start the local Gateway API Java process
2. Login to the LIVE IB account at
3. Send the HTTP request to the local Gateway and observed that this works.

So it works with live account but when I try demo account

1. Start the local Gateway API Java process
2. Login to the DEMO IB account at?
3. Send the HTTP request to the local Gateway and observed that it runs into Request failed with status code 500

The only difference between the two test cases is the fact that I logged into different accounts. Code and the HTTP request parameters are exactly the same.
If the market data is not available in the demo environment, how can I possibly run my algo and test it in demo before going live with risking real money?

I can see in my Axios debug print out the request is being sent correctly.
GET /v1/api/hmds/history?conid=586139726&period=5d&bar=1h&outsideRth=true

Re: Order Id in error messages

 

If I rephrase it for sake of my understanding:
In your experience, some request and at least reqPnL does act as an "On" switch about what is send back as "id" when an error is reported.
When this condition is triggered (trough for example a first successful? reqPnL) then any request like reqOrder that failed will report it's reqId instead of the Orderid.
Of course this impact only request that use an orderid, but allegedly impact every of them.
Right ?

What language are you using ?
Which version of the API ?
Are you connected as ClientID 0 ?

Speculation that come to my mind are:
1- This may happen on errors generated before IB server is even able to interpret the order ID in message sent. (this kind of error may contains the wording "....Java Parsing Error ....")
Then IB report what is the next easiest id at hand, which is the reqid.

If you use C++ with API > rev 10.XX I can imagine a reasons related to the way Decimal is used for quantity. (tough It's difficult to match your case and I don't see the link with reqPnL as a trigger)

2- You may be re-using an orderid of an already pending orderid (or use 0 ), then IB doesn't know what is the consistency of an error report that refer this orderId, as it was legit in the past, and IB then fall back on reqid. Never tried the case. So speculation here.
Farfetched but ...

It might be good to know more about a faulty 'Order oriented' request.
Which reqXXX failed , what are it's object content ? (What was the orderid and reqid ?)
What is the exact and full error message ?
If you make a reqAllOrders, are you sure your orderid is not in the reported orderid from openOrder call back ?
Does it happens on every subsequent order oriented request that error-ed ? or random and in an unpredictable manner ?

How to link 2 orders on 2 different contracts so that the 2nd one is executed only if the 1st one gets filled and have this sit on the IB side?

 

Hi,
Is there a way in the API to link two orders on two different options contracts so that the 2nd one is executed only if the 1st one gets filled?
So not at the same time like in a combo order.
?
Example:
Order 1: Buy 1 SPY 420 put
Order 2: Sell 1 SPY 415 put (only when order 1 is filled)
?
I want that to sit on the IB side and not have to monitor the execution in my code.
?
Thank you
F

Re: Order Id in error messages

 

As I said, as soon as I made reqPnLSingle request, all subsequent orders-related errors started to come with that reqId instead of orderId. There were no errors related to reqPnLSingle itself. I would not probably believe it if I had not seen it with my own eyes. As soon as I blocked the reqPnLSingle, everything was back to normal.
I can't rule out the possibility that something in my code was causing it, but it's still weird, considering the support confirmed it's normally expected behaviour.
I will try to recreate the conditions once I get a chance.

Re: updateMktDepthL2 missing data points

 

@fordfrog your description does not sound any less feature rich than what I an using. Pushing multi-threading to a lower level with topic queues is the right way to go and one of my roadmap items. And I fully agree with the use of provider, consumer, and connector interface abstractions to hide the TWS API details, for improved overall modularity, and to manage information flow even in applications that are not (yet ?) multi-threaded.
@´³¨¹°ù²µ±ð²Ô what i had on my mind is that your level of professionalism (which manifest through features like the extensive logging, possibility of replay, data analysis, overall debugability and many more) is much higher, the level that i would like to once achieve too, you inspire me :-)

Re: updateMktDepthL2 missing data points

 

I don't know enough about Python writing speed, but I am not sure you need multi-threading, because IB does buffer answers for you (to a limited extent but should be enough even with L2)
so you might make your life simpler doing your experiment with save (append) in same thread, as IB does hide for you it's own receiving thread.
Also you ought to to save 'Operation' in your CSV the only safe way to handle case 1,2 at a later stage.

Re: updateMktDepthL2 missing data points

 

I would not characterize your post as wasting the forum's time, John. I think it caused a nice collection of valuable information that already helped you and will likely help others when they interface TWS API with a multi-threaded application or attempt to implement completely asynchronous and event driven applications (as they should).

This is not that hard, but also not intuitively obvious. And as @buddy pointed out and @fordfrog reiterated, it is mostly about style and structure (architecture, design) and disciplined coding.

@fordfrog your description does not sound any less feature rich than what I an using. Pushing multi-threading to a lower level with topic queues is the right way to go and one of my roadmap items. And I fully agree with the use of provider, consumer, and connector interface abstractions to hide the TWS API details, for improved overall modularity, and to manage information flow even in applications that are not (yet ?) multi-threaded.

´³¨¹°ù²µ±ð²Ô


On Tue, Oct 31, 2023 at 02:10 PM, John wrote:
Thank you Jurgen, fordfrog, buddy for your replies, they are brilliant.

I realize now I wasted this forum's time on a data corruption issue due to multithreading, I hope these answers will be useful to others nevertheless.
This FX depth script is only for research purposes, the idea is just to collect a few day's worth of data and analyze it to see what alpha can be found if any.
So for now I used a bool flag that turns off writing for 0.1 seconds every hour during file saving, that's all and it works nicely for this humble purpose of sample data collection.

If/once I establish that this is something worth trading in production, I will definitely take a more robust approach for data collection, and will apply this feedback.
I have a larger app already in place for live trading which does have some of the suggested points, but it is incorrect on many levels raised here.
The below was especially useful to understand the right design approach, so thanks for that.

One way to eliminate the need for synchronization is to eliminate the global series object entirely:
  • Instead of storing the individual parameters from TWS API callbacks in lists with columns, define a class that carries all parameters for the callback and creates immutable objects that can be freely shared by all parts of your application.
  • Separate the data streams for multiple subscriptions/reqIds from each other so that they can operate in parallel and are not even aware of each other
  • Define a "consumer" interface that all classes implement that need to work with the data. In your case that could be
    • a MarketBook class that immediately handles the updates without having to store them in a list and that only remembers the last price and size update for each sell side and buy side slot (as @fordfrog suggested)
    • a FileLogger class that consumes the objects one at a time and saves them to file. This class could simply take advantage of "file output buffering" or keep a private list of a certain number of objects before it writes them to disk. That list is local to the logger and requires no synchronization. The logger class could also hide the fact that the logger actually runs in a separate thread. It would simply create a queue that the consumer interface feeds with data objects and that the logger thread reads and stores to file at its leisure and possibly with a lower priority than the real-time stream consumers.
    • There could be other users of the data that would simply implement the "consumer" interface as well.
    • Since each instance of the data is immutable, one and only one copy of the object exists at any point in time regardless of how may consumers or threads receive a copy.

Re: Order Id in error messages

 

As I concur with Jurgen, and looking at your last post I wonder if there isn't a misunderstanding.
Is it your reqPnLSingle that generate the error ?

Because your posts lead me to think that the error come back with the id of your reqPnLSingle? request id, then you should blame this request, and nothing related to an orderid.
The error message system does report to you the reqid of this faulty reqPnlSingle, right ?

General opinion:
reqPnlSingle is 'Positions' oriented, not 'Order' oriented, so what orderid(s) would you expect from a report trough an error in a reqPnlSingle? ?
Orders involved might be executed and objects expired a long time before you look at your Position PnL.

Note: (unsolicited suggestion)
As I need Orders level PnL (maybe this is what you look for here) for some deeper analysis, I did implement my own code for that using execDetails (because even a single order can be done with a plurality of transactions each at different price.
And aside of execDetails nothing did popup to me from IB literature as a smarter way to access this granularity of info (at least from API standpoint)

Re: Order Id in error messages

 

On Tue, Oct 31, 2023 at 08:40 PM, ´³¨¹°ù²µ±ð²Ô Reinold wrote:

You would not be the first person to receive incorrect information from good-meaning IBKR support staff.

+1

Re: Order Id in error messages

 

I hear you, but my tests clearly show that the absolute statement you received from IBKR support and as it is written in IBKR campus are not true. At least for my test scenarios.

And just to be sure, I added a few more unit tests to the suite such as using bogus or expired conIds for the reqPnlSingle call and using tradable instruments but cause a margin violation rejection. And the many test runs, without fail, all correctly returned the orderId as the error callback id.

You would not be the first person to receive incorrect information from good-meaning IBKR support staff. And the fact that they quoted the IBKR Campus text (which is in conflict with the official TWS API documentation) makes me even more suspicious about the response.

Can you share at least:
  • Details about the order (instrument, quantity, buy/sell, order type, simple order or complex order such as bracket etc)
  • Did you place the order when the market for the instrument was open, or during IBKR system maintenance periods over the weekend?
  • What was the order rejection reason and which instrument did your reqPnlSingle request PNL for?
  • Did you use the same orderId for the id parameter of the placeOrder call and the Order.orderId field, or did you leave the orderId field uninitialized?
  • What language, TWS API version, and TWS/IBGW version did you use?

Just trying to avoid that we spread incorrect information here. I am not prepared to believe this until I see TWS/IBGW logs showing that TWS/IBGW indeed sent an order rejection error message that contained requestIds from unrelated API requests. Maybe you could keep API message logging enabled in your TWS/IBGW for now and share the relevant log snippet in case you see the issue again.

´³¨¹°ù²µ±ð²Ô



On Tue, Oct 31, 2023 at 01:43 PM, bespalex wrote:
When I first encountered it a couple of days ago, I have raised a ticket with IB, so this is the exact quote from their reply after analyzing the logs they requested:
"the way error callback operates for the order rejection is by utilizing the reqID of the last request made via the API.
For more details on how the error callbacks handles reqID, you may explore the given link below:
"
Likewise, I have found out about ibkrcampus from them.
As soon as I excluded reqPnLSingle from my code everything was back to normal. I have tested it several times turning it back on and forth off.
Unfortunately, I did not save the initial chat with IB support that contained pieces of evidence. If I have a moment I will try to protocol what's happening.

2181 - 2200 of 53811