¿ªÔÆÌåÓý

ctrl + shift + ? for shortcuts
© 2025 Groups.io
Date

Re: VIX Futures trading

 

On Thu, Aug 10, 2023 at 01:45 PM, John wrote:
h future individually, also there is a lot more liquidity, than buying/selling the spread outright.
I think that the bid-ask spread during regular trading hours is more like 4-6 cents and not 10. its still quite expensive since 10 cents equals full spread on both individual legs. Based on your personal experience how much does it cost to trade a spread using individual legs as opposed to spreads. I am aware this isnt a TWS API question so forgive if I broke some rule here.


Re: Performance optimization when exporting data to excel

 

Well, weight in the fact that I am not a fan of Excel for handling fast pace data. (and here you deal with 'ticks' which is rated 2nd fastest data set of the API) Maybe other members have better experience.
Typically what seems an easy start may end up pretty painful, spending precious hours tweaking and 'workarounding' Excel limitations that will eventually fail to your expectations.

IMHO and without understanding if your purpose is to get an Excel spreasheet to look at, or to use Excel to compute and takje decision. Anyway you better organize "rendez-vous" in your code and get ride of "loop".
In short: you need a multi-thread method. yours seems single threaded. C# have good provision for it.

Assuming you never stop the 4th loop. Look to me that you need a 5th 'loop' (a thread in real), this is probably the most important for you within your current architecture, and it must be in a separate thread, this one only handle saving data.
Using SQL could be a simple solution as this will allow you to save faster (maybe) and more important collecting data from a central repository that by itself have all provision to handle concurrency of data access. This would preserve most of your existing code.

However depending upon how serious is your project you should use more of an appropriate language like Java or C# (in full) or C++ , it's not so hard to get some fundamental and it pay.
The API allows you to start any request you mention asynchronously and ASAP, Scanner answers are pretty slow, can be done at this moment. Be careful with contractdetails you may hit many pacing limitations.
IB have a pretty efficient and fast API, you can't keep up using Excel as the orchestrator.

Seems you use C#, keep the effort, if you don't know how to do multi-threading, start learning how to launch a thread, then either connect to a MySQL or learn how to share data between thread using a mutex, then ... many many details but later.


Re: Level2 data falls behind actual market

 

csv solved the problem. You were absolutely correct and saved me a boat load of money.? Much appreciated.? You are my new buddy!

On Tue, Aug 22, 2023 at 6:55 PM ebtrader via <jsiddique=[email protected]> wrote:
Using tail it is matching up with current market pricing in the after hours.? I will let you know if it works during regular market hours.

On Tue, Aug 22, 2023 at 6:35 PM ebtrader via <jsiddique=[email protected]> wrote:
I will try this file append.? I have an SSD drive.? No computations are happening. ?

On Sat, Aug 12, 2023 at 1:25 PM buddy <spy@...> wrote:

On Sat, Aug 12, 2023 at 02:01 PM, ebtrader wrote:

My buddy ran it with a laptop that has an Nvidia 2 series GeForce and it worked fine without falling behind.? I will upgrade my machine and then it should work.

Not THIS Buddy!

Please, I'm begging you, at least try dumping the data to a file first and watching it with . If you don't have tail, find it... in the WSL or install cygwin.

The test should be easy as pie, don't overthink it, just comment/ifdef out the SQL insert and replace it with a file append. Unless your computer has floppies with a 10Mbit network (i.e. ancient) everything should keep up. Then tune the db as previously suggested.

You haven't mentioned any kind of real-time computation happening on the CPU or GPU so the process is basically all network and disk I/O bound. If you need to upgrade anything it will be the network and/or disk first, and only then the [CG]PU; assuming you want to keep the data.

Be reasonable! With interest rates this high who can afford unnecessary capex?


Performance optimization when exporting data to excel

 




? I have a stock scanner project where I am working with TWS C# API to get data based on certain price parameters.

the data gets callbacked in several methods which I then am using with EPPlus to open and update my excel file.

the data comes back in the following order

1st loop (request stock symbols from API) using?reqScannerSubscription


for each symbol we have another loop to get contract details from API, (2nd loop) using reqcontractdetails

for each contract we request market data from API (3rd loop) using reqMktdata

for each market data we store as variable (4th loop) The callbacks from tickprice ticksize etc...
each method is a public virtual void method

for loop 2-4 we are opening and editing an excel file with epplus


? ? using (var package = new ExcelPackage(pathtofile))
? ? {
? ? ? ? // edit excel file with the data callback from API
? ?
? ? ? ? package.Save();
? ? }
Instead of opening and saving the excel file in each method, which costs about 5 seconds? for each iteration?

1) I could instead pass the variables to the last method in a array and then do it there.
2) I could use `LoadFromCollection` of Epplus instead of calling each cell, currently I am calling each cell like so

? ? worksheet.Cells[row, 1].Value = mycelldata; where row is sheet.Dimension.End.Row + 1
3) I could use sql database instead of using excel, (keep in mind I don't plan on having large databases (less then 50,000 entries)

has anyone had any experience on my 3 previous methods or would like to suggest something new to speed up performance??


Re: Level2 data falls behind actual market

 

Using tail it is matching up with current market pricing in the after hours.? I will let you know if it works during regular market hours.

On Tue, Aug 22, 2023 at 6:35 PM ebtrader via <jsiddique=[email protected]> wrote:
I will try this file append.? I have an SSD drive.? No computations are happening. ?

On Sat, Aug 12, 2023 at 1:25 PM buddy <spy@...> wrote:

On Sat, Aug 12, 2023 at 02:01 PM, ebtrader wrote:

My buddy ran it with a laptop that has an Nvidia 2 series GeForce and it worked fine without falling behind.? I will upgrade my machine and then it should work.

Not THIS Buddy!

Please, I'm begging you, at least try dumping the data to a file first and watching it with . If you don't have tail, find it... in the WSL or install cygwin.

The test should be easy as pie, don't overthink it, just comment/ifdef out the SQL insert and replace it with a file append. Unless your computer has floppies with a 10Mbit network (i.e. ancient) everything should keep up. Then tune the db as previously suggested.

You haven't mentioned any kind of real-time computation happening on the CPU or GPU so the process is basically all network and disk I/O bound. If you need to upgrade anything it will be the network and/or disk first, and only then the [CG]PU; assuming you want to keep the data.

Be reasonable! With interest rates this high who can afford unnecessary capex?


Re: Level2 data falls behind actual market

 

I will try this file append.? I have an SSD drive.? No computations are happening. ?

On Sat, Aug 12, 2023 at 1:25 PM buddy <spy@...> wrote:

On Sat, Aug 12, 2023 at 02:01 PM, ebtrader wrote:

My buddy ran it with a laptop that has an Nvidia 2 series GeForce and it worked fine without falling behind.? I will upgrade my machine and then it should work.

Not THIS Buddy!

Please, I'm begging you, at least try dumping the data to a file first and watching it with . If you don't have tail, find it... in the WSL or install cygwin.

The test should be easy as pie, don't overthink it, just comment/ifdef out the SQL insert and replace it with a file append. Unless your computer has floppies with a 10Mbit network (i.e. ancient) everything should keep up. Then tune the db as previously suggested.

You haven't mentioned any kind of real-time computation happening on the CPU or GPU so the process is basically all network and disk I/O bound. If you need to upgrade anything it will be the network and/or disk first, and only then the [CG]PU; assuming you want to keep the data.

Be reasonable! With interest rates this high who can afford unnecessary capex?


Linking an automatically exercised option to an underlying position

 

Hi all,

I'm looking to implement an options strategy which will BUY/SELL back options on expiration date to exit the position and avoid automatic exercising, however to build in redundancy for options which do get exercised through either technical issues on expiration day (WiFi down, application issues), or low-liquidity environments, it would be ideal to link a trade/position back to the exercised option through a field such as a permId. Due to my setup (and seemingly not able to distinguish position by client id), the resultant position in the underlying would be mixed with other existing positions in that security.

Ideally the `Trade.log` or `Trade.fills` (ib_insync) class variables for the option would recieve additional executions/logs to capture that it was exercised, but I don't believe it does this?

Any help on how others handle this case would be greatly appreciated! Many thanks,

Aidan


Re: MidPrice algo syntax, and other algo syntax

 

Bump.

From the FIX API, it appears that it requires the use of?MIDPX as order type instead of LMT / MKT.
That being said it is not documented anywhere in the TWS API.

Has anyone used it successfully?

Also how to know which exchange is it accepted with? I understand it only applies to US stocks.


Re: Can I place orders asyncronously like this without causing issues with the twsapi?

 

That sounds like a standard answer from a tier 1 support person. I would consider the counterpoints..

It's true there's a global interpreter lock, which may be in Python 3. It's also true that async code is single threaded, but in this circumstance you still need to be careful where you await. Finally, neither you nor IB mentioned anything about how many different instances of this program you may be running... if it's more than one you could certainly have a problem.

So again, since you haven't provided a picture of the whole system, there's no great point in speculation.

Of course testing is worthwhile, but static code analysis and formal verification are different beasts altogether. And, this is why I simply said "You might have problems but I won't conjecture on their likelihood."

I know my reply was somewhat esoteric. Therefore, to make up for that a tiny bit, I'd suggest that you use the "+= 1" idiom instead of adding constants to your order ID. In addition, you'll want to learn about different methods of doing an atomic increment in Python as well as how to leverage critical sections. You should be able to significantly "future proof" your code when and if necessary with those two things.

Still, these are all Python related (and concurrent programming) issues in general which the API can't address for you. Although I greatly enjoy the subject, it's a bit off-topic for this email group.


Re: Gateway loses historical market data every day

 

Ok, thanks! I need some time to process that. @Gordon Eldest No it wasn't exactly 300.
?


Re: Can I place orders asyncronously like this without causing issues with the twsapi?

 

I contacted IB, they told me the following:

Concurrency assumes that two task may overlap each other in execution. In python GIL(Global Interpreter Lock) prevents multiple threads from accessing the same python object. Only one thread can hold the GIL at a time and one thread must wait for another thread to release the GIL before running.

If i understand everything right - your code will run one thread that will asynchronously place bracket orders.In this case this task is still single-threaded but asynchronous and you should not encounter any issues with order id's if they are incremented during the execution of run_loop().

So anyway, I tested my code yesterday on paper like 20 times and it seems to work fine, I checked there were no order id clashes.

I also made another version where there was a second queue inside the function for the orders themselves, meaning if 2 api calls for price came back at the same time, the orders will be queued up, but it works exactly the same as the first version I described. Will try it again on paper today.


Re: Can I place orders asyncronously like this without causing issues with the twsapi?

 

On Sun, Aug 20, 2023 at 10:26 AM, omgwtfsalty@... wrote:

I am worried that I will place to orders at the exact same time and the order id's will get messed up.

You might have problems but I won't conjecture on their likelihood. They may not be exactly as you've described either, but there's no way to know much more from just a snippet of code. Details of the rest of the system need to be provided and considered... e.g. is this running in wsgi?

On Sun, Aug 20, 2023 at 10:26 AM, omgwtfsalty@... wrote:

Is there a way around this

Yes... of course. Potential solutions, however, have much more to do with Python and concurrent programming in general than they do with the IBKR API.

Check out my profile and follow the link to MIT's 6.031 class if you want to learn more about those topics. There's also a video by David Beasley in my profile... it's really great! But before you check those out, I suggest you sit through the 1986 videos of SICP and work through the book at the same time.... it's a masterpiece IMHO.

On Sun, Aug 20, 2023 at 10:26 AM, omgwtfsalty@... wrote:

does twsapi automatically mitigate this

Not really, this kind of thing is the responsibility of the end user. With a little work I'm sure you'll be able to come with something. Good luck!


Re: Could this happen: stp and trail in oca group both getting triggered?

 

Thats is great information again ´³¨¹°ù²µ±ð²Ô! I remember the post you are referring but I never really paid attention or understood the risc of overfilling, assuming the default behavior of oca would be sufficient.?

Thanks again, and will certainly add this OcaType to my oca orders. Probably setting it to 1.

Thanks again for your much appreciated help, greetings,
Raoul.

On Mon, Aug 21, 2023, 18:11 ´³¨¹°ù²µ±ð²Ô Reinold via <TwsApiOnGroupsIo=[email protected]> wrote:
We just had a different post where OCA overfill protection was discussed, Raoul. When you create the OCA group, do you set the attribute for each order or do you let IBKR use a default?

Chances are, that your OCA was of type 3 (see table at the bottom of the chapter) which does not protect against overfill. So you might need to set ocaType 1 or type 2 for your orders explicitly.

´³¨¹°ù²µ±ð²Ô

On Mon, Aug 21, 2023 at 10:51 AM, Raoul Suurmeijer wrote:
Hello all,
?
Just minutes ago, something strange happened in my algo trading. My algo always submits orders as an oca order. Parent being a limit order, connected are 3 child orders: stp, trail and mkt with time control. The trail is set much wider than the stp but I want to limit the max loss with the fixed stp.
?
In the call back after placing the MNQ order, size 1, I can clearly see all the order id's being used, all child orders in the same oca group. Everything perfect as expected and I am positioned +1 MNQ. The positions goes green, and the trail is going along.
?
At some point the stp and trail are at the same level. Then price drops and hits that level. You would expect that either of the two (stp or trail) to get executed and not the others. But afterwards I am left with -1 MNQ position. In the logging I see callbacks for executed orders for both the stp and trail causing a double closing sell. Again, all the order id's match with the initial orders as part of the same parent.
?
Is this behavior to be expected when using both a stp and trail in the same oca group and are at the same price level? If so, my algo should probably change stp level as soon as trail gets to the same level. Or remove it from the oca group (if that's possible).
?
Thanks in advance for sharing your insights and own experience, greetings,
?
Raoul
?
?
?


Re: Could this happen: stp and trail in oca group both getting triggered?

 

if i get it right, only one order is active at the target market, but ib monitors the other orders and activates the one when its rule is triggered and deactivates the one that was active so far.


Re: Could this happen: stp and trail in oca group both getting triggered?

 

In this case it could work but then what's the point of using oca group if just one order of the group is sent?


El lun, 21 ago 2023 a las 18:58, fordfrog via (<fordfrog=[email protected]>) escribi¨®:
what i recall, the blocking causes that only one order from the oca group is really submitted to the target market to be sure the overfill protection will work. it can be verified by color coding of the orders in tws.


Re: Could this happen: stp and trail in oca group both getting triggered?

 

what i recall, the blocking causes that only one order from the oca group is really submitted to the target market to be sure the overfill protection will work. it can be verified by color coding of the orders in tws.


Re: Could this happen: stp and trail in oca group both getting triggered?

 

I think it is not guaranteed even with the block option of the ocatype flag. It can execute two orders of the OCA at the exact same time. Check? ?

"Please note that because the OCA procedure is an automated process, there is no guarantee that requested cancellations and modifications will reach the specific exchange before an order has been executed."

El lun, 21 ago 2023 a las 18:11, ´³¨¹°ù²µ±ð²Ô Reinold via (<TwsApiOnGroupsIo=[email protected]>) escribi¨®:

We just had a different post where OCA overfill protection was discussed, Raoul. When you create the OCA group, do you set the attribute for each order or do you let IBKR use a default?

Chances are, that your OCA was of type 3 (see table at the bottom of the chapter) which does not protect against overfill. So you might need to set ocaType 1 or type 2 for your orders explicitly.

´³¨¹°ù²µ±ð²Ô

On Mon, Aug 21, 2023 at 10:51 AM, Raoul Suurmeijer wrote:
Hello all,
?
Just minutes ago, something strange happened in my algo trading. My algo always submits orders as an oca order. Parent being a limit order, connected are 3 child orders: stp, trail and mkt with time control. The trail is set much wider than the stp but I want to limit the max loss with the fixed stp.
?
In the call back after placing the MNQ order, size 1, I can clearly see all the order id's being used, all child orders in the same oca group. Everything perfect as expected and I am positioned +1 MNQ. The positions goes green, and the trail is going along.
?
At some point the stp and trail are at the same level. Then price drops and hits that level. You would expect that either of the two (stp or trail) to get executed and not the others. But afterwards I am left with -1 MNQ position. In the logging I see callbacks for executed orders for both the stp and trail causing a double closing sell. Again, all the order id's match with the initial orders as part of the same parent.
?
Is this behavior to be expected when using both a stp and trail in the same oca group and are at the same price level? If so, my algo should probably change stp level as soon as trail gets to the same level. Or remove it from the oca group (if that's possible).
?
Thanks in advance for sharing your insights and own experience, greetings,
?
Raoul
?
?
?


Re: Could this happen: stp and trail in oca group both getting triggered?

 

We just had a different post where OCA overfill protection was discussed, Raoul. When you create the OCA group, do you set the attribute for each order or do you let IBKR use a default?

Chances are, that your OCA was of type 3 (see table at the bottom of the chapter) which does not protect against overfill. So you might need to set ocaType 1 or type 2 for your orders explicitly.

´³¨¹°ù²µ±ð²Ô


On Mon, Aug 21, 2023 at 10:51 AM, Raoul Suurmeijer wrote:
Hello all,
?
Just minutes ago, something strange happened in my algo trading. My algo always submits orders as an oca order. Parent being a limit order, connected are 3 child orders: stp, trail and mkt with time control. The trail is set much wider than the stp but I want to limit the max loss with the fixed stp.
?
In the call back after placing the MNQ order, size 1, I can clearly see all the order id's being used, all child orders in the same oca group. Everything perfect as expected and I am positioned +1 MNQ. The positions goes green, and the trail is going along.
?
At some point the stp and trail are at the same level. Then price drops and hits that level. You would expect that either of the two (stp or trail) to get executed and not the others. But afterwards I am left with -1 MNQ position. In the logging I see callbacks for executed orders for both the stp and trail causing a double closing sell. Again, all the order id's match with the initial orders as part of the same parent.
?
Is this behavior to be expected when using both a stp and trail in the same oca group and are at the same price level? If so, my algo should probably change stp level as soon as trail gets to the same level. Or remove it from the oca group (if that's possible).
?
Thanks in advance for sharing your insights and own experience, greetings,
?
Raoul
?
?
?


Could this happen: stp and trail in oca group both getting triggered?

 

Hello all,

Just minutes ago, something strange happened in my algo trading. My algo always submits orders as an oca order. Parent being a limit order, connected are 3 child orders: stp, trail and mkt with time control. The trail is set much wider than the stp but I want to limit the max loss with the fixed stp.

In the call back after placing the MNQ order, size 1, I can clearly see all the order id's being used, all child orders in the same oca group. Everything perfect as expected and I am positioned +1 MNQ. The positions goes green, and the trail is going along.

At some point the stp and trail are at the same level. Then price drops and hits that level. You would expect that either of the two (stp or trail) to get executed and not the others. But afterwards I am left with -1 MNQ position. In the logging I see callbacks for executed orders for both the stp and trail causing a double closing sell. Again, all the order id's match with the initial orders as part of the same parent.

Is this behavior to be expected when using both a stp and trail in the same oca group and are at the same price level? If so, my algo should probably change stp level as soon as trail gets to the same level. Or remove it from the oca group (if that's possible).

Thanks in advance for sharing your insights and own experience, greetings,

Raoul




Re: Can I place orders asyncronously like this without causing issues with the twsapi?

 

I am not familiar enough with Python but a couple of things maybe be generic

1- you have a side 'thread' that deliver you a condition that look like a HTTP return code, right ?
aiohttp.ClientSession() which will use same credentials, hence same session as your own main thread.
2- your own thread will place a bracket order (and look like you understood the purpose of 'transmit')
Your concern is that your order id will be messed up, but with what ?

Your code should better take some precaution as you seems to me to mix 2 different technologies, the SDK and the REST.
You better call self.reqIds(-1) at least once yourself and track the call back of nextValidId before you use app.nextOrderId().?
Read the sequence (init+operation+report) in
and consider the 2 call back openOrder and orderStatus. you SHOULD listen to it.
Start keeping thing simple, there is no harm to call self.reqIds(-1) and it is fast