开云体育

ctrl + shift + ? for shortcuts
© 2025 Groups.io

TWS 50/sec limit


 

I need to get options chains for an underlying. Tws has a 50/sec limit on the data. The api loses connection also. What is the fix?


 

Limit requests to 50/sec. There is no other option.

2015-12-23 12:53 GMT+01:00 todell826@... [TWSAPI] <TWSAPI@...>:

?

I need to get options chains for an underlying. Tws has a 50/sec limit on the data. The api loses connection also. What is the fix?




--
Best regards,
Chernikov Aleksandr,
e-mail:?me@...
skype:?
RU:?+7-904-179-49-78


 

Can I filter by Delta and days till expiration to reduce the amount of data? Can I slow down the download of data. This roadblock is hard to believe. Options chains are basic info.


 

Can I get the data from another source and use it in my TWS API.?


rholowczak
 

Maybe I am not the only one with this question: When you say "option chains" are you only looking for the specifications (strike, expiration, right (call/put)) of options currently trading on an underlying?
Or are you looking real-time or historical for Bid and Ask prices, last trade prices, etc. for all of those option series?


Nick
 

开云体育

Disconnects are caused by an error in your program.

TWS does not disconnect for pacing violations.

There is no limit on the amount or speed of data sent by TWS to the API program.

Trying to create your own reality and then expect the computer to care about that reality will always fail.

On 12/23/2015 7:10 AM, todell826@... [TWSAPI] wrote:

?

Can I filter by Delta and days till expiration to reduce the amount of data? Can I slow down the download of data. This roadblock is hard to believe. Options chains are basic info.



 

开云体育

I found that 2 tws? never disconnect
tws also does not send all option data when requested
lots of? 0 on the bid /ask if 0 on the bid i usually take the strike -current + current or something like that



On 12/23/2015 11:07 AM, Nick news1000@... [TWSAPI] wrote:

Disconnects are caused by an error in your program.

TWS does not disconnect for pacing violations.

There is no limit on the amount or speed of data sent by TWS to the API program.

Trying to create your own reality and then expect the computer to care about that reality will always fail.

On 12/23/2015 7:10 AM, todell826@... [TWSAPI] wrote:
?

Can I filter by Delta and days till expiration to reduce the amount of data? Can I slow down the download of data. This roadblock is hard to believe. Options chains are basic info.




 

Please read my post in topic "Building an options trading algo using TWS. Need help". It explains in detail what I am trying to achieve.


 

rholowczak

I am using the Greeks for trade decisions.


 

I just requested the full option chain (all expiries) for the SPX.? The system returned 7646 contracts with an elapsed time of about one minute.? If you're getting timeouts or disconnects, I suspect you are doing something wrong.

The option chains are "Contract" objects which contain information about the item, probably the most important of which is the contract ID.? You then use the contract ID to subscribe price quotes, Greeks, etc.

[rwk]



At 06:19 PM 12/23/2015, you wrote:

Please read my post in topic "Building an options trading algo using TWS. Need? help". It explains in detail what I am trying to achieve.


 

If you connect via IB Gateway instead of TWS, you will be able to make about 120 requests/sec (don't have the exact number right now).


 

Confirming: java demo client loses connection when "Options" -> "Option
Chains" tab used to get all calls/puts for "IBM".

But it is just a poor design/implementation of java demo application itself.

*Short description*
When option chains coming back from IB each of contract added into new
table row and then request for market data for that contract fired by java
demo app right away. Obviously since there are hundreds of options for IBM
the above logic creates a few problems by completely uncontrollable
requests rate along with an attempt to subscribe to uncontrollable number
of tickers (more than 100 in our case, which is well known
poor-man-IB-imposed-limit:)

*Long description aka how to reproduce:*

Download and install the latest stable TWS:
on my mac I end up with this installer file: tws-stable-macosx-x64.dmg
MD5 (tws-stable-macosx-x64.dmg) = 38e3c828d2939752e34af3bd8fd60df4

Get latest stable IB API by visiting:
"IB API for Mac/Unix" Version: API 9.71 Release Date: April 24 2015
MD5 (twsapi_macunix.971.01.jar) = 6b8ed9a34a38d7bab329f2c533d478f3

# extract files:
jar xf twsapi_macunix.971.01.jar

# then follow this doc to run it: "Running the Java Test Client Sample
Program"


run java demo app
click "options" tab and leave default values (IBM)

click Go, after about 1-2 sec there's couple tables (Calls and Puts)
populated and in the log window of demo app we see that we're actually got
disconnected!

TWS will show following window:
You have requested data for 106 contracts simultaneously, which exceeds
your account current limit of 100. You may bla bla. . .
[image: Inline image 1]

Btw, java app will not get disconnected if we select "Use snapshot data"
checkbox.
If you wonder why, see this passage in the OptionChainsPanel.java file:

(quote)
if (snapshot) {
Util.sleep( 11); // try to avoid pacing violation at TWS
}

(end of quote)

the above lousy "sleep" is in milliseconds, which means it wont allow to
send more than 90 requests per second and it might work for some simple
solutions.

good luck,
Dmitry


On Thu, Dec 24, 2015 at 2:17 AM, tron001@... [TWSAPI] <
TWSAPI@...> wrote:



If you connect via IB Gateway instead of TWS, you will be able to make
about 120 requests/sec (don't have the exact number right now).



--
с ч.гл.ув.,
Дима Ш.

Dmitry's
<>
TWS API FAQ <>


[Non-text portions of this message have been removed]


rholowczak
 

Right- ?and based on other replies, my suggestion is that you separate out the functions into two steps:
1) Retrieving all of the contract ID's for all of the options series on a given underlying
2) Subscribing to market data for each of those IDs.

It is possible if you are attempting to combine these two operations at the same time (as the API demo application does - get a contractID and then subscribe to market data immediately) then you will hit the pacing violation.

I would suspect that there are some option series that your trading algorithm will not care about such as LEAPS and/or strike prices deep in the money or deep out of the money.

By splitting up the two operations, you can first build a table of all of the option series, then filter this according to what your application requires. ?Once you have that (hopefully much smaller) list of contractID's, then you can run another method to subscribe to the appropriate market data paying specific attention to the pacing limitations.



 

@Nick news1000,

I'm not sure how important it is, but the above exercise with java demo app
also shows that it is actually TWS who initiate the disconnect when client
brakes some rules (number of realtime subscribed tickers or exceed request
rate limits).

So then statement "Disconnects are caused by an error in your program."
seems to be not 100% right since there's no language/functionality scope
error - I mean the program itself is clean and does not crash nor does it
any wrong calls, but rather logical error (pacing) related to timing can
occur due to some reason ant then disconnect will be caused by TWS.

And then 2nd statement "TWS does not disconnect for pacing violations." is
not true. What I saw in described java demo app was:

*case-1: requested too many contracts simultaneously*
TWS does disconnect client when it brakes a limit (number of simultaneously
subscribed tickers).
TWS also brings nice modal popup alert with explanation as of why you've
been kicked out :)

*case-2: sent too many requests/second*
Then to check if similar disconnect (by TWS) happens in case of request
pacing I modified the demo app to send some bulk (about 950:)
reqContractDetails and saw similar behaviour: client kicked out by TWS
after 3 error/notes:

Max rate of messages per second has beein exceeded:max=50 rec=83 (1)
Max rate of messages per second has beein exceeded:max=50 rec=85 (2)
Max rate of messages per second has beein exceeded:max=50 rec=83 (3)
disconnected

D

ps: case-1 sometimes does not kick client out (if I subscribe to few
hundred of options with rate less than 50 per second to avoid pacing), but
I only tested it once around 1am eastern, so inconsistency may be due to
"funny time".

On Thu, Dec 24, 2015 at 2:43 AM, Dmitry Shevkoplyas <shevkoplyas@...>
wrote:

Confirming: java demo client loses connection when "Options" -> "Option
Chains" tab used to get all calls/puts for "IBM".

But it is just a poor design/implementation of java demo application
itself.

*Short description*
When option chains coming back from IB each of contract added into new
table row and then request for market data for that contract fired by java
demo app right away. Obviously since there are hundreds of options for IBM
the above logic creates a few problems by completely uncontrollable
requests rate along with an attempt to subscribe to uncontrollable number
of tickers (more than 100 in our case, which is well known
poor-man-IB-imposed-limit:)

*Long description aka how to reproduce:*

Download and install the latest stable TWS:
on my mac I end up with this installer file: tws-stable-macosx-x64.dmg
MD5 (tws-stable-macosx-x64.dmg) = 38e3c828d2939752e34af3bd8fd60df4

Get latest stable IB API by visiting:

"IB API for Mac/Unix" Version: API 9.71 Release Date: April 24 2015
MD5 (twsapi_macunix.971.01.jar) = 6b8ed9a34a38d7bab329f2c533d478f3

# extract files:
jar xf twsapi_macunix.971.01.jar

# then follow this doc to run it: "Running the Java Test Client Sample
Program"



run java demo app
click "options" tab and leave default values (IBM)

click Go, after about 1-2 sec there's couple tables (Calls and Puts)
populated and in the log window of demo app we see that we're actually got
disconnected!

TWS will show following window:
You have requested data for 106 contracts simultaneously, which exceeds
your account current limit of 100. You may bla bla. . .
[image: Inline image 1]

Btw, java app will not get disconnected if we select "Use snapshot data"
checkbox.
If you wonder why, see this passage in the OptionChainsPanel.java file:

(quote)
if (snapshot) {
Util.sleep( 11); // try to avoid pacing violation at TWS
}

(end of quote)

the above lousy "sleep" is in milliseconds, which means it wont allow to
send more than 90 requests per second and it might work for some simple
solutions.

good luck,
Dmitry


On Thu, Dec 24, 2015 at 2:17 AM, tron001@... [TWSAPI] <
TWSAPI@...> wrote:



If you connect via IB Gateway instead of TWS, you will be able to make
about 120 requests/sec (don't have the exact number right now).



--
с ч.гл.ув.,
Дима Ш.

Dmitry's
<>
TWS API FAQ <>


--
с ч.гл.ув.,
Дима Ш.

Dmitry's
<>
TWS API FAQ <>


[Non-text portions of this message have been removed]


 

All, I did extensive work with this java client and found it very frustrating.

I first added an expiration date to the contract definition and it began to work. This confirmed that the request exceeded the limits.

Then I created an elaborate (for me as a novice programmer) procedure to find the ATM strike and then figure out how far OTM I wanted to request. Then I was able to just put in a short sleep and change the expiry date to the next week in a loop and started to get some useful parts of the chain. The weakness to this is that I had to hardcode the expiry date and like today and next week, it's not always on a Friday. All of my work was tested using the demo acct so all this might not mean much.

The original poster is correct that IB wrote the client GUI and it is poorly designed regarding the option chain implementation.

Patrick

On Dec 24, 2015 11:21 PM, "Dmitry Shevkoplyas shevkoplyas@... [TWSAPI]" <TWSAPI@...> wrote:
>
> ?
>
> @Nick news1000,
>
> I'm not sure how important it is, but the above exercise with java demo app
> also shows that it is actually TWS who initiate the disconnect when client
> brakes some rules (number of realtime subscribed tickers or exceed request
> rate limits).
>
> So then statement "Disconnects are caused by an error in your program."
> seems to be not 100% right since there's no language/functionality scope
> error - I mean the program itself is clean and does not crash nor does it
> any wrong calls, but rather logical error (pacing) related to timing can
> occur due to some reason ant then disconnect will be caused by TWS.
>
> And then 2nd statement "TWS does not disconnect for pacing violations." is
> not true. What I saw in described java demo app was:
>
> *case-1: requested too many contracts simultaneously*
> TWS does disconnect client when it brakes a limit (number of simultaneously
> subscribed tickers).
> TWS also brings nice modal popup alert with explanation as of why you've
> been kicked out :)
>
> *case-2: sent too many requests/second*
> Then to check if similar disconnect (by TWS) happens in case of request
> pacing I modified the demo app to send some bulk (about 950:)
> reqContractDetails and saw similar behaviour: client kicked out by TWS
> after 3 error/notes:
>
> Max rate of messages per second has beein exceeded:max=50 rec=83 (1)
> Max rate of messages per second has beein exceeded:max=50 rec=85 (2)
> Max rate of messages per second has beein exceeded:max=50 rec=83 (3)
> disconnected
>
> D
>
> ps: case-1 sometimes does not kick client out (if I subscribe to few
> hundred of options with rate less than 50 per second to avoid pacing), but
> I only tested it once around 1am eastern, so inconsistency may be due to
> "funny time".
>
> On Thu, Dec 24, 2015 at 2:43 AM, Dmitry Shevkoplyas <shevkoplyas@...>
> wrote:
>
>
> > Confirming: java demo client loses connection when "Options" -> "Option
> > Chains" tab used to get all calls/puts for "IBM".
> >
> > But it is just a poor design/implementation of java demo application
> > itself.
> >
> > *Short description*
> > When option chains coming back from IB each of contract added into new
> > table row and then request for market data for that contract fired by java
> > demo app right away. Obviously since there are hundreds of options for IBM
> > the above logic creates a few problems by completely uncontrollable
> > requests rate along with an attempt to subscribe to uncontrollable number
> > of tickers (more than 100 in our case, which is well known
> > poor-man-IB-imposed-limit:)
> >
> > *Long description aka how to reproduce:*
> >
> > Download and install the latest stable TWS:
> > on my mac I end up with this installer file: tws-stable-macosx-x64.dmg
> > MD5 (tws-stable-macosx-x64.dmg) = 38e3c828d2939752e34af3bd8fd60df4
> >
> > Get latest stable IB API by visiting:
> >
> > "IB API for Mac/Unix" Version: API 9.71 Release Date: April 24 2015
> > MD5 (twsapi_macunix.971.01.jar) = 6b8ed9a34a38d7bab329f2c533d478f3
> >
> > # extract files:
> > jar xf twsapi_macunix.971.01.jar
> >
> > # then follow this doc to run it: "Running the Java Test Client Sample
> > Program"
> >
> >
> >
> > run java demo app
> > click "options" tab and leave default values (IBM)
> >
> > click Go, after about 1-2 sec there's couple tables (Calls and Puts)
> > populated and in the log window of demo app we see that we're actually got
> > disconnected!
> >
> > TWS will show following window:
> > You have requested data for 106 contracts simultaneously, which exceeds
> > your account current limit of 100. You may bla bla. . .
> > [image: Inline image 1]
> >
> > Btw, java app will not get disconnected if we select "Use snapshot data"
> > checkbox.
> > If you wonder why, see this passage in the OptionChainsPanel.java file:
> >
> > (quote)
> > if (snapshot) {
> > Util.sleep( 11); // try to avoid pacing violation at TWS
> > }
> >
> > (end of quote)
> >
> > the above lousy "sleep" is in milliseconds, which means it wont allow to
> > send more than 90 requests per second and it might work for some simple
> > solutions.
> >
> > good luck,
> > Dmitry
> >
> >
> > On Thu, Dec 24, 2015 at 2:17 AM, tron001@... [TWSAPI] <
> > TWSAPI@...> wrote:
> >
> >>
> >>
> >> If you connect via IB Gateway instead of TWS, you will be able to make
> >> about 120 requests/sec (don't have the exact number right now).
> >>
> >>
> >
> >
> >
> > --
> > с ч.гл.ув.,
> > Дима Ш.
> >
> > Dmitry's
> > <>
> > TWS API FAQ <>
> >
> >
>
> --
> с ч.гл.ув.,
> Дима Ш.
>
> Dmitry's
> <>
> TWS API FAQ <>
>
>
>
>


Nick
 

It is good you point this out. I should have been clear that my post was directed to the OP's claim that he could not download an option chain due to a fault in TWS.

Regarding "Disconnects are caused by an error in your program." - I suppose we could debate the meaning of "error" but I have no desire for that. I was trying to say that if he was getting a disconnect then he should be looking at his program for the cause, not TWS.

Regarding "TWS does not disconnect for pacing violations." - This was directed at his assertion that a large option chain reply was causing a pacing violation and that in turn was causing the disconnection. I was trying to say that pacing violations do not apply to replies sent by TWS. Although there are some pacing violations that don't cause disconnects (historical requests return a pacing violation error message) I should not have made an unqualified declaration.

So, thanks for being thorough and making things clear for those who may read these posts in the future. I will put effort into making posts that stand on their own.

On 12/25/2015 2:20 AM, Dmitry Shevkoplyas shevkoplyas@... [TWSAPI] wrote:
@Nick news1000,

I'm not sure how important it is, but the above exercise with java demo app
also shows that it is actually TWS who initiate the disconnect when client
brakes some rules (number of realtime subscribed tickers or exceed request
rate limits).

So then statement "Disconnects are caused by an error in your program."
seems to be not 100% right since there's no language/functionality scope
error - I mean the program itself is clean and does not crash nor does it
any wrong calls, but rather logical error (pacing) related to timing can
occur due to some reason ant then disconnect will be caused by TWS.

And then 2nd statement "TWS does not disconnect for pacing violations." is
not true. What I saw in described java demo app was:

*case-1: requested too many contracts simultaneously*
TWS does disconnect client when it brakes a limit (number of simultaneously
subscribed tickers).
TWS also brings nice modal popup alert with explanation as of why you've
been kicked out :)

*case-2: sent too many requests/second*
Then to check if similar disconnect (by TWS) happens in case of request
pacing I modified the demo app to send some bulk (about 950:)
reqContractDetails and saw similar behaviour: client kicked out by TWS
after 3 error/notes:


 

Ok everyone, we have beat that subject into the ground and?there is a solution. I however, am not a skilled developer and need help with the code. Can all interested parties jump back over to my original subject, "Building an options trading algo using TWS. Need some help!?"? In that topic,?I have detailed the trade entry portion. Is someone interested in developing the entire system for a fee?


 

I have found that even if you honor the 50/sec limit TWS pacing errors will occur.? TWS may queue the messages and at a later time send them to the server.? The 50/sec limit applies to the interactions with the server, not TWS.? If the data farm connection is broken and you cancel or initiate new data subscriptions, TWS will queue these.? When the data farm connection is restored they are sent generating a pacing violation.? Vagaries in TWS internal operations can also cause pacing errors.? You can reduce the occurrence, but I don't think there is a way to absolutely prevent pacing errors assuming you are pushing the limit.? You should be prepared to reconnect if the three pacing error limit is hit.


 

I know, it's a pretty old thread and Frank Bell implies that it may even impossible to constantly avoid the 50 msg/sec limit. But one question: what is counted as a message? If I sent a request to obtain contract details, is it counted as one message or is it counted as two: one for the request and one for the answer?


Nick
 

开云体育

This limit is for requests you send to TWS and does not concern any replies that TWS sends to you. TWS can send many more than 50 messages per second.

So in your example it would count as one request.

There is no way around this limit, although there may be a little slop as to when you get the error (you may not get it exactly on the 51st request).

If I remember right, you could run both a paper and live TWS and each would have its own 50/sec limit.


On 3/22/2022 12:12 PM, hansblafoo via groups.io wrote:

I know, it's a pretty old thread and Frank Bell implies that it may even impossible to constantly avoid the 50 msg/sec limit. But one question: what is counted as a message? If I sent a request to obtain contract details, is it counted as one message or is it counted as two: one for the request and one for the answer?