Hello I am currently have this in a foreach loop using

?

?if (hasCompleteData || hasPartialData)
? ? ? ? ? ? ? ? {
? ? ? ? ? ? ? ? ? ? ClientSocket.cancelMktData(cancelTickerId);
? ? ? ? ? ? ? ? ? ? MaxSubscriptioncancelledTickerIds.Add(cancelTickerId);

? ? ? ? ? ? ? ? ? ? logBuilder.AppendLine(hasCompleteData
? ? ? ? ? ? ? ? ? ? ? ? ? $"Max Subscription Data Cancel. Complete Data. Ticker Id: {cancelTickerId}"
? ? ? ? ? ? ? ? ? ? ? ? : $"Max Subscription Data Cancel. Partial Data. Ticker Id: {cancelTickerId}");

? ? ? ? ? ? ? ? ? ? return true; // Ticker was successfully canceled
? ? ? ? ? ? ? ?

?

?

My question is should i add sleep between each canceltickerid, I am batching about 300 canceltickerids at once and the tick call backs for each canceltickerids keeps firing,? On the log for example I could call canclemktdata for a specific tickerid and for up to 2 minutes I may still get tick call backs for that partciular tickerid, Will adding sleep allow for faster processing or should i do smaller batches?

#53544

It used to be the case that you can send a maximum of 50 messages per second via the API. So you would need to include some waiting time if you need to send more messages. However, latest versions of the API have a pacing function built-in. So it would not help if you include any waiting time when you send your messages. In my case: even though the pacing function is there, I still include some waiting time in case I send a batch of messages that is longer than 50 messages.

#53550

I think JG is correct. You are likely bumping at the 50 requests/sec API limit when you cancel 300 market data feeds.

Traditionally, TWS/IBGW did reject requests at rates of more them 50/sec with an error callback. But there was always the "PACEAPI" connection option that, when specified during your connection requests, would instruct TWS.IBGW to perform request throttling instead of rejecting requests. More recent TWS/IBGW versions now enable throttling by default and you'd have to use an API configuration field in TWS/IBGW to disable it.

I have been using TWS/IBGW throttling for years (first with the PACEAPI option and now just by default) and found it works very well. In the few occasions where I did exceed the 50 requests/sec limit, TWS/IBGW newer throttled below that rate and for short peaks allowed a few more than 50. So I think it is just fine to send requests without implementing any delays on your client side. And should IBKR increase the rate at some time in the future (they tend to relax limits over time), your clients would immediately take advantage of the higher rate.

Keep in mind, that the 50 requests/sec is the cumulative limit for all clients connected to TWS/IBGW at any point in time. To implement throttling on the client side, you'd have to coordinate all of your clients should there be more than one.

But coming back to your question. We don't know all of your requirements, but I can see a three basic approaches (assuming you indeed need to cancel all these feeds simultaneously):

You could continue with what you are doing now and simply ignore any data callbacks after you submitted the cancel request for a feed to hide the additional data from your application. But TWS API would be unable to accept additional requests for up to 300/50 = 6 seconds until all cancel requests have been satisfied. But that may be fine.
You could simply keep the subscriptions active all the time and never cancel them since there is no limit on the number of messages TWS/IBGW send to your client.
Or you could have two connections with TWS/IBGW.
- One for general requests (such as order placement etc) that you keep established for the entire lifetime of your client
- One for these temporary market data feeds that you establish when you start the subscriptions and disconnect (instead of calling cancel requests for all data feeds) when you are done with the market data. Closing the connection will cancel all outstanding data feeds. But there may be unintended consequences when closing the connection without cancelling the data feeds (there was a bug in TWS/IBGW in the past that made it important that feeds were cancelled before disconnect) and you'd have to experiment with that.

I assume you also experience a delay when you establish the 300 market data feeds, right? So it might take up to six seconds for the first market data callbacks for the last instrument on the list to arrive. So the best approach depends on the the balance of all of your requirements.

闯ü谤驳别苍

?

On Sun, Nov 24, 2024 at 01:21 AM, J G wrote:

Show quoted text

It used to be the case that you can send a maximum of 50 messages per second via the API. So you would need to include some waiting time if you need to send more messages. However, latest versions of the API have a pacing function built-in. So it would not help if you include any waiting time when you send your messages. In my case: even though the pacing function is there, I still include some waiting time in case I send a batch of messages that is longer than 50 messages.

#53551

interesting, In my case I need the cancelmktdata in case I run to Error 300, too many instruments subscribed. My experience is that in lag between cancelmktdata and IBKR cancelling the mktdata is that IBKR PACE doesn't actually cancel the first request, for example if I sent ticker ID 1-300 to be cancelled, I could sometimes get ticker id 245 cancelled before ticker 45,? (I continue getting ticker id 45 callbacks a long time before ticker 245, (which i confirm is cancelled by getting error Can't find EId) has anyone else had this?

#53552

开云体育

I think that given the large number of market data streams you have running, this might just be caused by your application not keeping up with the market data events. If you’re processing several hundred market data streams, you could easily be handling several thousand API callbacks per second at busy times

?

As API messages are read from the socket, they are placed in a queue. A separate thread (the reader thread) takes items off that queue, parses the message and calls the appropriate callback function, then proceeds to the next item. If the processing in the callbacks (which happens on the same reader thread) is such that new items are arriving on the queue faster than the reader thread and the callback function can process them, then the queue will just keep growing. So you could end up with potentially thousands of items on the queue wating to be processed. If you now cancel a ticker, the cancel message will be immediately sent to TWS and it won’t send you any more data for that ticker. But there may already be several, or tens, or hundreds of items for that ticker waiting in the queue, and they will just be processed when the reader thread gets to them, which could be long after the cancel was sent. This is because the cancelMktData method does nothing except send a cancel API message to TWS: it doesn’t attempt to remove any unprocessed messages for that ticker from the queue. This means that the API client program has to simply ignore any market data callbacks for a ticker after it has been cancelled.

?

It’s actually very easy to see this using IBKRs C# sample APP. If you start a number of very active market data streams, such as the ES, NQ, YM and DAX futures, and the major forex pairs (EUR.USD, GBP.USD, JPY.USD, etc) and add some market depth streams (ES, YM, NQ), and tick-by-tick etc, and make sure the busiest ones appear in the various grids, then run it at the US market open, the application will eventually get bogged down. Stop the tickers, and the data grid will continue to update for a while. DISCLAIMER: it’s a long time since I last did this, so it’s possible IBKR have enhanced the program to make it more efficient, but it certainly wasn’t originally written for efficiency (nor was the C# API itself, but that has certainly been improved enormously over the years.)

?

Perhaps you could give a summary of what processing you do for each tick?

?

Richard

?

Show quoted text

From: twsapi@groups.io <twsapi@groups.io> On Behalf Of H Liu via groups.io
Sent: 24 November 2024 19:01
To: twsapi@groups.io
Subject: Re: [TWS API] Best practice for cancelmktdata

?

interesting, In my case I need the cancelmktdata in case I run to Error 300, too many instruments subscribed. My experience is that in lag between cancelmktdata and IBKR cancelling the mktdata is that IBKR PACE doesn't actually cancel the first request, for example if I sent ticker ID 1-300 to be cancelled, I could sometimes get ticker id 245 cancelled before ticker 45,? (I continue getting ticker id 45 callbacks a long time before ticker 245, (which i confirm is cancelled by getting error Can't find EId) has anyone else had this?