Keyboard Shortcuts
ctrl + shift + ? :
Show all keyboard shortcuts
ctrl + g :
Navigate to a group
ctrl + shift + f :
Find
ctrl + / :
Quick actions
esc to dismiss
Likes
- Twsapi
- Messages
Search
|
Re: Possible bug in TWS API when requesting auction price, volume, and imbalance data for ASX-listed securities
Nick
You could enable the api log and see what TWS is sending over the socket.
toggle quoted message
Show quoted text
I haven't looked at the latest api but earlier versions did not inform TWS of the language used to implement the client api. In other words TWS does not know whether a client is python or Java or C++. If the data is correct in the api log but incorrect in python then that is useful. If the data is wrong in the api log it would be wrong in all languages. On 4/10/2022 7:35 PM, colintbowers@... wrote:
I also should have added to the original post that the auction imbalance displays correctly in the TWS GUI, which suggests fairly strongly that the problem is occurring somewhere in the API. |
|
Re: Possible bug in TWS API when requesting auction price, volume, and imbalance data for ASX-listed securities
I just thought I'd add to my original message: I'd be very interested in hearing whether anyone has experienced this bug on exchanges other than the ASX, and also whether anyone has experienced this bug using languages other than Python? I've joined the github project for TWS API, so if I could confirm that this behaviour only happens on Python that would certainly narrow things down.
I also should have added to the original post that the auction imbalance displays correctly in the TWS GUI, which suggests fairly strongly that the problem is occurring somewhere in the API. Cheers, Colin |
|
Re: Re-login failures with TWS auto-restart option.
Hi ´³¨¹°ù²µ±ð²Ô,
Thanks for the input. Yes, I get the same maintenance window stuff same as you every night, but we reconnect automatically with no problem. My logs show I was connected and running perfectly until the shutdown and TWS auto-restart . The only thing I can think of is that I'm running Win7 - is anyone else running it with no problems? Thanks, as always, and Be Well, Lou Dudka |
|
Re: Re-login failures with TWS auto-restart option.
All our TWS restarts worked without a hitch, Lou. They take place 5 min after yours at 16:30 Central, But we had an incident during the maintenance window on Tuesday April 5 at around 23:15 Central. TWS briefly looses connection during the maintenance window for one to five minutes pretty much every night:
On Tuesday, TWS lost connection at around 23:15 Central but did not send a 1100 message. Eventually, around 23:23 Central a "Re-login" box popped up and required a user interaction to re-login. Once clicked, TWS logged in and functioned properly. This has not happened in a long time (not for more than a year). ´³¨¹°ù²µ±ð²Ô |
|
Re: Re-login failures with TWS auto-restart option.
Nick
I had a successful TWS auto-restart at 5:15pm Eastern today, TWS version 981.3l.
toggle quoted message
Show quoted text
On 4/8/2022 5:36 PM, Lou Dudka wrote:
I had two re-login failures with TWS auto-restart option this week, Thursday and today Friday. |
|
Re: Building an Advanced Historical Stock Screener
¿ªÔÆÌåÓý
If I notice it, I just delete the file that stores the history and my puller will automatically repull it and capture the adjustments.? I only look at S&P500 and larger Nasdaq stocks so it rarely happens.? If I was more concerned about it, I would put in a
condition that if the stock or ETF changes by more than say 10% from one day to next, then repull the entire history.
From: [email protected] <[email protected]> on behalf of Mike via groups.io <m_saracena@...>
Sent: Wednesday, April 6, 2022 1:20 PM To: [email protected] <[email protected]> Subject: Re: [TWS API] Building an Advanced Historical Stock Screener ?
How are you adjusting for divs/splits/spin-offs?_
On Apr 6, 2022, at 4:13 PM, Crow <aaroncook394@...> wrote:
|
|
Re: Building an Advanced Historical Stock Screener
toggle quoted message
Show quoted text
On Apr 6, 2022, at 4:13 PM, Crow <aaroncook394@...> wrote:
|
|
Re: Building an Advanced Historical Stock Screener
¿ªÔÆÌåÓý
I have noticed that as well.? If I just do one or two requests, I get the data fast.? But then after about the third or so request, it slows down.? I think IB deprioritizes the request when they realize that it is complex and likely the input to an algorithm
rather than a human looking at a chart.
If you are doing this, I highly recommend caching your data.? I have an updater utility that will pull the entire history (daily bars) for a new symbol and stores it in a file.? Then the next time I request the symbol, it only pulls from the last day in the
history forward and then updates the cache each time.? I did that to save time, but also to minimize load on IB's servers.
From: [email protected] <[email protected]> on behalf of ´³¨¹°ù²µ±ð²Ô Reinold via groups.io <TwsApiOnGroupsIo@...>
Sent: Wednesday, April 6, 2022 12:14 PM To: [email protected] <[email protected]> Subject: Re: [TWS API] Building an Advanced Historical Stock Screener ?
I can confirm Richard's measurements, In fact some of my "1 day RTHOnly for 50 years" requests get served even faster.
On Wed, Apr 6, 2022 at 12:30 PM, Richard L King wrote:
|
|
What is manualOrderCancelTime param of Eclient::cancelOrder( int id, String manualOrderCancelTime)
¿ªÔÆÌåÓý
The latest TWS API for Java added a new required parameter to the existing method EClient::cancelOrder.? It now takes a string for manualOrderCancelTime which can be left null.? What does this parameter do?? Cancel an order if not already filled at a specific
time in the future?? It's not documented:
|
|
Re: Building an Advanced Historical Stock Screener
I can confirm Richard's measurements, In fact some of my "1 day RTHOnly for 50 years" requests get served even faster.
toggle quoted message
Show quoted text
On Wed, Apr 6, 2022 at 12:30 PM, Richard L King wrote:
|
|
Re: Building an Advanced Historical Stock Screener
¿ªÔÆÌåÓýNo, I manually (and quite tediously) scraped the symbols from a website and then edited them into a file in the format that my downloader command line program can use for input. ? You can get such a list of all US stocks from . This I better than what I used because you can ask it to display them all on one page. My approach was to copy the table, paste it into Excel, remove the unwanted columns, export it to a text file, then use NotePad++ to add the missing bits. ? Just for interest, I've attached the actual file that I input to the program. The boilerplate comments at the start show the most common commands that the program recognises. ? ? From: [email protected] <[email protected]> On Behalf Of tagerrish@...
Sent: 06 April 2022 17:44 To: [email protected] Subject: Re: [TWS API] Building an Advanced Historical Stock Screener ? Richard, did you manually enter every symbol into a list for the contract.symbol = ¡°ticker¡± |
|
Re: Building an Advanced Historical Stock Screener
It's vastly improved. There are still combinations of bar size and duration that cause lengthy waits, and my implementation takes care to avoid them. Though I've just realised that at least some of the limits are now much larger than I've coded for. For example, for 1-minute bars I was using 6 D as the longest duration (and making multiple requests if I needed more bars than that), but I just tried 20 D and that returns pretty much instantly; 30 D took 18 seconds. 50D was 30 seconds. And a proportion of that time was formatting the received results and writing them to the logfile. So I need to revise all my rules - it may even be that there are no restrictions now.
toggle quoted message
Show quoted text
As an example, you can download the entire daily history of Microsoft from 13 March 1986 in a single call, returning 9091 bars (today) in about 30 seconds. Use 50 Y as the duration. I'm pretty pleased with it, because I always hated the historical data implementation right from the start in about 2005. I still hate the API function, but at least it works well now! -----Original Message----- |
|
Re: Building an Advanced Historical Stock Screener
Nick
Yikes. Things have changed for the better. I know they relaxed the pacing limits but didn't think it was so dramatic for daily bars.
toggle quoted message
Show quoted text
Thanks for the info! On 4/6/2022 7:16 AM, Richard L King wrote:
Nick, I think you're dismissing the historical data mechanism too easily. |
|
Re: Building an Advanced Historical Stock Screener
Nick, I think you're dismissing the historical data mechanism too easily.
toggle quoted message
Show quoted text
I downloaded 10 daily bars for each of the S&P 500 stocks using my downloader program, and it did the lot in just 30 seconds. Daily bar downloads are so simple and quick that, certainly for small numbers of days, there is no suggestion of historical data API rate limiting.. (Whether the daily bars contain enough information to satisfy the OP's needs is another matter.) What there still is, of course, is the overall API input message rate limit 0f 50 per second. When I ran it again, I hit this limit and TWS broke the connection. My downloader doesn't implement explicit API message rate limiting, because it's normally only used for a handful or a few dozen of requests. For example it will reliably retrieve 1-minute bars for all S&P100 stocks for the current and previous session in around 9 seconds, and here the API message rate is quite low: one contract request and one hist data request per contract, so about 200 requests in 9 seconds. But in this case it had actually gone through all 500 contract requests and 266 of the historical data requests before TWS broke the connection, in just 5 seconds, which is about 150 input API messages per second. So it looks like the 50 messages per second API limit isn't very precise! So anyway, I think if the input rate were limited to say 40 per second, there should be no problem retrieving daily bars for all 6000 stocks in about 300 seconds, which doesn't seem too long to wait. But it may be that if you keep pumping in requests for that many stocks, even within the limiting rate, TWS might eventually balk, though I've no reason to suspect that it will. I should be able to test this fairly easily. My historical data mechanism still has an implementation of the original historical data pacing rules (remember the 60 requests per 10 minutes, etc?), but it's turned off by default: I could easily turn it back on and adjust the parameters suitably to fit within the required limit (or use the API connection parameter rate-limiter option). Richard -----Original Message----- |
|
Re: Building an Advanced Historical Stock Screener
Nick
Your best bet is the built-in scanners since you probably can't download history for those markets (around 6,000 stocks) in a timely manner with the tws api. The pacing limits are a moving target so you never know how many requests you can make per minute.
toggle quoted message
Show quoted text
reqMktData with frozen data might work you would have to see the timings for yourself. Note that there is no bulk symbol download in tws api. You have to request one symbol at a time which is not fast. If the available scanners don't have the conditions you want then you'll probably need a different data provider. On 4/5/2022 7:30 PM, tagerrish@... wrote:
I am trying to build an advanced stock screener using tws API and I am not sure if what I am trying to do is possible. I want to scan the entire NYSE and NASDAQ for a set of historical criteria to check if these criteria were met within the last 3 days. |
|
Building an Advanced Historical Stock Screener
I am trying to build an advanced stock screener using tws API and I am not sure if what I am trying to do is possible. I want to scan the entire NYSE and NASDAQ for a set of historical criteria to check if these criteria were met within the last 3 days. For example, I want to output a list of stocks that gained x% between today's close and the close 3 days ago. Is there a way to do this?
I found an example using the contract() object. In the example below the contract.symbol is defined as a single stock such as AAPL. Im trying to figure out if there is a way to scan all symbols against a list of custom conditions and then obtain the output for any tickers that match.? from ibapi.client import EClient from ibapi.wrapper import EWrapper from ibapi.contract import Contract from ibapi.ticktype import TickTypeEnum class TestApp(EWrapper, EClient): ??? def __init__(self): ??????? EClient.__init__(self, self) ??? def error(self, reqId, errorCode, errorString): ??????? print("Error: ", reqId, " ", errorCode, " ", errorString) ??? def tickPrice(self, reqId, tickType, price, attrib): ??????? print("Tick Price. Ticker Id:", reqId, "tickType:", TickTypeEnum.to_str(tickType), "Price:", price, end=' ') ??? def tickSize(self, reqId, tickType, size): ??????? print("Tick Size. Ticker Id:", reqId, "tickType:", TickTypeEnum.to_str(tickType), "Size:", size) def main(): ??? app = TestApp() ??? app.connect("127.0.0.1", 7497, 0) ??? contract = Contract() ??? contract.symbol = "AAPL" ??? contract.secType = "STK" ??? contract.exchange = "SMART" ??? contract.currency = "USD" ??? contract.primaryExchange = "NASDAQ" ??? app.reqMarketDataType(4)? # switch to delayed-frozen data if live is not available ??? app.reqMktData(1, contract, "", False, False, []) ??? app.run() if __name__ == "__main__": ??? main() |
to navigate to use esc to dismiss