Keyboard Shortcuts
Likes
Search
Error when requesting 20 years of price data for specific ticker [CHTR]
I have a strange issue when requesting data for a specific ticker: CHTR (for?Charter Communications Inc. - Class A). Below you can see a printout of my console, showing that price data for other test tickers is correctly retrieved. However, when reaching CHTR, it stops and says I don't have access to a database that can provide the data: Danilo #%% DATA REQUEST EXAMPLES
?
# VARIOUS EXAMPLES OF DATA REQUEST FROM INTERACTIVE BROKERS
?
from ibapi.client import EClient
from ibapi.wrapper import EWrapper
from ibapi.contract import Contract
from ibapi.common import TickerId
?
from threading import Thread
import time
?
import pandas
import sys
?
class IBapi(EWrapper, EClient):
?
? ? def __init__(self):
? ? ? ? EClient.__init__(self, self)
? ? ? ? self.data = []
? ? ? ? self.data_sent = -1 # used to check if full data requested returned
?
? ? def error(self, reqId:TickerId, errorCode:int, errorString:str, advancedOrderRejectJson = ""):
? ? ? ? ?if advancedOrderRejectJson:
? ? ? ? ? ? ?print("\nError. Id:", reqId, "\nCode:", errorCode, "\nMsg:", errorString, "\nAdvancedOrderRejectJson:", advancedOrderRejectJson)
? ? ? ? ?else:
? ? ? ? ? ? ?print("\nError. Id:", reqId, "\nCode:", errorCode, "\nMsg:", errorString)
?
? ? def historicalData(self, reqId, bar):
? ? ? ? self.data.append([bar.date, bar.close])
?
? ? def historicalDataEnd(self, reqId, start, end):
? ? ? ? self.data_sent = reqId
?
def run_loop():
? ? app.run()
?
app = IBapi()
app.connect('127.0.0.1', 7496, 0)
?
#Start the socket in a thread
api_thread = Thread(target=run_loop, daemon=True)
api_thread.start()
time.sleep(0.5) #Sleep interval to allow time for connection to server
?
?
# HISTORICAL DATA TEST OF DATABASES vs. API
?
# AAP (NYSE) and CHTR (Nasdaq):
# Had to sign-up for "NYSE (Network A/CTA) Billed by Broker" @ $45/month and "NASDAQ (Network C/UTP)" @ $25/month
# Didn't work with "NYSE American, BATS, ARCA, IEX, and Regional Exchanges (Network B)" @ $25/month
# "Worked" with "Cboe One" @ $5/month but data is inconsistent (e.g. multiple days with same price)
# The "NYSE American, BATS, ARCA, IEX, and Regional Exchanges (Network B)" was better to show streaming prices on TWS (with the Cboe One, live data disapears when using a Basket trade)
?
# Volvo worked with "Nordic Equity" @ 36 EUR/month
# Didn't work with "Turquoise ECNs and Gettex Europe" @ 9.5 GBP/month
# But didn't work with "European (BATS/Chi-X)" @ 31.5 GBP/month
?
# Nestle worked with "European (BATS/Chi-X) Equities" @ GBP 31.50/Month;?
# "SIX Swill Exchange" @ 32.5 CHF/Month partialy worked (would cause an error for 20 years of data request)
# "Tuquoise ECNs and Gettex" didn't work (same for Volvo - see above)
# Noticed that when using "SMART" for exchange getting different figures for Neste (vs. EOD and Yahoo Finance)
# Tried specific exchanges: AQXECH - no permission; BATECH: matched other databases; CHIXCH: permission but a few figures still wrong
# EBS - no permission; TRQXCH: no permission
?
contracts = {'NYSE listed 1':? ?['AAP',? ? 'SMART',? 'USD'],
? ? ? ? ? ? ?'NYSE listed 2':? ?['AGCO',? ?'SMART',? 'USD'],
? ? ? ? ? ? ?'Nasdaq listed 1': ['CHRW',? ?'SMART',? 'USD'],
? ? ? ? ? ? ?'Nasdaq listed 2': ['CSX',? ? 'SMART',? 'USD'],
? ? ? ? ? ? ?'Nasdaq listed 3': ['CHTR',? ?'SMART',? 'USD'],
? ? ? ? ? ? ?'Index':? ? ? ? ? ?['SPY',? ? 'SMART',? 'USD'],
? ? ? ? ? ? ?'Nestle':? ? ? ? ? ['NESN',? ?'BATECH', 'CHF'],
? ? ? ? ? ? ?'Volvo':? ? ? ? ? ?['VOLV.B', 'SFB',? ? 'SEK']}
?
#Create contract object
con = Contract()
con.secType = 'STK'
?
# short-term data
for c in contracts:
? ? con.symbol = contracts[c][0]
? ? con.exchange = contracts[c][1]
? ? con.currency = contracts[c][2]
?
? ? #Request Market Data
? ? reqId = 100
? ? app.data = []
? ? app.data_sent = -1
? ? app.reqHistoricalData(reqId, con, '', '2 W', '1 day', 'ADJUSTED_LAST', 1, 1, False, [])
? ? while app.data_sent != reqId:
? ? ? ? try:
? ? ? ? ? ? time.sleep(0.1)
? ? ? ? except:
? ? ? ? ? ? print("\nError while reading security "+contracts[c][0]+"\n")
? ? ? ? ? ? app.disconnect()
? ? ? ? ? ? sys.exit(1) # will exit code either due to an true error or if CTRL+C pressed
?
? ? # Work with "data" from IB API
? ? df = pandas.DataFrame(app.data, columns=['DateTime', 'Adj. Close'])
? ? df['DateTime'] = pandas.to_datetime(df['DateTime'], format='%Y%m%d')
? ? print('\n2 Weeks of price data for {}:\n'.format(con.symbol))
? ? print(df)
?
?
# long-term data
for c in contracts:
? ? con.symbol = contracts[c][0]
? ? con.exchange = contracts[c][1]
? ? con.currency = contracts[c][2]
?
? ? #Request Market Data
? ? reqId = 100
? ? app.data = []
? ? app.data_sent = -1
? ? app.reqHistoricalData(reqId, con, '', '20 Y', '1 day', 'ADJUSTED_LAST', 1, 1, False, [])
? ? while app.data_sent != reqId:
? ? ? ? try:
? ? ? ? ? ? time.sleep(0.1)
? ? ? ? except:
? ? ? ? ? ? print("\nError while reading security "+contracts[c][0]+"\n")
? ? ? ? ? ? app.disconnect()
? ? ? ? ? ? sys.exit(1) # will exit code either due to an true error or if CTRL+C pressed
?
? ? # Work with "data" from IB API
? ? df = pandas.DataFrame(app.data, columns=['DateTime', 'Adj. Close'])
? ? df['DateTime'] = pandas.to_datetime(df['DateTime'], format='%Y%m%d')
? ? print('\n20 years of price data for {}:\n'.format(con.symbol))
? ? print(df)
?
app.disconnect()
|
The TWS API Guide suggests for retrieval:
toggle quoted message
Show quoted text
For licensing reasons, IBKR has to be much more stringent when they provide you market data in machine readable form via the API than when they display charts within TWS. So I am not surprised that, for charting, TWS may receive more data than what you have subscribed to for and that you can retrieve unsubscribed but locally cached data after pulling up the chart in TWS. Hope this helps, ´³¨¹°ù²µ±ð²Ô PS. Going forward, please do not post large chunks of code. TWS API is language independent and discussions are more meaningful for the entire group when we focus on API behavior, not the implementation in one language. Chances are you get more and better responses that way, too. On Wed, Aug 9, 2023 at 01:46 PM, Rational-IM wrote:
|
Thank you for the tips - I will try tomorrow morning with the ISLAND exchange settings (before opening the chart I mentioned above) to check if it acquires the data without a glitch. I can't use "TRADES"? because it brings the price?unadjusted for dividends. See the detta for PCAR (a company that pays dividends):
What is quite scary is how different the prices are from another source (FactSet) - see below. But I will address this later: The price for ISLAND and SMART (using "ADJUSTED_LAST") is similar (a little scary that it isn't the same, but close enough) for CHTR: I will post back here re: data acquisition before trying the workaround (once you do it, the code works during the whole day). Thank you again, Danilo |
Interesting! |
Great catch from @ds-avatar: the dates are indeed different (so the difference in the last pair of screenshots are not meaningfull.
So this morning, I run my code but using ISLAND for CHTR. And it worked on the first trial (without my workaround of opening a chart with share prices for CHTR) - see below. Here is a puzzling part: @´³¨¹°ù²µ±ð²Ô Reinold allerted me to the following: "In general, a smart-routed historical data requests will require subscriptions to all exchanges on which a instrument trades". So I got the "validExchanges" info for the contracts I was using to ask for price data. See below that after each set of price data, there is now a "CONTRACT DETAILS" section. Note that for CHRW and CSX (two other Nasdaq-listed stocks) the "Valid Exchanges" are exactly the same as for CHTR. So not sure why the issue happens only with CHTR (and not the other Nasdaq names in my test list). Tomorrow I will run the code with "SMART"? for CHTR and see if the error happens again and report back. Thank you all for the help so far. |
If I rephrase your concern I understand it as: why different tickers with exactly same exchanges behave differently on historical request ? |