[email protected]

开云体育

Hello everybody,

Is there a way to dump somehow, complete IB's instruments database, together with all parameters (conID, supported exchanges, trading times, tick size etc.)? ?

Regards,

Alex

#45864

I think it's very very large DB
and it's updating every trading day, so it's hard.

But you can get information for particular instrument ticker...

#45872

As I remember for only NYSE it's about 1TB data is used for exchanges, financial products' prices etc for a day or an hour, it's not a good idea to store all the data in one computer, but you can collect data of specific tickets

Show quoted text

On Sun, Nov 22, 2020, 14:28 Mikhail Ershov <misha-ershov@...> wrote:

I think it's very very large DB
and it's updating every trading day, so it's hard.

But you can get information for particular instrument ticker...

Aleksandar Markelic

#45874

开云体育

Maybe it is a misunderstanding, I was not asking for historical prices or its volumes,
just about all tradable contracts including all its parameters.

For example MRK information what could be found on web Info&wlId=IB&conid=70101545&lang=en&ib_entity=

or directly from API:

ContractDetails [AggGroup] = 1, [BondType] = , [Callable] = False, [Category] = Pharmaceuticals, [Contract] = Contract [ComboLegs] = , [ComboLegsDescription] = , [ConId] = 70101545, [Currency] = USD, [DeltaNeutralContract] = , [Exchange] = SMART, [IncludeExpired] = False, [LastTradeDateOrContractMonth] = , [LocalSymbol] = MRK, [Multiplier] = , [PrimaryExch] = NYSE, [Right] = , [SecId] = , [SecIdType] = , [SecType] = STK, [Strike] = 0, [Symbol] = MRK, [TradingClass] = MRK, [ContractMonth] = , [Convertible] = False, [Coupon] = 0, [CouponType] = , [Cusip] = , [DescAppend] = , [EvMultiplier] = 0, [EvRule] = , [Industry] = Consumer, Non-cyclical, [IssueDate] = , [LastTradeTime] = , [LiquidHours] = 20201120:0930-20201120:1600;20201121:CLOSED;20201122:CLOSED;20201123:0930-20201123:1600;20201124:0930-20201124:1600;20201125:0930-20201125:1600, [LongName] = MERCK & CO INC, [MarketName] = MRK, [MarketRuleIds] = 26,26,26,26,26,26,26,26,26,26,26,26,26,26,26,26,26,26,26,26,26,26,26,26, [Maturity] = , [MdSizeMultiplier] = 100, [MinTick] = 0,01, [NextOptionDate] = , [NextOptionPartial] = False, [NextOptionType] = , [Notes] = , [OrderTypes] = ACTIVETIM,AD,ADJUST,ALERT,ALGO,ALLOC,AVGCOST,BASKET,BENCHPX,CASHQTY,COND,CONDORDER,DARKONLY,DARKPOLL,DAY,DEACT,DEACTDIS,DEACTEOD,DIS,GAT,GTC,GTD,GTT,HID,IBKRATS,ICE,IMB,IOC,LIT,LMT,LOC,MIDPX,MIT,MKT,MOC,MTL,NGCOMB,NODARK,NONALGO,OCA,OPG,OPGREROUT,PEGBENCH,POSTONLY,PREOPGRTH,PRICECHK,REL,RPI,RTH,RTHIGNOPG,SCALE,SCALEODD,SCALERST,SIZECHK,SNAPMID,SNAPMKT,SNAPREL,STP,STPLMT,SWEEP,TRAIL,TRAILLIT,TRAILLMT,TRAILMIT,WHATIF, [PriceMagnifier] = 1, [Putable] = False, [Ratings] = , [RealExpirationDate] = , [SecIdList] = List`1 [Capacity] = 1, [Count] = 1, [StockType] = COMMON, [Subcategory] = Medical-Drugs, [TimeZoneId] = EST (?stliche Normalzeit), [TradingHours] = 20201120:0400-20201120:2000;20201121:CLOSED;20201122:CLOSED;20201123:0400-20201123:2000;20201124:0400-20201124:2000;20201125:0400-20201125:2000, [UnderConId] = 0, [UnderSecType] = , [UnderSymbol] = , [ValidExchanges] = SMART,AMEX,NYSE,CBOE,PHLX,ISE,CHX,ARCA,ISLAND,DRCTEDGE,BEX,BATS,EDGEA,CSFBALGO,JEFFALGO,BYX,IEX,EDGX,FOXRIVER,PEARL,NYSENAT,LTSE,MEMX,PSX

Is there a way to get access to all instruments (tradable contracts) and it's all attributes?

Or at least to download all of them at least once per month?

Show quoted text

On Sun, Nov 22, 2020 at 10:20 PM AnerG corp <ahunatajanov@...> wrote:

As I remember for only NYSE it's about 1TB data is used for exchanges, financial products' prices etc for a day or an hour, it's not a good idea to store all the data in one computer, but you can collect data of specific tickets

On Sun, Nov 22, 2020, 14:28 Mikhail Ershov <misha-ershov@...> wrote:
I think it's very very large DB
and it's updating every trading day, so it's hard.

But you can get information for particular instrument ticker...

#45875

开云体育

If I were trying to do this, I'd write a web crawler starting with the exchange listings on IB's website at . By parsing the contents, I could then visit the page for each exchange, for example CBOE at and gather a list of all the contracts offered at the exchange, and then use contract details requests via the API to get the details of each contract.

?

There shouldn't be anything too technically difficult about this, but it's pretty tedious work (I did something similar in the early 2000s, to download the entire UK horse racing results database from the Racing Post website).

?

Richard

?

Show quoted text

From: [email protected] <[email protected]> On Behalf Of Aleksandar Markelic
Sent: 22 November 2020 23:07
To: [email protected]
Subject: Re: [TWS API] InteractiveBroker's instrument database

?

Maybe it is a misunderstanding, I was not asking for historical prices or its volumes,
just about all tradable contracts including all its parameters.

?

For example MRK information what could be found on web Info&wlId=IB&conid=70101545&lang=en&ib_entity=

?

or directly from API:

?

ContractDetails [AggGroup] = 1, [BondType] = , [Callable] = False, [Category] = Pharmaceuticals, [Contract] = Contract [ComboLegs] = , [ComboLegsDescription] = , [ConId] = 70101545, [Currency] = USD, [DeltaNeutralContract] = , [Exchange] = SMART, [IncludeExpired] = False, [LastTradeDateOrContractMonth] = , [LocalSymbol] = MRK, [Multiplier] = , [PrimaryExch] = NYSE, [Right] = , [SecId] = , [SecIdType] = , [SecType] = STK, [Strike] = 0, [Symbol] = MRK, [TradingClass] = MRK, [ContractMonth] = , [Convertible] = False, [Coupon] = 0, [CouponType] = , [Cusip] = , [DescAppend] = , [EvMultiplier] = 0, [EvRule] = , [Industry] = Consumer, Non-cyclical, [IssueDate] = , [LastTradeTime] = , [LiquidHours] = 20201120:0930-20201120:1600;20201121:CLOSED;20201122:CLOSED;20201123:0930-20201123:1600;20201124:0930-20201124:1600;20201125:0930-20201125:1600, [LongName] = MERCK & CO INC, [MarketName] = MRK, [MarketRuleIds] = 26,26,26,26,26,26,26,26,26,26,26,26,26,26,26,26,26,26,26,26,26,26,26,26, [Maturity] = , [MdSizeMultiplier] = 100, [MinTick] = 0,01, [NextOptionDate] = , [NextOptionPartial] = False, [NextOptionType] = , [Notes] = , [OrderTypes] = ACTIVETIM,AD,ADJUST,ALERT,ALGO,ALLOC,AVGCOST,BASKET,BENCHPX,CASHQTY,COND,CONDORDER,DARKONLY,DARKPOLL,DAY,DEACT,DEACTDIS,DEACTEOD,DIS,GAT,GTC,GTD,GTT,HID,IBKRATS,ICE,IMB,IOC,LIT,LMT,LOC,MIDPX,MIT,MKT,MOC,MTL,NGCOMB,NODARK,NONALGO,OCA,OPG,OPGREROUT,PEGBENCH,POSTONLY,PREOPGRTH,PRICECHK,REL,RPI,RTH,RTHIGNOPG,SCALE,SCALEODD,SCALERST,SIZECHK,SNAPMID,SNAPMKT,SNAPREL,STP,STPLMT,SWEEP,TRAIL,TRAILLIT,TRAILLMT,TRAILMIT,WHATIF, [PriceMagnifier] = 1, [Putable] = False, [Ratings] = , [RealExpirationDate] = , [SecIdList] = List`1 [Capacity] = 1, [Count] = 1, [StockType] = COMMON, [Subcategory] = Medical-Drugs, [TimeZoneId] = EST (?stliche Normalzeit), [TradingHours] = 20201120:0400-20201120:2000;20201121:CLOSED;20201122:CLOSED;20201123:0400-20201123:2000;20201124:0400-20201124:2000;20201125:0400-20201125:2000, [UnderConId] = 0, [UnderSecType] = , [UnderSymbol] = , [ValidExchanges] = SMART,AMEX,NYSE,CBOE,PHLX,ISE,CHX,ARCA,ISLAND,DRCTEDGE,BEX,BATS,EDGEA,CSFBALGO,JEFFALGO,BYX,IEX,EDGX,FOXRIVER,PEARL,NYSENAT,LTSE,MEMX,PSX

?

Is there a way to get access to all instruments (tradable contracts) and it's all attributes?

Or at least to download all of them at least once per month?

?

On Sun, Nov 22, 2020 at 10:20 PM AnerG corp <ahunatajanov@...> wrote:

As I remember for only NYSE it's about 1TB data is used for exchanges, financial products' prices etc for a day or an hour, it's not a good idea to store all the data in one computer, but you can collect data of specific tickets
?
On Sun, Nov 22, 2020, 14:28 Mikhail Ershov <misha-ershov@...> wrote:
I think it's very very large DB
and it's updating every trading day, so it's hard.

But you can get information for particular instrument ticker...

#45881

Never seen,
However I guess it doesn't exist (at least officially) because I understand why there NOT such feature:
1- bcs contract need daily update (There are addition/deleting/change daily!)
2- bcs the US market is already huge (> 14000 tickers excluding OTC)
If you add extra Can+Mex+Europe+China, It is unreasonable to manage and even pretty dangerous.

Way I do it (if anybody have better suggestion I listen!)
- Once a day i use a service like? https://www.alphavantage.co to get coarse but cheap IO to get updated ticker info,
- I correlate it with https://iexcloud.io/ for US market (IEX is more accurate)
- Then, and only then, every day I ask trough IBKR update but only for Contracts that are relevant to working position (existing or futur).
IBKR is not great in terme of IO so they can be overloaded/reject with request as simple as too many contract details.

Safety first:
I very much focus to get the EXACT "conID" for the instrument,
This is the key info, it rarely change (never seen) but is not said as being fixed forever. Like sid or cfig are
I always fetch information daily for contract associated with a new position, and put it in cache, expiring at midnight!. (they are not supposed to change during the day, emergency deletion work by halting during the RTH, and will disappear next day, rare case)

IBKR ContractDetails are reasonably fast to delay this request until setting an order on a new contract,
Otherwise try prefetch as soon as you guess you need it.