¿ªÔÆÌåÓý

ctrl + shift + ? for shortcuts
© 2025 Groups.io
  1. Twsapi
  2. Messages
Expanded Topics Messages Polls
Date
Re: How to Get Mkt Data from Scanned Symbols

 

I managed to get the contract details by invoking?self.reqMktData(self.count, scanned_contract, '', False, False, []).
But I dont know the Python syntax of extending the Scanner data with the Market Data details as availble on TWS.
i.e I can get my ranked stock symbols, I can get the req market data but I dont know how to tie both of them so that I get the same view in TWS Advanced Scanner result (with market data such as close, mkt capitalization, volatility etc...)?
Any pointers will be much appreciated.
Re: How to Get Mkt Data from Scanned Symbols

 

I can't speak for the python side, but reqMktData needs a contract with ConId and Exchange ("SMART") set generally to get US Listed quotes..??

If you have the ContractDetails object back from the scan, ContractDetails.Contract is what you'd send in the call to reqMktData..??

-Peter
How to Get Mkt Data from Scanned Symbols

 

Hi,
I have extracted the ScannerData via the wrapper and ranked as needed.
Need to find additional information for the scanned symbols which can be made available via reqMktData. Since I am new to Python, I am struggling to know how to add the code for reqMktData (essentially giving me the view I see on TWS Advance Scanner screen Below is my code (I am trying to get one symbol's MktData (which is MBB for a test).
Many thanks

# -*- coding: utf-8 -*-
"""
Created on Thu Mar 18 10:19:45 2021

@author: DJ
"""
#%% Imports
from ibapi.client import EClient
from ibapi.wrapper import EWrapper
from ibapi.utils import iswrapper
from ibapi.scanner import ScannerSubscription
from ibapi.contract import Contract
from ibapi.tag_value import TagValue
from ibapi.ticktype import TickTypeEnum

from threading import Thread
import time

#%% Client and Wrapper
class StockScanner(EWrapper, EClient):
? ? ''' Serves as the client and the wrapper '''

? ? def __init__(self, addr, port, client_id):
? ? ? ? EClient. __init__(self, self)
? ? ? ? self.data = [] #Initialize variable to store candle
? ? ? ?
? ? ? ? # Connect to TWS
? ? ? ? self.connect(addr, port, client_id)
? ? ? ? self.count = 0
? ? ? ?
? ? ? ? # Launch the client thread
? ? ? ? thread = Thread(target=self.run)
? ? ? ? thread.start()
? ?
? ? @iswrapper
? ? def scannerData(self, reqId, rank, details, distance, benchmark, projection, legsStr):
? ? ? ? # Print the symbols in the returned results
# ? ? ? ?print('{}: {}'.format(rank, details.contract.symbol))
? ? ? ? data1 = (rank+1, details.contract.symbol, details.contract.currency,
? ? ? ? ? ? ? ? ?details.contract.exchange, details.contract.secType
? ? ? ? ? ? ? ? )
? ? ? ? print(data1)
? ? ? ?
? ? ? ? #Request market data
? ? ? ? scanned_contract = Contract()
? ? ? ? scanned_contract.symbol = details.contract.symbol
? ? ? ? scanned_contract.secType = details.contract.secType
? ? ? ? scanned_contract.exchange = details.contract.exchange
? ? ? ? scanned_contract.currency = details.contract.currency
? ? ? ?
? ? ? ? if details.contract.symbol == 'MBB':
? ? ? ? ? ? data2 = reqMktData(self, self.count, scanned_contract, '', False, False, [])
? ? ? ? ? ? print(scanned_contract)
? ? ? ? ? ? print(data2)
? ? ? ? ? ? def tickPrice(self, reqID, tickType, price, attrib):
? ? ? ? ? ? ? ? print("Tick Price. Ticker ID: ", count, "ticktype: ", TickTypeEnum.to_str(tickType), "Price: ", price)

? ? ? ? self.data.append(data1)
? ? ? ? self.count += 1

? ? @iswrapper
? ? def scannerDataEnd(self, reqId):
? ? ? ? # Print the number of results
? ? ? ? print('Number of results: {}'.format(self.count))
? ? ? ?
? ?
? ? def error(self, reqId, code, msg):
? ? ? ? print('Error {}: {}'.format(code, msg)) ?
? ? ? ?
? ? ? ?
#%% Main
def main():

? ? # Create the client and connect to TWS
? ? client = StockScanner('127.0.0.1', 7497, 0)
? ? time.sleep(0.5)

? ? # Create the ScannerSubscription object ?
? ? ss = ScannerSubscription()
? ? ss.instrument = 'STK'
? ? ss.locationCode = 'STK.NASDAQ'
? ? # Top % Gainers Since Open
? ? # ss.scanCode = 'TOP_PERC_GAIN'
? ? ss.scanCode = 'HIGH_OPT_IMP_VOLAT_OVER_HIST'
? ? ss.abovePrice = 1.0
? ? ss.belowPrice = 1000.0

? ? # Set additional filter criteria
? ? tagvalues = []
? ? tagvalues.append(TagValue('changePercAbove', '0%'))

? ? # Request the scanner subscription
? ? client.reqScannerSubscription(0, ss, [], tagvalues)
? ?
? ?
? ? # Sleep while the request is processed
? ? time.sleep(5)
? ?
? ? #Export to CSV file ?
? ? import pandas
? ? df = pandas.DataFrame(client.data, columns=['Rank', 'Symbol', ?'Currency', 'Exchange', 'Sec Type'])
? ? df.to_csv('ScannedResult.csv')

? ? client.disconnect()
? ?
if __name__ == '__main__':
? ? main()??
Re: HOW TO SURVIVE THE IB TWS/GATEWAY RESET?

 

Good Mason!
You've got mail :)
F
Re: HOW TO SURVIVE THE IB TWS/GATEWAY RESET?

 

Yep I am 100% Python right now
Re: HOW TO SURVIVE THE IB TWS/GATEWAY RESET?

 

Thanks Bruce and Mason!

Looks like regularly saving the state is a good approach.

I program in Python but my backgound is more Matlab for signal and image processing.
I'm not fully at ease with object oriented programming concepts.

Do you use the IB Python API?

F
Re: HOW TO SURVIVE THE IB TWS/GATEWAY RESET?

 

My solution to this is that all of my code's state is maintained within a single class with a "set state" method, and every single time that method is called, the state is saved to a JSON file, and every time the class is constructed, it loads all state from that file. Since the rest of my code operates as an FSM, it can pick up right where it left off when I restart.
Re: HOW TO SURVIVE THE IB TWS/GATEWAY RESET?

 

Why don't you save your orders state and call status of each order after you receive a reset signal or after you determine TWS was reset?

It was said that Order Refs get reset but you can build different ways to cope with this.

-Bruce

On Thu, Mar 18, 2021, 4:32 PM Ace <fbaribeau1967@...> wrote:
Hi all!

I have a day trading automated strategy and now I would like to swing positions overnight and keep them while conditions are met.
I faced the nightly TWS reset which now also applies to the gateway.

My strategy is very dynamic with scaling in/out and this reset kills my loops and variables state.

I noticed this is a recurring issue that I've not found a clear solution for by looking through the forums.

So how do you go about surviving the reset for your swing trading strategies to get back exactly where you were at before the reset?

Thank you very much!
F

Re: HOW TO SURVIVE THE IB TWS/GATEWAY RESET?

Stuart Cracraft
 

¿ªÔÆÌåÓý

I second this and don¡¯t want ¡°resets¡±.?


On Mar 18, 2021, at 1:35 PM, Ace <fbaribeau1967@...> wrote:

?Hi all!

I have a day trading automated strategy and now I would like to swing positions overnight and keep them while conditions are met.
I faced the nightly TWS reset which now also applies to the gateway.

My strategy is very dynamic with scaling in/out and this reset kills my loops and variables state.

I noticed this is a recurring issue that I've not found a clear solution for by looking through the forums.

So how do you go about surviving the reset for your swing trading strategies to get back exactly where you were at before the reset?

Thank you very much!
F

HOW TO SURVIVE THE IB TWS/GATEWAY RESET?

 

Hi all!

I have a day trading automated strategy and now I would like to swing positions overnight and keep them while conditions are met.
I faced the nightly TWS reset which now also applies to the gateway.

My strategy is very dynamic with scaling in/out and this reset kills my loops and variables state.

I noticed this is a recurring issue that I've not found a clear solution for by looking through the forums.

So how do you go about surviving the reset for your swing trading strategies to get back exactly where you were at before the reset?

Thank you very much!
F
Re: Little tool for option chain contract creation

juxeiier
 

¿ªÔÆÌåÓý

I am not sure what you mean by all data.
So far, only the static contract data for options are requested.
But soon, also snapshots of actual market data will be included.
At some point, I will provide a pandas data frame to do work with data like bid/ask, greeks etc...

The storage mechanism relies on pickle:
For each symbol, for which you request option contracts, there will be a binary caching file written.
The script cleans that up, such that outdated contracts will get deleted.

Cheers,
Juergen

Am 18.03.2021 um 18:49 schrieb Bruce B:

Jurgen,

Thanks for sharing. I may incorporate your work in our program.

Would you say your program encompasses ALL types of data available from IBKR in regards to options or did you not include some parts?

Also, what is the storage mechanism?

Thanks,

On Thu, Mar 18, 2021, 5:41 AM juxeiier <juxeiier@...> wrote:
New version 0.5 is available, with new lookup functions and bugfixes.
Check new README at

Re: event when a stock reaches a certain price

 

Their last sentence is vague. Bad documentation like this I have seen in other places too.

Yes, I think the delay is in effect and so your quick requests were responded again and again with same data because your 11 seconds bottleneck was not up yet.

Most likely those quick - over time - requests were not even sent to IBKR at all and TWS decides and responds with "stale" data based on 11 second rule. So that could be the "cache".

- Bruce


On Thu, Mar 18, 2021, 2:34 PM §¡§Ý§Ö§Ü§ã§Ñ§ß§Õ§â §¹§Ö§â§ß§Ú§Ü§à§Ó <me@...> wrote:
My request in fact were duplicates - since I requested snapshots for 1500 instruments to get "almost real-time" data. Request ids were never repeated. Price did changed and if I had real-time subscription at the same time - it worked fine. Actually, I worked around it and used real-time data for important data and round-robin for everything else.
From IB docs - " It is important to note that a snapshot request will only return available data over the 11 second span; in some cases values may not be returned for all tick types."
So maybe this is expected behavior.?


On Thu, Mar 18, 2021 at 7:24 PM Bruce B <bruceb444@...> wrote:
Agreed - most things don't from IBKR do not? include timestamps and some may include fake timestamps from TWS. Dimitry I think has looked into this and posted before.

Is it also possible that maybe your requests were duplicates OR that price didn't change in between your multiple requests?

You can compare to TWS data or to other sources to confirm maybe.

I would be surprised if they send stale data over a live account as this has to be a serious? programming error. Can't be a network cache issue either as TCP ACKS would take care of those but maybe a different type of cache.


On Thu, Mar 18, 2021, 2:15 PM §¡§Ý§Ö§Ü§ã§Ñ§ß§Õ§â §¹§Ö§â§ß§Ú§Ü§à§Ó <me@...> wrote:
Where - I do not know (never bothered to check traffic between TWS and IB servers). From what I remember - you receive the same value over and over again.
And since there are no timestamps on data - no way to know what you got.

On Thu, Mar 18, 2021 at 6:42 PM Bruce B <bruceb444@...> wrote:
Chernikov,

Cached where? And what do you mean by *stale? Missing data points or delivered too late that may not be useful for strategies?

Thanks,


On Thu, Mar 18, 2021, 4:55 AM §¡§Ý§Ö§Ü§ã§Ñ§ß§Õ§â §¹§Ö§â§ß§Ú§Ü§à§Ó <me@...> wrote:
50 messages per second do not apply to this. However, there is some other limit, based on available market data lines.
But polling snapshots have another problem (at least it had a couple of years ago) - snapshot could be cached if polled too frequently. So instead of new prices, you are getting something stale.



On Thu, Mar 18, 2021 at 3:16 AM <sudhin_deshpande@...> wrote:
Instead of subscribing, why not poll each ticker to get the last price?
I think IB has a limit of 50 messages per second (correct me if I am wrong)
For 800 tickers, that would be 16 seconds. So you would poll each ticker about every 16 seconds.
Would that work for you?



--
Best regards,
Chernikov Aleksandr,
e-mail:?me@...
skype:?
RU:?+7-904-179-49-78



--
Best regards,
Chernikov Aleksandr,
e-mail:?me@...
skype:?
RU:?+7-904-179-49-78



--
Best regards,
Chernikov Aleksandr,
e-mail:?me@...
skype:?
RU:?+7-904-179-49-78
Re: event when a stock reaches a certain price

 

My request in fact were duplicates - since I requested snapshots for 1500 instruments to get "almost real-time" data. Request ids were never repeated. Price did changed and if I had real-time subscription at the same time - it worked fine. Actually, I worked around it and used real-time data for important data and round-robin for everything else.
From IB docs - " It is important to note that a snapshot request will only return available data over the 11 second span; in some cases values may not be returned for all tick types."
So maybe this is expected behavior.?


On Thu, Mar 18, 2021 at 7:24 PM Bruce B <bruceb444@...> wrote:
Agreed - most things don't from IBKR do not? include timestamps and some may include fake timestamps from TWS. Dimitry I think has looked into this and posted before.

Is it also possible that maybe your requests were duplicates OR that price didn't change in between your multiple requests?

You can compare to TWS data or to other sources to confirm maybe.

I would be surprised if they send stale data over a live account as this has to be a serious? programming error. Can't be a network cache issue either as TCP ACKS would take care of those but maybe a different type of cache.


On Thu, Mar 18, 2021, 2:15 PM §¡§Ý§Ö§Ü§ã§Ñ§ß§Õ§â §¹§Ö§â§ß§Ú§Ü§à§Ó <me@...> wrote:
Where - I do not know (never bothered to check traffic between TWS and IB servers). From what I remember - you receive the same value over and over again.
And since there are no timestamps on data - no way to know what you got.

On Thu, Mar 18, 2021 at 6:42 PM Bruce B <bruceb444@...> wrote:
Chernikov,

Cached where? And what do you mean by *stale? Missing data points or delivered too late that may not be useful for strategies?

Thanks,


On Thu, Mar 18, 2021, 4:55 AM §¡§Ý§Ö§Ü§ã§Ñ§ß§Õ§â §¹§Ö§â§ß§Ú§Ü§à§Ó <me@...> wrote:
50 messages per second do not apply to this. However, there is some other limit, based on available market data lines.
But polling snapshots have another problem (at least it had a couple of years ago) - snapshot could be cached if polled too frequently. So instead of new prices, you are getting something stale.



On Thu, Mar 18, 2021 at 3:16 AM <sudhin_deshpande@...> wrote:
Instead of subscribing, why not poll each ticker to get the last price?
I think IB has a limit of 50 messages per second (correct me if I am wrong)
For 800 tickers, that would be 16 seconds. So you would poll each ticker about every 16 seconds.
Would that work for you?



--
Best regards,
Chernikov Aleksandr,
e-mail:?me@...
skype:?
RU:?+7-904-179-49-78



--
Best regards,
Chernikov Aleksandr,
e-mail:?me@...
skype:?
RU:?+7-904-179-49-78



--
Best regards,
Chernikov Aleksandr,
e-mail:?me@...
skype:?
RU:?+7-904-179-49-78
Re: TWS app keeps exiting on my desktop

 

You might try to run the tws in gateway-mode.

If its stable then, you have to allocate more ressources.

By the way,? There is IB-Container , a simple script to setup a LXD-Container with autostart as cron-service and reverse-ssh-tunnel to setup a secret trading-place
Re: event when a stock reaches a certain price

 

Agreed - most things don't from IBKR do not? include timestamps and some may include fake timestamps from TWS. Dimitry I think has looked into this and posted before.

Is it also possible that maybe your requests were duplicates OR that price didn't change in between your multiple requests?

You can compare to TWS data or to other sources to confirm maybe.

I would be surprised if they send stale data over a live account as this has to be a serious? programming error. Can't be a network cache issue either as TCP ACKS would take care of those but maybe a different type of cache.


On Thu, Mar 18, 2021, 2:15 PM §¡§Ý§Ö§Ü§ã§Ñ§ß§Õ§â §¹§Ö§â§ß§Ú§Ü§à§Ó <me@...> wrote:
Where - I do not know (never bothered to check traffic between TWS and IB servers). From what I remember - you receive the same value over and over again.
And since there are no timestamps on data - no way to know what you got.

On Thu, Mar 18, 2021 at 6:42 PM Bruce B <bruceb444@...> wrote:
Chernikov,

Cached where? And what do you mean by *stale? Missing data points or delivered too late that may not be useful for strategies?

Thanks,


On Thu, Mar 18, 2021, 4:55 AM §¡§Ý§Ö§Ü§ã§Ñ§ß§Õ§â §¹§Ö§â§ß§Ú§Ü§à§Ó <me@...> wrote:
50 messages per second do not apply to this. However, there is some other limit, based on available market data lines.
But polling snapshots have another problem (at least it had a couple of years ago) - snapshot could be cached if polled too frequently. So instead of new prices, you are getting something stale.



On Thu, Mar 18, 2021 at 3:16 AM <sudhin_deshpande@...> wrote:
Instead of subscribing, why not poll each ticker to get the last price?
I think IB has a limit of 50 messages per second (correct me if I am wrong)
For 800 tickers, that would be 16 seconds. So you would poll each ticker about every 16 seconds.
Would that work for you?



--
Best regards,
Chernikov Aleksandr,
e-mail:?me@...
skype:?
RU:?+7-904-179-49-78



--
Best regards,
Chernikov Aleksandr,
e-mail:?me@...
skype:?
RU:?+7-904-179-49-78
Re: event when a stock reaches a certain price

 

Where - I do not know (never bothered to check traffic between TWS and IB servers). From what I remember - you receive the same value over and over again.
And since there are no timestamps on data - no way to know what you got.

On Thu, Mar 18, 2021 at 6:42 PM Bruce B <bruceb444@...> wrote:
Chernikov,

Cached where? And what do you mean by *stale? Missing data points or delivered too late that may not be useful for strategies?

Thanks,


On Thu, Mar 18, 2021, 4:55 AM §¡§Ý§Ö§Ü§ã§Ñ§ß§Õ§â §¹§Ö§â§ß§Ú§Ü§à§Ó <me@...> wrote:
50 messages per second do not apply to this. However, there is some other limit, based on available market data lines.
But polling snapshots have another problem (at least it had a couple of years ago) - snapshot could be cached if polled too frequently. So instead of new prices, you are getting something stale.



On Thu, Mar 18, 2021 at 3:16 AM <sudhin_deshpande@...> wrote:
Instead of subscribing, why not poll each ticker to get the last price?
I think IB has a limit of 50 messages per second (correct me if I am wrong)
For 800 tickers, that would be 16 seconds. So you would poll each ticker about every 16 seconds.
Would that work for you?



--
Best regards,
Chernikov Aleksandr,
e-mail:?me@...
skype:?
RU:?+7-904-179-49-78



--
Best regards,
Chernikov Aleksandr,
e-mail:?me@...
skype:?
RU:?+7-904-179-49-78
Re: Help getting realtime data stream via TWS API for TICK-NYSE and TRIN-NYSE

 

Hunter,

Thanks for that tip.

I can compare this to reqContractDetails() query and understand better.

For TWS > Financial Description window *right side (not left) information, is there a single API request call one can make to get all that data? (For example it includes Dividends, Valuations, Growth,...)

- Bruce

On Thu, Mar 18, 2021, 5:34 AM Hunter C Payne via <hunterpayne2001=[email protected]> wrote:
Go into TWS and right click on any instrument (TRIN or TICK) and select Financial Instrument Info -> Description.? The correct information to put into the Contract object will be specified on the left side of the window that pops up.? In the case of TICK you get:

Underlying = TICK-AMEX
Security Type = IND
Currency = USD
Exchange = AMEX
Symbol = TICK-AMEX

That's what you set for TICK-AMEX.

Hunter

On Thursday, March 18, 2021, 2:20:03 AM PDT, Ullas Diwakar <ullasdiwakar96@...> wrote:


Hi,

I want to get the data stream for the index TICK and TRIN. I read around the group and saw that i need to create the contract with the name TICK-NYSE and TRIN-NYSE with secType as "IND".

After that, I tried to subscribe to realtime data using both reqMktData and?reqRealTimeBars but I am not getting any ticks for it.

I want to know how to get the real time data stream for these two indices.?

Thanks
Re: Little tool for option chain contract creation

 

Jurgen,

Thanks for sharing. I may incorporate your work in our program.

Would you say your program encompasses ALL types of data available from IBKR in regards to options or did you not include some parts?

Also, what is the storage mechanism?

Thanks,

On Thu, Mar 18, 2021, 5:41 AM juxeiier <juxeiier@...> wrote:
New version 0.5 is available, with new lookup functions and bugfixes.
Check new README at

Re: event when a stock reaches a certain price

 

Chernikov,

Cached where? And what do you mean by *stale? Missing data points or delivered too late that may not be useful for strategies?

Thanks,


On Thu, Mar 18, 2021, 4:55 AM §¡§Ý§Ö§Ü§ã§Ñ§ß§Õ§â §¹§Ö§â§ß§Ú§Ü§à§Ó <me@...> wrote:
50 messages per second do not apply to this. However, there is some other limit, based on available market data lines.
But polling snapshots have another problem (at least it had a couple of years ago) - snapshot could be cached if polled too frequently. So instead of new prices, you are getting something stale.



On Thu, Mar 18, 2021 at 3:16 AM <sudhin_deshpande@...> wrote:
Instead of subscribing, why not poll each ticker to get the last price?
I think IB has a limit of 50 messages per second (correct me if I am wrong)
For 800 tickers, that would be 16 seconds. So you would poll each ticker about every 16 seconds.
Would that work for you?



--
Best regards,
Chernikov Aleksandr,
e-mail:?me@...
skype:?
RU:?+7-904-179-49-78
Re: TWS app keeps exiting on my desktop

Stuart Cracraft
 

¿ªÔÆÌåÓý

Thanks.?

On Mar 18, 2021, at 9:10 AM, JR <TwsApiOnGroupsIo@...> wrote:

?You might have a JVM log file with more information in case the JVM fails or gives up on TWS for some reason. The file is called? hs_err_pidXXXXX.log with XXXX elaced with the JVM process ID. You will find that file in the "current working directory" where the JVM was running. In many cases that is the Jts installation root folder. But it could be in one of your local folders if you start TWS in a different folder.

You might have quite a lot of those files if your TWS exists frequently.

JR


On Thu, Mar 18, 2021 at 10:44 AM, Stuart Cracraft wrote:
Nothing in the logs.

Not doing anything except letting it run.

I suspect general memory starvation on the instance and will increase its capacity (memory + compute + network) for a tech refresh.

On Mar 18, 2021, at 8:37 AM, mark collins <mark.collins@...> wrote:

Check the logs for errors, because that's not normal behaviour. If there's no errors it could be getting terminated by the os if it's taking a lot of resources, but almost certainly it's something you're doing and not noticing.

Best wishes,

M

On Thu, 18 Mar 2021, 14:27 Stuart Cracraft via groups.io, <smcracraft@...> wrote:
I login, go away, come back and it¡¯s exited.

How do I make it not exit.

All that happens every 30-60 minutes¡­

Bizarre.
7241 - 7260 of 53825