¿ªÔÆÌåÓý

ctrl + shift + ? for shortcuts
© 2025 Groups.io
  1. Twsapi
  2. Messages
Expanded Topics Messages Polls
Date
Re: reqHistoricalData - volume returned as 0.0

 
Edited

It looks like your contract object relates to NQU2 futures but on 20220401, NQM2 is still the front future. I just repeated your query and while NQM2 does have trades in pretty much every second in the 1/2 before market open on 20220401, NQU2 (which you are requesting had only very few trades the entire day and non just before market open. Therefore, volume = 0 and count = 0 values are correct.

As a matter of practice, we never create contract objects ourselves. We rather make a reqContraclDetails query (in this case NQM2:FUTURE:GLOBEX or NQU2:FUTURE:GLOBEX if that is what you really want) and use the returned contract object for calls such as reqHistoricalData. This also has the advantage that you do receive trading hour details for the next few days as well as the native time zone for the instrument. That is "US/Central" in this case.

You will get the most stable results if you specify times in the native timezone of the instrument since you do not have to deal with any complex timezone math. And if you are located outside of the US where your daylight saving times do not line up with those in the US (or in Arizona, where daylight saving is not observed, except in the north eastern part in the Navajo Nation) you can get your head spinning in no time and you will make mistakes. Consequently, we would have specified '20220401 08:30:00 US/Central'.

The long form timezone identifiers are much less error prone and IBKR has switched over to them pretty much everywhere. You can find a complete list in the

´³¨¹°ù²µ±ð²Ô


On Thu, Jul 7, 2022 at 05:46 PM, <shriki36@...> wrote:

Hey,

I am trying to pull out some historical data about NQ Future.?

The time frame is 1 sec for the last 30 min (for example) - The thing is that the volume is always returned as 0.0 which is doesn't make sense.

Example:??

ib.reqHistoricalData(contract, endDateTime='20220401 09:30:00 est', barSizeSetting='1 secs',
                                durationStr='1800 S', whatToShow='TRADES', useRTH=False, formatDate=2)

where contract is:?

contract = Future('NQ', '202209', 'GLOBEX', '', '20', 'USD')

Does anyone faced with this issue??

Another thing is the timezone which I need to understand how to make it aligned with the trading hours.?

?

Thanks for the help :)?


?On Thu, Jul 7, 2022 at 05:46 PM, <shriki36@...> wrote:
reqHistoricalData
reqHistoricalData - volume returned as 0.0

[email protected]
 

Hey,

I am trying to pull out some historical data about NQ Future.?

The time frame is 1 sec for the last 30 min (for example) - The thing is that the volume is always returned as 0.0 which is doesn't make sense.

Example:??

ib.reqHistoricalData(contract, endDateTime='20220401 09:30:00 est', barSizeSetting='1 secs',
                                durationStr='1800 S', whatToShow='TRADES', useRTH=False, formatDate=2)

where contract is:?

contract = Future('NQ', '202209', 'GLOBEX', '', '20', 'USD')

Does anyone faced with this issue??

Another thing is the timezone which I need to understand how to make it aligned with the trading hours.?

?

Thanks for the help :)?
Re: Cannot connect to Production API, test is OK.

 

Happy hunting, Lou. My gut tells me that this is not an issue? with IB or TWS. It is sufficient that some defunct process holds on to port 9707 for TWS to be denied access to that port. Those sockets in that state may not show up in netstat listings of established connections. A couple ways to check this are:

  • My favorite command "lsof". Try it as " lsof -iTCP -n" so that it only shows you the processes that use TCP resources.
  • Or you could go to the horse's mouth with "cat /proc/nettcp" but make sure you convert to numbers from hex to decimal or look for port "25EB"
´³¨¹°ù²µ±ð²Ô


On Thu, Jul 7, 2022 at 09:00 AM, Lou Dudka wrote:
´³¨¹°ù²µ±ð²Ô and joanmarcel119,

Again, thanks for your input.

Firstly, I don't use IB's ports simply because we may (and do) run multiple TWS's on the same machine.

I bought up another TWS on the same machine and tried to assign port 9707.? TWS refused it and said it was in use, which it wasn't.

I ran? netstat -ano -p tcp and port 9707 did not show up.

I think it's an IB problem, not a "network" problem.? It appears IB doesn't think it "let go" of the port.

I'll guarantee you when I re-boot the machine this weekend and assign 9707 to the "offending" TWS it will work.

This is not a big deal (now) to me since I'm up and running.? I consider myself a trader first and a technician second, but it does intrigue me.

I'll keep you posted.

Thanks, as always, Be Well,
Lou Dudka
Cannot connect to Production API, test is OK.

 

´³¨¹°ù²µ±ð²Ô and joanmarcel119,

Again, thanks for your input.

Firstly, I don't use IB's ports simply because we may (and do) run multiple TWS's on the same machine.

I bought up another TWS on the same machine and tried to assign port 9707.? TWS refused it and said it was in use, which it wasn't.

I ran? netstat -ano -p tcp and port 9707 did not show up.

I think it's an IB problem, not a "network" problem.? It appears IB doesn't think it "let go" of the port.

I'll guarantee you when I re-boot the machine this weekend and assign 9707 to the "offending" TWS it will work.

This is not a big deal (now) to me since I'm up and running.? I consider myself a trader first and a technician second, but it does intrigue me.

I'll keep you posted.

Thanks, as always, Be Well,
Lou Dudka
Re: Cannot connect to Production API, test is OK.

 

But that sounds more like a network issue than a TWS issue, Lou.

  • As joanmarcel119? said firewall or antivirus related that prevents access to port 9707
  • Or even more subtle, a port clash with another service on your machine. If that clash happens on the TWS/IBGW machine, you should get an error message or at least a log entry.

What does your infrastructure between your client and TWS/IBGW look like?

And finally, any specific reason that you do not use the IBKR standard port assignments?

  • TWS_Paper_Account 7497
  • TWS_Live_Account 7496
  • IBG_Paper_Acctount 4002
  • IBG_Live_Account 4001
´³¨¹°ù²µ±ð²Ô
Cannot connect to Production API, test is OK.

 

Hi All,

Fixed (cannot say "solved").

Out of desperation I changed the Prod Port from 9707 to 9777 and it connected.

I then changed the Port back to 9707 and received errors.

Put it back to 9777 and will proceed as if nothing happened .... I have backups of TWS from last week and if I really wanted to I could start the TWS with them.? Ah, it's just not worth the trouble ...

Thanks to all, as always, Be Well,
Lou Dudka
Cannot connect to Production API, test is OK.

 

joanmarcel119,

Thanks for your input.

It might be that except both Test and Prod are on the same computer using exactly the same code.

Thanks, as always, and Be Well,
Lou Dudka
Re: Cannot connect to Production API, test is OK.

 

maybe firewall or antivirus updated?or changed?


El mi¨¦, 6 jul 2022 a las 3:15, Lou Dudka (<loududka@...>) escribi¨®:
´³¨¹°ù²µ±ð²Ô,

Thanks, as always, for your complete and speedy response!

I'm (almost) ready to try replacing 981 with 10.12, simply because I've run out of ideas.

BTW both Prod and Test are running on the same machine, so all libraries, etc are the same.? I've even taken an XLS that's successfully connected to Test and changed the port to Prod and it hangs up. ? Any yes, as you pointed out, I double checked that the Ports are correct.? If I flip between the two (Prod and Test) API setup screens in TWS, they're exactly the same, with exception of the Port, as it should be.

The one thing I might try is bringing up another Prod TWS on the same machine and see if I can connect.? If I can, it would seem to point to the TWS connection itself.

I don't know what Richard's "Contract Inspector" is, but if the above fails, I'll spend some time researching it.

Thanks, as always, Be Well,udka
Lou Dudka

Cannot connect to Production API, test is OK.

 

´³¨¹°ù²µ±ð²Ô,

Thanks, as always, for your complete and speedy response!

I'm (almost) ready to try replacing 981 with 10.12, simply because I've run out of ideas.

BTW both Prod and Test are running on the same machine, so all libraries, etc are the same.? I've even taken an XLS that's successfully connected to Test and changed the port to Prod and it hangs up. ? Any yes, as you pointed out, I double checked that the Ports are correct.? If I flip between the two (Prod and Test) API setup screens in TWS, they're exactly the same, with exception of the Port, as it should be.

The one thing I might try is bringing up another Prod TWS on the same machine and see if I can connect.? If I can, it would seem to point to the TWS connection itself.

I don't know what Richard's "Contract Inspector" is, but if the above fails, I'll spend some time researching it.

Thanks, as always, Be Well,udka
Lou Dudka
Re: Cannot connect to Production API, test is OK.

 

We have not experienced any issues on our end, Lou, but then our setup is very different. Let me throw out a few thoughts, some or all of them you may have thought about yourself already:
  • Which version of TWS/IBGW are you running? Latest/Stable and Standalone/Autoupdate? Reason why I ask is that IBKR recently switched the stable track from the 981 level to the 10.12 level. There have been a lot of changes introduced in version 10 and I believe there are a couple new settings in the General API screen. Having said that, those TWS/IBGW changes should impact paper and real accounts equally.
  • Have you confirmed that the port for your real account in the TWS/IBGW API settings has not changed somehow? In other words, is the TWS/IBGW port really identical to the one you use in your client?
  • And finally, have you tried to get a "second opinion", meaning have you tried an API client that is very different from what you normally use? One that comes to mind is

Hope there is something here that gets you a step closer.

´³¨¹°ù²µ±ð²Ô

On Tue, Jul 5, 2022 at 06:36 PM, Lou Dudka wrote:

Hi All,

I suddenly (over the weekend) cannot connect on:
Call objTWSControl.m_TWSControl.Connect(connectionHost, connectionPort, clientId, False)
connectionHost is blank
connectionPort is correct in TWS API setup
clientId is correct and unique.

ONLY when trying to connect to Production.
I know we've all heard it before, but (to the absolute best of my knowledge) I've changed nothing.? This code is "as is" from IB and has not been changed ever.

If I simply switch (same XLS/VBA code) the connectionPort to the Test API port everything works.

My app (Excel/Win7/ActiveX) just hangs on the call. Old API code, don't know version.

Something changed over the weekend.

Thanks, as always, and Be Well,
Lou Dudka
Cannot connect to Production API, test is OK.

 

Hi All,

I suddenly (over the weekend) cannot connect on:
Call objTWSControl.m_TWSControl.Connect(connectionHost, connectionPort, clientId, False)
connectionHost is blank
connectionPort is correct in TWS API setup
clientId is correct and unique.

ONLY when trying to connect to Production.
I know we've all heard it before, but (to the absolute best of my knowledge) I've changed nothing.? This code is "as is" from IB and has not been changed ever.

If I simply switch (same XLS/VBA code) the connectionPort to the Test API port everything works.

My app (Excel/Win7/ActiveX) just hangs on the call. Old API code, don't know version.

Something changed over the weekend.

Thanks, as always, and Be Well,
Lou Dudka
Re: TWS Gateway: Requested market data is not subscribed

 

With a browser there's standard information passed back to the server (browser & version, OS & version, screen size, etc). When TWS or the Gateway connects who knows what is sent. It might send all sorts of data including it's own version number, the MAC address, the processor serial number, etc. Anything to identify a unique device.
Re: TWS Gateway: Requested market data is not subscribed

 

¿ªÔÆÌåÓý

I just wanted to add that I¡¯m not trying to suggest we should actually try to get round this restriction on any kind of real usage, just that this experiment might confirm that it is in fact the MAC address that is used to check for the same device.

?

We should of course abide by the terms and conditions that IB holds us to.

?

?

From: [email protected] <[email protected]> On Behalf Of Richard L King
Sent: 04 July 2022 10:48
To: [email protected]
Subject: Re: [TWS API] TWS Gateway: Requested market data is not subscribed

?

Interesting thought, but as that Wikipedia article also says, ¡®Many network interfaces, however, support changing their MAC address¡¯ and ¡®two NICs connected to two different networks can share the same MAC address¡¯.

?

So if your guess is correct, that would give a potential way to overcome this restriction. For example, I run a set of virtual machines under Hyper-V, and that allows the virtual network adapters¡¯ MAC address to be changed. So I could potentially set up clients on two different virtual networks, give them the same MAC address, and see if that bypasses the restriction. Might be an interesting experiment to try (when there¡¯s nothing better to do¡­).
Re: TWS Gateway: Requested market data is not subscribed

 

¿ªÔÆÌåÓý

Interesting thought, but as that Wikipedia article also says, ¡®Many network interfaces, however, support changing their MAC address¡¯ and ¡®two NICs connected to two different networks can share the same MAC address¡¯.

?

So if your guess is correct, that would give a potential way to overcome this restriction. For example, I run a set of virtual machines under Hyper-V, and that allows the virtual network adapters¡¯ MAC address to be changed. So I could potentially set up clients on two different virtual networks, give them the same MAC address, and see if that bypasses the restriction. Might be an interesting experiment to try (when there¡¯s nothing better to do¡­).
Re: TWS Gateway: Requested market data is not subscribed

 

My guess is that the device's MAC address is analyzed because it is unique to every device.
https://en.wikipedia.org/wiki/MAC_address
Sample code for most common use cases?

 

¿ªÔÆÌåÓý

Hi all,

I wonder if there are sample codes of recommended methods for the most common use cases. I know IB API has sample code, but it's on very low level (e.g. get price of a stock). Sample codes of such use cases can be very helpful:
  1. monitor the price of stock XYZ. If the price reaches x1, send a limit order of price x2. If the price drops to x3, cancel the order if not executed yet.
  2. get option chain of SPX, filter the expirations and strikes by some defined rules, and monitor the prices (bid/ask) for the options filtered. If some condition is met, send an order to buy a spread (buy x and sell y)
I know IB API takes an async approach and implementing the above use cases can have various ways. I think any method from an experienced programmer would be good to start. It teaches newbies how to organize all pieces into a functional trading software.

Thank you!
Re: TWS Gateway: Requested market data is not subscribed

 

Out of curiosity, how does it know/detect that is the "same device"?
reqHistoricalData returns live data with date in the future

 

Hi all

As topic title, have a bit of an unusual one. So sometimes IBKR returns live daily data with date in the future, has anyone come across this problem?

I can handle it easily by restarting Gateway, but it irks me to no end that it happens, and more importantly I'm concerned that I have missed something somewhere that will manifest in a much nastier way down the track.

To illustrate the problem, on this just past Friday (1st July):
Using IBC, I run my script at 845am AEST(Sydney), and using reqHistoricalData it backfills daily data correctly up to and including 30th of June, and then it sits quietly until 10am (AEST) for market open to update me with live data.

-------------------------------------------
2022-07-01 08:45:25.529 [GS] INFO? [JTS-PostAuthenticateS2-23] - Started on 20220630-22:45:25
2022-07-01 08:45:25.529 [GS] INFO? [JTS-PostAuthenticateS2-23] - Default timezone:Australia/Sydney name:Australian Eastern Standard Time (New South Wales) dst:3600000
/////////////////////TRUNCATED///////////////////////////////
2022-07-01 09:59:53.444 [GS] INFO? [JTS-CCPDispatcherS2-35] - CCP reported time: 2022-06-30 23:59:53.000 (GMT) [1656633593000], local time: 2022-06-30 23:59:53.442 (GMT), local time: 2022-07-01 09:59:53.442 (AEST), difference in seconds: -0.442, handling delta:0 [Msg1]
2022-07-01 10:00:11.010 [GS] INFO? [JTS-EServerSocketNotifier-93] - [1:157:157:1:0:90:-1:INFO] Sending historical data update [requestId: 2]...
2022-07-01 10:00:11.012 [GS] INFO? [JTS-EServerSocketNotifier-93] - [1:157:157:1:0:90:-1:DET] [90;2;541;20220702;40.830;40.740;40.840;40.710;40.82844666903462;413859]
-------------------------------------------

As a test, I sat and watched it feed me data from the '2nd of July' for more than an hour before manually handling it.

It seems that the Gateway reads my login screen settings correctly and knows that my computer is in AEST timezone, and gives me historical data correctly, but when it returns live data, it adds 1 day to the date which in this case, the 2nd of July being a Saturday and therefore not a trading day, makes no sense.

From my understanding of the IBKR github, with reqHistoricalData and keepUpToDate=True, there is no way to set a timezone as it just reads off the TWS login screen. I have checked my TWS login screen timezone, Windows timezone settings, combed through this group's topics as well as done a code review and nothing has come to mind.

If this wasn't an intermittent problem then I would just subtract 1 from the date to make it work properly or perhaps use reqRealTimeBars, but the fact that restarting Gateway can usually resolve this problem suggests to me the issue is somewhere else.

Has anyone else come across this or have any ideas/thoughts?

Cheers and thanks in advance
Re: Gateway disconnects multiple times daily at around 2pm, sometimes failing to reconnect

 

On Wed, Jun 29, 2022 at 06:05 PM, Kevin wrote:
e
I have the same experience as you Kevin, not sure why, since I've selected 'Asia' server in the TWS/Gateway login screen. I've found the IB response rate on tickets this year has also been abysmal, they must be snowed in. I might give them a call one of these days and see if they can offer any further insight or solutions. Otherwise, I'll keep using IBC to handle the disconnects
Re: tickPrice tick type "4" (last price) not being emitted for Forex?

 

¿ªÔÆÌåÓý

I can¡¯t imagine that they would have trade prices either. Try it and see for yourself?

?

As I understand it, the raw data for the Forex feeds from IDEALPRO come from a wide variety of different sources, and many (all?) of them just don¡¯t disseminate trade info.

?

?

From: [email protected] <[email protected]> On Behalf Of dam5h
Sent: 01 July 2022 19:41
To: [email protected]
Subject: Re: [TWS API] tickPrice tick type "4" (last price) not being emitted for Forex?

?

Got it, does that mean the the 5 sec real time bars also don't contain actual trade prices but rather mid points too?? I suppose tick-by-tick subscription won't have last either then, is that right?

Best,

Dave
4421 - 4440 of 53751