how long does historical pull take?

[email protected]

#34526

I am using the sample spreadsheet TwsDde to pull historical data via the API.

Symbl: QQQ

Type: STK

Exchange: NASDAQ

Currency: USD

End Date/Time:

20151005 18:25:18 GMT

Duration: 5 D

Bar Size: 5 (1 min)

What to show: MIDPOINT

RTH Only: 1

Date Format Style: 1?

In the CTRL column of the spreadsheet it says "PROCESSING" - How long should I wait for? Not sure if there is any progress.

Let me know if you need any additional information from me to help answer my question.

Thank you in advance!

#34527

You are probably asking for too much data in a single request.? I believe that you can only get 2 days of data when requesting 1 minute bars.?

There's a table of duration vs bar size limits somewhere in the documentation.

#34528

No, the limits have been increased. You can get up to 10 days of 1-minute bars now, at least with recent TWS versions.

But I think you’re right that the OP is asking for a lot of data, and the Excel DDE is probably rather (very?) slow at processing it.

From: TWSAPI@... [mailto:TWSAPI@...]
Sent: 20 November 2015 10:59
To: TWSAPI@...
Subject: [TWS API] Re: how long does historical pull take?

You are probably asking for too much data in a single request.  I believe that you can only get 2 days of data when requesting 1 minute bars.

There's a table of duration vs bar size limits somewhere in the documentation.

#34531
Edited

Hi Richard,

?

I seems missed "the limits have been increased" event and so far using the limits which circulated the group years ago. This is the table I use (and documented in faq):

?

I'm most interested in 1sec historical bars.

The table above suggests the limit for 1sec historical bars is 2000 bars (which is only about 33 minutes).

Could you point me to some doc which describes the latest changed limits please?

?

I checked "IB API - Guides" -> "Printable Reference Guide" and under "Valid Duration and Bar Size Settings for Historical Data Requests" they have old table, which says max duration for 1sec bars is 1800 (30 minutes) - same as last couple years.. and it is 3.(3) minutes shorter than actual max limit of 2000bars which is somehow not documented in official sources.

?

It would be awesome to see increase in number of bars because we allowed to send only so many reqests per minute (I allow next historical data request after 10.1 seconds after previous was sent per each connected TWS account).

?

thank you,

Dmitry

?

On Fri, Nov 20, 2015 at 6:12 AM, 'Richard L King' rlking@... [TWSAPI] <TWSAPI@...> wrote:

?

No, the limits have been increased. You can get up to 10 days of 1-minute bars now, at least with recent TWS versions.

But I think you’re right that the OP is asking for a lot of data, and the Excel DDE is probably rather (very?) slow at processing it.

From: TWSAPI@... [mailto:TWSAPI@...]
Sent: 20 November 2015 10:59
To: TWSAPI@...
Subject: [TWS API] Re: how long does historical pull take?

You are probably asking for too much data in a single request. I believe that you can only get 2 days of data when requesting 1 minute bars.

There's a table of duration vs bar size limits somewhere in the documentation.

?

?

--

с ч.гл.ув.,
Дима Ш.

?

Nick

#34532
Edited

I'm pretty sure I remember a sentence in the docs that says it's possible to exceed the stated bar limits but that the request may count as more than one for the purpose of pacing.

I suppose the only way to find out is experiment with it, but I've reached my limit of trying to document an undocumented and randomly changing api. I just use the old limits since I don't have a need to squeeze performance out of historical downloads.

On 11/20/2015 10:27 AM, Dmitry Shevkoplyas shevkoplyas@... [TWSAPI] wrote:

Show quoted text

?

Hi Richard,
?

I seems missed "the limits have been increased" event and so far using the limits which circulated the group years ago. This is the table I use (and documented in faq):

I'm most interested in 1sec historical bars.

The table above suggests the limit for 1sec historical bars is 2000 bars (which is only about 33 minutes).

Could you point me to some doc which describes the latest changed limits please?

?

I checked "IB API - Guides" -> "Printable Reference Guide" and under "Valid Duration and Bar Size Settings for Historical Data Requests" they have old table, which says max duration for 1sec bars is 1800 (30 minutes) - same as last couple years.. and it is 3.(3) minutes shorter than actual max limit of 2000bars which is somehow not documented in official sources.

?

It would be awesome to see increase in number of bars because we allowed to send only so many reqests per minute (I allow next historical data request after 10.1 seconds after previous was sent per each connected TWS account).

?

thank you,

Dmitry

?

On Fri, Nov 20, 2015 at 6:12 AM, 'Richard L King' rlking@... [TWSAPI] <TWSAPI@...> wrote:

?

No, the limits have been increased. You can get up to 10 days of 1-minute bars now, at least with recent TWS versions.

But I think you’re right that the OP is asking for a lot of data, and the Excel DDE is probably rather (very?) slow at processing it.

From: TWSAPI@... [mailto:TWSAPI@...]
Sent: 20 November 2015 10:59
To: TWSAPI@...
Subject: [TWS API] Re: how long does historical pull take?

You are probably asking for too much data in a single request. I believe that you can only get 2 days of data when requesting 1 minute bars.

There's a table of duration vs bar size limits somewhere in the documentation.

[Non-text portions of this message have been removed]

?
--

с ч.гл.ув.,
Дима Ш.

?

orionn2

#34622
Edited

hi Dmitry,

?

I have updated the table in the jTWSdump website:

?

http://www.tradingsoftwarelab.com/jtwsdump/table.html

?

showing increased limits of 5 Y (5 years) for 1M (1 month), 1W (1 week) and 1 day bars and also 10 D (10 days) for 5 mins, 3 mins, 2 mins and 1 min bars. These limits are for a single data request - much more data can be downloaded by setting the End Date/Time in another API request to a date before 5 years from now. In jTWSdump this can be easily managed by using the "Number of Queries" field.

?

Thanks to Richard L King for the heads-up.

#34629
Edited

great! thanks for update (updated faq)

?

D

On Wed, Dec 16, 2015 at 12:28 PM, orionn2 <no_reply@...> wrote:

?

?

hi Dmitry,

?

I have updated the table in the jTWSdump website:

?

?

showing increased limits of 5 Y (5 years) for 1M (1 month), 1W (1 week) and 1 day bars and also 10 D (10 days) for 5 mins, 3 mins, 2 mins and 1 min bars. These limits are for a single data request - much more data can be downloaded by setting the End Date/Time in another API request to a date before 5 years from now. In jTWSdump this can be easily managed by using the "Number of Queries" field.

?

Thanks to Richard L King for the heads-up.

?

?

?

--

с ч.гл.ув.,
Дима Ш.

?

开云体育

how long does historical pull take?