¿ªÔÆÌåÓý

ctrl + shift + ? for shortcuts
© 2025 Groups.io

Multiple symbol request speed


 

Hello i new here i try get data for multiple symbol,
about 580 +-,
but it take allot time and not always finish this.
this how i try make this.

from ibapi.client import EClient
from ibapi.wrapper import EWrapper
from ibapi.contract import Contract
import pandas as pd
import threading
import time
from datetime import timedelta
import datetime
tickers=list(pdToList[0:600]) #########-> list example i put here <-#########
listnumb = []
from ibapi.common import BarData
from random import random
start = datetime.datetime.utcnow()
class IBapi(EWrapper, EClient):
????def __init__(self):
????????EClient.__init__(self, self)
????????self.data = []
????def error(self, reqId, errorCode, errorString):
????????print("Error. Id: ", reqId, " Code: ", errorCode, " Msg: ", errorString)
????def historicalTicksBidAsk(self, reqId, ticks, done):
????????for i in listnumb:
????????????mama = int(''.join(filter(str.isdigit, i)))
????????????if str(reqId) == str(mama):
????????????????symbllll = str(i)
????????????????result = ''.join([i for i in symbllll if not i.isdigit()])
????????????????symbllll = str(result)
????????????????break
????????for tick in ticks:
????????????print("HistoricalTickBidAsk. ReqId:", reqId , "symb : " + symbllll, tick)
????????????
????# include this callback to track progress and maybe disconnectwhen all are finished
????def historicalDataEnd(self, reqId: int, start: str, end: str):
????????print("finished")
def run_loop():
????app.run()
app = IBapi()
app.connect('127.0.0.1', 4001, 123)
api_thread = threading.Thread(target=run_loop, daemon=True)
api_thread.start()
time.sleep(1)
reqId = 1
sym_dict = {}
start_time = time.time()
for sym in tickers:
????contract = Contract()
????contract.symbol = str(sym)
????sym_dict[reqId] = sym
????contract.secType = "STK"
????contract.exchange = "SMART"
????contract.currency = "USD"
????listnumb.append(sym + str(reqId))
????app.reqHistoricalTicks(reqId, contract, "20211022 04:09:40", "", 1, "BID_ASK", 1, False, [])
????reqId += 1


 

Try to obtain the data asynchronically



Example from ib-ruby

Source-code:

Documentation: https://ib-ruby.github.io/ib-doc/market_price.html

i.e.
> market_data = ["BMY", "CAT","DE","FL","GE","IBM","K","KO","T","VZ"].map{|y| IB::Stock.new( symbol: y).verify.first}
> market_data.map{|c| c.market_price(thread: true)}.join
> puts market_data.sort_by( &:symbol).map{|c| [c.symbol, c.misc.to_f].join " -> "? }.join "\n"
BMY -> 54.05
CAT -> 129.99
DE -> 147.87
FL -> 50.88
GE -> 8.02
IBM -> 121.7
K -> 61.98
KO -> 50.3
RH -> 113.2
T -> 30.32
VZ -> 60.23
?=> nil
>
It takes about 10 sec. to obtain the results.

have fun.


 

Hey i see its c++ not have make it faster in python ? i use theard in my code


 

How long does a successful run take?

I am wondering whether your loop exceeds the acceptable rate of requests (50 requests per second) and that your client gets penalized for that. In that case I can think of two approaches you could try:

First you could try the PACEAPI connection option. This option exists since TWS API 974 and makes TWS pace your requests for you. Here the release notes description:

Better Pacing: API messages sent at a higher rate than 50/second can now be paced by TWS at the 50/second rate instead of potentially causing a disconnection. This is now done automatically by the RTD Server API and can be done with other API technologies by invoking SetConnectOptions("+PACEAPI") prior to eConnect.

You could also add pacing code to your client code, meaning you could sleep for a small amount of time before each call to reqHistoricalTicks (say 50ms to 100ms) or larger amount after 10 or 20 calls.

Hope this helps.

´³¨¹°ù²µ±ð²Ô

PS: Using sleep calls is generally a bad idea when waiting for events or results in an asynchronous API such as TWS API. In fact you should remove the time.sleep(1) after api_thread.start() and suspend your thread until the connected() callback indicates a successful connection.

Sleep calls before requests for pacing is acceptable and sometimes necessary.


 

On Mon, Oct 25, 2021 at 01:23 AM, ´³¨¹°ù²µ±ð²Ô Reinold wrote:
SetConnectOptions("+PACEAPI")
i try it for 50 symbols but it take like 300 sec and not always work :X


 

I was able to do a little better than 50 symbols per 300 seconds with our Java framework, but I think there is an IBKR policy enforced limit of about 2 seconds per request once you go beyond +/-10 symbols.

Please keep in mind that IBKR is a brokerage and not a data provider. Also, the TWS API won't allow for operations that TWS does not need. And Bid/Ask information for 500 symbols is not needed for TWS. [Just to make sure. I do not work for or with IBKR in any capacity]

Your question intrigued the researcher in me and I tried a few things such as:

  • Placing all 500+ requests up front while observing the 50 requests per second limit)
  • Placing a rolling window of 10 to 50 requests at a time
  • Placing pretty much one request at a time

No matter what I did I ended up at about one result per 2 seconds. Or 100 seconds for 50 requests and 1,000 seconds for 500 requests.

Attached a couple charts.

´³¨¹°ù²µ±ð²Ô

On Sun, Oct 24, 2021 at 06:34 PM, <danndann4488@...> wrote:

On Mon, Oct 25, 2021 at 01:23 AM, ´³¨¹°ù²µ±ð²Ô Reinold wrote:
SetConnectOptions("+PACEAPI")
i try it for 50 symbols but it take like 300 sec and not always work :X


 

¿ªÔÆÌåÓý

"You could also add pacing code to your client code, meaning you could sleep for a small amount of time?before?each call to reqHistoricalTicks (say 50ms to 100ms) or larger amount after 10 or 20 calls."

I made a simple Java helper class to handle that.? I attached the code, same as pasted below.? It simply keeps track of when requests are made and then blocks long enough to avoid a pacing violation.? I call it right before calls to?reqMktData().? Seems to allow the code to run faster than the +PACEAPI.

import java.util.LinkedList;
import java.util.Queue;

public class PaceMaker {
private int maxrequests;
private Queue<Long> requesttimes;

public PaceMaker(int maxrequestspersecond) {
this.maxrequests = maxrequestspersecond;
this.requesttimes = new LinkedList<Long>();
}
//blocks to stay under max requests per second
public void submitRequest() {
requesttimes.add(System.currentTimeMillis());
while(!processQueue()) {
try {
Thread.sleep(2);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}

private boolean processQueue() {
long freetime = System.currentTimeMillis() - 1000;
while(!requesttimes.isEmpty() && requesttimes.peek() < freetime) {
requesttimes.poll();
}
return requesttimes.size() < maxrequests;
}
}


From: [email protected] <[email protected]> on behalf of ´³¨¹°ù²µ±ð²Ô Reinold via groups.io <TwsApiOnGroupsIo@...>
Sent: Sunday, October 24, 2021 3:23 PM
To: [email protected] <[email protected]>
Subject: Re: [TWS API] Multiple symbol request speed
?

How long does a successful run take?

I am wondering whether your loop exceeds the acceptable rate of requests (50 requests per second) and that your client gets penalized for that. In that case I can think of two approaches you could try:

First you could try the PACEAPI connection option. This option exists since TWS API 974 and makes TWS pace your requests for you. Here the release notes description:

Better Pacing: API messages sent at a higher rate than 50/second can now be paced by TWS at the 50/second rate instead of potentially causing a disconnection. This is now done automatically by the RTD Server API and can be done with other API technologies by invoking SetConnectOptions("+PACEAPI") prior to eConnect.

You could also add pacing code to your client code, meaning you could sleep for a small amount of time before each call to reqHistoricalTicks (say 50ms to 100ms) or larger amount after 10 or 20 calls.

Hope this helps.

´³¨¹°ù²µ±ð²Ô

PS: Using sleep calls is generally a bad idea when waiting for events or results in an asynchronous API such as TWS API. In fact you should remove the time.sleep(1) after api_thread.start() and suspend your thread until the connected() callback indicates a successful connection.

Sleep calls before requests for pacing is acceptable and sometimes necessary.