开云体育

[I sent this on Tuesday at 15:07 UK time, but it seems to have gone down a
hole, so I'll try again.]

Actually IB never accumulate anything (except total volume) and they don't
throttle. They just don't report every trade or every bid and ask.

You may find the following of interest. It is a composite of several posts
I've made here in recent months, and is based on some very intensive
research into how the IB data feed works that I undertook some years ago (as
far as I can see, it still works the same way today).

The IB datafeed is optimised to ensure that it keeps up with the market no
matter how busy the market is. As a result of this IB doesn't report every
trade, or every bid or ask, for any instrument in any exchange.

To accomplish this, it effectively sends a price snapshot for each
instrument at regular intervals. This interval seems to be about 300
milliseconds. For each of bid, ask, and last it compares the current price
and size with the values at the last snapshot. If the price is different it
sends both price and size. If the price is the same, but the size is
different it sends only the size. If both price and size are the same, it
doesn't send either. If there have been any trades since the last snapshot,
it sends the (accumulated) volume (so where the price and size haven't
changed but there have been one or more trades, this can be detected from
the increased volume).

By doing this, IB knows the maximum bandwidth required for each ticker, and
hence for each customer (since the number of tickers is limited), and so it
can size its servers to be able to cope with that load. If a market becomes
very busy, it makes no difference because it will still only send an update
three times a second or thereabouts, even if there have been 100 trades
during that second. This avoids the problem that every other data feed seems
to have, where the data will sometimes lag way behind the market at busy
times (with every other vendor I've used, I've had occasions where the data
could be anything up to two or three minutes behind the market).

There is an irritating side effect of this technique, which is that price
movements between shapshots may not be reported at all: for example if the
last price at snapshot 1 is 100, and then price moves up to 102 and then
back to 101 by snapshot 2, the price reported at snapshot 2 will be 101, and
the 102 price will not be reported at all. This leads to occasional
incorrect highs and lows of bars, but rarely by more than one tick: whether
that is significant depends very much on the trading strategy used.

In my experience, the number of occasions where the inaccuracies have any
significant effect is so small that it can be ignored, but that of course is
partly a function of the particular strategy I was using - note however that
it traded aggresively off 1-minute bars and used very tight stops, so you
might think it would be quite sensitive to inaccuracies. Besides, sometimes
the inaccuracies work against you and sometimes they help you, so overall I
think it pretty much balances out.

The above isn't a complete description, but it covers the basic mechanism. A
word of caution though: this is not an exact science. It would be nice if
what I said was an exact description of how it works, but you'll find odd
things happening occasionally, such as a volume update without a prior size
message where the increase in volume is not an exact multiple of the most
recent size message, or multiple last price/size messages sent at the same
time, or volume messages with a smaller volume than the previous one! But
most of the time my description is accurate.

Another point worth making is that the snapshots are taken at different
times for different TWS sessions. This is sensible from IB's point of view
because it means they don't have to send updates to all their thousands of
users at once. But it has the interesting side effect that if you run two
copies of TWS (eg different logins to the same account) you'll get slightly
different data from them, because the snapshotting is not being done
simultaneously.

By the way, one gotcha that I didn't mention above is that when both price
and size messages are sent (in a single TICK_PRICE socket message), TWS also
sends the size again in a separate TICK_SIZE message, but the volume is
correctly updated only once. I think the reason for this duplication is that
before the version 2 TICK_PRICE message was introduced, it didn't contain a
size field, so prices and sizes were always sent separately: if TWS didn't
send the duplicate size, then programs that relied on the separate TICK_SIZE
message would no longer work properly unless they were amended and
recompiled.

For backtesting, I've always used the IB realtime data which I collect in an
SQL database, and I've been perfectly happy with that. The fact is that
backtesting is very much only an approximation, especially if your strategy
uses any kind of limit orders: with market or stop orders you know that your
order would have been filled, but you can't be sure of what price you would
have got, whereas with limit orders you know at least the worst price you
would have got but you can't be sure you would have been filled at all
(unless price actually goes beyond your limit) - and your strategy might
behave quite differently subsequently depending on whether an order was
filled or not.

As an alternative to using the collected data for backtesting, one could
instead use the historical data supplied by IB - I believe that it reflects
the complete information sent by the exchange. The problem here is that IB's
historical data is not always complete, even for periods where the realtime
data feed was ok. One could also try a hybrid approach where you collect the
realtime data and then update it later on with whatever historical data you
can get from IB. But in my view, this is overkill.

Richard