¿ªÔÆÌåÓý

ctrl + shift + ? for shortcuts
© 2025 Groups.io

twsapi: Event driven garbage


 

grozzie

How much time elapsed during this series of events?

Kent

----- Original Message -----
From: "grozzie2" <grozzie2@...>
To: <twsapi@...>
Sent: Sunday, July 21, 2002 3:43 PM
Subject: twsapi: Event driven garbage


TWS is an event driven beast. No functions actually have any kind of
return to validate they actually did something, everything waits for
callbacks, to say events occurred.

Event driven models are great, provided, the event source reports the
events in the actual order they occur. TWS does not.

If you are basing code on the events from the TWS activeX component,
and building up logic for handling orders etc on it, be warned, I
have a captured log with the following sequence of events, and I'm
pondering what to do with it.

The state begins with a short position of 20 contracts in the es
minis. The program has a target cover buy in place, and a stop, for
20 contracts each, and joined as a one cancels all group.

Event 1 - TWS reports partial fill for 11 from target order
program modifies internal structures to show -9 left in the
position, with 11 bought.
Event 2 - TWS reports current position on ES as -9
program verifies this as correct.
Event 3 - TWS reports partial fill for 9 contracts, order filled
completely.
program modifies internal structures to show target order filled, and
removed from the operating state.
Event 4 - TWS reports current position on ES as -9 contracts
program goes HUH ?????????
program realizes, all buy orders are filled, and it still has a short
position of 9 contracts, places an order to buy 9 to cover.
Event 5 - TWS reports order to buy filled.
program modifies internal structures to show buy now filled, and
position back to 0
Event 6 - TWS reports position of 0
program verifies this is the correct position
Event 7 - TWS reports current position on ES as 9 contracts
program emits expletives into the debug log.


Computers are fast, and can react much faster than humans. What's
important to note in the above sequence, TWS reported a WRONG
position, and the program reacted. The position reported was correct
for the time BEFORE the second fill report, and this is significant,
because TWS reported the -9 position TWICE, once before, and once
after the fill for 9. It looks to me like the TWS internal state on
actual position size got delayed, and would have 'caught up' had my
program waited longer before doing order/position reconciliation. A
human cant react that fast, so if it displayed -9, and updated that
display to 0 a fraction of a second later, or even a couple seconds
later, a human would go 'great, filled'.

2 weeks ago, I realized, TWS events are not necessarily in sequence,
and had settled on the position report event as the 'atomic' point of
action, on the assumption, it would be the one event we could count
on to report things, and if we postponed all position management to
that event, everything would stay in synch. Up until now, it's
appeared that would work, but now i see it wont.

I now have logs of various orders of events during the process to
prove, without any possibility of it being otherwise, there is no TWS
event that you can rely on as an atomic thing, ie, it will be
correct. I have fills updated before positions, positions updated
before fills, and now i add to it, position reported late, or in
error. For now, i'll assume it was late, because if it was actually
in error, and TWS was not going to follow the second -9 report with a
0 report, then it will not be possible to build a 'safe' program
using this.

I can say the following with a certainty now. TWS is 2 or more
independant state machines internally, reporting events with no
relationships in time or order of events. If you are trying to
reconcile the events, and take actions based on the reports from tws
in real time, at the time of events, then the following holds true.
If you are trading more than quantity 1, at some point in time, you
will recieve event notifications that will come in an order that
breaks your logic. I have drawn out carefully the timing diagrams
from a number of event sequences in the area of partial fills, both
on entry and on exit. The last one provides a proof, there is no
event from TWS that is atomic, on which you can base decisions to
manage existing positions, and reconcile with fill reports. The TWS
position reports are independant of fill reports, and there will
ALWAYS be potential for discrepancies. The only way to do a
reconciliation, is to postpone it until a point in time that you can
verify there are no potential outstanding events.

It's time to re-design here.





To unsubscribe from this group, send an email to:
twsapi-unsubscribe@...



Your use of Yahoo! Groups is subject to


 

--- In twsapi@y..., "Kent Rollins" <kentr@m...> wrote:
grozzie

How much time elapsed during this series of events?

Kent
Total elapsed time of that sequence was just inder 1.5
seconds. I dont use the 'poll the socket on a timer'
methodology that the sample code uses, I have a dedicated
thread reading the socket from TWS, so my events arrive
in the rest of the application 'as they come in', with
no delays. The thread reading the socket is running as
the highest priority thread in the system. Since all the
events come from a single thread, that's reading the socket
serially, so there is no doubt i'm getting events from tws
in the order it's sending them.

I'm less interested in the time of the sequence, than the
order. The order there proves, position reports are not
sychronized with fill reports. That introduces a lot of
'issues'.


 

My first thought as I was reading your original post was to introduce a
small delay in processing responses. I agree completely that the message
order problem you have found is serious and needs fixin'.

IB has taken a difficult route in making this one app into a User interface,
an ActiveX interface, and a socket interface. Separate apps/libraries would
have been a better way to go.

Kent

----- Original Message -----
From: "grozzie2" <grozzie2@...>
To: <twsapi@...>
Sent: Sunday, July 21, 2002 10:34 PM
Subject: Re: twsapi: Event driven garbage


--- In twsapi@y..., "Kent Rollins" <kentr@m...> wrote:
grozzie

How much time elapsed during this series of events?

Kent
Total elapsed time of that sequence was just inder 1.5
seconds. I dont use the 'poll the socket on a timer'
methodology that the sample code uses, I have a dedicated
thread reading the socket from TWS, so my events arrive
in the rest of the application 'as they come in', with
no delays. The thread reading the socket is running as
the highest priority thread in the system. Since all the
events come from a single thread, that's reading the socket
serially, so there is no doubt i'm getting events from tws
in the order it's sending them.

I'm less interested in the time of the sequence, than the
order. The order there proves, position reports are not
sychronized with fill reports. That introduces a lot of
'issues'.





To unsubscribe from this group, send an email to:
twsapi-unsubscribe@...



Your use of Yahoo! Groups is subject to


marinindextrader
 

Kind of depressing Grozzie....Programming is not difficult at all,
just complex. And it is this sort of nonesense that makes it even
more complex...

What a bummer...

They should just turn over all the source to us and then pay us to re-
write the whole bloody mess...



Scott

--- In twsapi@y..., "grozzie2" <grozzie2@y...> wrote:
--- In twsapi@y..., "Kent Rollins" <kentr@m...> wrote:
grozzie

How much time elapsed during this series of events?

Kent
Total elapsed time of that sequence was just inder 1.5
seconds. I dont use the 'poll the socket on a timer'
methodology that the sample code uses, I have a dedicated
thread reading the socket from TWS, so my events arrive
in the rest of the application 'as they come in', with
no delays. The thread reading the socket is running as
the highest priority thread in the system. Since all the
events come from a single thread, that's reading the socket
serially, so there is no doubt i'm getting events from tws
in the order it's sending them.

I'm less interested in the time of the sequence, than the
order. The order there proves, position reports are not
sychronized with fill reports. That introduces a lot of
'issues'.


 

--- In twsapi@y..., "marinindextrader" <marinindextrader@y...> wrote:
Kind of depressing Grozzie....Programming is not difficult at all,
just complex. And it is this sort of nonesense that makes it even
more complex...

What a bummer...

They should just turn over all the source to us and then pay us to
re-
write the whole bloody mess...
Well, it can be made to work, and like i pointed out in the prior
message, if you can live without cross checks, there is always a
way. I cant live without cross checks, and as I press on, i'm
finding ways to get it accomplished. The key to doing reconciliation
between positions and orders, is to only do it at a time, when you
know it'll be a valid result.

I've had full automatic running here now for a few hours, doing lots
of full turn arounds (restart tws now and then to reprime the
balance :) and it's not had an order or position screw up, getting
tons of partials. Hehe, this program is actually lying in wait,
watching for the correct setup to generate lots of 'hard to handle
conditions', and then pasting bids and offers with the maximum size
it can, irrelavent of p/l, just churning orders to generate bad case
trace logs.

If this continues on for another hour like this, then we restart
the 'real' test plan tomorrow, and if it survives that, we'll be on
the 'real money account' test phase before the end of the week.

The model I'm using now, is by no means optimum, but, I think it
crosses over the line to the side of 'reliable'. For our
application, this is key, and essential. If this holds up, then we
can take a step back, and work with optimizing the model.