Keyboard Shortcuts
ctrl + shift + ? :
Show all keyboard shortcuts
ctrl + g :
Navigate to a group
ctrl + shift + f :
Find
ctrl + / :
Quick actions
esc to dismiss
Likes
Search
twsapi: Event driven garbage
grozzie
toggle quoted message
Show quoted text
How much time elapsed during this series of events? Kent ----- Original Message -----
From: "grozzie2" <grozzie2@...> To: <twsapi@...> Sent: Sunday, July 21, 2002 3:43 PM Subject: twsapi: Event driven garbage TWS is an event driven beast. No functions actually have any kind of return to validate they actually did something, everything waits for callbacks, to say events occurred. Event driven models are great, provided, the event source reports the events in the actual order they occur. TWS does not. If you are basing code on the events from the TWS activeX component, and building up logic for handling orders etc on it, be warned, I have a captured log with the following sequence of events, and I'm pondering what to do with it. The state begins with a short position of 20 contracts in the es minis. The program has a target cover buy in place, and a stop, for 20 contracts each, and joined as a one cancels all group. Event 1 - TWS reports partial fill for 11 from target order program modifies internal structures to show -9 left in the position, with 11 bought. Event 2 - TWS reports current position on ES as -9 program verifies this as correct. Event 3 - TWS reports partial fill for 9 contracts, order filled completely. program modifies internal structures to show target order filled, and removed from the operating state. Event 4 - TWS reports current position on ES as -9 contracts program goes HUH ????????? program realizes, all buy orders are filled, and it still has a short position of 9 contracts, places an order to buy 9 to cover. Event 5 - TWS reports order to buy filled. program modifies internal structures to show buy now filled, and position back to 0 Event 6 - TWS reports position of 0 program verifies this is the correct position Event 7 - TWS reports current position on ES as 9 contracts program emits expletives into the debug log. Computers are fast, and can react much faster than humans. What's important to note in the above sequence, TWS reported a WRONG position, and the program reacted. The position reported was correct for the time BEFORE the second fill report, and this is significant, because TWS reported the -9 position TWICE, once before, and once after the fill for 9. It looks to me like the TWS internal state on actual position size got delayed, and would have 'caught up' had my program waited longer before doing order/position reconciliation. A human cant react that fast, so if it displayed -9, and updated that display to 0 a fraction of a second later, or even a couple seconds later, a human would go 'great, filled'. 2 weeks ago, I realized, TWS events are not necessarily in sequence, and had settled on the position report event as the 'atomic' point of action, on the assumption, it would be the one event we could count on to report things, and if we postponed all position management to that event, everything would stay in synch. Up until now, it's appeared that would work, but now i see it wont. I now have logs of various orders of events during the process to prove, without any possibility of it being otherwise, there is no TWS event that you can rely on as an atomic thing, ie, it will be correct. I have fills updated before positions, positions updated before fills, and now i add to it, position reported late, or in error. For now, i'll assume it was late, because if it was actually in error, and TWS was not going to follow the second -9 report with a 0 report, then it will not be possible to build a 'safe' program using this. I can say the following with a certainty now. TWS is 2 or more independant state machines internally, reporting events with no relationships in time or order of events. If you are trying to reconcile the events, and take actions based on the reports from tws in real time, at the time of events, then the following holds true. If you are trading more than quantity 1, at some point in time, you will recieve event notifications that will come in an order that breaks your logic. I have drawn out carefully the timing diagrams from a number of event sequences in the area of partial fills, both on entry and on exit. The last one provides a proof, there is no event from TWS that is atomic, on which you can base decisions to manage existing positions, and reconcile with fill reports. The TWS position reports are independant of fill reports, and there will ALWAYS be potential for discrepancies. The only way to do a reconciliation, is to postpone it until a point in time that you can verify there are no potential outstanding events. It's time to re-design here. To unsubscribe from this group, send an email to: twsapi-unsubscribe@... Your use of Yahoo! Groups is subject to |
--- In twsapi@y..., "Kent Rollins" <kentr@m...> wrote:
grozzieTotal elapsed time of that sequence was just inder 1.5 seconds. I dont use the 'poll the socket on a timer' methodology that the sample code uses, I have a dedicated thread reading the socket from TWS, so my events arrive in the rest of the application 'as they come in', with no delays. The thread reading the socket is running as the highest priority thread in the system. Since all the events come from a single thread, that's reading the socket serially, so there is no doubt i'm getting events from tws in the order it's sending them. I'm less interested in the time of the sequence, than the order. The order there proves, position reports are not sychronized with fill reports. That introduces a lot of 'issues'. |
My first thought as I was reading your original post was to introduce a
toggle quoted message
Show quoted text
small delay in processing responses. I agree completely that the message order problem you have found is serious and needs fixin'. IB has taken a difficult route in making this one app into a User interface, an ActiveX interface, and a socket interface. Separate apps/libraries would have been a better way to go. Kent ----- Original Message -----
From: "grozzie2" <grozzie2@...> To: <twsapi@...> Sent: Sunday, July 21, 2002 10:34 PM Subject: Re: twsapi: Event driven garbage --- In twsapi@y..., "Kent Rollins" <kentr@m...> wrote: grozzieTotal elapsed time of that sequence was just inder 1.5 seconds. I dont use the 'poll the socket on a timer' methodology that the sample code uses, I have a dedicated thread reading the socket from TWS, so my events arrive in the rest of the application 'as they come in', with no delays. The thread reading the socket is running as the highest priority thread in the system. Since all the events come from a single thread, that's reading the socket serially, so there is no doubt i'm getting events from tws in the order it's sending them. I'm less interested in the time of the sequence, than the order. The order there proves, position reports are not sychronized with fill reports. That introduces a lot of 'issues'. To unsubscribe from this group, send an email to: twsapi-unsubscribe@... Your use of Yahoo! Groups is subject to |
marinindextrader
Kind of depressing Grozzie....Programming is not difficult at all,
toggle quoted message
Show quoted text
just complex. And it is this sort of nonesense that makes it even more complex... What a bummer... They should just turn over all the source to us and then pay us to re- write the whole bloody mess... Scott --- In twsapi@y..., "grozzie2" <grozzie2@y...> wrote:
--- In twsapi@y..., "Kent Rollins" <kentr@m...> wrote:grozzieTotal elapsed time of that sequence was just inder 1.5 |
--- In twsapi@y..., "marinindextrader" <marinindextrader@y...> wrote:
Kind of depressing Grozzie....Programming is not difficult at all,re- write the whole bloody mess...Well, it can be made to work, and like i pointed out in the prior message, if you can live without cross checks, there is always a way. I cant live without cross checks, and as I press on, i'm finding ways to get it accomplished. The key to doing reconciliation between positions and orders, is to only do it at a time, when you know it'll be a valid result. I've had full automatic running here now for a few hours, doing lots of full turn arounds (restart tws now and then to reprime the balance :) and it's not had an order or position screw up, getting tons of partials. Hehe, this program is actually lying in wait, watching for the correct setup to generate lots of 'hard to handle conditions', and then pasting bids and offers with the maximum size it can, irrelavent of p/l, just churning orders to generate bad case trace logs. If this continues on for another hour like this, then we restart the 'real' test plan tomorrow, and if it survives that, we'll be on the 'real money account' test phase before the end of the week. The model I'm using now, is by no means optimum, but, I think it crosses over the line to the side of 'reliable'. For our application, this is key, and essential. If this holds up, then we can take a step back, and work with optimizing the model. |
to navigate to use esc to dismiss