Keyboard Shortcuts
Likes
Search
Re: Running multiple algorithms in parallel
¿ªÔÆÌåÓýBuddy ? I¡¯m perfectly happy for you to ¡®harp on this¡¯ because what you discovered is an important issue. It was you who took your ball home, not me¡ ? By the way, it¡¯s not necessary to treat me as if I¡¯m a junior programmer that knows nothing about debugging, race conditions, etc. Look up the expression ¡®teach your grandmother to suck eggs¡¯ ¨C I¡¯m your grandmother!¡ ? You don¡¯t seem to have picked up on the fact that I have been running your script, and with the call to sleep it hasn¡¯t caused any problems. ? But I¡¯m perfectly willing to admit that I haven¡¯t run the script repeatedly in a loop as you suggest, and I note with interest what you say. So I¡¯ll do ?this tomorrow. ? I also want to mention something that you may or may nor be aware of. When clients make marketdata requests for a given contract, they all end up receiving exactly the same ticks (after the first few that just establish the current high, low, volume, previous close, etc).? So no matter how many clients are receiving data for that contract, there is only a single market data stream coming from IB¡¯s market data servers to TWS, and TWS just passes this data out via the individual API socket connections to the clients. That single stream represents 1 count off your ticker allowance. Moreover it¡¯s actually even better than this, because if both the live and the paper TWS are supplying data for particular contract, that again is just the identical data for both, and just one ticker count. ? So clearly there is some co-ordination going on between the TWSs and the market data servers, and I suspect that the fault lies in here somewhere. ? ? ? From: [email protected] <[email protected]> On Behalf Of buddy
Sent: Wednesday, April 5, 2023 6:24 PM To: [email protected] Subject: Re: [TWS API] Running multiple algorithms in parallel ? On Tue, Apr 4, 2023 at 05:38 PM, Richard L King wrote:
Richard, I hate to harp on this but I think your understanding of the bug is coming along and so I would like to point out, again, that cancelling the market data stream does not prevent the eventual deadlock. In fact it is very easy to refute that it does... just apply the proposed fix to the example and then run the example in a loop. In other words, apply this: to the example and then run in a forever loop, e.g: The deadlock may not occur immediately but will happen in somewhat short order. When I tried this the loop would hang by the 12th iteration with a very high likelihood; usually earlier. This behavior is certainly indicative of a race condition; be it in a destructor or reference counter as you suggest. This is why I was encouraging you to forego "issuing API commands discretely" and test the example programmatically and thoroughly. Race conditions often disappear in a debugger or when you "step" through code because the code timing is affected. Additionally, in a real-world application, attempting to "fix" the race condition by introducing an artificial delay wouldn't help since the program would be running in parallel at random overlapping times and therefore any artificial delay would be canceled out by the start times. Anyway, I'm glad you're feeling better. On Sat, Apr 1, 2023 at 10:14 PM, J¨¹rgen Reinold wrote:
J¨¹rgen, I'll be the first to admit that the bug I've been droning on about is a corner case. However... it is a very good example of why serious consideration should be given to the decision of implementing a multi-threading/processing design. Multi-threading/processing can sometimes be difficult to implement and often creates very tricky problems that even experienced folks have problems troubleshooting and understanding. And, even if the application design is solid, sometimes the cause is poor vendor support which can be out of the developers control. |