Keyboard Shortcuts
Likes
Search
Unexpected duplicate orders re-submitted many times within a short time
|
Today when the api-based program was running automatically, several unexpected duplicate orders were re-submitted many times within a short time.
?
These orders can only be re-submitted when there are no such orders in the open/live order list.
?
I happened to be in front of the computer, I noticed that at that moment, although such orders remain normal in the TWS, but the api-based program got the wrong information that the openOrder list is empty through openOrder() on?reqAllOpenOrders() . So the api-based program "thinks" that such orders don't exist at that moment and re-submitted them.
?
I hat to cancel all the orders and restarted the api-based program again, then the program runs correctly again and got the right information.
?
What will lead the openOrder() on reqAllOpenOrders() to getting the wrong information? How to avoid such errors?? |
|
Nick
开云体育You have to look at your log files to determine what actually happened.Was it that open order messages were never received? Was it that the open order messages contain incorrect information? Was it that your program tried to submit orders rapidly before the open order messages had a chance to arrive? There are many possible causes but you have to figure out what actually happened first. On 8/17/2017 9:00 AM, WHmKK Zeng via
Groups.Io wrote:
|
|
Thank you for your reply.
?
Sorrry maybe I didn't state clear enough above.
?
The fact is that the openOrder() got the wrong message that there was no open order existing at that moment(actually there were open orders exsisting at that moment and the TWS show them?
?
correctly).
?
Most of the time the program will get the right openOrder information, but somehow sometime it gets the wrong information that no open order exists without such error message as "Data Service?
?
error". Actually if the program got the "error message", no order will be submitted.
?
My concern is that?
1. what circumstances will possibly cause such "wrong enpmty oepn order information" to be received?
2. and what possible methods can be carried out in advance to avoied such errors in real time? (such erros are really fatal) |
开云体育I can tell you quite categorically that reqOpenOrders() always returns the correct information. If it didn’t, thousands of API programs around the world would stop working properly, IB would be inundated with support requests, and they would fix the problem pretty damn quick. Now, I don’t see anyone else complaining of a problem here, so my conclusion is that you are mistaken and you need to look more closely at what’s actually going on. ? Nick made some helpful suggestions which you seem to have ignored. ? It’s not even clear what you mean. You say ‘the openOrder() got the wrong message that there was no open order existing at that moment’. What do you mean by that? Do you mean you got the openOrderEnd() but no openOrder()s? Or did you get an error message? ? Have you ensured that transmit=true? ? Are you using a live account, a paper trading account, or a demo account? ? And which version of the API are you using anyway? ? Have you turned on API logging in TWS/Gateway so you can see exactly what’s being sent to the API? ? Here’s a piece of advice: the API works and it works well. If you’re using the beta version, you may come across a bug. But generally speaking, if someone complains that the API is not working then it’s 99.9% likely that there is a fault in their code, or their expectation of how it should work is incorrect. ? Richard ? ? ? From: [email protected] [mailto:[email protected]] On Behalf Of WHmKK Zeng via Groups.Io
Sent: 18 August 2017 00:50 To: [email protected] Subject: Re: [TWS API] Unexpected duplicate orders re-submitted many times within a short time ? Thank you for your reply. ? Sorrry maybe I didn't state clear enough above. ? The fact is that the openOrder() got the wrong message that there was no open order existing at that moment(actually there were open orders exsisting at that moment and the TWS show them? ? correctly). ? Most of the time the program will get the right openOrder information, but somehow sometime it gets the wrong information that no open order exists without such error message as "Data Service? ? error". Actually if the program got the "error message", no order will be submitted. ? My concern is that? 1. what circumstances will possibly cause such "wrong enpmty oepn order information" to be received? 2. and what possible methods can be carried out in advance to avoied such errors in real time? (such erros are really fatal) |
|
Ewald de Wit
Perhaps (wild guess) there is a mixup in the openOrder wrapper between responses to placeOrder and reqOpenOrders. Ewald On Fri, Aug 18, 2017 at 11:30 AM, Richard L King <rlking@...> wrote:
|
|
You have TWS open so you are submitting via TWS and not the Gateway. So perhaps you restarted TWS (as you must do each day) and your tool failed to "reconnect" to the new instance and was talking to the air, so to speak?
I would venture to say most people who submit multiple orders as part of an active trading strategy use the Gateway and not TWS.? The IB API can be slow to confirm trades via Order Status. This is an annoyance that must be programmed around. But the IB API does not drop orders, at least over my thousands of trades it hasn't. |
|
To everyone,
?
First of all, thank you all for your reply and good advice.
?
For the past few days, I've tried to catch more details about such error. However, it didn't happen again.?
?
In the mean time, I've added some secure-controls and self-check-fix mechanism into the program. So now I have confidence to let it run automatically.
?
?
Hi Richard,
?
Thank you for your suggestions. Now I've turned on API logging in TWS. So next time it can be traced back more clearly.
?
Btw, my post is no complaining of any kind, you have misunderstanding to this point.?
?
I really don't care "who" is wrong or complain something, my only concern is to avoid such errors in the real time.
?
?
Hi ?fantasy footbal,
?
Thank you for your suggestions. I've never tried Gateway. What is the advantages of Geteway over TWS?
?
You already have "thousands of trades" over your tool? Congratulations! I've just begun the journey :) |
|
Hi Zeng, I've witnessed this bug as well. I called placeOrder and immediately called reqAllOpenOrders and it did not return the OrderId submitted before openOrderEnd. I have not changed my software yet, but I have not seen it occur recently. I planned to fix this by creating a list of orderId's that I have submitted, checking that I see those in the openOrders returned, and if I don't see them, then I know the list is wrong (incomplete), and I should call reqAllOpenOrders again, and wait for the result, before proceeding with any trades on that symbol. I think the API is at fault here. I've not implemented this change in my software yet, so I'm not sure if it will cause secondary issues. As I said, I've not witnessed the bug recently so it's slipped down my list of priorities. I hope this helps, -Jason Sent with Secure Email.
|
开云体育Jason ? This isn’t a bug. When you place an order, it takes TWS a finite time to verify that the order is valid (eg to ensure it doesn’t breach your configured values for order size, etc, also margin checks and so on). TWS almost certainly does this on a different thread than the one it received the placeOrder API message on. TWS doesn’t consider an order as ‘open’ until all those checks have been completed. ? So if you immediately call req(All)OpenOrders after placing the order, TWS will sometimes receive this message before that processing is complete, and will respond with only those orders that are considered ‘open’. ? This isn’t a fault in the API: it’s an inevitable consequence of that fact that there are several processes separated by potentially many miles of wires and fibres involved in pretty much any aspect of the API: your app, TWS/Gateway, IB’s account server, IB’s contract database server, IB’s market data and historical data servers, the exchanges, etc etc. ? There are other situations where similar timing issues arise: for example req(All)OpenOrders might return an open order with a status of ‘Submitted’: however the reality could be that the order has actually been executed and filled at the exchange, but that information just hasn’t filtered back to TWS yet. ? So you need to code carefully whenever you do anything that depends on order status. ? Richard ? ? From: [email protected] [mailto:[email protected]] On Behalf Of Jason via Groups.Io
Sent: 24 August 2017 07:38 To: [email protected] Subject: Re: [TWS API] Unexpected duplicate orders re-submitted many times within a short time ? Hi Zeng, I've witnessed this bug as well. ? I called placeOrder and immediately called reqAllOpenOrders and it did not return the OrderId submitted before openOrderEnd. I have not changed my software yet, but I have not seen it occur recently. ? I planned to fix this by creating a list of orderId's that I have submitted, checking that I see those in the openOrders returned, and if I don't see them, then I know the list is wrong (incomplete), and I should call reqAllOpenOrders again, and wait for the result, before proceeding with any trades on that symbol. ? I think the API is at fault here. ? I've not implemented this change in my software yet, so I'm not sure if it will cause secondary issues. As I said, I've not witnessed the bug recently so it's slipped down my list of priorities. ? I hope this helps, -Jason ? ? Sent with Secure Email. ?
? |
开云体育You seem to object to my use of the word ‘complain’. A complaint is ‘a statement that something is unsatisfactory or unacceptable’. You stated that the API was sending you incorrect openOrder messages, and you clearly felt that this was unsatisfactory. I wasn’t trying to suggest that you or anyone else is whining or whingeing, which I suppose is another interpretation of ‘complaining’. I’m sorry if you made this interpretation, but I assure you that was not my intent. ? Richard ? ? From: [email protected] [mailto:[email protected]] On Behalf Of WHmKK Zeng via Groups.Io ? To everyone, ? First of all, thank you all for your reply and good advice. ? For the past few days, I've tried to catch more details about such error. However, it didn't happen again.? ? In the mean time, I've added some secure-controls and self-check-fix mechanism into the program. So now I have confidence to let it run automatically. ? ? Hi Richard, ? Thank you for your suggestions. Now I've turned on API logging in TWS. So next time it can be traced back more clearly. ? Btw, my post is no complaining of any kind, you have misunderstanding to this point.? ? I really don't care "who" is wrong or complain something, my only concern is to avoid such errors in the real time. ? ? Hi ?fantasy footbal, ? Thank you for your suggestions. I've never tried Gateway. What is the advantages of Geteway over TWS? ? You already have "thousands of trades" over your tool? Congratulations! I've just begun the journey :) |
|
Nick
I am curious as to what information order status adds to your processing. To me, once I submit an order it is active until I get an execution or cancel. Order status seems redundant. Is there some information in order status that simplifies or expands the abilities of your order processing?
toggle quoted message
Show quoted text
On 8/24/2017 2:37 AM, Jason via Groups.Io wrote:
I planned to fix this by creating a list of orderId's that I have submitted, checking that I see those in the openOrders returned, and if I don't see them, then I know the list is wrong (incomplete), and I should call reqAllOpenOrders again, and wait for the result, before proceeding with any trades on that symbol. |
|
Hi Jason,
?
Thanks a lot for sharing the experience.?
?
Otherwise I felt I was the only one on the world who had run across the situation. : )
?
Your ideas and information are really brilliant and very smart!?
?
?
Hi Nick,
?
It has nothing to do with order status or the api itself. I just added some secure-checks into my own program for risk-control purpose, probably for most of the people, these secure-checks are redundant. I agree with you, "once I submit an order it is active until I get an execution or cancel. Order status seems redundant."
?
?
Hi Richard,
?
You have misunderstood me from the very beginning.?
?
My post has noting to do with the word "complain" or "unsatisfactory".?
?
Everyone on the earth will run across some trouble or something unexpected from time to time somewhere somehow. It's normal and just part of life.
?
I just told the trouble I've just met, trying to get help and sharing the experience in the post. That is all. It has totally nothing to do with "complain" or "unsatisfactory".
?
Actually I am very grateful to IB. IB has provided such a wonderful and powerful platform and API, which make a lot of things possible and easier. |
Very interesting Richard, yes I think that's a plausible and probably correct explanation, So this implies that there isn't a singleton entity behind EClientSocket. And EClientSocket cannot be assumed to be a singleton on some client systems. I hadn't considered that before. Because otherwise all they needed to do was fix EClientSocket, or the thing behind it, to stash the orders and spoof the missing orders before calling openOrderEnd(). Then my client wouldn't need to do anything special, and if the server side implementation changes then I wouldn't be aware of it. Very interesting, I hope I've understood correctly. Thanks again for the explanation, -Jason Sent with Secure Email.
|
|
Hi Nick, maybe it's not necessary but I initially thought it would be necessary to check that the orders I submit have been received by IB. So I was looking for the orderId's. But it sounds like all of this is unnecessary since it seems that IB isn't in the habit of losing or breaking orders that I submit. I'll probably remove the redundant code. If I have time! (my wife is 8.5 months pregnant, so I'm running out of time for changes at this stage!) -Jason Sent with Secure Email.
|