Re: Multiple strategies in one account
In my?opinion Gordon's suggestion to build your own Position management is the best approach. We've been using that for almost a couple of decades now and it proves itself. I would recommend relying on order status (filled amount etc.) but I guess execDetails is appropriate as well. We chose to use order status call back as it is the most broker-agnostic approach - so if you one day decide to move to another broker, you can reuse your code.
The Position management is a very simple approach - just an in-memory dictionary of positions that you update on order?status change.Just make sure the appropriate semaphores are in place. You may also periodically serialize it to a XML or something file which you can load whenever?you restart your software.
A "Position" amount could be the aggregated result of multiples orders from multiples strategies. (that is probably your issue)
Seems like you look for a specific "Position" for each strategies on same symbol
This is not a concept that I saw IBKR supporting nor that I see easy to manage from TWS side (TWS handle aggregation of Position)
?
What I do is listen to execDetails to trace how each order did executed (it may execute in slice)
and there you can use orderId to consolidate a "Position" increment/decrement for each strategies. (you can use orderRef too, I use orderId just a safer choice as an index, it avoid mistakes filling custom orderRef that I consider as a "for Human use")
?
execDetails? is pretty powerful, alas transient you have to listen to the call back and store them, and its nearly the only safe method I know to understand what is happening /happened
it work in tandem with completedOrder call back that also report the orderId to better understand your orders completion.
Recommended practice is to crosscheck your execDetails analysis from time to time by fetching your position.
?
If this is just for forensic analysis, take a look at Flex reports, it may allow you to rebuild what happened.
but IMHO I recommend you familiarize yourself with execDetails , you most probably will need it soon.
?
I didn't saw IBRK granulating the API to handle every cases there are too many, but gives all the tools needed to do it yourself.
?
Note:
To better understand why I doubt IBKR will ever handle "sub position" it's interesting to mention the taxes optimization system IBKR offer, that said to avoid looking pedantic.
The underlying mechanism of BUY is more complex than it looks because you become owner of shares (for stocks LONG) that have unique id and this is done at specific purchase dates.
when you SELL, for sake of the example I assume you are LONG, IBKR elegantly does sell starting using the older shares you purchased (you can influence that but default mechanism is tuned for taxes optimization)
This is a hidden level of complexity, that may be to manage as part of your multiple strategies case because as the case maybe, your profit after taxes may depend upon your holding period. (in North America)
So to make your own taxes report matching IBKR taxes report you need to understand that too.
Generally IBKR mechanism improve your work, bad surprise happen mainly when using pyramidal strategies over various period.
?
?
?
|
Re: Multiple strategies in one account
A "Position" amount could be the aggregated result of multiples orders from multiples strategies. (that is probably your issue)
Seems like you look for a specific "Position" for each strategies on same symbol
This is not a concept that I saw IBKR supporting nor that I see easy to manage from TWS side (TWS handle aggregation of Position)
?
What I do is listen to execDetails to trace how each order did executed (it may execute in slice)
and there you can use orderId to consolidate a "Position" increment/decrement for each strategies. (you can use orderRef too, I use orderId just a safer choice as an index, it avoid mistakes filling custom orderRef that I consider as a "for Human use")
?
execDetails? is pretty powerful, alas transient you have to listen to the call back and store them, and its nearly the only safe method I know to understand what is happening /happened
it work in tandem with completedOrder call back that also report the orderId to better understand your orders completion.
Recommended practice is to crosscheck your execDetails analysis from time to time by fetching your position.
?
If this is just for forensic analysis, take a look at Flex reports, it may allow you to rebuild what happened.
but IMHO I recommend you familiarize yourself with execDetails , you most probably will need it soon.
?
I didn't saw IBRK granulating the API to handle every cases there are too many, but gives all the tools needed to do it yourself.
?
Note:
To better understand why I doubt IBKR will ever handle "sub position" it's interesting to mention the taxes optimization system IBKR offer, that said to avoid looking pedantic.
The underlying mechanism of BUY is more complex than it looks because you become owner of shares (for stocks LONG) that have unique id and this is done at specific purchase dates.
when you SELL, for sake of the example I assume you are LONG, IBKR elegantly does sell starting using the older shares you purchased (you can influence that but default mechanism is tuned for taxes optimization)
This is a hidden level of complexity, that may be to manage as part of your multiple strategies case because as the case maybe, your profit after taxes may depend upon your holding period. (in North America)
So to make your own taxes report matching IBKR taxes report you need to understand that too.
Generally IBKR mechanism improve your work, bad surprise happen mainly when using pyramidal strategies over various period.
?
?
?
|
Multiple strategies in one account
Hello,
I need to run several strategies in one account. How do I best keep them apart, unknowing of each other?
I can tag orders using the orderRef field, but as of now, all I can think of is to keep a local database with all information (and maybe reconcile it with IB now and then).
How else can I know which position belongs to which strategy?
Do you have any best practices, or tips, to share?
Thanks,
Peter
|
Re: No Security definition has been found for the request when running cpp sample code
Some suggestions: ? - Stop posting these screenshots: they’re almost impossible to read. Try to make life easy for recipients if you want help. Instead, copy the text from the terminal and paste it into your post, or into an attachment. If you really need to show a screenshot, take a high-resolution image and attach the image file.
- Make sure your log entries have timestamps. It’s impossible to tell whether all these things happened immediately consecutively, or spread out over time.
- The error 1100 means what it says: TWS/Gateway has lost its connection to the IBKR servers. Therefore it can’t process API requests until that connection is restored. It looks like your request was submitted while the connection was broken, so of course you get an error response.
- Note that the IBKR test clients are bare minimum demo code, and not production quality, so they don’t handle situations like this. When writing ‘real’ code, you have to pay attention to error conditions and act according – for example, don’t make API calls when there is no connection to TWS, or no connection from TWS to IBKR.
? Richard ?
toggle quoted message
Show quoted text
From: [email protected] <[email protected]> On Behalf Of xz2872 via groups.io
Sent: 06 December 2024 02:51
To: [email protected]
Subject: Re: [TWS API] No Security definition has been found for the request when running cpp sample code? runnig the test client multiple times gives different results..
|
Re: No Security definition has been found for the request when running cpp sample code
It might be my eyes, but the pictures you post are too small to read. And when zooming in they become blurry, thus unreadable.
|
Discrepancies between real-time and historical market data
Attached are the logs from my subscription to real-time and historical bar data yesterday. As you can see, there are discrepancies between the 'high' and 'Wap' values in the two datasets. I'm wondering if anyone can explain why this might be occurring. Any insights would be helpful.
?
RealTimeBars: JD,Time:1733404940,Open:36.28,High:36.28,Low:36.28,Close:36.28,Volume:500000000000,Wap:36280000000,Count:1
HistoricalBars: JD,BarData(date=datetime.datetime(2024, 12, 5, 8, 22, 20, tzinfo=backports.zoneinfo.ZoneInfo(key='US/Eastern')), open=36.28, high=36.29, low=36.28, close=36.28, volume=500.0, average=36.282, barCount=1)
?
Thanks?
xb
|
Re: reqRealTimeBars: data stops receiving for after-hour session
Please DO NOT delete messages. This topic is now useless for anyone searching the archives since your original problem statement and part of the discussion is gone.
闯ü谤驳别苍
?
On Fri, Dec 6, 2024 at 01:17 AM, xb wrote:
Thank you all guys, this issue has been resolved.
|
Re: reqRealTimeBars: data stops receiving for after-hour session
Thank you all guys, this issue has been resolved.
|
Re: No Security definition has been found for the request when running cpp sample code
runnig the test client multiple times gives different results..
|
Re: No Security definition has been found for the request when running cpp sample code
Thanks for the reply richard.
? When I send requests to the ib server, sometimes the data returns just fine, sometimes I get error 'Connectivity between IBKR and Trader Workstation has been lost' followed by? 'No security defintion has been found for the request'. I'm wondering what's causing this? and how should I handle this?
|
Tradiator writes: ? ‘What has happened since then that interfacing to IB has become as difficult as climbing the Everest mountain!’ ? I think that’s a little unfair. If you use the ActiveX API, it’s still just as easy, except that you don’t host it on a form – thankfully, because that was never sensible. You just have to instantiate it like any other object with the New operator. Wiring up events in Visual Basic.Net is just as easy as it was in VB6, and more flexible. You can see all this in the ActiveX sample I linked to earlier. ? The ActiveX API also has the benefit that events are fired on the main program thread, so that there is no need to be concerned with threading issues, and the UI can be accessed and modified directly. Though of course you can easily offload processing onto another thread where needed, and the async facilities in .Net make asynchronous working a delight. ? It’s certainly true that it’s more complex when using the CSharp API. I think there were some bad design decisions back in 2015, and I was fairly vocal about them on the GitHub repository, but my arguments fell on deaf ears. But it’s all water under the bridge, and it’s really not that hard to get it going. Certainly not in the Everest league (I went trekking there in 1984, up to 18000 feet, just mindblowing…). ? Richard
|
@Richard L King
?
> So if no-one objects, I’ll post that response here in due course (in case it might be useful for someone else in future), and I can then follow up privately with @Tradiator if he wants to try my suggestion.
? ? ? ? Richard, I would be very happy to try your solution.
? ? ? ? Thanking you in advance.
?
|
@Gordon Eldest
?
> ...I feel the need to share the feeling that you are "swimming against the flow".
? ? About 20 years ago I played a bit with the IB interface with VB6. It was as simple as dropping an .OCX control on a form and then calling its procedures. As simple as that!
? ? I had been able to request data and place orders in a very short time.
What has happened since then that interfacing to IB has become as difficult as climbing the Everest mountain!
For now, I just expected things to be as simple, or just slightly more complicated.
?
> 2- Although this business require well designed and safe code, a loosely type language like Basic is not a way to make your future life simple.
> You also increase the odd of uncontrolled polymorphic calls that could create hard to diagnose side effect.
> If your aren't familiar with C++ from your embedded system experience, go for Java, surely a strongly type language and also the best portable solution (I wonder why do you need to design a cross platform, solution)
? ? I am just beginning with Purebasic but I have read that it could use "strongly typed variables". As I am a C programmer I prefer those type of variables of course.
?
> 3- I don't understand your terminology, what IBKR codes are you calling the "windows socket solution" ?
> hints: the IBKR Client code sample ?
? ? Portions of code that I have found here and there, not supplied by IB.
> Where did you get the "TWSlib.dll" ? I just don't remember if IBKR does supply a precompiled binary.
> note: this may show how difficult it could be to give help if you don't comply with a standardized terminology, by default the IBKR API convention
? ? 2 precompiled files were supplied with the TWS API version that I recently downloaded (one for CPP I think).
?
> Last:
> Why don't you consider a mix with another REST data oriented vendor and using IBKR Web API (a REST API) ?
? ? This is the first time I hear about REST. Unfortunately for my current project, I am an electronician and not a computer scientist. There have been so many evolutions for the past 20-30 years.
|
@TRadiator ? This is in response to your post earlier today. Your statements are quoted in blue, followed by my replies. ? “I am sorry to be a bit stubborn but I prefer to start in Purebasic. Might change later eventually.” ? Well, all the advice you’ve received so far is DON’T. And that’s good advice. I think you should take it. Having said that, I’ll try to provide some responses to your statements that might help. ? “However I am struggling using the TWSlib.dll. I can connect to this lib but I am having problems to calling a function. What I would like to know is if TWSlib is the best option or is it better to use the windows socket solution ? I am not sure but it seems that the source code suppplied with TWS API of the other languages use the socket version ?” ? What do you mean when you say you can connect to TWSLib.dll? Do you mean you can reference the library, instantiate a TWS object and successfully call the eConnect() method? If so, that’s actually quite encouraging. If PureBasic can properly deal with ActiveX technology, then you should be able to use it quite successfully. ? What do you mean by ‘the windows socket solution’? I presume you mean the CSharpAPI.dll library? ?Every API implementation, no matter what language, use a socket connection to communicate with TWS/Gateway: there is no other way to communicate. Every API sends the same message in response to an API call. The TWSLib.dll (also known as the ActiveX API) is basically just a wrapper around CSharpAPI.dll. This wrapper uses Microsoft’s ActiveX technology to enable it to be used in environments that ‘speak’ ActiveX, such as Excel and other Office apps, Delphi, even Visual Basic 6. This version of the API is in some respects easier to program with than the CSharpAPI.dll, but it also has quite a few ‘gotchas’ that make it rather confusing. ? So assuming that PureBasic is a fully fledged ActiveX host, you should be able to use the full power of the API fairly straightforwardly, once you’ve got over the initial confusion. ? The other day I sent you a link to two sample programs written in Visual Basic .Net and C#, both using the CSharpAPI. As it happens, I also have a version that uses the ActiveX API. It does exactly the same things as the other two, but uses TWSLib.dll rather than CSharpAPI.dll. I’ve now uploaded that sample to the same location (see the SimpleTWSActiveXDemo folder, or SimpleTWSActiveXDemo.zip which is easier to download): ? (). ? Hopefully you’ll be able to use that code as a basis for something in PureBasic. I imagine the main differences will be in how you reference the project, the syntax for handling events, and of course the UI creation which would no doubt be quite an effort to port. I’d suggest starting with a much reduced version that has a button to connect/disconnect and handles the connectAck and errMsg events. Once that’s working you can build up with the reqMktData call and the tickPrice and tickSize events, etc. ? I should add that this version is also not using the latest API, but it works perfectly well. It doesn’t include the change from Double type to Decimal type for sizes, but that won’t matter for the moment (unless you want to trade crypto-currency contracts). I’ll try to get round to bringing these samples up-to-date? at some point. ? If you get stuck, email me privately (you can do this using the Reply to Sender facility). I’ll do what I can to help, but only if it looks like there’s a chance it can be made to work. ? ? “And I also have the impression that the socket version is more generic and makes the program independent of changes made by IB to TWSlib. What do you think ?” ? No, not really. The ActiveX API provides pretty much exactly the same capabilities as the CSharpAPI.dll (I think there are a couple of things that IBKR haven’t been able to provide for technical reasons, but I can’t remember what they are without going and looking – certainly not things that you’re likely to need any time soon). Changes to TWSLib always affect the client program one way or another. ? “Please help me, I really would like to start making TWSAPI calls and start working on the interesting part of the program.” ? ??????????????? I’ll try!... ? ? Richard ?
|
I have no objections at all, Richard, and documenting such a path may even be helpful for people who want to integrate other languages in the future.
I just want to avoid that the entire group will be exposed to the long and painful TWS API re-implementation process from scratch in PureBasic.
Looking forward to seeing what you came up with.
闯ü谤驳别苍
?
On Thu, Dec 5, 2024 at 12:54 PM, Richard L King wrote:
Can I just say that @Tradiator’s earlier post today has actually opened up a possibility to move forward with PureBasic. I’m in the middle of drafting a response to it that will indicate a way to proceed. It’ll be a while before I’m ready to post it though.
?
So if no-one objects, I’ll post that response here in due course (in case it might be useful for someone else in future), and I can then follow up privately with @Tradiator if he wants to try my suggestion.
?
闯ü谤驳别苍. If you’d rather I didn’t, let me know and I’ll just send it direct to him.
?
Richard
?
|
Can I just say that @Tradiator’s earlier post today has actually opened up a possibility to move forward with PureBasic. I’m in the middle of drafting a response to it that will indicate a way to proceed. It’ll be a while before I’m ready to post it though. ? So if no-one objects, I’ll post that response here in due course (in case it might be useful for someone else in future), and I can then follow up privately with @Tradiator if he wants to try my suggestion. ? 闯ü谤驳别苍. If you’d rather I didn’t, let me know and I’ll just send it direct to him. ? Richard ? ?
toggle quoted message
Show quoted text
From: [email protected] <[email protected]> On Behalf Of Tradiator
Sent: 05 December 2024 18:38
To: [email protected]
Subject: Re: [TWS API] Purebasic and TWS? Thank you all very much for your responses. I will carefully study all the information received and try not to bother too much in the future.
|
Thank you all very much for your responses.
?
I will carefully study all the information received and try not to bother too much in the future.
|
You received a lot of valuable feedback, the task you are trying to take on is massive, it will be a nightmare to maintain as a single person over long periods of time, and will possibly be a massive waste of time. I had not heard about PureBasic until your post, so that I am sure there is nobody to help you. And the code ChatGPT generated will not get you anywhere either, But go ahead, if that is what you want to do.
HOWEVER this is not the group to look for help for a task that you have been advised against. We are exclusively focused on the API implementations that IBKR provides. You may not expose our 6,000++ members to the long struggle you are going to have, That will create too much noise and I will have to lock the topic.
So here my advise:
- Don't do it. Stay with one of the many IBKR API implementations (C++, C#, VB, Java, Python) or and independent implementation with a large enough existing community (ib_async., Ruby, ...)
- At the heart, all IBKR TWS API implementations simply serialize the requests your client makes into messages that are sent to TWS/IBGW via the socket and, in return, a deserialize messages from TWS/IBGW into callbacks. Follow Mark Collins's advice and read and understand the EClient (requests) and EDecoder (responses) classes for a language you are most familiar with. You would have to create the equivalent to those two classes in PureBasic if you want to connect at the socket level.
- The wire-line message protocol is not documented, subject to change without notice, and does change with every major TWS/IBGW release. But several of our members and the ib_insync/ib_async community do maintain their own implementations. It can be done.
- If that is the route you want to go, implement a program that simply connects to TWS/IBGW via the socket and handles the few very basic messages required for the "HELLO" phase. But there are will be many additiona? messages with complex objects zou would have to implement eventually even if you'd only use a subset of TWS API. But I have attached (once more) the basic message flow you would have to implement:
- Create a socket with TWS/IBGW
- Send the FIRST_CLIENT_MESSAGE
- Wait for and receive the FIRST_SERVER_MESSAGE
- Send the API_START_API_MESSAGE
- Wait for and receive the MANAGED_ACCOUNTS_MESSAGE and NEXT_VALID_ID_MESSAGE
- Plus be prepared for ERROR_MESSAGEs since they can occur at any time after API_START
- IBKR takes care of all the little details by providing you with TWS API implementations for various languages. Making and maintaining a wrapper around TWSlib.dll for PureBasic might be a much more manageable task as long as TWSlib.dll and their threading code plays nicely with PureBasic. You might have to make minor adjustments to method signatures over time, but you can stay on the same TWSlib.dll version for months if not years unless you need features of newer versions.
But using an existing TWS API implementation will be by far your most productive path. And the only path for our group to provide you with any ongoing help.
One last thought. Assuming PureBasic provides a rich set of WEB/REST API tools, (formerly called Client Portal API) might be the simplest path for you to interact with your IBKR account from PureBasic.
闯ü谤驳别苍
?
?
?
?
On Thu, Dec 5, 2024 at 11:12 AM, Tradiator wrote:
I am sorry to be a bit stubborn but I prefer to start in Purebasic. Might change later eventually.
?
However I am struggling using the TWSlib.dll. I can connect to this lib but I am having problems to calling a function.
What I would like to know is if TWSlib is the best option or is it better to use the windows socket solution ?
I am not sure but it seems that the source code suppplied with TWS API of the other languages use the socket version ?
?
And I also have the impression that the socket version is more generic and makes the program independent of changes made by IB to TWSlib.
What do you think ?
?
Please help me, I really would like to start making TWSAPI calls and start working on the interesting part of the program.
?
?
|
Sorry if blunt, no intent
but IMHO and using my limited experience of IBKR I feel the need to share the feeling that you are "swimming against the flow".
?
1- Development should start easy and gradually increase in complexity,
Here it look to me you have a first mandatory daunting task to overcome, to create a non existing ABI lib for another language, able to do the telecom with the IBKR ABI,
And extend this later taking control and experiment the API itself (not a minor task and requiring a lot of trials)
All of this without any examples or help or support.
Then and only then will you be able to focus yourself on your algo.
?
2- Although this business require well designed and safe code, a loosely type language like Basic is not a way to make your future life simple.
You also increase the odd of uncontrolled polymorphic calls that could create hard to diagnose side effect.
(I already get bad feeling with your code that morph "clientid" to a string)
If your aren't familiar with C++ from your embedded system experience, go for Java, surely a strongly type language and also the best portable solution (I wonder why do you need to design a cross platform, solution)
?
3- I don't understand your terminology, what IBKR codes are you calling the "windows socket solution" ?
hints: the IBKR Client code sample ?
Where did you get the "TWSlib.dll" ? I just don't remember if IBKR does supply a precompiled binary.
note: this may show how difficult it could be to give help if you don't comply with a standardized terminology, by default the IBKR API convention
?
Last:
Why don't you consider a mix with another REST data oriented vendor and using IBKR Web API (a REST API) ?
It is surely more easy to implement and debug.
?
?
?
|
I'm with the rest. Please consider using python. I can use julia but python is more fun.
so in purebasic you'd
; PureBasic example for connecting to IB API and requesting account details
#PORT = 7497 ; TWS or Gateway default port
#HOST = "127.0.0.1"
; Establish a socket connection
If OpenNetworkConnection(#HOST, #PORT)
Debug "Connected to IB TWS/Gateway"
; Send handshake message (TWS Protocol version)
SendNetworkString(0, "API\0")
Delay(100)
; Request account details (e.g., account summary request)
requestID = 1
version = 1
message = "9" + Chr(0) + Str(version) + Chr(0) + Str(requestID) + Chr(0) + "All" + Chr(0)
SendNetworkString(0, message)
Delay(200)
; Receive and parse response
Repeat
buffer = ReceiveNetworkData(0, *memory, 4096)
If buffer > 0
result$ = PeekS(*memory, buffer)
Debug result$
EndIf
Until buffer <= 0
CloseNetworkConnection(0)
Else
Debug "Failed to connect to IB TWS/Gateway"
EndIf
and in python you'd
from ibapi.client import EClient
from ibapi.wrapper import EWrapper
from ibapi.contract import Contract
class IBApp(EWrapper, EClient):
def __init__(self):
EClient.__init__(self, self)
def error(self, reqId, errorCode, errorString):
print(f"Error: {reqId}, {errorCode}, {errorString}")
def accountSummary(self, reqId, account, tag, value, currency):
print(f"Account Summary. ID: {reqId}, Account: {account}, {tag}: {value} {currency}")
def accountSummaryEnd(self, reqId):
print(f"Account Summary End for request ID: {reqId}")
self.disconnect()
def main():
app = IBApp()
app.connect("127.0.0.1", 7497, clientId=1)
# Wait for the connection to establish
app.run()
# Request account summary
app.reqAccountSummary(9001, "All", "NetLiquidation,AvailableFunds")
if __name__ == "__main__":
main()
|
