Keyboard Shortcuts
Likes
- Twsapi
- Messages
Search
|
Re: Building an options trading algo using TWS. Need some help!
Ben Alex
Hi It might help if you expand on what you're after. Do you want some guidance on coding Java generally, some help using IB's Java library, or are you seeking someone to write some or all of code for you? Is your long-term goal to maintain the code yourself or did you want someone to continue with that?Java can seem daunting to learn, but it is well worth it if you have a few months available. While there are hundreds of books and tutorials out there, I know lots of professional Java developers who worked their way through (and Oracle to their credit keep it up-to-date with new Java improvements). If you are after trading system design advice or help with the IB Java library, this list will give you a lot of free advice. also includes lots of design, programming and API information if you haven't already seen it. Basically, if you're willing to invest the time and use the many high quality (free) resources out there, you can almost certainly write it yourself. One thing I'm curious on is does your strategy require human involvement or is it entirely algorithmic? What is the typical holding time and what data do you process (eg daily, minute, tick, bid-ask level)? Letting people know that makes a major difference to the sort of program you're trying to build (ie a GUI-driven decision support tool vs a headless black box that performs trades) and is more likely to appeal to someone who has already built that class of application and can quickly adapt their existing experience and/or code. Cheers Ben |
|
Building an options trading algo using TWS. Need some help!
I have been manually trading an options spread strategy and am looking to automate it. I am new to programming and am having difficulties getting started. I am trying to use java language in NetBeans. This is a profitable strategy and risk is easily controlled. Any help will be rewarding! |
|
Re: orderStatus for orders modified in TWS
Nick
¿ªÔÆÌåÓýOk, thanks. I never investigated this but tossed it in the hat since the OP is still looking for info.On 12/8/2015 1:05 PM, 'Richard L King'
rlking@... [TWSAPI] wrote:
? |
|
Re: orderStatus for orders modified in TWS
Nick
¿ªÔÆÌåÓýShow me the manual reference for your statement. What I see in the manual is
So where do you get that id zero can not be used in an api program? On 12/8/2015 12:48 PM,
fantasyfootball672@... [TWSAPI] wrote:
? |
|
Re: orderStatus for orders modified in TWS
The order was initially placed as a LMT order via the API, but was subsequently amended to a MKT order via TWS. Modifications in TWS to orders placed via the API are always notified to the API via the same clientID as placed the order.
From: TWSAPI@... [mailto:TWSAPI@...] Sent: 08 December 2015 17:09 To: TWSAPI@... Subject: Re: [TWS API] Re: orderStatus for orders modified in TWS You might look into using client id of 0. I don't know if this applies to your situation but normally a client id of 0 is required to be notified of orders entered in TWS. On 12/7/2015 6:17 PM, kimbleyoung@... <mailto:kimbleyoung@...> [TWSAPI] wrote: I actually cancels myself when amending an order via the API and haven't run into an issue....yet but do understand the race condition and actually leave a small pause in-between (not doing HFT here by any means). What I am actually trying to do though is simulate a delayed fill using the demo account by changing the order from a LIMIT to a MARKET so it gets filled immediately. When I do this in TWS it is filled but there is no orderstatus update triggered in the API. I wanted to know if this is a known limitation or a bug I'm hitting across. |
|
Re: Python TWS API
Richard,
I ran into another problem: the connection doesn't get dropped. Server Response: error, <error id=-1, errorCode=326, errorMsg=Unable connect as the client id is already in use. Retry with a unique client id.> I check on Help -> Data Connections on TWS and found the connection being kept alive despite I always called?conn.disconnect() after placing orders. ?Any idea on resolving this? Best, ? |
|
Re: orderStatus for orders modified in TWS
Nick
¿ªÔÆÌåÓýYou might look into using client id of 0. I don't know if this applies to your situation but normally a client id of 0 is required to be notified of orders entered in TWS.On 12/7/2015 6:17 PM,
kimbleyoung@... [TWSAPI] wrote:
? |
|
Re: orderStatus for orders modified in TWS
¿ªÔÆÌåÓýWell, I¡¯ve just tried this myself with the paper-trading system and I don¡¯t get the same result ¨C and I wouldn¡¯t expect to. I¡¯ve never seen a missed order status in 12+years of using the API. ? However I note that you are using the demo system, and it¡¯s possible that there¡¯s a bug there. Also I note from your first post that you¡¯re using pyalgotrade: I know nothing about this, but it¡¯s perfectly possible that what you¡¯ve found is a limitation or bug in it. ? What you should do to help diagnose where the problem lies is to turn on the API logging in TWS, then you¡¯ll be able to see exactly what TWS has sent. ? Here¡¯s an extract from my logfile that shows exactly what happened (some unnecessary details have been removed to aid readability): ? 2015-12-08 10:33:43.673 OUT: (PLACE_ORDER);¡;Order id=268435460;...Local Symbol=ZZ5;...Action=BUY;Quantity=1;Order type=LMT; 2015-12-08 10:33:44.521 IN: (OPEN_ORDER); ¡;Id=268435460;...Status=Submitted;... 2015-12-08 10:33:44.522 IN: (ORDER_STATUS); ¡;id=268435460;Status=Submitted;Filled=0;Remaining=1;... ? [At this point I changed the order from LMT to MKT in TWS] ? 2015-12-08 10:34:58.395 IN: (EXECUTION_DATA); ¡;ReqId=-1;OrderId=268435460;...Shares=1;Price=6192.5;... 2015-12-08 10:34:58.423 IN: (OPEN_ORDER); ¡;Id=268435460;...Status=PendingSubmit;... 2015-12-08 10:34:58.423 IN: ORDER_STATUS); ¡;id=268435460;Status=PendingSubmit;Filled=1;Remaining=0;... 2015-12-08 10:34:58.424 IN: (OPEN_ORDER); ¡;Id=268435460;...Status=Filled;... 2015-12-08 10:34:58.425 IN: ORDER_STATUS); ¡;id=268435460;Status=Filled;Filled=1;Remaining=0; 2015-12-08 10:34:58.426 IN: (OPEN_ORDER); ¡;Id=268435460;...Status=Filled;... 2015-12-08 10:34:58.426 IN: ORDER_STATUS); ¡;id=268435460;Status=Filled;Filled=1;Remaining=0;... ? There are several interesting points about this, and I hope you don¡¯t mind if I spell them out. ? ¡¤???????? It¡¯s worth having the ability to log what you¡¯ve received from the API immediately, before you do any processing. In my case I have my own API implementation which means I can log everything I send to or receive from the socket exactly as it came down the wire; the level of logging is configurable, so I don¡¯t normally log high intensity stuff like ticks, but I¡¯ve found from experience that it¡¯s worthwhile always logging everything to do with orders - ie, PLACE_ORDER, OPEN_ORDER, ORDER_STATUS, EXECUTION. If you¡¯re using IB¡¯s API code, you don¡¯t have quite this luxury, but you can still log details of every API call immediately before you make the call, and every API callback immediately after you receive it. ¡¤???????? You get multiple OPEN_ORDER and ORDER_STATUS callbacks relating to an order. ¡¤???????? The ORDER_STATUS callback is NOT a ¡®status change¡¯ indicator ¨C you can get more than one of them with the same order status. ¡¤???????? The EXECUTION_DATA callback is the definitive statement of a fill. In my opinion, it¡¯s better to use it rather than ORDER_STATUS, though the latter contains some potentially useful additional information. ¡¤???????? In the example above there are two callbacks after the EXECUTION_DATA callback which show the status as PendingSubmit. This illustrates an important principle regarding messages relating to orders: don¡¯t make any assumptions about the sequence in which such messages will arrive. ? Finally, the demo system really is the pits as far as developing a realistic trading system is concerned. If you¡¯re serious about this, you owe it to yourself to open an account so you can use the paper trading system, which gives you real data and a much closer approximation to what¡¯s likely to happen with a live account. Don¡¯t for a moment believe that you can go direct from a system developed against the demo system to live trading. ? Richard ? ? From: TWSAPI@... [mailto:TWSAPI@...]
Sent: 07 December 2015 23:17 To: TWSAPI@... Subject: [TWS API] Re: orderStatus for orders modified in TWS ? ? I actually cancels myself when amending an order via the API and haven't run into an issue....yet but do understand the race condition and actually leave a small pause in-between (not doing HFT here by any means). ? What I am actually trying to do though is simulate a delayed fill using the demo account by changing the order from a LIMIT to a MARKET so it gets filled immediately. When I do this in TWS it is filled but there is no orderstatus update triggered in the API. ? I wanted to know ?if this is a known limitation or a bug I'm hitting across. Posted by: kimbleyoung@... |
|
Re: Download page for IB Gateway...
couldvot
As explained to me by support we now have an "improved" way of downloading the Gateway via the IB logon screen, and selecting IB Gateway.
As explained by Ben's earlier post on Linux this downloads a whole lot of "stuff" including a complete java install. On my Ubuntu machine the new Gateway (ver 952) installs: $HOME/Jts? (the 2 gateway jars are in $HOME/Jts/ibg/952/jars) $HOME/.i4j_res (a whole java runtime) $HOME/.java .. but it does seem possible to run the new Gateway the "old" way if like me you prefer to keep things simpler and reduce memory footprint. This bash script worked ok and brought up the Gateway (I will need to test it more thoroughly before I use it in prod). #!/bin/bash PROCESS_NAME="ibgateway.GWClient" echo "starting IBG Gateway" nohup java -cp jars/jts.jar:jars/total.jar -Dsun.java2d.noddraw=true -Xmx256M ibgateway.GWClient . > ./ibg-gw.log & 2>&1 echo "Started $(date) PID:$(pgrep -f $PROCESS_NAME)" Andy |
|
Re: Python TWS API
¿ªÔÆÌåÓýThat¡¯s not quite true. When you change the API port in TWS¡¯s global configuration dialog, it pops up a message box asking if you want to apply the new setting immediately. (But maybe you¡¯re talking about the Gateway, and I haven¡¯t checked that.) ? From: TWSAPI@... [mailto:TWSAPI@...]
Sent: 08 December 2015 09:04 To: TWSAPI@... Subject: [TWS API] Re: Python TWS API ? ? I was stuck on the same problem for a few hours, and finally figured out that I was using the wrong port address. ?Also, it takes restarting of TWS to change ports. Posted by: kein520@... |
|
Re: orderStatus for orders modified in TWS
I actually cancels myself when amending an order via the API and haven't run into an issue....yet but do understand the race condition and actually leave a small pause in-between (not doing HFT here by any means).
What I am actually trying to do though is simulate a delayed fill using the demo account by changing the order from a LIMIT to a MARKET so it gets filled immediately. When I do this in TWS it is filled but there is no orderstatus update triggered in the API. I wanted to know ?if this is a known limitation or a bug I'm hitting across. |
|
Re: Tips for fastest performance with IBG?
Ben Alex
On Mon, Dec 7, 2015 at 11:31 AM, fantasyfootball672@... [TWSAPI] <TWSAPI@...> wrote: the traditional "command line" memory parameters are becoming deprecated, meaning garbage collection is essentially out of my hands I should clarify there are still hundreds of JVM garbage collector parameters you can tune. The permanent generation removal has been offset by the . My key point was JVM tuning options aren't likely to help very much in the case of IB Gateway due to its probable network IO bound workload, small heap and reasonable JIT hotspot defaulting. The non-JVM options are way more likely to give you value. But each have trade-offs (usually greater economic costs) for the improved latency outcomes. Even order replacement has costs with IB, so you should always check the fees involved. For example many folks aren't aware API placed orders (at least for US equities on the fixed pricing structure) cost 50% more than orders placed via TWS. So always . :-) |
|
Re: Tips for fastest performance with IBG?
Nick
I have seen this suggested by several different people but I have always been skeptical.
toggle quoted message
Show quoted text
If I submit an order at 9am that passes margin checks and then proceed to loose 90% of my account by noon, I can't bring myself to believe that IB will allow the now non-compliant order to fill when I change it. In other words I believe there has to be some validation at execution time, not just submit time. It may be that modifying an order has less overhead than submitting a new order but I can't believe that IB would expose itself to changes in a client's risk profile that happened after the order was submitted. On 12/6/2015 7:07 PM, Ben Alex ben.alex@... [TWSAPI] wrote:
* Place orders away from the market price so that risk management checks have already been completed, then modify the order when ready to invoke. |
|
Re: Tips for fastest performance with IBG?
Ben, this is great. I don't understand Java so I appreciate the "answer" that there isn't much that can be done since the source is closed and the traditional "command line" memory parameters are becoming deprecated, meaning garbage collection is essentially out of my hands. I also appreciate the "modify order" tip- that is very, very cool. My own trading tool is compiled C++ and is very fast, and the IB market data "feed" such as it is is almost certainly the biggest source of latency, especially with your tip for the credit workaround.
|
|
Re: Tips for fastest performance with IBG?
Ben Alex
What type of speed improvements are you looking for? When most people talk about speed and trading they are trying to achieve a specific latency outcome, ie completion of a given operation within a bounded amount of time. That's different from say the 99th percentile completion time, where outliers are of minimal consequence (eg batch applications, web applications etc). IB Gateway, TWS and IB Controller are JVM hosted applications that cannot be embedded within your own application (ie you can't run them in the same process). In addition, IB Gateway and TWS are closed source applications where you cannot look at (let alone improve) their threading, network use, object allocations/deallocations, retry/timeout mechanisms and so on. You're limited to using the provided wire level APIs (ie IB API or FIX). Assuming you need to use IB Gateway, the most you can do is (a) adjust JVM properties and (b) consider non-JVM issues. First, the good news. In general the easiest way to obtain the best performance for almost any Java application is to upgrade to the latest released JVM. As an aside the new Install4J-based distribution of TWS 954 bundles Java 1.8.0_65, which is pretty close to the latest (1.8.0_66 is ). Actual JVM configuration is also quite easy, although unlikely to help. That's because most JVM configuration latency wins come from adjusting memory management and JIT compilation settings. IB Gateway does not use a large amount of heap relative to normal Java applications. If you needed an 8+ GB of heap to run IB Gateway, tuning the garbage collector may prove useful. However for a few hundred megabytes (like IB Gateway uses) it isn't going to make much difference (and in many cases it's likely to actually be worse than using the defaults that Oracle have selected based on CPU class, memory size and operating system). The hotspot options can also control exactly when the JVM inlines certain methods. Like a (PGO) build for languages like C++, there's a requirement to wait for enough workload to complete so the routine code paths can be profiled (ie "warm up" the JVM). Doing it too soon (and there are JVM arguments that force it to happen immediately etc) and you will actually end up with worse performance than the default bytecode execution. But given IB Gateway is mostly a converter between local wire formats (IB API / FIX) and a proprietary wire format (to the IB backend), you are almost certainly going to be blocking on network IO as opposed to CPU or memory use. If you are wanting to play around with JVM options anyway, I assume your IBController install is based off the batch file. In that batch file is the JAVAOPTS setting. There's room for improvement, for example -XX:MaxPermSize option is now deprecated in Java 8, as the permanent generation has been removed (see in Java 8). What's an appropriate value? Well as of TWS 954 the officially-provided Install4J launch doesn't set any memory options for Java. So if you do anything to your JAVAOPTS, I'd be removing them unless benchmarking shows they're useful. A nice introductory book if you want to dig into JVM performance tuning is Scott Oak's . It covers you to use included JVM tools (eg visualvm, jcmd, jinfo, jstack etc) to get a better idea what's happening in your JVM and start tuning away. But again, you're extremely unlikely to gain very much improvement for IB Gateway. The biggest opportunity for meaningful latency improvement is likely to come from other techniques that have been mentioned on this list at various times. They basically boil down to order tuning, connection tuning and machine tuning: * Place orders away from the market price so that risk management checks have already been completed, then modify the order when ready to invoke. * Profile and control order routing (SMART vs directed orders). Note the execution price, liquidity rebate/charge and commissions though. * Colocate nearer IB. Commercial data centre hosting will have lower average latencies than SOHO connections. * Buy a commercial-grade, dedicated network connection to IB. Check out . * Reduce load on IB Gateway by not using it for market data. Specialist data vendors offer hyper-compressed, UDP-delivered feeds (eg Nanex). * Typical OS-level issues: latest versions, patches, malware/virus/privacy scanner settings, unnecessary services, network settings, swap config etc. The above isn't a complete list by any stretch. I've personally found minimal latency issues during realtime trading. To be using IB in the first place effectively rules out most low latency strategies. If you can expand on the specific speed improvements you're looking for, others might be able to chime in with more suggestions. On Mon, Dec 7, 2015 at 8:23 AM, fantasyfootball672@... [TWSAPI] <TWSAPI@...> wrote:
|