Keyboard Shortcuts
ctrl + shift + ? :
Show all keyboard shortcuts
ctrl + g :
Navigate to a group
ctrl + shift + f :
Find
ctrl + / :
Quick actions
esc to dismiss
Likes
- Twsapi
- Messages
Search
|
TwsActivex.xls cannot "request account updates"??
virusnme
I use TwsActiveX.xls excel sheet to connect to IB Gateway. The connection between IB Gateway and TwsActiveX.xls is fairly stable. However, on occasions when IB Gateway loses connection to IB Server, TwsActiveX.xls loses connection to IB Gateway. Eventually, IB Gateway reinstates a connection to IB server. And I will have to ensure that TwsActivex.xls reconnects to IB Gateway (via VBA).
I have figured out how to re-connect the TwsActivex.xls to IB Gateway but I am not able to "request account updates" via VBA! If anyone has figured out how to do it please do let me know. |
|
Re: Connect to TWS in main not in Thread (Java)
btw12342001
--- In TWSAPI@..., twsapi <no_reply@...> wrote:
Call me brian :). I have no idea how to change that login name. thanks so much for the tip! That actually helps me a lot. I still did not completely understood your earlier reply on how to handle returned data. In which method is the returned data handled?If you scroll down in Example1.java you will see the tickPrice method. This is overriding the method of the same name in ExampleBase. All data from the TWS socket gets sent to those methods in ExampleBase unless they are overridden. If you want more than one symbol you have to assign an separate id in the request and then do something in tickPrice based on the returned tickerId. I guess if the connection is not limited to just one thread you also run into synchronization issues between the threads. But as far as I understood you can initiate 6 or 8 connections but IB recommends to use as little as possible, is that correct?Yes, only use one connection. You don't even need the thread part, you could delete the "extends thread" part in ExampleBase and then just call run, however there's a thread.sleep in the run method and you would have to fix the logic. I don't know what your purpose is so it's hard to tell you what to do. I wrote a very simple 1 file, console based program in the files section under java. Maybe that will be easier to understand, it's also a netbeans project. |
|
Re: Connect to TWS in main not in Thread (Java)
Hi,
toggle quoted message
Show quoted text
Breaking in, I have a vague memory there may be people who said they open multiple connections in order to have a dedicated thread for each, but it is highly unusual and probably not necessary. Bottom line is if you do not have specific reason to make your life complicated, don't make your threading model more complex than it needs to be for your well-understood purposes. This certainly does not require multiple connections and you can still spawn threads to handle computations that can be factored in that way, if you like. I suggest you start with a fewest-thread-possible approach. Personally I choose 1 thread and do not use a separate reader thread. Sometimes the environment may present you with a separate reader thread and you can write some code to funnel the results from that back to the main thread. There have been endless arguments here about the incredible importance or incredible unnecessity of having additional threads for this or that. It is a big topic and has been discussed for weeks on end numerous times. If you want to learn about it I suggest your read the archives, but in the meantime stick with 1 thread, or 2 if that comes naturally in your context and you can manage it. A common suggestion is to use a single thread to dispatch data *quickly* to other threads so the main thread is close enough to always available to handle the next incoming message. Anything processed quickly can be handled on that thread and anything not can be put in a queue and handled elsewhere. (Meanwhile just start coding without worrying about that until you need to.) Regarding returned data the easy solution is to always override ALL callback methods and log the results from every callback. Then you have no doubt what data is returned through which callback, in case the documentation is vague about it (or non-existent). -Kurt On 7/16/13 12:45 AM, "twsapi" <no_reply@...> wrote:
Hi btw12342001, |
|
Re: NKD data
jayfunandgames
Nice, that works. Thanks a lot for the tip.
toggle quoted message
Show quoted text
--- In TWSAPI@..., Ed <news1000@...> wrote:
|
|
Re: NKD data
Ed
Try omitting local symbol and use expiration date of 201309 instead.
toggle quoted message
Show quoted text
On 7/16/2013 10:06 PM, jayfunandgames wrote:
|
|
NKD data
jayfunandgames
Trying to request NKD data, buy having a problem.
Using the following info Symbol: NKD Type: FUT Exchange: GLOBEX Local Symbol: NKD SEP 13 This is the format that I used when requesting YM futures, so I'm not sure what I'm doing wrong. Is there a good place that will show you the format needed for the different types of requests? Thanks, Jay |
|
Re: Tick Encoding/Decoding
Ed
I don't think there is a strict correlation between tick price/size
toggle quoted message
Show quoted text
and RTV. They seem to be completely different streams to me, with different sampling/aggregation. Remember that IB does not send each tick and you get about 3 updates per second. On 7/16/2013 4:15 AM, unatnahs57 wrote:
|
|
Re: Upcoming stock earnings dates
souqmate
Thanks for the link, rwk and Robert.
toggle quoted message
Show quoted text
Below are the one-liners to scrape Yahoo calendars on linux; cut 'n paste in your xterm (tcsh). European stocks appear in earnings.csv (since 2011), confCalls.csv (since 2013), in surprises.csv (since 2011), not in splits.csv nor ipo.csv. No Japanese stocks, as ramdukof bemoans; any hint welcome! Files with history (confCalls.csv, earnings.csv, ipo.csv, splits.csv, surprises.csv, marcro.csv) are readily available for 7 days at Good luck souqMate. ----------------------- # Earnings: since 19990401 (old format prior 20001031); header: "# Company,Symbol,Time (NY, or Before/After Mkt)" foreach d (19990401 19990402 ... 20130712) wget -q -O - " | sed -n '/>Conference</,/^colspan/H; ${x;s,\n, ,g; s,<td,\n<td,g; p}' | awk 'NR>1 && !(/colspan/)' | sed 's,<[^>]*>,,g; s, , ,g; s,&,\&,g; s/,//g' | awk '{l=NR%4; if(l==1) printf "'$d',"; if(l>0 && l<3) printf $0","; if(l==3){t=""; if(/Before/) t="Before"; if(/After/) t="After"; if(/ET/){split($1,a,":"); if(/pm/) a[1]+=12; t=a[1]":"a[2]} print t}}' >>! earnings.csv end # Conf Calls: since 20111014; header: "# Date,Time (NY),Symbol(if several: separated by blanks),Event" foreach d (20111014 20111015 ... 20130712) wget -q -O - " | sed -n '/>Access</,/size=1/H; ${x;s,\n, ,g; s,<td,\n<td,g; p}' | awk 'NR>1' | sed 's,<[^>]*>,,g; s, , ,g; s,&,\&,g; s/,//g' | awk '{l=NR%4; if(l==1){split($1,a,":"); if(/pm/) a[1]+=12; t=sprintf("%02d:%02d",a[1],a[2]); printf '$d'","t","} if(l==2) printf $0","; if(l==3) print $0}' >>! confCalls.csv end # Surprises, since 20021209; header: "# Date,Company,Symbol,Surprise (%),Reported EPS,Consensus EPS" foreach d (20021209 20021210 ... 20130712) wget -q -O - " | sed 's,</tr><tr,</tr>\n<tr,g; s,&,\&,g; s/,//g' | awk '/^<tr.*td><td$/,/^nowrap/{gsub(/^align/,"<align",$0); gsub(/td$/,"td>",$0); gsub(/<[^>]*>/,"",$0); print}' | awk '{l=NR%7; if(l==1) printf "'$d',"; if(l>0 && l<5) printf $0","; if(l==5) print $0}' >>! surprises.csv end # splits, since Jan 2000; header: "# Payable,Ex Date,Company,Symbol,Optionable?,Ratio,Announced" foreach y (00 01 02 03 04 05 06 07 08 09 10 11 12 13) foreach m (1 2 3 4 5 6 7 8 9 10 11 12) wget -q -O - " | grep ^noshade | sed 's,</table.*,,g; s,<tr,\n<tr,g; s,<td,\n<td,g; s,<[^>]*>,,g; s/,//g; s,&,\&,g; s, *,,g' | grep -v ^noshade | awk 'BEGIN{m["Jan"]=1;m["Feb"]=2;m["Mar"]=3;m["Apr"]=4;m["May"]=5;m["Jun"]=6;m["Jul"]=7;m["Aug"]=8;m["Sep"]=9;m["Oct"]=10;m["Nov"]=11;m["Dec"]=12} {l=NR%9; if(l==2||l==3||l==8){y=2000+'$y'; if(l==3) M=m[$1]; if(l==8 && m[$1]>M) y--; if(NF>0) $0=sprintf("%s%02d%02d",y,m[$1],$2)} if(l>1 && l<8) printf $0","; if(l==8) print $0}' >>! splits.csv end end # IPO pricings, since Jan 1998; header: "# OfferDate,Company,Symbol,SharesOffered[mln],ProposedPric[$],InitialPrice[$]" foreach y (98 99 00 01 02 03 04 05 06 07 08 09 10 11 12 13) foreach m (jan feb mar apr may jun jul aug sep oct nov dec) wget -q -O - " | sed 's,</TR>,</TR>\n,g' | grep '^<tr><td align' | sed 's,<td,\n<td,g; s,>M<,><,g; s,<[^>]*>,,g; s/,//g; s,\$,,g' | awk 'BEGIN{m["Jan"]=1;m["Feb"]=2;m["Mar"]=3;m["Apr"]=4;m["May"]=5;m["Jun"]=6;m["Jul"]=7;m["Aug"]=8;m["Sep"]=9;m["Oct"]=10;m["Nov"]=11;m["Dec"]=12} {l=NR%8; if(l==2){split($1,a,"-"); y=(a[3]>90? 1900:2000)+a[3]; $0=sprintf("%04d%02d%02d",y,m[a[2]],a[1])} if(l>1 && l<7) printf $0","; if(l==7) print $0}' >>! ipo.csv end end # macro-economy, since 20010102; header: "# date,time(NY),indicator,For,Actual,BriefingForecast,Expect,Prior,RevisedFrom" awk 'BEGIN{for(i=2001;i<2013;i++) for(j=1;j<53;j++) printf "%s%02d\n",i,j}' > ! /tmp/i foreach i ( `cat /tmp/i` ) wget -q -O - " | sed -n '/^From/,/^href/{s,<tr,\n<tr,g; s,More in-depth.*$,,; s,<[^<]*>, ,g; s, bcf,bcf,g; p}' | awk 'BEGIN{m["Jan"]=1;m["Feb"]=2;m["Mar"]=3;m["Apr"]=4;m["May"]=5;m["Jun"]=6;m["Jul"]=7;m["Aug"]=8;m["Sep"]=9;m["Oct"]=10;m["Nov"]=11;m["Dec"]=12} !(/^From|^href/){yr=substr('$i',1,4); d=sprintf("%s%02d%02d",yr,m[$1],$2); split($3,a,":"); hr=a[1]; mi=a[2]; if($4=="PM") hr+=12; t=sprintf("%02d%02d",hr,mi); txt=$5; for(n=6;n<NF-5;n++) txt=txt" "$n; OFS=","; print d,t,txt,$(NF-5),$(NF-4),$(NF-3),$(NF-2),$(NF-1),$NF}' >> macro.csv end # caution: prior 20120323, times are wrong: apply UTC2EST twice and subtract 3 hrs to get NY time. Corrected in the uploaded file. Corrections: 20090422 20090121 have Crude Oil at 1030 instead of 1035; and over 20080604-20081231 have it at 1035; over 2005-2010 can find Crude Oil on Wed and subsequent Thu (when holidays; discard Wed entry). Uploaded file is actually in tsv; change to csv: sed 's/,//g; s/\t/,/g' macro.tsv > macro.csv --- In TWSAPI@..., "rwk2095" <r@...> wrote:
|
|
Re: Tick Encoding/Decoding
unatnahs57
Yes its java, your tools sound quite interesting..
toggle quoted message
Show quoted text
The time lag was simply the local time when I get a IB response, in milliseconds. The lag I meant was the time lag between when i get tickPrice and RTVolume. Seems first I get "id=0 lastTimestamp=1373851042" response, then tickPrice/tickSize response and then I get RTVolume. Also, appears that tickSize response can come even after RTVolume, as if two threads are writing the callback response. --- In TWSAPI@..., "rwk2095" <r@...> wrote:
|
|
Re: Tick Encoding/Decoding
unatnahs57
Richard,
toggle quoted message
Show quoted text
Thank you for your detailed reply and for sharing your encoder 1. It seems to me that we get the last traded price(on a change as u say) and the all tickSize are useful and represent trade sizes at the last traded price. This checks out with the RTVolume output, as in most cases I find the tickSize add up to the volume reported in the RTVolume tick. I can construct the classes with this information, although was hoping I could get it from someone in the forum. I'll post a java class I've written so far. 2. I've started recording every every tick I get from tickPrice/tickSize methods in the format of the data I posted, one text file per instrument. files sizes for emini ES,YM,NQ are about 100Mb with depth information (also because of the long timestamp i use) rest are smaller daily. Have a 2TB network drive so its not an issue. 3. Problem is I don't use a data base yet and a fast Java disk file read for the futures file is still quite slow for that file size when running a backtest. I think compression would greatly speed things up for me, especially if decoding isn't much of an overhead. I will continue to save text files that are readable in say "R". --- In TWSAPI@..., "Richard L King" <rlking@...> wrote:
|
|
Re: Connect to TWS in main not in Thread (Java)
twsapi
Hi btw12342001,
toggle quoted message
Show quoted text
thanks so much for the tip! That actually helps me a lot. I still did not completely understood your earlier reply on how to handle returned data. In which method is the returned data handled? I guess if the connection is not limited to just one thread you also run into synchronization issues between the threads. But as far as I understood you can initiate 6 or 8 connections but IB recommends to use as little as possible, is that correct? Regards --- In TWSAPI@..., "btw12342001" <newguy@...> wrote:
Not much of automation when I need to authorize it manually every time :)To fix the authorizing part you just need to set trusted IP addresses in TWS. Add 127.0.0.1 for the local machine. |
|
Re: Upcoming stock earnings dates
ramdukof
I found that subscribing to Wall Street Horizon (2.5usd) there are available upcoming earnings dates in TWS. They are also available for the API.
Though testing this shows it seems not really functional. The TWS data lacks dates for many stocks. The API only returns symbols, no detail of type of event or date, and it also returns maximum 50 symbols, where TWS shows 250 (I assume the first upcoming 250). So not very functional, but may still may help some. Like Japanese stocks can be checked, and you may then enter manually in TWS to get accurate date. Haven't found Japanese earnings anywhere else on the web. If someone has, please share.. Have questioned API support about the problems with this feature, so will see if they have some clarifications. |
|
Re: Tick Encoding/Decoding
btw12342001
id=0 RTVolume=3066.00;4;1373851042866;1108;3065.15771661;falseThe time in your log is from your computer and the RTV time is a timestamp, I'm guessing from the exchange. You have to have your clock accurate in order to measure the delay till when you get the tick. When you set your clock don't forget the ping time from the NIST server or wherever you get it from. Frankly I don't think you can measure that accurately enough to matter even though XP and later OSes can measure very fine time intervals nowadays (much better than the old 1/18 sec). My guess is RTV arrives at home much later than 40 msec after the actual exchange trade. My ping time just to IB's server is 60msec. iii) Seems we can construct TRADE ticks using tickPrice stream ourselves, although VWAP might get messed up if data breaks in between.No, you need tickSize for when price doesn't change. You can get everything useful from the RTV tick except bid and ask prices, for that you need tick price. |
|
Re: Tick Encoding/Decoding
Ed
I would just add that the whole issue of timing under Windows is
toggle quoted message
Show quoted text
rather a mess. If you want to get resolution higher than the default you'll have to research a bit to find the implications and gotcha's. On 7/15/2013 11:07 AM, Richard L King wrote:
|
|
Re: Tick Encoding/Decoding
rwk, your 'understanding' that Windows cannot accurately measure time
intervals shorter than 65 msec is completely erroneous. If you're programming directly to the Windows API, you can use QueryPerformanceCounter and QueryPerformanceFrequency. In .Net, use the Stopwatch class. With these, you can measure pretty accurately down to the microsecond level. These all depend on the presence of a high-resolution timer in your PC's hardware. This doesn't appear to be much of a limitation - I've never come across a computer that doesn't have one. Also if you want to do timer-based actions, you can do these with very small intervals by using the timeBeginPeriod to set the minimum timer resolution (down to 1 millisec). Use createTimerQueue and CreateTimerQueueTimer to initiate the timers (can't remember offhand what the corresponding .Net timer facility is, and I don't have time to look it up). Richard From: TWSAPI@... [mailto:TWSAPI@...] On Behalf Of rwk2095 Sent: 15 July 2013 15:18 To: TWSAPI@... Subject: [TWS API] Re: Tick Encoding/Decoding Your original post (OP) makes more sense now. I was unfamiliar with TickUtils.jar because it appears to be Java, and I don't speak Java. It sounds like I am doing something similar to TickUtils.jar. I write quotes and trades into a file of fixed-length records, 20 bytes each, which I refer to as a blob (binary large object). I then load the blob into memory for detailed analysis at my leisure using a single read command. As Richard mentioned, a blob is not directly readable, but I have tools that extract pieces and put them out in .csv format for analysis in Excel. I also have a playback tool that can replay a segment of the day at whatever speed I select. If you're using the RTVolume generic, you might also specify the "mdoff" flag to suppress quotes in the regular quote stream so that you're seeing only trades. This saves bandwidth. If you need quotes too, you can get them from reqMktDepthEx(). I don't understand the time lag you mentioned. How are you measuring the lag? It's my understanding that Windows cannot accurately measure time intervals shorter than about 65 msec. I believe the regular data stream from reqMktDataEx() is unsuitable for understanding market microstructure. [rwk] --- unatnahs57 <no_reply@...> wrote: 1. I am making data request as: mClientSocket.reqMktData(id, contract, "",false); which returns the "last price and quantity in separate callbacks".mClientSocket.reqMktData(id, contract, "233", false); investigate further, I wanted to reply at the earliest). still gives me IB response for tickPrice, tickSize methods, so I'm actually using more bandwidth. when trade is executed and when we get RTVolume response. tickString Tick data, then trade size and then volume upto that point. Seems 2nd trade_size is not preceded with TRADE_PRICE if the price is same as last traded price. although VWAP might get messed up if data breaks in between. |
|
C# Option Contract Details
rwk2095
You should be able to get all options by calling reqContractDetailsEx(). Just leave strike, right, and possibly expiry blank/zero.
toggle quoted message
Show quoted text
[rwk] --- "andrewcmeier" <andrewcmeier@...> wrote: Still not sure how to get all options for a stock but will keep trying. |
|
Re: Tick Encoding/Decoding
rwk2095
Your original post (OP) makes more sense now. I was unfamiliar with TickUtils.jar because it appears to be Java, and I don't speak Java.
toggle quoted message
Show quoted text
It sounds like I am doing something similar to TickUtils.jar. I write quotes and trades into a file of fixed-length records, 20 bytes each, which I refer to as a blob (binary large object). I then load the blob into memory for detailed analysis at my leisure using a single read command. As Richard mentioned, a blob is not directly readable, but I have tools that extract pieces and put them out in .csv format for analysis in Excel. I also have a playback tool that can replay a segment of the day at whatever speed I select. If you're using the RTVolume generic, you might also specify the "mdoff" flag to suppress quotes in the regular quote stream so that you're seeing only trades. This saves bandwidth. If you need quotes too, you can get them from reqMktDepthEx(). I don't understand the time lag you mentioned. How are you measuring the lag? It's my understanding that Windows cannot accurately measure time intervals shorter than about 65 msec. I believe the regular data stream from reqMktDataEx() is unsuitable for understanding market microstructure. [rwk] --- unatnahs57 <no_reply@...> wrote: 1. I am making data request as: mClientSocket.reqMktData(id, contract, "", false); |
|
C# Option Contract Details
I am not sure if anyone has answered your question to your satisfaction yet but I wanted to give you what I have figured out for options.
toggle quoted message
Show quoted text
using request market data you would have something like this in c# client.RequestMarketData(i, new Option("BAC", "BAC130720C00012000", "", RightType.Undefined, 0), some collection, false, false); --- In TWSAPI@..., "andrewcmeier" <andrewcmeier@...> wrote:
|
|
Re: problem with contract details
dairen62
Good news... the problem will be fixed on 7/18. We just need to include the full expiration in the format of YYYYMMDD when invoking the reqContractDeta ils( ) method.
toggle quoted message
Show quoted text
--- In TWSAPI@..., dairen62 <no_reply@...> wrote:
|
to navigate to use esc to dismiss