开云体育

Hi all having some problems with my code. I want to qualify the Contracts on the SPX ( about 48) and it's not returning. I tried using the repl but nothing happens it just hangs. I tried leaving it for 30 minutes no luck. So I decided to look at the logs to see what is going on. I can download, decrypt and look at the api logs but they seem to be too poorly formatted to be correct. I am seeing a lot of sequences like this one

13:45:56:090 -> ---.73-1-1-U2005714--AccountOrGroup-U999999-BASE-
13:45:56:090 -> ----73-1-1-U2005714--AccountOrGroup-U999999-USD-
13:45:56:090 -> ---/73-1-1-U2005714--AccruedCash-9999.0093631-BASE-
13:45:56:090 -> ---)73-1-1-U2005714--AccruedCash-9999.82-USD-
13:45:56:090 -> ---/73-1-1-U2005714--CashBalance-9999999.9998-BASE-
13:45:56:090 -> ---,73-1-1-U2005714--CashBalance-999999999.86-USD-
13:45:56:090 -> ---.73-1-1-U2005714--CorporateBondValue-0.00-BASE-

which looks like accounting information and not api centric.

and nothing, that I can see, pertinent to my call. In the repl I see ( last few cells )

Option(symbol='SPX', lastTradeDateOrContractMonth='20231116', strike=4515.0, right='C', exchange='SMART', tradingClass='SPX'), Option(symbol='SPX', lastTradeDateOrContractMonth='20231116', strike=4520.0, right='C', exchange='SMART', tradingClass='SPX'), Option(symbol='SPX', lastTradeDateOrContractMonth='20231116', strike=4525.0, right='C', exchange='SMART', tradingClass='SPX')]

but in the log I see

14:21:49:686 <- 1-11-48-0-SPX-OPT-20231116-4510.0-C--SMART----SPX-0--1-0--
14:21:49:686 <- 1-11-49-0-SPX-OPT-20231116-4515.0-C--SMART----SPX-0--1-0--
14:21:49:686 <- 1-11-50-0-SPX-OPT-20231116-4520.0-C--SMART----SPX-0--1-0--
14:21:49:728 -> ---	58-1-6-1-
14:21:49:732 -> ---81-6-0.05-c70003-3-
14:21:49:734 -> ---46-6-6-45-1693422900-
14:21:49:734 -> ---1-6-6-4-30.00-1-0----45-6-6-49-0.0-
14:21:49:738 -> ---
2-6-6-8-4548----1-6-6-6-41.50-0-0----1-6-6-7-30.00-0-0----1-6-6-9-41.55-0-0-
14:21:49:739 -> ---1-6-6-1-31.70-75-1----1-6-6-2-32.00-77-1-
14:21:49:813 -> ---	58-1-7-1-

So I can't tell if this is corruption ( denoted by  ) or something else like a failed decryption. What I would like to see is just the api traffic and not the accounting information. I thought I specified just the api when I saved and decrypted the file from TWS desktop. Is there a guide for analyzing the logs? A youtube video would be excellent.

I hit send too quickly, sorry. The api log level is "DETAIL" and I did read the docs

Example Log Entry:

[0:9:9:1:1:0:3:DET]Socket request - [3;52;IBM;STK;null;0.0;2;SMART;null;null]

From this example, we can tell that a socket client with clientId=0 connected and made a request for market data. The version of the market data request, which was 3,implies what data should have been sent.

so the first line below makes sense to me BUT the next line ( from the actual log segment above) makes no sense to me.

14:21:49:686 <- 1-11-50-0-SPX-OPT-20231116-4520.0-C--SMART----SPX-0--1-0--
14:21:49:728 -> ---	58-1-6-1-

A YouTube video? Really? How about some popcorn and soda with that movie?

What you are looking at are the raw messages your client sends to TWS/IBGW through the socket each time it calls a request method ("<-") and the responses from TWS/IBGW to your client that eventually become callbacks ("->"). You can find the source code for the message sender (the code in your client that converts your request calls into messages) and the decoder (that parses TWS/IBGW responses and makes the callbacks) in your TWS API installation for the programming language of your choice. Reading that source code also shows you how each message is structured.

Each message starts with a binary 4 byte message length field. They are generally not printable characters and may show up as "bird dropping" or other funny characters in the log and have nothing to do with corruption.

The second field in each message is the message id code that identifies what the message is about. It looks like you use Python (judging from a similar post you made in the ib_insync group) so that you can find the definition of the message id code in the IN and OUT classes that are defined in message.py:

• The first three messages below have code 1, so they are requests your client made. And it looks like those requests were made with tickerIds 48, 49, and 50.
• The next message is a response from TWS/IBGW with code 58 for a MARKET_DATA_TYPE callback () that shows live data (1) was requested for an earlier market data request with tickerId 6 not shown in your post.
• Another response with code 81 is a TICK_REQ_PARAMS callback () that, among other info, shows a minTick of $0.05 for ticker 6.
• Then a code 46 callback for ticker 6 and tick type 45 ""
• ....

The account related information you see are code 73 ACCOUNT_UPDATE_MULTI responses from TWS/IBGW () and are indeed part of the API operation between your client and TWS/IBGW.

There can be any number of things going on, but what jumps at me is that you SMART route your SPX option market data requests. I don't have too much practical experience with SPX options, but there may just not be a lot of market data available through that route. One of the options users may want to comment here. I'd try contracts in your calls that route directly to the exchange where SPX options are traded and monitor in TWS that there is activity for those options.

Happy debugging.

闯ü谤驳别苍

I am a huge fan of youtube videos when it comes to this kind of thing but I get your point. I did find this and will be following up with IB when I meet up with them next month.

I read through the docs that IB produce but I didn't see anything that leapt out at me regarding the source code. Thanks for pointing that out. I am using python for this.

As I said in my follow up post I found the layout of the log and will run the test tomorrow using it, to me it reads that the message format differs from your version

The log entries for API requests have the format:

[ClientID:ClientVersion:ServerVersion:ClientType:Request:Response:Version:LogEntryType]

where

ClientID - the clientId used when connecting.

ClientVersion - identifies the client's request stream (for internal use).

ServerVersion - identifies the server's response stream (for internal use).

ClientType - the type of API connection: DDE = 0, Socket = 1.

Request - If greater than 0, indicates that the log entry is the result of an API client request. The number shown is the request identifier as listed in the "Outgoing Request Identifiers" section below.

Response - If greater than 0, indicates that the log entry is the result of a server response to the API. The number shown is the response identifier as listed in the "Incoming Response Identifiers" section below.

Version - identifies the version of the request or response message. The version changes when the message format changes.

LogEntryLevel - identifies the type of log entry (i.e. the log level as listed above)

Example Log Entry:

[0:9:9:1:1:0:3:DET]Socket request - [3;52;IBM;STK;null;0.0;2;SMART;null;null]

From this example, we can tell that a socket client with clientId=0 connected and made a request for market data. The version of the market data request, which was 3,implies what data should have been sent.

Perhaps I am reading BOTH incorrectly. Thank you for taking the time to give me more information. I wanted to match my call to qualify contracts to the api log as it "seemed" simple. Decrypt the log to a text file and search for the relevant client call using the IB doc message layout. I'll take a quick look at EClientSocket tomorrow to see where I went wrong.

Ewald ( ib_insync) has coded up an EXCELLENT notebook that I was using to test out jupyter lab and that's how I dug myself into this. Normally I would be using pluto and julia but I was too lazy to add in the julia kernel. Ho hum. Thanks again for the WONDERFUL analysis.You are doing a marvelous job.

Well, you are reading (outdated) documentation for TWS logs (as in Help->Troubleshooting->Diagnostics->TWSLogs) but you have posted API logs (as in Help->Troubleshooting->Diagnostics->APILogs). They are very different.

I was commenting on the APILogs you had posted that your documentation link does not cover. The best documentation for APILogs is, still, the API source code.

闯ü谤驳别苍

here's what I was looking at note it's entitled API Logging

I'll take a look at the code tomorrow. thanks again

I know. But when you click the "Copyrights and Trademarks" link in the bottom left of that page you will find that the page is based on "TWS Javahelp version 008, March 2, 2007" hence my comment that the page is outdated.

Everything was a little different 16 years ago and what they called "API Logging" then has some vague resemblance to what you'll find in TWSLog today:

From my TWSLog from today:

[JTS-EServerSocket-326] - [0:173:173:1:0:0:0:DET] ReqAutoOpenOrders(15)::[version=1,autoOpenOrders=false]

So forget about that ancient artifact. And before you spend a lot of time on log debugging, give direct routing to the SPX options exchange a try. Should not take you more than a few seconds.

闯ü谤驳别苍

PS. A couple more hints that the page you are reading has little to do with reality is that it says "it logs certain information to its 'log.txt' log file," Try to find that one.

And the page documents only 14 Request identifiers and 14 Response identifiers. In TWS API 10.24, the highest Request identifier is REQ_USER_INFO = 104 and the highest Response identifier is USER_INFO = 107.

so the upshot of this is that IB's documentation is in the code. Interesting, certainly something to raise with the great and good from Stamford next month. Ironically I'll be in 141 W. Jackson on Friday so I'll see what they have to offer. To be honest I am REALLY enjoying using ewald's ib_insync for python. He has an excellent way of documentation and it's a real pleasure to use jupyter labs with his code now that it's got an integrated debugger.

I'm fine with spending a couple of hours on this as it's fun. Now you have pointed me at the code documentation I can have a little relax. This seems like a simple protocol and the asynch side looks pretty straightforward. Might be nice to just take a peek with wireshark to see how interesting IB's encryption is. Thanks again for your REALLY helpful observations.