Keyboard Shortcuts
ctrl + shift + ? :
Show all keyboard shortcuts
ctrl + g :
Navigate to a group
ctrl + shift + f :
Find
ctrl + / :
Quick actions
esc to dismiss
Likes
Search
Re: Pulling reqMktData for multiple tickers
|
So lets step back a bit.? Conceptually any regular old program you wirte wiith the standard IBK client library (there are other ways to do it) has two (2) threads of execution.? The first is the main thread that starts running when you run your Python program.? The second is a looping "Inbound" thread created by the IBK client library on the socket connection that reads any incoming messages from IBK, parses them, and calls the appropriate "callback" function for that particular message. So this is Asychronous in the sense that the first thread can and will freely fire off requests to IBK and meanwhile the second thread is happily reading in all the responses and invoking the callback functions.??? An analogy is if you are the first thread you can write a bunch of letters to IBK and drop them off in the mailbox then the Postal Service delivers them to IBK.? Eventually, at some point, at their convience, IBK will answer those letters and the Postal Service will evenutally deliver them to your mailbox where, lets say your Wife gets all the mail in your home and distributes all the letters to the family.? The Postal Service is the internet and your Wife is the Second Thread. Notice, unless you the Thread 1, the sender,? and your Wife, Thread 2, somehow "meetup" you will never know IBK responsed to your letters. This is actually somewhat key, but I'll not get into it now. OK, so lets talk threads: Should you create N threads to fire off N requests?? Not really necessary.? To send a Request Msg to IBK is a few 10s of bytes pushed over the socket and over the internet which a single thread can do Bang, Bang, Bang in a few milliseconds.? Could you spawn a dedicated thread for each individual and every individual request?? Yes, but the overhead in creating a thread is probably on par with just sending a request alone. So given Thread Main ?- Send Req 1 - Send Req 2 - Send Req 3 versus Thread Main ?- Spawn Thread to Send Req 1 ?- Spawn Thread? to Send Req 2 - Spane Thread to Send Req 3 Is a lot of compexity for little to no gain.? In fact the second will be longer.? Sending a request is a near nothing in time so in this example the first approach is fine. OK so what about in the Inbound side after you ripped through your requests bang, bang, bang.?? IBK will start sending responses for all three request which will arrive in some arbitrary and mixed order.? So for three reqMktData requests you are going to get a stream of incoming TickXXX msgs all mixed together.? Say TickPrice, TickSize, ... for? AAPL, SPY,? AAPL, AMD, SPY, SPY, AMD, ... ticks etc. So these are all coming in and are handled by the Inbound Thread, Thread 2.? However, you know that you used ReqId 101, 102 and 103 respectively for each request. So if a TickSize msg shows up, you know that if the ReqId is 101 that that was the AAPL mktDataReq TickSize etc. So what to do?? Well as each message comes in you do whatever you like with each one.? Only interest in the BID and ASK,? then throw all the others away.? You could write the Bid and Ask values into a DB.? Or you put them into a hashmap into a vector/list ...? When you get the Bid and Ask msgs for each stock you can happily send a cancel mktDataReq message for ReqId XXX.? Notice you are donig N requests with N streamed reponses and we haven't created a bunch of threads. We certainly could. No need!! (so far) as we are just doing: Send mktDataReq, Wait on Bid and Ask Tick msgs, Write to a file,? Cancel Req. So when we would ever need threads at all. ? Yea, this would start to get a little subtle.? You might create and use a Native Thread Pool if say you were to stream N reqMktData requests for N stocks and you were doing complex computational Time Series Analysis on each data stream.? You might create N Green Threads if say you structured your code into Agent Classes where each Agent did a series of work requests and processing.? For example, say I have a OptionChainAgent where creating and running an Agent with a given symbol would call ReqMktData on the underlying stock, collect some data, cancel the request, request a ContractDetails to get the ConId, they use the ConId to get the Options with 1.5 StdDevs from a set number of Exchanges for a given date, then for each stirke do a reqMktData to some data.?? Here I might run _each_ Agent on its own thread and then each Agent may create even more threads to do some parts of the above work stream. Mine do. So one final thought as ti whether it is ever dealing with threads at all.? Yes, but avoid if at all possible.? Consider a Watchlist you created in the IBK's TWS with 50 stocks.?? IBK could loop through each one, one-by-one, creating a reqMktData, geting some data, canceling the request, update the Watchlist, loop to the next symbol, forever.? Very inefficient.? Sloooow.? Insteak IBK probably fires off 50 reqMktData requests bang, bang, bang, .... and as a msg arrives for? a particlur symbol updates the Watchlist immediately.??? Basically there is a crossover point where threads become necessary. Sorry I know this is fustrating because you'd be a lot happier if I just answered with a nice snippit of working code.? But I don't use Python.? And I did offer the opinion that for your (from what I know) very simple case you don't need to deal with Threads? at all.? Final Thougt: IF YOU ARE NOT AN EXPERIENCED PROGRAMMER WITH CONCURRENT AND PARALLEL PROGRAMS THEN JUST DON"T USE THREADS UNLESS YOU REALLY REALLY REALLY HAVE TO. On Wed, May 5, 2021 at 2:02 PM Patrick C <zenfiretrading@...> wrote:
|
to navigate to use esc to dismiss