thank you for the clarification. Gives me a better handle on
what's going on. I'm going to rationalize my data approach in 2024
as I have MANY experimental scripts ( python, julia, and now mojo)
which are run by cron jobs. I just shut one down that's been
running for 2 years and there's a 20gb csv with all the data.?
Thanks again for all you do and taking the time.
On 9/28/23 14:33, 闯ü谤驳别苍 Reinold via
groups.io wrote:
toggle quoted message
Show quoted text
Good luck with log reading. I hope it helps you find the root
cause.
I removed lots of log detail in the hope that would make things
clearer, but apparently it did not. To summarize the log one more
time:
- The real-time market data request for a Frankfurt based
instrument is handled by eufarm in Switzerland
- My TWS is in the US, did not have an active connection to
eufarm but automatically established one when needed, No
surprise here.
- The somewhat surprising observation is that the Historical
Market Data Farm euhmds is also needed, although my client did
not request any historical data. In fact TWS always connects
the corresponding HMDS farm before the actual data farm since
it provides certain instrument related information it needs
for the real-time market data subscription.
Since you push the envelope a little by requesting historical
data for 100 instruments I thought it might be good to make sure
the client is careful with other requests that inadvertently
cause HMDS queries.
闯ü谤驳别苍
On Thu, Sep 28, 2023 at 02:16 PM, dent wrote:
thank you AGAIN for such a detailed analysis.? I have turned
on the HMDS logging feature for extra detail, thanks again for
the heads up.? I am always a little nervous when there is a
hint of data flow consolidation. I can see why they would want
to do it but...
the issue I have with these logs is that I would have
expected the log to show the unique identifiers for the
request. Your request reqMktData( ADS:STK:FWB )? has no, that
I can see, identifier NOR does the log give me a CLEAR
indication of which dataserver got the request. I understand
that they have load balancers but it would be nice to be able
to read the logs more clearly.
I had not thought to look at the internal TWS logs partly
because I don't like having to decrypt them all the time. I
hoped that I could turn encryption off but I suppose that it's
necessary in this day and age.
Based on your wonderful ( as always) overview I am going
through all my linux boxes ( scattered over the house) and
check the crontab for TWS scripts that might be running that
add to my data pacing. I'm decrypting an api log now so I'll
try to figure out what's going on. I am not a fan of the TWS
log encryption preferring straightforward text so I can ssh
into the tws machine and work that way.
thanks again for EVERYTHING you do.
?
