I love Python but it has its own place and it's not good for real-time-ish parallel compute. You can hack around things but in the end it won't be crazy.

You can consider Go and node.js from mainstream languages (apart from C++).

I personally always wanted to try Elixir but never got around - perfect for highly concurrent stuff.?

On Sat, Mar 13, 2021 at 2:52 PM matt <rmdeboer82@...> wrote:

What would be the best way to do on-the-fly computations using Numpy, Pandas for trading using IB TWS API, writing to/from mySQL, and in the meanwhile receive and send data and orders?

From (and ) I conclude that CPU-bound programs, those that spend most of the their time processing data (like on the-fly computations using Pandas which is synchronous single threaded), solutions like threading (pre-emptive multitasking) and asyncio are inferior (and may disrupt the sending/receipt of data or introduce some lag or data drops) since there is only one Python interpreter on a single core/processor (asyncio aligns I/O processes more efficiently) that takes care of both the CPU computations and checking for socket data.

I know there is? a ib_insync library that implements cooperative multitasking (asyncio) in Python, but will the overall program be fast enough if you do computations using large dataframes? I read that on a socket level there might not be any limitations (“socket itself is actually thread-safe”), as suggested by . I do not assume that Ptyhon has it implemented this way since the Global Interpreter Lock is in place (single thread, synchronous). From :

IB.sleep(0); may be an alternative, but I do not think a Pandas groupby() can be interrupted once called.

Should I use the Multiprocessing library and do every task on its own processor (thus dataframe calculations, mySQL connection read/write, sending and receiving orders and data)?
Or should I combine some things using (I think some next level threading which can be combined with the normal way of interpretation of code by Python) or the (has to do with built-in async for Scikit-Learn, Pandas, etc. functions, if I understand correctly) library? I think this is next level, and may be an overkill.

Or should stick to Java (I do not know if there is a Pandas like alternative which allows easy manipulation of data and performing machine learning for example, but threading may be organized better)?

Any suggestions or tutorials?

(I read something about was well, but that’s a whole different story since it is not directly applicable to IB TWS API, and may be prone to delays resulting from single core processing/interpreting by Python as well, if I understand correctly)

What would be the best way to do on-the-fly computations using Numpy, Pandas for trading using IB TWS API, writing to/from mySQL, and in the meanwhile receive and send data and orders?

From (and ) I conclude that CPU-bound programs, those that spend most of the their time processing data (like on the-fly computations using Pandas which is synchronous single threaded), solutions like threading (pre-emptive multitasking) and asyncio are inferior (and may disrupt the sending/receipt of data or introduce some lag or data drops) since there is only one Python interpreter on a single core/processor (asyncio aligns I/O processes more efficiently) that takes care of both the CPU computations and checking for socket data.

I know there is? a ib_insync library that implements cooperative multitasking (asyncio) in Python, but will the overall program be fast enough if you do computations using large dataframes? I read that on a socket level there might not be any limitations (“socket itself is actually thread-safe”), as suggested by . I do not assume that Ptyhon has it implemented this way since the Global Interpreter Lock is in place (single thread, synchronous). From :

IB.sleep(0); may be an alternative, but I do not think a Pandas groupby() can be interrupted once called.

Should I use the Multiprocessing library and do every task on its own processor (thus dataframe calculations, mySQL connection read/write, sending and receiving orders and data)?
Or should I combine some things using (I think some next level threading which can be combined with the normal way of interpretation of code by Python) or the (has to do with built-in async for Scikit-Learn, Pandas, etc. functions, if I understand correctly) library? I think this is next level, and may be an overkill.

Or should stick to Java (I do not know if there is a Pandas like alternative which allows easy manipulation of data and performing machine learning for example, but threading may be organized better)?

Any suggestions or tutorials?

(I read something about was well, but that’s a whole different story since it is not directly applicable to IB TWS API, and may be prone to delays resulting from single core processing/interpreting by Python as well, if I understand correctly)

Hi Francois,

thx for the links.
All of this leaves me really speachless about the quality of the IB API.
I am a professional software developer since >30years, but a function like reqSecDefOptParams?plays in the primary league of "worst design never".
Not only does it return invalid data, but the concept inherently leads to massive server requests.

So sad, and since this topic is years old, IB obviously does not care.
Do they even have a valid email adress to their developers?

This is really unbelievable.
The quality of this API is really low, which is especially evident in this problem.
I have now implemented something with Yahoo finance, which gives me the valid option chains.
Then I create all needed contracts and serialize them to a file(because it takes an enormous amount of time to get them).
That way, the cached contracts can be reloaded and only new expiration chains well need to be added.
But still, why do we need to do this?
We deserve better from IB, since all this data is available in the option trader window.

I now struggle in reveiving the greeks for the options. Are they even there when markets are closed?

Hi,

I believe TWS comes with its own Java built into the installer.? I'm pretty sure I use that one.? TWS has never been the best written app but if you stick with stable its OK.?

I believe there are some weird security things built into IB's JVM they distribute which is why you can't make a stock JVM work.? I think if you dig into the IB JVM, you will probably find some security configs in the jre/lib directory (or inside of one of its sub-directories) that allow TWS to work.? Perhaps jre/lib/security/java.policy has been altered to only run IB signed jars or that TWS needs a custom Java security policy.? If you really want to hammer away at this that's where I would look.? As for the performance issues with TWS, perhaps the screen config you have is too busy or perhaps its using up all the pacing capacity and that's causing issues with refreshing the screen.? I have a hard time believing that upgrading the JVM would help.? The Java 8 JVM is really mature and the changes to it are mostly about very minor things that probably wouldn't impact the performance of a GUI app.

Hunter

On Monday, March 15, 2021, 11:32:32 AM PDT, John <jpelly@...> wrote:

No such luck for me - I let that error dialog sit and nothing happens; just sits there forever (rest of TWS will not start up).

I'd like to run TWS on my local Linux laptop, so hopefully someone else comments on this thread with some additional suggestions.

Correction to original message: I'm running Linux Mint 20.

Thanks all

John

On Mon, Mar 15, 2021 at 1:04 PM Ray Racine <ray.racine@...> wrote:

On latest Fedora Linux and I'm getting the exact same error as of today as well.? TWS ultimately starts up fine however.? I assume just not on the latest update :).

With regard to performance,? I've never had to do anything beyond adjusting the Max Heap. ? Performance is retalive, but I have some pretty big watch lists, charts with studies, plus API clients and performance has been fine for me.

I do get quite a few UI issues with menus and mouse pointing (not calculating the X-Y position correctly mostly) when I run under Sway / Wayland.??? The occasional annoying UI issues with XMonad / XOrg.? And no real UI issues to speak of with Gnome - Xorg or Wayland.

On Mon, Mar 15, 2021 at 11:52 AM John <jpelly@...> wrote:

I'm experiencing pretty bad performance of TWS GUI on Linux (Mint 19). I've tried to upgrade Java version that TWS uses, thinking that will help.

However I'm unable to change versions. Reading through the tws shell script, it looks like it pins the JRE version at 1.8.0_152. If I remove the check in the shell script and try to upgrade to openjdk version 1.8.0_282 (upgraded patch level), TWS won't start - says "we are unable to update the platform due to a networking error" (see attached screenshot)

Has anyone successfully been able to do this? It looks like you can upgrade Java versions on Windows so I was hoping to do it on Linux.