¿ªÔÆÌåÓý

ctrl + shift + ? for shortcuts
© 2025 Groups.io

Re: IB Gateway Memory Leak?


 

¿ªÔÆÌåÓý

I finally found some time to look into this.? Attached is a simple Java class that reproduces the issue.? It's around 700 lines of code, but most of that code is unimplemented required overloads.? When I started the program, IB Gateway was consuming 700 MB and after running the code (which took around 30 mins), the process was consuming over 4 GB.? The leak seems to be coming from request contract details (for option contracts).? Or at least that is the call that most contributes to the memory leak for my use case.? Importantly, IB Gateway still consumed 4 GB even after my attached puller program was closed.? I used JConsole to request several garbage collections on the IB Gateway, but that did not free up any memory.? If I leave the IB Gateway at the default max memory limit (768 MB I think), it will freeze up after only a few symbols.? My current work around is to use a very large memory limit and I have enough system memory to do that.? Interestingly, while I was looking at the IB Gateway using JConsole, I noticed that it had over 17,000 classes loaded.? That gives me some clue as to why it takes TWS so long to load...

I submitted a ticket with IB, but their uploader did not allow me to upload the attached source code.? So hopefully they will follow-up.? If course, you find find something in my code that I am doing wrong, please let?me know.? However, I don't see how my program could cause a memory leak in their separately-executing process and why it would still persist even after my program terminates.


From: [email protected] <[email protected]> on behalf of Ray Racine <ray.racine@...>
Sent: Thursday, June 24, 2021 9:16 AM
To: [email protected] <[email protected]>
Subject: Re: [TWS API] IB Gateway Memory Leak?
?
Been a long while since my java days ...

But you did ask about some possible things ...

AFAIK all of the JVM suppliers offer a tool or a procedure (send a POSIX signal to the JVM process)? to force a gc collection as well as a heap dump, event attaching an external monitor to a running? JVM etc.

Doing it low level and a bit raw (there are free and vendor supplied higher level tools for all this):
  1. Cause JVM to allocate a bunch of memory for a bit of time, then stop.
  2. Force a full GC.? (e.g. if using oracle jvm, via their jcmd tool etc.)
  3. Signal a "Heap Dump".
  4. Cause JVM to allocate a bunch of? memory over some time interval (say 5 mins) then stop.
  5. Force a full GC.
  6. Signal a "Heap Dump"
  7. Use vendor or other tool to analyze the Heap Dump.? Unless things changed even a grep | sort | count can be effective here.
You should clearly see some (it is usually one thing) run away count of object instances of some Class causing the leak.? It is rarely the class itself that is the issue, it is its use as an element of some collection container which makes it reachable thus? preventing gc'ing.

For an actionable bug report just hitting memory allocation hard for some duration to a "steady state", force gc, dump gc stats, hitting allocation hard again for 5-10 mins, force gc, dump gc stats should be sufficient.






On Wed, Jun 23, 2021 at 1:46 AM Crow <aaroncook394@...> wrote:
I notice that the IBKR Gateway will quickly run out of memory if left at the default Memory allocation setting of 768 MB.? I only notice this when I pull a lot of option chains such as 3 expirations, all strikes. I can usually get the Gateway to freeze up completely after about 5 to 7 symbols.? When I say freeze up, I mean that there is no response to my last request.? I can close my app, start a new one and it's still stuck.? If I increase the memory allocation to 26 GB, I can pull 500 symbols without running out of memory, but the memory used by the application steadily increases apparently without bound.? I only noticed this with Gateway after version 978 although it's possible that earlier versions were doing it too and I just did not notice because I already set the memory allocation very high.? Does anyone else notice this?? Is the application notorious for leaking memory?? Is there some call that I am supposed to do to free memory?? I assumed that after it passes the message to the app, then it simply deletes the message.? Perhaps not...

Join [email protected] to automatically receive all group messages.