Welcome to ib_async forum

#1

This is a new groups.io forum for ib_async (formerly ib_insync). I am testing it out to see if there will be enough interest to warrant the paid version as the free version now only handles up to 100 members.

Re: Welcome to [email protected]

#2

开云体育

Hi Everybody!

I’m in.

I’ll introduce myself while we are a small group :). So in short, I’m some kind of algorithm trader. And I’m interested in keeping this thing rolling.

Cheers ?

Https://github.com/gnzsnz/

Show quoted text

On 30 Apr 2024, at 20:21, Group Notification <[email protected]> wrote:

Hello,
Welcome to the [email protected] group at 开云体育, a free, easy-to-use email group service. Please take a moment to review this message.
Regards,
The [email protected] Moderator

Re: Welcome to [email protected]

#3

Welcome. Have you used Pystore? Sounds interesting.

Re: Welcome to [email protected]

#4

开云体育

Yes, I’m using pystore

I’m quite happy with it, except for some problems I found recently. Unfortunately it does not seem to be under active development, I have sent a pull request to have it working with latest pandas and dask but no answer so far.

I store daily price bars for many stocks and ETFs and it works quite well. The library is really simple so it’s easy to go through the code if needed.

I haven’t seen many options available on this space, there is ArticDB, but it’s not open source. And from what I understand a “production” DB requires a paid license. So I won’t spent time building my code around it.

Do you know any other alternatives on this space?

Regards,

Gonzalo

Show quoted text

On 3 May 2024, at 06:07, Mel <climbermel@...> wrote:

Welcome. Have you used Pystore? Sounds interesting.

Re: Welcome to [email protected]

#5

开云体育

I've been using mySQL, but you need a database server running.? I have one on? all the time for various things so I just use it.? I'm just interested in how it works and how it stores the data.?

Show quoted text

-------- Original message --------

From: "Gonzalo Saenz via groups.io" <yo@...>

Date: 2024-05-03 11:36 p.m. (GMT-08:00)

To: [email protected]

Subject: Re: [ib-async] Welcome to [email protected]

Yes, I’m using pystore

I’m quite happy with it, except for some problems I found recently. Unfortunately it does not seem to be under active development, I have sent a pull request to have it working with latest pandas and dask but no answer so far.

I store daily price bars for many stocks and ETFs and it works quite well. The library is really simple so it’s easy to go through the code if needed.

I haven’t seen many options available on this space, there is ArticDB, but it’s not open source. And from what I understand a “production” DB requires a paid license. So I won’t spent time building my code around it.

Do you know any other alternatives on this space?

Regards,

Gonzalo

On 3 May 2024, at 06:07, Mel <climbermel@...> wrote:

Welcome. Have you used Pystore? Sounds interesting.

Re: Welcome to ib_async forum

#6

Many thanks you setting this up. Hopefully we will have a thriving community as we once did . Good luck to us all and RIP Ewald . You will be always be remembered on this group.

Re: Welcome to ib_async forum

#7

Yes, hopefully it does well. Once more people get on here, I thinking of setting up a code repo. I have a repo on GitHub already for ib code, so I could clean that up to use. That would be more for code snippets such as getHistory calls and such.

Re: Welcome to [email protected]

#8

开云体育

Well I think pystore is a good package.

As I said I faced a few issues recently, mainly due to pandas and dask move from fast parquet to pyarrow, it has break a few things. The package is not under much maintenance, which is not great.

But being a relatively simple package I manage to do the necessary changes to have it working.

This is my pull request on pastors to move to pyarrow??so if want to test it pull from there. Because the official package is broken unless you find the right combination of pandas, dask and pyarrow.

It’s a lot simpler than having a database, and VERY scalable because it works with dask

Show quoted text

On 4 May 2024, at 09:23, Mel <climbermel@...> wrote:

I've been using mySQL, but you need a database server running.? I have one on? all the time for various things so I just use it.? I'm just interested in how it works and how it stores the data.?

-------- Original message --------
From: "Gonzalo Saenz via?" <yo@...>?
Date: 2024-05-03 11:36 p.m. (GMT-08:00)?
To:?[email protected]
Subject: Re: [ib-async] Welcome to?[email protected]

Yes, I’m using pystore

I’m quite happy with it, except for some problems I found recently. Unfortunately it does not seem to be under active development, I have sent a pull request to have it working with latest pandas and dask but no answer so far.

I store daily price bars for many stocks and ETFs and it works quite well. The library is really simple so it’s easy to go through the code if needed.

I haven’t seen many options available on this space, there is ArticDB, but it’s not open source. And from what I understand a “production” DB requires a paid license. So I won’t spent time building my code around it.

Do you know any other alternatives on this space?

Regards,
Gonzalo

On 3 May 2024, at 06:07, Mel <climbermel@...> wrote:

Welcome. Have you used Pystore? Sounds interesting.

Re: Welcome to [email protected]

#9

开云体育

What would be the use of dask when running ib?? Would you be able to distribute one loggin across multiple machines?

Show quoted text

-------- Original message --------

From: "Gonzalo Saenz via groups.io" <yo@...>

Date: 2024-05-06 4:23 a.m. (GMT-08:00)

To: [email protected]

Subject: Re: [ib-async] Welcome to [email protected]

Well I think pystore is a good package.

As I said I faced a few issues recently, mainly due to pandas and dask move from fast parquet to pyarrow, it has break a few things. The package is not under much maintenance, which is not great.

But being a relatively simple package I manage to do the necessary changes to have it working.

This is my pull request on pastors to move to pyarrow??so if want to test it pull from there. Because the official package is broken unless you find the right combination of pandas, dask and pyarrow.

It’s a lot simpler than having a database, and VERY scalable because it works with dask

On 4 May 2024, at 09:23, Mel <climbermel@...> wrote:

I've been using mySQL, but you need a database server running.? I have one on? all the time for various things so I just use it.? I'm just interested in how it works and how it stores the data.?

-------- Original message --------
From: "Gonzalo Saenz via?" <yo@...>?
Date: 2024-05-03 11:36 p.m. (GMT-08:00)?
To:?[email protected]
Subject: Re: [ib-async] Welcome to?[email protected]

Yes, I’m using pystore

I’m quite happy with it, except for some problems I found recently. Unfortunately it does not seem to be under active development, I have sent a pull request to have it working with latest pandas and dask but no answer so far.

I store daily price bars for many stocks and ETFs and it works quite well. The library is really simple so it’s easy to go through the code if needed.

I haven’t seen many options available on this space, there is ArticDB, but it’s not open source. And from what I understand a “production” DB requires a paid license. So I won’t spent time building my code around it.

Do you know any other alternatives on this space?

Regards,
Gonzalo

On 3 May 2024, at 06:07, Mel <climbermel@...> wrote:

Welcome. Have you used Pystore? Sounds interesting.

Re: Welcome to [email protected]

#10

开云体育

daks allows to distribute computation across a cluster. It basically allows to work with dataframes greater than your available memory. How? By applying computation and make it small enough before you receive it

So you can store high frequency price data, and just get last X bars for example.

I’m not there yet, I’m using mainly daily data. And parquet (pystore storage backend) is really good at keeping files small. I have RUS3000 companies going back 15years and it’s only 401M.?

I have developed a small layer on top of pystore and ib_insync (to be migrated to ib_async) and I can get my data collection updated daily

For in-memory data dask is an overkill. It’s a subset of pandas with cluster capabilities. But for pystore it offers the possibility to escalate if needed.

The most challenging scenario is data updates, for example every day add new data. Dask can do this without putting the whole file in memory

Dask is a big framework, you can distribute your own code across a cluster, and run machine learning in a cluster. I have not used it so far. But in principle I could run a few things across 2-3 PCs that I have here. I have work to do on my algorithm before going there.

Show quoted text

On 6 May 2024, at 18:05, Mel <climbermel@...> wrote:

What would be the use of dask when running ib?? Would you be able to distribute one loggin across multiple machines?

-------- Original message --------
From: "Gonzalo Saenz via groups.io" <yo@...>
Date: 2024-05-06 4:23 a.m. (GMT-08:00)
To: [email protected]
Subject: Re: [ib-async] Welcome to [email protected]

Well I think pystore is a good package.

As I said I faced a few issues recently, mainly due to pandas and dask move from fast parquet to pyarrow, it has break a few things. The package is not under much maintenance, which is not great.

But being a relatively simple package I manage to do the necessary changes to have it working.

This is my pull request on pastors to move to pyarrow??so if want to test it pull from there. Because the official package is broken unless you find the right combination of pandas, dask and pyarrow.

It’s a lot simpler than having a database, and VERY scalable because it works with dask

On 4 May 2024, at 09:23, Mel <climbermel@...> wrote:

I've been using mySQL, but you need a database server running.? I have one on? all the time for various things so I just use it.? I'm just interested in how it works and how it stores the data.?

-------- Original message --------
From: "Gonzalo Saenz via?" <yo@...>?
Date: 2024-05-03 11:36 p.m. (GMT-08:00)?
To:?[email protected]
Subject: Re: [ib-async] Welcome to?[email protected]

Yes, I’m using pystore

I’m quite happy with it, except for some problems I found recently. Unfortunately it does not seem to be under active development, I have sent a pull request to have it working with latest pandas and dask but no answer so far.

I store daily price bars for many stocks and ETFs and it works quite well. The library is really simple so it’s easy to go through the code if needed.

I haven’t seen many options available on this space, there is ArticDB, but it’s not open source. And from what I understand a “production” DB requires a paid license. So I won’t spent time building my code around it.

Do you know any other alternatives on this space?

Regards,
Gonzalo

On 3 May 2024, at 06:07, Mel <climbermel@...> wrote:

Welcome. Have you used Pystore? Sounds interesting.

Re: Welcome to ib_async forum

#11

开云体育

As we are on the subject of a thriving community.

Are there any plans of making this public? It’s on Mel’s discord, but shall it be published on the GitHub repo? If everybody is OK I would like to publish it in reddit’s algorithmic trading sub.

Anything else?

Show quoted text

On 4 May 2024, at 17:18, nkulki <nkulki@...> wrote:

Many thanks you setting this up. Hopefully we will have a thriving community as we once did . Good luck to us all and RIP Ewald . You will be always be remembered on this group.

Forum planning #forum_planning

#12

So we should do some planning before we invite the world here... We will want to consider some #hashtags as those allow us to have sub topics and also allow users to filter what topics they get emails in regard to. I set up this thread as an example of using #hashtags Publishing it on Reddit as Gonzalo suggested could send us pass the 100 user limit before we have had time to get it organized and need to start paying for it. It is currently set so that I have to approve all new members, if we have a few hundred dropping in all at once to check it out, that could be a lot of work. The nice thing about groups.io is that I can automate a lot of that, I just need to get everything set up for that.

Status of Ib-Async

#13

I thought it might be a good idea to start a thread to post how (the former) Ib_Insync, now Ib-Async, is progressing / evolving.
Just to keep an overview for everyone.

For starters, shortly after Ewald passed away, two people (atleast that I know of) stepped forward to take some responsabilities concerning the library: Mel and Mattsta.
Mel took initiatives concerning a user group forum (thanks to Mel we now have this 开云体育 group). Mattsta seems involved in maintaining the library code itself.

This is what I know. This is certainly not the whole story and perhaps not even entirely correct.
That's why I would like to encourage people to post here everything they know (and find out) about ongoing things about Ib-Async.

Thanks already!?

Re: Welcome to [email protected]

#14

OK that sounds fun, but it would be way beyond anything I would be doing. Here I thought having a server rack in my office was extreme... I have tried clusters yet.

Mel

Re: Welcome to [email protected]

#15

I have been trading options with IBKR since 2012.?
About 4 years ago I started using Ib_Insync to extract information from TWS, mainly data about my open positions (margin requirement etc.).
Later I started to extract option chain data from TWS for graphing.

This journey, connecting with TWS via the API, was also my journey to learn Python. So it was a double learning experience for me.
Needless to say, the software I have now is a complete mess, and I should clean it up some day ....
That said, I use this software every day and I hope that the Ib_Insync library will be kept up-to-date as "Ib-Async" for a long time to come....?

Re: Welcome to [email protected]

#16

I use it daily to get all my portfolio data such as positions, trades etc. I use the flex queries for that. Let me know if you need help cleaning up the code.

Mel

Re: Welcome to [email protected]

#17

I forgot to add, that I use PySimpleGUI to program the GUI of my Ib_Insync code.

Simple and lightweight. I like it a lot!

Re: Welcome to [email protected]

#18

开云体育

I may have to look at that.? I've just used tkinter since it is already part of python.?

Mel

Show quoted text

-------- Original message --------

From: "Ray Johnson via groups.io" <lasgo2000-1@...>

Date: 2024-05-08 4:25 p.m. (GMT-08:00)

To: [email protected]

Subject: Re: [ib-async] Welcome to [email protected]

I forgot to add, that I use PySimpleGUI to program the GUI of my Ib_Insync code.

Simple and lightweight. I like it a lot!

Re: Welcome to [email protected]

#19

PySimpleGUI uses tkinter, QT, WxPython, and/or Remi under the hood.

?

Re: Forum planning #forum_planning

#20

I agree with taking it slowly to start off since I'm new to this groups.io thing. Mel when you say you can automate a lot of the admin, how exactly would you accomplish that??

Before we open the floodgates, we need to decide what kind of forum we want this to be: do we want it to be exclusive to maybe only a select few dozen vetted members that can share coding and trading ideas without requiring constant support / moderating etc., or a more open, inclusive forum that is more dedicated to helping newbies get started and troubleshooting issues? Personally I would prefer the former but am not against being more inclusive, although my capacity for providing support and troubleshooting might not be up to the task lol!

As for a list of hashtags, here's a start:

#async_support, #ib_support, #coding, #trading_ideas, #algo_trading, #general?

(note no #python_support since there's plenty of resources out there for this!)