Re: using reqAccountUpdates to create pandas dataframe for each new update

#47303

Change the default PyCharm colour scheme?
I use the Intelli Light scheme in PyCharm, and just copied and pasted into a new .py document. Readibility is good.
KH

def updatePortfolio(self,
        contract: Contract,
        position: float,
        marketPrice: float,
        marketValue: float,
        averageCost: float,
        unrealizedPNL: float,
        realizedPNL: float,
        accountName: str):
    position_dict = {'symbol' : contract.symbol,
                     'position': position,
                     'value': marketValue}

Re: using reqAccountUpdates to create pandas dataframe for each new update

#47301

Sorry, I notice I can't see the code I posted very well. Some of the
code elements seem to show up so lightly that I can't see them.
If I drag my cursor across it to select it, then I can read it.
Maybe it has something to do with my PyCharm color scheme that makes cut and paste unusable.?
Or, maybe there is a trick I need to learn. Anyone? ?

Re: using reqAccountUpdates to create pandas dataframe for each new update

#47300

In your class __init__, add:
? ? self.positions = pd.DataFrame()

in def updatePortfolio, try this:

def updatePortfolio(self,
        contract: Contract,
        position: float,
        marketPrice: float,
        marketValue: float,
        averageCost: float,
        unrealizedPNL: float,
        realizedPNL: float,
        accountName: str):
    position_dict = {'symbol' : contract.symbol,
                     'position': position,
                     'value': marketValue}
    if contract.conId in self.positions.index:
        self.positions.loc[contract.conId] = pd.series(position_dict)
    else:
        self.positions = self.positions.append(
            pd.DataFrame(positions_dict, index=[contract.conId]))

    self.positions.to_csv('positions.csv')

?This may not be ideal, but it might get you on a path that works for now and can be improved upon later.

BTW, I have not tried the code, but despite any errors, it will hopefully get you started. ?

Re: Does anyone here track fills on touch in their systems for futures?

Nick

#47299

Yes, assuming it's not a popular price. If it's something commonly followed like previous high/low/close there might be a bunch of orders there already.

Show quoted text

On 5/27/2021 3:08 PM, Raoul Suurmeijer wrote:
So, putting in the sell limit order at 4000 really early would increase your chance of actually getting triggered I assume?

Re: Does anyone here track fills on touch in their systems for futures?

#47298

开云体育

So, putting in the sell limit order at 4000 really early would increase your chance of actually getting triggered I assume? My open buy/sell limit always has an attached bracket order, all of them entered several hours in advance, so it's probably first in line I guess ;-)

Regards,

Raoul

On 27-05-2021 18:59, Nick wrote:

Show quoted text

I haven't traded e-mini's for a while but there used to be thousands of contracts at each price near the current price. When you place a limit order you go to the back of the line for that price, so all contracts ahead of you have to fill before you will get filled.

I don't know the percentages but the closer you are to a popular price the lower the odds will be.

You could also look into market-if-touched orders to see if more fills but with the risk of slippage winds up being a net gain.

Note that stops always fill since they turn into market orders when touched.

On 5/27/2021 12:41 PM, Patrick C wrote:

Hey guys I'm about to run through a ton of trades by hand to check if the fills would have filled.. For example, I want to sell 1 E-Mini contract at 4000 and the high of the current candle is 4000. What are the probabilities that I got filled? (I know theres so much more too it)

Time held for each positions

#47297

Is there a way to pull the time held for each position? For example, I want to buy a stock, hold it for 100 minutes, and then sell it.

Other than manually adding a counter to each symbol after each minute with a loop, is there a way to pull (through the API) the timestamp the position was originally opened? or how long it has been held?

using reqAccountUpdates to create pandas dataframe for each new update

#47296

Is there a way of using reqAccountUpdates and updatePortfolio to create a dataframe with all positions, and then overwrite the dataframe for each refresh?

I am using reqAccountUpdates so that I can get the cash balance and the positions all in one place. My problem is that each symbol comes in separately, so if I do something like this:

```

? ? def updatePortfolio(self, contract: Contract, position: float, marketPrice: float, marketValue: float, averageCost: float, unrealizedPNL: float, realizedPNL: float, accountName: str):

? ? ? ? df = pd.DataFrame([contract.symbol, position, marketValue])

? ? ? ? df.to_csv('positions.csv')
```

The CSV file ends up with the last position only, since its overwriting it for each symbol.

Re: Does anyone here track fills on touch in their systems for futures?

Nick

#47295

I haven't traded e-mini's for a while but there used to be thousands of contracts at each price near the current price. When you place a limit order you go to the back of the line for that price, so all contracts ahead of you have to fill before you will get filled.

I don't know the percentages but the closer you are to a popular price the lower the odds will be.

You could also look into market-if-touched orders to see if more fills but with the risk of slippage winds up being a net gain.

Note that stops always fill since they turn into market orders when touched.

Show quoted text

On 5/27/2021 12:41 PM, Patrick C wrote:
Hey guys I'm about to run through a ton of trades by hand to check if the fills would have filled.. For example, I want to sell 1 E-Mini contract at 4000 and the high of the current candle is 4000. What are the probabilities that I got filled? (I know theres so much more too it)

Does anyone here track fills on touch in their systems for futures?

#47294

Hey guys I'm about to run through a ton of trades by hand to check if the fills would have filled.. For example, I want to sell 1 E-Mini contract at 4000 and the high of the current candle is 4000. What are the probabilities that I got filled? (I know theres so much more too it)

This whole system I am going through is for the E-Minis during live trading hours. I was thinking of doing 50% for the pass through

Does anyone trade E-Mini's and keep track of this in their own systems? I think 50% should give me a ball park to see if the non-fills will break the profitability of the trading system. I've always felt like IB was pretty good with getting fills on touch with a limit order being smart routed.?

But maybe some one has got this data from years of running. That would be nice :)

Re: ticks never received after requesting market data

#47293

开云体育

Natan

?

Sorry to nit-pick, but I think it's a little misleading to say that 'the API allows for a hang': that makes it sound like the API is at fault and should be doing something different.

?

Your 'hang' is caused by your code waiting for something that may never happen. This is particularly the case with way-out-of-the-money options: sometimes no-one in the entire world wants to buy or sell them at any price, so there may be no bid or ask, frozen or otherwise. You assume the API should always send something, even when there's nothing to send: but it just doesn't work that way, and never has.

?

You could try using snapshot market data requests and handling the tickSnapshotEnd callback instead of programming your own timeout. If you've got all the data you need before receiving that, then cancel the request and get on with the next one. Otherwise the tickSnapshotEnd will come about 11 seconds after the request, and then you know you've got all the data available. NB: I've never tried snapshots with frozen data, so I don't know for sure if this will work.

?

Richard

?

Show quoted text

From: [email protected] <[email protected]> On Behalf Of nsh2tman
Sent: 27 May 2021 05:02
To: [email protected]
Subject: Re: [TWS API] ticks never received after requesting market data

?

Thanks for your responses, guys.

My script is not a live trading tool but rather a scanner that generates a list of candidates for me.
It uses Frozen data types because I mostly work on it after trading hours.
I never get -1 as tickPrice or tickSize. It either comes or it doesn't.

I ended up adding a timeout, but what bugs me is that the api allows for a hang.

Thanks again,
Natan

?

Re: ticks never received after requesting market data

#47292

Hi

Since it's a scanner, and if you don't need a model price of an option, you can look at?reqHistoricalData(1, DAY, 15minutes bars, trades, RTH=true)

Show quoted text

On Thu, May 27, 2021 at 7:02 AM nsh2tman <natan.shtutman@...> wrote:

Thanks for your responses, guys.

My script is not a live trading tool but rather a scanner that generates a list of candidates for me.
It uses Frozen data types because I mostly work on it after trading hours.
I never get -1 as tickPrice or tickSize. It either comes or it doesn't.

I ended up adding a timeout, but what bugs me is that the api allows for a hang.

Thanks again,
Natan

?

Re: ticks never received after requesting market data

#47291

Thanks for your responses, guys.

My script is not a live trading tool but rather a scanner that generates a list of candidates for me.
It uses Frozen data types because I mostly work on it after trading hours.
I never get -1 as tickPrice or tickSize. It either comes or it doesn't.

I ended up adding a timeout, but what bugs me is that the api allows for a hang.

Thanks again,
Natan

?

Re: ticks never received after requesting market data

#47290

Hi

comments inline

Artyom

On Wed, May 26, 2021 at 8:50 AM nsh2tman <natan.shtutman@...> wrote:

Hi everyone,

My python script is looping through a list of option tickers (calls and puts with specific strike and expiration).
For each of them, I am sending reqMktData. I do it one at a time: I don't send the next request until I received all ticks that I want for the previous one and until I issue cancelMktData.
For each call or put the ticks I need are the latest bid, ask, and volume. 3 values for each and then move on.

Consider going fully async / concurrent. Call cancelMktData only when you received enough data for your algo.
See? for more details about concurrency of realtime data requests.

?

Most of the time it works perfectly, but for some tickers, one of the ticks is never received and then my script hangs.
For example, put bid is never received, or call ask never received, etc. When I check the TWS gui for the same option, the field for the missing tick is indeed empty.

If strike is (deep) OTM, you may never receive any bid / ask for it.?
Excerpt from? :

When??and??are reported as -1, this indicates that there is no data currently available. Most commonly this occurs when requesting data from markets that are closed. It can also occur for infrequently trading instruments which do not have open bids or offers at that time of the request. To receive frozen quotes (the last available bid/ask recorded by the system) invoke the function??with argument 2. Alternatively, to receive "delayed frozen" data from tickers without holding market data subscriptions, specify a market data type of 4. Frozen data is exclusive to default tick types-??are not available- and requires market data subscriptions.

I believe, after receiving -1 on those ticks, you have enough information to qualify a particular strike as being low liquid.
?

This usually happens for low liquidity options, but my script doesn't know this upfront.
And thus, I kill the script, remove the ticker from the list, and restart.

Questions:
1. is this a known issue
2. is there a workaround I can use (i am thinking of timeout as the last resort)

I would implement async.

?

3. is there a way to "sniff" the option for the existence of all necessary ticks before issuing reqMktData

Take liquid strike, record all ticks.

?

Thanks in advance!
Natan

hope this helps

?

Re: ticks never received after requesting market data

Nick

#47289

If the data is not in TWS it would be unlikely to be in the API.

You can't force the API to work the way you want so you have to adapt to whatever (possibly wacky) things it does. In this case you you either have a timeout or create an architecture where you request and process at the same time.

Doing things sequentially is logically cleaner but gets you farther and farther from real time processing.

Show quoted text

On 5/26/2021 12:05 AM, nsh2tman wrote:
Questions:
1. is this a known issue
2. is there a workaround I can use (i am thinking of timeout as the last resort)
3. is there a way to "sniff" the option for the existence of all necessary ticks before issuing reqMktData

ticks never received after requesting market data

#47288

Hi everyone,

My python script is looping through a list of option tickers (calls and puts with specific strike and expiration).
For each of them, I am sending reqMktData. I do it one at a time: I don't send the next request until I received all ticks that I want for the previous one and until I issue cancelMktData.
For each call or put the ticks I need are the latest bid, ask, and volume. 3 values for each and then move on.
Most of the time it works perfectly, but for some tickers, one of the ticks is never received and then my script hangs.
For example, put bid is never received, or call ask never received, etc. When I check the TWS gui for the same option, the field for the missing tick is indeed empty.
This usually happens for low liquidity options, but my script doesn't know this upfront.
And thus, I kill the script, remove the ticker from the list, and restart.

Questions:
1. is this a known issue
2. is there a workaround I can use (i am thinking of timeout as the last resort)
3. is there a way to "sniff" the option for the existence of all necessary ticks before issuing reqMktData

Thanks in advance!
Natan

Swift + TWS

#47287

Hello,

I am a Swift developer and I am beginning to trade programatically. Does anyone do their trading using Swift? I see I can use most of the rest and web socket end points to get everything that’s provided in the pre-made SDKs.

- Thanks

Re: the fields commission, max/min Commission in openOrder state are broken

#47286

why do you think its wrong? In the first case it lists commissions in the range [0.17, 0.75], in the second case it shows commissions as 2.0. Have you tried the Preview Order button in TWS? It should show the same thing. If two accounts are configured differently they can different commissions for the same or similar orders.

Re: TWS API | Options market data request issue.

#47285

Hello,
As I can see, in latest TWS 984.1h issue about "concurrect live session" is fixed,
I can't reproduce it. What bug with options market data do you mean?

the fields commission, max/min Commission in openOrder state are broken

#47284

hello friends,

i'm placing whatif orders to get the commission. it's twsapi ver 981.01 for python.?
the state received in openOrder is broken in respect to the commission fields,?
in the following way:

the paper account gives:

? > SELL 90 IWM @ LMT 230.34 [+2.00% : SELL 100.00%]

? ? profit:406.49, commission:1.7976931348623157e+308, profitable:False

? ? minCommission:0.1786933, maxCommission:0.7546933

while the live account gives:

? > BUY 11 IWM @ LMT 219.24 [+0.00% : BUY 12.50%]

? ? profit:None, commission:2.0, profitable:True

? ? minCommission:1.7976931348623157e+308, maxCommission:1.7976931348623157e+308

both accounts are at hk.

sys.float_info.max == 1.7976931348623157e+308

what do you think i'm doing wrong?

thanks in advance,
alex

bracket order , parent = 'STP' 135 errorString= Can't find order with id =101

[email protected]

#47283

Hi,

I have a classic working brakect order with parent?LMT?, takeprofit?LMT??, and stoploss?TRAIL

I changed the parent? type to STP, end recive the following error:
>>reqId= 102? ?errorCode=? 135? errorString= Can't find order with id =101

where 101 is the parent order ID.
thanks, Harris

?