Streaming market data from native python IB API
This the third in a chain of posts on the use of the native python API for interactive brokers. You have to read the first, and the second one, before this one.
It is an updated version of this older submit, which used a third birthday party API (swigibpy) which wraps across the C API. I've modified the code, however not the terrible attempts at humour.
In my last post we looked at gettinga single snapshot of historical prices. In this one we will look atstreamed prices - 'market tick data'. Rather than wait until the historical price feed ends with streaming prices we need to tell the streamer when to stop.
Note: This post has been updated to use a much better method for handling concurrency.
No flow rises better than its supply
Get the source code from this gist.
You'll also need the pandas library.
You may additionally need to examine the documentation.
No circulate drives whatever with out being restrained
The example code begins in a similar fashion to the historical data example; we make one of these weird client objects containing a server wrapper connection, make one of these slightly less weird contract objects (here it is for December 2018 Eurodollar futures), resolve it into a populated contract object (explained more fully here) and then shove that into a request for market data.from __main feature:
app = TestApp("127.0.0.1", 4001, 1) ibcontract = IBcontract() ibcontract.SecType = "FUT"ibcontract.LastTradeDateOrContractMonth="201706"
ibcontract.Symbol="GBL"
ibcontract.Trade="DTB"
## resolve the agreement
resolved_ibcontract=app.Resolve_ib_contract(ibcontract) tickerid = app.Start_getting_IB_market_data(resolved_ibcontract) time.Sleep(30)
Unlike the alternative features we've checked out so far there is not an internal loop here; instead we deliberately hang around while a few rate records comes in.
From TestClient.Start_getting_IB_market_data() approach:
def start_getting_IB_market_data(self, resolved_ibcontract,
tickerid=DEFAULT_MARKET_DATA_ID): """ Kick off market information streaming :param resolved_ibcontract: a Contract object
:param tickerid: the identifier for the request
:go back: tickerid """
self._market_data_q_dict[tickerid] = self.Wrapper.Init_market_data(tickerid)
self.ReqMktData(tickerid, resolved_ibcontract, "", False, False, []) go back tickerid
Ah sure its the same old stuff of setting up space in the self.Wrapper instance to get the facts after which name the tws server request function (strictly speakme its one of those 'EClient' whatdoyoucallits once more). However one difference is that we need to shop the TestClients pointer to the marketplace facts queue (within the dict self._market_data_q), considering the fact that we're going to be returning to it later.
Only useless fish swim with the circulation...
We now look inside the server wrapper object which gets populated as an instance into self.wrapper. As before there are a few EWrapper functions which get triggered whenever the market data arrives.There are in fact numerous methods for 'tickString', 'tickGeneric', 'tickSize' and 'tickPrice'; it appears a bit stochastic (quant speak: english translation completely bloody random and arbitrary) which of these strategies receives called whilst a tick arrives (a tick may be an replace to a charge or to a quoted length at the top degree of the order book). Lets observe the most established of those:
def tickGeneric(self, tickerid, tickType, cost): ## overriden technique this_tick_data=IBtick(self.Get_time_stamp(),tickType, cost)
self._my_market_data_dict[tickerid].Positioned(this_tick_data)All the code does is identify which type of tick it is and then add it to the Queue that lives in the appropriate part of self._my_market_data. You can look at the classes IBtick and tick to see how this is done. I'm using local time as the timestamp here, but again you can change this if you want.
Dipping our toe into the metaphorical market stream
from __main feature:
We can see what information we've received to date. This also clears the queue of information that has been transmitted out of the app.Wrapper storage. You can write this differently if you like of route.
An man or woman tick looks as if this:
>>> print(market_data1[1]) ask_price ask_size bid_price bid_size \ 2017-03-10 11:07:51.564816 NaN NaN NaN NaN canAutoExecute ignorabletick last_trade_price \ 2017-03-10 11:07:51.564816 None None 98.03 last_trade_size pastLimit 2017-03-10 11:07:51.564816 NaN None
In this example the tick became a change, instead of a quote. The length of the change arrives within the next tick.
>>> print(market_data1[2])
ask_price ask_size bid_price bid_size \
2017-03-10 11:07:51.564899 NaN NaN NaN NaN
canAutoExecute ignorabletick last_trade_price \
2017-03-10 11:07:51.564899 None None NaN
last_trade_size pastLimit
2017-03-10 11:07:51.564899 200 None
Notice they're proven as a single row of a pandas Data Frame. This is so we can try this:
market_data1_as_df=market_data1.As_pdDataFrame() print(market_data1_as_df)
The advantage of this technique might be clean later, after I talk interpretation.
Once within the move of history you can not get out
If we just let that baby run we'd be receiving streams of prices until the cows came home. So what we do back in the client world is say STOP I've had enough after a preset amount of time (we could also STOP when the N'th tick has arrived, or when there all the slots in the marketdata are tuple are filled, which would be easy enough to code up).
from __main feature:
time.Sleep(30)
market_data2 = app.stop_getting_IB_market_data(tickerid)
from TestClient.Stop_geting_IB_market_data() method:
def stop_getting_IB_market_data(self, tickerid): """
Stops the flow of market records and returns all the information we've got had on the grounds that we final requested for it :param tickerid: identifier for the request
:return: market statistics
"""
self.CancelMktData(tickerid) ## Sometimes a lag while this occurs, this prevents 'orphan' ticks appearing
time.Sleep(5) market_data = self.Get_IB_market_data(tickerid)
## output any errors whilst self.Wrapper.Is_error(): print(self.Get_error())
go back market_data
This may also go back any records that we have not yet captured with a preceding call to get_IB_market_data. Again experience free to alternate this.
Making the outcomes meaningful
To understand the consequences we can use the electricity of pandas to resample the dataframe. First of all we could glue collectively the 2 seperate buckets of data we have captured:
market_data2_as_df=market_data2.As_pdDataFrame() all_market_data_as_df=pd.Concat([market_data1_as_df, market_data2_as_df])
Now to look the bid-ask quoting activity, resolved to a one 2d resolution:
some_quotes = all_market_data_as_df.Resample("1S").Ultimate()[["bid_size","bid_price","ask_price", "ask_size"]] print(some_quotes.Head(10))
bid_size bid_price ask_price ask_size 2017-03-10 11:07:51 9952.0 98.030 98.040 3736.0 2017-03-10 11:07:fifty two 2653.0 NaN NaN 212.0 2017-03-10 eleven:07:fifty three 17250.0 ninety eight.1/2 ninety eight.0.5 9500.0 2017-03-10 eleven:07:54 3607.0 ninety eight.030 98.040 424.Zero 2017-03-10 11:07:fifty five 12992.0 NaN NaN 5920.0 2017-03-10 eleven:07:fifty six 10073.0 NaN NaN 3743.Zero 2017-03-10 eleven:07:fifty seven 9746.0 NaN NaN 3726.Zero 2017-03-10 eleven:07:fifty eight 8280.0 NaN NaN 4110.0 2017-03-10 11:07:fifty nine 17.0 NaN NaN 1723.0 2017-03-10 11:08:00 2920.0 NaN NaN 3248.Zero
You may want to competently ahead fill the prices (a nan is proven when there may be no updated cost).
Or the primary few trades, resolved to 10 milliseconds:
some_trades = all_market_data_as_df.Resample("10L").Closing()[["last_trade_price", "last_trade_size"]] print(some_trades.Head(10))last_trade_price last_trade_size 2017-03-10 11:07:fifty one.560 ninety eight.03 200.0 2017-03-10 eleven:07:fifty one.570 NaN NaN 2017-03-10 11:07:fifty one.580 NaN NaN 2017-03-10 eleven:07:51.590 NaN NaN
Here I'm using the 'remaining' method. You may also use a mean.
By the manner it is able to be a piece risky to average prices an excessive amount of; as an example if you sample costs all through the day after which take an average as your input into your buying and selling set of rules you'll underestimate the real quantity of volatility in the market. Similarly if you are trading high frequency stuff you will be the usage of the energetic nation of the order ebook and averaging average real time bars might be now not going to be a very clever element to do. Over this short term relative to my traditional buying and selling pace however its likely okay as in general all we are going to be disposing of is a little illusory volatility due to 'bid-ask' jump.
Also inspite of this averaging its nonetheless worth walking your charges via a 'soar detector' to ensure you do not trade off dirty prices displaying spuriously large actions; I see those about once a month for every tool I exchange!
Much tons extra on this problem on this submit
Islands within the circulate...
That is it for expenses. I use the historical statistics characteristic whenever I start trading a particular contract but additionally every day because it receives close charges. This makes my system self recuperating considering although it drops for a few days I will emerge as with every day charges at least being infilled. Also regularly non actively traded contracts nonetheless have close expenses, useful if you are the use of intra agreement spreads as a statistics input. Just be careful how you deal with intraday and final prices if you append them collectively.
Much tons extra on this problem on this submit
I use marketplace facts to get intraday fees in which a gadget requires that, and after I am pretty much to trade to test the market is liquid enough for what I want to do (or even just to test it's miles open considering I don't trouble keeping a holidays calendar for all my markets - I wouldn't want to spend extra than 10 mins of my time an afternoon strolling this gadget now would I?). Plus it allows me to dis-combination my trading charges into what's coming from the inner spread, the price of processing / execution delays and having to drop deeper into the order book.
Next at the menu will be placing an order! I will go away the trivial undertaking of building a machine which decides what the orders might be to the reader (trace: you might want to use the charge in a few manner).
This is the 1/3 in a series of posts. The first two posts are:
http://qoppac.Blogspot.Co.United kingdom/2017/03/interactive-agents-native-python-api.Html
http://qoppac.Blogspot.Co.United kingdom/2017/03/historic-facts-from-native-ib-pyhon-api.Html
The subsequent put up on putting orders is:
http://qoppac.Blogspot.Co.United kingdom/2017/03/putting-orders-in-native-python-ib-api.Html
