The worlds simplest execution algo
As you'll realize I run a fully automatic systematic buying and selling device. As its absolutely automated, due to my severe laziness, all of the trades are positioned into the marketplace completely automatically.
When I first started walking my machine I saved the execution manner extremely easy:
- Check that the best bid (if selling) or best offer (if buying) was large enough to absorb my order
- Submit a market order
Yes, what a loser. Since I am buying and selling futures, and my broker only has fancy orders for equities, this appeared the perfect alternative.
I then compounded my distress by way of creating a nice every day report to tell me how tons every alternate had cost me. Sure sufficient most of the time I become paying half of the interior unfold (the difference among the mid fee, and the bid or provide).
After a couple of months of this, and getting bored to death with seeing this file upload up my losses from buying and selling every day, I decided to chunk the bullet and do it well.
Creating cool execution algorithms (algos) isn't always my vicinity of deep information, so I had to paintings from first ideas. I additionally don't have a great deal enjoy of writing very complex fast event driven code, and I write in a slowish high stage language (python). Finally my orders aren't very huge, so there is no need to interrupt them up into smaller slices and song every slice. All this factors towards a simple algo being enough.
Only one greater element to don't forget; I get charged for enhancing orders. It isn't always a massive cost, and its really worth a whole lot less than the saving from smarter execution, however it nonetheless way that creating an algo that modifies orders an immoderate number of times in which this isn't always necessary in all likelihood isn't always well worth the more work or value.
Finally I cannot adjust a limit order and flip it into a market order. I would ought to cancel the order and post a brand new one.
What does it do?
A good human trader, wanting to execute a smallish buy order and not worrying about game playing or spoofing etc, will probably do something like this:- Submit a limit order, on the same side of the spread they want to trade, joining the current level. So if we are buying we'd submit a buy at the current best bid level. In the jargon this is skipive behaviour, waiting for the market to come to us.
- In an ideal world this initial order would get executed.We'll have gained half the spread in negative execution cost (comparing the mid versus the best bid).
- If:
- the order isn't being executed after several minutes,
- or there are signs the market is about to move against them, and rally
- or the market has already moved up against them
- ... then the smart trader would cut their losses and modify their order to pay up and cross the spread. This is aggressive behaviour.
- The new modified aggressive order would be a buy at the current best offer. In theory this would then be executed, costing half the spread (which if the market has already moved against us, would be more than if we'd just submitted a market order initially).
- If we're too slow and the market continues to move against us, keep modifying the order to stay on the new best offer, until we're all done
Although it really is it in a nutshell there are nonetheless some bells and whistles in getting an algo like this to paintings, and in such a way that it can deal robustly with something that gets thrown at it. Below is the detail of the algo. Although that is proven as python code, its now not executable on the grounds that I have not blanketed the various relevant subroutines. However it must come up with sufficient of an idea to code some thing similar up yourself.
Pre trade
It's somewhat dangerous dropping an algo trade into the mix if the market isn't liquid enough; this routine checks that.pretrademultiplier=four.0
def EasyAlgo_pretrade(ctrade, contract, dbtype, tws):
"""
Function easy algo runs before getting a new order
ctrade: The proposed exchange, an signed integer
agreement: object indicating what we are trading
dbtype, tws: handles for which database and tws API server we're coping with here.
Returns integer indicating length I am glad with
Zero means marketplace can't assist order
"""
## Get marketplace statistics (a list containing internal unfold and length)
bookdata=get_market_data(dbtype, tws, settlement, image=False, maxstaleseconds=5, maxwaitseconds=five)
## None way the API is not walking or the marketplace is closed :-(
if bookdata is None:
go back (0, bookdata)
## Check the market is liquid; the spread and the dimensions need to be inside sure limits. We use a multiplier due to the fact we are much less discerning with restriction orders - a huge spread ought to paintings in our favour!
Market_liquid=check_is_market_liquid(bookdata, agreement.Code, multiplier=pretrademultiplier)
if now not market_liquid:
go back (0, bookdata)
## If the marketplace is liquid, however perhaps the order is huge compared to the scale on the inside spread, we can reduce it all the way down to fit the order book.
cutctrade=cut_down_trade_to_order_book(bookdata, ctrade, multiplier=pretrademultiplier)
go back (cutctrade, bookdata)
New order
Not just one of the best bands in the eighties, also the routine you call when a new order request is issued by the upstream code.MAX_DELAY=0.03
def EasyAlgo_new_order(order, tws, dbtype, use_orderid, bookdata):
"""
Function easy algo runs on getting a new order
Args:
order - item of my order type containing the required change
tws - connection object to tws API for interactive brokers
dbtype - database we're gaining access to
use_orderid- orderid
bookdata- list containing exceptional bid and offer, and relevant sizes
"""
## The s, country, variable is used to make sure that log messages and diagnostics get stored proper. Don't fear too much approximately this
Log=logger()
diag=diagnostic(dbtype, system="algo", system3=str(order.orderid))
s=state_from_sdict(order.Orderid, diag, log)
## From the order book, and the change, get the price we would pay if aggressive (sideprice) and the charge we pay if we get skipive (offsideprice)
(sideprice, offsideprice)=get_price_sides(bookdata, order.Submit_trade)
if np.Isnan(offsideprice) or offsideprice==0:
log.Warning("No offside / restrict rate in marketplace facts so cannot issue the order")
go back None
if np.Isnan(sideprice) or sideprice==0:
log.Warning("No sideprice in marketplace statistics so dangerous to problem the order")
go back None
## The order item includes the rate recorded on the time the order was generated; test to look if a massive circulate seeing that then (have to be less than a 2nd, so not going until market records corrupt)
if not np.Isnan(order.Submit_price):
delay=abs((offsideprice/order.Submit_price) - 1.0)
if delay>MAX_DELAY:
log.Warning("Large flow considering the fact that submission - no longer buying and selling a limit order on that")
go back None
## We're glad with the order e-book, so set the restriction fee to the 'offside' - first-class provide if selling, pleasant bid if buying
limitprice=offsideprice
## We trade the order so its now a limit order with the proper fee
order.Regulate(lmtPrice = limitprice)
order.Alter(orderType="LMT")
## Need to translate from my item area to the API's native objects
iborder=from_myorder_to_IBorder(order)
settlement=Contract(code=order.Code, contractid=order.Contractid)
ibcontract=make_IB_contract(agreement)
## diagnostic stuff
## its vital to store this so we are able to tune what skiped off if orders go squiffy (a technical term)
s.Update(dict(limit_price=limitprice, offside_price=offsideprice, side_price=sideprice,
message="StartingPassive", Mode="Passive"))
timenow=datetime.Datetime.Now()
## The algo memory table is used to store state information for the algo. Key thing here is the Mode which is PASSIVE initially!
Am=algo_memory_table(dbtype)
am.Update_value(order.Orderid, "Limit", limitprice)
am.Update_value(order.Orderid, "ValidSidePrice", sideprice)
am.Update_value(order.Orderid, "ValidOffSidePrice", offsideprice)
am.Update_value(order.Orderid, "Trade", order.Submit_trade)
am.Update_value(order.Orderid, "Started", date_as_float(timenow))
am.Update_value(order.Orderid, "Mode", "Passive")
am.Update_value(order.Orderid, "LastNotice", date_as_float(timenow))
am.Near()
## Place the order
tws.PlaceOrder(
use_orderid, # orderId,
ibcontract, # contract,
iborder # order
)
## Return the order upstream, so it is able to be saved in databases and so forth. Note if this recurring terminates early it returns a None; so the upstream routine knows no order become located.
Return order
Action on tick
A tick comes from the API when any part of the inside order book is updated (best bid or offer, or relevant size).Within the tws server code I have a routine that keeps marketdata (a list with best bid and offer, and relevant sizes) up to date as ticks arrive, and then calls the relevant routine.
What does this set of functions do?
- If we are in a skipive state (the initial state, remember!)
- ... and more than five minutes has elapsed, change to aggressive
- if buying and the current best bid has moved up from where it started (an adverse price movement), change to aggressive
- if selling, and the current best offer has moved down from where it started (also adverse)
- If there is an unfavourable order imbalance (eg five times as many people selling than buying on the inside spread if we're also selling), change to aggressive.
- If we are in an aggressive state
- ... and more than ten minutes has elapsed, cancel the order.
- if buying and the current best offer has moved up from where it was last (a further adverse price movement), then update our limit to the new best offer (chase the market up).
- if selling and the current best bid has moved down from where it was last (a further adverse price movement), then update our limit to the new best offer
skipivetimelimit=five*60 ## max 5 minutes
totaltimelimit=10*60 ## max another 5 minute competitive
maximbalance=5.0 ## amount of imbalance we can replica with
def EasyAlgo_on_tick(dbtype, orderid, marketdata, tws, agreement):
"""
Function clean algo runs on getting a tick
Args:
dbtype, tws: handles for database and tws API
orderid: the orderid this is associated with a tick
marketdata: summary of the kingdom of cutting-edge internal spread
agreement: what we are certainly trading
"""
## diagnostic code
Log=logger()
diag=diagnostic(dbtype, system="algo", system3=str(int(orderid)))
s=state_from_sdict(orderid, diag, log)
## Pull out the whole thing we currently understand about this order
Am=algo_memory_table(dbtype)
change=am.Read_value(orderid, "Trade")
current_limit=am.Read_value(orderid, "Limit")
Started=am.Read_value(orderid, "Started")
Mode=am.Read_value(orderid, "Mode")
lastsideprice=am.Read_value(orderid, "ValidSidePrice")
lastoffsideprice=am.Read_value(orderid, "ValidOffSidePrice")
LastNotice=am.Read_value(orderid, "LastNotice")
## Can't discover this order in our state database!
If Mode is None or Started is None or current_limit is None or change is None or LastNotice is None:
log.Essential("Can't get algo reminiscence values for orderid %d CANCELLING" % orderid)
FinishOrder(dbtype, orderid, marketdata, tws, settlement)
Started=float_as_date(Started)
LastNotice=float_as_date(LastNotice)
timenow=datetime.Datetime.Now()
## If a purchase, get the pleasant offer (sideprice) and quality bid (offsideprice)
## If a sell, get the great bid (sideprice) and excellent provide (offsideprice)
(sideprice, offsideprice)=get_price_sides(marketdata, alternate)
s.Update(dict(limit_price=current_limit, offside_price=offsideprice, side_price=sideprice,
Mode=Mode))
## Work out how lengthy we've got been trading, and the time since we final 'observed' the time
time_trading=(timenow - Started).Total_seconds()
time_since_last=(timenow - LastNotice).Seconds
## A minute has elapsed due to the fact that we
if time_since_last>60:
s.Replace(dict(message="One minute due to the fact that remaining observed now %s, overall time %d seconds - waiting %d %s %s" % (str(timenow), time_trading, orderid, settlement.Code, contract.Contractid)))
am.Update_value(orderid, "LastNotice", date_as_float(timenow))
## We've run out of time - cancel any closing order
if time_trading>totaltimelimit:
s.Update(dict(message="Out of time cancelling for %d %s %s" % (orderid, contract.Code, settlement.Contractid)))
FinishOrder(dbtype, orderid, marketdata, tws, settlement)
go back -1
if not np.isnan(sideprice) and sideprice<>lastsideprice:
am.Update_value(orderid, "ValidSidePrice", sideprice)
if not np.isnan(offsideprice) and offsideprice<>lastoffsideprice:
am.Update_value(orderid, "ValidOffSidePrice", offsideprice)
am.Near()
if Mode=="Passive":
## Out of time (five minutes) for skipive behaviour: panic
if time_trading>skipivetimelimit:
s.Replace(dict(message="Out of time shifting to competitive for %d %s %s" % (orderid, settlement.Code, settlement.Contractid)))
SwitchToAggresive(dbtype, orderid, marketdata, tws, agreement, change)
go back -1
if np.Isnan(offsideprice):
s.Update(dict(message="NAN offside fee in skipive mode - waiting %d %s %s" % (orderid, agreement.Code, agreement.Contractid)))
return -five
if trade>0:
## Buying
if offsideprice>current_limit:
## Since we've put in our restrict the charge has moved up. We are not aggressive
s.Update(dict(message="Adverse fee flow transferring to aggressive for %d %s %s" % (orderid, contract.Code, settlement.Contractid)))
SwitchToAggresive(dbtype, orderid, marketdata, tws, agreement, change)
go back -1
elif trade<0:
## Selling
if offsideprice<current_limit:
## Since we've got installed our restrict the fee has moved down. We are now not aggressive
s.Update(dict(message="Adverse fee flow transferring to aggressive for %d %s %s" % (orderid, contract.Code, settlement.Contractid)))
SwitchToAggresive(dbtype, orderid, marketdata, tws, agreement, change)
go back -1
## Detect Imbalance (bid length/ask length if we are shopping for; ask size/bid size if we're selling)
balancestat=order_imbalance(marketdata, change)
if balancestat>maximbalance:
s.Replace(dict(message="Order e-book imbalance of %f evolved compared to %f, switching to competitive for %d %s %s" %(balancestat , maximbalance, orderid, settlement.Code, agreement.Contractid)))
SwitchToAggresive(dbtype, orderid, marketdata, tws, agreement, change)
go back -1
elif Mode=="Aggressive":
if np.Isnan(sideprice):
s.Replace(dict(message="NAN aspect fee in aggressive mode - waiting %d %s %s" % (orderid, settlement.Code, settlement.Contractid)))
return -five
if trade>0:
## Buying
if sideprice>current_limit:
## Since we have installed our restriction the rate has moved up further. Keep up!
S.Update(dict(message="Adverse rate circulate in aggressive mode for %d %s %s" % (orderid, settlement.Code, agreement.Contractid)))
SwitchToAggresive(dbtype, orderid, marketdata, tws, agreement, change)
go back -1
elif trade<0:
## Selling
if sideprice<current_limit:
## Since we've got put in our restrict the fee has moved down. Keep up!
S.Update(dict(message="Adverse rate circulate in aggressive mode for %d %s %s" % (orderid, settlement.Code, agreement.Contractid)))
SwitchToAggresive(dbtype, orderid, marketdata, tws, agreement, change)
go back -1
elif Mode=="Finished":
## do nothing, nevertheless have tick for a few reason
skip
else:
msg="Mode %s now not known for order %d" % (Mode, orderid)
s.Replace(dict(message=msg))
Log=logger()
log.Crucial(msg)
raise Exception(msg)
s.Update(dict(message="tick no motion %d %s %s" % (orderid, contract.Code, settlement.Contractid)))
diag.Near()
return zero
def SwitchToAggresive(dbtype, orderid, marketdata, tws, agreement, change):
"""
What to do... If we need to eithier change our present day order to an competitive restriction order, or pass an order is already competitive restriction fee
"""
## diagnostics...
Log=logger()
diag=diagnostic(dbtype, system="algo", system3=str(int(orderid)))
s=state_from_sdict(orderid, diag, log)
if tws is None:
log.Information("Switch to competitive did not get a tws... Cannot do anything in orderid %d" % orderid)
go back -1
## Get the final valid side charge (relevant charge if crossing the unfold) as this can be our new limit order
Am=algo_memory_table(dbtype)
sideprice=am.Read_value(orderid, "ValidSidePrice")
ordertable=order_table(dbtype)
order=ordertable.Read_order_for_orderid(orderid)
ordertable.Near()
if np.Isnan(sideprice):
s.Replace(dict(message="To Aggressive: Can't trade restrict for %d as got nan - will try again" % orderid))
go back -1
## updating the order
newlimit=sideprice
order.Regulate(lmtPrice = newlimit)
order.Alter(orderType="LMT")
iborder=from_myorder_to_IBorder(order)
ibcontract=make_IB_contract(agreement)
am.Update_value(order.Orderid, "Limit", newlimit)
am.Update_value(order.Orderid, "Mode", "Aggressive")
am.Near()
# Update the order
tws.PlaceOrder(
orderid, # orderId,
ibcontract, # contract,
iborder # order
)
s.Replace(dict(limit_price=newlimit, side_price=sideprice,
message="NowAggressive", Mode="Aggresive"))
return zero
def FinishOrder(dbtype, orderid, marketdata, tws, settlement):
"""
Algo hasn't worked, shall we cancel this order
""" diag=diagnostic(dbtype, system="algo", system3=str(int(orderid)))
s=state_from_sdict(orderid, diag, log) Log=logger()
if tws is None:
log.Information("Finish order did not get a tws... Can not do something in orderid %d" % orderid)
go back -1
Log=logger()
ordertable=order_table(dbtype)
order=ordertable.Read_order_for_orderid(orderid)
log.Information("Trying to cancel %d because easy algo failure" % orderid)
tws.CancelOrder(int(order.Brokerorderid))
order.Regulate(cancelled=True)
ordertable.Update_order(order)
do_order_completed(dbtype, order)
EasyAlgo_on_complete(dbtype, order, tws)
s.Update(dict(message="NowCancelling", Mode="Finished"))
Am=algo_memory_table(dbtype)
am.Update_value(order.Orderid, "Mode", "Finished")
am.Near()
go back -1
Partial or entire fill
Blimey this has actually worked, we've actually got a fill...def EasyAlgo_on_partial(dbtype, order, tws):
diag=diagnostic(dbtype, system="algo", system3=str(int(order.orderid)))
diag.W(order.Filledtrade, system2="filled")
diag.W(order.Filledprice, system2="fillprice")
return zero
def EasyAlgo_on_complete(dbtype, order_filled, tws):
"""
Function Easy algo runs on final touch of exchange
"""
diag=diagnostic(dbtype, system="algo", system3=str(int(order_filled.orderid)))
diag.W("Finished", system2="Mode")
diag.W(order_filled.Filledtrade, system2="crammed")
diag.W(order_filled.Filledprice, system2="fillprice")
Am=algo_memory_table(dbtype)
am.Update_value(order_filled.Orderid, "Mode", "Finished")
am.Near()
return zero
