Systems building - execution
People often get systematic and automated trading mixed up. The latter is a subset of the first. You can't have a system which is fully automated if it relies on discretionary input, no matter how small. But you can have a system which needs a human to make it run, even though there is no discretion, and its fully systematic.
The important region in which human beings are frequently used with systematic buying and selling is in execution. It's fairly not unusual for people to create software that analyses charges and produces a change, which the character then has to execute.
(At the large hedge fund I used to work for, we used a mixture of automated execution algos, and real people to execute trades. FWIW the best humans were better than the best algos, on certain kinds of markets)
This post isn't always for you. It's handiest for those who want to automate their execution as nicely, and keep away from the stress of truly trading.
 |
| If you don't need to be like this man, then study the rest of this submit. |
This is the fourth publish in a sequence giving guidelines on the nuts and bolts of constructing systematic buying and selling systems. You may want to look at my publish on constructing a easy execution algo, which I'll check with once in a while here.
Event pushed as opposed to Sample driven
As I've discussed before in this series there are two ways of running a trading system. The first method is event driven; you wait for something to happen (price hitting a particular level, or the next tick arriving) and then you take the appropriate action (buying or selling, or cancelling an existing order perhaps).
The second method, which I use, is sample driven (and there is no agreed nomenclature here, so you may hear it called other things). Here we periodically check / 'sample' the data we are interested in (eg prices), calculate what we want our optimal position to be, and then trade to achieve it.
The former device is maximum ideal to excessive frequency trading, and to rather easy algorithims (when you have to spend a minute analysing the tick occasion, then you lose any gain from walking an occasion pushed machine). It doesn't paintings as properly with, as an example, trading relative price structures.
Anyway this isn't always the vicinity to get into a debate approximately that is better. For my utility (exceptionally gradual trading) sample driven trading is more natural. There are also benefits in that debugging is less difficult, and code may be less complicated.
Much of this put up can be applicable to occasion driven structures, however those running pattern systems will get greater out of it.
Key principles
There are a number of factors, a few at the same time distinctive, to remember when designing a system to do execution. In precise there is a capability struggle among robustness and speed. To be specific, if I'm in a scenario in which the machine may do something probably risky which includes an wrong change, I might as an alternative postpone the exchange until the scenario may be resolved; despite the fact that the put off will be numerous days.
Because I need a totally automated system, that literally ought to be capable of run for months with out being touched or watched (rolls apart), this robustness is noticeably crucial.
The best function
To be clear then before the part of my program that does execution comes into the photo something else has created a theoretically foremost role, which turned into true in some unspecified time in the future in time, and with a few statistics (in practice then, for my simple technical machine, at a few charge).
I'll communicate extra in a later publish approximately whether or not this is a separate manner, or all one big satisfied feature, and how separate methods 'speak' to every other.
Essentially then the task of the execution thing of my device is to examine the top of the line position with the present day role; and if they are out of line, try to do a change that brings the real position back to what we need it to be.
If you want extra element on what I mean with the aid of 'out of line' you may ought to study my approaching e-book, www.Systematictrading.Org, chapter eleven.
What can cross incorrect
This all sounds very simple, however in truth there are a number of things which could go incorrect:
- Optimal position is unavailable
- The price on which the optimal is predicated is out of date
- We don't know what our current position is
- Our current position is out of date, as we've recently done a trade that will change it
- We're running two processes and they will both issue the same order, ending up with the position 'overshooting' the optimal
- The market is closed
- The market isn't liquid enough to do the adjusting trade required
- We might, for some reason, not want to do the adjusting trade
Writing exhaustive lists of 'what can pass incorrect' like this is essential before you write a sturdy system. We need to ensure all our tactics deal with those issues. Also my bias in the direction of simply computerized structures that run themselves, on which a delay in trading can be shrugged off, affects how these troubles are solved.
Getting the contemporary position: Storage and lining up
I won't discuss where the optimal position is stored, as this will be the subject of a future post. Just for now imagine there is a database table populated with it (as indeed is the case in my own system). However just to note that if we can't read the optimal position then we probably have a fairly broken database (or perhaps a file lock), and we shouldn't try and do any trading.
There are two main places you can store your current position:
- Your own database
- Your brokers database
* The NASA way of avoiding conflicts of this kind might be to have three sources of modern position and institute some form of voting set of rules. Sadly I can't think of a further, independent, source of position.
** References to my broker IB (Interactive Brokers), can truly seek advice from different brokers as nicely.
The solution I use is to make my database the primary source of current position. I then crosscheck this 3 times a day with the position the broker thinks I have. If there is a difference of opinion between me and the broker then I change the trading state to STOP (see more below about different states). Genuine differences of opinion are rare, and more often than not caused by a position cash settling or being forced to close by IB when for some reason I haven't closed it myself before expiry (discussed more here).
I also avoid creating new orders when I think, or understand, I actually have an present order (of which more beneath).
Lessons from the database managers
Let's take some of these problems I mentioned above and rephrase them barely:
- Our current record is out of date, as we've recently done a transaction that will change it
- We're running two processes and they will both issue the same transaction, ending up with the record being incorrect.
These problems then are exactly those faced by people who create database systems. The solution that they, and anyone involved in worrying about files being updated by multiple users / processes, is locking .
 |
| If I google (pix) 'database' this is the primary thing that comes up. It would be a lot cooler if databases truely got here with holograms of pieces of paper flying inside and outside of your server. |
I use locking in the following way. Locks are carried out at an device level (eg to the whole of the Gold destiny, no matter which contract we're buying and selling). The regular status of an device is unlocked. This approach we will problem a new order associated with that tool. Once an order is issued but the instrument turns into locked.
In a locked reputation no new orders may be issued for an device. Because I am notable paranoid I'll additionally take a look at with the IB API that there are no exquisite orders; even though the device is unlocked if the API reviews the existence of an incredible order (which should not show up... However...) approach I can not do any more.
The lock will handiest be eliminated as soon as the order is completely filled and the modern function up to date to mirror that, or the order is cancelled via my software (of which greater later). Because I can omit fills, and because of other gremlins, I actually have a periodic (hourly) manner which checks to peer if there are any units which might be locked, however with no first-rate orders pronounced by using the IB API. If there are, then they're automatically unlocked.
This may additionally look like paranoia. Why now not just take a look at for oustanding orders from the IB API, and don't alternate in the event that they exist? Because there may be a tiny period of time where IB will say 'yeah, no great orders' but the applicable fill hasn't but arrived (or the fill may be neglected somehow) and my modern-day function hasn't been updated, meaning I might try to trouble the identical change once more. By periodically casting off locks that apparently are not wanted I do open myself as much as this taking place, however it is much much less probably than if I used the presence of IB tremendous orders as my handiest take a look at to look if it's far secure to exchange.
Enemy of the kingdom
I won't always want to trade an instrument, or do certain types of trade. For this reason I have a number of trading states which an instrument can be in:
- OK - allow all kinds of trades
- NOOPEN - do not open any new trades, but allow any existing orders to continue.
- NOINCREASE - do not do any trades that will increase our position, but allow closing trades. I use this a lot; for example when I recently wanted to get out of my German 2 year bond position.
- NOCLOSE - do not do any trades that will reduce our position. It's hard to think of a situation where this would make sense, but I hate asymmetry.
- STOP - do not open any new trades, and cancel any existing orders. Once existing orders are cancelled, this migrates to 'NOOPEN'
- CLOSE - immediately try and issue a trade to close the entire position (but which will respect trading limits; see below).
Locks and states are special; locks are carried out and can be eliminated by using software program. The state can be changed via software (including while opinions on cutting-edge positions vary); but can simplest be reset to 'adequate' by using human fingers.
Rolling rolling rolling...
Applicable to future traders best (or unfold bets, alternatives... Others where positions can't be held indefinitely and want to be 'rolled over' to the following maturity). You must have already examine my submit on the mechanics of rolling. Suffice to say that we may be:
- Not rolling, in which case we'll buy and sell the current contract
- Passively rolling, when we'll issue reducing orders in the current contract, and increasing orders in the new one
- Actively rolling with a spread trade, when we'll issue a single order to (if long) sell the current contract and buy the new one
- Actively rolling with an individual leg trade, when we'll issue two orders to close the current contract, and open a position in the new one
The crucial factor right here is that the code first wishes to discover the roll state, work out which aggregate of orders makes experience, and then wishes to have the good judgment to issue one or two orders inside the modern-day and/or new settlement; or a spread order.
Open all hours
There isn't any factor trying to exchange if no person wants to alternate with you. There are 5 motives I can think of why you won't be capable of alternate:
- The market is closed today (a regular closure, depending on the day)
- The market isn't open right now (a regular closure, depending on the hour).
- The market is unexpectedly closed.
- There isn't enough liquidity in the market to trade
- You have been banned from the market, or blown up and lost all your money; or upset your broker.
 |
| Don't reproduction this guy in case you don't want to be banned from the market |
Leaving apart the ultimate factor; for the primary two points we will of path download and create tables storing market holidays and market opening instances. Beware this is lots of work; and you need to then waste your existence stressful about daytime savings adjustments. This would not cope with factor three of direction.
For point four you are going to have to test the liquidity earlier than you change. I'll speak underneath precisely what a liquidity test would possibly involve.
I pretty an awful lot pick the laziest possible option; that is to deal with the primary four points all in the same way as a loss of liquidity. What I do in exercise then is have a desk of when I recognise markets must be open and liquid(ish). During the times laid out in that table if I feel the want to exchange I will periodically check to peer if that marketplace is liquid. If it is, then I go ahead and trade (if trading is wanted). If no longer, then I don't. Not liquid ought to imply the market is on excursion today, or has closed early for a few motive, or it would just now not be liquid.
Note that if I was even lazier, and simply checked all markets 'all'* the time, then I'd be frequently checking a number of closed markets that is a waste of processing energy and bandwidth.
* My execution process runs from 10pm to 8pm the next day, local UK time. It is kicked off Sunday evening, Monday evening .... Thursday evening. The two hours of down time is used to run reports and create backups.
Liquid liquid everywhere and no longer a drop to drink
I said above we want to test for liquidity. There are many ways to measure liquidity of direction. In my easy world, wherein I most effective have level 1 records (and L2 anyhow might be corrupted by using spoofers), I use the subsequent easy take a look at:
- Is the inside bid-ask spread in the market smaller than X (where X is some value I've set for each market, which depends on how I'm executing the order)?
- Is the size available at the bid and ask larger than the trade I want to do?
Note that if the market is closed then the values for bid-ask unfold and size can be NAN and NAN respectively, so with the aid of definition there isn't enough liquidity.
I'll talk in a 2d how the method of execution affects the specified maximum spread. Just to observe that if the size to be had is smaller than the exchange I need to do (however is greater than zero), and I want to do more than one settlement, the execution code will lessen the size of the exchange so its viable given the available size.
There is a further check I will do at this stage. Remember from above the theoretically optimal position, which was true at some point in time, and with ... some price. So I need to have a concept of staleness, and a concept of maximum price movement.
To put it another way, given the speed of my system, I would be concerned if I'd calculated my optimal position based on a price from more than a hour ago. By the way this will happen if you're running seperate optimal position calculation and trading processes as I am; and the latter kicks off in the morning before the former has had a chance to update the price. It won't happen if you have a single pipeline process 'collect price -> calculate optimal position -> create order', or if you're using event driven systems. Again this kind of stuff will be the subject of another post.
Secondly I'd be involved if the rate had moved given that I checked it. If for example a market had moved more than four daily trendy deviations in charge inside the hour or much less in view that I ultimate grabbed a price, then it is quite likely my most suitable function is out of date, and I will need to recalculate it before trading. Otherwise the hazard is I might buy based totally at the old charge, and then quickly promote once more, once a new price has percolated via the machine.
The messy commercial enterprise of really executing
When you simply execute you can:
- Issue a market order, that will most probably cross the spread
- Use your own algo, as I do (much, much more is here).
- Use a brokers algo. IB have a good set of algos for equities, fewer for futures.
- Use a third party algo. Unfortunately the good ones are only available to institutional traders. I can recommend QB for interest rate products.
I will usually use my personal algo, although for manual trades (see later) I deliver myself the option of the use of a market order. Eventually I'd assume to have a suite of algos for distinctive devices; so I have the ability to do that equipped in my database, although I don't but use it. Algo evaluation is a technology; just like comparing different buying and selling systems it has very noisy statistics and again most effective massive institutions can realistically collect enough information to make significant comparisions (and possibly no longer even then).
As I alluded to above I even have distinctive liquidity requirements depending on whether I am issuing a market order or the use of my algo. If I'm issuing a market order I want the unfold to be tight (normally a unmarried tick, or whatever the minimal increment is, until the marketplace commonly trades a bit wider), due to the fact I understand for positive I'll be paying half of of it. If I'm the usage of my algo I am less choosy; a miles large unfold would possibly imply I seize extra of it after I submit my initial lot on the bid (if I'm shopping for) or provide (selling). This is genuine to a degree - very big spreads suggest the market might be dangerously thin and I wouldn't try and issue an order.
Safety first
The scariest element approximately fully automated trading is the doomsday situation; what if this issue simply goes terrible and trades like loopy?
 |
| What my automatic system may say if I tried to forestall it trading. Although, simply to be clear, my call isn't always Dave. |
Although I've been ultra careful there is still an outside chance something could go wrong. To avoid this I use daily, and 'lifetime' trade limits.
So before I simply issue an order the software program will take a look at to peer how many contracts I'd achieved these days already, and whether or not I will smash that cumulative restriction. If so it will reduce the trade (sometimes all the way down to zero). I additionally have 'lifetime' limits. In exercise if I'm going on excursion I will set the lifetime limits to a reasonably huge degree of buying and selling I'd expect to do over that duration (truely much less than N x day by day limit, wherein N is the quantity of days I'm away, otherwise they might not bind), after which once I get returned I'll reset them to big numbers.
Limits are set by using looking at values in my backtest, and so are contract established.
In my modern setup lively roll trades (as spreads or legs) are exempt from exchange limits, even though as soon as done they be counted closer to the limit (frequently which means you can't do any greater trading that day after a roll). It might probable be better to be counted rolls seperately towards some other, larger, limit. That is on my, very long, to-do listing.
Note that IB API also has limits, but they are not instrument dependent. Unless you're trading only stocks, or just one future, this makes them pretty useless. So I turn them off.
Correcting errors
Stuff takes place. And whilst it takes place you need to restoration the mess it creates.
The first form of stuff, which I've already mentioned, is where my role and the IB API function get out of line. Normally what has occurred here is that I issued an order which turned into stuffed, and I neglected seeing the fill; or IB closed out or settled a function which I hadn't got around to rolling in time.
To deal with this my software has the ability to do a manual fill or a balancing order. The former is where I update an existing, unfilled, order in my database to include the fill that actually happened. The latter is where I create an entirely new order in the database to reflect the trade that happened. After I've done this I should be in a situation where my, and my brokers, record of position are perfectly matched.
The other kind of stuff is sometimes I might want to trade outside of the normal execution process. For example suppose I have a contract I have to roll today. I've tried passive rolling, issuing spread orders, and even individual leg orders. None of these have filled, so what I will do is issue a manual trade from the command line, and then manually override the normal execution method to do a market order. Unlike a manual fill or a balancing order a manual trade will be sent to IB for execution, as well as going in the database.
Done
Hurray. Instead of being tied in your pc, following its robot commands to execute trades, you could now do something else as an alternative.
In the next publish I'll look at working out how a lot lovable moolah you are making.
This is the fourth submit in a series on constructing systematic trading structures.
The first three posts are:
http://qoppac.Blogspot.Co.United kingdom/2015/04/gadget-building-records-seize.Html
http://qoppac.Blogspot.Co.United kingdom/2015/05/structures-building-futures-rolling.Html
http://qoppac.Blogspot.Co.United kingdom/2015/06/structures-constructing-figuring out-positions.Html
The next put up is right here:
http://qoppac.Blogspot.Co.United kingdom/2015/08/structures-building-accounting.Html
