How-Tos and Examples ===================== How to code a trading strategy on Blueshift ------------------------------------------- On Blueshift you can use the full power of Python to code your strategy logic. To do that, you follow - roughly - the following steps - Have a clearly defined strategy logic Blueshift will run your strategy as you have coded it. Make sure your strategy logic clearly identifies all scenarios and has an appropriate logical flow. - Identify instruments, input data and variables Clearly identify the assets that you are going to trade, the data that is required to generate trade entry/ exit and any variables you need to track. - Identify the events handlers that suit the strategy Blueshift offers a number of ways to respond to the market with different choice of event handlers. Choose the one that suits your case the best - Initialize your strategy properly Use the :ref:`initialize` to make sure your strategy has a proper starting state. For example define your trading assets, as well as initialise the variables you need to track. You can optionally ``parameterise`` your strategy here. - Write efficient and robust strategy Use the event handlers from step 3 to write down the strategy. Separate the parts of the logic in individual functions (so that it is easy to debug and easy to tweak). Fetch data only once in each of the strategy iterations (instead of in each function where this data is used) and pass on to different functions. Choose the order placing functions correctly, depending on your strategy logic. Also, validate data to check for missing values and stale data. Below are some guidelines on various steps involved in the process of writing an effective strategy. What is the Python support on Blueshift ---------------------------------------- Blueshift support all legal Python code (version 3.6 or higher) subject to a few restrictions Blueshift has a comprehensive collection of `white-listed` modules that you can import as usual. +-----------+-------------------------------------------------------------------------+ |package | use case | +===========+=========================================================================+ |bisect |An useful array sorting package. | +-----------+-------------------------------------------------------------------------+ |cmath |Provides access to mathematical functions for complex numbers. | +-----------+-------------------------------------------------------------------------+ |cvxopt |Package for convex optimization. | +-----------+-------------------------------------------------------------------------+ |cvxpy |A "nice and disciplined" interface to cvxopt. | +-----------+-------------------------------------------------------------------------+ |datetime |For manipulating dates and times in both simple and complex ways. | +-----------+-------------------------------------------------------------------------+ |functools |Higher-order functions and operations on callable objects. | +-----------+-------------------------------------------------------------------------+ |hmmlearn |For unsupervised learning and inference of Hidden Markov Models. | +-----------+-------------------------------------------------------------------------+ |hurst |for analysing random walks and evaluating the Hurst exponent. | +-----------+-------------------------------------------------------------------------+ |arch |ARCH and other tools for financial econometrics. | +-----------+-------------------------------------------------------------------------+ |keras |A deep learning API running on top of TensorFlow. | +-----------+-------------------------------------------------------------------------+ |math |Provides access to the mathematical functions defined by the C standard. | +-----------+-------------------------------------------------------------------------+ |numpy |Package for scientific computing with Python. | +-----------+-------------------------------------------------------------------------+ |pandas |High-performance, easy-to-use data structures and data analysis tools. | +-----------+-------------------------------------------------------------------------+ |pykalman |Implements Kalman filter and Kalman smoother in Python. | +-----------+-------------------------------------------------------------------------+ |pytz |Allows accurate and cross platform timezone calculations. | +-----------+-------------------------------------------------------------------------+ |random |Random number generators for various distributions. | +-----------+-------------------------------------------------------------------------+ |scipy |Efficient numerical routines for scientific computing. | +-----------+-------------------------------------------------------------------------+ |sklearn |For machine learning in Python. | +-----------+-------------------------------------------------------------------------+ |statsmodels|For statistics in Python. | +-----------+-------------------------------------------------------------------------+ |talib |For technical analysis in Python. | +-----------+-------------------------------------------------------------------------+ This covers a range of useful modules from technical indicators to advanced machine learning programs. If you attempt to import and use any other packages not listed here, you will get an import error. There are a few other restrictions as listed below. - certain built-in functions (e.g. `type`, `dir` etc.) are restricted on the platform. - identifier (variables, functions etc.) names should not start or end with underscore. - while you can use the ``print`` function in your strategy code, the maximum output is restricted. - async programming and generator functions are not allowed on the platform. Also looping with `while` is banned, use a for loop instead. How to create and use variables -------------------------------- The strategy code is a collection of functions that are called by the Blueshift event loop at appropriate times. You can use the normal Pythonic way to create and use local variables for use within each individual function. For accessing the same variables across functions, we recommend using the :ref:`context` variable. Since this is a Python object, you can add attributes to it to store your variable. Also since this variable is passed in all the :ref:`event callbacks`, you can access this variable and its attributes in all functions. This makes it a superior way to pass around variables across your strategy functions (instead of using, say, global or module-level variables). .. note:: There are certain restrictions on variable names that you can use. Apart from being a legal Python identifier, it also must not start or end with underscore ('_'). In addition, there are some built-in :ref:`attributes` of the context variable and user variable name should not clash with them (else the strategy will crash with errors). See the point on asset fetching below to see an example. How to fetch assets in strategy code ------------------------------------ Use the :ref:`symbol` API function to convert an asset ticker or symbol to an :ref:`asset` object. This object can then be used in any API functions (e.g. to place order or fetch data) that require an `asset` object as an input. To use this function, import it from the ``blueshift.api`` module in your strategy code. .. code-block:: python from blueshift.api import symbol, order_target from blueshift.api import get_datetime def initialize(context): # convert the ticker "TCS" to the TCS asset # we can also shorten it by direct assignment # context.asset = symbol('TCS') asset = symbol('TCS') context.asset = asset def handle_data(context, data): # maintain 1 unit position in TCS stock order_target(context.asset, 1) For more on what symbol to use for an asset, please see :ref:`symbology`. Note, here we are using the `context` object as a store of strategy variables (the asset(s) to trade in this case). We can use the `context` object for all variables our strategy needs to track. Fetching Equity Futures instruments ++++++++++++++++++++++++++++++++++++ Strategy code can fetch futures instruments as either dated or rolling assets. For dated instruments, specify the ticker as ``SYM``, where `SYM` is the underlying symbol. For rolling futures, use ``SYM-I`` for the first futures (near-month) and ``SYM-II`` for the far-month. .. code-block:: python from blueshift.api import symbol def initialize(context): acc_dated_futures = symbol('ACC20210826') acc_first_futures = symbol('ACC-I') acc_second_futures = symbol('ACC-II') .. important:: Rolling futures will always be resolved to dated futures in live trading and the positions will be tracked in terms of the dated futures. For backtesting, positions are tracked in terms of rolling futures. Also placing order with rolling assets may be restricted after a cut-off period each trading day if the underlying broker requires it. The cut-off time is typically 15 minutes before the market close. Fetching Equity Options instruments ++++++++++++++++++++++++++++++++++++ Fetching options instruments are similar to futures. Use the symbology ``SYMTYPE`` to fetch a specific option, where `SYM` is the underlying `` is the expiry date, `TYPE` is the option type (can be either ``CE`` or ``PE`` for call and put respectively) and `STRIKE` is the strike price without any leading or trailing zeros. For rolling options, replace the expiry with expiry identifier. For strikes specified in terms of offset, replace the `STRIKE` part with offset specifications as described in the :ref:`symbology`. .. code-block:: python from blueshift.api import symbol def initialize(context): # Aug 21 call at 1000 strike for ABC asset1 = symbol('ABC20210826CE1000') # near-month ATMF+100 call asset2 = symbol('ABC-ICE+100') # current-week ATMF put symbol('ABC-W0PE-0') How to place orders ------------------- Use the :ref:`ordering functions` for placing orders. The first argument must be an asset object. Blueshift offers a number of ways to place orders, including auto-sizing and targeting orders. We recommend :ref:`target order` functions for placing orders from a strategy. The family of targeting order functions work by checking the current positions and outstanding orders for the asset at the time of placing order, and place orders for incremental amounts, if any, to achieve the specified target. The target can be in terms of units, or percent of the current portfolio value, or total value of the required position in the specified asset. An example is given below. .. code-block:: python from blueshift.api import symbol, order_target def initialize(context): context.asset = symbol('TCS') def handle_data(context, data): # maintain 1 unit position in TCS stock order_target(context.asset, 10) In the above example, the :ref:`handle_data` function is called every minute, which in turn calls the `order_target` API function. The first time this function is called, a new order for 10 stocks of TCS is placed. The next time this function is called (and in any subsequent calls), the target, i.e. 10 stocks of TCS in our algo positions is already achieved. So the incremental quantity required is 0, and hence no further orders are sent out to the broker anymore, as long as the target position is maintained. This works very differently if we did not use a target function. For example, if we used simply the basic :ref:`order` function, for each call (initial or subsequent) a fresh order of 10 units will be sent to the broker. Let's look at another example. .. code-block:: python from blueshift.api import symbol, order_target_value, schedule_function from blueshift.api import date_rules, time_rules def initialize(context): context.asset = symbol('TCS') schedule_function(rebalance, date_rules.every_day(), time_rules.market_close(hours=2, minutes=30)) def rebalance(context, data): # maintain INR 10,000 position in TCS stock order_target_value(context.asset, 10000) In this example, the rebalance function is called everyday, 2.5 hours before the market close. For the first time, this will place an order worth INR (broker currency) 10,000 of TCS shares. In subsequent calls, if the market price of TCS shares remain unchanged, no further orders will be sent. If the prices go down, the positions will fall below 10,000 and to maintain the target, algo will send buy orders to achieve 10,000 in value. If the prices go up and the opposite will happen (sell orders). Order targeting in terms of portfolio percent works similarly, but to safeguard against market movement and order failing due to lack in buying power, a haircut (usually 2% of the current portfolio value) is applied before calculating the required quantities. For example, if the portfolio value is $10,000 and an order target percent of 0.25 is specified, the computed target value will be $10,000 (`portfolio value`) X 0.98 (`haircut`) X 0.25 (`target`) or $2450. From there it will follow the order target value behaviour as above. Note, a value or percent target does not guarantee the value of the resulting positions or the execution price. .. important:: We recommend using targeting functions for placing order, unless there is a strong reason not to. This reduces the chance of an order machine-gunning (sending the same order many times over, due to bugs in user strategy logic). .. important:: If rolling assets are used in strategy, they are treated in the following ways: in backtest, all order functions accept dated assets. Rolling assets are not accepted for targetting order functions, except for futures. The order object will always have the dated asset. The resulting position may be rolling (for futures) or dated (otherwise), the former is for convenience. In live mode, usually all rolling symbol specification will result in dated asset, and hence asset returned by the `symbol` function can be freely used in order functions, targetting or otherwise. In such cases, both the order objects and the resulting positions will have the same (dated) assets. Additionally, specifying `product_type` for orders may also result in a different asset for orders and positions than the input asset specified. For equities, for example, `product_type` margin may create a different asset for the orders and position (:ref:`EquityMargin`, than the input asset used in the ordering function (which may simply be of `Equity` type). It is important to keep this in mind, when fetching and checking order assets and comparing them with position assets. You can use the API method `get_asset_from_order` to determine the asset to track in positions for a given (valid) order ID. How to fetch price data for signal generation ---------------------------------------------- Use the :ref:`data` object for fetching historical or current data points. See examples below. .. code-block:: python from blueshift.api import symbol def initialize(context): context.assets = [symbol('TCS'), symbol('WIPRO')] def handle_data(context, data): prices = data.history(context.assets, 'close', 100, '1m') for asset in context.assets: sig = generate_signal(prices[asset]) def generate_signal(price): sig = 0 # apply your data analysis logic here # note, in this case price is a pandas series with the # closing price of the asset return sig Note, although we are using price data for each asset in the `generate_signal` function (a custom function we created), the data query is done in one place, and for all assets together. Also, since we are using only the 'close' price, we queried only for that field. This is an efficient way to query data (instead of calling `data.history` for each asset separately inside the `for` loop or inside the `generate_signal` function). For more on how to query data see :ref:`data.current` and :ref:`data.history`. .. warning:: Note, the `current` and the `history` method returns different types of objects based on the types of the input arguments. The returned object type is the simplest possible, depending on the number of assets, number of fields queried and whether we asked for current or historical data. see the function documents for the expected returned data type. How to write strategy code -------------------------- Blueshift is an event driven engine. Use the :ref:`event callbacks` to write your strategy logic. Your strategy should always include the :ref:`initialize` function (otherwise it is **NOT** a valid Blueshift strategy). Based on your underlying trading logic, you have a number of options to arrange your strategy flow. Below are some examples, that assume we have a signal function as below that checks the asset prices and determines if a trade to be initiated or not. .. code-block:: python import talib as ta # import the ta-lib for RSI calculation def signal_func(asset, price): # TODO: enter your trading logic here. The `price` # parameter is assumed to be a pandas series with closing # price for the assets at 1 minute candles. Below example # shows a simple RSI based entry logic and assume the asset # is shortable - i.e. margin equities or F&Os rsi = ta.RSI(price, 14) if rsi < 30: return 1 # buy signal elif rsi > 70: return -1 # sell signal else: return 0 # neutral signal In the above example, the signal function evaluates a simple RSI based entry condition. .. danger:: Note the above signal function does not trigger only on cross-over but for the entire duration the condition is true. For example, it will trigger a buy signal as long as RSI<30, not just the first time it crosses below 30. If you trigger a basic :ref:`order`, it will generate a fresh order each time the signal function is evaluated (not just when the cross-over happens). The appropriate order function in this case is :ref:`targeting functions` Alternatively, you can modify the above function to trigger only on cross-over (by remembering the last RSI value in the strategy code, for e.g. storing it as a context variable attribute). Strategy that trades periodically ++++++++++++++++++++++++++++++++++ Strategies that run (check trading signal and enter a position) on a periodic basis are best handled by the :ref:`schedule_function`. Assume our strategy checks for entry/ exit every 5 minutes. We can code that as shown below .. code-block:: python import talib as ta from blueshift.api import symbol, schedule_function from blueshift.api import date_rules, time_rules def initialize(context): context.freq = 5 context.quantity = 1 context.assets = [symbol('TCS'), symbol('WIPRO')] schedule_function(rebalance, date_rules.every_day(), time_rules.every_nth_minute(context.freq)) def rebalance(context, data): prices = data.history(context.assets, 'close', 50, '1m') for asset in context.assets: price = prices[asset] signal = signal_func(asset, price) order_target(asset, signal) def signal_func(asset, price): rsi = ta.RSI(price, 14) if rsi < 30: return 1 # buy signal elif rsi > 70: return -1 # sell signal else: return 0 # neutral signal Note that we have initialised the strategy in the `initialize` function that defines the stocks we want to trade and also the trade frequency and the trade size. Secondly, we have split the logic in functions - for this simple case, only two (`rebalance` and `signal_func`). Finally, we are querying data efficiently, only once per iteration (per trade frequency) and fetching data for all assets at one go. Strategy that trades conditionally ++++++++++++++++++++++++++++++++++ Sometimes, we may have to enter or exit based on condition or state of the algo. We tweak the above RSI strategy for this example: we still use the same signal function, but want to enter once (and hold), and only in one stock (whichever triggers the RSI condition first). We can code this strategy as follows. .. code-block:: python import talib as ta from blueshift.api import symbol, schedule_once, schedule_later from blueshift.api import date_rules, time_rules def initialize(context): context.freq = 5 context.quantity = 1 context.traded = False context.assets = [symbol('TCS'), symbol('WIPRO')] schedule_once(rebalance) def rebalance(context, data): if context.traded: # do nothing if already traded return # not traded, check for RSI signal prices = data.history(context.assets, 'close', 50, '1m') for asset in context.assets: price = prices[asset] signal = signal_func(asset, price) if signal !=0: # if an entry signal, place the order and mark # traded, break out of the for loop order_target(asset, signal) context.traded = True break if not context.traded: # if not traded, schedule itself again to run in 5 minutes schedule_later(rebalance, context.freq) def signal_func(asset, price): rsi = ta.RSI(price, 14) if rsi < 30: return 1 # buy signal elif rsi > 70: return -1 # sell signal else: return 0 # neutral signal Note that in the above example, we use a combination of :ref:`schedule_once` and :ref:`schedule_later` to run the `rebalance` function conditionally. This capability gives a powerful way to express your strategy logic. How to check order status ------------------------- There are roughly two ways to do that. We can either use the :ref:`get_open_orders` to fetch a dict (keyed by order IDs) of all orders currently open (i.e. not completed, cancelled, or rejected). Else, we can use the :ref:`get_order` function to fetch an order by its order ID. Check the `status` attribute of the :ref:`order` object to know its status. See :ref:`OrderStatus` to know how to interpret it. How to check open positions --------------------------- See :ref:`here` for more details and code sample. How to use stoploss and take-profit ------------------------------------ On Blueshift, adding a :ref:`stoploss` or a :ref:`take-profit` target is just a convenience API function that automatically checks the price level at the frequency of the event loop (i.e. one minute). Additionally, it also enforces a cool-off period (typically 30 minutes). Example below shows how to add a stoploss and take-profit to our original RSI strategy above (shows only the relevant part). .. code-block:: python from blueshift.api import set_stoploss, set_takeprofit def rebalance(context, data): prices = data.history(context.assets, 'close', 50, '1m') for asset in context.assets: price = prices[asset] signal = signal_func(asset, price) order_target(asset, signal) set_stoploss(asset, 'PERCENT', 0.01) # stoploss of 1% set_takeprofit(asset, 'PERCENT', 0.01) # stoploss of 1% If we are exiting a position by a signal (i.e. not triggered by a stoploss or a take-profit exit), it is recommended to remove the corresponding stoploss and take-proft targets as well. This makes the algo run more efficiently. .. warning:: The asset to place order and the asset to track the stoploss or take-profit must be consistent. See the caveat under :ref:`placing trades` for more. It is usually safer to query positions and then place stoploss or take-proft on the assets from the positions dictionary. How to parameterise my strategy ------------------------------- On Blueshift, you can parameterise your strategy so that you can launch backtests or live executions with dynamic input at the time of launch (instead of hard-coding them in the strategy). This is done in two steps. First make sure you have put all your parameters in a dictionary named `params` and have set it as an attribute of the `context` variable in the `initialize` function. Just after that, call the :ref:`set_algo_parameters` function to set the `params` dictionary as your parameters definition for the strategy. Use appropriate default values for your parameters while defining the dictionary. .. code-block:: python from blueshift.api import set_algo_parameters def initialize(context): # strategy parameters context.params = {'my_param1':0, 'my_param2':42} set_algo_parameters('params') The `set_algo_parameters` API call binds the context attribute `params` as strategy parameters. You can use any other variable name as well, but `params` is recommended for easier tracking and understanding. In the second step, define your parameters while creating the strategy on the plaform. Once both are done successfully, you will get options to select parameters while launching your strategy on the platform. Be careful to exactly match the name of your parameters while creating your strategy parameters (else the default values will be seen by the strategy). .. important:: If you are defining your strategy that accepts parameters as above, it is highly recommended that you add validation in each of the parameter values before using them in the strategy. This is because the parameters entered during launch and passed by the platform to your strategy can potentially be corrupted (mis-format, bad inputs etc.) Good practices to follow for strategy building ----------------------------------------------- Algorithmic trading can be advantageous as machines are faster than humans. But they are faster when things go wrong as well. It is of utmost importance that we take proper steps to make our strategies fault-tolerant. Below is a (non-exhaustive) set of points to keep in mind before you take your strategy live. - Strategy is designed to be fault-tolerant for corrupt input data At the base level, check if the data you have received is very much different from the last datapoints your strategy had. Most exchanges usually follow a market volatility control for publicly traded securities. These limit stock price movements and also enforce cool-off periods in such circumstances. If your algo received some extreme data points, it is highly likely they are wrong. Even if they are true, the market volatility control mechanism probably has already been triggered. If your strategy is not particularly designed to exploit these situations, it is a good practice to pause any trading activity till saner data arrive. - Strategy has necessary risk controls in place This is, again, an absolute must. At the minimum, it should control the max number of orders it can send (to control machine gunning), the max size of each order (machines do fat fingers too) and a kill switch (a percentage loss below which it should stop automatically). Blueshift® has all these features, and then some more. You can put controls based on the maximum position size, maximum leverage or even declare a white-list (black-list) of assets that the algo can (cannot) trade. - Checking/ cancelling pending open order before placing new orders This one is an absolute must for live trading. A best practice is usually to cancel all open orders before placing fresh orders, or, updating the existing orders as per the algo signals. Else it is easy to end up with a machine gun order scenario - the algo firing up and queuing up orders faster than they can be processed. See the code snippet below. We use the `get_open_orders` and `cancel_order` API functions to handle pending orders, **before** placing fresh orders. - Order placing and signal generations are isolated into specific functions A strategy that places orders from multiple functions can mess up really fast. Make sure all your orders are placed only through a specific function. In such a design, chances of unforeseen mis-behaviours are considerably less. - Strategy is not over-sensitive to latency Orders will be sent and processed over regular internet. So any latency sensitive strategies (like cross exchange arbitrage or market-making) are bound to suffer. Also internet connections are prone to interruptions or even complete outages. The strategy should be robust to such scenarios. Things to avoid while writing a strategy ---------------------------------------- - A strategy generating orders at a very high rate These are more prone to instability, and also perhaps lose more in round trip trading costs than they make. A hair-trigger signal generation method may result in such a scenario. So make sure your signal generation is robust and expected holding periods are consistent with points above. - A strategy that triggers orders continuously for the same prevailing condition If you are trading based on some technical indicators, say `RSI`, you usually want to place an order when the indicator crosses a threshold (a change of state). The intention is not to generate orders constantly as long as the indicator stays below (or over) that threshold (a state). If you are using a state-based signal, make sure the order functions are targeting in nature (e.g. the ubiquitous `order_target_percent`), not absolute (e.g. `order`). In case you are using absolute orders, make sure your signal generation is based on change of state, not the state itself - A strategy trading close to the account capacity Margin calls can put an automated strategy out of gear. Always ensure the account is funded adequately, so that the algo runs in an expected way. Risk management and monitoring makes all the difference between blowing out the bank roll and making handsome profit. The above points will take care of some aspects of risk management, especially in automated trading set-up. But bank-roll management, position sizing etc, are still some points that need to be deliberated carefully before taking a strategy live. Also it is absolutely necessary we keep a close watch on the algo performance.