Cointegration is used in Statistical Arbitrage to find best Pair of Stocks (Pair Trading) to go long in one stock and short(Competitive peers) another to generate returns. Statistical Arbitrage(StatArb) is all about mean reversion, looking for deviation in the spreads and expecting mean reversion from the spread.
So Whats the Problem with Correlation?
Often people use correlation in pair trading to identify high correlated pairs and then expect mean reversion from the spreads. But look into the following example where X and Y are the random time-series data diverging and both moving in the same direction and highly correlated. But do you think we can do a pair trading on top of it where there is no mean reversion among the spreads?
[wp_ad_camp_5]
What is Co-Integration
Co-Integration helps in identifying best stock pairs where the spread could revert to mean value. Co-Integration looks for stationary pair where the mean of the spread is fixed. Whenever the spread is deviating from the mean it generates trading opportunity and the spread will possibly revert back to the mean value.
Let me explain with a funny example which explains Co-Integration in a better way. “A drunken man is walking on the road along with his dog chained and tied up with the drunkard’s hand. When the man is drunk and he is expected to walk random and the chained dog is also expected to walk random(assume a small little puppy 🙂 ). The maximum distance between them could be the length of rope holding the chained dog and it is always fixed. Whenever the distance/spread between the Drunken Man and the Dog goes near to the max distance we can expect a mean reversion in the distance to the mean” In simple words the drunken man and the dog both are Co-Integrated.
If two stocks are highly correlated then both the stocks will move in the same direction most of the time however the magnitude of the moves is unknown and spread can keep increasing as long as it could as shown in the above example. However Co-Integration looks for mean reversion in the spread/distance and the spreads are tradeable. Augmented Dicky Fuller test is generally used to identify with a certain level of confidence whether the spread between two stocks or time series is stationary and cointegrated or not.
Augmented Dickey-Fuller (ADF) Test
The Augmented Dicky Fuller test is a hypothesis test that a signal contains a unit root,we want to reject this hypothesis. The test gives a pValue, the lower this number the more confident we can be that we have found a stationary signal. pValues less than 0.5 are considered to be good mean reverting stock pairs. Some of the experts even look for values pValues less than 0.1. pValues above 0.1 are likely to be non statinary and trading such stock pairs are not advisable.
Above image shows Cointegration and Correlation Dashboard between Sun Pharma and Cipla Futures since Dec 2014 to till date which shows the P-Value is 0.05 (Highly Co-Integrated) and also highly Correlated (0.834) and possibly a best pair to look for long term mean reversion in the spread.
The second example showing Infy and TCS Hourly Future charts with High Co-Integration (0.05) and High Correlation (0.843) and possibly a best pair to look for short term mean reversion in the spread.
Computing Co-Integration in Amibroker
Since Co-Integration is a statistical model it is relatively difficult to code in AFL Programming Language we rely on Amibroker with Python COM Server and statistical computing python packages like numpy(to handle arrays), Pandas(to handle time series data) and statsmodels(to do ADF test) where the close arrays of two stock pair are passed from Amibroker and the CoIntegration is computed by python and revert back to Amibroker.
If you are not sure how to install and configure python and its statistical packages like numpy, statsmodels, pandas then go thro the video tutorial here which explains how to install python library zipline – a backtesting package right from the scratch
Steps to follow in Amibroker
1)Download CoIntegration-AFL Set and Unzip it
2)Copy the file coint.py to \\python2.7\\bin\\ folder. And execute the file with the command python coint.py in your command prompt as shown below
3)Copy the PyCoint.afl file and paste the file in \\Amibroker\\Formulas\\Basic Charts Folder
4)Open a New Blank Chart and apply PyCoint.afl to it. Now right click over the charts and goto parameters and enter the two symbols for which you want to compute Co-Integration. You should be able to see Correlation(22 Period) and Co-Integration values displayed in a dashboard which you can use for your further Statistical Arbitrage(Pair Trading) analysis.
Note : Co-Integration is calculated based on the visible data displayed data in the charts. If you do zoom/Unzoom or Changing the timeframe you may end up with Co-Integration value calculated only for the visually displayed price on the screen.
Nice article. Is it possible to create simple explorer in AmiBroker to check co-integration between a lot of stocks?
I tried but amibroker is crashing with unhandled exception here is the code https://gist.github.com/marketcalls/3ea6f74b51d1bdb1ee5c
However using seaborn python package we can plot externally here is the sample implementation code from quantopian https://www.quantopian.com/posts/update-pairs-trading-notebook-now-with-cloning
I’ve installed libraries
– numpy-MKL-1.8.2.win-amd64-py2.7.exe
– statsmodels-0.7.0-4b55fa4.win-amd64-py2.7.exe
– scipy-0.14.0.win-amd64-py2.7.exe
After running “C:\Python27\bin>python coint.py” in command line I get:
”
Traceback (most recent call last):
File “coint.py”, line 8, in
import statsmodels.api as stat
File “C:\Python27\lib\site-packages\statsmodels\__init__.py”, line 8, in
from .tools.sm_exceptions import (ConvergenceWarning, CacheWriteWarning,
File “C:\Python27\lib\site-packages\statsmodels\tools\__init__.py”, line 1, in
from .tools import add_constant, categorical
File “C:\Python27\lib\site-packages\statsmodels\tools\tools.py”, line 9, in
from statsmodels.datasets import webuse
File “C:\Python27\lib\site-packages\statsmodels\datasets\__init__.py”, line 5, in
from . import (anes96, cancer, committee, ccard, copper, cpunish, elnino,
File “C:\Python27\lib\site-packages\statsmodels\datasets\anes96\__init__.py”, line 1, in
from .data import *
File “C:\Python27\lib\site-packages\statsmodels\datasets\anes96\data.py”, line 90, in
from statsmodels.datasets import utils as du
File “C:\Python27\lib\site-packages\statsmodels\datasets\utils.py”, line 14, in
from pandas import read_csv, DataFrame, Index
ImportError: No module named pandas
”
Should be installed pandas library also?
Yes pandas too required. It is a dependency library.
Thanks. I’ve managed to successfully installed packages. It’s the total list required:
– numpy-MKL-1.8.2.win-amd64-py2.7.exe
– statsmodels-0.7.0-4b55fa4.win-amd64-py2.7.exe
– pandas-0.13.0.win-amd64-py2.7.exe
– patsy-0.2.1-1_py27.exe
– python-dateutil-1.5.win-amd64-py2.7.exe
– scipy-0.14.0.win-amd64-py2.7.exe
After running C:\Python27\bin>python coint.py
I got:
Registering COM Server
Registered: PyCoint
and now it works 🙂
Glad it worked for you! Yeh python pandas has too much dependencies. Thanks for posting the list of libraries.
I tried using pycoint . My amibroker 5.4 hangs when i m trying to load coint charts . i m getting “ERROR: Formula not found – no chart to display”
Some of the Gfx functions used are not compatible with 5.4. IT is recommended to try above 5.7 and above and it is always recommended to install python, libraries and then compile the Cointegration code and then apply the charts in amibroker.
Rajendra, issues is not with python installation. As python com server is running proeprly.. I even tried with iir.py . That was also working fine.. it seems some issue with pycoint.afl only.. thanks for pointers.. will try to debug afl .. Can u suggest any way to get logs from com to ami, so that debug can be done easily?
It’s not working
what issue you are facing?
Very Good article.. Feel finding pair trades should be easier. Once configured and used Pair trade finder was very easy to use. Software is free bu need to have a dollar account with them. Not if we can do pair trade in foreign exchanges.
Thanks that motivates me a lot! Now doing little more with python!
Sir
Is it not possible to make this afl without python ?
Nope coz such complex financial model packages are not inbuilt in Amibroker and so we are doing the computation externally in python.
How do you debug Python code?
Could you also plot non AmiBroker charts? For example by matplotlib importing.
Will provide in the next update working on Python based Co-Integration Map for Multiple Scripts.
Great!
Sir,
I Cant Find the exact link for the below Libraries (Download Link)
— numpy-MKL-1.8.2.win-amd64-py2.7.exe
— statsmodels-0.7.0-4b55fa4.win-amd64-py2.7.exe
— pandas-0.13.0.win-amd64-py2.7.exe
— patsy-0.2.1-1_py27.exe
— python-dateutil-1.5.win-amd64-py2.7.exe
— scipy-0.14.0.win-amd64-py2.7.exe
Pls Help me out in this
you dont need to find the links to download. Simply use pip installer commands to install these packages.
Dear Rajandran,
Do you cover this topic in your TRAINING PACK?
Have you planned any training sessions in the coming days.
I know there is seminar but find it difficult to travel.
Yes it will be covered. more details on the amibroker afl workshop agenda visit here
This AFL is very very useful to find proper trades, Thank you so much
Rajandran,
i am not able to install numpy as u shown in the video, all i have is .whl files but cant install it as u done in that video like windows installer pls help
i also tried by command line its showing error ” unable to find vcvarsall.bat”
i googled it i got a solution to install vcc libraries but error still same
pls help
Thank you for this solution…
I’ve tried to install on Win7x64 and on Win XP and I can say that installation of all those python packages is lil bit tricky..so it would be great to collect all packages in one archive file and share it.
Both times i finally have COM server launched but… when i apply AFL code on the chart it stopped off with unhandled error.
Win7 x64 – shows COleException error, then Amibroker crashed and no responding at all
Win XP – also shows COleException error, but when i close error message window, Amibroker continue to work
Make sure you are using latest version of Amibroker
atleast 5.7 or above
Hi,
Followed steps in blog, got below message. However getting COM/Object is null in Amibroker 5.7. Pls help.
Registering COM Server
Registered: PyCoint
Hi,
Nice Post!
I followed all the steps mentioned in post and I am getting the below mentioned error in Amibroker.
COM/object handle is null Formulas\Basic Charts\Py2Ami.afl
Can you please help me to resolve this.
Not working with Python 3.6
Dear, Mr.Rajandran.
Thank you for your article. I am very interested in this topic.
Unfortunately, I don’t know how to handle Python. But I did everything as you wrote. Version 2.7 could not be installed because it is no longer supported. But I managed to install version 3.8 and install all the libraries. Unfortunately, at the stage when you enter in the command line coint.py -it gives an error. File “C:\Python3\lib\site.py”, line 178 file=sys. stderr) SyntaxError: invalid syntax Please help me fix the error and run the script. What do I need to do?! Thank you in advance for your prompt response/