I want to backtest a large number of exchange pairs and strategies at once with Python's backtesting.py
wrap up
I want to do FX backtesting in Python! backtesting.py seems to be easy to use! But one exchange pair and one trading strategy: only strategies can be backtested ... I want to backtest a large number of exchange pairs and strategies at once .... Let's make your own class! / * This is my first post, so please forgive me in various ways * /
Module used
numpy
pandas
seaborn
matplotlib
talib
datetime
pytz
mplfinance
dask
tqdm
backtesting
read_hst
Please install the above module as pip with conda etc. By the way, the contents of read_hst are below.
Preparation of historical data
Please prepare by referring to the URL below. I save it in I: \ FX \ Alpari-Demo on the external HDD.
How to download historical data from Alpari https://www.hungarianbest.com/mt4backtest/alpari_download/
What you can do
Putting .hst into a DataFrame like this ...
Read the spread from the csv file ...
Run……
Check the result with .result ()
Visualize the results with .result_plot () ...
Use .all_bt_plot () to run backtesting.py's .plot () all the way ...
Satisfied because I was able to do what I wanted to do (≧ ▽ ≦)
Contents of the created .ipynb
I uploaded it to the URL below, so if you want it, please (≧ ▽ ≦) …… Or rather, the explanation is troublesome, so please check the source code and understand it (´ ･ ω ･ `) Well, I think the metamorphosis that I want to backtest with Python is about me ... https://ux.getuploader.com/hage_fx/download/50
Contents of read_hst.py
read_hst.py
#!/usr/bin/env python
# -*- coding: utf-8 -*-
import argparse
import zipfile
import pandas as pd
import os
import numpy as np
def zip2hst(fullpath):
"""Extract zip file.
Usage:
zip2hst('~/Data/USDJPY.zip')
> ~/Data/USDJPY.hst
zip2hst('USDJPY.zip')
> USDJPY.hst
args:
fullpath: Zip filename or path
return:
Extract filename or path
"""
if zipfile.is_zipfile(fullpath):
with zipfile.ZipFile(fullpath, 'r') as zf:
zf.extractall() #Zip unzip
ziplist = zf.namelist()
if not len(ziplist) == 1:
print('There are {} files in zipfile. Try again.'.format(len(ziplist)))
raise IOError
hstfile = ziplist[0]
return hstfile #Returns only the full path or file name
else: #If it is not a zip file, return it as it is
return fullpath
def tickdata(filepath):
"""binary to pandas DataFrame using numpy.
reference: (´ ・ ω ・ ｀ ；)I'm sorry-Read MT4 history file with python
http://fatbald.seesaa.net/article/447016624.html
(16)[MT4] Historical data (hst file) format
https://www.dogrow.net/fx/blog16/
"""
with open(filepath, 'rb') as f:
ver = np.frombuffer(f.read(148)[:4], 'i4')
if ver == 400:
dtype = [('time', 'u4'), ('open', 'f8'), ('low', 'f8'), \
('high', 'f8'), ('close', 'f8'), ('volume', 'f8')]
df = pd.DataFrame(np.frombuffer(f.read(), dtype=dtype))
df = df['time open high low close volume'.split()]
elif ver == 401:
dtype = [('time', 'u8'), ('open', 'f8'), ('high', 'f8'), ('low', 'f8'), \
('close', 'f8'), ('volume', 'i8'), ('spread','i4'), ('real_volume','i8')]
df = pd.DataFrame( np.frombuffer(f.read(), dtype=dtype) )
df = df.set_index(pd.to_datetime(df['time'], unit='s', utc=True)).drop('time', axis=1)
return df
def read_hst(fullpath):
"""Extracting hst file from zip file.
Usage:
import hst_extract as h
df = h.read_hst('~/Data/USDJPY.zip')
args:
fullpath: zip / hst file path
return:
pandas DataFrame
"""
hstfile = zip2hst(fullpath) # Extract zip in current directory.
print('Extracting {}...'.format(hstfile))
df = tickdata(hstfile) # Convert binary to pandas DataFrame.
#If fullpath is given something other than an hst file, delete the file
if not os.path.splitext(fullpath)[1] == '.hst':
os.remove(hstfile)
return df
def main():
"""Arg parser
usage: bin/read_hst.py [-h] [-c] [-p] filenames [filenames ...]
Convering historical file (.hst) to csv or pickle file.
positional arguments:
filenames
optional arguments:
-h, --help show this help message and exit
-c, --csv Convert to csv file
-p, --pickle Convert to pickle file
`stockplot/bin/read_hst.py -cp ~/Data/USDJPY.zip ~/Data/EURUSD.zip`
Extracting '~/Data/USDJPY.zip' and '~/Data/EURUSD.zip' then save to
* '~/Data/USDJPY.csv' and '~/Data/EURUSD.csv' as csv file.
* '~/Data/USDJPY.pkl' and '~/Data/EURUSD.pkl' as pickle file.
"""
description = 'Convering historical file (.hst) to csv or pickle file.'
parser = argparse.ArgumentParser(prog=__file__, description=description)
parser.add_argument('filenames', nargs='+') #One or more file names
parser.add_argument('-c', '--csv', action='store_true', help='Convert to csv file')
parser.add_argument('-p', '--pickle', action='store_true', help='Convert to pickle file')
args = parser.parse_args()
filenames = args.filenames
csv = args.csv
pickle = args.pickle
if not filenames:
raise KeyError("Enter a valid filenames")
elif not (csv or pickle):
raise KeyError("Enter a valid output - filetype '-c'(--csv) or '-p'(--pickle).")
else:
for filename in filenames:
df = read_hst(filename) # convert historical to pandas Dataframe
basename = os.path.splitext(filename)[0]
if csv:
outfile = basename + '.csv'
df.to_csv(outfile)
yield outfile
if pickle:
outfile = basename + '.pkl'
df.to_pickle(outfile)
yield outfile
if __name__ == '__main__':
for convert_filename in main():
print(convert_filename)
Reference URL
Overwhelming thanks! (≧ ▽ ≦)
Backtest FX with "Backtesting.py"! : Python https://mmorley.hatenablog.com/entry/fx_backtesting01 (´ ・ ω ・ ｀ ；) I'm sorry "I tried the back test" http://fatbald.seesaa.net/category/25188000-26.html Module backtesting.backtesting https://kernc.github.io/backtesting.py/doc/backtesting/backtesting.html#backtesting.backtesting.Backtest&gsc.tab=0 Make MT4 historical data workable on python or save it in csv https://qiita.com/u1and0/items/6a690f6b0080b8efc2c7 How to create a Qiita account, how to write an article, how to post https://qiita.com/predora005/items/342b50859e2aeafc39b6
Recommended Posts