DIY Excel Accounting System | 2018

Step 1

Download the base file from Facebook

Copy to your local drive and make changes (refer to video) in 'TabList' to activate the clickable buttons.

To be continued...

Uploading Python file To Get Stock Data From Yahoo Finance| 2018

Just uploaded the Python file to my Facebook group (link below)

Get_Stock_Python_File

Updated Version @ March 27, 2018


*By clicking the link above, you are assumed to accept any disclaimer terms
stated in the linked post 

#get_stock_data_from_web, #yahoo_finance, #python, #pandas_datareader, 

Remote Accessing Raspberry Pi 3 From Win 10| 2018-03-25

Just captured the screenshots for the process setting up remote access to Pi3 from Win 10
Click the link below for detail instructions.



Remote Accessing Raspberry Pi3 From Win 10

Notes | Entry-Level Stock Price Analysis | Stats On Stock Price Percentage Change N Days After

Quoted from this page

1) [Import Libraries]
We need at least 2 Python libraries (pandas and numpy) to read stock data (step 3) and calculation.

2) [Define Data Source]
In case you have the stock data in CSV or Excel format, you can simply
store it in variable (e.g. data_source) assigning the file path (e.g. r'your_path', see pic)

If you want to get data feed online, you might:
 (i) write code on another Python file + store the result in local drive + assign variable to that path
(see this video https://www.youtube.com/watch?v=BKezm2_hLNw)
or
(ii) wrtie code on the SAME Python file + store the result in local drive + assign variable to that path

I preferred method (i) because of the quicker load time and cleaner code to maintain.
I ran into problem with method (ii).  I could fix that by creating multiple variables but still, it's not
very convenient for me.

3) [Store Data In Tabular Format]
Simply use the 'pandas' package to read your source file in tabular format.
The 'dropna()' simply clear whatever row that has non-numeric/datetime value inside.

4) [Define Target Column(s)]
This is the real start of our solution.
We first set the target.  In this case, my question is as follows:

"I want to know when something happens (triggering event), what do I get (target_event)."

From pic_1 and pic_3, I want to know [the percentage change N days after [trigger_event] happens.

That's why I need to define [the percentage change] by using the '.pct_change()' method after
the column (e.g. Close).

5) [Define Triggering Event(s)]
I define the 'triggering event' in the variable 'criteria' which can further be divided into
many sub-sections (e.g. 'tcriteria' means target criteria; 'dcriteria' means day period...)
The variable names are up to you (e.g. you can call 'trigger_event' instead of 'criteria')

In Pic_1, (see 5a), I filter out those with [today's stock price change %] is UP or DOWN more than 1%
(e.g. abs(df['day_chg'] ) > 1)
You might change the 'tcriteria' to something like

my_cat_s_birthday = '2018-02-08'
Replace 'tcriteria' with 'my_cat_s_birthday'

It's totally up to you.

The reason I split 'criteria' into (5a - tcriteria) and (5b - dcritiera) is
the greater flexibility. 
Usually, I want to know whatever triggering event (5a) happens in whatever period (5b).
Through adding a separate period criteria (5b - dcritiera), it's easier to maintain the code.

6) [Define LookUp Period]
Usually, we want to know 'what will [the stock price] be [N days after today].
This is the part to define.

In Pic_1, we want to know 'the percentage change' (df['day_chg']) 1 day after today, so
our code is

df['Ndaychg'] = df['day_chg'].shift(-1)

Since we will be building more interactive app for data analysis,
I suggest generalizing the above formula like this:

lookforwardday = 1   <- you can change this to input(whatever number here, see Pic_3)

df['Ndaychg'] = df['day_chg'].shift(lookforwardday)

7) [Print Result]

Note that you can print 'whatever you want' from your dataset.

But in my case, I want to know the 'mean', 'standard deviation', and 'median' of 'columns that I want to view'.

Given that, I split my print action into (7a) and (7b)
(7a) is simply printing ALL information (e.g. include Open High Low CLose ...) that meets the criteria
stated in step (5)

In Pic_1, you notice the latest day (the bottom row, 2018-02-28 has [NdayChg] = NaN, not a number.
because, on Feb 28, 2018, we don't know what will happen the day after, Mar 1, 2018.

In Pic_3, we define the 'lookforwardday' to be 3.  That's why, we won't know what will 3 (trade days)
after will be, on Feb 26, and Feb 28, 2018.


(7b) Based on the info (see 7a), I add the 'columns that I want to view the stats'
(e.g. .loc[:, ['Vol','Ndaychg']]) 

Put it simple,

df[critieria]   add  .loc[all rows, ['column1', 'column2','column_whatever']]  then add
.agg(['mean','std','median'])  

Convert Date Format In Python | Datetime 'strftime()'| 2017

Python Datetime 'strftime()' | Convert Date to Text Format

01:00 - 03:30 | Get Stock Data With Yahoo Finance
03:31 - 06:04 | Convert Date Format To Text
06:05 - 06:43 | Slicing doesn't work due to keyword "&" "and"

Quick note:

I couldn't slice the dataframe based on column 'df['new_date']' because the keyword "and"
was not used when coding in Python IDLE.

In Sublime Text 3, I used "&" instead of "AND" and the slicing worked.

So, remember to use "and" instead of "&" when coding in Python IDLE.

Install Google Chrome Other Than C Drive| Windows| 2017

First of all, thank Matt Skaggs for his article regarding Google Chrome Installation Other Than C Drive:
http://smallbusiness.chron.com/change-installation-path-google-chrome-42689.html

He introduced the way to use the "Junction" program to create link between the original Chrome installation location (normally, "%LocalAppData%\Google\Chrome" ) and the destination folder
(wherever you want, e.g. D:\Program Files\Google\Chrome)

Here's my video showing the installation process.

#install_google_chrome_other_than_c_drive, #Matt_Skaggs, #Windows_7, #2017,

Run Python With VSCode On USB | Anaconda Envs| Windows 10| 2017-11-14

Why Use Visual Studio Code

I am happy with Sublime Text 3 for my Python coding.
But since VSCode is free, I'd like to give it a try.
To be honest, the task runner setup was more complicated that I expected.

Anyhow, I managed to set this up on a USB.

Python Environment

1) Python 3.6.3 - Anaconda , with environment being created with Anaconda Navigator
2) Operating System: Windows 10 

Steps

1) Install Anaconda (location being set to D drive (drive other than default C))

2) Create Environment With Anaconda Navigator (Optional)
This step is optional if you're happy with the Anaconda root installation.
I tend to create different projects in different environments (Python virtual environment),
just a matter of habit.

 

3) Download VSCode and Setup

This can be quite tricky when you need to setup the task runner to run Python code.

Create Python Backtest With Jupyter Notebook Anaconda 3.5.0.1 On Windows 10| 2017-11-11

Backtest With Jupyter Notebook Anaconda 3.5.0.1

Just walkthrough the whole process of creating Python backtest in Anaconda, Windows 10.

The process is broken down into 5 major steps:
00x | Install Anaconda
10x | Get online stock (e.g. AAPL) | Market Data
20x | Create trade strategy | Strategy Class
30x | Generate trade signal | Signal Class
40x | Create investment portfolio | Portfolio Class
50x | Perfomance review | PnL + Graphing

A timeline is provided below for easy navigation:
In case you just want to use the Anaconda root setup, skip step 'Ana002'

mm:ss
00:00 - 03:02 ||Ana001| Install Anaconda
03:04 - 10:02 ||Ana002|Setup different environment other than root - optional step*
10:04 - 19:02 ||Ana101| Get Apple stock data from Yahoo Finance, save to local drive as Excel
19:04 - 29:11 ||Ana201| Create Strategy - calculate moving averages
29:12 - 35:53 ||Ana202| Create Strategy - get stock trend
35:55 - 41:22 ||Ana301| Get Trade Signal and Generate Order
41:24 - 49:50 ||Ana401402| Create Investment Portfolio and Trade Details
49:52 - 59:03 ||Ana501502| Calculate Profit and Loss and Graphing

* install optional Python packages that your project requires
** the strategy mentioned assumes zero borrowing cost so that whenever short position occurs,
we can borrow unlimited amount of money without needing to pay a penny of interest.
This is not real in real world.  So just be aware of that.

Here's the 59-min video(Complete Walkthrough - Jupyter Notebook Anaconda):

*** this video is rendered in 1.05x speed; feel free to visit the original speed videos below.

Install Anaconda 3.5.0.1(Ana001)

Setup Environment In Anaconda 3.5.0.1(Ana002) - Optional

Get Online Stock Data + Save To Local Drive(Ana101) 

Form Trade Strategy - Calculate Moving Average (Ana201) 

Form Trade Strategy - Get Stock Trend (Ana202) 

Generate Trade Signal - Get Stock Trend (Ana301)

Create Investment Portfolio (Ana401402) 

Measuring Performance & Graphing (Ana501502)

Backtest With Python Default Editor

My Thought

Comparing to buidling Python strategy backtest in default Python editor,
the Anaconda Jupyter Notebook takes a bit of time to get used to.
It takes time to load up the Jupyter Notebook while such load time is minimal in Python default editor.

But from layman's perspective, setting up multiple environments and installing packages
through the Anaconda Navigator are intuitive.
Anaconda basically organizes the folders under the 'env' folder.
That's good folder management.

In the future, I'll try using Sublime Text 3 to route to the Anaconda's environment folder.
See if I can take advantage of Sublime Text' speed while maintaining proper environment folder
management.

If you find this article helpful, feel free to discuss on my Facebook:

www.facebook.com/clueple


Disclaimer:
Everything shown in this video is just the author's personal experience with Anaconda.
All mentioned assumptions and tactics are used as demonstration for education purpose.
Trade details, not limited to, leverage level, deposit amount, transactions fees, and
any costs associated with the strategy are not meant to be complete.

No investment decision should be based on any information provided in this video.
No professional advice are provided or implied whatsoever.

Viewers are solely responsible for any loss associated to consumption of this video.

Setup Jupyter Notebook & Anaconda Environment On Windows 10

Jupyter Notebook

Just record how to create Anaconda environment to use Jupyter Notebook in folders other than the root folder, Windows 10.

#create_anaconda_environment_on_win10, #jupyter_notebook_on_anaconda, #use_jupyter_notebook_in_folders_other_than_root_folder,

How I Create My 1st Backtest With Python | Example: Google (GOOG)

Finally finished up my 1st backtest with Python.
Just shoot this video to help remind myself the basics
and hopefully, this can help people get their feet wet
on Python coding ... (advanced users feel free to jump around using the time table below)

Here, I am showing you 2 videos where the first one is the Python setup (from scratch)
and the second one is the backtest creation.

YouTube video link - Install Python & Related Packages:

*Note that I installed Python to specific folder "py362"  but you can feel free to decide the destination folder location.
**Also, I setup the Sublime Text 3 as my code editor.  Again, you can pick your own editor.
In the backtest creation video, I used the Python built-in editor (IDLE).
***I created the virtualenv (virtual environment) for my trade algo.
You don't need that for backtest.  But it's highly recommended if you want to build Python projects, other than trade algo.

YouTube video link - Creating backtest:
https://www.youtube.com/watch?v=D73NyNb2YZs

Time Table |English Version
hh:mm:ss
00:00:09 - 00:08:24 | part 1| Get stock data from Yahoo Finance + Save to Excel
00:08:26 - 00:14:49 | part 2.1| Strategy class - defininng basic parameters
00:14:50 - 00:21:21 | part 2.2| Strategy class - calculate moving averages
00:21:22 - 00:30:53 | part 2.3| Strategy class - calculate trend details
00:30:54 - 00:37:58 | part3.1| Signal class - calculate trade signal and order
00:30:59 - 00:45:51 | part4.1| Portfolio class - calculate trade amount
00:45:52 - 00:52:26 | part4.2| Portfolio class - calculate cash balance and asset position
00:52:26 - 00:57:56| part 5.1| calculate profit and loss + plot graph


For part 4.2 "cash_balance and asset position",
you can read Isaac Kwan's code for traditional stock trade style.
He offered great help in explaining the use of abstract-based class of Python (I am still figuring that out :))

https://gitlab.com/snippets/1681494


Python script: (underlying stock being changed to 0388.HK)
https://github.com/clueple/algo/blob/master/hkex_full_example.py

Isaac Kwan's link on part 4.2 - cash balance and asset position
https://gitlab.com/snippets/1681494



Finally, please read the video disclaimer
and by no means, this video be used for real trade.

Setup Python 3.6.2 Virtual Environment For Trade + Machine Learning Setup| Windows 10 Sublime Text 3

00：00 - 01：20 | Install Python
01:27 - 02:02 | Install virtualenv to system Python
02:06 - 03:02 | Create virtual environment
03:13 - 03:58 | Download Sublime Text 3
04:02 - 06:28 | Setup Sublime Text 3 Python path
06:29 - 07:31 | Setup Sublime Texxt 3 layout
07:38 - 08:36 | Install Python packages for trade
08:37 - 09:56 | Example reading stock data

Part 1

Part1

In this video
I'll be showing how to setup Python on Windows 10

Look up for the Python installation from your browser
and install the one that fits your system

Here I'm downloading the 64-bit Windows version

Save it to your download folder
and double click to install

Untick things that are not needed
and custom install to the drive you want

Untick the optional features
and set the destination folder

In this case
I'll be installing Python to F drive folder "py362"

Hit "OK" and start installation

You can now remove the Python installer

Now we're going to install virtualenv

Open command prompt

Navigate to the system Python location
which is F:\py362\Scripts

from there
we pip install virtualenv

Hit [Enter]
and wait for the process

you can type pip list to check whether virtualenv is installed to the system

Part 2

We're now creating a folder in F drive to hold our virtual environment

Go to command prompt
Go to F drive and type md space "Trade"
We'll be using this folder to hold our Python trade projects

Part 3

Open a new command prompt

and navigate to our system Python location
which is F:\py362\Scripts

hit [Enter] and install a new virtual environment
to the path
F:\Trade\TradeEnv
then hit [Enter] to start installation

Exit command prompt
and go to the path to see whether the virtual environment is installed

Jump back to browser
and download Sublime Text 3
Click to download the one that fits your system

Once it's downloaded
extract all files
then cut and paste the Sublime Text folder
to your destination folder

Open up the Sublime Text folder
and doubleclick the execute file to start programming

Let's start typing some Python code
Save it as a Python file

Before you run the code
you go to Tools -> Build System and tick Python

But when you hit [Ctrl] [B] to run the code
you get error message

That's why we need to create new build
to point to our virtual environment

From the untitled sublime build file
we need to paste some codes

We can look up for these codes from web

Before we save
we need to change the "python" path
to our virtual environment

I just name it with my environment's name
so that I can easily remember

Go back to your Python file
and tick the Sublime build we've just created
and run the Python code to check if it works

Yeah! It works

Part 4

If you want to move the output panel
from bottom to right

simply hit [Ctrl], [Shift], [P]
and type "install package"

Hit [Ctrl], [Shift], [P] again
and type "install"
Hit it and you'll be able to type "buildview"

click the only selection
and wait for the installation

Now you can hit [Alt] [Shift] [2]
and see a split in between

Simply run the code again
and drag the result panel
to the right

Then you can now see the right side
displays the code result

Now
we're going to pip install several packages for trade

Open command prompt

Navigate to the virtual environment's Scripts folder
pip install pandas
This is a popular package to read and process data

We then pip install xlrd
to let us read and write Excel file with Python

Pip install matplotlib
We use this one to plot graph

We also need beautifulsoup 4 to grab web data

exit command prompt

I'm going to create a new Python file to test these packages
Before saving, make sure you're coding in the virtual environment

Save the file

We're going to read this csv file with Python packages

Let me type and run the code

Ooops I type the wrong file format excel instead of csv

Go back to change to csv in our code
and press [Ctrl] [B] to run it

There we go!

Thanks for watching


Please note that you can also install numpy + mkl, scipy, and sklearn for machine learning

Feel free to discuss trade algo at
www.facebook.com/clueple

Setup Sublime Text 3 To Use Python Virtualenv on Windows

Using Virtualenv in Sublime Text 3 (Windows)

I was just looking for this setup for long.
Finally, found that pointing the path in the Sublime Text 3 build file (video 05:14 - 08:07)
can make it work.

The reason why I setup virtual environment because I need to separate different projects
based on purposes.

In this video, I pip install 2 packages - pandas and xlrd - to my virtual environment "myTradeEnv" to read the stock data from a csv file.

When you're in Sublime Text, type your Python code and save as a new Python file.
Then you go to Tools -> new build system and paste your code there.

Once you save that Sublime Text build file, get back to your Python file
and hit [Ctrl] + [B] (build) to run your code.


00:00 - 01:23 | Download and install Python
01:23 - 03:01| install virtualenv to system
03:19 - 04:09| create virtual environment myTradeEnv
04:22 - 05:11| download sublime text 3
05:14 - 08:07| setup sublime text 3 to run Python virtual environment
08:10 - 11:10 | test running python code in virtual environment to grab stock data from a csv file

Running Python On USB | WinPython

Just tried using Python on USB via WinPython.
It's pretty handy.

Simply double-click the WinPython execute file downloaded from its official website.
And you'll see a zip folder.
Unzip it to wherever you want (in my case, a USB stick).

You're ready to code.

Simply double-click the icon "Spyder" to open up the code editor.

Try Loading Video Source With OpenCV in Python Virtual Environment

I tried out the New Boston's OpenCV script to load up a video source.

I could get the video feed when only loading the webcam but not the capture card being hooked up to my laptop.

I did get the choice to select video source.  But still, couldn't get it worked.

If you know how to grab the video source to play on PC, please let me know.

Many thanks.