How to set cron for regular Python scraping on Sakura server.
Introduction
It seems that you have to set cron to scrape regularly, and when I searched it, it was too confusing in an old article, so I described my own method.
I wrote about scraping last time, so if you want to scrape, please see the link below. https://qiita.com/kei_kei_kei/items/730ba09e7830d5c00a1c
Let's actually set it
Open a terminal and log in to the Sakura server.
Execute the following command.
ssh account name@Initial domain(xxx.sakura.ne.jp)
You will be asked for a password, so enter the password required when entering the domain ID.
%I was able to log in when appeared.
Sakura server python version is old
Since the version of python is old, everyone is desperate to upgrade it, but it's annoying.
pathIt's even more annoying because I have to change it, so leave it as it is.
However! If you move it as it is, an error will occur.
Traceback (most recent call last):
File "train_mnist.py", line 4, in <module>
import requests
ImportError: No module named requests
Local is Python3. Sakura is Python 2.
Locally
pip install (External module name)
I think it worked after installing it.
Sakura also needs to install an external module.
Sakura's version is too low to use pip.
If you notice, I'll try to install it with pip, but I can't.
So please install the required modules as below
python -m install requests
This time I installed requests,
If you want to install Beautiful Soup, use the following command
python -m install beautifulsoup4
I will install the necessary modules like this.
If you can install the external module to be used this time, the work in the terminal is over.
We will set up the Sakura server.
Log in to the contracted Sakura server
Click the cron settings link in ⓵ in the application settings
Press ⓵ to go to the screen where the additional link for the new item in ⓶ appears. Just press ⓶ Add new item.
We will set how often cron is run.
⓵
cd /home/My account name/www/Directory with files;<Half-width space>/usr/local/bin/python<Half-width space>対象ファイル.py
In ⓶
As described in the explanation, describe what time of month, what time, and every minute you want to move.
In the image, it is executed every 2 minutes.
Click the ⓷ submit button when you have finished entering
### Press Send to return to the cron setting screen, so set the environment variables
here
Environment variable settings: ``` PYTHONPATH```
Environment variable values: ``` home / account name / lib / python```
Enter the above and click Add
### Completion of registration! !!
# Let's check if it actually works.
If you have not entered `` `1> / dev / null``` in Sakura's cron settings, you will receive an email.
If there is an error in the email, check the error text as it is written.
If there is an error, it will be resolved according to the error.
# Finally check required
Sakura's Python default encoding is ascii.
So, if you want to speak Japanese, please change the default.
If you want to check the default encoding, go back to the beginning and open Sakura Terminal.
Log in and type `` `python``` to execute the following command.
% python
import sys print sys.getdefaultencoding()
You can check the default encoding.
Since I only needed the Int type this time, I didn't have to change it, but if you need it, please try it.
## Click here for the site created with cron settings
[Completed site: https://compadre.matrix.jp/aisekiya/](https://compadre.matrix.jp/aisekiya/)
Recommended Posts