On my server I have a very simple webapp developed in Python with Flask. Its code is hosted on GitHub in a private repository. Before, every time I modified the app on my PC, then pushed it to GitHub, I had to connect to the Linux server to kill it, git pull and restart it. It was annoying but now everything is happening automatically!
How can my Python Flask web app deploy itself to my Linux server when I push to GitHub?
I have put in place a very simple solution that I’ll explain below in details. But here is a summary of what I needed and the solutions I chose:
- The app running on my server must be informed when a new version is pushed to GitHub. This is done using a GitHub webhook which calls the web app itself, but with the proper security checks to ensure that only GitHub can trigger such an event.
- Since the code is stored in a private repository, the server needs credentials to fetch the newer version. But I did not want to store an SSH key giving access to my whole GitHub account. So, I created an SSH deploy key which only gives access to the desired repo in read-only mode.
- The whole process is orchestrated using a simple shell script and the app runs as a systemd service
The sequence is the following:
Triggering notifications 🔗
The definition of the webhook is done in the repository settings and here’s what I use:
The interesting parameters are:
- 🔗 Payload URL: URL of the webhook listener on my web app, that GitHub will request. I included a random string in the path to ensure that it isn’t called by a robot (anyway it would be secure as we’ll see after).
- 🤫 Secret: this is really important! It allows to secure your webhook and ensure that no one other than GitHub can call it. It is a shared secret in the form of a simple string that must be configured on both GitHub and the app side. I generated a random UUID using Python (better than a common password):
>>> from uuid import uuid4 >>> str(uuid4()) '7e867072-9ffc-4fff-985c-84b039631ea3'
- 🔐 SSL verification: my web app is exposed on HTTPS and its certificate is valid (thanks to Let’s Encrypt) so I enabled SSL verification since it’ll work and as recommended.
- ↗️ Just the push event: I only want to be notified about “git push” events, not other events happening the repository (like fork, new issue, etc.)
Receiving notifications 🔗
Now that GitHub is configured to call the webhook in my app, I need to handle it. Here is the Python code that I wrote:
@app.route('/webhook', methods=['POST']) def webhook(): # X-Hub-Signature-256: sha256=<hash> sig_header = 'X-Hub-Signature-256' if sig_header in request.headers: header_splitted = request.headers[sig_header].split("=") if len(header_splitted) == 2: req_sign = header_splitted computed_sign = hmac.new(secrets.webhook, request.data, hashlib.sha256).hexdigest() # is the provided signature ok? if hmac.compare_digest(req_sign, computed_sign): # create a thread to return a response (so GitHub is happy) and start a 2s timer before exiting this app # this is supposed to be run by systemd unit which will restart it automatically # the  syntax for lambda allows to have 2 statements threading.Thread(target=lambda: [time.sleep(2), os._exit(-1)]).start() return "ok"
First of all, the app validates that the webhook call is indeed coming from GitHub and not from someone who guessed the URL. It follows the securing your webhooks guide. Legitimate requests will contain the
X-Hub-Signature-256 header with value
<hash> is a HMAC SHA256 signature of the request payload, with the shared secret with GitHub that I created and configured. There are some checks to ensure that the header is present and with the expected format, then the HMAC is computed with
hmac.new(). Notice how the code uses
hmac.compare_digest() to have a time-constant comparison, thus preventing timing attacks. Webhooks initially used the
X-Hub-Signature header with a HMAC SHA1 signature, but it’s being deprecated by GitHub for security reasons.
With all of this, if a robot or someone malicious requests
/webhook, the request will be ignored unless they know the secret 😉
The payload sent by GitHub in the request contains a lot of interesting data useful in some cases (who pushed, what are the commits, which files change, etc.) but here I don’t use any.
If the signature is good, the application exits to trigger the update. There is just a little quirk to wait 2 seconds before exiting, so the function has time to returns “ok” in the response to GitHub. It allows GitHub to show in the recent deliveries that the webhook call was good and show a nice green tick:
I also decided to return “ok” in all cases, whether everything is good or not. It makes debugging harder but it leaks less technical information. As you prefer!
SSH deploy key 🔗
Now I needed credentials to pull the new code from my private repository. I created a new SSH key (without passphrase) and declared it as a deploy key in the repo settings (instead of my account’s settings):
This SSH key gives access to this repository only, not to the others in my GitHub account. Moreover, I didn’t check “allow write access” when adding it, so the given access is read-only which is even better 😌
My app runs under a service account called “scripts” with its own home folder. So, the private and public key files are stored in
/home/scripts/.ssh like usual. This folder being outside of the app folder, I’m certain the keys aren’t exposed inadvertently by the web server.
Service and update script 🔗
The app is running as a service using systemd. Here is the simple service configuration file I use, saved as
[Unit] Description=my app [Service] Type=simple ExecStartPre=/bin/bash /home/scripts/myapp/update.sh ExecStart=/usr/bin/python3 /home/scripts/myapp/main.py WorkingDirectory=/home/scripts/myapp/ User=scripts Group=scripts Restart=always RestartSec=5 StartLimitInterval=60s StartLimitBurst=3 [Install] WantedBy=default.target
You’ll notice that it declares a shell script in
ExecStartPre, which runs before starting the app itself declared in
And here is the
update.sh script in charge of fetching new code from GitHub:
#!/bin/bash date > updated.txt # ensure to get origin's data and discard any local change git fetch git reset origin/master --hard git pull
The current date and time are written in
updated.txt. I use it in the webapp to include the latest update time in an HTML comment to easily check if it works fine. Then the newer code is fetched and all eventual local modifications are erased.
Here’s how I was able to fully automate a simple deployment of a small Python Flask web app on my server. This doesn’t apply to all cases (for ex. more complex web applications that can’t just
exit() in the middle of something!), and I’m sure many things can be improved, but it should give you some good ideas and a working basis! 💪