Auto deploy Python Flask web app on GitHub push feature image

On my server I have a very simple webapp developed in Python with Flask. Its code is hosted on GitHub in a private repository. Before, every time I modified the app on my PC, then pushed it to GitHub, I had to connect to the Linux server to kill it, git pull and restart it. It was annoying but now everything is happening automatically!

How can my Python Flask web app deploy itself to my Linux server when I push to GitHub?

I have put in place a very simple solution that I’ll explain below in details. But here is a summary of what I needed and the solutions I chose:

  • The app running on my server must be informed when a new version is pushed to GitHub. This is done using a GitHub webhook which calls the web app itself, but with the proper security checks to ensure that only GitHub can trigger such an event.
  • Since the code is stored in a private repository, the server needs credentials to fetch the newer version. But I did not want to store an SSH key giving access to my whole GitHub account. So, I created an SSH deploy key which only gives access to the desired repo in read-only mode.
  • The whole process is orchestrated using a simple shell script and the app runs as a systemd service

The sequence is the following:

Webhook 🔗

Triggering notifications 🔗

The definition of the webhook is done in the repository settings and here’s what I use:

The interesting parameters are:

  • 🔗 Payload URL: URL of the webhook listener on my web app, that GitHub will request. I included a random string in the path to ensure that it isn’t called by a robot (anyway it would be secure as we’ll see after).
  • 🤫 Secret: this is really important! It allows to secure your webhook and ensure that no one other than GitHub can call it. It is a shared secret in the form of a simple string that must be configured on both GitHub and the app side. I generated a random UUID using Python (better than a common password):
>>> from uuid import uuid4
>>> str(uuid4())
'7e867072-9ffc-4fff-985c-84b039631ea3'
  • 🔐 SSL verification: my web app is exposed on HTTPS and its certificate is valid (thanks to Let’s Encrypt) so I enabled SSL verification since it’ll work and as recommended.
  • ↗️ Just the push event: I only want to be notified about “git push” events, not other events happening the repository (like fork, new issue, etc.)

Receiving notifications 🔗

Now that GitHub is configured to call the webhook in my app, I need to handle it. Here is the Python code that I wrote:

@app.route('/webhook', methods=['POST'])
def webhook():
    # X-Hub-Signature-256: sha256=<hash>
    sig_header = 'X-Hub-Signature-256'
    if sig_header in request.headers:
        header_splitted = request.headers[sig_header].split("=")
        if len(header_splitted) == 2:
            req_sign = header_splitted[1]
            computed_sign = hmac.new(secrets.webhook, request.data, hashlib.sha256).hexdigest()
            # is the provided signature ok?
            if hmac.compare_digest(req_sign, computed_sign):
                # create a thread to return a response (so GitHub is happy) and start a 2s timer before exiting this app
                # this is supposed to be run by systemd unit which will restart it automatically
                # the [] syntax for lambda allows to have 2 statements
                threading.Thread(target=lambda: [time.sleep(2), os._exit(-1)]).start()
    return "ok"

First of all, the app validates that the webhook call is indeed coming from GitHub and not from someone who guessed the URL. It follows the securing your webhooks guide. Legitimate requests will contain the X-Hub-Signature-256 header with value sha256=<hash>. The <hash> is a HMAC SHA256 signature of the request payload, with the shared secret with GitHub that I created and configured. There are some checks to ensure that the header is present and with the expected format, then the HMAC is computed with hmac.new(). Notice how the code uses hmac.compare_digest() to have a time-constant comparison, thus preventing timing attacks. Webhooks initially used the X-Hub-Signature header with a HMAC SHA1 signature, but it’s being deprecated by GitHub for security reasons.

With all of this, if a robot or someone malicious requests /webhook, the request will be ignored unless they know the secret 😉

The payload sent by GitHub in the request contains a lot of interesting data useful in some cases (who pushed, what are the commits, which files change, etc.) but here I don’t use any.

If the signature is good, the application exits to trigger the update. There is just a little quirk to wait 2 seconds before exiting, so the function has time to returns “ok” in the response to GitHub. It allows GitHub to show in the recent deliveries that the webhook call was good and show a nice green tick:

I also decided to return “ok” in all cases, whether everything is good or not. It makes debugging harder but it leaks less technical information. As you prefer!

Gunicorn specifics 🔗

If your Python app is running through Gunicorn (or similar), a simple os._exit(-1) will not be sufficient to run the newer code because it will just restart a new worker with the existing code and continue forever. One solution is to trigger a restart of the service with this line instead near the end of the code above:

threading.Thread(target=lambda: [time.sleep(2), os.system('systemctl restart myapp')]).start()

Note though that it will only work if your app runs as root, which is not recommended. But there are options to allow an unprivileged user to restart a service.

Thanks to Eduardo Heisenberg for sharing this tip with us

SSH deploy key 🔗

Now I needed credentials to pull the new code from my private repository. I created a new SSH key (without passphrase) and declared it as a deploy key in the repo settings (instead of my account’s settings):

This SSH key gives access to this repository only, not to the others in my GitHub account. Moreover, I didn’t check “allow write access” when adding it, so the given access is read-only which is even better 😌

My app runs under a service account called “scripts” with its own home folder. So, the private and public key files are stored in /home/scripts/.ssh like usual. This folder being outside of the app folder, I’m certain the keys aren’t exposed inadvertently by the web server.

Service and update script 🔗

The app is running as a service using systemd. Here is the simple service configuration file I use, saved as /lib/systemd/system/myapp.service:

[Unit]
Description=my app

[Service]
Type=simple
ExecStartPre=/bin/bash /home/scripts/myapp/update.sh
ExecStart=/usr/bin/python3 /home/scripts/myapp/main.py
WorkingDirectory=/home/scripts/myapp/
User=scripts
Group=scripts
Restart=always
RestartSec=5
StartLimitInterval=60s
StartLimitBurst=3

[Install]
WantedBy=default.target

You’ll notice that it declares a shell script in ExecStartPre, which runs before starting the app itself declared in ExecStart.

And here is the update.sh script in charge of fetching new code from GitHub:

#!/bin/bash
date > updated.txt

# ensure to get origin's data and discard any local change
git fetch
git reset origin/master --hard

git pull

The current date and time are written in updated.txt. I use it in the webapp to include the latest update time in an HTML comment to easily check if it works fine. Then the newer code is fetched and all eventual local modifications are erased.

Conclusion 🔗

Here’s how I was able to fully automate a simple deployment of a small Python Flask web app on my server. This doesn’t apply to all cases (for ex. more complex web applications that can’t just exit() in the middle of something!), and I’m sure many things can be improved, but it should give you some good ideas and a working basis! 💪