Develop a Python Flask Application with Docker and Deploy to AWS – Part 2 Serve a Flask App with Docker

aws flask python Jan 12, 2019

Overview

In this portion of the series we will be:

Creating a simple flask app with a few RESTful methods
Testing out our RESTful methods using curl.
Future proofing our development environment for easy and stress free deployment.

Get the Code

Clone the github repo to see the full code.

Create a flask app

Flask is a python framework for creating web applications. It has all kinds of functionality, but here we will be using just a small portion to create a few RESTful web routes.

A Note about Directory Structures

I tend to sit down with a paper and colored pens and figure out how I want my application organized. Generally, each service is in its own directory, which is then in its own docker container. A service is a single application, such as our flask app, a database, a job queue, etc. For this particular example we only have one service, the flask app, but hopefully you can see how important it is to organize your applications.

├── build_docker_image.sh
├── docker-compose.yml
└── flask_app
├── Dockerfile
├── __init__.py
├── download_files.sh
├── flask-app-environment.yml
├── flask_app.py
└── start_flask_app.sh

For this portion of the series we will focus on everything happening in the flask_app folder.

Flask App Definition

First of all, we have to define our flask application. At a minimum, this imports Flask and creates an app. Of course, we want to do more than this.

CORs, Headers, and Access Control

You will notice that there are all kinds of headers added here. This is to enable requests from front end dev environments (such as localhost:4200 if you are developing using an angular frontend), or to allow for requests from IP addresses besides the one that the flask app is hosted on. You will notice: response.headers["Access-Control-Allow-Origin"] = "*". You almost never want to use "*", but instead have a list of IP addresses that are allowed access. Most of the time I am working behind a firewall and don't really care, but if you have an IP address open to world you will probably want to restrict it somewhat (or not).

Beyond this tidbit security is completely beyond the scope of this article.

Run your flask application

We are going to run our flask app using gunicorn, which is a fairly standard way of running flask applications both in production and development. You can get more indepth information on the parameters of gunicorn by looking at its documentation. The important thing to note here is that we are binding to 0.0.0.0:5000, which will be important for our docker use later, and telling gunicorn to run a 'flask_app:app', which corresponds to our app declaration in flask_app.py

gunicorn --workers=2 --bind=0.0.0.0:5000 --keep-alive=2000 --timeout=2000 --log-level=debug --reload flask_app:app

If you're not using the github repo, be sure to put this in a bash file , flask_app/start_flask_app.sh, and make it executable before moving onto the next section.

Build the Docker Image

Now we get to have some fun and build containers!! This container is slightly more involved than the example in Part 1 of the series. I wanted to keep that one quite simple, and to demonstrate how you can add layers of complexity to your build system and containers in order to build real world applications. Hopefully, you can also see how to debug containers, by adding your functionality step by step. The example Dockerfile I used in Part 1 is the base of every python project I deploy. First I install any additional system packages I need, then I bootstrap my python conda environment, and then I decide how I want to deploy my application.

Dockerfile Commands

ENV

Set environmental variables in the docker container. Example:

ENV hello world

Sets the environmental variable hello to world.

COPY

Copy files IN your build context to your docker container, without mounting those directories at runtime.

RUN

Execute a command in the shell during build time.

Examples: Install packages, bootstrap environments

CMD

Execute a command in the shell during startup time

Examples: Start a webserver, bring up a database

Inspect the Docker file

If you haven't cloned the github repo get the Dockerfile here:

wget https://gist.githubusercontent.com/jerowe/5de67ea90886472666ac905464e5fbdf/raw/e51c17afa1e6c70d7ef26c52163ef5dcae9a7f90/deploy-python-flask-docker-series-Dockerfile -O Dockerfile

You will notice that this Dockerfile is exactly the same as the Dockerfile in Part 1, with some additions. After we build our conda environment we copy over the rest of the flask_app directory. Finally, we call CMD to tell the container that at runtime we want to start our flask applications. We split building the environment and copying the files into two separate RUN and COPY commands to take advantage of docker's caching system. We don't want to have to rebuild the conda environment just because we made some changes to the flask app.

Build the docker image using docker build

In the next section I will show you my preferred method of building docker files using docker-compose, but its important to know what is happening under the hood.

docker build --rm -t flask_app:latest flask_app

Once you execute this command, you will see all manner of building going on, including adding packages.

Running the flask app

Now that we've built our application we can run it! We will tell docker to expose port 5000, and then we can test out our server from our host machine.

docker run -p 5000:5000 flask_app

If all went according to plan you will see some startup debug information. At the end of all that you will see the startup message from gunicorn.

[2019-01-19 08:00:03 +0000] [8] [INFO] Starting gunicorn 19.9.0
[2019-01-19 08:00:03 +0000] [8] [DEBUG] Arbiter booted
[2019-01-19 08:00:03 +0000] [8] [INFO] Listening at: http://0.0.0.0:5000 (8)
[2019-01-19 08:00:03 +0000] [8] [INFO] Using worker: sync
[2019-01-19 08:00:03 +0000] [11] [INFO] Booting worker with pid: 11
[2019-01-19 08:00:03 +0000] [12] [INFO] Booting worker with pid: 12
[2019-01-19 08:00:03 +0000] [8] [DEBUG] 2 workers

In the flask app we created a health endpoint that does nothing but spit back what we give it. Now we are posting the data 'hello' and 'world', and expecting that back. Try it out and see.

curl -X POST \
http://localhost:5000/health \
-H 'Cache-Control: no-cache' \
-H 'Content-Type: application/json' \
-d '{"hello": "world"}'

If you wish to kill the container to move onto the next step, press Ctrl+C.

Using File Mounts for Dev Work

When it comes time to deploy our application to production land we want all our files and data in the application. However, when we develop we want our application to update in real time. Gunicorn already knows to update in realtime by setting the --reload option.

docker run -p 5000:5000 -v flask_app:/home/flask/flask_app:Z flask_app

Now when we make changes to our application they will be reflected in the container, and gunicorn will know to restart the server.

I hope you can see what a powerful method of development this is. We can have our deployment behavior all ready, copy the flask application into the container, but continue to develop with updates made in real time, and not having to constantly rebuild the docker image.

Wrapping Up

That's all for now! Hopefully you have a good understanding of how to deploy your python application into a docker container, what tricks to use to run a web application from your container (ie exposing ports, running your webserver using CMD), and how to use file mounts to continue developing your application before deployment.

Bioinformatics Solutions on AWS Newsletter

Get the first 3 chapters of my book, Bioinformatics Solutions on AWS, as well as weekly updates on the world of Bioinformatics and Cloud Computing, completely free, by filling out the form next to this text.

Bioinformatics Solutions on AWS

If you'd like to learn more about AWS and how it relates to the future of Bioinformatics, sign up here.

We won't send spam. Unsubscribe at any time.