TL;DR: In this article, you will see how to build and deploy a serverless Django API with Zappa.
Introduction
Serverless technology was developed from the idea of allowing developers to build and run applications without server management. Servers, of course, still exist, but developers and administrators don't need to manage them. This concept was heralded by AWS when it open-sourced its Serverless Application Model (SAM) framework. It then went on to release Chalice, a flask-like Python framework for building serverless applications. You can learn about building Chalice applications and APIs in this article.
With all the goodness that we are presented with, deploying serverless applications can be difficult when using frameworks like Django. Debugging in production environments can be hard too. Hence, there is a need for a way to deploy painlessly and also provide a means to debug and also monitor logs in production.
Zappa is an open-source Python tool created by Rich Jones of Gun.io for developers to create, deploy and manage serverless Python software on API Gateway and AWS Lambda infrastructure. Zappa allows you to carry out deployment with speed and ease in the most cost-efficient way possible.
In this article, you will learn how to develop serverless Django RESTful APIs, use Zappa to deploy them to AWS, and add authentication with Auth0.
Features and Benefits of Zappa
Some of the benefits provided by Zappa include:
- Automated deployment: With a single command, you can package and deploy your Python project as an AWS serverless app. You can also update and destroy your app with a single command in the different staging environments. You can even deploy your application in different AWS regions.
- No server maintenance: in compliance with the serverless ecosystem, Zappa takes away the maintenance of your servers from your list of tasks.
- Out-of-the-box Scalability: Zappa enables quick infrastructure scaling.
- Great Usability: Zappa provides a single settings file where you can do all your configuration as well as environment variables. It can be stored in JSON or YAML format.
- SSL certificates: Zappa provides a dedicated certify command which grants you an access to SSL certificates from different provides such as let's Encrypt and AWS
Using Zappa with Django
To use Zappa for your Django projects, ensure you have met the following prerequisites:
- Have a machine running Python 3.6, 3.7, or 3.8
- Can create a virtual environment with either venv or virtualenv
- Have an AWS account.
- Configured AWS credentials
- Have basic experience with Python and the Django framework
If you don't have any of the specified Python versions installed, you can download Python here. You can use Venv, Virtualenv, or Pyenv to create virtual environments. If you do not have an AWS account yet, you can sign up for free. You may follow these instructions to configure your AWS credentials.
Building a Django RESTful API
Companies need an employee management system to keep details about each employee. It could be in a form of web application where new staff could be added or removed. What's more, staff details should also be presented.
An employee would have attributes like:
- name
- age
- salary
- location
Hence, in this section, you will create an API that could do the following actions:
- List all employees
- Show the details of an employee
- Add a new employee
- Edit the details of an employee
- Delete an employee
The endpoints could then be as follows:
- List all employees:
GET /api/employees
- Show the details of an employee:
GET /api/employees/{id}
- Add a new employee:
POST /api/employees
- Edit the details of an employee:
PUT /api/employees/{id}
- Delete an employee:
DELETE api/employees/{id}
Installing Dependencies
Create a folder where the files of this project will reside. Name it django_zappa
or any other name. Also, navigate to the new folder:
mkdir django_zappa && cd django_zappa
Create a new virtual environment and activate it:
python -m venv env
source env/bin/activate
Use pip
to install the needed dependencies on the CLI as shown below:
pip install Django==2.1.9 djangorestframework==3.10.0 zappa
We used the Pip package manager that comes pre-installed with Python to install the following packages:
Django
: version 2.1.9 of the Django package. Installing this earlier version helps avoid an SQLite version conflict that could arise when deploying Zappadjangorestframework
: the version 3.10.0 of the Django REST framework package for creating APIszappa
: Zappa package for deployment
Scaffolding the Project
Now, you will create the Django project and application. Use the django-admin
command to create a new Django project called company
:
django-admin startproject company
Then, create a new app called api
inside the company
parent directory. This app will contain the API code:
cd company
django-admin startapp api
Next, open the settings.py
file of the Django project and add rest_framework
and api
to the list of installed apps.
# company/company/settings.py
...
INSTALLED_APPS = [
...
'rest_framework',
'api',
]
...
Creating the Model, Database Migrations and Serializer
Here, you will create a model for employee API that will determine which employee details will be saved in the database. Go to the models.py
file inside the api
application directory and add the following code:
# company/api/models.py
from django.db import models
class Employee(models.Model):
name = models.CharField(max_length=225)
email = models.EmailField()
age = models.IntegerField()
salary = models.CharField(max_length=60)
location = models.CharField(max_length=125)
Next, you need to create migrations to seed the model into the database:
python manage.py migrations
python manage.py migrate
Then, you will create a serializer that will allow Django views to return an appropriate response to the users' requests.
Create a new serializers.py
file in the api
directory and add the following code:
# company/api/serializers.py
from rest_framework import serializers
from api.models import Employee
class Serializer(serializers.ModelSerializer):
class Meta:
model = Employee
fields = '__all__'
Creating Views and URLs
You need to make views that will handle the logic of the HTTP actions when users make requests to the API endpoints. The views will also interact with models via the serializer.
Go to the views.py
file of the api
directory and add the following code:
# company/api/views.py
from rest_framework import generics
from api.models import Employee
from api.serializers import EmployeeSerializer
class EmployeeList(generics.ListCreateAPIView):
queryset = Employee.objects.all()
serializer_class = EmployeeSerializer
class EmployeeDetail(generics.RetrieveUpdateDestroyAPIView):
queryset = Employee.objects.all()
serializer_class = EmployeeSerializer
The code above has 2 views. The first one, called EmployeeList
lists all employees and also allows the creation of a new employee. The other view, EmployeeDetail
, allows the creation, retrieve, update, and deletion of particular employees. It accepts a single model instance.
Next, create URLs for the views respectively. Navigate to the urls.py
file in the project sub-directory and add the following code:
# company/api/urls.py
...
from django.urls import path
from api.views import EmployeeList, EmployeeDetail
urlpatterns = [
...
path('api/employees/', EmployeeList.as_view()),
path('api/employees/{id}', EmployeeDetail.as_view()),
]
In the code above, you added paths to the two views created earlier. The URLs were also defined as api/employees/
and api/employees/{id}
respectively. The second URL accepts the id
as an integer parameter, which is a primary key, int:pk
. This is for the single model instance to be fetched when requests are made at the endpoint.
Now, the API has been successfully created. Next, Zappa will be used to deploy the API to AWS Lambda.
Testing Locally
Use the runserver
command in your terminal to start the built-in Django server so you can access the API in your browser.
python manage.py runserver
Navigate to127.0.0.1:8000
in your browser. You should see a page that shows a list of accessible endpoints like this:
If you navigate to api/employees
endpoint at 127.0.0.1:8000/api/employees, you will see the Django REST framework browsable API:
You have confirmed that the API works well in your local environment.
Deploying with Zappa
Before you deploy the Django API with Zappa, you have to initialize Zappa in the project:
zappa init
When you ran zappa init
, you should get a command-line output that looks like the following:
███████╗ █████╗ ██████╗ ██████╗ █████╗
╚══███╔╝██╔══██╗██╔══██╗██╔══██╗██╔══██╗
███╔╝ ███████║██████╔╝██████╔╝███████║
███╔╝ ██╔══██║██╔═══╝ ██╔═══╝ ██╔══██║
███████╗██║ ██║██║ ██║ ██║ ██║
╚══════╝╚═╝ ╚═╝╚═╝ ╚═╝ ╚═╝ ╚═╝
Welcome to Zappa!
Zappa is a system for running server-less Python web applications on AWS Lambda and AWS API Gateway.
This init command will help you create and configure your new Zappa deployment.
Let's get started!
Your Zappa configuration can support multiple production stages, like 'dev', 'staging', and 'production'.
What do you want to call this environment (default 'dev'):
Follow the prompts to name your staging environment and private S3 bucket where the project files will be stored.
- It will prompt you to request whether you want to deploy globally or not.
- Then, it will detect your application type as Django.
- It will also locate the settings file as
company.settings
. - Finally, it will create a
zappa_settings.json
file in your project directory.
By the end of the prompts, you should get an output like this:
Done! You can also deploy all by executing:
$ zappa deploy --all
After that, you can update your application code with:
$ zappa update --all
To learn more, check out our project page on GitHub here: https://github.com/Miserlou/Zappa
and stop by our Slack channel here: https://zappateam.slack.com
Enjoy!,
~ Team Zappa!
The zappa_settings.json
file is crucial for your deployment because it contains the deployment settings.
{
"dev": {
"aws_region": "us-west-2",
"django_settings": "company.settings",
"profile_name": "default",
"project_name": "company",
"runtime": "python3.7",
"s3_bucket": "zappa-xxxxxxxxx"
}
The zappa_settings.json
file may, however, contain the information for other regions if you choose global deployment while initializing Zappa.
You will deploy your project with the zappa deploy
command where stage-name could be dev
or any other stage name you use when initializing Zappa:
zappa deploy stage-name
Upon successful deployment, you should get a URL where you can access your API on the internet. It should look like this:
https://5vbm7e4hn2.execute-api.us-west-2.amazonaws.com/dev
Copy the URL generated for your application and add it to the list of ALLOWED_HOSTS
inside the settings.py
file of the project:
# company/company/settings.py
...
ALLOWED_HOSTS = ['127.0.0.1', '9dgf4q6wx8.execute-api.us-west-2.amazonaws.com',]
...
Then, update the deployment:
zappa update dev
Keep in mind that you need to update the deployment with the command above when making any changes in your project.
Serving Static Files
At this point, you may need to serve static files so that default Django styles can be active in the deployed stage.
Creating an S3 bucket
- Go to the Amazon S3 console and select
Create bucket
- Give your bucket a unique name. The name must start with a lowercase character or number. It must not contain an uppercase character. The length should be between 3 and 63 characters.
Note that you can't change a bucket name after creating it.
- Select the AWS region where you want your bucket to be hosted.
- Under the Bucket settings for Block Public Access, make sure you uncheck Block all public access
- Note the name of the S3 bucket that you created
Go to the Permissions
tab of your S3 bucket and navigate to the Cross-Origin resource sharing (CORS) section. Click the Edit
button and add the following configuration:
[
{
"AllowedHeaders": ["*"],
"AllowedMethods": ["GET"],
"AllowedOrigins": ["*"],
"MaxAgeSeconds": 3000
}
]
Configuring Django Settings for Handling Static Files
Install django-s3-storage
library for Django to work with S3:
pip install django-s3-storage
Next, open the settings.py
file and add the following:
# company/company/settings.py
import os
...
STATIC_ROOT = os.path.join(BASE_DIR, 'staticfiles')
S3_BUCKET_NAME = "put-the-name-of-the-bucket-you-created-here"
STATICFILES_STORAGE = "django_s3_storage.storage.StaticS3Storage"
AWS_S3_BUCKET_NAME_STATIC = S3_BUCKET_NAME
# to serve the static files from your s3 bucket
AWS_S3_CUSTOM_DOMAIN = '%s.s3.amazonaws.com' % S3_BUCKET_NAME
STATIC_URL = "https://%s/" % AWS_S3_CUSTOM_DOMAIN
# if you hosted your static files on a custom domain
#AWS_S3_PUBLIC_URL_STATIC = "https://static.yourdomain.com/"
Now, update the deployment with the static files.
zappa update dev
zappa manage dev "collectstatic --noinput"
The browsable API should render with its styles appropriately like below:
Adding Authentication with Auth0
You need an Auth0 account to use Auth0 APIs for authenticating your Django APIs with Auth0. You can create an Auth0 account if you do not have one.
Click the Create API button on your Auth0 dashboard to create an API. Then, go to the Permissions
tab of the newly created API to allow read access.
Now, install the libraries needed for authentication:
pip install cryptography==2.8 django-cors-headers==3.1.1 drf-jwt==1.13.3 pyjwt==1.7.1 requests==2.22.0
Next, open the settings.py
file of the project and add the ModelBackend
and the RemoteUserBackend
to the list of Authentication backends.
# company/company/settings.py
AUTHENTICATION_BACKENDS = [
'django.contrib.auth.backends.ModelBackend',
'django.contrib.auth.backends.RemoteUserBackend',
]
The code above allows Django to link the Django users database with your Auth0 users database.
Next, update the middleware in the settings.py
file as thus:
# company/company/settings.py
MIDDLEWARE = [
...
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.auth.middleware.RemoteUserMiddleware',
]
The RemoteUserMiddleware
middleware connects the user in the Auth0 Access Token to the user in the Django authentication system. The AuthenticationMiddleware
handles authentication. Make sure you add the AuthenticationMiddleware
after the SessionMiddleware
to avoid an AssertionError
.
Next, navigate to the api
application directory and create a new file called utils.py
. Then, add the following:
# company/api/utils.py
from django.contrib.auth import authenticate
def jwt_get_username_from_payload_handler(payload):
username = payload.get('sub').replace('|', '.')
authenticate(remote_user=username)
return username
The code above consists of a function jwt_get_username_from_payload_handler
that accepts the authentication payload
, which is the Access Token. It then maps the sub
field from the Access Token to the username variable. The authenticate
method imported from the RemoteUserBackend
creates a remote user in the Django authentication system. Then, a User object is returned for the username.
Next, navigate back to the settings.py
file of the project and IsAuthenticated
to the DEFAULT_PERMISSION_CLASSES
. Also, add the JSONWebTokenAuthentication
to the DEFAULT_AUTHENTICATION_CLASSES
like this:
# company/company/settings.py
REST_FRAMEWORK = {
'DEFAULT_PERMISSION_CLASSES': (
'rest_framework.permissions.IsAuthenticated',
),
'DEFAULT_AUTHENTICATION_CLASSES': (
'rest_framework_jwt.authentication.JSONWebTokenAuthentication',
'rest_framework.authentication.SessionAuthentication',
'rest_framework.authentication.BasicAuthentication',
),
}
In the code above, we have defined settings for permissions, i.e., what can be accessed after authentication. We also defined the type of authentication that we want as JWT.
Next, set the JWT_AUTH
variable for the JWT authentication library in Django.
Add the following code to your settings.py
file. Make sure you replace the JWT_AUDIENCE
with your Auth0 API identifier and the JWT_ISSUER
with your Auth0 domain.
# company/company/settings.py
JWT_AUTH = {
'JWT_PAYLOAD_GET_USERNAME_HANDLER':
'auth0authorization.utils.jwt_get_username_from_payload_handler',
'JWT_DECODE_HANDLER':
'auth0authorization.utils.jwt_decode_token',
'JWT_ALGORITHM': 'RS256',
'JWT_AUDIENCE': 'YOUR_AUTH0_API_IDENTIFIER',
'JWT_ISSUER': 'https://YOUR_AUTH0__DOMAIN/',
'JWT_AUTH_HEADER_PREFIX': 'Bearer',
}
Next, navigate to the utils.py
file and add a function to get the JSON Web Token Key Sets (JWKS) from your dashboard to verify and decode Access Tokens.
# company/api/utils.py
...
import json
import jwt
import requests
def jwt_decode_token(token):
header = jwt.get_unverified_header(token)
jwks = requests.get('https://{}/.well-known/jwks.json'.format('YOUR_AUTH0__DOMAIN')).json()
public_key = None
for jwk in jwks['keys']:
if jwk['kid'] == header['kid']:
public_key = jwt.algorithms.RSAAlgorithm.from_jwk(json.dumps(jwk))
if public_key is None:
raise Exception('Public key not found.')
issuer = 'https://{}/'.format('YOUR_AUTH0__DOMAIN')
return jwt.decode(token, public_key, audience='YOUR_AUTH0_API_IDENTIFIER', issuer=issuer, algorithms=['RS256'])
Next, create two methods in the views.py
file of the api
application to check the scopes granted from the access_token
:
# company/api/views.py
...
from functools import wraps
import jwt
from django.http import JsonResponse
def get_token_auth_header(request):
"""Obtains the Access Token from the Authorization Header
"""
auth = request.META.get("HTTP_AUTHORIZATION", None)
parts = auth.split()
token = parts[1]
return token
def requires_scope(required_scope):
"""Determines if the required scope is present in the Access Token
Args:
required_scope (str): The scope required to access the resource
"""
def require_scope(f):
@wraps(f)
def decorated(*args, **kwargs):
token = get_token_auth_header(args[0])
decoded = jwt.decode(token, verify=False)
if decoded.get("scope"):
token_scopes = decoded["scope"].split()
for token_scope in token_scopes:
if token_scope == required_scope:
return f(*args, **kwargs)
response = JsonResponse({'message': 'You don\'t have access to this resource'})
response.status_code = 403
return response
return decorated
return require_scope
In the code above, the get_token_auth_header
method gets the Access token from the authorization header while the requires_scope
checks if the Access Token obtained contains the scope required to allow access to some specific parts of the application.
Now, you can protect endpoints by adding the authentication_classes
and permission_classes
decorators to methods that need authentication and permission scopes, respectively.
# company/api/views.py
from rest_framework import permissions
from rest_framework_jwt.authentication import JSONWebTokenAuthentication
...
rest_framework_jwt.authentication.JSONWebTokenAuthentication
class EmployeeList(generics.ListCreateAPIView):
authentication_classes = [authentication.JSONWebTokenAuthentication]
permission_classes = [permissions.IsAuthenticated]
...
Now, your application has Auth0 authentication enabled. Update the deployment for the authentication to be active:
zappa update dev
Conclusion
This tutorial has taken you through the process of building Django RESTful APIs and deploying them as serverless applications on AWS Lambda with Zappa. You also learned how to configure authentication using Auth0. Now, you can go on and use the knowledge gained in your projects building APIs and deploying serverless Python applications.
Let us have your suggestions and questions in the comments section below. Thanks.