In this post I’ll describe the required steps to configure a Django Pyhton Application for Cloud Foundry.

Cloud Foundry

Cloud Foundry (CF) is an open source Platform as a Service (PaaS) solution developed mainly by the company Pivotal. Cloud Foundry is controlled by the Cloud Foundry Foundation where, among others,EMC, VMware, IBM and SAP are members.
Similar to Open Stack there are commercial products of Cloud Foundry like Pivotal Web Services, Pivotal CF and IBM Bluemix.

Pivotal Web Services

For my tests I used the Pivotal Web Services (PWS). It is Pivotal’s public cloud offering for Cloud Foundry. PWS is running the elastic runtime on Amazon Webservices (AWS) and services in the marketplace are provided by other public cloud offerings. Service are things like databases, PostgresSQL provided by ElephantSQL, application monitoring, New Relic, message broker, RabbitMQ provided by CloudAMQP, and more:

You can get a 60 Day free trial for PWS. All of the above services also offer a free service plan for limited usage. For my example I leveraged ElephantSQL and New Relic. How to use New Relic will be described in a later post.

Python Buildpack

Buildpacks provide framework and runtime support for your applications. Currently Cloud Foundry provides buildpacks for Go, Java, Node.js, PHP, Ruby and Python. All buildpacks, beside the Java one, are based on Heroku buildpacks. Heroku is also a platform as a service (PaaS) but not based on Cloud Foundry. It is also possible that you create your own buildpack or e.g. use one that is shared on GitHub.

Environment Variables

The biggest difference between CF and Heroku is, which made me struggle, how they differently handles environment variables to inject information, like database connection information, into the container at runtime.
For example does Heroku add an environment variable (config var) “DATABASE_URL” for the database connection string (e.g. “postgres://foo:foo@heroku.com:5432/hellodb”) or “NEW_RELIC_LICENSE_KEY” for the new relic license key.
In contrary does Cloud Foundy add an environment variable “VCAP_SERVICES”, which is formatted in JSON, and stores all information for connected services. Cloud Foundry also offers the ability to create custom environment variables additionally . These variables can store any kind of strings.
Here is an example for a VCAP_SERVICES variable containing information for PostgreSQL and New Relic:

{  
   "elephantsql":[  
      {  
         "name":"name",
         "label":"elephantsql",
         "tags":[  
            "New Product",
            "Data Stores",
            "relational",
            "Data Store",
            "postgresql"
         ],
         "plan":"turtle",
         "credentials":{  
            "uri":"postgres://rccmcbbt:f28JL8sfsdfdsfsdsezqZm4xk@babar.elephantsql.com:5432/rccmcbbt",
            "max_conns":"5"
         }
      }
   ],
   "newrelic":[  
      {  
         "name":"newrelic",
         "label":"newrelic",
         "tags":[  
            "Monitoring"
         ],
         "plan":"standard",
         "credentials":{  
            "licenseKey":"7251234dd1d1a3453dg90f5879e2d823"
         }
      }
   ]
}

There has to be a “but” – it seems there is one exception in Cloud Foundry. Beside the “VCAP_SERVICES” environment variable CF is creating a “DATABASE_URL” variable based on the “VCAP_SERVICES” one. Sadly I haven’t found any documentation for the Python buildpack but the variable is mentioned in the documentation for the Ruby buildpack.
In a later post I will describe how to handle the missing “NEW_RELIC_LICENSE_KEY” for the New Relic configuration.

Configuration files

The Python buildpack needs a couple of files to be present in the root folder of your Python application. These are namely: “runtime.txt”, “requirements.txt”  and “Procfile”

The “runtime.txt” specifies which version of Python should be loaded. A list of supported runtimes can be found on GitHub.

runtime.txt:

python-3.4.3

The “requirements.txt” specifies which Python models should be installed by “pip”. The content of this file can be generated by using

pip freeze > requirements.txt

requirements.txt:

Django==1.7.1
WhiteNoise==1.0.4
gunicorn==19.1.1
psycopg2==2.5.4
dj-database-url==0.3.0

The most obvious package we are loading is Django, our Python Web framework. WhiteNoise is used to serve static files and Gunicorn is used as our Python WSGI HTTP Server. For the Python PostgreSQL adapter I use Psycopg2. And finally dj-database-url is used to simplify the usage of the “DATABASE_URL” environment variable in the Django “settings.py”.
If you want to use a MySQL database you will have to replace psycopg2 with mysqlclient “mysqlclient==1.3.4” (supports Python 3).

The third and last file “Procfile” is used to define what should be started in the instance.

Procfile:

web: gunicorn myapp.wsgi --workers 2

This “Procfile” is going to start the python application “myapp” using gunicorn with two workers. “myapp”  will have to be replaced by the name of your application.

Django configuration for Cloud Foundry

Before we are able to publish our django application to Cloud Foundry we have to modify “settings.py” to use the “DATABASE_URL” for the database configuration and the “static” variables to use WhiteNose for serving our static files.

settings.py:

import dj_database_url
DATABASES = {'default': dj_database_url.config()}

STATIC_URL = '/static/'
STATIC_ROOT = os.path.join(BASE_DIR, 'staticfiles')
STATICFILES_STORAGE = 'whitenoise.django.GzipManifestStaticFilesStorage'

The function “dj_database_url.config()” loads the “DATABASE_URL” and transforms it into the django format for configuring the database setttings.
Please note that the buildpack will automatically run “manage.py collectstatic” to collect the static files from all of your applications.

“wsgi.py” has to be modified to use WhiteNose for serving our static files. Please make sure that you replace “myapp.settings” with the name of your application.

wsgi.py:

import os
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "myapp.settings")

from whitenoise.django import DjangoWhiteNoise
from django.core.wsgi import get_wsgi_application

application = get_wsgi_application()
application = DjangoWhiteNoise(application)

Once the application is deployed to Cloud Foundry, which I’ll show in a minute, we need to create the database tables and the admin user. On heroku you can simply use the heroku cli to run any command in an “One-Off Dyno“. Cloud Foundry currently does not support a way to run a shell command in an easy way. The only option is to run once a custom script, instead of using the “Procfile”, the first time you push your application to Cloud Foundry.

init_db.sh:

#!/bin/sh
echo "------ Create database tables ------"
python manage.py migrate --noinput

echo "------ create default admin user ------"
echo "from django.contrib.auth.models import User; User.objects.create_superuser('admin', 'admin@myapp.local', 'Passw0rd')" | python manage.py shell

echo "------ starting gunicorn  ------"
gunicorn myapp.wsgi --workers 2

This short bash script will create the database tables and create a default admin user with password ‘Passw0rd’

Push Django App to Cloud Foundry

The last and fun part is to push the application to Cloud Foundry. To be able to push an application easier we create a manifest file which describes the application.

manifest.yml:

---
applications:
- name: cf-dango
  instances: 1
  memory: 128M
  command: null
  services:
  - cf-django-pgsql

We give our application a name, how many instances we want to start, its memory requirement and lastly the name of our database service which we want to bind.

Finally we are ready to push our application using the Cloud Foundy CLI.

Commands:

cf login -a https://api.run.pivotal.io
cf create-service elephantsql turtle cf-django-pgsql
cf push --no-route -c "bash ./init_db.sh"
cf push

With the first command we login into the Pivotal Web Services. The command will ask you for your username and password. Next we create our PostgreSQL database. “elephantsql” is the name of the PostgreSQL service (see the picture of the service marketplace) and “turtle” is the name of the free plan.
If you deploy your app for the first time you will have to run our “init_db.sh” script once and then push your app again but without specifying the script.

Now your django application should successfully run on Cloud Foundry!!!

In one of the next posts I will describe the required steps to uses New Relic on Pivotal Web Services with Python.

Example App on GitHub

I published a very simple django app on GitHub, called “cf-vLab-event“. This app is already modified to run on Cloud Foundry and could help you as a starting point. If you successfully deployed the app to Pivotal Web Services it should look like this: