djangoappengine

Djangoappengine contains all App Engine backends for Django-nonrel, e.g. the database and email backends. In addition we provide a testapp which basically consists of a modified manage.py file to support the App Engine development server, and all corresponding settings in order to specify App Engine as the backend for Django-nonrel. Use it as a starting point if you want to use App Engine as your database for Django-nonrel. We've also published some details in the Django on App Engine blog post.

Take a look at the documentation below and subscribe to our Django-nonrel blog for the latest updates.

Documentation

Installation

Make sure you've installed the App Engine SDK. On Windows simply use the default installation path. On Linux you can put it in /usr/local/google_appengine. On MacOS it should work if you put it in your Applications folder. Alternatively, on all systems you can add the google_appengine folder to your PATH (not PYTHONPATH) environment variable.

Download the following zip files:

Unzip everything.

The django-testapp folder contains a sample project to get you started. If you want to start a new project or port an existing Django project you can just copy the manage.py, settings.py, and app.yaml files into your project and adapt settings.py and app.yaml to your needs.

Copy the following folders into your project (e.g., django-testapp):

  • django-nonrel/django
  • djangotoolbox/djangotoolbox
  • django-dbindexer/dbindexer
  • djangoappengine

That's it.

Alternatively, you can of course clone the respective repositories and create symbolic links instead of copying the folders to your project. That might be easier if you have a lot of projects and don't want to update each one manually.

Management commands

You can directly use Django's manage.py commands. For example, run manage.py createsuperuser to create a new admin user and manage.py runserver to start the development server.

Important: Don't use dev_appserver.py directly. This won't work as expected because manage.py runserver uses a customized dev_appserver.py configuration. Also, never run manage.py runserver together with other management commands at the same time. The changes won't take effect. That's an App Engine SDK limitation which might get fixed in a later release.

With djangoappengine you get a few extra manage.py commands:

  • manage.py remote allows you to execute a command on the production database (e.g., manage.py remote shell or manage.py remote createsuperuser)
  • manage.py deploy uploads your project to App Engine (use this instead of appcfg.py update)

Supported and unsupported features

Field types

All Django field types are fully supported except for the following:

  • ImageField
  • ManyToManyField

The following Django field options have no effect on App Engine:

  • unique
  • unique_for_date
  • unique_for_month
  • unique_for_year

Additionally djangotoolbox provides non-Django field types which you can use on App Engine or other non-relational databases. These are

  • ListField
  • BlobField

The following App Engine properties can be emulated by using a CharField in Django-nonrel:

  • CategoryProperty
  • LinkProperty
  • EmailProperty
  • IMProperty
  • PhoneNumberProperty
  • PostalAddressProperty

QuerySet methods

You can use the following field lookup types on all Fields except on TextField and BlobField

  • __exact equal to (the default)
  • __lt less than
  • __lte less than or equal to
  • __gt greater than
  • __gte greater than or equal to
  • __in (up to 500 values on primary keys and 30 on other fields)
  • __range inclusive on both boundaries
  • __startswith needs a composite index if combined with other filters
  • __year
  • __isnull (does not work on ForeignKeys)

Using django-dbindexer all remaining lookup types will automatically work too!

In addition you can use

  • QuerySet.exclude()
  • Queryset.values() (only efficient on primary keys)
  • Q-objects
  • QuerySet.count()
  • QuerySet.reverse()
  • ...

In all cases you have to keep general App Engine restrictions in mind.

Model inheritance only works with abstract base classes:

class MyModel(models.Model):
    # ... fields ...
    class Meta:
        abstract = True # important!

class ChildModel(MyModel):
    # works

In contrast, multi-table inheritance (i.e. inheritance from non-abstract models) will result in query errors. That's because multi-table inheritance, as the name implies, creates separate tables for each model in the inheritance hierarchy, so it requires JOINs to merge the results. This is not the same as multiple inheritance which is supported as long as you use abstract parent models.

Many advanced Django features are not supported at the moment. A few of them are:

  • JOINs
  • many-to-many relations
  • aggregates
  • transactions (but you can use transactions from App Engine's SDK if you really need them)
  • QuerySet.select_reated()

Other

In addition the following features from App Engine are not supported

  • entity groups (we don't yet have a GAEKeyField, but it should be trivial to add if you really need it)
  • batch puts

Indexes

It's possible to specify which fields should be indexed and which not. This also includes the possibility to convert a TextField into an indexed field like CharField. You can read more about this feature in our blog post Managing per-field indexes on App Engine.

Zip packages

Since you can't upload more than 3000 files on App Engine you sometimes have to create zipped packages. Luckily, djangoappengine can help you with integrating those zip packages. Simply create a "zip-packages" directory in your project folder and move your zip packages there. They'll automatically get added to sys.path.

In order to create a zip package simply select a Python package (e.g., a Django app) and zip it. However, keep in mind that only Python modules can be loaded transparently from such a zip file. You can't easily access templates and JavaScript files from a zip package, for example. In order to be able to access the templates you should move the templates into your global "templates" folder within your project before zipping the Python package.