Thomas Wanschik on July 07, 2011

F() objects and QuerySet.update() support in djangoappengine

As some of you already noticed, we added supported for QuerySet.update() and F() objects as part of some features we currently need for our startup pentotype. Finally, it's possible to use simple transactions on App Engine via Django-nonrel :) Let's see how it works.

Django's beauty

In order to show what we gain by using Django's QuerySet.update() method and F() objects let me illustrate the code differences between the old djangoappengine version and the new one using a simple transaction which increments the value of a counter by a specific amount. In both cases we have the following model definition:

from django.db import models

class Accumulator(models.Model):
    counter = models.IntegerField()
    name = models.CharField(max_length=500)

The old version looks like this:

from google.appengine.ext import db

def increment_counter(pk, amount):
    obj = Accumulator.objects.get(pk=pk)
    obj.counter += amount

db.run_in_transaction(increment_counter, 1, 5)

whereas the new one looks like this:

from django.db.models import F
Accumulator.objects.filter(pk=1).update(counter=F('counter') + 5)

Obviously, it's more elegant to update a counter using QuerySet.update() and F() objects because it's much clearer and we have to write less code, thus resulting in more productivity.

Additionally, it's possible to do more complicated transactions, for example, updating multiple entities using Django's elegant syntax:

Accumulator.objects.filter(name__startswith='simple').update(counter=F('counter') + 1)

Note, that this will update each entity in its own transaction. With the old version we would have to iterate over each entity explicitly and start the transaction:

counters =  Accumulator.objects.filter(name__startswith='simple')

for counter in counters:
   db.run_in_transaction(increment_counter,, 5)

Again, we benefit from writing less code.

However, one of the main benefits of QuerySet.update() and F() objects is that the code becomes even more portable between backends than before. For example, the MongoDB backend also has support for F() objects and QuerySet.update(), so the same code can be used for transactional updates on App Engine and MongoDB. Moreover, the same code runs transactionally safe on SQL databases, too!

What's next

Support for QuerySet.update() and F() objects is the foundation of portable transactions over different databases. In combination with the django-dbindexer it now becomes possible to add support for automatically sharded counters as described in Sharding with Django on App Engine! In other words, we could add an index definition in order to tell django-dbindexer to automatically shard specific updates. Stay tuned!