We really weren't posting often enough recently. Now we'll make up for it with an advanced new asset manager called django-mediagenerator. Those of you who used app-engine-patch might still remember a media generator. This one is completely rewritten with a new backend-based architecture and muchos flexibility for the shiny new HTML5 web-apps world (see the feature comparison table). In this post I'll give you a quick intro and after that I'll make another post about some crazy stuff you can do with the media generator.
Why oh why?
What is an asset manager and why do you need one? Primarily, asset managers are tools for combining and compressing your JS and CSS files into bundles, so instead of many small files your website visitors only need to download a single big JS file and a single big CSS file. This is important because request latency has a much bigger impact on your site's load times than file size. You should definitely read Yahoo's Exceptional Performance and Google's Speed pages to learn more about how to improve your site's performance.
The second important task of an asset manager is to help you with handling HTTP caches. This is done by renaming your files, such that they contain a version tag. For example
main.css could be renamed to
main-efe88bad66a.css. Whenever the file's contents change the version tag is updated, so the browser will not use the cached version of your file, but download the updated file.
Django already has lots of existing solutions for managing your JS and CSS files and images, so why oh why do we make yet another one? Well, they don't provide the flexibility we need:
- Integration with Sass, PyvaScript, pyjs (the Python->JS compiler used in Pyjamas), etc.
- Flexible backend system for other converters (CleverCSS, etc.)
- Versioning for everything (including images)
- Support for image spriting
- Uncompressed and uncombined output during development with
runserverfor easier debugging
- Works in sandboxed environments like App Engine
Similar to django-compress, django-mediagenerator stores bundles in the file system. The bundles are defined in settings.py. Some people prefer to define bundles in their templates. Why don't we define bundles in templates?
- It doesn't work in sandboxed hosting environments like App Engine because all files have to be statically pre-generated in advance
- It can lead to unnecessary bundles and thus slow page loads if different pages have only slightly different scripts
- The configuration is not very flexible (you can only list a few JS/CSS files)
- It adds unnecessary checks to every request whether file contents have changed
Even if you'd say that these definitions belong into the templates the disadvantages are much bigger than that little increase in "comfortability".
Let's install it!
We tried our best to make the media generator easy to use while keeping it flexible. On the development server we provide a middleware for serving files. If you want to generate files for production you just run
manage.py generatemedia. We also provide two simple template tags for referencing media files in your templates.
So, let's install the media generator. Just download and extract the source code and run
setup.py install. Then, go to your project and edit your settings.py:
It's important that the middleware is the very first middleware in the list. Otherwise media files won't be cached correctly on the development server.
During development via
manage.py runserver media is served at
MediaMiddleware. The media generator stores production media in a
_generated_media folder. In production, the
_generated_media folder should be served from
PRODUCTION_MEDIA_URL by your web server.
MediaMiddleware is automatically disabled in this case, so Django will not serve any media in production. The
MEDIA_DEV_MODE setting specifies whether you're in development or production mode and whether to use development or production URLs (more on that in the templates section).
In the last line we've added the
static folder in the project's root directory to the media search path. The media generator will look for media files in that search path. Additionally, all
static folders in your
INSTALLED_APPS are added to the media search path. Only the admin app is removed, by default. Note that app-specific media should be placed in a subfolder:
app/static/app/.... This is very similar to Django templates which are placed in
Defining JS and CSS bundles
All required JS and CSS files must be explicitly listed via bundles in settings.py:
Bundles are defined as tuples where the first entry is the bundle name and the remaining entries list the file names that should be combined. Here, we have a
main.css bundle which combines
css/design.css. The second bundle is named
main.js and it combines
If you want to generate the media files for production you can just run
manage.py generatemedia. This will store all generated files and copy all images into the
_generated_media folder. Also, this creates a
_generated_media_names.py module which stores the mapping from unversioned file names to versioned file names.
You can tell the media generator to minify your JS and CSS files via YUICompressor in settings.py:
In your project you'll need a convention where to find the YUICompressor jar file. In this case we assume that it's in the project's parent folder and called
Let's add media to our templates
Now you know how to define bundles, but that's pretty useless if you can't reference those bundles from within your templates. Now this time I'll make it really simple. :)
In your template you first need to add the media generator template library via
Then you can include JS and CSS directly using e.g.:
This will automatically generate
<script> tags. In development mode (
MEDIA_DEV_MODE = True) your files are not combined, so this will generate multiple
<script> tags instead of a single one. This is very useful because your JS tracebacks will point directly to the file that caused an exception instead of a huge spaghetti code soup file (in that case only
grep can save you).
Optionally, you can also specify the CSS media type via:
Image URLs can be generated using e.g.:
URLs in CSS files
Whenever you write an URL via
url(some/relative/path...) in your CSS files the URL gets rewritten to the actual generated file name. This is only done with relative URLs. Absolute URLs stay untouched (e.g., those that start with
Example: If you have a CSS file in
css/style.css and you want to access
img/icon.png you would write
Sass is treated differently (we'll explain Sass in the next post). In Sass you'd write
url(img/icon.png). In other words, you always write the full media path to the file, without the leading
PRODUCTION_MEDIA_URL, of course. Why the difference? When you
@import a Sass module we lose all information about the imported code's original location. Thus, we can't support relative URLs like in CSS. In order to make your code easier to understand and more reusable we decided to use full paths, instead.
Web server cache settings
In order to get the maximum possible performance out of the media generator you have to configure proper caching for your
- Disable ETags because they cause unnecessary
Cache-Control: public, max-age=31536000
This will guarantee that your media files are only fetched once on the first visit. All subsequent requests will retrieve the media files from the browser's cache which will help make your website blazingly fast!
Installation on App Engine
Add the following handlers to your app.yaml:
If you use Django-nonrel that's all you need.
If you use some alternative Django setup (app-engine-patch, Django helper, etc.) you'll also need to add this at the top of your main.py handler (or whatever you've called your handler in app.yaml):
This will enable Python's
subprocess module which is needed by some media generator backends. Note that Django-nonrel uses a safer solution, but the code above is easier to integrate into the other Django helpers.
How to reload pages utilizing the browser cache
When you press CTRL+R or F5 in your browser all cached files get invalidated and refreshed. This can take a while if you have many media files. Since the media generator automatically takes care of versioning your media files even on the local development server you can use a more efficient technique to reload your pages: Click in the URL bar (or use CTRL+L) in your browser and press enter. That will reload the page without invalidating the cache. That way, only modified files will be reloaded. All other files will be retrieved from the cache.
Update: Current versions of Google Chrome have a bug in the preconnect handling code. Chrome creates multiple connections although the development server is only single-threaded. This causes lockups during reloads because one connection is blocking the development server while the other is trying to communicate with the blocked instance. You can fix this by starting Chrome with
--disable-preconnect. This way only one connection is created and the lockups are gone.
Now it's your turn
In the repository you can find a sample project with a little CSS example. If you have installed Django it should work out-of-the-box.
This post should get you started with the most common use-cases. There's a lot more that can be done with the media generator. We'll talk about the really exciting stuff in the next media generator post.
Update: The next post is Using Sass with django-mediagenerator