Django tips: laying out an application

An entry published by James Bennett on September 10, 2006, Part of the category Django. 13 comments posted.

Continuing the theme of dealing with common questions from the Django mailing lists and IRC channel, today we’ll look at how to organize the various bits of a Django-based project or application.

Projects versus applications

This is really more of a separate (though related) question, but understanding the distinction Django draws between a “project” and an “application” is a big part of good code layout. Roughly speaking, this is what the two terms mean:

An application tries to provide a single, relatively self-contained set of related functions. An application is allowed to define a set of models (though it doesn’t have to) and to define and register custom template tags and filters (though, again, it doesn’t have to).
A project is a collection of applications, installed into the same database, and all using the same settings file. In a sense, the defining aspect of a project is that it supplies a settings file which specifies the database to use, the applications to install, and other bits of configuration. A project may correspond to a single web site, but doesn’t have to — multiple projects can run on the same site. The project is also responsible for the root URL configuration, though in most cases it’s useful to just have that consist of calls to include which pull in URL configurations from inidividual applications.

Views, custom manipulators, custom context processors and most other things Django lets you create can all be defined either at the level of the project or of the application, and where you do that should depend on what’s most effective for you; in general, though, they’re best placed inside an application (this increases their portability across projects).

Default project-level file layout

When you run django-admin.py startproject, Django will automatically create a new directory containing four files:

__init__.py, which will be empty. This file is required to tell Python that the directory is a Python module and can be imported (and imported from).
manage.py, which provides a number of convenience functions for working with the project.
settings.py, which will be the project’s settings file.
urls.py, which will be the project’s root URL configuration.

Generally you don’t need to modify this layout, and for compatibility and consistency it’s probably best if you don’t. If you do want to change things, though, here’s what it’s safe to do:

You can put your settings in a file that’s not called settings.py — Django finds out where your settings are by looking at the environment variable DJANGO_SETTINGS_MODULE, not by looking for a file with a specific name.
You can put your root URL configuration somewhere that’s not called urls.py — Django looks at the ROOT_URLCONF setting to find out where your URL configuration lives.

manage.py and __init__.py should be left alone.

Default application-level file layout

When you run manage.py startapp, Django creates a sub-directory of your project directory, and creates the following files:

__init__.py, which serves the same purpose as at the project level.
models.py, which should hold the application’s model classes.
views.py, which is for any custom views the application wants to provide.

The __init__.py and models.py files (or, if you want to split up your models across multiple files, a directory called models which can act as a Python module) are required; without __init__.py, Python won’t be able to import from the application, and Django is hard-wired to expect models in a file or module called models. The views.py file, however, is optional and you can delete it if you won’t be providing any views, or rename it if you want to call it something else (though for sake of consistency it’s probably best not to rename it).

Extra special stuff

There are four “special” locations inside your application which can be used in order to take advantage of specific features, so if you want to use these features you don’t have a whole lot of choice in how you set them up:

To define custom template tags or filters, you must create a sub-directory in the application’s directory called templatetags, and it must contain a file named __init__.py so that it can be imported as a Python module.
To define unit tests which will automatically be noticed by Django’s testing framework, put them in a module called tests (which can be either a file named tests.py or a directory called tests). The testing framework will also find any doctests in that module, but the preferred place for those is, of course, the docstrings of the classes or functions they’re designed to test.
To provide custom SQL which will be executed immediately after your application is installed, create a sub-directory called sql inside the application’s directory; the file names should be the same as the names of the models whose tables they’ll operate on; for example, if you have an app named weblog containing a model named Entry, then the file sql/entry.sql inside the app’s directory can be used to modify or insert data into the entries table as soon as it’s been created.
To provide custom Python functions which will run when the application is installed, put them in a file named management.py, and use Django’s internal dispatcher to connect your functions to the post_syncdb signal.

That last one deserves a bit more explanation, so let’s look at it in detail.

Internally, Django uses a package called PyDispatcher to enable its various bits to communicate cleanly with each other. Basically, the dispatcher works like this:

Various parts of Django, as well as other applications, define plain objects called “signals”.
Code which wants other things to be notified of something happening tells the dispatcher to send a particular signal.
Code which wants to be notified when something happens uses the dispatcher’s connect method to listen for a particular signal.

For example, if you wanted to set up a function which would execute any time a new application is installed, you could create a file in your application called management.py, and put this code in it:

from django.dispatch import dispatcher
from django.db.models import signals
 
def my_syncdb_func():
    # put your code here...
    
dispatcher.connect(my_syncdb_func, signal=signals.post_syncdb)

Several of the applications bundled with Django use this trick to do various things:

django.contrib.sites listens to find out when it’s been installed, and creates the default “example.com” site object, which is needed for the admin to function.
django.contrib.contenttypes listens for any new apps being installed, and creates new ContentType instances for all the models being installed.
django.contrib.auth listens on two fronts: when the auth app is installed, it prompts you to create a superuser, and when any new app is installed, it creates permissions for that app’s models.

This works because the syncdb function in manage.py imports the management files from all the installed and soon-to-be-installed apps in your project; that makes sure any app which needs to can take advantage of the dispatcher.

Other useful conventions

Of course, that doesn’t cover all the things you might want to do within Django, so naturally people end up wondering how they should organize the rest of their code. Generally there’s no need to require standardized layouts or locations for certain functions, but it can be helpful to adhere to a few conventions. So to provide an example, here’s how I generally lay out any additional bits I need in things that I’m working on.

At the project level, I generally don’t add a whole lot; it’s rare that I need to do something that doesn’t make more sense as part of an application. However, if you’re going to be running multiple projects which all use some or all of the same applications, it can be useful to organize your settings carefully. Jacob has, once or twice, pointed out the trick that we use at the Journal-World, and I think it’s fairly useful: we have our code base in one directory structure, and settings files in another, with a “default” settings file at the top level of the settings tree. Settings files for individual sites import the default settings and override anything they need to, or add any extra settings they require. Because Django doesn’t require settings files to live in the same directory tree as the applications their projects use, this is extremely easy and extremely useful to do.

At the application level, I usually drop in a few more files depending on exactly what the application is going to be using:

If the application defines any custom manipulators, I put them in a file called forms.py instead of in the views file.
If there are multiple custom managers in the app, I put them in a file called managers.py instead of the models file.
If I’m defining any custom context processors, I put them in a file called context_processors.py.
If I’m setting up any custom dispatcher signals, they go in a file called signals.py.
If the application is setting up any syndication feeds, the feed classes go in a file called feeds.py. Similarly, sitemap classes go in sitemaps.py.
Middleware classes go in a file called middleware.py.
Any miscellaneous code which doesn’t clearly go anywhere else goes in a file or module called utils.

I don’t always use all of that functionality in an app, but if I do it’s nice to have a convention for it, so that I only need to remember to do from appname import forms or from appname import feeds.

And one more thing…

This is something I’m still in the process of developing, but I also like to have an easy way to test the dependencies my applications have, and to make sure everything is configured properly. The easiest way to do this, in my experience, is to put some code in the application’s __init__.py which checks for everything the application will need. I wrote up a simple example in a post on the Django developers list, and I’m still working on improving that; there are functions tucked away in django.core.management which will let you test not only whether an application can be imported and is listed in the INSTALLED_APPS setting, but whether it’s actually been installed into the database.

How do you do it?

I think I’ve covered everything I know of, and every trick I use, for laying out a Django project or application cleanly; if you see something here you like, feel free to start using it. And if I’ve overlooked something cool that you know of, please post it in a comment and let the world know about it :)

← Django tips: documentation and resources Dear John →

On September 11, 2006, Dagur said:

Good post! But I have one question. Have you been in the situation that your views.py file is so big that you wanted to split it into more files? If so, how did you do it?

On September 11, 2006, Joseph Rose said:

I was wondering if I could call the settings file whatever I wanted to. How do I specify this when running manage.py scripts?

On September 11, 2006, janders said:

Great article, thanks. However, the settings file(s) for 1) multiple projects for a single site, or, 2) single project with multiple sites never seems to be concretely laid out in the various articles that I read. What are the specifics for these two cases? Where do the settings files go in the directory? What are their names? How does this tie into the Sites functionality.

On September 11, 2006, Brice said:

Joseph: you would do the following : from django.conf import settings

On September 11, 2006, Jay said:

Dagur, I do this all the time. Most recently I have a views.py file for regular views and then a jsviews.py for all views responsible for ajax stuff, like returning json. Just create you new views.py file named whatever you want and reference that in your urls.py file instead of the standard views.py

Example: (r’^widget/search/$’, ‘myproject.widget.views.search’), (r’^widget/(S+)/$’, ‘myproject.widget.jsviews.view_widget’),

On September 11, 2006, Tom von S. said:

Joseph:

I’m a Django rookie, but for one project I created a file called devsettings.py and a script called development.sh. The development.sh script looked something like this:

#!/bin/bash

python manage.py —settings=devsetttings “$@”

I would then just use that script to run manage.py against the devsettings.py file:

./development.sh runserver

On September 11, 2006, James Bennett said:

Dagur, if you want to keep the simplicity of doing myproject.myapp.views but still have the convenience of multiple files, put them in a directory called views (making sure it contains an __init__.py file) and delete views.py — then you can import from files inside the views directory just as you’d import from any Python module.

Joseph, just use the —settings option for manage.py to specify where your settings file is.

On September 11, 2006, Alan Green said:

I’ve used the post_syncdb signal in all my projects, but was never sure whether it was a part of the Django API or just a convenient implementation side-effect. Thanks for clearing it up a little.

On September 12, 2006, Joe Murphy said:

Note — on #3, under “Extra Special Stuff,” the filename “sql/Entry.sql” oughta be “sql/entry.sql”

On September 19, 2006, bora said:

?????????? XD

On September 19, 2006, bora said:

chinese? http://forum.ubuntu.org.cn/weblogs/upload/4/225696793450fe4e044621.png

On September 26, 2006, Lee Causier said:

In response to Tom Von S., I thought I’d share this. For the ulcompsoc site, I wrote a little script called “manage_wrapper.py” (that seems to work on both Windows and Linux) for running manage.py:

#!/usr/bin/python
import sys, os, socket
def dowrap(appdir, hostname, moreargs):
    settingsmodule = “ulcompsoc.%s_settings” % hostname
    os.chdir(appdir)
    os.environ[‘PYTHONPATH’] = os.pathsep.join([“ulcompsoc”, “third-party”, “third-party/docutils-extras”])
    os.environ[‘DJANGO_TEMPLATE_PATH’] = (“%s/ulcompsoc-templates” % appdir)
    arglist = [sys.executable, ‘%s/ulcompsoc/manage.py’ % appdir, ‘—settings=%s’ % settingsmodule] + moreargs
    os.spawnv(os.P_WAIT, sys.executable, arglist)

if (__name__ == “__main__”):
   dowrap(os.path.dirname(sys.argv[0]), socket.gethostname(), sys.argv[1:])

and, another called “run.py” for “running” it intelligently - eg from Windows Explorer, or from Pida’s Project Execute option:

#!/usr/bin/python
import sys, os, socket, manage_wrapper
appdir = os.path.dirname(sys.argv[0])
hostname = socket.gethostname()
manage_wrapper.dowrap(appdir, hostname, [‘runserver’, ‘%s:8080’ % hostname])

Hope this helps :)

On October 7, 2006, Paul said:

For some projects I use one pattern that I don’t think is mentioned above — for lack of a better term I’ll call it app-as-project.

It’s really only good for sites that are not using apps from other sources (and not likely to yield apps that you’ll want to extract). I add the project name to INSTALLED_APPS, create models.py and views.py right there in the project directory, and use one urls.py for all URLs on the site. For templates, you only need the app_directories loader, with a single templates directory right there in the project directory.

I typically also have a docroot and a logs directory there, and the DB file if I’m using SQLite.

Comments for this entry are closed. If you'd like to share your thoughts on this entry with me, please contact me directly.