Django tips: get the most out of generic views

Recently at work I had the chance to revisit an application I’d written fairly early on in my tenure at World Online; I was still getting used to doing real Django development and to some of the quirks of our development environment, and ended up writing a lot more code than I needed to, so I was happy to be able to take a couple days and rewrite large chunks of the application to be more efficient and flexible.

The biggest win was more careful use of generic views, which this particular application wasn’t making much use of originally. To put it into perspective, switching as many things as possible to generic views has meant that the application’s view code shrunk by about a third while gaining functionality.

In some situations that might not be much of a difference, but with an application that weighs in at a few thousand lines of code, shaving a full 33% off one of its components is a huge win, even if it just does the same things afterward, but in this case the application gained more functionality and flexibility. So let’s look through some things generic views can do to cut down the size of your code.

Vanilla views

The simplest case is when you find you’re writing code which does nothing but grab a plain list of objects from a model and display them. In this case you don’t have to write a view at all; the generic list_detail.object_list view will happily do that for you, and will throw in pagination for free (via the optional paginate_by parameter).

Similarly, if you’re just showing a chronological archive of objects, the date-based generic views will probably be the only things you need; this blog, for example, runs almost completely on date-based generic views (even the home page uses one).

Of course, you already know that; the real thing I want to get at is how you can do more complex things with generic views, because that’s the thing a lot of people (myself included, once upon a time) tend to miss.

Extra filtering

Let’s suppose that you have a collaborative to-do list application, where each to-do list belongs to a specific user. You want to have a nice page that each user can go to and look at all of their own lists; for example, Bob should be able to go to the URL /lists/bob/ and see his to-do lists.

The (apparent) problem here is that it doesn’t look like we can tell the generic view about the filtering we want to do; the view has no way of “knowing” that it should look for a username in the URL, so we can’t rely on that. And while we could hard-code separate dictionaries of keyword arguments for each user, and set up a line in the URL configuration for each one (/lists/alice/, /lists/bob/, /lists/carol/ and so on), that doesn’t scale at all.

But we can do this with a generic view — we just have to give it a little help, in the form of a wrapper function. Way back in June, Malcolm wrote about how to do this, but it hasn’t gotten much attention. So let’s give some more attention to the technique, because it’s extremely powerful.

The first thing you’d do is add a line in your URL configuration like this: 

('r^lists/(?P<username>\w+)/$', 'myproject.myapp.views.user_lists'),

This says that any URL matching /lists/<username>/ should go to the user_lists view in your application. Now let’s see what that view looks like: 

from django.views.generic.list_detail import object_list
from myproject.myapp.models import TodoList
 
def user_lists(request, username):
    todo_lists = TodoList.objects.filter(owner__username__exact=username)
    return object_list(request, queryset=todo_lists)

And you’re done.

The only thing holding this up was that the generic view wants to be passed a QuerySet which contains the objects it’s supposed to filter or display, so all we’ve done here is write a two-line function which builds the correct QuerySet (by reading the username out of the URL to do the filtering) and then calls the object_list generic view with that QuerySet as an argument.

This works because generic views — and, in fact, all views in Django — are just Python functions, so other functions can build up lists of arguments, call them and even return their responses directly. So any time you find yourself thinking “I could use a generic view for this if only I could somehow filter a little first”, remember that you can filter a little first by writing a short wrapper function that calls the generic view.

Getting more out of a generic view

Another common situation is needing to display a single object, or a list of objects — just the sorts of things generic views are good at — but also needing to add one or two extra things to show up in the template.

Again, it can be wrappers to the rescue: all of the generic views take an optional argument, extra_context, which should be a dictionary of variables and values to pass through into the template context. Continuing with the to-do list example, let’s say we want to add a couple things to the view we wrapped above: the number of to-do items the user currently has open, and any high-priority items they have open.

Here’s the code: 

from django.views.generic.list_detail import object_list
from myproject.myapp.models import TodoList, TodoItem
 
def user_lists(request, username):
    todo_lists = TodoList.objects.filter(owner__username__exact=username)
    open_items = TodoItem.objects.filter(todolist__owner__username__exact=username)
    open_item_count = open_items.count()
    priority_items = open_items.filter(priority__exact='high')
    return object_list(request, queryset=todo_lists,
                       extra_context={'open_item_count': open_item_count,
                                      'priority_items': priority_items})

And now the open_items and priority_items variables will be available, with the correct values, in the template.

Doing extra work

Another common pattern is needing to do take some sort of related action, like logging, before returning the final view. And again, it’s really easy to do this with a short wrapper around a generic view. If we had a user profile model which stored the last date/time users checked their to-do lists, we could once again add a couple lines to our wrapper and have it work: 

import datetime
from django.views.generic.list_detail import object_list
from myproject.myapp.models import TodoList, UserProfile
 
def user_lists(request, username):
    profile = UserProfile.objects.get(user_username__exact=username)
    profile.last_list_check = datetime.datetime.today()
    profile.save()
    todo_lists = TodoList.objects.filter(owner__username__exact=username)
    return object_list(request, queryset=todo_lists)

Now, each view of a user’s to-do lists will reset the last_list_check field of their profile to the current date and time.

Learn it, love it, use it

There are tons of other uses for this technique, but hopefully I’ve got you thinking about them now and you’ll figure out how to use them in your next Django project. Writing wrappers like these around generic views when you need some extra processing is probably one of the most important and powerful idioms of everyday Django development; it can drastically cut down the amount of code you have to write and increase the options you have available (because even with just their stock set of options, generic views are pretty darned flexible).

Unfortunately, it seems to be one of the most-overlooked idioms as well; more than once I’ve explained this trick to people, even seasoned, experienced programmers, only to have their eyes go wide as they say, “wait, can you really do that?” You can do that, and you should whenever possible — the generic views are there, and built the way they are, for good reasons. Learning to use them effectively will save you time and effort on the path to writing better applications.

Comments

EspenG
November 16, 2006
#

This was a really nice write up! Thanks for all those nice tricks I did not know about, and please bring me more.

Espen

James Wheare
November 16, 2006
#

You’ll also need to pass in the request object, like this:

return object_list(request, queryset=todo_lists)

Oh and another syntactic trick is to take advantage of kwargs, so instead of setting all your parameters in the object_list call, you can first define a dictionary of parameters then pass that in instead. 

params = {
    queryset=todo_lists
    extra_context = {
        ‘open_item_count’: open_item_count, 
        ‘priority_items’: priority_items,
    },
}
return object_list(request, **params)
James Bennett
November 16, 2006
#

Arg. I knew I’d have a stupid typo in there somewhere. And since I copy/pasted the code from the first sample to do the other two, I ended up with three.

Thanks for catching it.

Kevin Cooke
November 16, 2006
#

Maybe I’m just being dense, but would this technique work for pulling different data from different objects onto the same page? Like, recent todos on one part of the page, recent blog posts on another? Or do I write custom views for that?

James Wheare
November 16, 2006
#

You can pass objects from as many other models as you like in the extra_context parameter. 

from noodles.blog.models import Blog, Sandwich
extra_context = {
    ‘blog_posts’: Blog.objects.latest()[5],
    ‘tasties’: Sandwich.objects.filter(filling=’yum!’),
}

etc. etc.. =)

Hank Sims
November 16, 2006
#

In your to-do list example — it’d probably be pretty tough to get the admin interface to display only Bob’s items to Bob, though, right?

bin
November 16, 2006
#

Nice writeup James.

Wouldn’t it be nice to have generic views (REST APIs) for:

  • filesystem operations (on *nix for a start). Loose coupling with Python’s OS module. In this way, I have extended patrickk’s FileBrowser a little.

  • subversion interface (e.g. for keeping templates or translation under revision control) using pysvn. This was actually my first attempt trying to grasp Django. The results were basic, but pleasing.

  • RDF data (for any purpose, really!) You could store metadata about anything, anywhere - for example using Sparta or straight RDFLib. Imagine the leverage of being able to define RDF triples (or even inferential statements…) with generic views. By the way, a recent django-users post touched on RDF briefly. (James, perfect idea to develop an RDF serializer…) Hey, what about interfacing to DCMI Metadata Terms? And while we’re at it, why not replace Django’s URL.conf with RDF graphs hooking namespaces?

  • Python Imaging Library operations; In conjunction with the above-mentioned extension of FileBrowser, I implemented some basic PIL functionality using RESTful urls.

  • relaying Ma.gnolia, Flickr or any other proprietary web service’s API calls to django’s generic views. Google Calendar, Yahoo Finance, YouTube… Especially the REST-oriented services would be relatively easy to implement. Imagine being able to extend URL.conf with proprietary namespaces.

  • OpenID, Geodata… you name it.

In my excitement, I just set up (http://code.google.com/p/metaapi/). Something like ‘A set Django generic views relaying to proprietary web service APIs or Python modules’.

Please, share your views on my twisted approach. I would really appreciate that.

Bye for now, bin

James Bennett
November 16, 2006
#

In your to-do list example — it’d probably be pretty tough to get the admin interface to display only Bob’s items to Bob, though, right?

Yes. It could be done with some work, but it shouldn’t be done. The admin app is not intended for that level of control; it’s meant for site staff who are trusted with access to all content of a particular type.

Steve Wedig
November 17, 2006
#

Nice post :)

I would love to use generic views more like this, but my problem is that we can only do the front half of a wrapper. To do row-level authorization, I need to work with the returned object(s) before returning a response. However generic views themselves return responses, so by then I’m out of luck (I think).

Any suggestions?

noway
November 17, 2006
#

Steve: You can do anything to the queryset before you send it off. So you could go through it result, by result, and filter it through a loop if you felt compelled. The generic view takes a request, and queryest, and returns a response.

Although there are probably somethings you could do to your database to make the row-level authorization more simple.

Scanner
November 17, 2006
#

I like generic views for private mockups, but the biggest problem I have with them is I frequently need to add a set of permissions for privacy. Such as I have a large number of bits of data submitted (and owned) by various users. You can browse and see all of your own data, you can also browse and see other user’s data if they have either permitted it to be globally visible, or visible to you, or to a group that you are in.

I know I can customize the result I return on a case by case basis, but it ends up being simpler to just write the view myself each time.

Although I suppose basically writing a bunch of views that just wrap the generic view lets me worry about the specifics in each case and let the generic view do its heavy lifting.

Baczek
November 18, 2006
#

Re: row level permissions - I think that querysets should grow a way to filter them (as in builtin filter(), not filter() the method) when they get executed, e.g.

f = lambda x: foo(x) == y and bar(x) != z qs.add_filter_func(f)

prints filtered results, query is unmodified

print qs

Baczek
November 18, 2006
#

Doh, markdown syntax :P 

f = lambda x: foo(x) == y and bar(x) != z
qs.add_filter_func(f)
# prints filtered results, query is unmodified
print qs
MerMer
November 20, 2006
#

James,

Excellent - as usual. Your step by step tutorials have been a big help to me, particularly as I came to Django without any programing background.

I think I’ve spotted a small typo…

Shouldn’t , ” This says that any URL matching //lists/ should go to the user_lists view in your application. “ read

“his says that any URL matching lists/ should go to the user_lists view in your application.”

James Bennett
November 20, 2006
#

Yeah, I originally had it the other way around, and then missed changing it in that spot.

Thomas Weholt
February 9, 2007
#

I’ve just started using django and I’m having a hard time wrapping my brain around the use of generic views; does it need a template? How come none of the examples on how to use generic views shows an example template if that’s the case? And a screenshot showing the output of a generic view?

James Bennett
February 9, 2007
#

Thomas, the docs on generic views explain what variables are made available to generic views, and what template names they use by default (as well as how to change the names of the templates they’ll use, on a per-view basis).

As for output of a generic view, well, you’re looking at one — this page is served from the generic date_based.object_detail view. The entry and information about it come from the generic view, and other stuff comes out of template tags I’ve written.

Add a comment

You may use Markdown syntax in your comment, but raw HTML will be removed. By posting a comment here, you are agreeing to the terms of my comment policy.