Django tips: get the most out of generic views

An entry published by James Bennett on November 16, 2006, Part of the category Django. 17 comments posted.

Recently at work I had the chance to revisit an application I’d written fairly early on in my tenure at World Online; I was still getting used to doing real Django development and to some of the quirks of our development environment, and ended up writing a lot more code than I needed to, so I was happy to be able to take a couple days and rewrite large chunks of the application to be more efficient and flexible.

The biggest win was more careful use of generic views, which this particular application wasn’t making much use of originally. To put it into perspective, switching as many things as possible to generic views has meant that the application’s view code shrunk by about a third while gaining functionality.

In some situations that might not be much of a difference, but with an application that weighs in at a few thousand lines of code, shaving a full 33% off one of its components is a huge win, even if it just does the same things afterward, but in this case the application gained more functionality and flexibility. So let’s look through some things generic views can do to cut down the size of your code.

Vanilla views

The simplest case is when you find you’re writing code which does nothing but grab a plain list of objects from a model and display them. In this case you don’t have to write a view at all; the generic list_detail.object_list view will happily do that for you, and will throw in pagination for free (via the optional paginate_by parameter).

Similarly, if you’re just showing a chronological archive of objects, the date-based generic views will probably be the only things you need; this blog, for example, runs almost completely on date-based generic views (even the home page uses one).

Of course, you already know that; the real thing I want to get at is how you can do more complex things with generic views, because that’s the thing a lot of people (myself included, once upon a time) tend to miss.

Extra filtering

Let’s suppose that you have a collaborative to-do list application, where each to-do list belongs to a specific user. You want to have a nice page that each user can go to and look at all of their own lists; for example, Bob should be able to go to the URL /lists/bob/ and see his to-do lists.

The (apparent) problem here is that it doesn’t look like we can tell the generic view about the filtering we want to do; the view has no way of “knowing” that it should look for a username in the URL, so we can’t rely on that. And while we could hard-code separate dictionaries of keyword arguments for each user, and set up a line in the URL configuration for each one (/lists/alice/, /lists/bob/, /lists/carol/ and so on), that doesn’t scale at all.

But we can do this with a generic view — we just have to give it a little help, in the form of a wrapper function. Way back in June, Malcolm wrote about how to do this, but it hasn’t gotten much attention. So let’s give some more attention to the technique, because it’s extremely powerful.

The first thing you’d do is add a line in your URL configuration like this: 

('r^lists/(?P<username>\w+)/$', 'myproject.myapp.views.user_lists'),

This says that any URL matching /lists/<username>/ should go to the user_lists view in your application. Now let’s see what that view looks like: 

from django.views.generic.list_detail import object_list
from myproject.myapp.models import TodoList
 
def user_lists(request, username):
    todo_lists = TodoList.objects.filter(owner__username__exact=username)
    return object_list(request, queryset=todo_lists)

And you’re done.

The only thing holding this up was that the generic view wants to be passed a QuerySet which contains the objects it’s supposed to filter or display, so all we’ve done here is write a two-line function which builds the correct QuerySet (by reading the username out of the URL to do the filtering) and then calls the object_list generic view with that QuerySet as an argument.

This works because generic views — and, in fact, all views in Django — are just Python functions, so other functions can build up lists of arguments, call them and even return their responses directly. So any time you find yourself thinking “I could use a generic view for this if only I could somehow filter a little first”, remember that you can filter a little first by writing a short wrapper function that calls the generic view.

Getting more out of a generic view

Another common situation is needing to display a single object, or a list of objects — just the sorts of things generic views are good at — but also needing to add one or two extra things to show up in the template.

Again, it can be wrappers to the rescue: all of the generic views take an optional argument, extra_context, which should be a dictionary of variables and values to pass through into the template context. Continuing with the to-do list example, let’s say we want to add a couple things to the view we wrapped above: the number of to-do items the user currently has open, and any high-priority items they have open.

Here’s the code: 

from django.views.generic.list_detail import object_list
from myproject.myapp.models import TodoList, TodoItem
 
def user_lists(request, username):
    todo_lists = TodoList.objects.filter(owner__username__exact=username)
    open_items = TodoItem.objects.filter(todolist__owner__username__exact=username)
    open_item_count = open_items.count()
    priority_items = open_items.filter(priority__exact='high')
    return object_list(request, queryset=todo_lists,
                       extra_context={'open_item_count': open_item_count,
                                      'priority_items': priority_items})

And now the open_items and priority_items variables will be available, with the correct values, in the template.

Doing extra work

Another common pattern is needing to do take some sort of related action, like logging, before returning the final view. And again, it’s really easy to do this with a short wrapper around a generic view. If we had a user profile model which stored the last date/time users checked their to-do lists, we could once again add a couple lines to our wrapper and have it work: 

import datetime
from django.views.generic.list_detail import object_list
from myproject.myapp.models import TodoList, UserProfile
 
def user_lists(request, username):
    profile = UserProfile.objects.get(user_username__exact=username)
    profile.last_list_check = datetime.datetime.today()
    profile.save()
    todo_lists = TodoList.objects.filter(owner__username__exact=username)
    return object_list(request, queryset=todo_lists)

Now, each view of a user’s to-do lists will reset the last_list_check field of their profile to the current date and time.

Learn it, love it, use it

There are tons of other uses for this technique, but hopefully I’ve got you thinking about them now and you’ll figure out how to use them in your next Django project. Writing wrappers like these around generic views when you need some extra processing is probably one of the most important and powerful idioms of everyday Django development; it can drastically cut down the amount of code you have to write and increase the options you have available (because even with just their stock set of options, generic views are pretty darned flexible).

Unfortunately, it seems to be one of the most-overlooked idioms as well; more than once I’ve explained this trick to people, even seasoned, experienced programmers, only to have their eyes go wide as they say, “wait, can you really do that?” You can do that, and you should whenever possible — the generic views are there, and built the way they are, for good reasons. Learning to use them effectively will save you time and effort on the path to writing better applications.

← →

On November 16, 2006, EspenG said:

This was a really nice write up! Thanks for all those nice tricks I did not know about, and please bring me more.

Espen

On November 16, 2006, James Wheare said:

You’ll also need to pass in the request object, like this:

return object_list(request, queryset=todo_lists)

Oh and another syntactic trick is to take advantage of kwargs, so instead of setting all your parameters in the object_list call, you can first define a dictionary of parameters then pass that in instead. 

params = {
    queryset=todo_lists
    extra_context = {
        ‘open_item_count’: open_item_count, 
        ‘priority_items’: priority_items,
    },
}
return object_list(request, **params)
On November 16, 2006, James Bennett said:

Arg. I knew I’d have a stupid typo in there somewhere. And since I copy/pasted the code from the first sample to do the other two, I ended up with three.

Thanks for catching it.

On November 16, 2006, Kevin Cooke said:

Maybe I’m just being dense, but would this technique work for pulling different data from different objects onto the same page? Like, recent todos on one part of the page, recent blog posts on another? Or do I write custom views for that?

On November 16, 2006, James Wheare said:

You can pass objects from as many other models as you like in the extra_context parameter. 

from noodles.blog.models import Blog, Sandwich
extra_context = {
    ‘blog_posts’: Blog.objects.latest()[5],
    ‘tasties’: Sandwich.objects.filter(filling=’yum!’),
}

etc. etc.. =)

On November 16, 2006, Hank Sims said:

In your to-do list example — it’d probably be pretty tough to get the admin interface to display only Bob’s items to Bob, though, right?

On November 16, 2006, bin said:

Nice writeup James.

Wouldn’t it be nice to have generic views (REST APIs) for:

In my excitement, I just set up (http://code.google.com/p/metaapi/). Something like ‘A set Django generic views relaying to proprietary web service APIs or Python modules’.

Please, share your views on my twisted approach. I would really appreciate that.

Bye for now, bin

On November 16, 2006, James Bennett said:

In your to-do list example — it’d probably be pretty tough to get the admin interface to display only Bob’s items to Bob, though, right?

Yes. It could be done with some work, but it shouldn’t be done. The admin app is not intended for that level of control; it’s meant for site staff who are trusted with access to all content of a particular type.

On November 17, 2006, Steve Wedig said:

Nice post :)

I would love to use generic views more like this, but my problem is that we can only do the front half of a wrapper. To do row-level authorization, I need to work with the returned object(s) before returning a response. However generic views themselves return responses, so by then I’m out of luck (I think).

Any suggestions?

On November 17, 2006, noway said:

Steve: You can do anything to the queryset before you send it off. So you could go through it result, by result, and filter it through a loop if you felt compelled. The generic view takes a request, and queryest, and returns a response.

Although there are probably somethings you could do to your database to make the row-level authorization more simple.

On November 17, 2006, Scanner said:

I like generic views for private mockups, but the biggest problem I have with them is I frequently need to add a set of permissions for privacy. Such as I have a large number of bits of data submitted (and owned) by various users. You can browse and see all of your own data, you can also browse and see other user’s data if they have either permitted it to be globally visible, or visible to you, or to a group that you are in.

I know I can customize the result I return on a case by case basis, but it ends up being simpler to just write the view myself each time.

Although I suppose basically writing a bunch of views that just wrap the generic view lets me worry about the specifics in each case and let the generic view do its heavy lifting.

On November 18, 2006, Baczek said:

Re: row level permissions - I think that querysets should grow a way to filter them (as in builtin filter(), not filter() the method) when they get executed, e.g.

f = lambda x: foo(x) == y and bar(x) != z qs.add_filter_func(f)

prints filtered results, query is unmodified

print qs

On November 18, 2006, Baczek said:

Doh, markdown syntax :P 

f = lambda x: foo(x) == y and bar(x) != z
qs.add_filter_func(f)
# prints filtered results, query is unmodified
print qs
On November 20, 2006, MerMer said:

James,

Excellent - as usual. Your step by step tutorials have been a big help to me, particularly as I came to Django without any programing background.

I think I’ve spotted a small typo…

Shouldn’t , ” This says that any URL matching //lists/ should go to the user_lists view in your application. “ read

“his says that any URL matching lists/ should go to the user_lists view in your application.”

On November 20, 2006, James Bennett said:

Yeah, I originally had it the other way around, and then missed changing it in that spot.

On February 9, 2007, Thomas Weholt said:

I’ve just started using django and I’m having a hard time wrapping my brain around the use of generic views; does it need a template? How come none of the examples on how to use generic views shows an example template if that’s the case? And a screenshot showing the output of a generic view?

On February 9, 2007, James Bennett said:

Thomas, the docs on generic views explain what variables are made available to generic views, and what template names they use by default (as well as how to change the names of the templates they’ll use, on a per-view basis).

As for output of a generic view, well, you’re looking at one — this page is served from the generic date_based.object_detail view. The entry and information about it come from the generic view, and other stuff comes out of template tags I’ve written.

Comments for this entry are closed. If you'd like to share your thoughts on this entry with me, please contact me directly.

ponybadge