Django tips: auto-populated fields

One of these days I’m going to run out of frequently-asked Django questions to answer. But today isn’t that day, so let’s look at one of the most common questions people ask: how do you set up a model with one or more fields that never show up in add/edit forms, and get automatically populated with some specific value?

A simple example

Let’s say you’re writing a to-do list application, and you want to have fields on the list model to store the date it was created and the time it was last updated. The model might look something like this: 

class TodoList(models.Model):
    title = models.CharField(maxlength=100)
    created = models.DateField()
    updated = models.DateTimeField()

Now, in an ideal world you’d never publicly display the created and updated fields in a form; you’d just have them filled in automatically with the right values. So let’s make that happen by editing the model slightly: 

class TodoList(models.Model):
    title = models.CharField(maxlength=100)
    created = models.DateField(editable=False)
    updated = models.DateTimeField(editable=False)
    
    def save(self):
        if not self.id:
            self.created = datetime.date.today()
        self.updated = datetime.datetime.today()
        super(TodoList, self).save()

Of course, you’ll want to add import datetime up at the top of your models file. Anyway, there are two changes that stand out here:

  1. The created and updated fields now have editable=False. This means that they will never show up in an automatic manipulator or in the admin. They’re still required, they just won’t be displayed anywhere.
  2. There’s now a custom save method on the model which does the actual work. The first time a new to-do list is saved, it won’t have an id because there’s no row for it in the database yet. So the if not self.id check tells us that the to-do list is being saved for the first time, and we drop in a single line which fills the current date into the created field. We also want the updated field to be updated on every save, so we add a line which sets that to the current date and time. Then the super call saves the list.

It really is that easy, and it works for a lot of field types — just give the field editable=False and fill it in with a custom save method. If the field is only supposed to be filled in for the first save, use the if not self.id trick to accomplish that. The only thing that will vary is how you fill in the fielf; if you wanted to automatically populate a SlugField, for example, you could import the slugify function from django.template.defaultfilters and use that to generate a slug from some other field.

But the thing people really clamor for is a foreign key to the auth User model which automatically fills in the current user, for doing something like a “created_by” field. This trick won’t work there, because models intentionally have no access to the “current user” — that decoupling makes it possible to use Django for things that aren’t web applications (and thus don’t have HttpRequest instances which carry user instances around with them), but it keeps this trick from working out-of-the-box for that particular case.

So how can we auto-populate a user into a model?

The Holy Grail

First, let’s change the model to have a created_by field: 

class TodoList(models.Model):
    title = models.CharField(maxlength=100)
    created = models.DateField()
    updated = models.DateTimeField()
    created_by = models.ForeignKey(User)
    
    def save(self):
        if not self.id:
            self.created = datetime.date.today()
        self.updated = datetime.datetime.today()
        super(TodoList, self).save()

Of course, you’ll want to add from django.contrib.auth.models import User up at the top of the file so you can reference User like that. Now, solving this is going to require two short pieces of code: a custom manager and a custom manipulator. Here’s the manager class: 

class TodoListManager(models.Manager):
    def create_list(self, title, user):
        new_list = self.model(title=title, created_by=user)
        new_list.save()
        return new_list

Then make it the default manager for the TodoList class by adding objects=TodoListManager() in the model definition. All of the normal querying methods will work as expected, because we haven’t overridden any of them. But we have added a new method: create_list, which expects a title and a User instance, and creates a new to-do list using them (the created and updated fields will, as before, be automatically filled in when the new list is saved). To see how that works, let’s look at the manipulator: 

from django import forms
class TodoListForm(forms.Manipulator):
    def __init__(self, request):
        self.fields = (
            forms.TextField(field_name='title', length=30, maxlength=100, is_required=True),
        )
        self.request = request
    
    def save(self, new_data):
        return TodoList.objects.create_list(new_data['title'], self.request.user)

Here’s a simple view which puts it all together: 

@login_required
def create_todo_list(request):
    manipulator = TodoListForm(request)
    if request.POST:
        new_data = request.POST.copy()
        errors = manipulator.get_validation_errors(new_data)
        if not errors:
            manipulator.do_html2python(new_data)
            new_list = manipulator.save(new_data)
            return HttpResponseRedirect(new_list.get_absolute_url())
    else:
        errors = new_data = {}
    form = forms.FormWrapper(manipulator, new_data, errors)
    return render_to_response('create_todo_list.html', {'form': form})

The trick here is that creating the custom manipulator takes the HttpRequest as an argument, and later when you use it to save, it pulls the User out of that request and uses it to create the new to-do list. The view has the login_required decorator to make sure there will always be a valid user.

And voilà. You have a to-do list which automatically gets the current user assigned to its created_by field. Not hard at all, right? And as a bonus, the create_list method on the manager is useful in other circumstances as well — any time you have access to a valid User object, you can call this method to create a new list with that user filled in.

One caveat

Now, some people are probably going to be unhappy with the fact that the automatic user population doesn’t integrate into Django’s admin application in any way. That’s unavoidable, and my own humble opinion is that this isn’t something that should — the admin is for people who are 100% trusted to do the right thing with your site’s content, so if you’re doing all your model creation through the admin you should just show the created_by field and trust your site staff to fill it in correctly.

But for the (probably more common) case where users are creating content, it’s great: just point them at a view like the one I’ve listed above, and everything will happen as it should.

Comments

Jay Parlar
November 2, 2006
#

Should the manager not be: 

class TodoListManager(models.Manager):
    def create_list(self, title, user):
        new_list = self.model(title=title, created_by=user)
        new_list.save()
        return new_list
James Bennett
November 2, 2006
#

Yup, that was me typing too fast. Fixed it.

Tony
November 3, 2006
#

At the moment, I’m populating a file upload instance with something like this

and then using that as the way of passing back who the logged in user is.

This looks very cool, but I’m having a hard time understanding why it’s needed? (and now I’ve definitely shown my ignorance!)

Arthur Case
November 3, 2006
#

Great article thanks :)

I’ve had a request for a project I’m working on to make an auto-populated password field, so this will come in handy.

One thing I’m not sure is possible is to have a field hidden when you add something in the Django admin app (in this case a password field for a new user) and then when you are editing this record have the field become visible so the auto-made password can be changed and viewed in the admin page?

Vidar
November 3, 2006
#

The use of a custom save method is a fine thing but in your first example (create and update fields) it is not necessary. Simply use the auto_now and auto_now_add arguments for the DateField and DateTimeField fields. Note that you will still have to set Editable=False.

James Bennett
November 3, 2006
#

I didn’t use auto_now or auto_now_add for two reasons:

  1. They’ve always been a little bit flaky and I don’t really trust them.
  2. DateField and DateTimeField are the only ones they work with, and I wanted to provide a way to generically populate any sort of field.
Alexander
November 3, 2006
#

maybe copy is replaced by copycopy because in the syntax highlighter copy is in both commonlibs and builtins (see http://www.b-list.org/media/js/dps/shBrushPython.js)

James Bennett
November 3, 2006
#

Nicely done. I hadn’t had time to dig into the highlighter and figure it out, but that seems to have cleared things up. Now I just need to remember not to use the copy module in any examples…

Eric Florenzano
November 3, 2006
#

For fields that need to be automatically populated and shown to the user (say, the subject line of a reply to a post on a message board), I’ve found a much easier way to do it…

On line 12 of the view, change: errors = new_data = {} to something like: errors = {} new data = {‘subject’, topic.subject}

(that’s assuming you’ve already queried up a ‘topic’, using get_topic_or_404, or whatever)

And that’s it!

Shimon
November 4, 2006
#

Correct me if I’m wrong but there seems to be a bit of a violation of DRY going on in your example in that the list of fields in TodoListManager needs to be kept in sync with the TodoList model.

Also, if a required field is added to the model - the 2 extra classes need to be manually updated.

Wouldn’t it be easier to just use the method outlined in the cookbook at http://code.djangoproject.com/wiki/CookBookManipulatorWithPostpopulatedFields

i.e. new_data[‘field’] = val

James Bennett
November 5, 2006
#

Well, any custom manager or custom manipulator potentially needs to be updated if you change the fields on the model they work with — there’s no way around that.

And post-filling fields by inserting stuff into new_data has always felt a bit hackish to me, even when I’ve used it. It also gets really ugly for certain types of fields, because manipulators expect expect the data to be in the format that would result from a Django FormField submitting over HTTP. To munge a value into a DateTimeField, for example, you have to produce a string representation of the date in the right format and inject that into one key of new_data, then produce a string representation of the time in the right format and inject that into another. Eventually, the overhead of all the formatting and special-casing you have to do to inject those values properly becomes impractical.

And remember that “practicality beats purity” is one of the lines in the Zen of Python ;)

Shimon
November 5, 2006
#

It’s kind of a pitty that the [seemingly common] case of creating an object with a form where some of the fields are filled in programmatically is so involved.

Why doesn’t the manipulator have a fill_model(new_data) method instead of save - then instead of:

manipulator.save(new_data)

You could have: obj = manipulator.fill_model(new_data) obj.field1 = val1 obj.field2 = val2 obj.save()

Or alternative pass a function pointer into the save method that gets access to the object just before save is called which can then fill in the fields.

def callback( obj ): obj.field1 = ‘val1’ …

manipulator.save(new_data, callback)

David, biologeek
November 6, 2006
#

I didn’t use auto_now or auto_now_add for two reasons: 1. They’ve always been a little bit flaky and I don’t really trust them.

Can you be a bit more explicit about that?

Brice Carpentier
November 6, 2006
#

I didn’t use auto_now or auto_now_add for two reasons: 1. They’ve always been a little bit flaky and I don’t really trust them.

Can you be a bit more explicit about that?

I think that what he means is that this keywords usually stop working from revision to revision, mostly when using SQLite.

James Bennett
November 6, 2006
#

Can you be a bit more explicit about that?

:)

A few of the reasons:

  • #89 (this one bit me the first time I ever tried Django)
  • #572
  • #555
  • #1030 (still unresolved)
  • #1056 (with choice comment from Jacob that “auto_now_add should go away”)
  • #1195
  • #1584

If I were to make a list of “things in Django that probably cause more problems than they solve”, auto_now_add and auto_now would probably be very high up on it.

David, biologeek
November 6, 2006
#

Ok, thanks :)

Now that’s clear, I will use your tip asap.

William McVey
November 16, 2006
#

Be careful in auto-populating slugs with slugify in the manner listed. Since the save() occurs after validation, there is the potential for a slug which might be defined with unique=True to be constructed via slugify() and hit a duplicate. I’ve written a small wrapper around slugify that will attempt to build a UniqueSlug. It’s on the django wiki at http://code.djangoproject.com/wiki/SlugifyUniquely

James Bennett
November 16, 2006
#

Yeah, I’ve only used the auto-slugification on a couple of things so far, and then only when I was slugifying from the contents of a guaranteed unique field. It’s definitely something to be careful with.

William McVey
November 17, 2006
#

James, I’m sure you’re aware of the issues, so I followup for the sake of the other readers of the post… slugify() when called on contents of two unique fields isn’t guarenteed or even expected to return two unique slugs. When you said “slugifying from the contents of a guaranteed unique field” I’m sure you probably meant to say “slugifing from fields guaranteed to result in unique slugification.”

zetarde
December 18, 2006
#

I was wondering why don’t you use the newforms API in the otherwise excellent example? That would simplify the solution a bit AFAICS…

James Bennett
December 18, 2006
#

Mostly because:

  1. The newforms system wasn’t really in a usable state when I wrote this up, and
  2. I don’t really know enough about it to feel comfortable writing tutorials for people.
Thomas
February 1, 2007
#

James -

Are you learning newforms? Is that a subject we could get a tutorial writeup on in the near future? I could sure use one… (I’m working on my first two comercial django sites even now)

Thanks for all your helpful tips and keep up the great work!

  •  Thomas
James Bennett
February 2, 2007
#

Thomas, I did finally get around to newforms, and I’ve got some stuff cooking. Give it a little while :)

Add a comment

You may use Markdown syntax in your comment, but raw HTML will be removed. By posting a comment here, you are agreeing to the terms of my comment policy.