The B-List

Django tips: using properties on models and managers

While working on a little side project this week, I ran into a couple of very common use cases that often result in a lot of extra typing:

1. Defining a BooleanField, or an IntegerField or CharField with choices which will, logically, break up instances of the model into certain groups which need to be accessed often.
2. Repeatedly wanting to calculate a value based on the values of several fields of a model.

Let’s look at how to handle these common cases, while reducing the extra typing and making them behave in an extremely intuitive fashion.

The set-up

Consider as an example this model, which might be found in a project-management or customer-relations application:

ORG_TYPE_CHOICES = ((1, 'Internal'), (2, 'External'))

class Organization(models.Model):
    name = models.CharField(maxlength=200)
    slug = models.SlugField(prepopulate_from=('name',))
    contact_person = models.ForeignKey(User)
    mailing_address = models.TextField()
    add_date = models.DateTimeField(default=models.LazyDate())
    org_type = models.IntegerField(choices=ORG_TYPE_CHOICES)
    is_active = models.BooleanField(default=True)
    last_paid = models.DateField(blank=True, null=True)

Now, it’s going to be extremely common to want to fetch all the organizations of one type — “internal” or “external” — and do something with that list. That means typing things like this over and over again:

external_orgs = Organization.objects.filter(org_type__exact=2)

That’s a lot of typing. Worse, it’s a lot of repetitive typing. Fortunately, Django provides an easy way to save ourselves that typing: we can define a custom manager which will do the filtering for us:

class OrganizationManager(models.Manager):
    def internal(self):
        return self.get_query_set().filter(org_type__exact=1)
    
    def external(self):
        return self.get_query_set().filter(org_type__exact=2)

Then, since we didn’t override anything, it’s safe to make this the default manager for the Organization model, by adding this line in the model definition:

objects = OrganizationManager()

Now instead of doing lots of repetitive typing, we can just do, say,

Organization.objects.internal().filter(is_active__exact=True)

And we’ll get all the “internal” organizations that are currently active.

Now, suppose that for “external” organizations we want to charge a monthly subscription fee, and disable their login if they haven’t paid this month. We could define a method on the Organization model which checks that the organization is external, is active and has paid within the last thirty days:

def is_paid_up(self):
    return self.org_type == 1 and self.is_active and self.last_paid > \
           (datetime.date.today() - datetime.timedelta(30))

Then whenever we need to check this, we can grab the Organization object into a variable, say we’ll call it o, and do

if o.is_paid_up():
    # code here to let them into the application
else:
    # deny access

Let’s make that even better

At this point we’re rolling along pretty well and have saved ourselves quite a bit of typing: we no longer need to explicitly call filter every time we need to separate out “internal” or “external” organizations, and we don’t need to do lots of cumbersome attribute checks to find out if an organization should be allowed access to the application (since we can just check the return value of is_paid_up(). Cleaning up the resulting application code that much would make a pretty decent “Django tip” in itself (especially the custom manager bit — managers are quite possibly my favorite new feature from magic-removal/0.95), but saving ourselves some typing isn’t everything; we also want the code to make intuitive sense, and we can go one step better on that.

You’ll notice that internal and external are instance methods of the OrganizationManager, and that is_paid_up is an instance method of the Organization model; that means we have to remember to type parentheses every time we use them (except in a template), which is a minor annoyance because, really, these don’t feel like they should be methods that you have to call — it’d feel cleaner and more intuitive to just do

if o.is_paid_up:
    # let them in
else:
    # lock them out

So let’s fix that, by making all these methods we’ve defined into properties.

In Python, properties provide an easy way of wrapping up something that isn’t a “simple” attribute (i.e., one that doesn’t just hold a value) and making it look and behave like it is. Creating a property is extremly simple: depending on the exact type of property you want, you just define a couple of methods and pass them as arguments to property. So, for example, we could set up is_paid_up as a property by doing this:

def _get_is_paid_up(self):
    return self.org_type == 1 and self.is_active and self.last_paid > \
           (datetime.date.today() - datetime.timedelta(30))

is_paid_up = property(_get_is_paid_up)

In this case we only want to get the value of is_paid_up, not set or delete it, so we only need to define one method to pass into property, but for other situations you might want to define extra methods to handle those (the Django model documentation includes an example of a property which has both “get” and “set” methods. Also, note that the method used to create the property has an underscore in front of its name — so it’s _get_is_paid_up — which is a convention in Python for marking something as “private” to the class (Python doesn’t really enforce private attributes or methods, but using the underscore is a way of telling those who read your code that something is meant only for internal use).

Now we can just do if o.is_paid_up everywhere in our code, and it will work — we don’t have to remember everywhere that we’re actually getting the return value of a method or type the parentheses to call it, because making it into a property makes it behave like any attribute that holds a simple value.

And we can rewrite our custom manager along the same lines:

class OrganizationManager(models.Manager):
    def _get_internal(self):
        return self.get_query_set().filter(org_type__exact=1)
    
    def _get_external(self):
        return self.get_query_set().filter(org_type__exact=2)
    
    internal = property(_get_internal)
    external = property(_get_external)

Now we can do Organization.objects.internal.filter(is_active__exact=True), or any other type of operation that a QuerySet can do (just don’t try do to Organization.objects.internal.all() — Organization.objects.internal already is the set of all “internal” organizations, and trying to do all() on it will raise an AttributeError).

This doesn’t save us tons of typing like the initial setup of the custom manager and the is_paid_up method did, but it does make our code cleaner and makes internal, external and is_paid_up behave more intuitively, and that’s a big, big win.

Comments

Jeff Croft
August 18, 2006
#

Nice!

I’ve also been playing with custom managers a lot lately as I re-write my blog and personal site apps for magic removal. But, I have generally defined separate managers, rather than redefining objects, like this:

class LiveEntryManager(models.Manager):
    def get_query_set(self):
        return super(LiveEntryManager, self).get_query_set().filter(status__exact=2)

add to my model like this:

live_entries = LiveEntryManager()

and then used like this:

entries = Entry.live_entries.get(id=1)

Is there any advantage to doing what you’ve done, redefining the objects() manager, versus just creating my own?

James Bennett
August 18, 2006
#

Doing the LiveEntryManager is actually the exact method I’m using right now on this site (since, like you, I want to be able to have “draft” entries that don’t show up publicly).

But as I ever-so-slowly refactor the code to get it into shape for a public release, I think I’m going to move toward the property trick, and doing live_entries, or probably just live as a property on the manager, which will let me stick to just a redefined objects and avoid some of the pitfalls of having multiple managers on a class (for example: you have to make sure that if you’ve overridden get_query_set you keep objects as the default manager so that you won’t accidentally make some objects uneditable in the admin).

James Bennett
August 18, 2006
#

Malcolm,

Mostly it’s to do with two things that creep up when multiple managers get involved:

1. There’s the problem of having custom managers which override get_query_set in a way which excludes certain objects (e.g., the LiveEntryManager Jeff and I are talking about); if you don’t keep objects as the default manager, you’ll lose the ability to edit those excluded objects in the admin.
2. There’s the problem of specifying a manager for use in the admin; there’s an option to do this, but when editing an object the admin uses an automatic ChangeManipulator, which uses the model’s default manager no matter what you’ve told the admin to use.

Also, philosophically I feel like using a different manager class means “I’m going to interact with this model in a completely different way” — if I want to keep the default get_query_set and other methods, and just add some new filtering/querying capabilities for a model, I’m probably going to do them up as properties on a single custom manager and override objects with it.

Adam Endicott
August 18, 2006
#

I’ve only used custom managers in one app, and I went the multiple managers route to do the same thing as your internal/external example. But when I defined my managers, I also did

objects = models.Manager()

So I could have my cake and eat it too. This lets me use o.objects to access all organizations, and o.internal/o.external to access access specific types.

This has worked quite well for me, and seems to address your two issues. It doesn’t, however, address the philosophical issue, which seems to me to be a fine reason to go the other way if it feels right to you.

James Bennett
August 18, 2006
#

Adam, that’s what I ended up doing, because there wasn’t any other way which consistently worked.

Note, however, that even doing this requires working up some convenience models on related objects — my Category model, for example, has to have an extra method which fetches the related entries using the LiveEntryManager, because the related manager for a many-to-many relationship is always the default manager of the related model.

Add a comment