Just so you know, Django is a smug, arrogant framework that doesn’t play nice with others. The developers are smug, arrogant assholes who don’t care what anyone else does or what other tools are available, and who always insist on doing things their way. Django absolutely refuses to support cool “standard” Python tools and libraries unless they were implemented first for Django. Just stay the hell away from it, and teach those goddamned Django people a lesson by doing so!
Or at least, that’s the impression you’d get from reading the rants some people periodically launch against it. I understand why this particular dead horse gets dragged out and flogged every once in a while (and I understand why, on several occasions, the flogging has been done by developers of other Python web frameworks), but understanding why it happens doesn’t make it any less wrong. And of course there’s an entry in the Django FAQ about why we’ve “reinvented” a number of things, and has been pretty much since day one, but for some reason people either don’t read it or just dismiss it out of hand.
So let me try to set the record straight.
The “NIH” stands for “Not Invented Here”, and is a common ingredient of bad software projects — rather than looking at what’s available and figuring out what they can re-use, such projects ignore existing work and proceed to re-invent everything for their own use. If it didn’t come out of the project originally, it doesn’t get used.
From the outside, especially now, it’s easy to fling that accusation at Django: we have our own object-relational mapper, our own templating system, our own middleware system… it sure looks like we’re suffering from a major case of NIH. But that’s a misleading impression, and to understand why we have to go back a ways in time.
Django began its life a number of years ago and coalesced, one piece at a time, out of the needs of the developers here at World Online. The key word here is “a number of years”; components of Django predate a lot of the things we’re often accused of “reinventing”. That shoots a big, gaping hole in the NIH argument, because it’s really hard to use a third-party tool that doesn’t exist yet.
Django has certainly evolved a lot since then, and high-quality tools have come into existence and matured independently of Django, and — in my own admittedly biased opinion — I think we’re doing an OK job of appropriately using them as we go. For example:
gettext.
I could keep going on and on about the various third-party tools and standards Django uses and all the ways you can integrate more stuff if you want to, but hopefully the above will be enough to indicate that anyone accusing us of having NIH blinkers is full of it. Nevertheless, there are two things that keep coming up over and over again…
One thing people ask a lot (well, one particular person asks it a lot, at least) is why, since Django can speak WSGI, it implements its own middleware system instead of taking advantage of the existing ecosystem of WSGI middleware. This seems like classic developer arrogance, right?
Well, not really. If you want to use WSGI middleware instead of Django’s, then go for it — we won’t yell at you if use a WSGI gzip module instead of Django’s GzipMiddleware, for example.
The problem is that you can’t always use WSGI middleware or other common WSGI tools. Django is heavily committed to also working under mod_python; if nothing else, it’s the standard deployment setup here at World Online, and the one we recommend to our clients when we sell them Django-based applications. And mod_python, according to its official FAQ, doesn’t really do WSGI yet; if you need mod_python and WSGI, you need to go fetch a third-party module and set it up yourself.
It’s kind of hard to use WSGI middleware when you don’t have a WSGI server.
And it’s a very good argument in favor of Django having its own middleware system which implements common needs like content encoding, caching, error handling, etc. It’s also why Django has two slightly different entry and exit points — one speaks mod_python and the other speaks WSGI. It’s not fun or pretty to need that, but what are you gonna do?
It’s true that now there are approximately eleventy billion Python templating toolkits out there, and some of them are pretty darned good. So why does Django still have its own different templating system? Off the top of my head, I can think of a few good reasons:
It’s also kind of amusing that some people apparently like Django’s template system so much that they’ve built standalone versions of it for use elsewhere (even in other languages — there’s a templating system available now for Rails that uses identical syntax to Django’s).
There’s a big difference between NIH and “we looked at what was available already and didn’t find anything suitable”. There are a number of places where Django has chosen to do its own thing, but those have been informed choices, made with an understanding of what existing tools were available.
So could we stop flogging the dead horse?
Comments for this entry are closed. If you'd like to share your thoughts on this entry with me, please contact me directly.
Interestingly, I started my Python Framework trek in TurboGears. It was a really neat thing at first, but the more complicated stuff I wanted to do, the more cumbersome it became. Now, this is MY OPINION and this happened during the 0.8 days (much has changed since then), I feel a lot of the problems I had were due to the jilted-ness of the framework.
For those who don’t know, TurboGears was built using pre-existing components; CherryPy as a base framework, Kid templates, SQLObject for ORM, and MochiKit for easy AJAX. TurboGears itself is the glue that holds them all together, with a bunch of excellent helper functions that make developing easier.
This approach has been the biggest factor in my abandonment of the platform. The glue and pieces don’t seem to stand very firm without a lot of magic. Additionally, there have been many unfixable bugs since they existed with CherryPy or SQLObject, etc.
The Django approach was pure enlightenment. Every piece was designed to work together and much work has been put into doing things right. With TurboGears it seems they spend most of their time chasing bugs and creating temporary workarounds.
What most people don’t realize is that, while you can pull out any piece of TurboGears and replace it with another, you can do the same thing with Django. Therein lies the beauty of Python. Anybody who complains that Django is too restrictive doesn’t quite get Python, if you ask me.
Finally, it seems that lots of the critics don’t understand that Django is not new. I suppose they only understand that it is new to them.
Hi James,
Nice writeup! Some of the points raised in defence of Django do make sense.
In my opinion, one considerable pitfall will be consistent co-existence of Django’s ORM and SQLAlchemy. The native ORM does have shortcoming, not merely related to it’s expressive power, but rather related to db-engine support, model inheritance, model migration. In this sense, it’s no longer a match against SA.
The adoption of SA support therefore seemed inadvertible even before the trunk branch was created. At this point, adoption of SA seems to render the admin interface useless.
As a result, a subsequent possible trap might be the admin interface’s reliance on attributes inside the model definition. Maybe DHH had a point in objecting to this strategy…
RoR’s Streamlined project has an increasing array of features that leave Django’s monolithic admin interface behind in many repects.
Jeff Croft’s immediately rejection of Streamlised on the basis of the mere fact that it’s approach is not dynamic in the sense that it depends on code generation might be misleading. After all, what’s dynamic about Django’s admin interface without proper support for model migration…?
Django’s admin could of course be extended and improved, but there seems to be no priority for this. What are Django’s priorities nowadays anyway? Development efforts have been in decline lately and checkins to trunk seem quite unstructured and ad hoc.
Another issue is AJAX support. Django’s slogan in this case is: “roll your own, it’s easy!”. Not binding Django to a particular proprietary AJAX framework makes sense to many. On the other hand, if it’s so easy to ingrate, why not implement helper functionality for at least one of Mochikit, Dojo, YUI, …, ?
But then again, Django is often quoted as a framework with a strong content-publishing focus. Then who needs RoR’s proclaimed ‘appy’-ish AJAX helpers? Moreover, if Django’s attributes would really act in support of content-driven web services - why is there no revision support in the admin interface, or decent auditing and logging capabilities, internationalization support through model inheritance…
These are issue that came to mind randomly while typing this comment. I’m in no ways a RoR fan, on the contrary, but Django has a series of known deficiencies. Deficiencies are unavoidable, but shouldn’t we be honest about them in the first place?
Unbiased scrutiny and developer devotion have brought Django where it is today. Let’s make ik better - starting tomorrow! (or don’t we work on Sundays? ;)
Best regards, bin
Your ruining the reputation here James.
next you’ll be telling people how easy it is to use, and how helpful the IRC channel is.
you don’t actually want people to start using Django do you?
but seriously, I always thought the NIH parts of django were great, as a non-python person initially I wanted something cohesive, and where all the things worked together, and I didn’t have to figure out where to download what, and where the documentation for X was. Django did that for me. After using django now for over a year, I haven’t really seen a need for going out of the NIH areas.
I guess the question about NIH could also be rephrased as, do we want to attract existing python developers to django or non-python developers to django.
Nice entry James. Like Jarad and Ian stated, this integration of components is a something that i find compelling about Django.
The problem I have with Django atm is the 7 i think branches django has open which are “feature complete” but are still to be merged into trunk because of the lack of time which the lead developers have to review the code.
A number of issues which are brought up on the mailing lists and in the IRC channel are requests for features which are essentially complete but aren’t in trunk. People are often saying “when will this be merged” with the response being “when the lead developers aren’t busy”.
Maybe its time to delegate some of the responsibility to the other developers, allowing for development to continue when Jacob and Adrian are tired up doing their day jobs.
No, because that’s not how the SA integration will work. The plan is to keep the Django model-definition syntax the same, and the core model attributes the same across both ORMs — this will involve a compatibility layer which translates between Django and SQLAlchemy. That means that anything which is used to introspecting standard Django models (like the admin) will continue to function.
Django’s admin interface is dynamic, in that it doesn’t generate static views the way most of the Rails “auto admin” plugins do — it generates it interface fresh on every single request by introspecting the current state of the model.
As for changing the fields in a model, well, people who change their models often enough to need automated schema-migration tools are, in my opinion, asking for trouble. It’s not good development practice in anyone’s book to be changing your data models that often, and it’s just bound to get you into nasty problems no matter how good your migration tools are.
I’ve literally written volumes on why that would be a bad idea. Poke through the archives of this blog, or the Django mailing lists, for my reasoning on the subject.
It depends on what you mean by “revision support” or “auditing and logging capabilities”. Django’s admin app doesn’t automatically save diffs of every change to every instance of every model, but it does log every action taken in the admin, with a description of who did what. Being able to arbitrarily diff versions of an object, “roll back” to previous versions, etc. are much larger problems that aren’t tied solely to administration, and are being addressed in a branch right at this very moment — which makes me wonder how familiar you actually are with Django.
And “internationalization through model inheritance”? What does that mean, exactly? We’ve got an internationalization system; I suspect what you’re talking about is more on the lines of content translation, and is an entirely different can of worms (and not one to be solved by “model inheritance”).
Well. I don’t want to offend any of the people who’ve developed stuff in there, but I’d say probably only three of the branches are “feature complete”: multi-db, per-object permissions and generic-auth. The rest are still largely a mixed bag, and most of them exist because we wanted a centralized place for our Summer of Code students to post their work and get feedback.
Of the SoC projects I’m not sure which will eventually land in trunk in some form, and which won’t. I’ve got reservations about a couple of those branches, but that’s a story for another day.
Also, the time of the core dev team isn’t necessarily the issue here — some of the features being developed need someone else to test them, because we literally don’t have use for them. The same goes for the experimental Oracle and MS-SQL support; none of us use Oracle or MS-SQL, and none of us have access to systems running those databases, so we need people from the community to step up and contribute (and, in fact, we’ve been saying that pretty loudly on the lists).
Sweet, sweet acid… Does it take an expert to speak ones mind?
Don’t be annoyed at my apparent lack of familiarity with Django. There’s more to lack than Django… In another sense though, being a Gypsy myself, we share a common heritage.
James, you’re a good Defender of the Crown. Keep it up!
Best regards, bin
Er… modpython_gateway doesn’t have to be third-party. It’s in the public domain, so grab it and distribute it with Django with whatever changes you feel are necessary. Heck, you can even claim you wrote it (but web-sig archives might disagree ;).
Robert, it doesn’t have to be, but it is; if you want WSGI under mod_python right now, then as far as I can tell you have to go to a different site, download an extra module, and configure mod_python differently. It’s hard to tell, from what little discussion there is of it on the mod_python list, whether that situation will change anytime soon. The fact that mod_python and the WSGI module for it are distributed under different licenses could be a bit of a hurdle; mod_python is under the Apache license, while the WSGI module is GPL, and according to the FSF those licenses aren’t compatible.
I’m shocked, baffled, and a little bit flattered that someone would consider my opinion on this to matter at all. I’m not really a programmer, I know very little about programming, and my opinion should certainly not be considered that of an expert or that of the Django team (of which I am not an official part in any way).
As you said yourself, bin — it doesn’t take an expert to speak one’s mind. The code generation seems like a bad idea to me, but just because I said it doesn’t mean anyone should consider it anything more than me spouting off at the mouth. :)
I’m really disappointed to hear that Django isn’t a smug, arrogant framework programmed by elitist @ssholes. That’s why I wanted to use Django in the first place! :P
I find your entry, as usual, very lucid and well thought out.
It seems to me like the whole RoR vs Django vs TurboGears thing boils down to one main difference when it comes to the developers:
Django was grown by developers/designers that had a specific need for it, and then opened to the community… TG was developed for/by the community, and (maybe I’m wrong?) so was RoR… so while those other two might be more inclined to make it “play nice” with everything (that’s their goal!), the Django folk just don’t have a NEED for Django to do that (hence the apparent “arrogance” and “feet dragging” on their part).
Maybe I’m wrong… I don’t know that much about the RoR or TG folk (or Django for that matter!)
That being said, I prefer the Django approach,but in a sensible way… making your own ORM might be a bit much (complex) but I don’t see a problem with making your own template system (its a template system folk, not the space shuttle control program).
I guess I’m one of those that gave the aforementioned impression. I had a good rant about the Django ORM trampling all over the foreign key policy of the database. However, I’m still using Django and will continue to do so. Why? because everything else is pertty much spot on. I’m still trying to work around my little “problem”, but in the mean time using Django for what it does best, getting web sites put together neatly and swiftly.
Most of the issues about lack of Oracle and MSSsql support, as well as my original gripe are hightly likely to go away with the SQLAlchemy integration.
Reading lots of articles, people moaning about year old bugs not fixed, year old patches not yet applied and following the svn development there is an impression that the developers are doing what THEY want to do to acheve their ends. If that helps the rest of us though, then that’s fine. If not, then anyone can write their own code to do their own thing. Who knows, it might eventually make its way into the core code.
I’ve looked very closely at many of the other frameworks, all very clever and currently working well, but most of them feel “disjointed”, being collections of other stuff. I also worry about one part of the framework being abandoned. I know that the plug-in architecture of those frameworks means that you could use another “thing” for the job, but that’s not very comforting if you’ve spent several weeks (or more) constructing your code, only to realise that you’ve probably got to do one patrt all over again because component xzy is the current flavor as opposed to component abc that you are using now.
Django just feels more “solid” and constructed from one end to the other to all work together in the first place.
I was, in fact, speaking about my own modpython_gateway module, which was the first and is not under any license other than “public domain”, by which I mean you can do anything you want to with it. I wasn’t aware of anyone else’s. Grab mine here: http://projects.amor.org/misc/wiki/ModPythonGateway and distribute it with Django. You have my full permission.
I don’t think it was a wise idea to open like that. How many of the same short sighted people who don’t look into things are going to read the first paragraph of your post, take it as your opinion and move on.
Wouldn’t it be in better style to open with an overview or atleast blockquote the opening? Being a well respected web-figure I’d hate for the people to get the wrong idea about Django because they didn’t read the whole post.
That said, it’s your blog and you can write however you want to.
I also wanted to observe some of the comments are excellent. Great to see a blog getting so many comments from so many involved people that’s so cool.
I also wanted to comment that Django has a BSD license and often those other third party packages people say Django should be using are not compatible with a BSD licenses.
Oh and I wanted to “shout out” that I’m against the whole AJAX integration thing for a number of reasons. Having it doesn’t force people to use it but it does add maintenance and reduced diversity, theres so many good ones, just pick one and go with it! I recommend people check out http://jquery.com/ it’s a whole ball of Javascript you might just like. Also integrating an XMLRPC framework goes against the whole idea that Django outputs stuff for HTTP not HTML, HTML might be what gets sent over HTTP a lot of the time but how is a JavaScript library good for the guy who’s putting out PDFs ro CSVs? Why not include one of the plethora of libraries for doing that?
I’d rather the busy core developers worked on merging some of the exciting stuff going on!
I think we can safely assume that anybody who’s thick enough not to read the sarcasm in that first paragraph is somebody we needn’t worry about as a threat.
As a sidenote on the main developer time bottlenecks, even if it was supposed to be a funny ticket closing, remarks like these do support the impression users of Django might get of ‘the crew’. Or: how to scare away your testers from entering tickets…
I’ve been bouncing around web frameworks for years now, in both Java and Python. Django ranks above everything I’ve used. Everything is intuitive, and if not at first, it will feel that way after you realize how slick it was designed. URL mapping, templating, ORM, middleware, template tags/filters… it all comes together so nicely with Django. The amount of “plumbwork” that is required for other frameworks is definitely minimized with Django, allowing a seasoned Django veteran to get a substantial wireframe site up in almost no time. The right combination of Python/Django knowledge, Javascript frameworks/skills, and CSS slick-ness can really produce winning results in minimal time.
I’ll also agree with Django on the WSGI argument. I only use mod_python w/apache for deployed sites, so the built-in Django support is definitely appreciated.
I’ll stand behind Django every chance I get. Thanks for the writeup James!
Fred, that ticket was originally closed by Adrian saying that it was a backwards-incompatible change he didn’t think was worth making, and was reopened with a snarky comment about “if you just don’t have time, say so”. So it was promptly re-closed with “we just don’t have time”.
I don’t see how that’s “scary” in any way; the original reasoning about it being a backwards-incompatible change, and being easily fixed with a custom auth class for people who absolutely need it, was sound, and when it was reopened with snark it was reclosed with snark. Maybe it’s just me, but I like seeing that kind of thing ;)
As usual, Zope has been dealing with this kind of stuff for years now.
Zope has gotten the same criticism from Python developers: Zope is NIH. It requires you to drink a lot of koolaid. And it’s true: Zope did suffer from inventing everything itself. An important reason is because there wasn’t anything suitable available back when Zope started almost 10 years ago. Entirely informed choices were made, not NIH ones. In many areas of the Python web development space, the Zope community was one of the first to have the need.
While a very good excuse, we also have suffered from NIH on occasion. We got used to looking inward for our solutions, not outward to what other Python web developers were doing. We had good reasons when we were one of the first, but those reasons disappeared.
We wanted to play with others, sure, but we invited people to play in our playground more than we joined other people’s playground.
So we set out and tried to fix it. Those involve technical changes (using Twisted, WSGI adoption) and cultural fixes both. We’ve been at this for some years now. Some things we continue to invent ourselves, where we think we should push the state of the art, but we also have a lot of awareness of what we shouldn’t reinvent. Most importantly, our culture has been changing to be more outward looking.
So Django project: expect more of this criticism in the future. You might’ve made an entirely reasonable choice after evaluating alternatives to implement something yourself, only for some library or framework to come along next year that does it a lot better, and people will be complaining Django is suffering from NIH. It might even become true if you don’t watch out - be sure to continue to look outward.
What’s so bad of NIH if your business depends on it?
I would recommend everybody to roll their own framework if their business relies on the platform of their choice heavily.
Google, YouTube, Yelp and others don’t bet their companies on a pre-made framework like Django or RoR. (Mind you Google did use Django internally but only in small groups I think) They use their own frameworks, just like Lawrence Journal uses Django (their own framework) and 37signals.com uses Rails (their own). I guess that leaves TurboGears in a strange land huh? Not having its own mother company.
My only objection in using other frameworks if I have my own company is that if I knew my app is slow because there are bottleneck codes written in Python or Ruby, I might have a hard time trying to pinpoint where it is because I did not invent the framework myself.
Not too long ago DHH wrote a case in his blog where he has to go down and code C in a “bottleneck” section of his (or 37signals) web-app. This dude knows Rails so he could detect problems faster than any of the general users (and also find a solution for it in that matter). I would assume the same applies for Lawrence Journal and Django.
If someone asked me to make a website for them say a small scale website at a community college, definitely I would use framework (and definitely it would be Django). If a city (say Vancouver, the place where I live) needs a website, Django would be still of my choice. But if I want to have my own startup creating web-portal or the next YouTube, I would roll my own framework.
Kudos to Django team for making their framework general enough to cater the needs of Python/Web developer in general.
i have to disagree with the reason “We cant use WSGI middleware because we need to support mod_python”. anyone familiar with WSGI or mod_python knows thats at best a completely lame excuse. there are several adapters available for mod_python, and if they have licensing problems, WSGI/mod_python adapters are incredibly simple to write (feel free adapt the one in Myghty, its BSD licensed), typically 30-40 lines of code - and configuration in mod_python is also quite trivial. both frameworks literally require a single callable which takes a request, dumps out content, returns a return value. some small adaption of the ingoing and outgoing arguments and youre on your way - the Django community could start using WSGI middleware and begin to phase out its own redundant services.
I don’t think the django middleware will be going anywhere anytime soon… while many middleware components (such as gzip…) don’t really belong in django (job of the gateway/server to do that, blablabla) for some middlewares (auth, trans) it just makes sense to keep’em “closer” to django.
Not that I like the idea of multiple “layers” of middleware or anything… maybe they could just call/treat it as something else.
Really? Because, um, I’m familiar with both, and I think it’s a pretty darned good reason. When mod_python ships built-in WSGI support, then I think it’ll be time for Django to think about dropping the old-style mod_python handling stuff. But only think about it, mind you — we have compatibility issues to deal with (Adrian reverted something out of Django just today because of compatibility issues with versions of mod_python we want to support).
I think your contradicting your self here. You say how Django developers are not smug and arrogant, but then you go on to defend design decision Django has made, instead of suggesting and analyzing things they could be doing (like Jeff Croft http://www2.jeffcroft.com/2006/jul/20/top-ten-things-suck-about-django/).
Instead of saying, “The problem is that you can’t always use WSGI middleware or other common WSGI tools. Django is heavily committed to also working under mod_python” You might say: “Until support for WSGI under Apache is complete, Django will choose to use its own middleware.”
Stop criticizing the criticizers and get back to your normal design patterns/ideas/django tips writing!
It was a good read though, and thats why I continually check you writings on web development.
On the whole I think this is both a good write up and good comments. I have one bone to pick though, and that is that better data base migration is needed because, while we shouldn’t need to change our models after the fact, in the real world we do. I could be the best designer in the world, but if my client can’t make up their mind or completely misjudged their needs, I’m probably going to need to redo the models. Sure it’s their fault, but my job is to make them happy. so it happens. For small business development, I definitely need to be developing that way and my clients are never going to be perfect. Personally, I have the SQL chops to do it in bash with sql files, so that’s ok, but it is certainly a minor annoyance and could be bigger for others.
I would have to also say that while I love Django to bits, there is most certainly more arrogance in the Django dev community than I have encountered in other OS tools I use. I would hate to see this get in the way of developer momentum. As many have pointed out, the genius of Linus is not in his code, but in his extremely effective management of the open source approach …