The future of web standards
The world of standards-based web design and development has been undergoing something of a shake-up these past few days; Andy Clarke’s “CSS Unworking Group” seems to have opened the floodgates to expressions of dissatisfaction with the current method of progress (or lack thereof) in developing and standardizing new features for web developers and designers. Alex Russell’s “The W3C Cannot Save Us” and my friend and former colleague Jeff Croft’s “Do we need a return to the browser wars?” continue the theme, as does Stuart Langridge’s “Reigniting the browser wars”, which popped up as I was finishing the first draft of this post.
Ultimately, I think this boils down to two problems.
The first problem, not to put too fine a point on it, is that progress in developing new standards is glacial at best. HTML went from initial concept to version 4.01 in less than a decade, but has stayed pinned at 4.01 since before the turn of the millennium (XHTML is no better; XHTML 1.0 was deliberately identical to HTML 4.01 except for the XML syntax, and XHTML 1.1 really didn’t add much, focusing mostly on reorganization and modularization). Similarly, CSS has been sitting at version 2 of its spec since 1998; CSS 2.1 is still no more than a “Candidate Recommendation”.
The second problem is that innovation on the Web today is largely taking place through leveraging non-standardized or even proprietary technologies: Flash is popping up in all sorts of unexpected places, Microsoft and Adobe are both working on next-generation “rich internet application” runtimes and the biggest buzzword of them all — “AJAX” — is based around a formerly Microsoft-only technique (XMLHttpRequest) which has found its way into other browsers.
The combination of these two problems leads to a very real worry: that whatever chance there might have been for a truly interoperable Web will vanish, with useful content and applications disappearing into the proprietary walled gardens which provide the only genuine opportunity for new features and capabilities.
The two problems — the slow pace of standardization and innovation through as-yet-non-standard features — are obviously intertwined. The question at hand, then, is how to solve them together: how to produce a process whereby standards bodies can respond quickly to innovative ideas, and wherein innovators will encourage standardized, interoperable implementations of their ideas.
While I don’t have a solution, I do have some thoughts on some of the problems with the discussion so far, and some ideas on where to look for a successful model to follow. So let’s dive in.
The false dilemma of standardization models
One of the major issues in resolving this problem is that huge amounts of discussion have essentially assumed a false dilemma, that there are only two ways to run a standards body:
- A closed-door, pay-to-play system, as the W3C is perceived to be.
- A howling mob which runs by consensus of the participants.
Of course, these aren’t the only two options, but quite a few people (Daniel Glazman, for example) seem to behave as if they are. The first thing we need to do, then, is throw that out and recognize that there’s actually a fairly broad continuum of options in between these two extremes; in other words, what we should be looking for is a balance between the input of people who use and develop for the Web, and people who develop browsers and attendant technologies.
Finding the balance
This brings us to a new question: how do we find the proper balance between the competing interests of Web vendors and Web users/developers? Personally, I think the answer is to look at the available history: the world of web standards is not really breaking new ground in needing to properly strike this sort of balance, and there’s already a long and rich history of groups going through precisely this process, which anyone who’s interested in reforming web standards should be looking at.
I’m primarily thinking of open-source software development, which has been through this already many times over: there’s plenty of open-source software out there which needs to appeal both to “average joes” and to multinational corporations, and even a few examples of projects which have successfully done so. The Linux kernel is one such; its model is neither “closed-door” nor “howling mob”, but rather something in between. Linux is not a dictatorship, but it is not a democracy; Linus Torvalds and his trusted “lieutenants” ultimately retain control over the project, but input from any interested party is accepted and taken into consideration. And though not all opinions are equal, the process of deciding which opinions are given more weight than others seems to be largely pragmatic and meritocratic.
The result is that large corporations can participate and contribute without turning Linux into a closed-door effort, and smaller/independent developers can participate and contribute without turning Linux into a howling mob. That’s quite an accomplishment, and it’s something that should not be overlooked by anyone who wants to reform web standards.
And the same is true in many other successful open-source projects: Perl, Python and Ruby, for example, all have open development processes, but ultimately remain in the control of a single “BDFL” and/or a few “lieutenants”. And the success of these projects across a broad spectrum of their target markets shows that this process can work extremely well: a general patten of open input and discussion, with a few experienced and trustworthy folks who steer the process and have the power to make final decisions when things would otherwise get bogged down, manages to avoid the downsides of both of the extremes of the false dilemma outlined above.
W3C’s on first, WHAT’s on second
Of course, this raises the question of whether the WHATWG, which has been working steadily on refining, improving and extending various standards for several years, will be able to accomplish the same thing in the web-standards world that Linux and other successful open-source projects have accomplished in the software-development world.
At this point I don’t honestly know, and I don’t think anyone really does, though there’s no shortage of opinions. People far smarter and more experienced than I have come down on both sides, with some feeling that WHATWG is the right way to go and others feeling that it’s a fool’s errand. There are some encouraging signs, though:
- Though the group is open and anyone can join and participate, it is very definitely steered by its own “BDFL”: Ian Hickson, who possibly knows more about web browsers and the browser industry than anyone else alive. Ian’s participation in WHATWG is extremely similar to that of BDFLs in successful open-source projects: he knows the subject matter as well as anyone, is willing to entertain and respond to input from practically anyone, and — speaking as someone who’s been reading the WHATWG list off and on since September 2004 — has a good track record of sifting through the conflicting interests and finding good compromises.
- Despite being an extremely young group, the WHATWG has already produced some notable results; Apple, Mozilla and Opera are all actively working on implementing various stable portions of WHATWG proposals, and the group has shown a willingness to respond to useful features which do not originate within itself (as in the case of the offline storage API, which benefits by taking cues from, and cross-pollinating with, Google Gears).
Some folks will inevitably point to the recent tempest in a teapot over media codecs as an example of the WHATWG being too beholden to corporate interests, but I’m not convinced (though, to be fair, I’m also not convinced that the HTML spec should be in the business of telling people which media formats to use); unlike many people who began frothing at the mouth when they heard about it, I actually followed the discussion and so I don’t really see anything to complain about. The spec no longer specifically recommends Ogg Theora, but it still leaves implementors on the hook to provide support for unencumbered, interoperable media. And in the long run the “unknown unknown” (to borrow a phrase from the Tao of Donald Rumsfeld) that represents may prove worse for large companies than the known unknown of implementing Theora; the current language of the draft is somewhat akin to a dentist telling you that painful tooth will have to come out someday.
At any rate, it’s still too early to tell whether the WHATWG will work, especially for the case of CSS; the WHATWG’s work to date has largely focused on HTML and the DOM, as indicated by the current WHATWG draft specification.
The Microsoft bogeyman
The largest perceived obstacle to any reform of web standards is a big question mark surrounding Microsoft. Though a member of the W3C and a participant in various working groups, Microsoft’s implementations of the resulting standards have ranged from lackluster to laughable and this, combined with Internet Explorer’s dominant market share in the browser world, naturally leads to a concern that any effort which doesn’t have a firm commitment from Microsoft will inevitably fail.
But at this point, honestly, I’m not convinced that Microsoft is relevant. At least, not in the way everyone seems to think they’re relevant.
If you’ve never read Joel Spolsky’s article “Fire and Motion”, do so now, because what I’m about to argue won’t make much sense if you haven’t. Go ahead and read the whole article, because it’s good, but the thing you want to take away from it is the fact that a large part of Microsoft’s dominance is due to the phenomenon Joel so eloquently describes by analogy to infantry combat tactics:
The competition has no choice but to spend all their time porting and keeping up, time that they can’t spend writing new features. Look closely at the software landscape. The companies that do well are the ones who rely least on big companies and don’t have to spend all their cycles catching up and reimplementing and fixing bugs that crop up only on Windows XP. The companies who stumble are the ones who spend too much time reading tea leaves to figure out the future direction of Microsoft. People get worried about .NET and decide to rewrite their whole architecture for .NET because they think they have to. Microsoft is shooting at you, and it’s just cover fire so that they can move forward and you can’t, because this is how the game is played, Bubby.
Now, stop and look at Internet Explorer 7 (er, I mean, “Windows Explorer 7”). IE7 is, despite what Microsoft’s PR people would like you to believe, the result of Microsoft being pinned down under covering fire from the rest of the industry. Mozilla, Apple and Opera were pelting them with fire from all directions — tabbed browsing, better privacy, better security, all sorts of features people actually cared about — and Microsoft found itself on the receiving end of its own strategy. And, just as attempts to constantly keep pace with Microsoft resulted in lots of software vendors turning out sub-par products, Microsoft’s own response to the unfamiliar need to play catch-up resulted in a sub-par browser.
The same thing is actually happening to Microsoft on several fronts right now:
- The increased visibility and success of Apple means Microsoft is playing catch-up in the operating-systems business. The result is that Vista was a flop; all the features which might have been interesting were gutted long ago in an attempt to get the thing out the door and provide a public competitor to OS X.
- The utter dominance of Flash on the Web, and the new threats of AIR and JavaFX, have Microsoft playing catch-up on another front, resulting in the extension of their desktop APIs into the browser through Silverlight.
Microsoft is pinned; they’re stuck trying to catch up to what everybody else is already doing, while the competition just keeps piling on new features and new technologies. They’re not dead yet, of course, and are far from it: IE is still the dominant browser and Windows is still the dominant operating system. But there’s definitely been a sea change in the industry: Microsoft, the unstoppable juggernaut, is vulnerable and is having to play catch-up to maintain its dominance. To borrow a phrase from Lewis Carroll, they’re having to run as fast as they can, just to stay where they are.
And so Microsoft really isn’t relevant to the future of web standards; any compelling new development that comes from the rest of the industry will be just another form of fire and motion, and Microsoft will have no choice but to keep pace, regardless of whether they participated in the process.
Where to go from here
I don’t really know. Right now I’m just making some observations and engaging in a little analysis of the problem, because that’s what I do. I do think that the major points I’ve outlined above are all important to any actual solution:
- Any replacement process will need to strike a balance between the “closed-door” and “howling mob” extremes, starting with the realization that those aren’t the only options.
- The open-source software world is probably a good place to look for successful ways to do this.
- The WHATWG shows promise, but it’s too early to tell if it’s the real solution.
- Microsoft’s participation or non-participation is no longer relevant.
But beyond that I’m not sure where to go. For now I’m keeping my eye on the WHATWG (as I’ve been doing for several years, because for the most part they’re the ones who are actually getting things done), but something better may certainly come along. I think an important next step for the folks who are dissatisfied with the status quo is to decide what to do. Andy Clarke has proposed some ideas, but amusingly derides the idea of operating by consensus whilst proposing things that could only work if accepted by consensus (and largely seems to fall into the false dilemma outlined above). It’s all well and good to know that a different direction is desired, but it’s equally important to know which direction you want to go in. So I’ll close with a thought to that effect from one of my philosophical heroes, G.K. Chesterton (from the opening chapter of his wonderful book Heretics):
Suppose that a great commotion arises in the street about something, let us say a lamp-post, which many influential persons desire to pull down. A grey-clad monk, who is the spirit of the Middle Ages, is approached upon the matter, and begins to say, in the arid manner of the Schoolmen, “Let us first of all consider, my brethren, the value of Light. If Light be in itself good—” At this point he is somewhat excusably knocked down. All the people make a rush for the lamp-post, the lamp-post is down in ten minutes, and they go about congratulating each other on their unmediaeval practicality. But as things go on they do not work out so easily. Some people have pulled the lamp-post down because they wanted the electric light; some because they wanted old iron; some because they wanted darkness, because their deeds were evil. Some thought it not enough of a lamp-post, some too much; some acted because they wanted to smash municipal machinery; some because they wanted to smash something. And there is war in the night, no man knowing whom he strikes. So, gradually and inevitably, to-day, to-morrow, or the next day, there comes back the conviction that the monk was right after all, and that all depends on what is the philosophy of Light. Only what we might have discussed under the gas-lamp, we now must discuss in the dark.
Feed of latest entries
December 17, 2007
#
Interesting insights that Apple/Opera/Mozilla (among others in many areas of the markets in which Microsoft competes) have actually been forcing Microsoft to play catch-up. Maybe that explains why Microsoft won’t commit to supporting the standards work that’s coming out of the triumvirate’s collaboration: they’re too far behind and don’t really want to encourage more standards to be developed.
December 17, 2007
#
Thank you James for an insightful journey through the web standards problematics. I find it really refreshing and promising that there are still people like you who don’t use topics like web standards to paint black and white but who rather do something to explain what it all boils down to. I’ve read too many articles on web standards recently that felt like short sighted populist speeches that lead nowhere.
December 17, 2007
#
I’m with you all but for a couple things. First CSS 2.1 being “no more than a ‘Candidate Recommendation’” is hardly a negative thing. From the W3 Process page:
From a developer’s perspective it’s done — the spec is depending on browsers to implement it. So the ball is in the browser vendors’ court, not the W3C’s.
Second, what of the relaunched HTML WG (which is really just the WHAT WG legitimized in W3C clothing)? Arguably some of what you describe is already at work there. Ian Hickson, Dave Hyatt and Maciej Stachowiak are some of the most meritorious browser-level minds and are leading the discussions there.
Of course there are flame wars and idiots abound, but that will never be prevented outright; it’s more important to see how those disputes are handled and how forward progress is ensured. As far as I can tell there has been steady forward progress since the group’s creation, despite the naysayers’ and zealots’ apparent will to halt progress in the face of imperfect development.
As for Microsoft, I think they still have the dangerous power to separate from the standards community as a whole and pursue selfish goals along with the separatist zealots who are already frothing. It’s easy to forget that the vast majority of web authors and consumers couldn’t care less for web standards and, furthermore, use Microsoft products almost exclusively on a daily basis. A split with Microsoft on the parting side would be disastrous for the open web in general and should be prevented consciously by all of those with power in the web community.
December 17, 2007
#
Ideal scenario. Microsoft releases their quirks mode rendering engine as open source. Mozilla et al adopts said quirks mode renderer - saving them from wasting their time and effort in re-inventing the (not round) wheel. IE8 adopts the Mozilla standards mode rendering engine - saving them time and effort in trying to drag their apparently terminally broken code into the future. At a stroke, all Windows browsers feature the best of both worlds. And web designers can code with either Microsoft tools or to standards safe in the knowledge that whatever code they write it will just work. Everyone benefits (except Apple and Linux).
Likely scenario. Microsoft release IE8 with non-standard standards mode. Web designers face another five years of frustration.
December 17, 2007
#
EXCELLENT article, i think this is one of the top 3 so far on programming reddit…
Now, after the deserved and honest praise :) I must say that I have big doubts that the W3C will manage to reach to the community on that.
As sad as it is, flash is some kind of standard already. I see only more and more proprietary solutions emerging, and the W3C is dependent on that income so… no real way to survive it.
There is only one way I think would make sense and this is: -) If an open standard makes more “sense” or gives you an advantage. I.e. .ogg is used instead of .mp3 sometimes because you want to avoid paying royalties. Exactly these little advantages must be combined together.
December 17, 2007
#
Just one remakr…
“Ideal scenario. Microsoft releases their quirks mode rendering engine as open source.”
I dont think this will ever happen. Have you looked at Microsoft’s XNA? Porting it to Linux would not be that difficult. The problem is that MS forbids this - thus securing their monopoly on the GAMES market.
December 17, 2007
#
Interesting stuff, and thought-provoking. However I remain unconvinced of this oft-repeated notion, that Microsoft are irrelevent. We will be lumped with the Internet Explorer albatross for many years to come, and even if Apple/Opera/Mozilla collectively invent the browser equivalent of cold fusion, the majority of people on the web will remain Internet Explorer users through ignorance, familiarity or because it’s what their employer/school/university gives them.
I see three potential routes out of this situation. One: Apple’s desktop share continues to increase until it reaches some sort of tipping point and people are buying Macs left-right-and-centre (okay, that’s sort of happening already, but they still remain a tiny percentage). Two: Opera’s lawsuit is successful and obligates Microsoft to start bundling alternative browsers with Windows (which? Opera’s?). This has so many ifs buts and maybes that it’s hardly worth betting the house on. Three: Microsoft release Internet Explorer 8, which by some miracle happens to support standards well…
So, to quote Private Hudson in Aliens; we’re in some real pretty sh*t now.
December 17, 2007
#
I agree with the others — Microsoft may be irrelevant from a super-high level, as in “We don’t need for IE7’s tabbed browsing, it’s a good idea, let’s just do it today!”.
But implementation is a different story. I still can’t use PNGs with alpha transparency (without tricks) because of IE6. If I did, some large percentage of my traffic would see a broken experience.
Microsoft continues to have the single largest influence over the technologies I can put on my webpages. That’s relevance.
December 17, 2007
#
Ben, all it took for Firefox to steal noticeable market share from IE was tabbed browsing, a perception of better security and some good word-of-mouth marketing. Imagine what could be done by, say, showing a sales manager a contact-management app that runs offline as well as on? It wouldn’t have to be blatant — none of that hideous “best viewed in Browser X” stuff — but just the same sort of subtle “hey, did you know…” marketing that’s already getting the job done.
And, frankly, the way Microsoft is already reacting shows that they think they’re on the defensive even if they’re technically not.
December 17, 2007
#
Seconding Brad Fults, to some extent at least. Your article would carry more weight if you demonstrated better grasp of the facts upon which you lay your arguments, although admittedly individually minor. With regard to CSS, you say it has been stuck at version 2 (or revisions thereof) since 1998, but CSS3 has in fact come along very nicely and has even started to get implemented in WebKit.
Personally, I’m concerned a lot of the anti-standards people are throwing out the proverbial baby with the bath water. While IE7 isn’t quite there yet, we are finally to the point where all the main powerful features of CSS2 have total support amongst the other browsers, along with standard XHTML or XML. I personally definitely remember the browser wars, and they SUCKED from a developer’s point of view. It was a pain in the ass to have to write many different versions of a page, and to be so heavily restricted to the lowest common denominator. Even if it isn’t perfect, having XML that can simply be turned into anything with XSLT and CSS is very very pleasant. I can go all out for Firefox/Opera/Safari (or anything WebKit based which is more and more), safe in the knowledge that I can also make something to fit IE with just another stylesheet. No nasty hacks. Also can target screen readers, bots, and other non-traditional viewing agents.
I certainly appreciate the problems with some of the standards that exist, and I wouldn’t mind seeing a faster way of progressing along (such as having an “unstable/stable branch” like in a lot of OSS). But at the same time a lot of what we have is in fact really good, and I don’t want to have that get thrown away purely for the sake of change. Having a lot of people be able to use something, even if it isn’t the technically best, has a value all by itself.
December 17, 2007
#
The single most useful thing Microsoft could do in IE8 would be to support SVG. I don’t really care about the future of html, css or javascript anymore - all I want is universal support for SVG.
December 18, 2007
#
I’ve been doing standards-based web development since 2001. I know the specs inside and out. I follow the relevant mailing lists and other discussions. I’m at a loss as to which facts I’m supposed to have a “better grasp” of.
No new version of CSS has reached W3C recommendation status in nearly ten years. This is what you call “coming along nicely”?
Where exactly do you get the idea that I, or anyone else in this discussion, is “anti-standards”?
With IE6 still significant and IE7 still subpar, I find that somewhat unlikely.
You argument would carry more weight if it appeared that you’d actually read what I wrote.
December 18, 2007
#
Dunno what version of IE7 you use Xoa but there isn’t enough of CSS2 properly done in it to make a good site without hacks/workarounds. Heck, there’s still some CSS1 things they don’t have right yet. IE7 also completely fails at handling XHTML properly and instead either offers you a download prompt when viewing a proper XHTML site or it falls back into quirks mode and starts rendering everything wrong. Not acceptable at all and hardly the support you talk about.
James has written a well thought out and articulate article, which you appear to have read about 2 words of before trying to attack him. I will admit that if you know what you’re doing, a script (server side mainly, too many hacks needed to make cross-browser JavaScript work right, most browser detections scripts don’t actually work) can be used to serve up different style sheets without hacks, but why is there another style sheet? It should just work, needing another style sheet is the same as needing hacks, you’re just doing it in a different way, since hacks are just used to serve different style information to different browsers.
I will admit that CSS3 has some nice stuff in it, but at the rate of adoption I’ll be using DreamWeaver 20 to manage my home, life, and all my sites (yay, feature bloat) before CSS3 can be reliably used on the web.
December 18, 2007
#
Another great article. Thanks for the wrap-up.
Without wanting to detract from the context at all, might I recommend a spell check on future posts? I know you’re probably pressed for time, James, but clarity — especially in this uncertain world — is still everything. :)
December 18, 2007
#
Its an interesting angle you take with Microsoft. I am not convinced you can just discard their relevance though.
Its pretty clear MS did the absolute minimum to “catch up” with IE7. The average Windows user no longer perceives a benefit to installing an alternate browser. For MS this is a job well done and it buys them some time to re-arm. Steve Ballmer has been quoted on Linux saying “[…] cancer that attaches itself in an intellectual property sense to everything it touches.” MS are still interested in maintaining proprietary technologies and forcing lock in, but now days they have better camouflage.
Let hope you are right about them not being relevant to future web standards.
December 19, 2007
#
This is probably one of the more well-considered articles I’ve had to read this week. :) I especially thought your last quotation was very pertinent.
Brad Fults is right, CSS2.1 is pretty much done in the writing-the-specification dept: the ball is indeed in the browser developers’ court. CSS2.1 cannot move forward any further without a test suite and two implementations that pass it.
Note also that according to the current W3C process, CSS1 and CSS2 would not be Recommendations, they would be only Candidate Recommendations. The process was changed to include the CR status and the test suite+implementation requirements after REC-CSS2 was published.
December 20, 2007
#
Best thing I’ve read on the topic. Really, very well done.
December 20, 2007
#
Nicely done. Interesting because today I was listening to Douglas Crockford on YUI Theater discussing some of these same issues and his frustrations with the standards bodies.
December 21, 2007
#
Hi James. Thanks for a fascinating read. There’s been a lot of hyperbole around this issue recently, and the throw the baby out with the bathwater approach has gotten a lot more attention than it deserves. I’m not sure I agree that any stakeholder’s participation is irrelevant. I would rather say it was relevant and preferred but not essential.
I’m trying to find my way with some embryonic thoughts about standards relating to the core Web fonts, @font-face and the future of Web typography, so this was particularly useful. Thanks.
December 22, 2007
#
Thanks, James. You really opened a young developer’s eyes to a problem that is undoubtedly going to blow up in our faces quite soon. Thanks for the links to the other bloggers out there as well.
Here is what I don’t understand. I still feel that a lot of browsers are missing some of the correct XML XHTML and CSS2 standards. Won’t moving forward almost make those standards obsolete and more likely to be ignored? I feel like this is a real reason as to why W3C is so hesitant to move forward. We are just now seeing in Mozilla FF3 and IE8, whose predecessors hold 95 percent at least of active Web browsers, true adoption of those standards. Imagine what the future holds for developers who finally don’t have to open up a second machine to check that their Web pages work on all fronts.
This whole debacle has my head spinning, and I have to agree with you that it is hard to hold an opinion because every choice has clear unintended consequences. Thanks again.
December 24, 2007
#
Interesting article. You said:
The whatwg, the blogs, the different mailing-lists, the bug trackers are among the sources of input of the W3C HTML 5 document to be published as a first Working Draft in January 2008.