Standardization Solution

In yesterday’s entry I talked a lot about the problems caused by the lack of enterprise software standardization.
I realize I didn’t say much about the problem or a proposed solution.
The problem isn’t a lack of standards – many blog entries and articles have been written about the plethora of WS-* proposals.
The problem is the lack of an independently defined and universally endorsed Web services architrecture. Microsoft has one architecture, IBM has another – Sun supports a variation in J2EE specifications that isn’t the same as either, and so on.
Another problem is the lack of any central authority or leadership over the Web services standardization process. Microsoft and IBM have their own process and have produced a number of specs (most of the leading specs in fact), W3C has its process and has produced some specs, OASIS has yet another process and is working on some other specs, WS-I has another process and has defined some profiles of other specs, OMG has yet another process and has contributed a couple of proposals, and so on.
It’s not about abuse of the community process, as this editorial in SD Times says. The market makes standards, committees don’t. Many specifications created in committe are never implemented, and many specifications created in a closed process never succeed, either.
What’s needed is an independent, user-driven (or user monitored) process capable of resolving disputes based in vendor competition (and of course vendors by definition compete with each other so even in an open process we can’t reasonably expect vendors to always be able to come to consensus on their own – at least not in ways that meet the customers’ main requirements) according to which of the competing proposals best meets user requirements.
Despite the fact that the OASIS folks get pissed at me for saying so, I believe that the W3C would be the best place to try to establish such a process, which is why I raised the issue with them. They already have a good rigorous process, including a strong IP policy, significant and helpful staff involvement, and a track record of gaining adoption in the market. Not to mention the influence of their well-known director, Sir Tim.
We in fact try to define a Web services architecture at the W3C from January 2002-January 2004. I spent the majority of those two years as an editor of the spec, which isn’t bad, but did not really accomplish the goal. The specification contains some good information but in particular we tried and failed to define an architecture that allows the industry to know which among the many WS-* proposals is really a useful and necessary part of Web services, and which isn’t.
One reason we had trouble gaining consensus on that was that the group included strong proponents of both the Semantic Web and Web services, and it is often difficult to find good compromises between those communities. (Thus the reason why I was asking whether “one Web” really makes sense since we have so much trouble getting the two communities to see eye to eye.)
Another, perhaps more important, reason, is that the current W3C process does not really have a good way to resolve disputes rooted in a conflict over commercial interests (which as I said is pretty much the root of the problem). The W3C process is designed to promote consensus in a technical discussion, and it does that pretty well. But there’s really no way to break a logjam caused by a commercial conflict of interest such as whether the Microsof, IBM, Sun, or Oracle architecture is the winner.
The OMG process does include such a mechanism, for example, and years ago when I was working on committees at TMF/SPIRIT, there was a way to raise issues to a customer council for resolution. For another example, X/Open (now the Open Group) had an organization that gave customers and vendors different roles – the customers created the requirements, the vendors created the specifications, and the customers and had an opportunity to review the specifications before they were adopted.
This raises all kinds of other contentious issues, such as the role of customers or specification users in the standards process. Should they be expected to be fully participating committee members, etc. Standards are defined for the use of vendors – that is, vendors are the ones who implement them. But they are established for the benefit of the customers – at least that’s the theory. So there needs to be a good balance, but that’s tricky. As I said it’s the market that determines a standard, and therefore vendors with marketshare try to influence customers to adopt the specifications they like.
It is not reasonable, however, for customers to expect vendors to be able to agree by themselves on the adoption of useful, meaningful standards – that is, the ones that truly are able to solve the software productivity issue – since different vendors will emerge from the process as winners or losers, depending on how well their products align with whatever is adopted. We see this kind of struggle in the consumer electronics field all the time, such as the current struggle over the next generation DVD format. This is, again, why independent leadership is so important, since progress toward standardization inevitably causes lumps and bumps.