More on WS-* Complexity

Poking around the Web today I noticed a few more posts on the subject of Web services complexity. A pretty good summary is available here.
Like Mike Champion, I am a veteran of many, many REST vs SOAP debates at the Web Services Architecture WG (check the public email archives and you will see all every argument ever made ;-). The trouble with many of the arguments, as Geoff Arnold points out so eloquently here is that the REST folks (and ok, maybe my friend Mark Baker is on the extreme here, but I’m not sure they aren’t all zealots) seem to believe that everything else is wrong.
As I tried to point out in my previous blog entry on this topic, I pretty much agree with Mike that it’s a case of finding the right tool for the right job. But as I also pointed out, Amazon.com said last year that 20% of their developers use SOAP. This is exactly the kind of information you won’t hear from the REST folks since they don’t like to acknowledge the possibility that other valid solutions exist.
One funny thing about this kind of noise starting again (or maybe it just never went away, maybe it was me that wasn’t paying attention for a while), is that a new study is just out saying that the Web services market is exploding. The projection is $950M in 2004 and $6.2B by 2008.
Granted, the article doesn’t say much about the basis for the study, but it does characterize the market as being appropriate for “unified corporate portals” and helping solve the enterprise integration problem. Those of us who work in the enterprise software field know that the enterprise integration problem remains among the CIO’s top concerns (it is often cited as the top). Web services, and SOAs based upon them, have tremendous potential for addressing this problem, and therefore are of tremendous significance, at least for the world’s IT systems.
Simon St. Laurent’s blog provides a good perspective on the whole “XML vs SOAP” debate, albeit acknowledging he has no interest in enterprise software issues. I would point out though that CORBA has succeeded in exactly the place where Web services are trying to get – as an enterprise integration technology, and an enabler of SOA. Most of the world’s largest SOAs today are implemented using CORBA. So CORBA is not a bad comparison.
Among other things, though, Simon points out a blog by Nelson Minor on the topic of doc-literal Web services. Most of the time, the REST folks who raised all the fuss at the W3C preferred to contrast REST to the rpc-oriented Web services style.
Some of us have been saying for years now that Web services aren’t very good RPC technologies. In a recent blog entry I mentioned that using doc literal is probably the best way to use Web services. In his blog, another former WSA veteran, Chris Ferris, said that he agreed.
Typically the REST folks don’t like to hear things like this from Web services advocates, either, since it also tends to undermine their arguments. They also don’t like to consider a mixed world in which the REST approach may have its place for simple interactions and Web services may be needed for more complex transactions.
Personally, I like the idea of using XML over HTTP for everything, but I don’t think it’s practical. Too many other transports are already in use — you can’t just rip everything out and replace it all with XML/HTTP equivalents. (Although from a certain perspective, it appears this is what’s happening with the WS-* specs, it isn’t. The WS-* specs are defining features that primarily map to existing execution environments rather than replace them.)
I also agree that the whole WS-* thing is getting out of hand. My interpretation of the new WS-Transfer and WS-Enumeration specs is that they are intended to provide XML/Web services equivalents for the .NET (previously Windows DNA) concepts of ADO record sets (ok, so we now also have JDO for the Java world). And maybe this is necessary because IBM et. al are taking Web services off toward the Grid with the Web Services Resource Framework set of specs, and not everyone is interested in the Grid (or it’s too much of an IBM thing). But the problem with the WS-* specs isn’t a technological problem, despite some REST folks wishing it were. It is a vendor politics issue, and it simply needs independent leadership.
Referring again to CORBA, one of the reasons for its success as enterprise computing infrastructure (and granted this is an historical success, but nonetheless one any standards-based technology can aspire to – and of course it’s also worth pointing out the J2EE dependency upon CORBA) was that it had an architecture, (pretty) clearly defined and administered by OMG. The WS-* community badly needs an independent body (like OMG was for CORBA vendors) to determine whether specs like the WSRF or Transfer/Enumeration are relevant and appropriate to include.
With all due respect to Don, Chris, and Felipe, who have written a very good document on Web Services Architecture, the responsibility for defining the Web Services Architecture is not a single vendor’s. I realize that I spent almost two years in a failed attempt to help get W3C become that place and write that spec. I also realize that many of the REST advocates felt that it was an appropriate failure.
By the way, does anyone know of a recent REST market study? That might help clarify the debate somewhat.

Advertisements

23 responses to “More on WS-* Complexity

  1. Hmmm – since you trackbacked me (and there’s a hefty load of bile here, though not directly aimed at me) I guess I feel the need to respond.
    From my perspective, the problem isn’t that “the REST folks don’t like to hear things like this from Web services advocates, either, since it also tends to undermine their arguments.” The problem is that the SOAP and WS-* people tried to sell their specialty product to a very broad audience.
    I’ll admit that there may be people who actually want the features SOAP provides and REST doesn’t, maybe even for reasons other than relentless marketing. Counter to the past few years of Web Services propaganda, however, I’d also argue that they’re a minority of cases, in projects if not in wallets. Most of the time using SOAP is just adding wasted overhead in the service of an architecture that isn’t generally necessary.
    Most developers don’t need CORBA, nor do most most developers need Web Services. The real problem here from my perspective isn’t whether or not SOAP sucks, but that gleeful vendors tried to pretend the market for it was much larger than it actually was, and weren’t keen on hearing from people who pointed out that most of the time SOAP isn’t a particularly clean solution. In fact, much of time, it’s poison.
    XML-based technologies seem particularly susceptible to the “if we standardize it, everyone will use it” fallacy. Somehow people seem to have absorbed the standardization aspect of XML while missing its flexibility and the fact that it was explicitly designed as “SGML for the Web”. They’ve just kept going with an insane urge to create more sort-of standards…

  2. Simon,
    Boy, that was fast! Thanks for the reply.
    I certainly did not intend any of my comments as bilious, although with this whole debate how things are interpreted often depends on where you’re coming from (and again, no, this isn’t aimed at you, just my general experience in this seemingly endless debate).
    I for one actually believe in a middle ground, but unfortunately I don’t seem to have much luck getting any recognition for that from what I call the REST crowd. I think doc literal is the way to go for SOAP, not rpc-oriented.
    I am also a big fan of the Web and I am among those traditional old distributed computing guys who argue in favor of adapting those systems toward the SGML model. But in my life I have always had trouble with those who view things dogmatically, religiously, or in terms of simple black and white.
    If I say I do not think the REST folks are entirely wrong, that isn’t saying I think they are entirely right. So not only can’t I be one, but I have to be argued with.
    Eric

  3. > By the way, does anyone know of a recent REST market study?
    Gartner’s Web services hype cycle 2004 piece http://www3.gartner.com/DisplayDocument?doc_cd=120919 (subscription only or pay big bucks, but the TOC is revealing) is the nearest thing I’ve seen. It notes that REST is at the peak of hype (a bad thing) and XML over HTTP is off the hype charts (a good thing).
    Now that’s a bit puzzling, since an awful lot of people use “REST” and “XML over HTTP” as synonyms. How can one be as comfortable as an old shoe, and the other something that only zealots would possibly find a use for? Having been tutored by Mr. Baker and Dr. Fielding on this topic for a couple of years, I think I understand the distinction, but one wonders what the Gartner analyst had in mind?
    My guess, assuming that he has followed this debate in the excruciating detail to which Eric and I have been subjected, is that “XML over HTTP” involving using HTTP as a *transport* mechanism to move around documents that are interpreted in a fairly application-specific way, e.g. as an invoice. “REST” would involve using HTTP or a compatible protocol as a *transfer* mechanism to create (PUT), read (GET), update (POST) and delete (DELETE) representations of abstract resources in the way that Fielding describes and Baker prescribes. In other words, you can use XML over HTTP in a non-RESTful way, and Gartner appears to be concluding that the vast majority of users do just that.

  4. Mike,
    That’s an important distinction, and I will definitely be more careful about equating XML over HTTP and REST in the future.
    It does also mean that when Amazon.com says their developer community is using XML over HTTP, it does not necessarily mean they are using REST.
    Eric

  5. “If I say I do not think the REST folks are entirely wrong, that isn’t saying I think they are entirely right. So not only can’t I be one, but I have to be argued with.”
    Well, as long as you keep selling SOAP as a general solution, I’d be surprised if they didn’t argue with you.
    Using doc-literal solves one aspect of SOAP’s ugliness, but it’s hardly an answer to either REST’s architectural objections or the broader “do I really need this mess anyway?” questions that seem to be voiced more and more frequently.
    Using doc-literal is a good start. Dumping the extra SOAP tags and surrounding infrastructure where possible is better. (Sam Ruby pretends the SOAP markup doesn’t hurt anything, but he’s hardly a neutral player either.)
    If you want to stop arguing with REST people, you might at least come up with something like:
    “Plain XML over HTTP – RESTful work – is fine for most situations. If by some chance what you really want is an XML-based equivalent to mainframe messaging systems, come over here and look at this WS-* stack…”
    That preserves your market while giving up on the “we’re better than the Web” rhetoric of the early Web Services marketing period. The strategies of pretending REST is a weak alternative or not even mentioning it (as Jeff Webb noted and as the MSFT WS overview does) just create easy targets. And I haven’t heard a lot of REST advocates argue that the Web is or should be the equivalent of mainframe messaging.
    (Honestly, I’m surprised that the reaction to the vapid and largely unfulfilled promises of the early WS days hasn’t been stronger. The REST folks seem pretty mild to me, all those PowerPoint presentations considered.)

  6. People do XML over HTTP because it’s cheap and easy, no question. I think REST is what emerges when people have been using HTTP for a while and pause to think about how to do it right, as well as cheap.
    (Which I think means Gartner’s data is registering meme awareness, not actual practice.)

  7. Simon,
    Ok, but I guess there’s still the middle ground to be found. In beween all the WS-* specs and the ones that are really useful.
    I could say what you suggest, but it’s the opposite of what I think. I think that SOAP and WSDL are fine for most applications, but if you really need to use XML over HTTP, and hand code some REST interfaces for some simple Web-facing interactions, then you can and should.
    I am not sure I want to get anyone to stop arguing, just to stop thinking as if there’s only one possible solution to the entire range of distributed computing problems.
    I also would recommend that anyone who’s thinking of publishing Web services take a serious look at exposing the same service in two formats the way Amazon.com does.
    With regard to the vendor hype – sure, that happens. It’s been a fact of life in this industry for a long time. I am not sure what actual harm you think it does.
    As I told Alice LaPlante last week:
    http://www.webservicespipeline.com/trends/47902916;jsessionid=SKTMVX0MLW0KMQSNDBCSKHQ
    I think what everyone should be paying attention to is what people actually end up using commercially. If Amazon.com is one of the best examples of commercial use of Web services, then it’s good to pay attention to what they’re doing and try to understand what’s really getting adopted and used. And it’s both formats today.

  8. WS Complexity

    The WS Stack has a variable geometry – constantly taking on a different shape, with different architectural properties.

  9. “Ok, but I guess there’s still the middle ground to be found. In beween all the WS-* specs and the ones that are really useful.”
    Perhaps I’m misreading this, but if the middle ground you see is _within_ the WS-* specs, that may explain why the REST people aren’t flocking to your ideas.
    “I think that SOAP and WSDL are fine for most applications, but if you really need to use XML over HTTP, and hand code some REST interfaces for some simple Web-facing interactions, then you can and should.”
    I think the reverse. SOAP and WSDL are nonsense for most applications, merely separating programmers from what’s actually going over the wire, letting them pretend they’re not responsible for it. It’s not like developers using REST and XML+HTTP typically spend weeks writing their own XML parsers and processors anyway. I argue that developers should SOAP and WSDL _only_ if forced to do so. There are a few unfortunates who really need all those extra capabilities SOAP provides.
    If you can’t see the “actual harm” vendor hype causes, I doubt it’s worth explaining.
    I agree that “what everyone should be paying attention to is what people actually end up using”, though I find your limitation to “commercially” to be putting blinders on for no useful reason.
    In evaluating that use, I think Mike’s right that XML over HTTP should be looked at as a separate category from REST, but I have to say that given their status as technologies whose foundations have been stable for ever-closer to a decade and which have no marketing staff, both of those technologies are doing well.
    Perhaps best of all, they show little sign of drowning themselves in new specifications, and will be here for a long time to come, whether or not the RESTifarians and the WS-* community come to terms with each other.

  10. Simon’s doing a fine job at representing my views, in particular the bit about the middleground. As I see it, there are many architectural styles which have adopted interface constraints (REST, styles that describe email, FTP, IRC, instant messaging, etc..), and many which haven’t (SOA/WS, OMA/CORBA, DCE, …), and that will continue to be the case for all of time, I’m certain. But note that *every single* system which was ever deployed at Internet scale adopted such a constraint. REST isn’t the “One True Way”, because there is no One True Way. But it’s a heckuva lot better than a style which fails to pass the most basic of litmus tests for integration at a very large scale.

  11. Also, regarding “the REST folks who raised all the fuss at the W3C preferred to contrast REST to the rpc-oriented Web services style”, that’s not true. While I’ve certainly been a vocal critic of that approach, for the past year or more I’ve been pointing out the problems with document-oriented Web services. In particular I’ve pointed out that this style is architecturally far worse off than RPC, since it has all the downside of RPC (too many interfaces), with none of the upside (largely self-descriptive messages). What I’ve been trying to get across was that if you want true loosely coupled document-orientation, you need to adopt an interface constraint, as ProcessMessage or REST.
    Seriously, if it’s document orientation that you want, can you think of a more document-oriented style than REST? *shrug* I suppose old habits – of service-specific interfaces – die hard.

  12. If I understand this correctly, what we seem to be talking about is a naming issue.
    — Anything to do with Web services is bad, since it supports the concept of custom interfaces (i.e. not even any of the WS-* specs are worth keeping)
    — Fixed interfaces are a key part of the proven success of the Web, therefore any integration or interoperability technology seeking to provide a solution on the scale of the Web needs to use them
    — A view of technology in terms of its commercial viability is tantamount to wearing blinders to its real value (so the market studies don’t matter)
    — We only need one tool in the toolbox, since RESTful use of XML over HTTP can solve any and all interoperability requirements (and furthermore is the best approach since it follows the proven Web architecture)
    — But what do I do if I want to integrate my Tuxedo, CORBA, TibCo, MQSeries, JMS, J2EE, CICS, IMS, SAP, etc? These systems all support “enterprise qualities of service” such as reliability, security, and transactions. Do I invent my own XML mappings for each from the fixed interfaces?
    — Aha, but these integration scenarios are not Internet scale, therefore we can use custom interfaces
    — XML might be ok to use with custom interfaces as long as we’re not trying to reinvent the Web (or anything like it)
    — Therefore rather than just throwing out all the specs and starting over, we should just change the name to “Melvin” services (or something, sorry Melvin šŸ˜‰

  13. Shucks! I forgot to say “and these systems all also have different transports that wouldn’t recognize the HTTP interfaces”
    Sorry

  14. Are you _trying_ to parody yourself, Eric?
    It’s pretty clear to me now why the REST folks aren’t interested in the conversation you offer. Can’t say I am much either, after that last snarl. Your blog, though, so adios!

  15. You claim, “Amazon.com said last year that 20% of their developers use SOAP. This is exactly the kind of information you won’t hear from the REST folks since they don’t like to acknowledge the possibility that other valid solutions exist.” I don’t know what other RESTafarians say about this, but if you ever listened to me talk about REST, it’s one of the first things you hear. I love saying that four out of five developers prefer REST when given a choice. šŸ™‚

  16. Simon,
    You are correct in at least on aspect of your most recent comment, which is that I don’t expect true REST believers to listen to me. This debate has been going on for more than two years (at least) and while I am willing to concede many points to the REST advocates, they consistently appear unwilling to concede any points to the Web services advocates.
    I take it from your reply that you count yourself among those who don’t wish to debate the relevant points since your reply is on a personal level rather than to any of the points I’m trying to bring up about the importance of custom interfaces to enterprise integration or the commercial value of Web services.
    I also really dislike absolutists of any stripe since I try to view the world in practical terms. To go back to the Amazon.com example one more time, I do agree with Eliotte that it’s good to point out that people tend to prefer the XML over HTTP services to the SOAP services. If you had read my blog you would also notice that I agree with Jeff Webb’s blog entry about the relative simplicity of the XML interface for VB programmers, not only for Web developers.
    But the fact that people are also using the SOAP interfaces is important. The fact that Web services are being used to solve significant enterprise integration problems is also important. The simple fact is that the importance of SOAP and Web services tend to get overlooked in the REST debates since the REST folks seem to feel that they need to convince the world it’s going down the wrong track.
    Things just are not, and never will be, that simple.

  17. I completely appreciate that my position must come across as “REST is all you need”, but it simply isn’t, and I’ve said so on numerous occasions. Twice now I’ve had to use a non-uniform (though still very general) semantic in order to do efficient bulk state updates. In my experience at developing Internet scale, enterprise-class, document-centric, integration solutions, I’d say that REST is very suitable for about 80% of what one might need, and near-REST is suitable for the other 20% (keeping in mind this is for document-centric solutions). SOA/WS is not near-REST, despite being – sort of – document-centric.
    People use SOAP in the SOA style for many reasons. One may very well be that they prefer it to using HTTP/XML or SOAP in a RESTful manner. But you need to also consider that they might be using it because it’s “the cool thing” that the ISVs are hyping, and/or because they aren’t familiar with how to solve their problems using the Web or REST. Only time will tell what was really happening, but in the meantime you can’t claim that “adoption == success”, because it doesn’t, only deployment does. Here’s one measure of deployment; http://www.markbaker.ca/2002/04/WebServicesGrowth/

  18. Okay, hot topic, so I’m going to put my foot in it, ask some stupid questions, and hopefully learn something.
    Some background: I develop healthcare and research applications, both GUI client/server apps and browser-based apps. I’ve never built an application that sends XML as its data format — not counting XHTML. šŸ˜‰ I have read Fielding’s REST thesis, and I understand it (on a theoretical level). OTOH, I am only vaguely aware of the WS stack, but have no real knowledge of it.
    So, my personal context is established – I’m vastly ignorant. šŸ™‚ My stupid questions, then, are: what are the use cases of the WS stack? What problem spaces is it designed to address, where does it overlap with REST/XML/HTTP, and why is it better, and for which areas of overlap? I know that REST/XML/HTTP is relatively simple, while WS is complex and growing. Why is the complexity necessary? What problems does it solve?
    By asking these questions, I want to cut through the tools vendors’ marketing hype, and get a reasonable understanding of why I should consider using WS-* rather than REST/XML/HTTP. Short of reading the specifications themselves (which I will of course do if I decide to use WS in a project — and will have to do eventually anyway), how can I tell when my project needs to use WS, rather than an alternative?

  19. Oh, just to be clear, I’ve gone ahead and read the quick bits you linked to in this article, and also your last article and the one about doc/lit-style. Doc/lit seems reasonably simple, but since I’m not yet familiar with the alternative(s) in WS, I can’t yet evaluate it. And I’ll read the Microsoft intro to WS on the commute home tonight.

  20. Pete –
    Ok, this is actually an excellent challenge, and I think it presents a great opportunity for us all to debate and test out some of these points in a real scenario.
    What I’d like to do is start a new blog entry for this and get the discussion rolling by posting my response.
    I am traveling today, so I won’t be able to get to it for a couple of days, but I wanted to reply immediately to say yes, let’s see what we can do to help out here.
    Eric

  21. Hi Pete!,
    If you work in the healthcare you’re probably familar with HIPPA regulations (For those of you with no prior work experience in this sector, ‘Portability’ and ‘Accountability’ factor into the acronym). I’m sure most readers of this blog have a firm grasp of the first term.
    There has been much discussion about the ‘complexity’ of the WS-* specifications. Are the WS-* specs complex? … compared to what? … VB, Java, Obstruction Theory? This degenerates into pointless, acrimonious grand-standing knotted in self-serving circular references.
    If you are involved in the transfer of commerial (as opposed to retail, the web metric dejour) funds, patient information, legal contracts or correspondence and you do not have a firm grasp of ‘confidentiality’, ‘integrity’, ‘authenticity’ and ‘reliability’ you may become uncomfortably familiar with litigation.
    The WS-* stack, at least as I understand it, is composited … don’t need a feature … don’t bother. If REST can provide comparable functionality with greater ease, please demonstrate.
    Specs from the W3C, OASIS et al. are targeted toward ‘implementers’, You may enjoy these during more masochistic moments, as I have, but they are probably not a good launch point given your ostensible interests.
    As others have suggested, quite some time ago, doc/lit is the approach to research.

  22. Eric,
    I look forward to your next entry.
    Milton,
    Um, it’s “HIPAA”, not “HIPPA” (Health Information Portability and Accountability Act of 1996).
    “Are the WS-* specs complex? … compared to what?” Well, the question isn’t how complex the specs are, the question is whether the usefulness of the technology justifies the complexity. That is, can the same requirements be met using simpler technologies, and how actually necessary are the requirements? I don’t think there’s any argument that the WS stack is complex, but where is it necessary? Everything should be as simple as possible, but no simpler. So in what situations does the REST approach fail, and WS succeeds?
    Hopefully Eric and the other respondents will be able to hash out the proper uses of WS as compared to other architectures. I don’t have the knowledge necessary to do more than pick apart the responses.
    “The WS-* stack… is composited… If REST can provide comparable functionality with greater ease, please demonstrate.” Um, no. šŸ™‚ As I wrote, I haven’t the knowledge to make a reasonable case for either side. More to the point, the overall argument (it seems to me) is not about whether REST can do everything WS can. The argument seems to be that using the WS stack is often not necessary, because REST apps do enough and are simpler to build. So, my question is, what can WS do that REST apps cannot, and when would I need to do that?
    Eric’s next entry will hopefully answer those questions, and the RESTifarians will have an opportunity to explain how REST apps would solve the same issue, if they can. If we’re lucky, we may be able to come to some sort of consensus as to when and where WS is the proper and preferred approach, and when REST apps are “good enough”.
    Or have I missed something somewhere (very likely)?

  23. SOAP vs REST

    Does anyone else think the W3C have lost control of their own standards process? The WS-* specification collection seems to get bigger and bigger. Not surprisingly there are now calls for a WS-Architecture to try define how all these WS-* specificatio…