Time to Stop the SOA Hype?

Granted I am basing this generalization on a single data point, something that an IT manager I happened to be sitting next to on the plane told me, but I think the initial wave of SOA evangelism is done and we can (as an industry I mean) stop hyping it now. I think people know what it is now, and are ready to decide whether or not it’s something they want to do.

Definition of Software Architecture

Last week in San Francisco I had dinner with my good ol’ Microsoft MVP solutions architecture buddy Beth Massi and her significant other, Alan Griver. We had a great time weaving our way through the crowds at the St. Patrick’s Day block party on Front Street but we could not get close enough to the bar to get a beer and ended up at the nearby Gordon Biersch’s for dinner.
During the conversation over dinner Beth gave about the best definition of software architecture I’ve heard – I’m not sure I have the wording exactly right, but it was basically that software architecture is all about figuring out how to put together things that were never designed to work together in the first place.
Meaning things like J2EE and .NET were not designed to work together but since they are often used in the same companies, someone has to figure it out.

Boeing Factory Tour

One of the top tourist attractions in the Seattle area is the Boeing factory tour.
IONA sells a lot of software to Boeing, and yesterday after introducing our new CEO, Peter Zotto, and providing a technology update on our new products and open source initiatives, Peter, Tony Frey, Dan Stein, Al Davies, and I were treated to a factory tour by one of the Boeing executives. I had heard about other IONA employees getting to go on this tour, and I had been jealous ever since. But not anymore 😉
Tens of thousands of Boeing employees use IONA software every day in the process of manufacturing commercial aircraft, and even though we were well aware of that fact it was still a real experience to see airplanes actually being assembled, to get a close look at the massive 777 engines, stand inside an empty 747 freighter, and sit in a 747 cockpit during power on testing.
IONA software is used to power the DCAC/MRM application. The project began in 1996 and started going into production in 1997 through 1999. According to the stats on the company’s website, as of July 2001 39,250 Boeing employees were using the system, and its purpose is to streamline the commercial aircraft manufacturing process.
With the downturn in business following 9/11 the numbers of manufacturing workers are down from their pre-9/11 peak, but the application is still used in the manufacturing process for every commercial aircraft currently in production.
Like our Western Region sales director Tony Frey said, it makes you proud to be American. It’s not just the unbelievable size, scale, and complexity of the job, but also the spirit, enthusiasm, and pride of the people who work there. There is a real “can do” attitude, which is apparent in the enthusiasm everyone has for the new 747-8 model and of course for the soon-to-be-in-production 787.
During the visit and the tour there was quite a bit of talk about the 787. They are already refitting the factory for it. It’s due to start the production cycle later this year and begin filling orders in 2007. This is the answer to the Airbus 380, or perhaps the challenge, since Boeing is taking an opposite view of what the airline industry, and by implication the traveling public, really wants. The choice is between a mega liner capable of carrying 555 people on two full decks and a smaller, efficient, and relatively inexpensive liner capable of taking 210-330 people point to point. The fact that the Airbus 380 requires some new airport infrastructure seems it may present a challenge to overcome (although the Airbus website says this will not be a major problem).
In any case this will be a very interesting drama to unfold over the next five to 10 years. The Airbus is already in its first flight, but Boeing is countering with the 747-8, which expands the upper deck the length of the plane (although it will, like the 787, not be in full service till 2008). According to the Boeing website it will carry 450 passengers in a typical three-class configuration.
The factory in Everett where they turn out the “twin aisle” airplanes is the largest enclosed space in the world: 86 acres. Looking out across the roof it had the appearance of a cityscape. Mount Rainer was visible in the distance past one end, and Mt Hood across the other. Employees came up to jog during lunch or after work. When it was cold they’d jog or walk in the tunnels in the basement, where they had at one time set up the charts and plans for building the first 747, since the enclosing part of the building hadn’t been finished yet when the first 747s rolled off the line.
There were a lot of signs warning against the dangers of FOD – foreign object debris. Wrenches left inside wing assemblies, rags inside engines, debris on the runway that could get sucked into the powerful turbines and wreck a multi million dollar engine.
The science involved, the attention to detail, the engineering – really impressive. The GEMS machines that literally stitch together entire wing assemblies, automatically drilling, deburring, riveting, and polishing thousands of holes. The 777 wing laid out on the floor 170 feet long with supports measuring the precise halfinch of distance and height. The attention to sealing up the fuel tanks. The way in which parts are delivered using trains, special trucks, and container ships. The history of the company – the daring and risk taking and attention to every detail. It is a privilege and honor to work with these guys, and I hope the relationship continues for a long, long time.

SOA Approach to IT

Another great thing about attending the InfoWorld SOA Executive Forum was getting a chance to meet people, such as Phil Windley, whose blog I really like (check out his excellent summaries of the Forum keynotes), and Todd Biske, who also has a very interesting blog. It was great to put a face to a name, as they say.
Todd and I and David Chappell also spent a few minutes talking about our recent interactions on the Service Orientated Architecture discussion group. In general this has been a good experience, although as with any public forum you get the occasional strange or off topic post. But overall I’d recommend checking out the archives and joining if you haven’t already.
One thing that continues to strike me from the discussions is how many people still seem not to understand services, SOA, and Web services very well. Vendor suspicion runs very high. I am often left wondering whether the industry has really done the job we set out to do here.
It therefore bears repeating that SOA is an approach, not a technology. The customer side presenters at the SOA Forum emphasized this. Some of them also said that they only use 20-40% of the software capability they own. Another consistent theme is that current IT environments are too brittle and hard to change to support business needs.
Everyone seems to agree that SOA has the potential for resolving these problems, if applied correctly. But perhaps the fact that customers need to adopt a new approach to solve these problems means that vendors also need to adopt a new approach.
I think it was someone on the SOA discussion list who said that IT vendors today are like going to an automobile dealer and having the dealer tell you what kind of car you want to buy. In the past it has been much more about the technology than the approach, meaning the vendors who developed the technology had to be the experts on how to apply it.
Now with customers deciding on the approach independently of technology, the roles are reversing. Customers are starting to tell vendors what kind of technology they need by creating their SOA based designs independently of their technology decisions.
This strikes me as a very good trend, and appropriate for where we are in the evolution of the software industry, helping to lead us to a place of resolution for the current frustrations with enterprise software.

InfoWorld Executive SOA Forum March 2006

The InfoWorld SOA Executive Forum remains an interesting and apparently popular event about a year after it first started. This is the third one, and I’ve had the pleasure to attend all three as a participant on one of Jon Udell’s various panels (here’s his description of the ones from yesterday).
Did I ever mention what a great guy Jon is? (Did I also mention that he reads my blog? 😉
Seriously, it has always been a pleasure to participate in Jon’s panels. He always comes up with good, challenging questions and I always learn something from the other panelists. I thought this one went very well because we had a good conversation on a range of topics, but someone came up to me afterward and said there wasn’t enough controversy for his taste…I think he was joking.
Anyway this time we were talking about the various development paradigms available for SOA, including RPC, asynchronous messaging, and document oriented (which I actually tend to equate with asynchronous messaging but you can pass documents using RPC, or maybe more correctly request/response since this is how HTTP works for example).
In the end, the enterprise probably needs some combination of all of the above. But how to choose? Go by whatever the developers are most familiar and comfortable with? It is true that subcultures in distributed computing tend to spring up around these different approaches since they each have their tricks and best practices and lessons (to be) learned.
But I do not think so. I think it’s more important to choose the best tool for the job since some application requirements are better served by one or the other.
Here’s my rule of thumb, which I explained in a bit more detail during the panel yesterday. I think it depends on the significance of the reply to the application and especially to the user.
For example, if you are authorizing a credit card, and you get a successful reply to your message, you expect to be able to use that credit card on the next message to buy something. The significance of a synchronous, RPC-style reply is that the database has been updated.
On the other hand, you may need to know is that the message was reliably captured for later processing. For example, you’ve placed an order with Amazon.com and the reply says that Amazon has received your order and will process it later.
One other interesting point of discussion was who should we make things easy for? The developer of the service or the consumer of the service? If we make the programmer creating a service do a bit more work to produce a more useable and useful service, isn’t that time and energy well spent? As opposed to just assuming that all services can and should be easily and automatically generated by tools, which is kind of where we seem to be now with Web services tools.
Overall the event is really good because it focuses on lessons learned from customers working on SOAs, and also because Jon and his colleagues manage to focus on technology and business topics of general interest.
Yesterday’s keynote was by Jeff Gleason of Transamerica, who told us how they had started an SOA pilot but cancelled it because it was at the project level. Their problem was that they were already suffering from project based architecture.
He also described how their core policy rating application had dozens of other applications integrated with it, making changing the core business application too difficult because of the impact of change on all the other applications depending on it. SOA as an approach helped resolve this issue.
He also described the way they inventoried existing systems to see what could be reused, identifying deltas representing new development, and how they categorized business events and the functions and services associated with them.
The InfoWorld folks have promised to post the presentations so I will post a link when they do.

XML for RPG

I was surprised and amazed to see that RPG supports XML now, but perhaps I shouldn’t be.
At this point I will have to admit having used RPG more than 20 years ago but I believe it was RPG II (the article in the lastest issue of IBM Systems is about a new release of RPG IV). It was an interesting time because at the same time I was submitting RPG II and Mark IV (similar to RPG) programs to a remote mainframe to produce batch reports I was also programming a Sycor micro using TAL II (its assembly language) to capture and feed the data to the mainframe from cassette tapes… And no, the cassettes did not make what you would call musical noises when you put them in your home or car cassette player 😉
In those days we were not thinking about a language that would run across different hardware systems, and a hardware-neutral operating system was similarly not something that seemed to be in the cards. I learned RPG because that was how you got reports out of the mainframe, and I learned TAL II because that was how you programmed a Sycor to send data to the mainframe.
The news about RPG and XML seems like a significant milestone in the evolution of software. It is an indication of the growing adoption of XML and provides further evidence that existing programs don’t get replaced as much as they get extended, enhanced, or perhaps improved (if that isn’t too much of a leap in the characterization of what’s going on here).

Distributed, Not Stacked

For a long time the software industry focused on building up stacks of technology designed for developing new applications.
But that’s not what customers need anymore. For the most part, they already have enough features, functions, and capabilities. In some cases probably even more than enough.
What customers need is a better approach. One that allows them to distribute a new kind of software to improve what they already have and make it more useful.
We are at a very interesting point in the software industry maturity cycle. People don’t need more IT, they need better IT.
In other words, when software started out we used to think broadly in terms of what new features and functions were needed for new applications to be developed, what previously manual part of the business needed to be automated, what computers couldn’t yet do but should. This adoption cycle drove a particular type of innovation. Everyone was busy inventing new features and functions so that more and more business transactions could be automated.
But we don’t need to do that anymore. We have pretty much already automated all business transactions. What we need is to find a better way to do it, to share data across applications, to reuse existing investments, to eliminate duplication and cost, and to establish a more predictable way to modify and change the applications.
In short, the industry needs a different kind of software. Not the stack of features and functions designed for new application development, but a kernel of distributed software capable of modernizing and improving existing software — providing just the right amount of added value (and at the right price of course) to solve today’s problems.
Yesterday’s software just won’t do it – certainly not at any reasonable price.

The ESB Question

I think the issues with ESBs that Joe McKendrick highlights in some of his recent posts have to do with the way in which some vendors have implemented the concept rather than with the concept itself.
Enterprises need infrastructure for SOA deployment. In the debate about ESBs everyone can at least agree on the fact that the ESB is designed to meet requirements for SOA infrastructure.
However as Joe and others correctly point out, most vendors interpret this requirement as an excuse to reuse, repurpose, or reface their existing message bus technology with Web services and a bunch of fancy add ons.
The dilemma for us was whether or not to align ourselves with the ESB product category since we have a completely different approach, one that meets SOA requirements far better. Unfortunately, there isn’t another product category that’s a better fit for us. So I guess we just have to try to gain consensus that ours is the right approach ;-).
In the debate the need for a distributed infrastructure is recognized, like the one used in the Web. That’s exactly what we have done. We live at the application endpoints, service enabling them (you don’t need a Web server or an app server) and connecting the services however you want – peer to peer, point to point, or even hub and spoke (although we would recommend against this pattern for SOA).
Our architecture is definitely the right way to go, though. It’s microkernel based with plug ins, endpoint oriented, multi-communications protocol, multi-data format, with high performance, language neutrality, and enterprise quality of service – all based on WSDL and WS-*. This is what our customers told us they needed and wanted – and among our customers are some of the largest deployed SOAs in the world.
I believe this is also the reason Anne Thomas-Manes doesn’t like to include IONA in the ESB category. I hate it when we get lumped in with the other guys that just don’t get it, and we end up getting criticized along with the JMS centric crowd. An ESB is not JMS + Web Services +++ Not if it’s really going to meet customer requirements for SOA infrastructure.