Enterprise applications Toolkit

All aboard! The enterprise service bus pulls in

Tags:
Web Services,
API,
ESB,
Enterprise Service Bus

David Berlind ZDNet.com

Published: 29 Jul 2004 11:45 BST

But with nothing but message queues (and the systems they abstract), the potential for mayhem and complexity still exists. At the very least, directly connecting message queues to one another still represents a form of hardwiring that could benefit from some additional abstraction. For example, in order to connect message queues directly to other systems or message queues means that the queue might have to be reprogrammed every time systems get moved, changed, or replaced. If there are only two nodes in your integration matrix, as there might have been in the days where integration was so difficult that you wouldn't fathom manually wiring more than a couple of systems, this is no big deal.

As integration became more commonplace and more nodes were thrown into the mix, however, it became clear that the message queue didn't need the additional responsibility of connectivity. It's one thing to integrate, and it's another to make sure the entire system scales well. Distracting the host systems or the message queues with tasks that can be taken on by something else runs the risk of gumming up the works.

When a single transaction in your point-of-sale system results in interactions with your inventory system (is the product in stock?), your credit card processing service (is the card valid?), your ledger, your CRM system (do we have updated customer information?), and your shipper (multiple drop ship?), the last thing you want your message queue to have to worry about is where all those things are. Especially since, for whatever reasons, you might have other point-of-sale systems for other lines of business interacting with the same services. Rather than maintain all that connectivity data in the queues, one better way to do it is to maintain the routing information in a central location that can serve as yet another layer of abstraction.

Peter Linkin, senior director of product marketing for BEA's WebLogic, put it to me this way: "What was needed was a system that just told us what the message was, what the contents were, where it should go, and what the quality of service for it should be. Then, that was handed off to the next level which is something like a central post office. The postmaster says 'send me all your messages from all these outpoints, and I will intermediate. I'll make sure they get sent individually and reliably to all the end points.' It's a message broker that's driven by business process. The end points don't have to know about each other so there's ignorance at each end and the logic of the business process is in the plumbing."

If two point-of-sales systems need to reach your CRM system as a part of their business process, the message gets packaged and ready for sending to the CRM system, but no particular destination is specified. When that message pops off the top of the queue and reaches a broker that's like the equivalent of a central post office, the postmaster says, "Oh, I know where the CRM system is" and then ensures that the message is reliably delivered according to the prescribed quality of service.

The beauty of this broker-like design is that it operates very much like the Internet's DNS. If for example, you decide to switch your CRM system from Siebel to Salesforce.com, you don't have to reprogram all your queues. You just reprogram the central directory to say "the CRM system is over there."

Although the overall architecture has evolved to have more primary parts (the transactional system, the message queue, the central post office), the distribution of responsibility results in less complexity, as well as less cost from a maintenance perspective. It also allows components of the workflow logic to be moved to where they can be most efficient. For example if some customer data is sent to your CRM system and other customer data is sent to another one, the brokering logic that makes that decision can be put in central router rather than in the message queues.

In transactional systems, where a high number of systems that have to be touched after one infinitesimal change, not only are the number of touch points reduced (which drives down the maintenance costs), the system can perform better since each of the discreet parts gets to hone in on their core competencies without the distractions of problems that other parts are better suited to work on.

Another benefit of this infrastructure framework is that it introduces a level of fault tolerance into the system. If the primary CRM system has failed or isn't responding, the central post office will not only detect the fault, but know where to re-route the traffic. In a very small way, this reliability aspect is one reason that services-oriented architectures (SOAs) are being associated with concepts like utility and on-demand computing.