ESB Architecture & Lifecycle Definition

A Brief Introduction

Progress Software developed a comprehensive definition of the Sonic ESB, including its development and deployment lifecycle at an architectural level, using industry-standard modeling techniques. The following is an overview of the ESB definition, which is fully defined in the whitepaper The Sonic ESB: An Architecture and Lifecycle Definition.

Service-oriented architecture (SOA) is fast becoming the systems and software approach of choice for the majority of medium and large enterprises.1 According to the principles of SOA, IT professionals design, implement and deploy information systems from components that implement discrete business functions. These components, called "services", can be distributed across geographic and organizational boundaries, can be independently scaled and can be reconfigured into new business processes as needed. This flexibility provides a range of benefits for both the IT and business organizations.

Core to the IT infrastructure supporting SOA is the enterprise service bus (ESB) which connects and mediates all communications and interactions between services. Given the importance of the enterprise service bus (ESB) to SOA implementation - and its affects on costs, development and deployment processes - it is important to understand the functions and structure of the ESB and how its design permits rapid change in, easy connections to, and clear visibility and control of services and processes in a SOA-based application.

Figure 1 shows an example of SOA used to integrate and mediate a multichannel financial trade process. This represents a combination of existing systems "wrapped" with a services interface and new services created to enhance the business process. This example illustrates how an equity trade order would originate from an Order Management Service, undergo a transformation of data elements to support the requirements of the trading system and the Compliance Service, flow through to trigger a fund transfer with the Funds Transfer Service, and ultimately get logged as a completed transaction in the Logging Service. Rather than an integrated application, in this case the trade process is executed as a sequence of heterogeneous process steps, coordinated by the SOA infrastructure.

Figure 1: Example of SOA Application

In this example the ESB is providing the connectivity with routing between services, the transformation of XML data to allow services with varying interfaces to communicate, and the business process sequencing of services.

Sonic Software introduced the industry's first ESB and has continued to be the market leader in ESB functionality. As such, the Sonic ESB ® (hereafter referred to simply as "ESB") sets a reference mark for the SOA marketplace. The ESB is complemented by additional products in the Sonic SOA Infrastructure family including its Workbench, Orchestration Server, XML Server, Collaboration Server and Database Service.

The approach being taken in this full ESB definition effort is to describe each of the key ESB functional areas through exposition of its function and benefits accompanied by structure diagrams based on an industry-standard approach, UML (Unified Modeling Language). UML Class diagrams provide a definitive architectural definition for the ESB that is useful for learning and discussion. Figure 1 employs an ESB diagramming notation introduced by David Chappell in his book Enterprise Service Bus.

Services, Communication and Mediation

Core to the support of a service-oriented architecture (SOA) is the support of services and their communications. Figure 2 shows a UML Class diagram with an overview of the ESB structures supporting services, communications and mediation.

Figure 2: Overview Architecture for Services, Communication and Mediation

ESB Services (left of Figure 2)

ESB services may be any of a number of types. Typical ESB services which implement mediation are: XML Transformation, Content-Based Routing and User Defined Mediation. An XML Transformation Service provides data conversion from one XML document to another according to an XSLT specification. A Content-Based Routing Service directs messages to various other service endpoints according to configured routing rules. A User-Defined Mediation Service implements a function such as logging or custom validation which complements the core business logic. A User-Defined ESB Service connects to business services (see below) to implement core business logic.

ESB services execute within the ESB Container. The ESB container instantiates ESB services and provides the systems management interface for its services, the management of communication protocols, the capability to have multiple threads executing services for scalability, and the standard capability to execute distributed processes.

Connected Business Services (right of Figure 2)

Connected services implement the business logic for ESB-integrated applications. Types of business services include: Web Services, Legacy Systems, JMS (Java Message Service) Applications and all varieties of Application Server Applications. Each of these types of business services can communicate with each other through the ESB, and with ESB services to perform mediation. In Figure 1 the Order Management Service is a connected service which connects in through an HTTP Connection. The Compliance Service, Trading System and Funds Transfer Service are business services that have been interfaced directly to the ESB as user-defined ESB services.

Communications and ESB Processes (center of Figure 2)

This section of Figure 2 shows the core capability for services (both ESB and business) to communicate and to execute an ESB Process which is a sequence of steps invoking services or other ESB processes. Figure 1 shows an ESB process called Trading Process that has five steps.

An ESB service sends and receives messages through references to ESB endpoints and ESB addresses. For example, an ESB service such as the compliance service in Figure 1 can configure its Exit to refer to the ESB address of another service to receive output messages - in this example that would be the Trading Service. Alternatively, the ESB address can refer to an ESB process.

An ESB process is defined by an ESB Itinerary which contains a sequence of Process Steps. Each process step can invoke one (or multiple) ESB services, other ESB processes, or an external Web service. The ESB itinerary is carried with its associated ESB message providing a completely distributed, and hence reliable, execution environment for the ESB process. There is no central processing engine for the itinerary as each ESB container can process and route the next step in the ESB itinerary. This provides both efficient execution without the need to communicate to a central site and reliability as processing can continue even with some losses of connectivity.

Each ESB service can have its entry and exit endpoints changed without requiring changes to their implementations.

The ESB Endpoint provides the capability for ESB services to send and receive messages with a specified Quality-of-Service, for example "best-effort", "reliable" or "guaranteed, once-and-only-once". The connection tells what supporting messaging channel is used by an ESB endpoint to connect with an ESB service or a business service. This messaging channel can use an ESB native connection, or for external systems can use Web Service Connection, JMS Connection or JCA (J2EE Connector Architecture) Adapter.

The ESB Connection employs user configurable asynchronous and synchronous delivery semantics including "publish-and-subscribe", "send-and forget" and "request-reply". In the example in Figure 1 the ESB connection between the Order Management Service and the Trading Process is Web Services request-reply. If a business or ESB service needs to change the manner in which it connects, this can be affected without requiring changes to its implementation. When services require differing types of connections, the ESB provides the mediation between the differing protocols.

The ESB Message class and the sends and receives associations between ESB endpoint and ESB message create the basic structure for one ESB service to send a message to another ESB service through an ESB service's Dispatcher. The dispatcher manages the sending and receiving of ESB messages between ESB services and external endpoints, in effect implementing the arrows between process steps shown in Figure 1. This dispatcher also manages ESB processes.

Industry-standard WSDL (Web Services Definition Language) Definitions can be created for ESB services and ESB processes. This allows the use of standard tools and integration with other SOA infrastructure components such as a UDDI Registry. For example, in Figure 1 the entire Trading Process ESB process is exposed as a Web service.

Key benefits of this approach to implementing ESB services, mediation and connection to business services include:

• Business agility through rich support for mediation services including XML transformation and built-in support for distributed ESB processes.
  
  
• Broad reach across many kinds of business services and communications protocols; integration of heterogeneous legacy and new Web service elements.
  
  
• Distributed process execution, performance and reliability even in highly-distributed deployments.

ESB Development, Deployment and Production Management

Intrinsic to the ESB is rich support for the entire services life-cycle from development to deployment to management. Figure 3 shows a UML Class diagram with an overview of some of the key structures for this support. The figure is divided to reflect the migration of ESB services from development systems to staging/test systems to a distributed production environment. A critical requirement for the Sonic SOA Suite is to allow such a migration to occur in the easiest error-free manner possible.

Figure 3: Overview Architecture for ESB Development, Deployment and Production Management

Development Systems (left of Figure 3)

The top of this diagram section shows Development Tools while the bottom, labeled "Development ESB" defines the core components of the ESB infrastructure which are used in each stage of the ESB Lifecycle from development to staging/test to production environments.

The Workbench provides each developer with a set of tools for ESB development and deployment including creation of custom ESB Service Type implementations. For example, in Figure 1, the Logging Service would be a custom ESB service type. With a set of Resource Editors, in concert with third party and industry standard development environments, developers create Resource Files which are stored in the ESB Repository (a configuration repository). The Sonic Workbench provides for the creation of Development Projects containing all of the configuration files for a specific service development project in support of a composite or ESB application. In the trading example this would include files describing the ESB process and it's WSDL file, used to expose the process as a Web service. It would also include the Logging Service implementation, among other resource files.

Editors include: the ESB Process Editor for visually creating and editing business process definitions tying together multiple ESB services, the XLST Editor for creating the XLST Stylesheet defining an XML transformation and the Routing Rules Editor to create the configuration file used by content-based routing services.

The Management Console provides the management framework for configuring the Sonic SOA infrastructure for ESB services, ESB containers and the supporting SonicMQ multi-protocol communication server elements. The Management Console allows for the configuration of new instances of ESB services and ESB containers to allow an easy "configure-and-go" approach to managing the SOA infrastructure. Using this console, for example, the ESB Connection between all services in the Figure 1 trade process would be set to "once-and-only-once" to prevent the chance of any message losses.

Staging/Test Systems and Deployment Tools (center of Figure 3)

Moving configurations from Development to Staging/Test to Production environments is done with the Deployment Tool. Selected resource files and configurations are moved through the creation of intermediate archive files and the use of the Deployment Map File to map service and messaging parameters to operate properly in the targeted environment where quality of service settings, security policies and ESB addresses may differ. The diagram shows the promotion and deployment of configurations from the development directory service to the staging/test directory service.

Production Systems (right of Figure 3)

The deployment tools are also used to promote and deploy ESB configurations from the staging/test directory service to the Production ESB Repository.

The management console is used to place ESB containers and their configured services onto multiple distributed systems where interaction policies, transport bindings and communication layer security policies are established.

The ESB provides highly reliable support for distributed systems. As shown in the diagram section labeled "ESB Distributed", an ESB Repository Cache is used to provide local copies of the configuration files used by ESB containers and ESB services executing on distributed systems. This provides the ability for distributed systems to be both high-performance and reliable, and allows continued independent operation even when the production directory service is unavailable or unreachable. This same execution environment is available, but not illustrated, in both the staging/test and development environments.

Having multiple distributed systems in an ESB provides scaling of ESB service capacity across machines and locations. Each distributed system shares the same configuration for reporting faults and sending status back to the management console. In the financial trade process example, it would be possible to replicate the transform service on multiple execution machines such that if it were needed between each step in the process (a likely situation) that there would be multiple instances running across a network of machines that could be recruited to address peak.

The administrator uses ESB services and ESB processes to automate the handling of these errors and notification events in a service-oriented fashion, leveraging the same ESB infrastructure. For example, if errors occurred in the Compliance Service step of the trade process depicted in Figure 1, the normal fault endpoint of that service could be configured to route the error back to the requestor using a reply-to semantic or a separate exception process could be defined to receive the fault on it's entry endpoint.

Key benefits of this approach to implementing ESB development, deployment and management include:

• Rich set of development and management tools allow for easy "configure-and-go" deployment.
  
  
• Deployment tools provide support for the ESB lifecycle and allow for the automated mapping of configurations and movement of implementation resources from development to staging/test to production environments.
  
  
• Distributed execution environment including directory caching allows for high-performance, scalable and continuously-available SOA systems.

For a more thorough definition of the Sonic ESB, including a complete UML class diagram, download the whitepaper The Sonic ESB: An Architecture and Lifecycle Definition.

1Heffner, Randy, "Delivering Real World SOA", Forrester Research, 2005.

<출처: http://www.sonicsoftware.com/products/sonic_esb/architecture_definition/index.ssp>

[1편] SOA와 Service의 정의
[2편] BPM을 통한 서비스 분류하기
[3편] ESB를 이용하여 Service Enabling 하기
[4편] SOA 잘 돌리기(계획)

[3편] ESB를 이용하여 Service Enabling 하기

우리는 "[2편] BPM을 통한 서비스 분류하기"를 통해서 어떻게 서비스를 분류해 낼지에 대해 이야기를 했다. 실질적인 업무의 예를 들지 못한것이 아쉬움으로 남는다. - 추후 적절한 예시가 있으면 업데이트 하겠다.

3편에서는 둥실 둥실 떠있는 서비스 들을 어떻게  Enable 시킬 것인가?

예를 들면, 대출 신규 신청이란 어플리케이션이 있다고 가정하자.

이 업무를 수행하기 위해서
1. 고객정보 조회
2. 대출 신규신청
3. 고객 담보대출 신용도 조회
4. 대출 신규신청 취소
5. 승인 / 반려

위의 5가지 서비스가 도출 되었다고 가정하자.
기존의 방식으로 J2EE Application으로 구현한 경우 아래의 그림과 같이 될것이다.

기존방식으로 구현

SOA가 추구하는4가지 목표(from oracle)

ESB를 사용한 구현

위의 그림은 ESB를 활용하여 <신규대출 신청>이란 업무를 재구성한 것이다.

이와 같이 J2EE App에서 ESB를 통해 서비스에 대한 결과를 얻을수 있고, 내부적인 비즈니스 서비스에 대한 연결은 ESB를 통하여 구현하면, <신규대출신청> 이란 어플리케이션과 각 서비스간의 Loosely Coupled한 구조로 연결될 수 있고, 또한 서비스의 재사용이 가능하며, <신규대출신청>이란 업무에 새로운 서비스가 추가 또는 삭제될 때에도 자유로이 대응할 수 있다.
자세히 표현되지는 않았지만, 기본적으로 ESB에서 서비스에 연결될때는 표준프로토콜을 사용하는 것이 일반적이다.( 특정 프로토콜이 사용되는 경우도 있다.)

ESB를 활용하여 구성된 서비스를 enabling 시킴으로서 SOA의 목표에 부합될 수 있는 환경을 구축할 수 있다.