Tuesday, October 14, 2008

Mashups and SOA

The recent buzz in the IT industry apart from SOA is "Mashups". While SOA mainly focus on Web Services, Mashups deal with bringing data from various systems together.

Wikipedia defines Mashup as a web application that combines data from more than one source into a single integrated tool. RSS, Web Pages, Databases, Web Services, and even custom code can be sources for Mashups. Data can be made available using different protocols like Web Services (SOAP), REST, or RSS.

Based on the type of activities that are done with the mashups, mashups can be categorized as data mashups, consumer mashups, and business mashups, etc. While there is no clear separation among the mashups the underlying concepts or technologies are the same.

We can classify mahsups as next generation Portals as there are many similarities in the functionality and concepts. At present there are no specific standards (just for the mashups) like portals which have JSR 168, JSR 286 etc. but follow some of the XML, Web Services and REST standards.

Some of the aspects or focuses of SOA are governance and service reuse or service composition. With some of the commercial Mashup tools both of these things can be easily done and infact service composition is the goal of mashups. Lately Data Services are also focus of rapid development and SOA and these data services are focus of Mashups also.

Some of the known mashup tools in the market/industry are WSO2 Mashup Server (open source), JackBe Presto, Serena Business Mashups, Nexaweb Enterprise Web Suite, RSSBus, Apatar (open source), Kapow's Web Harvesting Tools, Backbase, etc. Most of these tools focus on bringing RIA usage to the users apart from mashups.

If properly used in SOA for the service composition and to feed the data for portals or to serve as portals these can be potential candidates for good ROI.

Tuesday, September 30, 2008

Mock Web Services with WSO2-ESB

Mock Web Services are not actual web services but behave like real web services. In general these services have interfaces defined but the implementation (business logic and coding) is not yet done. Mock Web Services (will use Mock Services for simplicity) are helpful for the service consumers or for the developers who are developing services based on some other web services. If the depending services are not yet available this will delay their development. With the help of Mock Services one can start coding/developing or consuming the services. Also when the actual services can not be accessed for any reason then mock services are helpful.

For the demo of Mock Services, I am considering Amazon Web Services particularly AlexaWebSearch service listed at http://wsearch.amazonaws.com/doc/2007-03-15/AlexaWebSearch.wsdl. Let us say that, Amazon has published the interface at that given url but the service is not yet ready. It is known how the request and response look like. Now with the help of Mock Services, the users that are interested in can start developing their application and can simulate some test cases.

Using Mock Services encourage some of best practices (1) Contract First Design (2) Decouples the dependency of deliverables (Initially).

In this post, I will explain how to develop Mock Services using WSO2-ESB.

(1) Download and install latest WSO2-ESB (1.7.1 today) from here.
We will download binaries, after download extract the zip file to c:\. It will create the following structure.















From now on wards C:\wso2esb-1.7.1 is referred as %ESB_HOME%
The default ports & passwords used by ESB are mentioned below:

RMIRegistryPort: 1099 RMI Registry port used for JMX
HTTPPort: 8280 HTTP port used by the underlying
Axis2/Synapse
HTTPSPort: 8243 HTTPS port used by the
underlying Axis2/Synapse
DerbyPort: 1528 Internal Derby Database port

userid/password: admin/admin
esb/esb

You can change any of the above ports or userid/passwords by changing the %ESB_HOME%
\webapp\WEB-INF\classes\conf\server.xml

ESB Management Console port (default) is 9444 which can be changed by changing %ESB_HOME%\tomcat\conf\tomcat.properties file.


(2) You can start the ESB by running the windows batch file %ESB_HOME%\bin\wso2-esb.bat.
Access the managment console by opening the page at https://localhost:9444/esb















After successful logging, it will take us to the following screen where the left side has links for proxies, sequences, endpoints, tasks, registry, configuration, statistics..etc most of them are self explanatory (at least for the folks in SOA or trying to be in SOA)
















Adding Mock Service is a three step procedure. (a) Determine the response for the mock service operations and prepare xslt (b) Define the sequence (c) Create Proxy Service and apply the xslt and sequences. We can combine these steps into one step if we start with creating proxy. In this example we will create mock service in three steps.

(3) Access the WSDL from http://wsearch.amazonaws.com/doc/2007-03-15/AlexaWebSearch.wsdl and save the contents to a file(ex: AlexaWebSearch.wsdl). Sample services request and responses are provided here http://docs.amazonwebservices.com/AlexaWebSearch/2007-03-15/SOAP_SoapAuthenticationArticle.html. Save the request and responses to AlexaWebSearch-Request.xml and AlexaWebSearch-Response.xml files, we are going to use this sample response for our Mock Service.

We have the following soap response in the AlexaWebSearch-Response.xml

<?xml version='1.0' encoding='utf-8'?>
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
<soapenv:Header/>
<soapenv:Body>
<ns1:SearchResponse xmlns:ns1="http://wsearch.amazonaws.com/doc/2007-03-15/">
<ns1:SearchResult>
<ns1:SearchTerms>java</ns1:SearchTerms>
<ns1:EstimatedNumberOfDocuments>87568000</ns1:EstimatedNumberOfDocuments>
<ns1:Documents>
<ns1:Document>
<ns1:Url>http://java.com/en/download/index.jsp</ns1:Url>
<ns1:Title>Download Free Java Software</ns1:Title>
<ns1:Bytes>11360</ns1:Bytes>
<ns1:Checksum>AAAAAAABAkY=</ns1:Checksum>
</ns1:Document>
</ns1:Documents>
</ns1:SearchResult>
<ns1:ResponseMetaData>
<ns1:RequestId>09b4accb-ff51-4145-9988-25d38dfcb705</ns1:RequestId>
</ns1:ResponseMetaData>
</ns1:SearchResponse>
</soapenv:Body>
</soapenv:Envelope>

Modify it as followed to make it xsl transformation which we will use for sending the custom response from the mock service.

<xsl:stylesheet version="2.0"
xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
<xsl:output method="xml"/>
<xsl:template match="/">

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
<soapenv:Header/>
<soapenv:Body>
<ns1:SearchResponse xmlns:ns1="http://wsearch.amazonaws.com/doc/2007-03-15/">
<ns1:SearchResult>
<ns1:SearchTerms>java</ns1:SearchTerms>
<ns1:EstimatedNumberOfDocuments>87568000</ns1:EstimatedNumberOfDocuments>
<ns1:Documents>
<ns1:Document>
<ns1:Url>http://java.com/en/download/index.jsp</ns1:Url>
<ns1:Title>Download Free Java Software</ns1:Title>
<ns1:Bytes>11360</ns1:Bytes>
<ns1:Checksum>AAAAAAABAkY=</ns1:Checksum>
</ns1:Document>
</ns1:Documents>
</ns1:SearchResult>
<ns1:ResponseMetaData>
<ns1:RequestId>09b4accb-ff51-4145-9988-25d38dfcb705</ns1:RequestId>
</ns1:ResponseMetaData>
</ns1:SearchResponse>
</soapenv:Body>
</soapenv:Envelope>
</xsl:template>
</xsl:stylesheet>

Add the contents of the above xslt file to integrated registry by clikcing "Integrated Registry" followed by "XSLT", then provide a name (SnowBoard) and click create to save the xslt in the registry as shown below






Follow similar steps for creating WSDL called AlexaWebSearch inside registry.


(4) Now it is time to define what to do with the Mock Service request. Once a request to ESB comes, we need to take the request and ignore where it should be routed and then send the previously built custom response back.

To add a sequence to the registry click "Sequences" and then "Add" button in the management console.

After that click "Add Mediator" and select "Filters"-->"In" and again add "Filters"-->"Out" as shown below.




After that, click "Add Mediator" inside "In Mediator" and then select "Transform"-->"XSLT"


Click the "Registry Browser" button next to "Key" form value field and then browse "XSLT", pick "AlexaWebSearch" and then click "updated" as shown below:









Now we have defined the outgoing message structure in the sequence. We need to indicate that we are going to send the response and we will strip down the "header" (if any in the request) from the outgoing message and then we are going to send out the message.

To indicate that we are sending response back, add the property named "RESPONSE" and set its value to "true" as shown below





To remove the header from the response follow as shown below.





Finally to send the response back add the "send" sequence as shown below and then click "Save As" and save the sequence.








This whole sequence is represented in the Apache Synapse (which is used in the WSO2-ESB) in the sequences as below, which you can find it in the configuration:

<syn:sequence statistics="enable" name="AlexaWebSearch" trace="enable">
<syn:in>
<syn:xslt key="XSLT/AlexaWebSearch"/>
<syn:property name="RESPONSE" value="true"/>
<syn:header name="To" action="remove"/>
<syn:send/>
</syn:in>
<syn:out/>
</syn:sequence>

(You can avoid creating the whole sequnce manually, isntead you can copy paste the above sequnecne text inside the configuration and then hit "update" button which will create the sequence and after that save the configuration.)

Now we have the mock response and the behavior that is required to act like a mock service. We are ready to define the proxy service (mock service) and attache the above defined behavior and response objects to it.

(5) Create a new proxy from the ESB Management console by clicking "Proxy Services" and after that "Add" as shown below.
















This will lead to the form for defining and creating a proxy service as shown below:
















Enter the proxy service name as "AlexaWebSearch" and then click select the drop down for the "Incoming Messages" and choose "AlexaWebSearch" sequence from the declared sequnces as shown below





After that click advanced and disable unwated protocols and then click "Next" and "Next". Click the Registry Browser and browse the registry and pick the "WSDL" and then click finish.






You can verify the newly created Mock Service (proxy) by cliking "Proxy Services" on the left.

Enable the logs and statistics as shown below for this newly created service.

Congrats...now we have created a mock service successfully using WSO2-ESB. Irrespective of the request message, we will be getting only one response back as per our "xslt". One can configure various responses based on the user needs or based on the incoming requests.

(6) I have used SoapUI for testing this Mock Service and below are the request and responses.

Wednesday, September 10, 2008

SOA Management Systems

The goal of this post is to introduce SOA Management Systems and some of their capabilities.

Service Oriented Architecture (SOA) is now-a-days an important buzz word for most of the moderately big organizations. While most of the organizations are trying to achieve it through "Web Services" there can be other implementations as well. With the increasing popularity of SOA and its inception via Web Services, web service management or SOA Management is a key step for success.


There are many products in the industry which are providing such solutions, Amberpoint, Progress Actional, Tibco ActiveMatrix, Oracle ClearApp, etc. With the help of these tools one can separate the security concerns to an extent and can focus primarily on deliverables (developing code for business logic rather than for security). Some of the other features are logging, security (authorization, authentication, SAML, etc), transaction control/management, service level agreement (SLA) based policies and various custom policies.


Based on the deployment architecture, application/web servers used and hardware/software some of the tools can provide visibility inside the web services by instrumenting the class loaders. While it is totally based on the requirements whether to go for this additional visibility or not, going for this will create lot of information which may cause adverse performance issues.


Most of the products enforce policies based on Proxy mechanism. While proxies gives power also comes with the tightly coupling architecture with them.


In summary there are many SOA Management Systems out there in the market and most of them separate the concerns for logging, security, and policies there by helping in delivarables and return on investment (ROI).


Note: Amberpoint, Progress Actional, Tibco ActiveMatrix, Oracle ClearApp are registered trademarks of respective companies.

Friday, September 5, 2008

Oracle acquired ClearApp

Oracle acquired ClearApp a SOA application service management (ASM) solution provider. ClearApp specializes in providing solutions for J2EE in ASM. Oracle not only can boost its SOA market with this acquisition but also can provide a top-down visibility in application management. With this top-down visibility one can get faster problem resolution and there by increasing Return On Investment (ROI).

Somewhat similar competitive players in the market are Actional (Progress Actional), Amberpoint etc.

read more here:
http://www.clearapp.com/
http://www.oracle.com/clearapp/clearapp-general-presentation.pdf
http://www.oracle.com/clearapp/clearapp-faq.pdf