Category: Uncategorized

JSON Enrich Mediator for WSO2 ESB

Introduction

JSON support for WSO2 ESB [1] was introduced sometime back. But only small number of mediators support manipulating JSON payloads. In this article I am going to introduce a new mediator called JsonEnrichMediator [2], which work quite similar to existing Enrich mediator [3], but aiming JSON payloads. The specialty of this mediator is, since this is working with native JSON payload, JSON payload will not be converted to an XML representation. Hence there won’t be any data loss due to transformations.

Please note that this is a custom mediator I have created and will not ship with WSO2 ESB pack.

Configuring Mediator

  1. Clone the GitHub repository: https://github.com/Buddhima/JsonEnrichMediator
  2. Build the repository using maven (mvn clean install)
  3. Copy the built artifact in target folder to ESB_HOME/repository/components/dropins
  4. Download json-path-2.1.0.jar [5], json-smart-2.2.jar [6] and put them to the same folder (dropins).
  5. Start WSO2 ESB (sh bin/wso2server.sh)

Sample Scenario

For this article I am using a sample scenario which moves a JSON property within the payload. For that you need to add the following API to WSO2 ESB.

<api xmlns="http://ws.apache.org/ns/synapse" name="sampleApi" context="/sample">
<resource methods="POST" uri-template="/*">
<inSequence>
<log level="full"/>
<jsonEnrich>
<source type="custom" clone="false" JSONPath="$.me.country"/>
<target type="custom" action="put" JSONPath="$" property="country"/>
</jsonEnrich>
<respond/>
</inSequence>
</resource>
</api>

The above configuration will take the value pointed by JSONPath “$.me.country” and move it to the main body. You can find further details about JSONPath at the location [4].

Once the API is deployed, you need to send following message to ESB.


curl -H "Content-Type: application/json"
-X POST -d '{
"me":{
"country": "Sri Lanka",
"language" : "Sinhala"
}
}'
http://127.0.0.1:8280/sample

The output of the ESB should look like below


{
"me": {
"language": "Sinhala"
},
"country": "Sri Lanka"
}

Conclusion

I have shown a simple use-case of using JSON Enrich Mediator. You can see the comprehensive documentation at the code repository [2].

References

[1] WSO2 ESB JSON support : https://docs.wso2.com/display/ESB500/JSON+Support

[2] Code Repository for JSON Enrich Mediator : https://github.com/Buddhima/JsonEnrichMediator

[3] WSO2 ESB Enrich Mediator : https://docs.wso2.com/display/ESB500/Enrich+Mediator

[4] JSON Path documentation : https://github.com/jayway/JsonPath/blob/json-path-2.1.0/README.md

[5] json-path-2.1.0 : https://mvnrepository.com/artifact/com.jayway.jsonpath/json-path/2.1.0

[6] json-smart-2.2.1 : https://mvnrepository.com/artifact/net.minidev/json-smart/2.2.1

WSO2 ESB Endpoint Error Handling

Introduction

WSO2 ESB can be used as an intermediary component to connect different systems. When connecting those systems the availability of those systems is a common issue. Therefore ESB has to handle those undesirable situations carefully and take relevant actions. To cater that requirement outbound-endpoints of the WSO2 ESB can be configured. In this article I discuss two common ways of configuring endpoints.

Two common approaches to configure endpoints are;

  1. Configure with just a timeout (without suspending endpoint)
  2. Configure with a suspend state

Configure with just a timeout

This would suitable if the endpoint failure is not very frequent.

Sample Configuration:

<endpoint name="SimpleTimeoutEP">
    <address uri="http://localhost:9000/StockquoteService">
    <timeout>
        <duration>2000</duration>
        <responseAction>fault</responseAction>
    </timeout>
    <suspendOnFailure>
        <errorCodes>-1</errorCodes>
        <initialDuration>0</initialDuration>
        <progressionFactor>1.0</progressionFactor>
        <maximumDuration>0</maximumDuration>
    </suspendOnFailure>
    <markForSuspension>
        <errorCodes>-1</errorCodes>
    </markForSuspension>
</address>
</endpoint>

 

In this case we only focus on the timeout of the endpoint. The endpoint will stay as Active for ever. If a response does not receive within duration, the responseAction triggers.

duration – in milliseconds

responseAction – when response comes to a time-out message one of the following actions trigger.

  • fault – calls the fault-sequence associated
  • discard – discards the response
  • none – will not take any specific action on response (default action)

The rest of the configuration avoids the endpoint going to suspend state.

If you specify responseAction as “fault”, you can define define customize way of informing the failure to the client in fault-handling sequence or store that message and retry later.

Configure with a suspend state

This approach is useful when connection failures are very often. By suspending endpoint, ESB can save resources without unnecessarily waiting for responses.

In this case endpoint goes through a state transition. The theory behind this behavior is the circuit-breaker pattern. Following are the three states:

  1. Active – Endpoint sends all requests to backend service
  2. Timeout – Endpoint starts counting failures
  3. Suspend – Endpoint limits sending requests to backend service

Sample Configuration:

<endpoint name="Suspending_EP">
    <address uri="http://localhost:9000/StockquoteServicet">
    <timeout>
        <duration>6000</duration>
    </timeout>
    <markForSuspension>
        <errorCodes>101504, 101505</errorCodes>
        <retriesBeforeSuspension>3</retriesBeforeSuspension>
        <retryDelay>1</retryDelay>
    </markForSuspension>
    <suspendOnFailure>
        <errorCodes>101500, 101501, 101506, 101507, 101508</errorCodes>
        <initialDuration>1000</initialDuration>
        <progressionFactor>2</progressionFactor>
        <maximumDuration>60000</maximumDuration>
    </suspendOnFailure>
</address>
</endpoint>

 

In the above configuration:

If endpoint error codes are 101504, 101505; endpoint is moved from active to timeout state.

When the endpoint is in timeout state, it tries 3 attempts with 1 millisecond delays.

If all those retry attempts fail, the endpoint will move to suspend state. If a retry succeed, then endpoint will move to active state.

If active endpoint receives error codes 101500, 101501, 101506, 101507, 101508; endpoint will directly move to suspend.

After endpoint somehow moves to suspend state, it waits initialDuration before attempting any furthermore. Thereafter it will determine the time period between requests according to following equation.

Min(current suspension duration * progressionFactor, maximumDuration)

In the equation, “current suspension duration” get updated for each reattempt.

Once endpoint succeed in getting a response to a request, endpoint will go back to active state.

If endpoint will get any other error codes (eg: 101503), it will not do any state transition, and remain in active state.

Conclusion

In this article I have shown two basic configurations that would be useful to configure endpoints of WSO2 ESB. You can refer WSO2 ESB documentation for implementing more complex patterns with endpoints.

References

WSO2 ESB Documentation: https://docs.wso2.com/display/ESB500/Endpoint+Error+Handling#EndpointErrorHandling-timeoutSettings

Timeout and Circuit Breaker Pattern in WSO2 Way: http://ssagara.blogspot.com/2015/05/timeout-and-circuit-breaker-pattern-in.html

Endpoint Error Codes: https://docs.wso2.com/display/ESB500/Error+Handling#ErrorHandling-codes

Endpoint Error Handling: http://wso2.com/library/articles/wso2-enterprise-service-bus-endpoint-error-handling/

Classic Mistakes in Software Development

Introduction

Software development is a complicated activity. Hence the people work in the project can do mistakes that could affect the project. Researchers have reviewed number of software projects and identified a set of mistakes that can be seen throughout projects. They have mentioned that those mistakes might not be only causes for slow development. To slip a project into slow development, all you need to do is to make one big mistake. However to achieve efficient development, you need to avoid all mistakes.

The set of mistakes that researchers have identified is known as “Classic Mistakes”. Those bad practices have been chosen so often, by so many people. And those mistakes have predictable bad-results on the development of the project.

Four categories of classic mistakes:

  1. People related
  2. Process related
  3. Product related
  4. Technology related

People related classic mistakes

This kind of mistakes talks about how to avoid mistakes among team mates. This kind of mistakes affect directly to the development speed and it is crucial to rectify those.

Undermined motivation – Studies have shown that giving suspicious talks at the beginning, asking to work overtime reduces the motivation of the people. Sometimes team leaders take long vacations while team is working overnights. The researchers highlighted that team lead has to work along with other team members is a positive motivation.

Weak personnel – If a team need an efficient development throughout the project, the recruitment needs to hire talented developers. Also carefully filter people who could do most of the work until the end of the project.

Uncontrolled problem employees – Failure to take actions for problems with team members and team leads will eventually affect the development speed. Some higher management should actively look into those and sort out.

Heroics – Heroics within the team increases the risk and discourages cooperation among the other members of the team

Adding people to a late project – Adding new people when the project is behind schedule, can take more productivity away from team members.

Noisy, crowded offices

Friction between developers and customers – Need to increase the communication between customers and developers.

Unrealistic expectations – Setting deadlines earlier without any proper reasons can lengthen the development schedule.

Process related classic mistakes

This type of mistakes talks about issues that may arise in management and technical methodologies.

Overly optimistic schedules – This sort of scheduling will result in failure by under-scoping the project and hurt long-term morale and productivity of the developers.

Insufficient risk management – If projects risks are not actively managed, the project will lead in to slow-development mode.

Contractor failure – weak relationship with contractors can lead to slow-down the project

Insufficient planning

Short-changed upstream activities – Start coding without properly design the project plans will costs 10 or 100 times than doing it with properly designed plans.

Short-changed quality assurance – Eliminating design and code reviews, eliminating test planning and do only perfunctory testing will reduce the development of the project and ends up with major bugs.

Omitting necessary tasks from estimates – People forget about the less visible tasks and those tasks add up.

Code-like-hell programming – Developers should be sufficiently motivated rather forcing them to work hard.

Product related classic mistakes

This type of mistakes talks about which can affect the outcome of the project.

Requirements gold-planting – More requirements that are not really necessary, and pay less attention on complex features

Feature creep – On average 25% of requirements can be changed and affect the project schedule.

Developer gold planting – It is frequent that developers attempt to try new technologies that they saw in other projects, which is not actually necessary.

Technology related classic mistakes

This type of mistakes is about technologies use during the project.

Silver-bullet syndrome – Thinking that certain approach will solve every issue, and that approach has not already used by developers (eg: Object oriented design)

Overestimated savings from new tools or methods – New practices will introduce a new risk as team has to go through a learning-curve to become familiar.

Switching tools in the middle of a project – Using new tools will add a learning curve, rework and inevitable mistakes to project schedule

Lack of automated source-code control – If two or more developers are working on the same part of the project, it is necessary to adhere to source-code control practices. If not developers have to spend time on resolving conflicting changes.

Conclusion

In this article I have mentioned several mistakes that could done by people during a project life time. There could be many other mistakes which can lead to slow-down a project. However at least you should avoid these well-known classic mistakes.

JDBC Message Store for WSO2 ESB

Introduction

Message stores in WSO2 ESB is important in implementing various scenarios like store & forwarding, and reliable delivery. They can be used to persist messages during mediation or even after. Currently synapse uses in-memory message stores, which is incapable of persists after execution. It is also consuming memory too. JMS message stores can save messages, but it need more additional resources and speed becomes slower. So as a good alternative for those is a JDBC message store. This has been added to WSO2 ESB 4.9.0 onwards.

JDBC message store

In this implementation ESB uses the same implementation of message store already in ESB but in a different form. It is designed closely similar to the design of JMS message store implementation in WSO2 ESB. It uses a JDBC connector to connect to external relational databases (eg: mysql, h2 are tested).

WSO2_jdbc_store_foward
JDBC Message Store for store and Forward Pattern

 

From the very beginning of the designing phase JDBC Message Store focused on eliminating the difficulties that faced when using JMS queues as message stores. So following list of aims are achieved from JDBC store,

Easy to connect – It’s rather easy to connect with databases compared to JMS queues
More operations on data – In JMS queues methods like random selecting messages are not supposed to support. But with JDBC it become a reality, and has operations very close to in-memory stores.
Fast transactions – In test I have seen that JDBC stores are capable of handling about 2300 transactions per second which is 10 times faster than the existing system.
Work with high capacity and long-time – Since JDBC Stores uses databases as the medium to store data, and can depend on up to Terabytes of data. It is also generally accepted that Databases are capable of handling data for long-time compared to JMS queues.
After having tests in different backgrounds with different configurations, I have seen that the outcomes of JDBC message store has achieved more than expected at the initial stages.

In this store implementation, the message is converted to serializable Java object, so it to be able to store as a blob.
Construction of the persisting message has two basic parts JDBC Axis2 Message and JDBC Synapse Message. Combination of those two will produce the Storable Message , which is sent to database.

wso2_jdbc_store_message
Constructing a Storable Message

Other than message constructing classes following are explanation on what rest of the classes are doing.

JDBCMessageStore– Provides the fundamental interface to external parties by encapsulating the underlying implementations. This class exposes store, poll, get, clear, peek and other generic methods of messages stores to outside parties.
JDBCMessageStoreConstants – This class defines the related constant values for JDBC message store. This class make it easy for maintain JDBC store implementation by gathering all the constants in to a single place.
JDBCConfiguration – This class was defined to provide necessary utility functionalities to JDBC operations. Basically it deals with creating connections and terminating connections, querying database tables etc.
JDBCMessageConverter – This class is to help with converting SOAP messages in to serializable Java objects and the reverse process after querying the required. This works as an adaptor between database and ESB.
JDBCProducer – This is to produce messages into store which is used by the store mediator.
JDBCConsumer – This is to consume messages from a message-store, and to be used by message processors.
Those classes along with the classes mentioned previously, creates a successful JDBC Message Store.

Configuration of JDBC Message Store

To use JDBC Message store customer has to add the required JDBC support. There after following configuration will allow any message processor to use JDBC message store as same as other message store. Configuration can be specified as an inline or points to a datastore (which gives you additional control over database).

<store messageStore="MyStore"/>

<messageStore class="org.apache.synapse.message.store.jdbc.JDBCMessageStore" name="MyStore">

(
<parameter name="store.jdbc.driver">com.mysql.jdbc.Driver</parameter>
<parameter name="store.jdbc.connection.url">jdbc:mysql://localhost:3306/mystore</parameter>
<parameter name="store.jdbc.username">root</parameter>
<parameter name="store.jdbc.password"></parameter>
<parameter name="store.jdbc.table">store_table</parameter>

|

<parameter name="store.jdbc.dsName">reportDB</parameter>
<parameter name="store.jdbc.table">store_table</parameter>

)

</messageStore>

In-lined Data Source

store.jdbc.driver – Database driver class name
store.jdbc.connection.url– Database URL
store.jdbc.username – User name for access Database
store.jdbc.password – Password for access Database
store.jdbc.table – Table name of the database

External Data Source

store.jdbc.dsName – The name of the Datasource to be looked up
store.jdbc.table – Table name of the database
Optionally;
store.jdbc.icClass – Initial context factory class. The corresponding java environment property is java.naming.factory.initial
store.jdbc.connection.url– The naming service provider url . The corresponding java environment property is java.naming.provider.url
store.jdbc.username – This is corresponding to the java environment property java.naming.security.principal
store.jdbc.password – This is corresponding to the java environment property This is corresponding to the java environment property java.naming.security.principal

Database script

For creation database table you can use following scripts.

MySQL :
CREATE TABLE jdbc_store_table(
indexId BIGINT( 20 ) NOT NULL AUTO_INCREMENT ,
msg_id VARCHAR( 200 ) NOT NULL ,
message BLOB NOT NULL ,
PRIMARY KEY ( indexId )
)

H2 :
CREATE TABLE jdbc_store_table(
indexId BIGINT( 20 ) NOT NULL AUTO_INCREMENT ,
msg_id VARCHAR( 200 ) NOT NULL ,
message BLOB NOT NULL ,
PRIMARY KEY ( indexId )
)

You can create similar SQL script according to your database.

Sample

First you need to put the relevant database driver into repository/components/lib folder. [2]
Following sample configuration is based on a mqsql database name “mystore” and table “store_table”.

<proxy xmlns="http://ws.apache.org/ns/synapse"
       name="MessageStoreProxy"
       transports="https http"
       startOnLoad="true"
       trace="disable">
   <description/>
   <target>
      <inSequence>
         <property name="FORCE_SC_ACCEPTED" value="true" scope="axis2"/>
         <property name="OUT_ONLY" value="true"/>
         <property name="target.endpoint" value="StockQuoteServiceEp"/>
         <store messageStore="MyStore"/>
      </inSequence>
   </target>
   <publishWSDL uri="http://localhost:9000/services/SimpleStockQuoteService?wsdl"/>
</proxy>

<messageStore xmlns="http://ws.apache.org/ns/synapse"
              class="org.apache.synapse.message.store.impl.jdbc.JDBCMessageStore"
              name="MyStore">
   <parameter name="store.jdbc.password"/>
   <parameter name="store.jdbc.username">root</parameter>
   <parameter name="store.jdbc.driver">com.mysql.jdbc.Driver</parameter>
   <parameter name="store.jdbc.table">store_table</parameter>
   <parameter name="store.jdbc.connection.url">jdbc:mysql://localhost:3306/mystore</parameter>
</messageStore>

<messageProcessor xmlns="http://ws.apache.org/ns/synapse"
                  class="org.apache.synapse.message.processor.impl.forwarder.ScheduledMessageForwardingProcessor"
                  name="ScheduledProcessor"
                  messageStore="MyStore">
   <parameter name="max.delivery.attempts">5</parameter>
   <parameter name="interval">10</parameter>
   <parameter name="is.active">true</parameter>
</messageProcessor>

Message processor also can be added via management console through following section:

wso2_message_store_screen
JDBC Message Processor

 

Use axis2 client as follows to send messages to the proxy:

ant stockquote -Daddurl=http://localhost:8280/services/MessageStoreProxy

Rough comparison between JMS and JDBC message store performance

For the testing the selected database details are as follows:

Type : MySQL on XAMPP server for database (JMS is just for comparison)
Storage Engine : MyISAM
Machine Details : Ubuntu 14.04 on Intel® Core™ i7-4800MQ CPU @ 2.70GHz × 8 with 16 GB RAM
Messages sent via : JMeter client

# of Threads Messages/Thread Total Messages Throughput / sec
JMS Store Producer JDBC Store Producer
1 100 100 63.9 552.5
1 500 500 62.8 566.3
1 1000 1000 63.5 577.4
1 2000 2000 62.3 629.3
1 3000 3000 62.8 649.9
10 10 100 73.7 108.8
10 50 500 70.4 511.8
10 100 1000 70.7 928.5
10 200 2000 71.3 1537.3
10 300 3000 72.3 1827
50 10 500 71.6 494.1
50 100 5000 72.5 3494.1
100 250 25000 73.4 5529.8
500 100 50000 7055.2

(Note: At small number of messages, JDBC figures have not reached stable throughput)

References:

[1] Pull request containing the implementation – https://github.com/wso2/wso2-synapse/pull/91

[2] MySQL connector – https://dev.mysql.com/downloads/connector/j/

[3] WSO2 ESB 4.9.0-ALPHA – https://github.com/wso2/product-esb/releases/tag/esb-parent-4.9.0-ALPHA

 

Basics of Virtualization

Introduction

Today with Cloud Computing, “Virtualization” has become a buzz word. Many hosting services tend to use this technology since it’s more cost effective. So during last few weeks, I too get involved with this term, so thought of having look at what it really means. In this case a book[1] and a video[2] I found was really helpful. But I need to say that technology is rapidly changing. So you may find there are new ways to do “Virtualization”.

What is Virtualization

You have to understand that virtualization is not cloud computing, but a component of Cloud Computing. As the book[1] says: “… the process of converting physical IT resources into virtual IT resources.” So first you need to have physical resources to do virtualization. Resouces in the definition can be;

  1. Servers
  2. Storage
  3. Networks etc.

To explain virtualization, there are few other terms you need to understand.

  • Host – The actual physical hardware use for virtualization
  • Host Operating System – The operating system which runs on-top-of Host hardware
  • Guest Operating System – The operating system which runs on-top-of virtualized layer
  • Hypervisor (Virtual Machine Monitor) – The software piece which enables virtualized software to access underlying hardware

Why Virtualization

There are several reasons why people are so interested about virtualization

  • Hardware Independence – Virtualized software is decoupled with underlying hardware, so that it can be moved from one-to-another just as moving a file
  • Server Consolidation – Instead of having multiple servers to run many servers, now all virtualized servers can run on the same server and share underlying hardware
  • Resource Replication – Virtual servers saves on virtual hard disks, which can apply simple file operations. So for en example, you can replicate the same server by copy the virtual hard disk

How to Virtualize

Types of virtualization depends on where the Hypervisor lies. So from the top-level, there are 2 kinds of virtualizations you can see (as mentioned in [1] & [2]):

  1. Hardware-based Virtualization
  2. Operating System-based Virtualization

So let’s dive more deep into these 2 areas.

Hardware-based Virtualization

This means, take the bare hardware and install hypervisor. So hypervisor will provide basic functionality. Then you can install operating systems on-top-of hypervisor. So in this case you don’t need any host operating system. Therefore you need to have a client machine which need to manage the hypervisor.  Those management software can do magical things such as moving operating systems between several hardware servers etc. This kind of virtualization is optimized for performance. The issue with this approach is compatibility with hardware devices. Because device drivers may not be available for hypervisors when compared to operating systems. So following diagram shows how hardware-based virtualization can be shown (taken from [1]).

Hardware Based Hypervisor
Hardware Based Hypervisor

 

 

Operating System-based Virtualization

This kind of virtualization you may have already experienced too (without knowing 🙂 ). In this case hypervisor  is used to abstract the host operating system. Simple use-case is using Virtual Box to run Windows on-top of Ubuntu, where Virtual Box act as the hypervisor. This approach will eliminates the  hardware compatibility issues but incurs more overhead as host operating system needs resources in addition to guests. So following diagram shows how operating system-based virtualization can be shown (taken from [1]).

Operating System-Based Virtualization
Operating System-Based Virtualization

 

As I mentioned in the introduction, today there are new concepts introduced to this field. One such interesting thing is Docker[3], which has become more popular recently. So before the conclusion of this post I thought of keeping a note on Docker too.

Docker

Docker from outside can be view as similar approach to Operating System-based virtualization. But the difference is that Docker has able to eliminate the overhead of the guest operating system. With Docker they have introduced the concept of “Containers” which can run on Linux platforms currently and with additional library on Windows too. In Docker world the hypervisor is Docker Engine which enables you to do all sorts of things to a Docker Container. You can play with Docker through the online try-it console[4].

Docker

 

Conclusion

Through this post, I tried to express my view on basic concepts on Virtualization in computer world. In here I mentioned why virtualization is important and benefits of it. After that I expressed how virtualization can be achieved and brief introduction to Docker too. Other than mentioned things various companies tries to make this virtualization more efficient and various technologies are upcoming. So this is just a glimpse of the virtualization world and if you are interested, you can refer the given resources and expand your knowledge!

 

References

[1] Cloud Computing: Concepts, Technology & Architecture by Thomas Erl

[2] Introduction to Virtualization – https://www.youtube.com/watch?v=zLJbP6vBk2M 

[3] Docker – https://www.docker.com/ 

[4] Docker Try It! – https://www.docker.com/tryit/

Creating HTTP Server with Java

Introduction

After a busy period I got a chance to look something different from usual work. This time its about HTTP server implementation using Java. Most of Java programmers have implemented TCP socket servers when stepping in to Java world. Thereafter developers compare Java with other languages, and think that implementing a server with Java is pretty hard. This is because they are not aware about Java HttpServer [1]. Also you may wonder that there are use-cases which you can easily accomplish with HttpServer while need more effort on alternative methods. So in this post, I’m going to introduce a very simple use-case of it.

Usecase

The example following explains implementing a HTTP server which accepts POST request containing data and send the same payload back to the client (a basic version of a echo server).
A sample request :
————————————–
POST /test HTTP/1.1
Host: localhost:8080
Cache-Control: no-cache

TEST DATA HERE
————————————–

Implementation

Implementation of the usecase contains only 3 high-level steps.

  1. Create a HttpServer object with suitable port number
  2. Add Context suites for your needs (will describe soon)
  3. Start the server

Adding a Context

A context describes the pattern in request url for a matching Handler. A Handler is responsible for further processing of the request. For that a Handler should implement a HttpHandler with overriding ‘handle(HttpExchange he)’ method. Within this handle method, you can extract request and build the response.
Working with HttpExchange in handle method should follow the following sequence [2]:

  1. getRequestMethod() to determine the command
  2. getRequestHeaders() to examine the request headers (if needed)
  3. getRequestBody() returns a InputStream for reading the request body. After reading the request body, the stream is close.
  4. getResponseHeaders() to set any response headers, except content-length
  5. sendResponseHeaders(int,long) to send the response headers. Must be called before next step.
  6. getResponseBody() to get a OutputStream to send the response body. When the response body has been written, the stream must be closed to terminate the exchange.

 

In addition to that HttpExchange [2] contains several additional methods to extract more information from the request and change the response.

Sample

Now let’s move on to the implementation of the above mentioned use-case.


import com.sun.net.httpserver.Headers;
import com.sun.net.httpserver.HttpExchange;
import com.sun.net.httpserver.HttpHandler;
import com.sun.net.httpserver.HttpServer;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.net.HttpURLConnection;
import java.net.InetSocketAddress;
import java.util.List;
import java.util.Map;
import java.util.Set;
import java.util.logging.Level;
import java.util.logging.Logger;

/**
 *
 * @author BUDDHIMA
 */
public class ServerTest {

    public static void main(String[] args) {
        try {

            // Bind to port 8080
            HttpServer httpServer = HttpServer.create(new InetSocketAddress(8080), 0);

            // Adding '/test' context
            httpServer.createContext("/test", new TestHandler());

            // Start the server
            httpServer.start();

        } catch (IOException ex) {
            Logger.getLogger(ServerTest.class.getName()).log(Level.SEVERE, null, ex);
        }

    }

    // Handler for '/test' context
    static class TestHandler implements HttpHandler {

        @Override
        public void handle(HttpExchange he) throws IOException {
            System.out.println("Serving the request");

            // Serve for POST requests only
            if (he.getRequestMethod().equalsIgnoreCase("POST")) {

                try {

                    // REQUEST Headers
                    Headers requestHeaders = he.getRequestHeaders();
                    Set<Map.Entry<String, List<String>>> entries = requestHeaders.entrySet();

                    int contentLength = Integer.parseInt(requestHeaders.getFirst("Content-length"));

                    // REQUEST Body
                    InputStream is = he.getRequestBody();

                    byte[] data = new byte[contentLength];
                    int length = is.read(data);

                    // RESPONSE Headers
                    Headers responseHeaders = he.getResponseHeaders();

                    // Send RESPONSE Headers
                    he.sendResponseHeaders(HttpURLConnection.HTTP_OK, contentLength);

                    // RESPONSE Body
                    OutputStream os = he.getResponseBody();

                    os.write(data);

                    he.close();

                } catch (Exception e) {
                    e.printStackTrace();
                }
            }

        }
    }
}

Conclusion

Through this post I wanted to give a very brief introduction to Java HttpServer, which is not popular among many developers. I believe this will help you to build embedded HTTP servers and quickly create a backend servers according for your needs.

References

[1] Documentation on HttpServer class : http://docs.oracle.com/javase/7/docs/jre/api/net/httpserver/spec/com/sun/net/httpserver/HttpServer.html
[2] Documentation on HttpExchange class : http://docs.oracle.com/javase/7/docs/jre/api/net/httpserver/spec/com/sun/net/httpserver/HttpExchange.html
[3] Sample with extracting parameters from a request : http://www.rgagnon.com/javadetails/java-get-url-parameters-using-jdk-http-server.html
[4] Sample of a file download : http://www.rgagnon.com/javadetails/java-have-a-simple-http-server.html

Local Transport in WSO2 ESB

Introduction

WSO2 ESB [1] is considered as the fastest 100% open-source enterprise service bus on the planet. It has number of features that cater for different enterprise-integration scenarios. It also supports number of transports including NHTTP, JMS, VFS, Local, SMS, Mail and domain-specific transports such as FIX, HL7. Among those transports, recently I got an opportunity to work a bit on Local Transport. So this post is based on what I have experienced with Local Transport.

Local Transport [2] was first introduced to WSO2 ESB in version 4.0. It helps to communicate with proxy-services in an efficient manner. This gain has obtained by using In-JVM calls when calling to proxy-services. Considering technical side, the sender of Local Transport is implemented based on org.apache.axis2.transport.local.NonBlockingLocalTransportSender , and there’s no receiver implementation.

Enabling Local Transport

To enable Local Transport, you need to follow the following steps:
1. Go to /repository/conf/carbon.xml
2. Replace local://services/ with https://${carbon.local.ip}:${carbon.management.port}${carbon.context}/services/

3. Go to /repository/conf/axis2/axis2.xml
4. Comment the following two lines:

<transportReceiver name="local" class="org.wso2.carbon.core.transports.local.CarbonLocalTransportReceiver"/>
<transportSender name="local" class="org.wso2.carbon.core.transports.local.CarbonLocalTransportSender"/>

5. Add the following line:

<transportSender name="local" class="org.apache.axis2.transport.local.NonBlockingLocalTransportSender"/>

If you need to use local transport with callout mediator, you do not need to perform configuration mentioned in this section as callout mediator requires blocking local transport which is configured by default in WSO2 ESB distribution.

Scenarios

Sample scenario [3] can be find along with WSO2 documentation.
Sample URL: https://docs.wso2.com/display/ESB481/Sample+268%3A+Proxy+Services+with+the+Local+Transport

In the sample you can see there are 3 proxy-services. Once client sends a message to ESB, LocalTransportProxy, SecondProxy and StockQuoteProxy get called sequentially. In that scenario communication between proxy-services handles by In-JVM calls, and that makes less overhead on network traffic.

You can try the same execution chain by resetting the Enabling Local Transport settings and replacing “local://localhost” prefixes with “http://localhost:8280 ” . Then the intercommunication with proxies happen across the network.

If you capture the network traffic with Wireshark for the above 2 scenarios (TCP traffic is filtered);

With Local Transport:

with_local

Without Local Transport:

without_local

As depicts in the first image, with Local Transport you can observe just the requests and responses which are exchanged with external parties (Client and BE service). But without Local Transport, you can see the proxy-service calls happening through the network.

To consider Local Transport

Though Local Transport seems to be an efficient way to communicate, it comes with several limitations which make Local Transport is not the choice.

  1. Local Transport cannot be used to send REST API calls, which require the HTTP/S transports.
  2. WS-Security cannot be used with the local transport. Since the local is mainly used to make calls within the same VM, WS-Security is generally not required in scenarios where it is used.
  3. If you want to make calls across tenants, you should use a non Local Transport even if they run from the same VM.

Conclusion

This post is based on the facts I found about Local Transport so far. So it’s your integration scenario which decides whether Local Transport caters or not. But it’s worth always go for the most efficient way to get the full benefit of service integration software.

References

[1] WSO2 ESB – http://wso2.com/products/enterprise-service-bus/

[2] Local transport – https://docs.wso2.com/display/ESB481/Local+Transport

[3] Local Transport Sample – https://docs.wso2.com/display/ESB481/Sample+268%3A+Proxy+Services+with+the+Local+Transport