WebSocket with Spring Boot

Introduction

In this post I am going to talk briefly about developing a WebSocket based application using Spring Boot framework. WebSocket[1] is a full duplex protocol allows bi-directional communication. At the moment, widely using protocols such as HTTP is uni-directional and use long polling mechanisms to achieve bi-directional behavior. But using protocols such as WebSocket allows sending requests from server to client-side. However unlike HTTP, WebSocket does not form a strong application-level protocol. WebSocket sits as a thin layer on-top-of TCP protocol and allow application developers to come-up with a high-level messaging protocol design.

STOMP

Simple Text-Oriented Messaging Protocol (STOMP) is selected by the Spring framework for its WebSocket support. STOMP uses message brokers to provide bi-directional communication.

Server-side Code

Server-side code of a websocket server is as follows:


import org.springframework.messaging.handler.annotation.MessageMapping;
import org.springframework.messaging.handler.annotation.SendTo;
import org.springframework.stereotype.Controller;

@Controller
public class GreetingController {

@MessageMapping("/hello")
 @SendTo("/topic/greetings")
 public Greeting greeting(HelloMessage helloMessage) throws Exception {
 return new Greeting("Hello," + helloMessage.getName() + "!");
 }
}

In the above code @MessageMapping annotation is used to map the messages come with “/hello” in the path. @SendTo is used to specify the destination of the result. Here, HelloMessage and Greeting are bean classes which can be found in [2].

The server-side configuration is as follows:


@Configuration
@EnableWebSocketMessageBroker
public class WebsocketConfigurer extends AbstractWebSocketMessageBrokerConfigurer {

@Override
 public void configureMessageBroker(MessageBrokerRegistry registry) {
 registry.enableSimpleBroker("/topic");
 registry.setApplicationDestinationPrefixes("/app");
 }

@Override
 public void registerStompEndpoints(StompEndpointRegistry stompEndpointRegistry) {
 stompEndpointRegistry.addEndpoint("/gs-guide-websocket").withSockJS();
 }
}

WebsocketConfigurer class is annotated as the configuration class for the application. By configureMessageBroker method it enables message broker and add “/topic” as topic it holds. “/app” is used to identify messages which need to send to the controller.

In the second method, “/gs-guide-websocket” is used as a endpoint which clients can connect. So let’s move in to the client-side code. It contains 2 files, a static HTML and a JS file.

<html>
<head>
<title>Hello WebSocket</title>
	<link href="/webjars/bootstrap/css/bootstrap.min.css" rel="stylesheet">
<!--	<link href="/main.css" rel="stylesheet">-->
<script src="/webjars/jquery/jquery.min.js"></script>
    <script src="/webjars/sockjs-client/sockjs.min.js"></script>
<script src="/webjars/stomp-websocket/stomp.min.js"></script>
    <script src="/app.js"></script>
</head>
<body>
<noscript>
<h2 style="color: #ff0000">Seems your browser doesn't support Javascript! Websocket relies on Javascript being
enabled. Please enable
Javascript and reload this page!</h2>
</noscript>
<div id="main-content" class="container">
<div class="row">
<div class="col-md-6">
<form class="form-inline">
<div class="form-group">
<label for="connect">WebSocket connection:</label>
<button id="connect" class="btn btn-default" type="submit">Connect</button>
<button id="disconnect" class="btn btn-default" type="submit" disabled="disabled">Disconnect
</button>
</div>
</form>
</div>
<div class="col-md-6">
<form class="form-inline">
<div class="form-group">
<label for="name">What is your name?</label>
<input type="text" id="name" class="form-control" placeholder="Your name here...">
</div>
<button id="send" class="btn btn-default" type="submit">Send</button>
</form>
</div>
</div>
<div class="row">
<div class="col-md-12">
<table id="conversation" class="table table-striped">
<thead>
<tr>
<th>Greetings</th>
</tr>
</thead>
<tbody id="greetings"></tbody>
</table>
</div>
</div>
</div>
</body>
</html>

var stompClient = null;

function setConnected(connected) {
$("#connect").prop("disabled", connected);
$("#disconnect").prop("disabled", !connected);
if (connected) {
$("#conversation").show();
}
else {
$("#conversation").hide();
}
$("#greetings").html("");
}

function connect() {
var socket = new SockJS('/gs-guide-websocket');
stompClient = Stomp.over(socket);
stompClient.connect({}, function (frame) {
setConnected(true);
console.log('Connected: ' + frame);
stompClient.subscribe('/topic/greetings', function (greeting) {
showGreeting(JSON.parse(greeting.body).content);
});
});
}

function disconnect() {
if (stompClient != null) {
stompClient.disconnect();
}
setConnected(false);
console.log("Disconnected");
}

function sendName() {
stompClient.send("/app/hello", {}, JSON.stringify({'name': $("#name").val()}));
}

function showGreeting(message) {
$("#greetings").append("
<tr>
<td>" + message + "</td>
</tr>
");
}

$(function () {
$("form").on('submit', function (e) {
e.preventDefault();
});
$( "#connect" ).click(function() { connect(); });
$( "#disconnect" ).click(function() { disconnect(); });
$( "#send" ).click(function() { sendName(); });
});

Finally the pom file of the project is as follows:

<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>1.4.1.RELEASE</version>
</parent>
<groupId>com.buddhima.websocket</groupId>
<artifactId>websocket-try</artifactId>
<version>1.0-SNAPSHOT</version>

<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-websocket</artifactId>
</dependency>

<dependency>
<groupId>org.webjars</groupId>
<artifactId>webjars-locator</artifactId>
</dependency>
<dependency>
<groupId>org.webjars</groupId>
<artifactId>sockjs-client</artifactId>
<version>1.0.2</version>
</dependency>
<dependency>
<groupId>org.webjars</groupId>
<artifactId>stomp-websocket</artifactId>
<version>2.3.3</version>
</dependency>
<dependency>
<groupId>org.webjars</groupId>
<artifactId>bootstrap</artifactId>
<version>3.3.7</version>
</dependency>
<dependency>
<groupId>org.webjars</groupId>
<artifactId>jquery</artifactId>
<version>3.1.0</version>
</dependency>
</dependencies>

<properties>
<java.version>1.8</java.version>
</properties>

<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
</build>
</project>

 

References

[1] WebSocket protocol RFC – https://tools.ietf.org/html/rfc6455

[2] Using WebSocket to build an interactive web application – https://spring.io/guides/gs/messaging-stomp-websocket/

[3] WebSocket Support – https://docs.spring.io/spring/docs/current/spring-framework-reference/html/websocket.html

Start Using Java 8 Lambda Expressions

Introduction

Java 8 comes with a bunch of new features for its developers. One such improvement is Lambda expressions. Lambda Expressions allows Java developers to use functions as values, and pass it as a value to a method. This might be familiar for people from functional programming background, but a bit difficult for people who tends to think in object-oriented mindset. In this article, I am going through series of examples to show how we can use lambda expressions in coding.

Prerequisites

Before you start, you need to install Java 8 in your machine and create a project using Java 8. Here I’m going describe things in more abstract-level. So you should possess knowledge on using your IDE for basic things such as creating classes, executing Java code etc.

Example Scenario

For this article I have selected a scenario where you need to go though set of books and select books based on different criteria. Those selection criteria are list all the books, list books which are novels, and list title of the books written in 20th century. So let’s start coding.

Creating entity class

First step of solving such problem is to create an entity class called Book which can represent a single instance of a book. So here it is.


public class Book {

private String name;
 private String author;
 private int year;
 private String language;
 private String category;

public Book(String name, String author, int year, String language, String category) {
 this.name = name;
 this.author = author;
 this.year = year;
 this.language = language;
 this.category = category;
 }

public String getName() {
 return name;
 }

public void setName(String name) {
 this.name = name;
 }

public String getAuthor() {
 return author;
 }

public void setAuthor(String author) {
 this.author = author;
 }

public int getYear() {
 return year;
 }

public void setYear(int year) {
 this.year = year;
 }

public String getLanguage() {
 return language;
 }

public void setLanguage(String language) {
 this.language = language;
 }

public String getCategory() {
 return category;
 }

public void setCategory(String category) {
 this.category = category;
 }

@Override
 public String toString() {
 return "Book{" +
 "name='" + name + '\'' +
 ", author='" + author + '\'' +
 ", year=" + year +
 ", language='" + language + '\'' +
 ", category='" + category + '\'' +
 '}';
 }
}

I think I no need to explain the above simple Java class as it should be familiar to you.

Conventional Method

This is what we usually do to solve this type of problems.

public class BookFinderExample1 {

    public static void main(String[] args) {
        List<Book> books = Arrays.asList(
                new Book("Moby-Dick", "Herman Melville", 1851, "EN", "Novel"),
                new Book("War and Peace", "Leo Tolstoy", 1869, "RU", "Novel"),
                new Book("The Three Musketeers", "Alexandre Dumas", 1844, "FR", "Novel"),
                new Book("Les Miserables", "Victor Hugo", 1862, "FR", "Fiction"),
                new Book("Journey to the West", "Wu Cheng'en", 1592, "ZH", "Fiction"),
                new Book("Wild Swans", "Jung Chang", 1991, "ZH", "Biography"),
                new Book("The Reader", "Bernhard Schlink", 1995, "DE", "Novel"),
                new Book("Perfume", "Patrick Suskind", 1985, "DE", "Fiction")
        );

        // 1. print all books
        System.out.println("Print all books");
        printAllBooks(books);

        // 2. print all novels
        System.out.println("Print all novels");
        printAllNovels(books);

        // 3. print all books in 20th century
        System.out.println("Print all books in 20th century");
        printAllIn20thCentury(books);
    }

    private static void printAllBooks(List<Book> books) {
        for (Book book : books) {
            System.out.println(book.toString());
        }
    }

    private static void printAllNovels(List<Book> books) {
        for (Book book : books) {
            if (book.getCategory().equals("Novel"))
                System.out.println(book.toString());
        }
    }

    private static void printAllIn20thCentury(List&lt;Book&gt; books) {
        for (Book book : books) {
            if (book.getYear() > 1900 && book.getYear() < 2001)
                System.out.println(book.getName());
        }
    }
}

First it created a list of books (I won’t going to repeat this step in next examples). Then we create 3 methods which serves our purpose. And we call those methods one by one. Though this fulfills out requirement, seems that’s not a scalable solution. Once we have a new requirement, we need to create a new method and call it.

Using Generic Solution

If we carefully look into the methods we have implemented, they all do a common thing. They iterate though a given list of books, checks a condition and perform an action (eg: print the book). So we can use interfaces for this and stick with just one method. Let’s see how.

public class BookFinderExample2 {

    public static void main(String[] args) {
        List<Book> books = Arrays.asList(
                .......
        );

        // 1. print all books
        System.out.println("Print all books");
        printBooks(books, new Checker() {
            public boolean check(Book book) {
                return true;
            }
        }, new Action() {
            public void perform(Book book) {
                System.out.println(book.toString());
            }
        });

        // 2. print all novels
        System.out.println("Print all novels");
        printBooks(books, new Checker() {
            public boolean check(Book book) {
                return book.getCategory().equals("Novel");
            }
        }, new Action() {
            public void perform(Book book) {
                System.out.println(book.toString());
            }
        });

        // 3. print all books in 20th century
        System.out.println("Print all books in 20th century");
        printBooks(books, new Checker() {
            public boolean check(Book book) {
                return (book.getYear() > 1900 && book.getYear() < 2001);
            }
        }, new Action() {
            public void perform(Book book) {
                System.out.println(book.getName());
            }
        });
    }

    private static void printBooks(List<Book> books, Checker checker, Action action) {
        for (Book book : books) {
            if (checker.check(book)) {
                action.perform(book);
            }
        }
    }

    interface Checker {
        boolean check(Book book);
    }

    interface Action {
        void perform(Book book);
    }
}

In the above solution I have introduced two interfaces and both of them contains a single method exposed. With the use of just one method printBooks and objects implementation of those two interfaces we have achieved the same results as before. Now we have taken out the action and inject in to the printBooks method. The instances of Checker and Action is created with anonymous inner classes.

The above seems to be fine for generalizing the solution but the syntax seems to be so tedious. So far we have not used anything new in Java 8. Let’s use Java 8 new features to ease our coding.

Using Lambda Expressions

Lets have a look at one of our anonymous inner class

new Action() {
    public void perform(Book book) {
        System.out.println(book.toString());
    }
}

The above is an anonymous inner class created for Action interface. Action interface has only a single method called perform. That method takes one argument and prints it. So such Anonymous Inner classes can be written as follows.

(Book book) -> System.out.println(book.toString())

Since the Action interface has only one method, we no need to say it’s name. Since it has one line of code, no need to use curly-brackets. Now with single argument, no need of using brackets, and specifying argument type.

book -> System.out.println(book.toString())

That’s a lambda expression. So let’s substitute lambda expressions.

public class BookFinderExample3 {

    public static void main(String[] args) {
        List<Book> books = Arrays.asList(
                ....
        );

        // 1. print all books
        System.out.println("Print all books");
        printBooks(books, book ->  true, book -> System.out.println(book.toString()));

        // 2. print all novels
        System.out.println("Print all novels");
        printBooks(books, book -> book.getCategory().equals("Novel"), book -> System.out.println(book.toString()));

        // 3. print all books in 20th century
        System.out.println("Print all books in 20th century");
        printBooks(books, book ->  (book.getYear() > 1900 && book.getYear() < 2001), book -> System.out.println(book.getName()));
    }

    private static void printBooks(List<Book> books, Checker checker, Action action) {
        for (Book book : books) {
            if (checker.check(book)) {
                action.perform(book);
            }
        }
    }

    @FunctionalInterface
    interface Checker {
        boolean check(Book book);
    }

    @FunctionalInterface
    interface Action {
        void perform(Book book);
    }
}

We have introduced 2 interfaces for our work. But is it necessary? No. Java SDK developers have identified this and provide us a set of already defined interfaces inside java.util.function package. All we have to do it, reuse them!

public class BookFinderExample4 {

    public static void main(String[] args) {
        List<Book> books = Arrays.asList(
                ...
        );

        // 1. print all books
        System.out.println("Print all books");
        printBooks(books, book ->  true, book -> System.out.println(book.toString()));

        // 2. print all novels
        System.out.println("Print all novels");
        printBooks(books, book -> book.getCategory().equals("Novel"), book -> System.out.println(book.toString()));

        // 3. print all books in 20th century
        System.out.println("Print all books in 20th century");
        printBooks(books, book ->  (book.getYear() > 1900 && book.getYear() < 2001), book -> System.out.println(book.getName()));
    }

    private static void printBooks(List<Book> books, Predicate<Book> checker, Consumer<Book> action) {
        for (Book book : books) {
            if (checker.test(book)) {
                action.accept(book);
            }
        }
    }
}

In the above example we have used two interfaces Predicate and Consumer. you can find more information about those in Oracle’s official website. You can find plenty of such interfaces which can help your coding.

Using streams

We are not going to stop our simplification from there. If we look in to the printBooks method, it iterates through the list of books and perform things mention in the interface implementations. This kind of iteration is called external iteration, because for-loop iterates the list. In Java 8, streams provide the functionality of internal iteration, which is implemented by converting the list in to a stream. The stream can be considered as a conveyor belt which brings list items one by one. We can use filters etc., to filter-out the required elements and so what every action you like. So when using streams our code would look like as follows.

public class BookFinderExample5 {

    public static void main(String[] args) {
        List<Book> books = Arrays.asList(
                ...
        );

        // 1. print all books
        System.out.println("Print all books");
        books.stream()
                .filter(book ->  true)
                .forEach(book -> System.out.println(book.toString()));

        // 2. print all novels
        System.out.println("Print all novels");
        books.stream()
                .filter(book -> book.getCategory().equals("Novel"))
                .forEach(book -> System.out.println(book.toString()));

        // 3. print all books in 20th century
        System.out.println("Print all books in 20th century");
        books.stream()
                .filter(book ->  (book.getYear() > 1900 && book.getYear() < 2001))
                .forEach(book -> System.out.println(book.getName()));
    }

}

Conclusion

We have started our example from conventional coding and step-by-step brings the Java 8 new features such as lambda expressions, streams and forEach. Similarly you can apply Java 8 concepts in your development as well. Resources in references section below will help you to find more information about Java 8 Lambda Expressions.

References

[1] Oracle’s Java 8 website : http://docs.oracle.com/javase/8/docs/

[2] Oracle’s website on util-functions : https://docs.oracle.com/javase/8/docs/api/java/util/function/package-summary.html

[3] Java Brains Tutorial on Lambda Expressions : https://www.youtube.com/watch?v=gpIUfj3KaOc&list=PLqq-6Pq4lTTa9YGfyhyW2CqdtW9RtY-I3

Microservices with Spring Boot

Introduction

Currently enterprise application development is more interested towards building them as microservices. This trend started about 2 years back and some organizations take this as an opportunity to do a complete re-write of their products. To help developing microservices, several organizations have done framework implementations. In here I am talking about using Spring Boot to create a very basic microservice.

Use-case

This system is about handling patient records. So it is more like an CRUD application. To persist data, I am using a Mongo DB (embedded version). First, let’s see what would be structure of this project.

proj-structure

Fist you need to create a project with the above structure. You may find maven arc-types which helps to do that. Next the pom file should be created properly. Here, I’m showing the important sections of the pom file.

 <parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>1.4.1.RELEASE</version>
</parent>
...
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-mongodb</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-jersey</artifactId>
</dependency>

<dependency>
<groupId>de.flapdoodle.embed</groupId>
<artifactId>de.flapdoodle.embed.mongo</artifactId>
</dependency>

<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>

</dependencies>

 

Application.java file contains the main method to start the microservice. So it should looks like as follow:

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;

@SpringBootApplication
public class Application {

 public static void main(String[] args) {
     SpringApplication.run(Application.class, args);
 }
}

ApplicationConfig.java file is used to provide configurations to Spring framework. Here we provide the location of the service and REST-Template. So it should look like follows:

 

import org.glassfish.jersey.server.ResourceConfig;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.web.client.RestTemplate;

import javax.inject.Named;

@Configuration
public class ApplicationConfig {
    @Named
    static class JerseyConfig extends ResourceConfig {
        public JerseyConfig() {
            this.packages("com.project.capsule.rest");
        }
    }

    @Bean
    public RestTemplate restTemplate() {
        RestTemplate restTemplate = new RestTemplate();
        return restTemplate;
    }
}

Next we can extend MongoRepository and create PatientReportRepository. This is very interesting capability of Spring framework as it can convert method names in to SQL queries directly.

import com.project.capsule.bean.PatientReport;
import org.springframework.data.mongodb.repository.MongoRepository;
import java.util.List;

public interface PatientReportRepository extends MongoRepository&lt;PatientReport, String&gt; {

 public List<PatientReport> findByName(String name);

 public List<PatientReport> findByNameLike(String name);

 public List<PatientReport> findByTimeBetween(long from, long to);

}

Now let’s create the bean class, PatientReport

 

import com.fasterxml.jackson.annotation.JsonIgnoreProperties;
import org.springframework.data.annotation.Id;
import java.util.Map;

@JsonIgnoreProperties(ignoreUnknown = true)
public class PatientReport {

@Id
public String id;

public String name;
public int age;
public String sex;
public String doctorName;
public long time;
public String reportType;
public Map<String, Object> reportData;
}

Finally the service class, PatientReportService. You can define any number of methods and implement a custom logic.

import com.project.capsule.PatientReportRepository;
import com.project.capsule.bean.PatientReport;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RequestParam;

import javax.inject.Named;
import javax.ws.rs.*;
import javax.ws.rs.core.MediaType;
import javax.ws.rs.core.Response;
import java.util.*;

@Named
@Path("/report")
public class PatientReportService {

@Autowired
private PatientReportRepository repository;

@POST
@Path("")
@Consumes(MediaType.APPLICATION_JSON)
public Response storePatientReport(@RequestBody PatientReport patientReport) {
repository.save(patientReport);
return Response.status(201).build();
}

@GET
@Path("{id}")
@Produces(MediaType.APPLICATION_JSON)
public PatientReport retrievePatientReport(@PathParam("id") int id) {
PatientReport patientReport = repository.findOne(String.valueOf(id));
return patientReport;
}

@POST
@Path("find")
public List<PatientReport> findReports(@RequestParam Map<String, Object> map) {
List<PatientReport> patientReports = new ArrayList<PatientReport>();
Map<String, PatientReport> resultantMap = new HashMap<String, PatientReport>();
List<PatientReport> resultantReports;

if (map.containsKey("name") && map.get("name") != null) {
String patientName = (String) map.get("name");
if (!patientName.trim().equalsIgnoreCase("")) {
resultantReports = repository.findByNameLike(patientName);

for (PatientReport report : resultantReports)
resultantMap.put(report.id, report);

}
}
return patientReports;
}

}

Once you run the Application.java file, microservice will start from port 8080. You can change the post by giving argument “-Dserver.port=8090” etc. Thereafter you can use a REST client to send HTTP requests and see how it works!

References

[1] https://spring.io/blog/2015/07/14/microservices-with-spring

[2] http://blog.scottlogic.com/2016/11/22/spring-boot-and-mongodb.html

[3] https://dzone.com/articles/spring-boot-creating

JSON Enrich Mediator for WSO2 ESB

Introduction

JSON support for WSO2 ESB [1] was introduced sometime back. But only small number of mediators support manipulating JSON payloads. In this article I am going to introduce a new mediator called JsonEnrichMediator [2], which work quite similar to existing Enrich mediator [3], but aiming JSON payloads. The specialty of this mediator is, since this is working with native JSON payload, JSON payload will not be converted to an XML representation. Hence there won’t be any data loss due to transformations.

Please note that this is a custom mediator I have created and will not ship with WSO2 ESB pack.

Configuring Mediator

  1. Clone the GitHub repository: https://github.com/Buddhima/JsonEnrichMediator
  2. Build the repository using maven (mvn clean install)
  3. Copy the built artifact in target folder to ESB_HOME/repository/components/dropins
  4. Download json-path-2.1.0.jar [5], json-smart-2.2.jar [6] and put them to the same folder (dropins).
  5. Start WSO2 ESB (sh bin/wso2server.sh)

Sample Scenario

For this article I am using a sample scenario which moves a JSON property within the payload. For that you need to add the following API to WSO2 ESB.

<api xmlns="http://ws.apache.org/ns/synapse" name="sampleApi" context="/sample">
<resource methods="POST" uri-template="/*">
<inSequence>
<log level="full"/>
<jsonEnrich>
<source type="custom" clone="false" JSONPath="$.me.country"/>
<target type="custom" action="put" JSONPath="$" property="country"/>
</jsonEnrich>
<respond/>
</inSequence>
</resource>
</api>

The above configuration will take the value pointed by JSONPath “$.me.country” and move it to the main body. You can find further details about JSONPath at the location [4].

Once the API is deployed, you need to send following message to ESB.


curl -H "Content-Type: application/json"
-X POST -d '{
"me":{
"country": "Sri Lanka",
"language" : "Sinhala"
}
}'
http://127.0.0.1:8280/sample

The output of the ESB should look like below


{
"me": {
"language": "Sinhala"
},
"country": "Sri Lanka"
}

Conclusion

I have shown a simple use-case of using JSON Enrich Mediator. You can see the comprehensive documentation at the code repository [2].

References

[1] WSO2 ESB JSON support : https://docs.wso2.com/display/ESB500/JSON+Support

[2] Code Repository for JSON Enrich Mediator : https://github.com/Buddhima/JsonEnrichMediator

[3] WSO2 ESB Enrich Mediator : https://docs.wso2.com/display/ESB500/Enrich+Mediator

[4] JSON Path documentation : https://github.com/jayway/JsonPath/blob/json-path-2.1.0/README.md

[5] json-path-2.1.0 : https://mvnrepository.com/artifact/com.jayway.jsonpath/json-path/2.1.0

[6] json-smart-2.2.1 : https://mvnrepository.com/artifact/net.minidev/json-smart/2.2.1

WSO2 ESB Endpoint Error Handling

Introduction

WSO2 ESB can be used as an intermediary component to connect different systems. When connecting those systems the availability of those systems is a common issue. Therefore ESB has to handle those undesirable situations carefully and take relevant actions. To cater that requirement outbound-endpoints of the WSO2 ESB can be configured. In this article I discuss two common ways of configuring endpoints.

Two common approaches to configure endpoints are;

  1. Configure with just a timeout (without suspending endpoint)
  2. Configure with a suspend state

Configure with just a timeout

This would suitable if the endpoint failure is not very frequent.

Sample Configuration:

<endpoint name="SimpleTimeoutEP">
    <address uri="http://localhost:9000/StockquoteService">
    <timeout>
        <duration>2000</duration>
        <responseAction>fault</responseAction>
    </timeout>
    <suspendOnFailure>
        <errorCodes>-1</errorCodes>
        <initialDuration>0</initialDuration>
        <progressionFactor>1.0</progressionFactor>
        <maximumDuration>0</maximumDuration>
    </suspendOnFailure>
    <markForSuspension>
        <errorCodes>-1</errorCodes>
    </markForSuspension>
</address>
</endpoint>

 

In this case we only focus on the timeout of the endpoint. The endpoint will stay as Active for ever. If a response does not receive within duration, the responseAction triggers.

duration – in milliseconds

responseAction – when response comes to a time-out message one of the following actions trigger.

  • fault – calls the fault-sequence associated
  • discard – discards the response
  • none – will not take any specific action on response (default action)

The rest of the configuration avoids the endpoint going to suspend state.

If you specify responseAction as “fault”, you can define define customize way of informing the failure to the client in fault-handling sequence or store that message and retry later.

Configure with a suspend state

This approach is useful when connection failures are very often. By suspending endpoint, ESB can save resources without unnecessarily waiting for responses.

In this case endpoint goes through a state transition. The theory behind this behavior is the circuit-breaker pattern. Following are the three states:

  1. Active – Endpoint sends all requests to backend service
  2. Timeout – Endpoint starts counting failures
  3. Suspend – Endpoint limits sending requests to backend service

Sample Configuration:

<endpoint name="Suspending_EP">
    <address uri="http://localhost:9000/StockquoteServicet">
    <timeout>
        <duration>6000</duration>
    </timeout>
    <markForSuspension>
        <errorCodes>101504, 101505</errorCodes>
        <retriesBeforeSuspension>3</retriesBeforeSuspension>
        <retryDelay>1</retryDelay>
    </markForSuspension>
    <suspendOnFailure>
        <errorCodes>101500, 101501, 101506, 101507, 101508</errorCodes>
        <initialDuration>1000</initialDuration>
        <progressionFactor>2</progressionFactor>
        <maximumDuration>60000</maximumDuration>
    </suspendOnFailure>
</address>
</endpoint>

 

In the above configuration:

If endpoint error codes are 101504, 101505; endpoint is moved from active to timeout state.

When the endpoint is in timeout state, it tries 3 attempts with 1 millisecond delays.

If all those retry attempts fail, the endpoint will move to suspend state. If a retry succeed, then endpoint will move to active state.

If active endpoint receives error codes 101500, 101501, 101506, 101507, 101508; endpoint will directly move to suspend.

After endpoint somehow moves to suspend state, it waits initialDuration before attempting any furthermore. Thereafter it will determine the time period between requests according to following equation.

Min(current suspension duration * progressionFactor, maximumDuration)

In the equation, “current suspension duration” get updated for each reattempt.

Once endpoint succeed in getting a response to a request, endpoint will go back to active state.

If endpoint will get any other error codes (eg: 101503), it will not do any state transition, and remain in active state.

Conclusion

In this article I have shown two basic configurations that would be useful to configure endpoints of WSO2 ESB. You can refer WSO2 ESB documentation for implementing more complex patterns with endpoints.

References

WSO2 ESB Documentation: https://docs.wso2.com/display/ESB500/Endpoint+Error+Handling#EndpointErrorHandling-timeoutSettings

Timeout and Circuit Breaker Pattern in WSO2 Way: http://ssagara.blogspot.com/2015/05/timeout-and-circuit-breaker-pattern-in.html

Endpoint Error Codes: https://docs.wso2.com/display/ESB500/Error+Handling#ErrorHandling-codes

Endpoint Error Handling: http://wso2.com/library/articles/wso2-enterprise-service-bus-endpoint-error-handling/

Classic Mistakes in Software Development

Introduction

Software development is a complicated activity. Hence the people work in the project can do mistakes that could affect the project. Researchers have reviewed number of software projects and identified a set of mistakes that can be seen throughout projects. They have mentioned that those mistakes might not be only causes for slow development. To slip a project into slow development, all you need to do is to make one big mistake. However to achieve efficient development, you need to avoid all mistakes.

The set of mistakes that researchers have identified is known as “Classic Mistakes”. Those bad practices have been chosen so often, by so many people. And those mistakes have predictable bad-results on the development of the project.

Four categories of classic mistakes:

  1. People related
  2. Process related
  3. Product related
  4. Technology related

People related classic mistakes

This kind of mistakes talks about how to avoid mistakes among team mates. This kind of mistakes affect directly to the development speed and it is crucial to rectify those.

Undermined motivation – Studies have shown that giving suspicious talks at the beginning, asking to work overtime reduces the motivation of the people. Sometimes team leaders take long vacations while team is working overnights. The researchers highlighted that team lead has to work along with other team members is a positive motivation.

Weak personnel – If a team need an efficient development throughout the project, the recruitment needs to hire talented developers. Also carefully filter people who could do most of the work until the end of the project.

Uncontrolled problem employees – Failure to take actions for problems with team members and team leads will eventually affect the development speed. Some higher management should actively look into those and sort out.

Heroics – Heroics within the team increases the risk and discourages cooperation among the other members of the team

Adding people to a late project – Adding new people when the project is behind schedule, can take more productivity away from team members.

Noisy, crowded offices

Friction between developers and customers – Need to increase the communication between customers and developers.

Unrealistic expectations – Setting deadlines earlier without any proper reasons can lengthen the development schedule.

Process related classic mistakes

This type of mistakes talks about issues that may arise in management and technical methodologies.

Overly optimistic schedules – This sort of scheduling will result in failure by under-scoping the project and hurt long-term morale and productivity of the developers.

Insufficient risk management – If projects risks are not actively managed, the project will lead in to slow-development mode.

Contractor failure – weak relationship with contractors can lead to slow-down the project

Insufficient planning

Short-changed upstream activities – Start coding without properly design the project plans will costs 10 or 100 times than doing it with properly designed plans.

Short-changed quality assurance – Eliminating design and code reviews, eliminating test planning and do only perfunctory testing will reduce the development of the project and ends up with major bugs.

Omitting necessary tasks from estimates – People forget about the less visible tasks and those tasks add up.

Code-like-hell programming – Developers should be sufficiently motivated rather forcing them to work hard.

Product related classic mistakes

This type of mistakes talks about which can affect the outcome of the project.

Requirements gold-planting – More requirements that are not really necessary, and pay less attention on complex features

Feature creep – On average 25% of requirements can be changed and affect the project schedule.

Developer gold planting – It is frequent that developers attempt to try new technologies that they saw in other projects, which is not actually necessary.

Technology related classic mistakes

This type of mistakes is about technologies use during the project.

Silver-bullet syndrome – Thinking that certain approach will solve every issue, and that approach has not already used by developers (eg: Object oriented design)

Overestimated savings from new tools or methods – New practices will introduce a new risk as team has to go through a learning-curve to become familiar.

Switching tools in the middle of a project – Using new tools will add a learning curve, rework and inevitable mistakes to project schedule

Lack of automated source-code control – If two or more developers are working on the same part of the project, it is necessary to adhere to source-code control practices. If not developers have to spend time on resolving conflicting changes.

Conclusion

In this article I have mentioned several mistakes that could done by people during a project life time. There could be many other mistakes which can lead to slow-down a project. However at least you should avoid these well-known classic mistakes.

Java Virtual Machine

Introduction

From this post I thought of discussing how underlying components work together for a successful execution of a Java program. Content in this post consists of a collection of knowledge I gathered after going though several articles on this topic.

Most of us never bothered about internal stuffs related to Java, because IDEs have made our life a lot easier. But it’s worth if you have some idea about internal stuffs as if you happened to work with an enterprise level application, and faced with memory issues. In this article, I though of going from basic concepts to an intermediate level.

JVM, JRE, JDK

In brief “JVM (Java Virtual Machine) is an abstract machine. It is a specification that provides runtime environment in which java bytecode can be executed.” Which implies JVM is merely a concept. I’ll come back to JVM later to discuss more.

JRE (Java Runtime Environment) provides a runtime environment (implementation of JVM). JRE contains the concrete implementation of JVM (eg: HotSpot JVM, JRockit, IBM J9), set of libraries and other files need for JVM. Different vendors release their own JRE, based on a reference.

JDK is for developers to create Java program. JDK consists of JRE + development tools such as ‘javac’.

One important thing to remember is all JVM, JRE and JDK are platform-dependent.

jdk2
JVM, JRE, JDK (source: javatpoint.com)

 

JVM in Detail

It is said that JVM is;

  1. A specification where working of Java Virtual Machine is specified. But implementation provider is independent to choose the algorithm. Its implementation has been provided by Sun and other companies.
  2. An implementation Its implementation is known as JRE (Java Runtime Environment).
  3. Runtime Instance Whenever you run a java program, an instance of JVM is created.

JVM is responsible for following operations:

  • Loads code
  • Verifies code
  • Executes code
  • Provides runtime environment

Internal Architecture of JVM

jvminternal
(source: javatpoint.com)
  1. Classloader : Classloader is used to load class files. The embedded classloader in JVM is also called “primordial class loader”. Depending on the class name, classloader can search for the .class file in directory structure. User can also define classloaders (called “non-primordial class loader”) if required.
  2. Method (Class) Area : Also called non-heap area, it has 2 subsections. “Permanent Generation – This area stores class related data from class definitions, structures, methods, field, method (data and code) and constants. Can be regulated using -XX:PermSize and -XX:MaxPermSize. It can cause java.lang.OutOfMemoryError: PermGen space if it runs out if space”. OutOfMemoryError happens when class definitions are accumulated. The other section, Code Cache is used by JIT to store compiled code (hardware specific native code).
  3. Heap : Area allocated for runtime data. This area is shared by all threads. We can use  -Xms and -Xmx JVM options to tune the heap size. In most of the time “java.lang.OutOfMemoryError” error occurs because heap is getting exhausted. Heap consists of 3 sub-areas:
    1. Eden (Young) “New object or the ones with short life expectancy exist in this area and it is regulated using the -XX:NewSize and -XX:MaxNewSize parameters. GC (garbage collector) minor sweeps this space”
    2. Survivor – “The objects which are still being referenced manage to survive garbage collection in the Eden space end up in this area. This is regulated via the -XX:SurvivorRatio JVM option”
    3. Old (Tenured) – “This is for objects which survive long garbage collections in both the Eden and Survivor space (due to long-time references of course). A special garbage collector takes care of this space. Object de-alloaction in the tenured space is taken care of by GC major”

      Analysis on “java.lang.OutOfMemoryError” can be done by taking a heap-dump when the incident occours. You can refer a case-study on analyzing for a real-world case at here: http://javaeesupportpatterns.blogspot.com/2011/11/hprof-memory-leak-analysis-tutorial.html

  4. Stack : This area is specific to a thread. Each thread has it’s stack which is used to store local variables and regulates method invocation, partial result and return values. This space can be tuned with -Xss JVM option.
  5. Program Counter Register : It contains the address of the Java virtual machine instruction currently being executed.
  6. Native Stack : Used for non-Java code, per thread allocation
  7. Execution Engine : This contains 3 parts;
    1. A virtual processor
    2. Interpreter, which reads the bytecode and execute the instructions
    3. Just-In-Time(JIT) compiler: “It is used to improve the performance. JIT compiles parts of the byte code that have similar functionality at the same time, and hence reduces the amount of time needed for compilation.Here the term ‘compiler’ refers to a translator from the instruction set of a Java virtual machine (JVM) to the instruction set of a specific CPU.”

Conclusion

In this post I want to gather a colllection of knowledge I found related to JVM from various articles. All those areticles are mentioned in Reference section below. I would like to thank all those original authors, and hope you will get some knowledge out of this as well.

References

  1. JVM internals tutorial – http://www.javatpoint.com/internal-details-of-jvm
  2. About JVM Memory – https://abhirockzz.wordpress.com/2014/09/06/jvm-permgen-where-art-thou/
  3. Understanding Java PermGen – http://www.integratingstuff.com/2011/07/24/understanding-and-avoiding-the-java-permgen-space-error/
  4. Java HotSpot VM Options – http://www.oracle.com/technetwork/articles/java/vmoptions-jsp-140102.html
  5. Java Classloaders – http://www.javaworld.com/article/2077260/learn-java/learn-java-the-basics-of-java-class-loaders.html
  6. Heap Dump Analysis with VisualVM – http://www.javaworld.com/article/2072864/heap-dump-and-analysis-with-visualvm.html
  7. Real-world Example for Heap Dump Analysis with MAT – http://javaeesupportpatterns.blogspot.com/2011/11/hprof-memory-leak-analysis-tutorial.html