Company

Category ‘Java’

Introducing a Query tool as an Elasticsearch plugin (part 1)

March 12th, 2013 by
(http://blog.trifork.com/2013/03/12/introducing-a-query-tool-as-an-elasticsearch-plugin-part-1/)

Es

In the past few weeks I have been working with Elasticsearch. I was missing a plugin to look at the data, create queries and evaluate different facets. That was when I decided to start working on a plugin that enables you to do just this.

I have been working on AngularJS together with Twitter Bootstrap, therefore the choice as to which technology to use was not a difficult one. I also used some additional libraries, but I'll tell you more on this later on.

Why did I put the part 1 in the title? I am going to split this information into two parts. This part deals with the setup of the plugin, the libraries I used and the functionality I implemented. Then the next part deals with providing more details about interacting with Elasticsearch; how I use the facets, create the queries, etc. I will also write down some lessons learned with respect to AngularJS in a later blog post.

Now let's move on and have a look at the features:

Read the rest of this entry »

Searching with the elasticshell

March 7th, 2013 by
(http://blog.trifork.com/2013/03/07/searching-with-the-elasticshell/)

elasticshell
So as promised here is a sequel to my previous post Introducing the elasticshell. Let's start exactly where we left off...

What about search?
We of course need to search against the created index. We can provide queries as either json documents or Java QueryBuilders provided with the elasticsearch Java API, which are exposed to the shell as they are.
Read the rest of this entry »

Introducing the elasticshell

March 6th, 2013 by
(http://blog.trifork.com/2013/03/06/introducing-the-elasticshell/)

elasticshell
A few days ago I released the first beta version of the elasticshell, a shell for elasticsearch. The idea I had was to create a command line tool that allows you to easily interact with elasticsearch.

Isn't elasticsearch easy enough already?
I really do think elasticsearch is already great and really easy to use. However, on the other hand there is quite some API available and quite some json involved too. Also, interacting with REST APIs requires a tool other than the browser to use the proper http methods and so on. There are different solutions available: some of them are generic, like curl or browser plugins, while others are elasticsearch plugins like head or sense, that you can use to send json requests and see the result, still in json format. What was missing is a command line tool, something that plays the role of the mongo shell in the elasticsearch world. That's ambitious, isn't it?

In the meantime the es2unix tool has been released by Drew, a member of the elasticsearch team. The interesting approach taken there is to hide all the json and show only text in a nice tabular format, providing an executable command that makes possible to pipe its output to other unix commands like grep, sort and awk. That's a great idea, and an even greater result I must say.

A json friendly environment
I decided to take another approach: provide an environment that makes it easier to play around with all that json. That's why I started writing a javascript shell, where json is native and it's relatively easy to provide auto-suggestions directly within json objects. I also wanted to use the elasticsearch Java API, which are complete, performant, and powerful, allowing to even fire a new node if needed.
Read the rest of this entry »

Axon Framework 2.0 Released!

January 22nd, 2013 by
(http://blog.trifork.com/2013/01/22/axon-framework-2-0-released/)

After laying the ground work for this release about a year ago, we now proudly announce the next major release of Axon Framework! The 2.0 release is a big step forward compared to the previous version, as it contains a thorough restructuring of components, making it even easier to build scalable and extensible applications.
Read the rest of this entry »

How to write an elasticsearch river plugin

January 10th, 2013 by
(http://blog.trifork.com/2013/01/10/how-to-write-an-elasticsearch-river-plugin/)

Up until now I told you why I think elasticsearch is so cool and how you can use it combined with Spring. It’s now time to get to something a little more technical. For example, once you have a search engine running you need to index data; when it comes to indexing data you usually need to choose between the push and the pull approach. This blog entry will detail these approaches and goes into writing a river plugin for elasticsearch.

Read the rest of this entry »

Properly testing Spring MVC controllers

December 11th, 2012 by
(http://blog.trifork.com/2012/12/11/properly-testing-spring-mvc-controllers/)

In this post I want to introduce you to the Spring MVC testing framework, a way to properly test your Spring MVC controllers.

The way I used to do it

I have always found unit testing Spring MVC controllers nearly useless because I felt they don't test anything useful for the most part. If something would break in the implementation, those unit tests would rarely fail. To explain what I mean by this, let me give you an example. Say you want to unit test the following controller method:

@RequestMapping(value = "/people/{groep}", method = RequestMethod.GET)
public String listPeopleInGroup(@PathVariable String group, ModelMap modelMap) {
    List<Person> people = peopleService.listPeople(group);
    modelMap.put("people", people);
    return "peopleList";
}

All this controller method does is take the "group" parameter from the URL and list all people who are in that group. But it has a bug in it I didn't notice. Usually you would unit test this controller by just mocking the peopleService using EasyMock or Mockito, pass in a fake Model and off you go! This is what the unit test in that case typically would look like (using JUnit, Mockito and Hamcrest):

public class PeopleControllerTest {

    @InjectMocks
    PeopleController controller;

    @Mock
    PeopleService mockPeopleService;

    @Before
    public void setUp() throws Exception {
        MockitoAnnotations.initMocks(this);
    }

    @Test
    public void testListPeopleInGroup() {
        List<Person> expectedPeople = asList(new Person());
        when(mockPeopleService.listPeople("group")).thenReturn(expectedPeople);

        ModelMap modelMap = new ModelMap();
        String viewName = controller.listPeopleInGroup("group", modelMap);

        assertEquals("peopleList", viewName);
        assertThat(modelMap, hasEntry("people", (Object) expectedPeople));
    }
}

Let's run this unit test ...


Green light! Wow that's amazing, this controller works like a charm! But oh wait, let me try to run this in a browser now.

What did we test?

Hmm, ok something obviously went wrong here. Why did the unit test not cover this? Well, as I mentioned before: it's useless! Because what did it test exactly?

  • A call to the peopleService is made
  • The return value from the peopleService is put inside the ModelMap
  • The correct view name is returned

What didn't we test?

Some examples of things the unit test didn't cover:

  • That it responds to the correct url
  • That the PathVariable correctly takes the value from the "{group}" part in the url
  • That the method accepts GET requests
  • That any class-level annotations are also working as you expect
  • That any handler interceptors are called before and/or after executing this method
  • That any other methods in the controller are called beforehand, like those
    annotated with @ModelAttribute or @InitBinder.
  • That binding from parameters to an object works the way you expect
  • That any validation errors were registered by validators

Introducing the Spring MVC test framework

Recently a colleague of mine pointed me towards a little framework created as part of the Spring Framework called spring-test-mvc. This framework makes unit testing controllers a lot more meaningful! It actually has the ability to test all the aspects of a controller method I couldn't test before. Let's see if I can reproduce the above bug using Spring Test MVC.

public class PeopleControllerTest {

    @InjectMocks
    PeopleController controller;

    @Mock
    PeopleService mockPeopleService;

    @Mock
    View mockView;

    MockMvc mockMvc;

    @Before
    public void setUp() throws Exception {
        MockitoAnnotations.initMocks(this);
        mockMvc = standaloneSetup(controller)
                .setSingleView(mockView)
                .build();
    }

    @Test
    public void testListPeopleInGroup() throws Exception {
        List<Person> expectedPeople = asList(new Person());
        when(mockPeopleService.listPeople("someGroup")).thenReturn(expectedPeople);

        mockMvc.perform(get("/people/someGroup"))
                .andExpect(status().isOk())
                .andExpect(model().attribute("people", expectedPeople))
                .andExpect(view().name("peopleList"));
    }
}

The above unit test almost needs no explanation. It reads like reading a book. We perform a get request, check if the status is OK (status 200), check if a model attribute exists named "people" and check if the view name is "peopleList". Now let's run this unit test ...

Yes we did it! We are now essentially seeing the same error message as we saw in the browser. So what mistake could I have made in my controller method that a 'normal' unit test failed to cover? Let's take another quick look at the code.

@RequestMapping(value = "/people/{groep}", method = RequestMethod.GET)
public String listPeopleInGroup(@PathVariable String group, ModelMap modelMap)

Oh yes, of course! I made a typo in the path variable inside the url string. Of course "groep" doesn't match the method argument named "group" so the request fails. Let's correct my mistake.

@RequestMapping(value = "/people/{group}", method = RequestMethod.GET)
public String listPeopleInGroup(@PathVariable String group, ModelMap modelMap)

Run it again ...

This proves my point. A unit test written using Spring Test MVC closely resembles a real request made by a browser. That's why this test is a lot more meaningful. The fact that this test is now passing makes it very likely that it will work in a browser as well. The fact that the other unit test passed, didn't tell us anything.

Testing a REST interface

Spring Test MVC is especially useful when building a REST interface which for example returns JSON responses. The framework contains all sorts of stuff for easily building a test request and carefully examining the response.

Let's say we have a controller that listens to the URL "/people" and adds a person to the database whenever a POST request was received on that URL. The request body would be a JSON representation of the person to add. The response body will hold JSON that tells us what the database identifier of the person is that was just added, and also gives us a list of all people that have been added so far. This is how you would test this:

mockMvc.perform(post("/people")
        .contentType(MediaType.APPLICATION_JSON)
        .body("{\"firstName\":\"Tom\", \"lastName\":\"van Zummeren\"}".getBytes()))
        .andExpect(status().isCreated())
        .andExpect(jsonPath("$.identifier", equalTo("123")))
        .andExpect(jsonPath("$.allPeople[*].firstName", hasItem("Tom")));

verify(mockPeopleService).persistPerson(new Person("Tom", "van Zummeren"));

In this example we see a few new things. The content type is set on the request, a request body is given, and the response is checked using an expression language provided by the "json-path" framework.

This test tests so many things now. It's not just calling the controller method, it's also testing:

  • The mapping of the request body to a Person object
  • The status code that was set on the response
  • That it supports the given content type
  • That it listens to a POST on the /people URL
  • The mapping of the response object is transformed correctly to JSON

Conclusion

Spring Test MVC is indispensable if you want to test your Spring MVC controllers. Simply testing the controller methods without including the Spring MVC framework itself, is useless. Spring Test MVC will be included in the Spring 3.2 release (so I'm told) but for now it can be found on Github: https://github.com/SpringSource/spring-test-mvc

Axon Framework 2.0-rc1 released

November 30th, 2012 by
(http://blog.trifork.com/2012/11/30/axon-rc1-released/)

Axon 2 has hit an important milestone today, as we have published the first Release Candidate. This release marks a major milestone towards the final 2.0 release, expected early January. All features on the roadmap have been implemented in this release. The coming weeks, these features will be refined and fine-tuned for optimal performance and reliability to ensure Axon 2.0 helps meet the demands applications face today.

Read the rest of this entry »

Basic Axon Framework sample using vert.x and angular.js

November 27th, 2012 by
(http://blog.trifork.com/2012/11/27/basic-axon-framework-sample-using-vert-x-and-angular-js/)

Some people on the Axon Framework mailing list were asking for more basic samples, preferably using something like vert.x. Since I am familiar with Axon as well as vert.x I thought I could help out. Therefore I have created a very basic application based on vert.x using the Axon Framework. The application is all about maintaining a list of TODO items. These items are shared between all connected web browsers. You have to be able to create a new TODO item and you have to be able to mark them as completed.

During my visit of Devoxx I learned about a front-end framework called AngularJS. This is a very nice addition to the vert.x technology stack. Since I always use Bootstrap for my front-end work, the front-end should look familiar to a lot of you. The following images shows the end result of the application. If you want to know how I created the sample, read on.

ScreenShotAxonTodoList Read the rest of this entry »

And that was Devoxx 2012, partnering with 10gen and a lot of knowledge gaining

November 17th, 2012 by
(http://blog.trifork.com/2012/11/17/and-that-was-devoxx-2012-partnering-with-10gen-and-a-lot-of-knowledge-gaining/)

Last week a couple of my colleagues and I were in Antwerp visiting the Devoxx conference. In this blog post we try to give you an idea about what we did & learned:

The sessions

I attended a lot of sessions, if you want to know more I urge you to read on. I briefly discuss the following talks:

  • Keynote: Geek leaks
  • New features in Java 8
  • Security and http headers
  • Developer tools in google chrome
  • OAuth2
  • Skaling software with akka
  • Last but not least: AngularJS

Read the rest of this entry »

Spring Insight plugin for the Axon CQRS framework

November 13th, 2012 by
(http://blog.trifork.com/2012/11/13/axon-insight-plugin/)

Introduction

In a previous blog post we introduced the Spring Insight module that's part of SpringSource's tc Server developer edition and gives you, well, insight into what's taking up the time to process requests in your applications. Insight does this by instrumenting your application at load time using AspectJ: a single call to your application will result in a so-called trace consisting of various operations, and for each of these operations it's measured how long it takes to execute. An operation can provide additional details: for example, when sending a query to a database using a JDBC Connection the actual SQL statement including prepared statement bindings will be stored in the operation.

Spring Insight is not a profiler: it doesn't create operations for each and every little thing that happens, but only for 'significant' events, like a Spring-MVC controller invocation, the start of a transaction or communication with a database. These events are called 'operations'. It does allow assigning operations to so-called endpoints. An endpoint could be a controller method, or a method on a class annotated with one of Spring's stereotype annotations like @Service or @Repository for example. For these endpoints Insight can provide statistics across traces, so that you can measure the health of these endpoints during the lifespan of your application based on configurable thresholds.

Insight's plugin model

Spring Insight consists of a core plus a set of plugins that are shipped with the tool. These plugins define the actual operations and endpoints that Insight knows about and exposes. One of the nice things about Spring Insight is that the plugins use a public API that's not restricted to the built-in plugins: you can build your own plugins to teach the tool new tricks.

Although you could do this on a per-application basis to expose metrics relevant to your particular app, you wouldn't usually write a dedicated plugin for that. Insight already exposes several application-level methods as operations if they're part of your stereotype-annotated Spring beans, and you can use their set of annotations to expose additional application-specific operations and endpoints easily .

Plugins are much more useful for framework developers: their framework might contain several methods that would be interesting to expose as Insight operations or even endpoints to show users what the framework is doing and how long that takes. Earlier this year, that's exactly what we did for the Axon CQRS framework that's being developed within Trifork.

This blog briefly discusses the plugin's implementation. All source code has been added to the Axon Framework itself and is available on Github.

Update:
After publishing this blog entry, VMware has contacted  us to host the source code for this plugin in their public GitHub repository, so that the plugin can be shipped out-of-the-box with the Spring Insight distribution. That means that the plugin sources are no longer found under the Axon repository. The link above has been updated to reflect this change.

Read the rest of this entry »