Trifork Blog

Category ‘Java’

API Specification first – Avoid chaos in REST endpoint development.

November 17th, 2017 by
(http://blog.trifork.com/2017/11/17/api-specification-first-avoid-chaos-in-rest-endpoint-development/)
A typical approach for API development, is to let devs develop your REST API, story-by-story, repetitive and boring to build and often leading to a non coherent API.
With an API First approach, you maintain control of endpoints created for your Resource Server.

 

In this video we demonstrate how to define an API with the Open-API specification,
generate REST controllers and DTO’s, in a Spring Boot application and finally validate the API implementation against the specification.

 

Using Axon with PostgreSQL without TOAST

October 9th, 2017 by
(http://blog.trifork.com/2017/10/09/axon-postgresql-without-toast/)

The client I work for at this time is leveraging Axon 3. The events are stored in a PostgreSQL database. PostgreSQL uses a thing called TOAST (The Oversized-Attribute Storage Technique) to store large values.

From the PostgreSQL documentation:

“PostgreSQL uses a fixed page size (commonly 8 kB), and does not allow tuples to span multiple pages. Therefore, it is not possible to store very large field values directly. To overcome this limitation, large field values are compressed and/or broken up into multiple physical rows”

As it happens, in our setup using JPA (Hibernate) to store events, the DomainEventEntry entity has a @Lob annotation on the payload and the metaData fields (via extension of the AbstractEventEntry class):

For PostgreSQL this will result in events that are not easily readable:

SELECT payload FROM domainevententry;

| payload |
| 24153   |

The data type of the payload column of the domainevententry table is OID.

The PostgreSQL JDBC driver obviously knows how to deal with this. The real content is deTOASTed lazily. Using PL/pgSQL it is possible to store a value in a file. But this needs to be done value by value. But when you are debugging your application and want a quick look at the events of your application, this is not a fun route to take.

So we wanted to change the data type in our database to something more human readable. BYTEA for example. Able to store store large values in, yet still readable. As it turned out, a couple changes are needed to get it working.

It took me a while to get all the pieces I needed. Although the solution I present here works for us, perhaps this could not be the most elegant of even the best solution for everyone.
Read the rest of this entry »

How to send your Spring Batch Job log messages to a separate file

April 14th, 2017 by
(http://blog.trifork.com/2017/04/14/how-to-send-your-spring-batch-job-log-messages-to-a-separate-file/)

In one of my current projects we’re developing a web application which also has a couple of dozen batch jobs that perform all sort of tasks at particular times. These jobs produce quite a bit of logging output when they’re run, which is important to see what has happened during a job exactly. What we noticed however, is that the batch logging would make it hard to quickly spot the other logging performed by the application while also running a batch job. In addition to that, it wasn’t always clear in the context of what job a log statement was issued.
To address these issues I came up with a simple solution based on Logback Filters, which I’ll describe in this blog.

Logback Appenders

We’re using Logback as a logging framework. Logback defines the concept of appenders: appenders are responsible for handling the actual log messages emitted by the loggers in the application by writing them to the console, to a file, to a socket, etc.
Many applications define one or more appenders and them simply list them all as part of their root logger section in the logback.xml configuration file:

<configuration scan="true">

  <appender name="LOGSTASH" class="net.logstash.logback.appender.LogstashTcpSocketAppender">
    <destination>logstash-server</destination>
    <encoder class="net.logstash.logback.encoder.LogstashEncoder"/>
  </appender>

  <appender name="FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
    <file>log/server.log</file>
    <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
      <fileNamePattern>log/server.%d{yyyy-MM-dd}.log</fileNamePattern>
      <maxHistory>30</maxHistory>
    </rollingPolicy>
    <encoder>
      <pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %mdc %-5level %logger{36} - %msg%n</pattern>
    </encoder>
  </appender>
  <root level="info">
    <appender-ref ref="LOGSTASH"/>
    <appender-ref ref="FILE"/>
  </root>

</configuration>

This setup will send all log messages to both of the configured appenders. Read the rest of this entry »

Writing less code

November 23rd, 2016 by
(http://blog.trifork.com/2016/11/23/writing-less-code/)

Have you had that feeling that you have to write too much code to build simple functionality? Some things just feel repetitive, they feel you should be not have to write them yourself, instead a framework should make your life easier.

Recently I’ve been building a project in Java/Spring, and after some time I started wondering about alternatives and how to build the same functionality with less code.

There is lots of alternative frameworks and multiple ways of building rest endpoints in Java/Spring.

  • Building the controller/service/dao layers manually in Spring ; https://spring.io/guides/tutorials/bookmarks/
  • Using spring-data-rest to export your spring-data repositories ; https://spring.io/guides/gs/accessing-data-rest/
  • Groovy/grails RestfulController ; https://examples.javacodegeeks.com/jvm-languages/groovy/grails/grails-rest-example/
  • Python/django django-rest-framework ; http://www.django-rest-framework.org/tutorial/6-viewsets-and-routers/
  • etc

Examples

Below some abbreviated examples of how a simple rest endpoint looks for each approach. To actually run the examples, you’ll need check out the tutorials mentioned earlier. My goal here is a quick comparison of how you do things in each framework.

Read the rest of this entry »

Personalised city trip itinerary using integer linear programming

February 1st, 2016 by
(http://blog.trifork.com/2016/02/01/personalised-city-trip-itinerary-using-integer-linear-programming/)

Case

As a research project I have developed an itinerary service. The idea started when I was doing a hackathon with colleagues for the city of Amsterdam (see earlier post). I wanted to recommend an itinerary to a tourist visiting the city of Amsterdam. Furthermore, I wanted to make the itinerary based on the user’s interests to recommend interesting places and activities for him in the city. If the user is interested in modern art for example, the recommendation scores for modern art museums will increase for that user.

The tool divides the duration of the tourist’s stay into separate time slots. For example, a single day could be divided in 3 time slots: morning, afternoon, evening. POIs get a different recommendation score for each time slot they can be visited. The Vondelpark for example could be less recommended on Monday morning because of expected rain or because of expected crowds. On Tuesday morning the Vondelpark could be recommended because of an interesting event or nice weather.

The itinerary tool will try to limit travel time between the recommended POIs (point of interests). In this way the tourist will not waste time on travailing. The user can also set a budget for the entire itinerary.

By taking all these considerations into account the tool should be able to aid the user in making good decisions about which places to visit and when to visit them.

Here is a screenshot of the user interface of the itinerary tool that was created as part of the research project:

image01

Read the rest of this entry »

Controlling Java with the Leap Motion

November 17th, 2015 by
(http://blog.trifork.com/2015/11/17/controlling-java-with-the-leap-motion/)

Leap Motion Controller

The Leap Motion Controller is a device that uses two cameras to track the hands and fingers. This makes it possible to use gestures for controlling the computer or applications. It is possible to buy or download applications through the Leap Motion app store, but there is also an SDK for different languages available to integrate the controller in your own application.

With this article I aim to give an insight in the usability of the Leap Motion Controller in combination with Java. For this I describe the controller and Java API itself and have written an example application which uses the controller. The application is written in Java and is available on github.

Functionality of the controller

The basic functionality for the controller and API is working without problems. This makes it possible to make the interaction with devices and computers more intuitive. In the next screenshot an example is shown from the supplied Visualizer application with the detected hands and fingers.
Read the rest of this entry »

City-wide crowd management in Amsterdam

November 10th, 2015 by
(http://blog.trifork.com/2015/11/10/city-wide-crowd-management-in-amsterdam/)

As most residents and visitors of Amsterdam know, every year more people are visiting Amsterdam, city wide events like GayPride, Koningsdag and MuseumNacht are getting bigger and more frequent, putting more strain on the city’s infrastructure and all people living in the city center.

That’s why this November 7th, Amsterdam Marketing organized the Museumn8 hackathon to allow developers to come up with creative and innovative solutions for improving improving mobility, navigation and crowd management in the city. Twenty teams eventually participated.

Trifork (Rienk Prinsen, Marleine van Kampen, Marijn van Zelst) and weCity (David Kat, Luc Deliance) teamed up and joined the hackathon to give their take on solving this problem. Their solution:

city-live-logo

By transforming the advertisement billboards of JCDecaux into large information screens displaying real-time information, visitors can get informed about activities and interesting places in the vicinity of the billboard. They receive live crowd information, travel times to and queue lengths at museums and even recommendations where to go next. Read the rest of this entry »

Recognizing commercials using the Alphonso API

September 21st, 2015 by
(http://blog.trifork.com/2015/09/21/recognizing-commercials-using-the-alphonso-api-2/)

Liberty Global organized the Hack & Play Appathon in Ziggo dome on September 15th and 16th. More than 20 teams of hackers, designers and programmers were invited to create an app or a game for the Liberty Global product Horizon set-top box. Team Trifork joined with Dennis de Goede (Design & Frontend), Tony Abidi (Devops) and myself (Front & Backend).

Alphonso added another challenge to the appathon: Create the best integration with the Alphonso platform. Integration challenge? Sounds like a Trifork challenge to me.

FirstPrice

 

Read the rest of this entry »

A wrinkle in time

June 30th, 2015 by
(http://blog.trifork.com/2015/06/30/a-wrinkle-in-time/)

Leap second

I clearly remember the morning of Sunday July 1, 2012, almost three years ago. I was at church, actually, when I got a call from one of our clients: “The website doesn’t seem to be working.” All I could check at that point was that, indeed, the website was not responding. So I called our sysadmin, who found that even SSH-ing into the machine running the site was taking much longer than usual. Finally, restarting everything solved the problem, but we were still unsure about what had happened.

As we found out later, it was related to the leap second, a phenomenon I had not even heard of until then. Read the rest of this entry »

Documenting your REST APIs

May 8th, 2015 by
(http://blog.trifork.com/2015/05/08/documenting-your-rest-apis/)

Whenever you deliver some API that is to be consumed by another party, you will get the inevitable question of providing documentation. Probably every developer’s least favorite task.

In Java there is javadoc, but that doesn’t cut it if you are delivering a Web Service API. In that realm we already know WSDL for SOAP based Web Services. Then again, every developer seems to prefer REST based Web Services these days and those are not WSDL based… So what then? That is a question with multiple answers. In the last few years there have been three different open-source projects that have tried to give an answer to this: swagger, RAML and API BluePrint. Of those Swagger has been around the longest and arguably gained the largest following.

Based on the completely subjective criteria ‘it needs to support Java’, ‘what about Spring MVC?’ and ‘can you deliver it to the customer by Monday?’ I decided to take a stab at documenting an existing API using Swagger. It is written in something that can run on a JVM, has Spring MVC support (via third party libraries) and seemed to be the easiest to set up based on their examples and various other online resources.

Read the rest of this entry »