Testing SSL (HTTPS) clients with Junit and Jetty

July 14th, 2010

I recently ran into a situation where a web service client had issues when invoking a webservice via HTTPS. To reproduce, I needed a lightweight, JUnit-based test to reproduce the problem and write a regression test. Here is how I got it to work:

First, I used the basic HTTP test setup. Now however, I needed Jetty to provide a HTTPS connector. For this to work, Jetty – just like any other webserver, such as apache – needs a server certificate and a private key in a keystore. To create the keystore, I simply followed the excellent Jetty SSL guide. Namingly, I did this (using Ubuntu, see the above link for alternatives):

openssl genrsa -des3 -out jetty.key
openssl req -new -x509 -key jetty.key -out jetty.crt
openssl pkcs12 -inkey jetty.key -in jetty.crt -export -out jetty.pkcs12
$JAVA_HOME/bin/keytool -importkeystore -srckeystore jetty.pkcs12 -srcstoretype PKCS12 -destkeystore keystore

Once I had the keystore set up, I placed it in the src/main/resources folder (the root of the classpath, in case you’re not a maven user) and extended the HttpTestServer like so:

import java.net.URL;

import org.junit.Ignore;
import org.mortbay.jetty.Connector;
import org.mortbay.jetty.security.SslSocketConnector;

/**
 * A test server for testing SSL requests.
 *
 * @author Olaf Otto
 */
@Ignore
public class SslTestServer extends HttpTestServer {
	private static final int HTTPS_PORT = 50443;

	public SslTestServer() {
	}

	public SslTestServer(String mockData) {
		super(mockData);
	}

	@Override
	protected void configureServer() {
		super.configureServer();
		Connector secureConnector = createSecureConnector();
		getServer().addConnector(secureConnector);
	}

	private Connector createSecureConnector() {
		SslSocketConnector connector = new SslSocketConnector();
		connector.setPort(HTTPS_PORT);
		URL keystoreUrl = getClass().getClassLoader().getResource("keystore");
		connector.setKeystore(keystoreUrl.getFile());
		connector.setKeyPassword("test");
		return connector;
	}

	public static void main(String[] args) {
		SslTestServer server = new SslTestServer();
		try {
			server.start();
		} catch (Exception e) {
			throw new RuntimeException(e);
		}
	}
}

(I added the @Ignore annotation as I placed the class in the test sources folder, though it’s not a test…)
You might use it in your JUnit-based test like so:

import org.junit.After;
import org.junit.Before;
import org.junit.Test;

/**
 * @author Olaf Otto
 */
public class MySslTest {
	private SslTestServer _server;

	@Before
	public void startServer() throws Exception {
		_server = new SslTestServer();
		_server.start();
	}

	@After
	public void stopServer() throws Exception {
		_server.stop();
	}

	@Test
	public void mySsltest() {
		_server.setMockResponseData("<xml>My client expects this response</xml>");
		// Perform tests on port 50443...
	}
}

Here is the whole package as a maven 2 project (including the keystore with a self-signed certificate). Download, unzip, and give it a try (mvn test).

No Comments Posted in: , , , J2EE, java, maven 2, open source

Lightweight testing of (webservice) HTTP clients with JUnit and Jetty

July 14th, 2010

If you’re working with webservice clients you will certainly have noticed the complexity of integration-testing your webservice clients. Building webservice clients can already be quite a complex task, but providing a mock-up webservice backend delivering useful test responses is quite often just to much work, if not impossible, since many web service backends are very complex constructs. Furthermore, setting up a backend for each integration test consumes valuable build time.

As an alternative, one may simply replace the backend with a mock HTTP server that does one thing: deliver the expected webserver response – i.e. XML data – when called by the client. All you need to set this up is a HTTP server and a pre-recorded service response (quite often this comes with WS client specifications). If you don’t have this data, you can record it, for example using a proxy server in between your WS client and a real WS backend, or a tool such as wireshark.

The recorded XML response thus represents your assumptions against which you test the client. This makes your tests fast, lightweight and a lot better to understand.

Here is my Jetty-based test HTTP server:

package de.sysbsb.test;

import static javax.servlet.http.HttpServletResponse.SC_OK;
import static org.apache.commons.io.IOUtils.write;

import java.io.IOException;

import javax.servlet.ServletException;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

import org.apache.commons.io.IOUtils;
import org.junit.Ignore;
import org.mortbay.jetty.Handler;
import org.mortbay.jetty.HttpConnection;
import org.mortbay.jetty.Request;
import org.mortbay.jetty.Server;
import org.mortbay.jetty.handler.AbstractHandler;

/**
 * A server for answering HTTP requests with test response data.
 *
 * @author Olaf Otto
 */
@Ignore
public class HttpTestServer {
	public static final int HTTP_PORT = 50036;

	private Server _server;
	private String _responseBody;
	private String _requestBody;
	private String _mockResponseData;

	public HttpTestServer() {
	}

	public HttpTestServer(String mockData) {
		setMockResponseData(mockData);
	}

	public void start() throws Exception {
		configureServer();
		startServer();
	}

	private void startServer() throws Exception {
		_server.start();
	}

	protected void configureServer() {
		_server = new Server(HTTP_PORT);
		_server.setHandler(getMockHandler());
	}

	/**
	 * Creates an {@link AbstractHandler handler} returning an arbitrary String as a response.
	 *
	 * @return never <code>null</code>.
	 */
	public Handler getMockHandler() {
		Handler handler = new AbstractHandler() {

			public void handle(String target, HttpServletRequest request, HttpServletResponse response, int dispatch) throws IOException, ServletException {
				Request baseRequest = request instanceof Request ? (Request) request : HttpConnection.getCurrentConnection().getRequest();
				setResponseBody(getMockResponseData());
				setRequestBody(IOUtils.toString(baseRequest.getInputStream()));
				response.setStatus(SC_OK);
				response.setContentType("text/xml;charset=utf-8");
				write(getResponseBody(), response.getOutputStream());
				baseRequest.setHandled(true);
			}
		};
		return handler;
	}

	public void stop() throws Exception {
		_server.stop();
	}

	public void setResponseBody(String responseBody) {
		_responseBody = responseBody;
	}

	public String getResponseBody() {
		return _responseBody;
	}

	public void setRequestBody(String requestBody) {
		_requestBody = requestBody;
	}

	public String getRequestBody() {
		return _requestBody;
	}

	public static void main(String[] args) {
		HttpTestServer server = new HttpTestServer();
		try {
			server.start();
		} catch (Exception e) {
			throw new RuntimeException(e);
		}
	}

	public void setMockResponseData(String mockResponseData) {
		_mockResponseData = mockResponseData;
	}

	public String getMockResponseData() {
		return _mockResponseData;
	}

	protected Server getServer() {
		return _server;
	}
}

You can use it in a JUnit test like so:

package de.sysbsb.test;

import java.io.IOException;
import java.io.InputStream;

import org.apache.commons.io.IOUtils;
import org.junit.After;
import org.junit.Before;
import org.junit.Test;

/**
 * @author Olaf Otto
 */
public class MyClientTest {
	private HttpTestServer _server;

	@Before
	public void startTestServer() throws Exception {
		_server = new HttpTestServer();
		_server.start();
	}

	@After
	public void stopTestServer() throws Exception {
		_server.stop();
	}

	@Test
	public void someClientTest() throws IOException {
		_server.setMockResponseData("<xml><testdata>hello, world</testdata></xml>");
		// or as a much more elegant alternative:
		_server.setMockResponseData(getRecordedResponse("testdata.xml"));
		// run your client tests, using the HttpTestServer.HTTP_PORT
	}

	private String getRecordedResponse(String recordedResponseFile) throws IOException {
		InputStream testDataStream = getClass().getClassLoader().getResourceAsStream(recordedResponseFile);
		return IOUtils.toString(testDataStream);
	}
}

Here are is the HTTP test server and a JUnit test example as a maven 2 project.

No Comments Posted in: , , , J2EE, java, maven 2, open source

Solved: Xserver lock-ups in Ubuntu 10.04 with intel GFX

June 23rd, 2010

After installing 10.04 I experienced a GDM lockup a couple of times a day – the symptoms varied from the window management not reacting to left mouse clicks to a complete freeze of the Xserver. This bug has caused a lot of noise across distributions, for instance in 590109 and 538563. The symptoms are log messages such as


[mi] EQ overflowing. The server is probably stuck in an infinite loop.
...
Backtrace:
...
... /usr/lib/xorg/modules/drivers/intel_drv.so
...

in the :0… logfiles in /var/log/gdm when the problem occurs.

This is, however, not a kernel, XServer or GDM issue, but caused by a problem with the intel-linux driver. The problem is apparently fixed as of version 2:2.11.0-1ubuntu1 of the xserver-xorg-video-intel driver. Since this is not (yet) an update in the official distribution repo, I did the following to solve the issue:

1: Optional: Update to the latest stable kernel

Reason: IMO, upstream Intel drivers are usually build against the latest stable kernel

Goto http://kernel.ubuntu.com/~kernel-ppa/mainline/v2.6.34-lucid/ and download the headers… _all.deb, headers-generic …..deb and linux-image… .deb for your platform (in most cases, i368). install the generic headers package, then the all headers package, then the image.

A word of warning: If you install this custom kernel you will not receive security updates for it from the automatic distribution updates.

2: Add the X Updates PPA to your sources.list

In a terminal, type:


sudo gedit /etc/apt/sources.list

And add at the bottom of the file (if not already present):


deb http://ppa.launchpad.net/ubuntu-x-swat/x-updates/ubuntu lucid main #X-Updates PPA
deb-src http://ppa.launchpad.net/ubuntu-x-swat/x-updates/ubuntu lucid main #X-Updates PPA

2.1: Trust the software packages from the x-updates PPA

sudo apt-key adv –recv-keys –keyserver keyserver.ubuntu.com AF1CDFA9

3: Update the driver


sudo apt-get update
sudo apt-get dist-upgrade

Reboot – the problem should be solved.

No Comments Posted in: , , System engineering, open source, ubuntu

Creating HDR images in Ubuntu with Luminance (QTPFSGUI)

February 6th, 2010

For those who are not (yet) in possession one of the latest digital cameras with build-in HDR capabilities or the more experimental folks, there is a fantastic free tool to create HDR images by combining multiple shots of the same scene: Its called qtpfsgui.

That is, it used to be called that and is still available in Ubuntu under this name (sudo apt-get install qtpfsgui). However, qtpfsgui is discontinued since mid 2009 and has been replaced by the Luminance HDR project.

Since qtpfsgui crashes under ubuntu 9.10 when attempting to save any HDR image, I downloaded the latest luminance version, compiled and installed it like so:

  1. Download the luminance source and unpack the folder.
  2. Install the dependencies required to compile luminance:
    sudo apt-get install qt4-qmake libexiv2-dev libopenexr-dev fftw3-dev libtiff4-dev libqt4-dev g++ libgsl0-dev
  3. Compile luminance (takes a few minues) and install it. Change to the unpacked luminance folder and do:
    qmake
    make
    sudo make install

That´s it! you now have luminance installed and it should be in your main menu under applications>graphics.

It works perfectly with Karmic.
One of the best things about luminance is that you can play a lot with the algorithms and parameters used to tonemap the HDR into a LDR image. In contrast to many other (commercial) tools, you actually get to know which algorithms are used, who created them and can read a little more on how they work (if you are not to opposed to mathematics) by googling up the corresponding papers.

No Comments Posted in: , , open source, ubuntu

Per-assembly filtering with the maven-assembly-plugin

January 18th, 2010

The maven assembly plugin is an extremely powerful tool when it comes to creating custom distributions (aka assemblies) of your artifacts for individual platforms etc. However, the ability to create multiple variants of an artifact within a single build conflicts with the standard maven approach of using multiple build profiles and executing a single build for each profile to generate artifact variants.

This also means that mavens resource filtering is not very useful for individual assemblies, since you can only replace placeholders such as ${someValue} with the same value for all assemblies (since resources are only filtered once, using the active profile(s), for all assemblies).

You can, however, configure the maven assembly plugin to use individual filter property files for each assembly using the <execution> section of the plugin configuration, like so:

<plugin>
  <artifactId>maven-assembly-plugin</artifactId>

  <version>2.2-beta-5</version>
  <executions>
    <execution>
      <id>production</id>
      <goals>

        <goal>single</goal>
      </goals>
      <configuration>
        <filters>
          <filter>${project.basedir}/src/main/assembly/production.properties</filter>

        </filters>
        <finalName>${artifactId}-production-${version}</finalName>
        <ignoreDirFormatExtensions>true</ignoreDirFormatExtensions>
        <descriptors>
          <descriptor>src/main/assembly/assembly.xml</descriptor>

        </descriptors>
      </configuration>
      <phase>package</phase>
    </execution>
    <execution>

      <id>integration</id>
      <goals>
        <goal>single</goal>
      </goals>
      <configuration>

        <filters>
          <filter>${project.basedir}/src/main/assembly/integration.properties</filter>
        </filters>
        <finalName>${artifactId}-integration-${version}</finalName>
        <ignoreDirFormatExtensions>true</ignoreDirFormatExtensions>

        <descriptors>
          <descriptor>src/main/assembly/assembly.xml</descriptor>
        </descriptors>
      </configuration>
      <phase>package</phase>

    </execution>
  </executions>
</plugin>

This snippet uses the same assembly descriptor (you can use any assembly descriptor, actually) to build individual artifacts for production and integration. Note that the ${project.basedir}/ prefix is a workaround for MASSEMBLY-150 and is nedded to avoid a nasty Failed to create assembly: Error filtering file ... Error loading property file 'src/main/...' error when executing the mojo from outside the module directory (say, for instance, in a mvn release:prepare…).

Filtering must be enabled in the assembly descriptor, like so:

<?xml version="1.0" encoding="UTF-8"?>
<assembly xmlns="http://maven.apache.org/xsd/assembly"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

  xsi:schemaLocation="http://maven.apache.org/xsd/assembly http://maven.apache.org/xsd/assembly-1.1.1.xsd">
  ...
  <fileSets>
    <fileSet>
      <filtered>true</filtered>
      ...
    </fileSet>

  </fileSets>
</assembly>

Thanks to John Casey for suggesting this in MASSEMBLY-430!

No Comments Posted in: maven 2, open source

Obtaining the default charset (aka platform encoding) in JAVA

November 9th, 2009

… is really simple since JDK 1.5:

java.nio.charset.Charset.defaultCharset()

So please don’t use messy, old workarounds such as this:

byte [] byteArray = {'a'};
InputStream inputStream = new ByteArrayInputStream(byteArray);
InputStreamReader reader = new InputStreamReader(inputStream);
String defaultEncoding = reader.getEncoding();

Which, unfortunately, is one of the first things you stumble upon when searching for this topic.

No Comments Posted in: , java, open source

Intel linux drivers and new kernel gfx support reach mature state

October 22nd, 2009

The Linux Kernel and the XORG video rendering have been undergoing some significant improvements in the last year, with Intel’s linux open source team bringing in a lot of refactorings an architectural improvements.

I myself suffered from the lack of support for recent integrated intel gfx cards in notebooks and thus followed the excellent Intel Linux graphics performance guide using the bleeding-edge configuration, i.e. with the latest (non-stable) builds of the xorg / intel gfx drivers and the most recent kernels. However, this configuration was (not quite unexpected) somewhat unstable and had a lot of issues.

This phase is now over. I am happy to say that after upgrading to Kernel version 2.6.32 and the xserver-xorg-video-intel driver 2:2.9.0-1ubuntu2~xup~3 the graphics support is now fast, reliable and stable. Desktop effects are back working like a charm, also in a multi-monitor setup and with sending the computer to hibernation and so forth.

I am quite optimistic that this state might make it into the upcoming Ubuntu release (Karmic), thus eliminating a lot of frustration laptop users have been experiencing with their intel gfx cards.

No Comments Posted in: , , ubuntu

Using RPC-style, encoded Axis 1 webservices with spring-remoting

October 12th, 2009

Preamble

When working with enterprise integration you will quite often deal with legacy systems integration.

I recently had the case where integration of webservices was requested, and much to my surprise there are huge issues when attempting to integrate old webservices (as in pre axis2 and pre jax-ws).

These webservices use outdated or unsupported methods, such as rpc-style/encoded communication, or even worce, outdated elements in the XML messages, for instance <multiref>’s. Having to integrate such services does of course imply that you cannot change anything on the server side, and thus such an outdated format must still be consumed.
This is a huge problem, since current WS implementations, such as the popular axis2 framework, do simply not support these formats.

What surprised me the most about this is that webservices are designed to make systems independent by defining a common communication and data format, which is the exact opposite of what is going on here – and these formats are not that old, we are probably talking 4-5 years.

Developers facing these problems are taking rather desperate measures to work around these problems, such as using on-the-fly XSLT transformation to convert incoming and outgoing WS messages. However, this binds your application even stronger to the outdated format and specific service data.

In my case I wanted to use Spring-remoting. Since the service was RPC-style, I wanted to use the JaxRpcPortProxyFactoryBean to create an on-the-fly proxy implementing the service interface. What I did not want to do is having to write any additional code to consume the service. Furthermore, I wanted the complex objects transferred by the services to be represented by standard JAVA beans, and not be generated using the wsdl2java axis1 tools. Here is how I got this to work.
Read the rest of “Using RPC-style, encoded Axis 1 webservices with spring-remoting”

1 Comment Posted in: , , J2EE, System architecture, java, open source

Using apache to redirect from the root “/” context to a webapp context

October 12th, 2009

Just a short note:

The best way of doing this avoiding endless recursion is to use apaches RedirectMatch rule, like this:


RedirectMatch ^/$ http://targethost/mywebapp

Easy, agreed. But i’ve seen (and done it wrong my self) quite often…

No Comments Posted in: apache, java, open source, tomcat

Sorry guys, server crash…

September 11th, 2009

Folks,

There was a hardware crash at the hosting center that has caused a complete exchange and re-installation of this server.
I’ve got older backups running now, recent posts are not up yet but will be within the next week.
Posts may still look crappy and outtakes may occur, but for now, a least the older stuff is back online…

Greetings
Olaf

No Comments Posted in: , , java

Older Entries