Archive for the ‘open source’ Category
« Older EntriesTesting SSL (HTTPS) clients with Junit and Jetty
Wednesday, July 14th, 2010
Posted in: https, junit, ssl, unit test J2EE, java, maven 2, open source.
I recently ran into a situation where a web service client had issues when invoking a webservice via HTTPS. To reproduce, I needed a lightweight, JUnit-based test to reproduce the problem and write a regression test. Here is how I got it to work:
First, I used the basic HTTP test setup. Now however, I needed Jetty to provide a HTTPS connector. For this to work, Jetty – just like any other webserver, such as apache – needs a server certificate and a private key in a keystore. To create the keystore, I simply followed the excellent Jetty SSL guide. Namingly, I did this (using Ubuntu, see the above link for alternatives):
openssl genrsa -des3 -out jetty.key openssl req -new -x509 -key jetty.key -out jetty.crt openssl pkcs12 -inkey jetty.key -in jetty.crt -export -out jetty.pkcs12 $JAVA_HOME/bin/keytool -importkeystore -srckeystore jetty.pkcs12 -srcstoretype PKCS12 -destkeystore keystore
Once I had the keystore set up, I placed it in the src/main/resources folder (the root of the classpath, in case you’re not a maven user) and extended the HttpTestServer like so:
import java.net.URL;
import org.junit.Ignore;
import org.mortbay.jetty.Connector;
import org.mortbay.jetty.security.SslSocketConnector;
/**
* A test server for testing SSL requests.
*
* @author Olaf Otto
*/
@Ignore
public class SslTestServer extends HttpTestServer {
private static final int HTTPS_PORT = 50443;
public SslTestServer() {
}
public SslTestServer(String mockData) {
super(mockData);
}
@Override
protected void configureServer() {
super.configureServer();
Connector secureConnector = createSecureConnector();
getServer().addConnector(secureConnector);
}
private Connector createSecureConnector() {
SslSocketConnector connector = new SslSocketConnector();
connector.setPort(HTTPS_PORT);
URL keystoreUrl = getClass().getClassLoader().getResource("keystore");
connector.setKeystore(keystoreUrl.getFile());
connector.setKeyPassword("test");
return connector;
}
public static void main(String[] args) {
SslTestServer server = new SslTestServer();
try {
server.start();
} catch (Exception e) {
throw new RuntimeException(e);
}
}
}
(I added the @Ignore annotation as I placed the class in the test sources folder, though it’s not a test…)
You might use it in your JUnit-based test like so:
import org.junit.After;
import org.junit.Before;
import org.junit.Test;
/**
* @author Olaf Otto
*/
public class MySslTest {
private SslTestServer _server;
@Before
public void startServer() throws Exception {
_server = new SslTestServer();
_server.start();
}
@After
public void stopServer() throws Exception {
_server.stop();
}
@Test
public void mySsltest() {
_server.setMockResponseData("<xml>My client expects this response</xml>");
// Perform tests on port 50443...
}
}
Here is the whole package as a maven 2 project (including the keystore with a self-signed certificate). Download, unzip, and give it a try (mvn test).
Lightweight testing of (webservice) HTTP clients with JUnit and Jetty
Wednesday, July 14th, 2010
Posted in: http clients, junit, test, webservices J2EE, java, maven 2, open source.
If you’re working with webservice clients you will certainly have noticed the complexity of integration-testing your webservice clients. Building webservice clients can already be quite a complex task, but providing a mock-up webservice backend delivering useful test responses is quite often just to much work, if not impossible, since many web service backends are very complex constructs. Furthermore, setting up a backend for each integration test consumes valuable build time.
As an alternative, one may simply replace the backend with a mock HTTP server that does one thing: deliver the expected webserver response – i.e. XML data – when called by the client. All you need to set this up is a HTTP server and a pre-recorded service response (quite often this comes with WS client specifications). If you don’t have this data, you can record it, for example using a proxy server in between your WS client and a real WS backend, or a tool such as wireshark.
The recorded XML response thus represents your assumptions against which you test the client. This makes your tests fast, lightweight and a lot better to understand.
Here is my Jetty-based test HTTP server:
package de.sysbsb.test;
import static javax.servlet.http.HttpServletResponse.SC_OK;
import static org.apache.commons.io.IOUtils.write;
import java.io.IOException;
import javax.servlet.ServletException;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import org.apache.commons.io.IOUtils;
import org.junit.Ignore;
import org.mortbay.jetty.Handler;
import org.mortbay.jetty.HttpConnection;
import org.mortbay.jetty.Request;
import org.mortbay.jetty.Server;
import org.mortbay.jetty.handler.AbstractHandler;
/**
* A server for answering HTTP requests with test response data.
*
* @author Olaf Otto
*/
@Ignore
public class HttpTestServer {
public static final int HTTP_PORT = 50036;
private Server _server;
private String _responseBody;
private String _requestBody;
private String _mockResponseData;
public HttpTestServer() {
}
public HttpTestServer(String mockData) {
setMockResponseData(mockData);
}
public void start() throws Exception {
configureServer();
startServer();
}
private void startServer() throws Exception {
_server.start();
}
protected void configureServer() {
_server = new Server(HTTP_PORT);
_server.setHandler(getMockHandler());
}
/**
* Creates an {@link AbstractHandler handler} returning an arbitrary String as a response.
*
* @return never <code>null</code>.
*/
public Handler getMockHandler() {
Handler handler = new AbstractHandler() {
public void handle(String target, HttpServletRequest request, HttpServletResponse response, int dispatch) throws IOException, ServletException {
Request baseRequest = request instanceof Request ? (Request) request : HttpConnection.getCurrentConnection().getRequest();
setResponseBody(getMockResponseData());
setRequestBody(IOUtils.toString(baseRequest.getInputStream()));
response.setStatus(SC_OK);
response.setContentType("text/xml;charset=utf-8");
write(getResponseBody(), response.getOutputStream());
baseRequest.setHandled(true);
}
};
return handler;
}
public void stop() throws Exception {
_server.stop();
}
public void setResponseBody(String responseBody) {
_responseBody = responseBody;
}
public String getResponseBody() {
return _responseBody;
}
public void setRequestBody(String requestBody) {
_requestBody = requestBody;
}
public String getRequestBody() {
return _requestBody;
}
public static void main(String[] args) {
HttpTestServer server = new HttpTestServer();
try {
server.start();
} catch (Exception e) {
throw new RuntimeException(e);
}
}
public void setMockResponseData(String mockResponseData) {
_mockResponseData = mockResponseData;
}
public String getMockResponseData() {
return _mockResponseData;
}
protected Server getServer() {
return _server;
}
}
You can use it in a JUnit test like so:
package de.sysbsb.test;
import java.io.IOException;
import java.io.InputStream;
import org.apache.commons.io.IOUtils;
import org.junit.After;
import org.junit.Before;
import org.junit.Test;
/**
* @author Olaf Otto
*/
public class MyClientTest {
private HttpTestServer _server;
@Before
public void startTestServer() throws Exception {
_server = new HttpTestServer();
_server.start();
}
@After
public void stopTestServer() throws Exception {
_server.stop();
}
@Test
public void someClientTest() throws IOException {
_server.setMockResponseData("<xml><testdata>hello, world</testdata></xml>");
// or as a much more elegant alternative:
_server.setMockResponseData(getRecordedResponse("testdata.xml"));
// run your client tests, using the HttpTestServer.HTTP_PORT
}
private String getRecordedResponse(String recordedResponseFile) throws IOException {
InputStream testDataStream = getClass().getClassLoader().getResourceAsStream(recordedResponseFile);
return IOUtils.toString(testDataStream);
}
}
Here are is the HTTP test server and a JUnit test example as a maven 2 project.
5 CommentsSolved: Xserver lock-ups in Ubuntu 10.04 with intel GFX
Wednesday, June 23rd, 2010
Posted in: intel, lucid, Xserver System engineering, open source, ubuntu.
After installing 10.04 I experienced a GDM lockup a couple of times a day – the symptoms varied from the window management not reacting to left mouse clicks to a complete freeze of the Xserver. This bug has caused a lot of noise across distributions, for instance in 590109 and 538563. The symptoms are log messages such as
[mi] EQ overflowing. The server is probably stuck in an infinite loop.
...
Backtrace:
...
... /usr/lib/xorg/modules/drivers/intel_drv.so
...
in the :0… logfiles in /var/log/gdm when the problem occurs.
This is, however, not a kernel, XServer or GDM issue, but caused by a problem with the intel-linux driver. The problem is apparently fixed as of version 2:2.11.0-1ubuntu1 of the xserver-xorg-video-intel driver. Since this is not (yet) an update in the official distribution repo, I did the following to solve the issue:
1: Optional: Update to the latest stable kernel
Reason: IMO, upstream Intel drivers are usually build against the latest stable kernel
Goto http://kernel.ubuntu.com/~kernel-ppa/mainline/v2.6.34-lucid/ and download the headers… _all.deb, headers-generic …..deb and linux-image… .deb for your platform (in most cases, i368). install the generic headers package, then the all headers package, then the image.
A word of warning: If you install this custom kernel you will not receive security updates for it from the automatic distribution updates.
2: Add the X Updates PPA to your sources.list
In a terminal, type:
sudo gedit /etc/apt/sources.list
And add at the bottom of the file (if not already present):
deb http://ppa.launchpad.net/ubuntu-x-swat/x-updates/ubuntu lucid main #X-Updates PPA
deb-src http://ppa.launchpad.net/ubuntu-x-swat/x-updates/ubuntu lucid main #X-Updates PPA
2.1: Trust the software packages from the x-updates PPA
sudo apt-key adv –recv-keys –keyserver keyserver.ubuntu.com AF1CDFA9
3: Update the driver
sudo apt-get update
sudo apt-get dist-upgrade
Reboot – the problem should be solved.
No CommentsCreating HDR images in Ubuntu with Luminance (QTPFSGUI)
Saturday, February 6th, 2010
Posted in: HDR, luminance, qtpfsgui open source, ubuntu.
For those who are not (yet) in possession one of the latest digital cameras with build-in HDR capabilities or the more experimental folks, there is a fantastic free tool to create HDR images by combining multiple shots of the same scene: Its called qtpfsgui.
That is, it used to be called that and is still available in Ubuntu under this name (sudo apt-get install qtpfsgui). However, qtpfsgui is discontinued since mid 2009 and has been replaced by the Luminance HDR project.
Since qtpfsgui crashes under ubuntu 9.10 when attempting to save any HDR image, I downloaded the latest luminance version, compiled and installed it like so:
- Download the luminance source and unpack the folder.
-
Install the dependencies required to compile luminance:
sudo apt-get install qt4-qmake libexiv2-dev libopenexr-dev fftw3-dev libtiff4-dev libqt4-dev g++ libgsl0-dev -
Compile luminance (takes a few minues) and install it. Change to the unpacked luminance folder and do:
qmake
make
sudo make install
That´s it! you now have luminance installed and it should be in your main menu under applications>graphics.
It works perfectly with Karmic.
One of the best things about luminance is that you can play a lot with the algorithms and parameters used to tonemap the HDR into a LDR image. In contrast to many other (commercial) tools, you actually get to know which algorithms are used, who created them and can read a little more on how they work (if you are not to opposed to mathematics) by googling up the corresponding papers.
Per-assembly filtering with the maven-assembly-plugin
Monday, January 18th, 2010
Posted in: maven 2, open source.
The maven assembly plugin is an extremely powerful tool when it comes to creating custom distributions (aka assemblies) of your artifacts for individual platforms etc. However, the ability to create multiple variants of an artifact within a single build conflicts with the standard maven approach of using multiple build profiles and executing a single build for each profile to generate artifact variants.
This also means that mavens resource filtering is not very useful for individual assemblies, since you can only replace placeholders such as ${someValue} with the same value for all assemblies (since resources are only filtered once, using the active profile(s), for all assemblies).
You can, however, configure the maven assembly plugin to use individual filter property files for each assembly using the <execution> section of the plugin configuration, like so:
<plugin>
<artifactId>maven-assembly-plugin</artifactId>
<version>2.2-beta-5</version>
<executions>
<execution>
<id>production</id>
<goals>
<goal>single</goal>
</goals>
<configuration>
<filters>
<filter>${project.basedir}/src/main/assembly/production.properties</filter>
</filters>
<finalName>${artifactId}-production-${version}</finalName>
<ignoreDirFormatExtensions>true</ignoreDirFormatExtensions>
<descriptors>
<descriptor>src/main/assembly/assembly.xml</descriptor>
</descriptors>
</configuration>
<phase>package</phase>
</execution>
<execution>
<id>integration</id>
<goals>
<goal>single</goal>
</goals>
<configuration>
<filters>
<filter>${project.basedir}/src/main/assembly/integration.properties</filter>
</filters>
<finalName>${artifactId}-integration-${version}</finalName>
<ignoreDirFormatExtensions>true</ignoreDirFormatExtensions>
<descriptors>
<descriptor>src/main/assembly/assembly.xml</descriptor>
</descriptors>
</configuration>
<phase>package</phase>
</execution>
</executions>
</plugin>
This snippet uses the same assembly descriptor (you can use any assembly descriptor, actually) to build individual artifacts for production and integration. Note that the ${project.basedir}/ prefix is a workaround for MASSEMBLY-150 and is nedded to avoid a nasty Failed to create assembly: Error filtering file ... Error loading property file 'src/main/...' error when executing the mojo from outside the module directory (say, for instance, in a mvn release:prepare…).
Filtering must be enabled in the assembly descriptor, like so:
<?xml version="1.0" encoding="UTF-8"?>
<assembly xmlns="http://maven.apache.org/xsd/assembly"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/xsd/assembly http://maven.apache.org/xsd/assembly-1.1.1.xsd">
...
<fileSets>
<fileSet>
<filtered>true</filtered>
...
</fileSet>
</fileSets>
</assembly>
Thanks to John Casey for suggesting this in MASSEMBLY-430!
1 CommentObtaining the default charset (aka platform encoding) in JAVA
Monday, November 9th, 2009
Posted in: Charset, Platform encoding java, open source.
… is really simple since JDK 1.5:
java.nio.charset.Charset.defaultCharset()
So please don’t use messy, old workarounds such as this:
byte [] byteArray = {'a'};
InputStream inputStream = new ByteArrayInputStream(byteArray);
InputStreamReader reader = new InputStreamReader(inputStream);
String defaultEncoding = reader.getEncoding();
Which, unfortunately, is one of the first things you stumble upon when searching for this topic.
No CommentsUsing RPC-style, encoded Axis 1 webservices with spring-remoting
Monday, October 12th, 2009
Posted in: Axis 1, Jax-ws, Spring-Remoting J2EE, System architecture, java, open source.
Preamble
When working with enterprise integration you will quite often deal with the integration of legacy systems.
I was recently tasked with the integration of webservices and much to my surprise there are huge issues when attempting to integrate old webservices (i.e. pre axis-2 and pre jax-ws webservices).
These webservices use outdated or unsupported methods, such as rpc-style/encoded communication, or even worce, outdated elements in the XML messages, for instance <multiref>’s. Having to integrate such services of course implies that you cannot change anything on the server side, and thus the outdated format must still be consumed by your webservice client.
This however is a huge problem, since the current WS implementations, such as the popular axis2 framework, do simply not support these outdated formats.
What surprised me the most about this is that webservices are designed to make systems independent by defining a common communication and data format, which is the exact opposite of what is going on here – and these formats are not that old, actually, they where pretty common just about 5 years ago.
Developers facing these issues are taking rather desperate measures to work around these problems, such as using on-the-fly XSLT transformation to convert incoming and outgoing WS messages. However, this binds your application even stronger to the outdated format and specific service data.
In my case I wanted to use Spring-remoting. Since the service was RPC-style, I wanted to use the JaxRpcPortProxyFactoryBean to create an on-the-fly proxy implementing the service interface. What I did not want to do is having to write any additional code to consume the service. Furthermore, I wanted the complex objects transferred by the services to be represented by standard JAVA beans, and not be generated using the wsdl2java axis1 tools since they contain a lot of ugly dependencies. Here is how I got this to work.
Read the rest of “Using RPC-style, encoded Axis 1 webservices with spring-remoting”
Using apache to redirect from the root “/” context to a webapp context
Monday, October 12th, 2009
Posted in: apache, java, open source, tomcat.
Just a short note:
The best way of doing this avoiding endless recursion is to use apaches RedirectMatch rule, like this:
RedirectMatch ^/$ http://targethost/mywebapp
Easy, agreed. But i’ve seen (and done it wrong my self) quite often…
No CommentsRunning mvn:release with Subversion 1.5.x
Monday, March 23rd, 2009
Posted in: J2EE, SCM, System engineering, java, maven 2, open source, subversion.
When attempting to prepare a release using maven and the maven-release-plugin, you are currently very likely to see your build fail with a message such as:
[INFO] Executing: svn --non-interactive copy --file /tmp/...commit . https://subversion..../tags/...
[INFO] Working directory: ...
[INFO] ------------------------------------------------------------------------
[ERROR] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Unable to tag SCM
Provider message:
The svn tag command failed.
Command output:
svn: Commit failed (details follow):
svn: File '...' already exists
If you do, you are using subversion 1.5.x which no longer supports tagging from a local working copy, thus causing the unfortunately very misleading error message.
This is a known issue of the subversion sourcecode management (SCM) and there is a simple workaround:
Read the rest of “Running mvn:release with Subversion 1.5.x”
Modular development with OSGI about to be very, very popular – finally.
Monday, March 2nd, 2009
Posted in: J2EE, JPA, OSGI, System architecture, eclipselink, java, open source, spring.
It’s quite exciting to see the tremendous progress in development tools, patterns, frameworks and libraries since the release of JAVA 1.0 in 1996.
Within just thirteen years, development improved from the first applets, from the dark ages of the first EJB standardization attempts and a lack of suitable development processes into an age where aspect oriented programming, dependency injection and a freely available range of lightweight yet extremely powerful frameworks, libraries and API’s finally permitted the humble programmer to do what he always wanted to do – analyze problems, and elegantly write them down without spending all of his precious time on the syntax.
I would even go as far as saying that there was no object orientation before programming crosscutting concerns and using dependency injection became established mechanisms.
Think about it: The original goal of OO is not to use inheritance, encapsulation, identity and polymorphism, but to express the real world of things – our reality – using an abstract world of things, while preserving most of the properties and relationships we observe in reality.
Before AOP, before being able to separate the description of things from creating, configurating and connecting concrete instances through dependency injection frameworks, this goal was hard to achieve.
Modules larger than classes with OSGI
Now however, we are about to reach a new stage in development of object orientation in JAVA: Developing modules that are larger than classes, modules which can, once properly identified, be developed independently using all the wonderful technology described beforehand. The technology used to achieve this has a name: OSGI. This technology has quite a long history of development, but a certain lack of elegance and light-weightiness barred this component model from becoming a real hit in the worldwide development community. This is about to change.
Read the rest of “Modular development with OSGI about to be very, very popular – finally.”
