On May 29, 2011 christopher wrote: Content Negotiation with Spring 3 MVC & Velocity

One of my pet sub­jects is con­tent nego­ti­a­tion. There are many ben­e­fits to a REST­ful approach to web appli­ca­tion archi­tec­ture but few which fill me with such a curi­ous delight as the spec­ta­cle of see­ing loads of dif­fer­ent rep­re­sen­ta­tions of the same under­ly­ing resource being returned at the flick of an Accept header. That prob­a­bly makes me a very weird and geeky per­son but what the hell: con­tent nego­ti­a­tion is fun! … espe­cially when it is achieved with prac­ti­cally zero effort.

Any­how, some of the approaches I have seen to con­tent nego­ti­a­tion seem a lit­tle “over­weight”, fre­quently involv­ing sig­nif­i­cant amounts of cod­ing in order to add sup­port for new rep­re­sen­ta­tions. To be fair, they are intended to pro­vide über solu­tions to every con­ceiv­able sce­nario — which is great when you encounter one of those 1%-of-the-time sit­u­a­tions. Nor­mally, how­ever, you will sim­ply be return­ing a new text-based for­mat. Surely that should be a breeze, right? Well, yes and no.

One of the great improve­ments to Spring in ver­sion 3 is its sup­port for REST and con­tent nego­ti­a­tion in par­tic­u­lar. It also pro­vides almost-out-of-the-box solu­tions for pro­duc­ing JSON and XML rep­re­sen­ta­tions of a view model that might oth­er­wise be ren­dered as HTML (i.e. when we don’t ask for either JSON or XML specif­i­cally). But some­times they don’t quite give you the results you want and cus­tomi­sa­tion of the out­put is nigh-on impos­si­ble, so you have to go back to writ­ing some code. Which is a pain. But hold on a sec­ond … am I not already using a mech­a­nism for seri­al­iz­ing a Java object graph as text in order to pro­duce HTML pages? Indeed I am. In my case, in my hobby project which I call “Appo­site” (see my post on end-to-end BDD test­ing), I am using Apache Veloc­ity

In the post on BDD test­ing, I setup a sin­gle test case which asserted that a “admin­is­tra­tor” could log in to an “admin­is­tra­tion dash­board”. Let’s take up that case and try not just to make it pass but also to make the result­ing page view­able in a num­ber of dif­fer­ent for­mats (e.g. atom, rss, xml, json, plain text etc). Really, I should be writ­ing BDD tests specif­i­cally to cover each of these “view as for­mat” sce­nar­ios but, as they are not really part of the appli­ca­tion require­ments and I am only doing it for demon­stra­tion pur­poses, I am going to skip that for now.

Once again, let’s take this step-by-step and begin by set­ting up our basic Spring web appli­ca­tion. First off, we need to add the Spring depen­den­cies to the pom.xml, if not already present:

    <properties>
        <spring.version>3.0.5.RELEASE</spring.version>
        <spring.security.version>${spring.version}</spring.security.version>
    </properties>

First up, under the DRY prin­ci­ple, I declare the Spring ver­sion num­ber as a property.

        <dependency>
            <groupId>org.springframework</groupId>
            <artifactId>spring-context</artifactId>
            <version>${spring.version}</version>
        </dependency>
        <dependency>
            <groupId>org.springframework</groupId>
            <artifactId>spring-core</artifactId>
            <version>${spring.version}</version>
        </dependency>
        <dependency>
            <groupId>org.springframework</groupId>
            <artifactId>spring-web</artifactId>
            <version>${spring.version}</version>
        </dependency>
        <dependency>
            <groupId>org.springframework</groupId>
            <artifactId>spring-webmvc</artifactId>
            <version>${spring.version}</version>
        </dependency>
        <dependency>
            <groupId>org.springframework.security</groupId>
            <artifactId>spring-security-core</artifactId>
            <version>${spring.security.version}</version>
        </dependency>

The Spring frame­work is well mod­u­larised, so there are a num­ber of dif­fer­ent depen­den­cies required. It can become a lit­tle con­fus­ing when try­ing to deter­mine which Spring depen­den­cies one actu­ally requires but this is prefer­able, in my view (and, obvi­ously, the view of the peo­ple at Spring), to includ­ing a sin­gle mas­sive jar con­tain­ing tonnes of stuff you don’t need. The key mod­ules in the list above are:

spring-context
Con­tains the Spring appli­ca­tion con­text classes and anno­ta­tions to enable you actu­ally con­fig­ure and cre­ate a Spring context.
spring-core
For the pur­poses of our web appli­ca­tion, the most impor­tant aspect of the spring-core.jar is that it is the home for the new, uni­fied con­ver­sion API intro­duced with Spring 3. This rep­re­sents a sig­nif­i­cant improve­ment over the pre­vi­ous PropertyEditor based sup­port for data binding.
spring-web
This is obvi­ously nec­es­sary for a web appli­ca­tion, con­tain­ing, as it does, most of the basic Servlet API inte­gra­tion code includ­ing the ContextLoaderListener and request map­ping annotations.
spring-webmvc
Con­tains a num­ber impor­tant sup­port classes and servlet API inte­gra­tions for cre­at­ing MVC web apps; notably the DispatcherServlet and the MvcNamespaceHandler (more on which below). Con­ven­tion­ally, this is also used to pro­vide view classes, such as Spring’s Veloc­ity and Freemarker imple­men­ta­tions. How­ever, I will be util­is­ing instead my own Veloc­ity inte­gra­tion library, because it is, of course, much bet­ter! (It is my spring-velocity library that makes the con­tent nego­ti­a­tion we are going to be using pos­si­ble so, whilst I won’t go into the imple­men­ta­tion in detail, I will note some key dif­fer­ences with the usual inte­gra­tion with Veloc­ity pro­vided by Spring out-of-the-box.)
spring-security-core
Spring Secu­rity will be used to pro­vide user authen­ti­ca­tion ser­vices since the test case I want to pass requires this.

Because I am using my own Spring-Velocity inte­gra­tion library, I also need to declare my maven repos­i­tory in the POM:

    <repositories>
        <repository>
            <id>christophertownson-com-public</id>
            <url>http://christophertownson.com/mvn/</url>
        </repository>
    </repositories>

And add the library as a dependency …

        <dependency>
            <groupId>com.christophertownson</groupId>
            <artifactId>spring-velocity</artifactId>
            <version>0.0.2</version>
            <exclusions>
                <exclusion>
                    <groupId>commons-logging</groupId>
                    <artifactId>commons-logging</artifactId>
                </exclusion>
            </exclusions>
        </dependency>

I am exclud­ing commons-logging (which is picked up as a tran­si­tive depen­dency via Veloc­ity and Spring) here because I am using slf4j so nei­ther need nor want it.

Finally, I also want to use the XStreamMarshaller and MappingJacksonJsonView to pro­duce default XML and JSON rep­re­sen­ta­tions, respec­tively, so I need to declare run­time depen­den­cies on XStream and Jackson-Mapper:

        <dependency>
            <groupId>com.thoughtworks.xstream</groupId>
            <artifactId>xstream</artifactId>
            <version>1.3.1</version>
            <scope>runtime</scope>
        </dependency>
        <dependency>
            <groupId>org.codehaus.jackson</groupId>
            <artifactId>jackson-mapper-asl</artifactId>
            <version>1.6.2</version>
            <scope>runtime</scope>
        </dependency>

Before we start, let’s remind our­selves of the test scenario:

Scenario: I cannot access the administration dashboard unless I am logged in
 
Given I am not logged in
When I go to the administration dashboard
Then I am asked to login
Then I enter the administrator credentials
Then I am redirected to the administration dashboard

Clearly, we are going to need some kind of “admin dash­board con­troller” to sat­isfy this test, so let’s begin with that:

public class DashboardAdminControllerTest {

    private DashboardAdminController controller;

    @Before public void setup() {
        controller = new DashboardAdminController();
    }

    @Test public void shouldReturnEmptyModelWhenIGetDashboard() {
        assertThat(controller.getDashboard(), equalTo((Map<String, Object>) new HashMap<String, Object>()));
    }

}

This sim­ple test should be suf­fi­cient for now: all it asserts is that there is a “get dash­board” method which, for now, just returns an empty model. That should be easy enough to pass …

@Controller public class DashboardAdminController {

    @RequestMapping(value = "/admin", method = RequestMethod.GET) public Map<String, Object> getDashboard() {
        return new HashMap<String, Object>();
    }

}

Some things to notice:

  • In the cre­ation of the first con­troller here, we are set­ting up a name­space and nam­ing con­ven­tion for “admin­is­tra­tion con­trollers” whereby they will live in the org.apposite.controller.admin pack­age and be named *AdminController.
  • The con­troller knows only about the model and noth­ing about view names (there are some unsat­is­fac­tory points to come where it will need to know about the lat­ter and I will have to com­pro­mise with the frame­work, but that is another story).
  • The unit test cov­ers only what the con­troller, as a class, knows about (i.e. the model). The con­fig­u­ra­tion (request map­ping, in this case) is pro­vided as meta­data on the class and implic­itly cov­ered by the end-to-end func­tional test.

We’re also going to need some kind of “login con­troller”. This is cross-cutting func­tion­al­ity but is per­formed by a “User”, so I’m going to cre­ate a UserController for this purpose:

public class UserControllerTest {

    private UserController controller;

    @Before public void setup() {
        controller = new UserController();
    }

    @Test public void shouldReturnEmptyModelWhenIGetLogin() {
        assertThat(controller.login(), equalTo((Map<String, Object>) new HashMap<String, Object>()));
    }

}
@Controller public class UserController extends AbstractController {

    @RequestMapping(value = "/users/login", method = RequestMethod.GET) public Map<String, Object> login() {
        return new HashMap<String, Object>();
    }

}

Because nei­ther con­troller is yet required to gen­er­ate any model data, they are both extremely simple.

Next, we need to pro­vide some way to map the request to a view tem­plate. For this pur­pose, Spring pro­vides the RequestToViewNameTranslator inter­face. I’m obvi­ously using TDD here (if you aren’t these days, what are you doing?!), so I begin with a test for my implementation:

public class ViewNameTranslatorTest {

    private ViewNameTranslator vnt;

    private MockHttpServletRequest req;

    @Before public void setup() {
        vnt = new ViewNameTranslator();
        vnt.setUriPattern("^/((?:admin/)?[a-zA-Z0-9-_]+)(?:\\.[a-zA-Z0-9]+)?(/|/([a-zA-Z0-9-_]+)(?:\\.[a-zA-Z0-9]+)?(/|/([a-zA-Z0-9-_]+)/?)?)?(?:\\.[a-zA-Z0-9]+)?");
        vnt.setEntityIdentifierPattern("^[0-9]+$");
        vnt.setEntityIndex(1);
        vnt.setActionIndex(3);
        vnt.setEntityActionIndex(5);
        vnt.setDefaultAction("list");
        vnt.setReadAction("read");
    }

    @Test public void shouldReturnAdministratorsDashboardView() throws Exception {
        givenRequest("GET", "/admin");
        thenViewNameIs("/admin/list");
    }

    @Test public void shouldReturnLoginFormView() throws Exception {
        givenRequest("GET", "/users/login");
        thenViewNameIs("/users/login");
    }

    private void givenRequest(String method, String uri) {
        givenRequest(method, "/apposite", uri);
    }

    private void givenRequest(String method, String contextPath, String uri) {
        req = new MockHttpServletRequest(method, contextPath + uri);
        req.setContextPath(contextPath);
    }

    private void thenViewNameIs(String name) throws Exception {
        assertThat(vnt.getViewName(req), is(name));
    }

}

Now, you are prob­a­bly (and should be) think­ing “What the hell is going on in that setup method?” I will admit that I am jumping-the-gun a lit­tle here, but it is worth doing, I think: one of my non-functional require­ments for this appli­ca­tion is that all URIs are pre­dictable on the basis of a pat­tern descrip­tion (i.e. a reg­u­lar expres­sion). In other words, it is con­ven­tional. This can become restric­tive but, on the whole, I feel that the con­sis­tency it lends to the appli­ca­tion is desir­able from both a developer’s and a user’s per­spec­tive: “usabil­ity” is a phe­nom­e­non that can be described in terms of “intu­itive­ness” which, in turn, can be described as a form of pre-reflective pat­tern recog­ni­tion. Usabil­ity is a vital con­sid­er­a­tion because, tele­o­log­i­cally, it describes a ten­dency to facil­i­tate, rather than hin­der, intent and action (whether that is of a devel­oper extend­ing a code base or a user attempt­ing to com­plete some sce­nario). There­fore, all my URIs will take the fol­low­ing form, where each ele­ment is optional:

  1. A name­space (e.g. “admin”)
  2. An entity name
  3. An entity identifier
  4. An action name (ide­ally, we would be able to encap­su­late the con­cept of an “action” entirely within the use of HTTP verbs but there are occa­sions where it is nec­es­sary to pro­vide URIs that include an action in order to eas­ily sup­port con­ven­tional human inter­ac­tion via a web browser).
  5. A file exten­sion (which can used to over­ride the content-type spec­i­fied by the Accept header where necessary).

My ViewNameTranslator imple­men­ta­tion will, con­se­quently, have some URI pat­terns defined with cap­ture group indexes set cor­re­spond­ingly so that it can parse out the rel­e­vant con­stituent parts of my URIs. Nonethe­less, the main thing to note is that the test asserts that the view name for the URI GET /admin will be /admin/list and that, for GET /users/login, the view name will be /users/login.

@Component public class ViewNameTranslator implements RequestToViewNameTranslator {

    private Integer actionIndex;

    private String defaultAction;

    private Integer entityActionIndex;

    private RegularExpression entityIdentifierPattern;

    private Integer entityIndex;

    private String readAction;

    private RegularExpression uriPattern;

    @Override public String getViewName(HttpServletRequest request) throws Exception {
        String uri = request.getRequestURI().replaceFirst(request.getContextPath(), "");
        if (!uriPattern.matches(uri)) return null;
        List<String> groups = uriPattern.groups(uri);
        String entity = groups.get(entityIndex);
        String action = groups.size() > actionIndex ? groups.get(actionIndex) : defaultAction;
        String entityAction = groups.size() > entityActionIndex ? groups.get(entityActionIndex) : null;
        String ext = FilenameUtils.getExtension(uri);
        return "/" + entity + "/" + (entityIdentifierPattern.matches(action) ? entityAction != null ? entityAction : readAction : action) + (isNotBlank(ext) ? "." + ext : "");
    }

    @Autowired public void setActionIndex(@Value("${org.apposite.view.ViewNameTranslator.actionIndex}") Integer actionIndex) {
        this.actionIndex = actionIndex;
    }

    @Autowired public void setDefaultAction(@Value("${org.apposite.view.ViewNameTranslator.defaultAction}") String defaultAction) {
        this.defaultAction = defaultAction;
    }

    @Autowired public void setEntityActionIndex(@Value("${org.apposite.view.ViewNameTranslator.entityActionIndex}") Integer entityActionIndex) {
        this.entityActionIndex = entityActionIndex;
    }

    @Autowired public void setEntityIdentifierPattern(@Value("${org.apposite.view.ViewNameTranslator.entityIdentifierPattern}") String entityIdentifierPattern) {
        this.entityIdentifierPattern = new RegularExpression(entityIdentifierPattern, Flag.CASE_INSENSITIVE);
    }

    @Autowired public void setEntityIndex(@Value("${org.apposite.view.ViewNameTranslator.entityIndex}") Integer entityIndex) {
        this.entityIndex = entityIndex;
    }

    @Autowired public void setReadAction(@Value("${org.apposite.view.ViewNameTranslator.readAction}") String readAction) {
        this.readAction = readAction;
    }

    @Autowired public void setUriPattern(@Value("${org.apposite.view.ViewNameTranslator.uriPattern}") String uriPattern) {
        this.uriPattern = new RegularExpression(uriPattern, Flag.CASE_INSENSITIVE);
    }

}

The things to note about this class are:

  1. It is anno­tated with @Component: it is a Spring bean. No bean ID is nec­es­sary. Sim­ply by hav­ing a Spring bean that imple­ments RequestToViewNameTranslator in your appli­ca­tion con­text, Spring will detect and use it as appropriate.
  2. The RegularExpression class I am util­is­ing is a helper class pro­vided by my lit­tle com­mons library which sim­ply wraps java.util.regex.Pattern to make it eas­ier to use.
  3. The var­i­ous con­fig­urable prop­erty val­ues are obtained from an application.properties file (this con­tains prop­er­ties that are inter­nal to the appli­ca­tion itself and are required but which might fea­si­bly be over­rid­den using another prop­er­ties file taken from the tar­get run­time envi­ron­ment. In accor­dance with the unit test, I there­fore have the fol­low­ing application.properties defined:
org.apposite.view.ViewNameTranslator.uriPattern = ^/((?:admin/)?[a-zA-Z0-9-_]+)(?:\\.[a-zA-Z0-9]+)?(/|/([a-zA-Z0-9-_]+)(?:\\.[a-zA-Z0-9]+)?(/|/([a-zA-Z0-9-_]+)/?)?)?(?:\\.[a-zA-Z0-9]+)?
org.apposite.view.ViewNameTranslator.entityIdentifierPattern = ^[0-9]+$
org.apposite.view.ViewNameTranslator.entityIndex = 1
org.apposite.view.ViewNameTranslator.actionIndex = 3
org.apposite.view.ViewNameTranslator.entityActionIndex = 5
org.apposite.view.ViewNameTranslator.defaultAction = list
org.apposite.view.ViewNameTranslator.readAction = read

Dear Spring peo­ple: can you please add an optional=true attribute to the @Value anno­ta­tion so that value injec­tion from a prop­er­ties file does not throw if left uncon­fig­ured? Then I could have defaults in the class and then only have to con­fig­ure when I need to over­ride. Thanks.

Next, let’s con­fig­ure some Veloc­ity basics and make some tem­plates for the admin dashboard:

com.christophertownson.spring.velocity.RedirectViewResolver.order = 0
com.christophertownson.spring.velocity.VelocityViewResolver.order = 2
org.springframework.web.servlet.view.ContentNegotiatingViewResolver.order = 1

First, I set the chain­ing order of the var­i­ous “view resolvers” that I will be using. The order is:

  1. Redi­rect view resolver comes first. We can always detect eas­ily if it is a redi­rect view (because the view name starts with redirect:. More­over, these need to be han­dled dif­fer­ently. There­fore, we get them out of the way with at the start of the chain so there is no need for fur­ther processing.
  2. Next we go to the ContentNegotiatingViewResolver so that it can mar­shall the request and con­struct a sorted set of can­di­date view names (in order of file exten­sion over­ride or Accept header pref­er­ence) on the basis of a media type map­ping (e.g. given an appro­pri­ate media types map­ping, a request result­ing in the view name /home with an Accept: text/json header might result in the view name set /home.json, /home, giv­ing sub­se­quent view resolvers or con­fig­ured default views the oppor­tu­nity to sat­isfy the request using the clients pre­ferred representation).
  3. Finally, we come to my cus­tom VelocityViewResolver. This will look for an exist­ing Veloc­ity tem­plate cor­re­spond­ing to the view name. This means, for exam­ple, that, when used in con­junc­tion with the ContentNegotiatingViewResolver, we could con­fig­ure a MappingJacksonJsonView on the lat­ter to serve a default JSON rep­re­sen­ta­tion (seri­al­ized object graph) but, should we wish to cus­tomise that rep­re­sen­ta­tion, we could sim­ply drop in a /home.json.vm tem­plate: the VelocityViewResolver would con­se­quently indi­cate to the ContentNegotiatingViewResolver that it could sat­isfy a request to rep­re­sent the resource “/home” as JSON and would be del­e­gated to in order to sat­isfy it accord­ingly in pref­er­ence to the MappingJacksonJsonView. Because it works off of the same media types map­ping as the ContentNegotiatingViewResolver, the result­ing VelocityView pro­duced by the VelocityViewResolver will dynam­i­cally be able to deter­mine the cor­rect content-type to use when stream­ing the con­tent back to the client.

Next a lit­tle con­fig­u­ra­tion that is unfor­tu­nately nec­es­sary only because Spring’s otherwise-extremely-handy @Value anno­ta­tion has no “optional” or “default” attributes:

com.christophertownson.spring.velocity.DefaultVelocityInitialisationStrategy.order = 0
com.christophertownson.spring.velocity.VelocityToolsInitialisationStrategy.order = 1
com.christophertownson.spring.velocity.VelocityToolsWebInitialisationStrategy.order = 2

com.christophertownson.spring.velocity.VelocityConfiguration.defaultMediaType = application/xhtml+xml
com.christophertownson.spring.velocity.VelocityConfiguration.defaultLayoutTemplate = /common/layouts/layout.vm
com.christophertownson.spring.velocity.VelocityConfiguration.layoutContextKey = layout
com.christophertownson.spring.velocity.VelocityConfiguration.prefix = /templates
com.christophertownson.spring.velocity.VelocityConfiguration.screenContentKey = screen_content
com.christophertownson.spring.velocity.VelocityConfiguration.suffix = .vm
com.christophertownson.spring.velocity.VelocityConfiguration.toolboxUrl = /WEB-INF/toolbox.xml

com.christophertownson.spring.velocity.SpringViewHelperRenderInterceptor.xhtml = true

This just sets the order of a bunch of “ini­tial­i­sa­tion strate­gies” and prop­er­ties for Veloc­ity. As I say, whilst they need to be configur-able, there should be lit­tle need to ever change many of the above val­ues because most are merely sen­si­ble defaults. If some­one out there knows of a way to option­ally inject val­ues from prop­er­ties files using Spring anno­ta­tions, I would dearly love to hear from you!

And then a lit­tle more veloc­ity con­fig­u­ra­tion so that it is really work­ing the way we want it to …

input.encoding = UTF-8
output.encoding = UTF-8

directive.foreach.maxloops = 1000
directive.set.null.allowed = true

resource.loader = webapp, classpath

classpath.resource.loader.description = Velocity Classpath Resource Loader
classpath.resource.loader.class = org.apache.velocity.runtime.resource.loader.ClasspathResourceLoader

webapp.resource.loader.description = Velocity Web Application Resource Loader
webapp.resource.loader.class = org.apache.velocity.tools.view.WebappResourceLoader
webapp.resource.loader.path = /WEB-INF
webapp.resource.loader.cache = false
webapp.resource.loader.modificationCheckInterval = 10

runtime.log.logsystem.class = org.apache.velocity.runtime.log.Log4JLogChute
runtime.log.logsystem.log4j.logger = org.apache.velocity
runtime.log.invalid.references = false

velocimacro.library = /templates/common/macros/macros.vm,/org/springframework/web/servlet/view/velocity/spring.vm
velocimacro.library.autoreload = true

I won’t detail what each of these options achieves except to point out that, on line 22, I spec­ify the use of 2 macro libraries: one con­tain­ing my own macros and the one pro­vided by Spring (which is very use­ful for form binding).

Any­way, now that we’re mostly con­fig­ured, let’s cre­ate an /admin/list.vm tem­plate to serve the default rep­re­sen­ta­tion of the admin dash­board page:

<h2>Administration Dashboard</h2>

There we go. Nice and basic, I think you’ll agree (it just matches the sim­ple asser­tion from the func­tional test that the head­ing will be “Admin­is­tra­tion Dashboard”).

I’m going to cre­ate my login form as a macro, because I thnk I can safely assume that, at some point, I will want to be able to put the form in mul­ti­ple pages:

#macro(loginForm)
<form class="login" method="post" action="$linkTool.relative('/j_spring_security_check')">
    <fieldset>
        #label('j_username' 'Username') <input type="text" id="j_username" name="j_username" maxlength="255" />
        #label('j_password' 'Password') <input type="password" id="j_password" name="j_password" maxlength="40" />
        #submitInput('Login')
    </fieldset>
</form>
#end

#macro(label $for $label)<label for="$for">$label</label>#end

#macro(submitInput $value)<input type="submit" value="$value" />#end

Con­se­quently, all I need to do to cre­ate my login page is to cre­ate the fol­low­ing template:

#loginForm()

Let’s quickly cre­ate a default lay­out template …

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN" "http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" version="-//W3C//DTD XHTML 1.1//EN">
    <head>
        <meta http-equiv="content-type" content="application/xhtml+xml;charset=utf-8" />
        <title>Apposite</title>
    </head>
    <body>
        <div id="header"><h1>Apposite</h1></div>
        <div id="content">${screen_content}</div>
        <div id="footer"><p>&copy; Apposite 2011</p><!-- obligatory paranoid copyright notice --></div>
    </body>
</html>

… and we’re almost there. We just need a lit­tle applicationContext.xml and web.xml jiggery-pokery and we’re pretty much done:

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
    xmlns:context="http://www.springframework.org/schema/context"
    xmlns:mvc="http://www.springframework.org/schema/mvc"
    xmlns:security="http://www.springframework.org/schema/security"
    xmlns:util="http://www.springframework.org/schema/util"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
        http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-3.0.xsd
        http://www.springframework.org/schema/mvc http://www.springframework.org/schema/mvc/spring-mvc-3.0.xsd
        http://www.springframework.org/schema/security http://www.springframework.org/schema/security/spring-security-3.0.3.xsd
        http://www.springframework.org/schema/util http://www.springframework.org/schema/util/spring-util-3.0.xsd">

    <context:property-placeholder location="classpath:application.properties,classpath:environment.properties" system-properties-mode="OVERRIDE" />

    <context:component-scan base-package="org.apposite,com.christophertownson.spring.velocity" />
    
    <security:http auto-config="true" disable-url-rewriting="true" path-type="regex">
        <security:intercept-url pattern="/admin/?.*" access="${apposite.security.admin.role.name}" />
        <security:form-login login-page="/users/login" default-target-url="/" authentication-failure-url="/users/login"/>
        <security:logout logout-url="/users/logout"/>
    </security:http>
    
    <security:authentication-manager>
        <security:authentication-provider>
            <security:user-service>
                <security:user name="${apposite.security.root.user.name}" password="${apposite.security.root.user.password}" authorities="${apposite.security.admin.role.name}"/>
            </security:user-service>
        </security:authentication-provider>
    </security:authentication-manager>

    <mvc:annotation-driven />

    <mvc:default-servlet-handler />

    <util:properties id="velocityProperties" location="classpath:velocity.properties" />

    <bean id="velocityToolboxFactory" class="org.apache.velocity.tools.config.XmlFactoryConfiguration">
        <constructor-arg type="boolean" value="false" />
    </bean>

    <util:map id="mediaTypes">
        <entry key="json" value="application/json"/>
        <entry key="xml" value="application/xml"/>
        <entry key="rss" value="application/rss+xml" />
    </util:map>

    <bean class="org.springframework.web.servlet.view.ContentNegotiatingViewResolver">
        <property name="order" value="${org.springframework.web.servlet.view.ContentNegotiatingViewResolver.order}"/>
        <property name="mediaTypes" ref="mediaTypes" />
        <property name="defaultViews">
            <list>
                <bean class="org.springframework.web.servlet.view.json.MappingJacksonJsonView"/>
                <bean class="org.springframework.web.servlet.view.xml.MarshallingView">
                    <constructor-arg>
                        <bean class="org.springframework.oxm.xstream.XStreamMarshaller"/>
                    </constructor-arg>
                    <property name="contentType" value="application/xml;charset=UTF-8"/>
                </bean>
            </list>
        </property>
    </bean>

</beans>

It’s worth run­ning through this file step-by-step …

  1. On line 14, I load the prop­er­ties in order such that application.properties will be over­rid­den by environment.properties which will in turn be over­rid­den by sys­tem properties.
  2. On line 16, I load anno­tated com­po­nents from the name­spaces org.apposite (to get my Con­trollers) and com.christophertownson.spring.velocity (to get my Veloc­ity setup).
  3. On lines 18–30, I con­fig­ure a very basic Spring Secu­rity setup, using hard-coded users and pass­words in the clear. This can be improved in future. Note, how­ever, that I do exter­nalise the user and role name con­fig­u­ra­tion details into appli­ca­tion or envi­ron­ment prop­er­ties: this will make some things eas­ier as things progress.
  4. On line 32, I declare annotation-driven MVC because this is a great addi­tion to Spring that basi­cally sets up every­thing you need to use anno­tated con­trollers (as I am).
  5. On line 34, I declare the default servlet han­dler. Another good new XML short­hand, this sets up han­dling of sta­tic resources by the container’s default servlet (as it says on the tin).
  6. On line 36, I instan­ti­ate a java.util.Properties instance with the bean ID velocityProperties so that this is auto-wired into the default Veloc­ity setup achieved via the component-scan of the com.christophertownson.spring.velocity pack­age. Sim­i­larly, on lines 38–40, I instan­ti­ate a Veloc­ity tool­box fac­tory, so that I can use Veloc­ity tools using the new tool­box for­mats intro­duced with Veloc­ity Tools 2.
  7. On lines 42–6, I con­fig­ure the media types map­ping: this is a bi-directional map used by both the content-negotiating view resolver and the Veloc­ity view to deter­mine either (a) the file exten­sion to use for a requested content-type or, inversely, (b) the content-type to use for a given file exten­sion (falling back to a default content-type con­fig­ured in appli­ca­tion or envi­ron­ment prop­er­ties when no content-type file exten­sion is present). To add sup­port for new for­mats, all we need to do is add it to the map­ping and drop in cor­re­spond­ing tem­plates for any resources for which you want a rep­re­sen­ta­tion in that content-type to be available.
  8. Last, but by no means least, on lines 48–62, I instan­ti­ate the ContentNegotiatingViewResolver, con­fig­ur­ing it with the media types map and giv­ing it the MappingJacksonJsonView and XStreamMarshaller as default views (so that we can get a JSON or XML rep­re­sen­ta­tion of any URI, if we so desire).

To get the whole thing work­ing, so that the Spring appli­ca­tion con­text is fired-up when the built WAR is deployed or started, we just need a stan­dard web.xml that declares the rel­e­vant servlets and fil­ters for Spring and Spring Security:

<?xml version="1.0" encoding="UTF-8"?>
<web-app version="2.4" xmlns="http://java.sun.com/xml/ns/j2ee"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://java.sun.com/xml/ns/j2ee http://java.sun.com/xml/ns/j2ee/web-app_2_4.xsd">
    <display-name>apposite</display-name>
    <description>A publishing application</description>

    <context-param>
        <param-name>contextConfigLocation</param-name>
        <param-value>classpath:org/apposite/applicationContext.xml</param-value>
    </context-param>

    <listener>
        <listener-class>org.springframework.web.context.ContextLoaderListener</listener-class>
    </listener>

    <filter>
        <filter-name>springSecurityFilterChain</filter-name>
        <filter-class>org.springframework.web.filter.DelegatingFilterProxy</filter-class>
    </filter>

    <filter>
        <filter-name>httpMethod</filter-name>
        <filter-class>org.springframework.web.filter.HiddenHttpMethodFilter</filter-class>
    </filter>

    <filter>
        <filter-name>encoding</filter-name>
        <filter-class>org.springframework.web.filter.CharacterEncodingFilter</filter-class>
        <init-param>
            <param-name>encoding</param-name>
            <param-value>UTF-8</param-value>
        </init-param>
        <init-param>
            <param-name>forceEncoding</param-name>
            <param-value>true</param-value>
        </init-param>
    </filter>

    <filter-mapping>
        <filter-name>springSecurityFilterChain</filter-name>
        <servlet-name>apposite</servlet-name>
    </filter-mapping>

    <filter-mapping>
        <filter-name>httpMethod</filter-name>
        <servlet-name>apposite</servlet-name>
    </filter-mapping>

    <filter-mapping>
        <filter-name>encoding</filter-name>
        <servlet-name>apposite</servlet-name>
    </filter-mapping>

    <servlet>
        <servlet-name>apposite</servlet-name>
        <servlet-class>org.springframework.web.servlet.DispatcherServlet</servlet-class>
        <load-on-startup>1</load-on-startup>
    </servlet>

    <servlet-mapping>
        <servlet-name>apposite</servlet-name>
        <url-pattern>/</url-pattern>
    </servlet-mapping>

</web-app>

Note that the <servlet-name> is “appo­site” and not “spring” (or some­thing sim­i­larly generic) — it is a minor detail but this can result in some clearer log mes­sages dur­ing startup, espe­cially if you have a num­ber of Spring webapps run­ning in the same con­tainer. It also means that the DispatcherServlet will be look­ing for a WEB-INF/apposite-servlet.xml file: I could con­fig­ure it to point to the main app con­text file but, instead, I usu­ally opt to just stick an empty appli­ca­tion con­text file there — his­tor­i­cally, Spring rec­om­mend putting your con­troller and map­ping dec­la­ra­tions in this file but that becomes unnec­es­sary when you are using anno­tated con­trollers. I am not a fan of hav­ing mul­ti­ple appli­ca­tion con­text XML files, what­ever the sup­posed ratio­nale for divid­ing them up: if you find you are hav­ing to do too much “pointy-bracket-configuration”, then sim­ply hid­ing it across many files is not the answer!

In addi­tion to the DispatcherServlet (which dis­patches requests to the right con­troller), the DelegatingFilterProxy (which is here set­ting up the Spring Secu­rity fil­ter), and the ContextLoaderListener (which is start­ing the main appli­ca­tion con­text for access by the Spring Secu­rity fil­ter and dis­patcher servlet), I am also using the very use­ful HiddenHttpMethodFilter (to tun­nel POST or GET requests from browser form sub­mis­sions to more appro­pri­ate HTTP verbs) and the CharacterEncodingFilter (which does exactly what it says on the tin).

Now we are all ready to go and, given our end-to-end test­ing setup, we should be able to exe­cute mvn test -PFunctionalTests from a com­mand prompt at the project root and see the appli­ca­tion started up, fol­lowed by the web dri­ver test being run and passing.

As you may recall, I wanted to do a lit­tle more than just pass the test: I wanted to be able to get dif­fer­ent rep­re­sen­ta­tions of the same resource. Given the steps so far, it is pos­si­ble to view GET /admin in default JSON or XML views gen­er­ated by XStream or Jackson-Mapper sim­ply by adding a .xml/.json file exten­sion to the URI or (bet­ter) by issu­ing the request with an Accept: application/xml or Accept: application/json header. How­ever, as there is no real “model” (object graph) asso­ci­ated with this sim­ple page, you will begin to see some of the lim­i­ta­tions of these default views … but, sim­ply by drop­ping in a new tem­plate or two, we can selec­tively over­ride the use of these default XML/JSON views:

$screen_content

First we cre­ate a totally “neu­tral” lay­out (because oth­er­wise our default one would start try­ing to wrap our JSON or XML or what­ever inside an HTML page lay­out). Now we can cre­ate, for exam­ple, an XML tem­plate for the admin dash­board page (leav­ing aside the ques­tion of the use­ful­ness of such a rep­re­sen­ta­tion of this resource for the time being):

#set($layout="/common/layouts/nolayout.vm")<?xml version="1.0" encoding="UTF-8"?>
<admin><dashboard><title>Administration Dashboard</title></dashboard></admin>

Now, of course, that is not a stun­ningly use­ful exam­ple but I have no doubt that you can imag­ine a num­ber of bet­ter use cases your­self. Within the “Appo­site” appli­ca­tion, for instance, I have a con­cept of a “Cal­en­darEvent” within the domain model (which rep­re­sents an “adver­tised, pub­lic event sub­mit­ted by a user” as opposed to, say, an “appli­ca­tion event”). I use this form of content-negotiation to deliver HTML, Atom, RDF, RSS, and ICS rep­re­sen­ta­tions of events to the user from the same URI (with no addi­tional pro­gram­ming required), whilst also being able to pro­vide links for browsers within HTML pages by using the equiv­a­lent URI with the desired format’s file exten­sion. If you com­pare this approach to the frankly painful process of adding RSS/Atom sup­port using Spring’s own AbstractRssFeedView and Rome, I think you will be pleas­antly sur­prised. Object graphs are object graphs and text is text. Turn­ing the for­mer into the lat­ter should be a generic and abstract activ­ity that requires no pro­gram­ming, only a syn­tac­ti­cal descrip­tion of the trans­for­ma­tion (which is what a tem­plate is). Of course, there are more com­plex, binary for­mats that you may also wish to deliver for which this style of con­tent nego­ti­a­tion is not appro­pri­ate … but, then, you are not tied to it. Because the Veloc­ity View resolver lives at the end of the view resolver chain and will return null if it can­not locate an exist­ing tem­plate, you can add your View imple­men­ta­tions for more com­plex cases as default views on the ContentNegotiatingViewResolver and just use Veloc­ity selec­tively to deliver cus­tomised rep­re­sen­ta­tions of spe­cific resources. To be hon­est, I have not yet had to sat­isfy any use cases that can­not be met by this sim­ple method of con­tent nego­ti­a­tion for text-based formats.

On May 17, 2011 christopher wrote: We Are An Author”

Book cover: Site and Non-Site

The cover: even if it looks as though it should be about agri­cul­ture, it is about art and phi­los­o­phy. Honest.

To para­phrase Mar­garet Thatcher’s hilar­i­ously delu­sional Freudian slip in which she revealed her monar­chi­cal self-image, “We are an author!”

Hav­ing sat on my PhD the­sis for nearly 6 years (some­times lit­er­ally), after an email enquiry from a pub­lisher, I fig­ured I may as well put it into print. I am con­se­quently delighted to announce that you can now pur­chase my the­sis from Ama­zon at the bar­gain price of £71. Put it on your Christ­mas lists now.

For those of you who haven’t read it yet (if you are one of these few: what have you been doing all these years?!), it is basi­cally about the rela­tion­ship between the phi­los­o­phy of Mar­tin Hei­deg­ger and what is referred to as “site-specific” art. How­ever, there is also quite a bit of stuff on the phi­los­o­phy of tech­nol­ogy, which is a sub­ject that has been my increas­ing pre­oc­cu­pa­tion since completion.

From the blurb:

The notion of site-specific art is one that has been used exten­sively within art the­ory and prac­tice since the late 1960s. How­ever, in the process of its var­i­ous util­i­sa­tions and inter­pre­ta­tions the con­cept would seem to have been emp­tied almost entirely of mean­ing such that, in the words of Miwon Kwon, it becomes merely a token of crit­i­cal­ity or pro­gres­siv­ity. Many works through­out his­tory have had a spe­cific rela­tion to a site. What, if any­thing, dis­tin­guishes mod­ern and con­tem­po­rary works that draw on the con­cept or to which it is ascribed? By revis­it­ing the philo­soph­i­cal basis of the prob­lem through an analy­sis of the work of Mar­tin Hei­deg­ger and Mau­rice Merleau-Ponty, Dr Christo­pher Town­son addresses the prob­lem of the mean­ing of site-specificity as a con­cept, then elab­o­rat­ing this through case stud­ies on James Tur­rell and Robert Smith­son. As a con­se­quence, sig­nif­i­cant con­clu­sions can be drawn not only with regard to site-specific art past and present but also for art his­tory as a discipline.

Who knows? Now that I have pub­lished one, maybe I shall get into the habit. It is ter­ri­bly nar­cis­sis­tic, but there is some­thing rather pleas­ing about see­ing your hard work in print. I am very much look­ing for­ward to receiv­ing my com­ple­men­tary copy. I am really hop­ing is it has a good “new book” smell.

On May 16, 2011 christopher wrote: End-to-End Testing With Maven, Jetty & JBehave

One of those seem­ingly triv­ial top­ics of debate amongst soft­ware devel­op­ers which is liable to irk me is the sub­ject of depen­den­cies. There is noth­ing more frus­trat­ing than check­ing out a project only only to dis­cover that it has no end of “exter­nals” and other assorted envi­ron­men­tal depen­den­cies that are out­side of the con­trol of the project itself. Ide­ally, the struc­ture and build sup­port for an exe­cutable project (e.g. a web appli­ca­tion) should make it pos­si­ble for it to be run “out-of-the-box” because three sig­nif­i­cant costs are incurred where this is not the case:

  1. Time-to-start” for any new devel­oper is increased.
  2. The appli­ca­tion becomes more frag­ile through expo­sure to change in exter­nal dependencies.
  3. End-to-end func­tional, in-container test­ing of the appli­ca­tion becomes cor­re­spond­ingly more com­plex, the setup process for which now has to effec­tively doc­u­ment and keep pace with changes to exter­nal sys­tems (which, in prac­tice, are often not known until after your tests start fail­ing, lead­ing to wasted time debug­ging the cause of failure).

The argu­ment I some­times hear against my approach is that it attempts to cre­ate a mono­lithic sys­tem (the “one ring to rule them all” syn­drome) and that sep­a­ra­tion into “mod­ules” helps to cre­ate smaller, more man­age­able appli­ca­tions (smaller: maybe; more man­age­able: def­i­nitely not, in my view). “Mod­u­lar­i­sa­tion” and “service-oriented” approaches need not incur the loss of compile-time safety (which is an advan­tage of a lan­guage like Java) nor the frag­men­ta­tion of inte­gra­tion con­cerns. To demon­strate this, and for my own enjoy­ment and edu­ca­tion, I have recently been work­ing in my spare time on putting together a small web appli­ca­tion that I call “Appo­site”. In this and sub­se­quent posts, I would like to share with you what I see as some of the key struc­tural and archi­tec­tural approaches that I am adopt­ing, begin­ning with the assump­tion that the entire appli­ca­tion must always remain exc­etable as a stand­alone arte­fact using just the build script so that it is pos­si­ble to do both end-to-end func­tional test­ing of the appli­ca­tion as part of the build and to make it pos­si­ble to do “live cod­ing” (where changes in com­piled code are quickly vis­i­ble within a run­ning instance.

The appli­ca­tion itself is a very con­ven­tional Java web appli­ca­tion build using the now-standard stack of Spring and Hiber­nate. For builds, I am using Maven. For test­ing, I am using JUnit (of course), Web­Driver, and JBe­have (whilst arguably not as good as cucum­ber it is eas­ier to use with Java projects and fits more nicely with my “out-of-the-box” non-functional require­ments). As you can see, hardly ground­break­ing stuff … but some­thing I still see done “wrong” in so many places and by so many peo­ple (which is some­what inve­vitable by virtue of its popularity).

Get­ting the basic project struc­ture in place

Let’s begin from the very basics, assum­ing a stan­dard Maven web appli­ca­tion project structure:

  • appo­site
    • pom.xml
    • src
      • main
        • java
        • resources
        • webapp
          • WEB-INF
            • web.xml
      • test
        • java
        • resources

In the pom.xml, we declare the test­ing depen­den­cies we are going to be using, start­ing with JUnit:

        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>4.8.2</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.jbehave</groupId>
            <artifactId>jbehave-core</artifactId>
            <version>3.3.2</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.seleniumhq.selenium</groupId>
            <artifactId>selenium-common</artifactId>
            <version>${selenium.version}</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.seleniumhq.selenium</groupId>
            <artifactId>selenium-firefox-driver</artifactId>
            <version>${selenium.version}</version>
            <scope>test</scope>
            <exclusions>
                <exclusion>
                    <groupId>commons-logging</groupId>
                    <artifactId>commons-logging</artifactId>
                </exclusion>
            </exclusions>
        </dependency>
        <dependency>
            <groupId>org.seleniumhq.selenium</groupId>
            <artifactId>selenium-support</artifactId>
            <version>${selenium.version}</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.springframework</groupId>
            <artifactId>spring-test</artifactId>
            <version>3.0.5.RELEASE</version>
            <scope>test</scope>
        </dependency>

Func­tional tests are inher­ently slower to run that unit tests and we do not nec­es­sar­ily want to run them all the time. There­fore, we want to exe­cute them only dur­ing Maven’s inte­gra­tion test phase and, even then, only when we spec­ify a func­tional test pro­file. To achieve this, we begin by adding the maven-failsafe-plugin to the build sec­tion of our POM:

            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-failsafe-plugin</artifactId>
                <version>2.8</version>
            </plugin>

By default, this will run JUnit tests that match the pat­tern **/*IT.java dur­ing the inte­gra­tion test phase of the build. You can stick with the default, how­ever I pre­fer the slightly more descrip­tive nam­ing con­ven­tion **/*FunctionalTest.java — that can yield slightly over-long test names but it is at least blind­ingly clear what sort of test your test class is! To ensure my pre­ferred test nam­ing con­ven­tion does not con­flict with the stan­dard sure­fire plu­gin defaults, I con­fig­ure excludes and includes in the surefire-plugin:

            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-plugin</artifactId>
                <version>2.8.1</version>
                <configuration>
                    <includes>
                        <include>**/*Test.java</include>
                    </includes>
                    <excludes>
                        <exclude>**/*FunctionalTest.java</exclude>
                    </excludes>
                </configuration>
            </plugin>

By default in Maven, the webapp source folder is not on the test class­path. It makes this kind of in-container func­tional test­ing much eas­ier if it is. To place the webapp folder on the class­path, you can use the build-helper-maven-plugin:

            <plugin>
                <groupId>org.codehaus.mojo</groupId>
                <artifactId>build-helper-maven-plugin</artifactId>
                <version>1.5</version>
                <executions>
                    <execution>
                        <id>add-test-resource</id>
                        <phase>generate-test-sources</phase>
                        <goals>
                            <goal>add-test-resource</goal>
                        </goals>
                        <configuration>
                            <resources>
                                <resource>
                                    <directory>src/main/webapp</directory>
                                </resource>
                            </resources>
                        </configuration>
                    </execution>
                </executions>
            </plugin>

Next in the build plu­g­ins we need to declare and con­fig­ure the jetty-maven-plugin so that we can fire-up the entire web appli­ca­tion using mvn jetty:run:

            <!-- You need to specify -Djetty.port=${port} if 8080 is already bound on the build machine -->
            <plugin>
                <groupId>org.mortbay.jetty</groupId>
                <artifactId>jetty-maven-plugin</artifactId>
                <version>7.2.0.v20101020</version>
                <dependencies>
                    <!-- you can declare what would commonly be your container-provided dependencies here, such as log4j etc -->
                </dependencies>
                <configuration>
                    <webAppConfig>
                        <contextPath>/${project.artifactId}</contextPath>
                    </webAppConfig>
                    <jettyConfig>src/test/resources/jetty.xml</jettyConfig>
                    <useTestClasspath>true</useTestClasspath>
                    <scanIntervalSeconds>10</scanIntervalSeconds>
                    <stopKey>${project.artifactId}-stop</stopKey>
                    <stopPort>9999</stopPort>
                </configuration>
            </plugin>

Note the use of the jetty.xml jet­ty­Con­fig there: I use this to declare a JNDI data­source (this needs to be a jetty server con­fig — using a jetty web appli­ca­tion con­fig will result in hor­rific mem­ory leaks around data­base con­nec­tions with the reg­u­lar restarts that you may well want if you are doing “live cod­ing” against a run­ning jetty instance using this build con­fig) so that all my app has to know is the JNDI name and the actual details of this will always be encap­su­lated within the con­tainer (in this case, the test har­ness). In Appo­site, I am using an in-memory HSQLDB data­base for test­ing, so I declare c3p0 and HSQLDB as container-provided depen­den­cies of the jetty plu­gin and my jetty.xml looks like this:

<?xml version="1.0"  encoding="UTF-8"?>
<!DOCTYPE Configure PUBLIC "-//Mort Bay Consulting//DTD Configure//EN" "http://jetty.mortbay.org/configure.dtd">
<Configure class="org.eclipse.jetty.server.Server">
    <New class="org.eclipse.jetty.plus.jndi.Resource">
        <Arg>jdbc/apposite</Arg>
        <Arg>
            <New class="com.mchange.v2.c3p0.ComboPooledDataSource">
                <Set name="driverClass">org.hsqldb.jdbcDriver</Set>
                <Set name="jdbcUrl">jdbc:hsqldb:mem:apposite</Set>
                <Set name="user">sa</Set>
                <Set name="password"></Set>
            </New>
        </Arg>
    </New>
</Configure>

The final touch in the POM is to setup a func­tional tests pro­file so that we can exe­cute the in-container tests using mvn test -PFunctionalTests (or -PWhateverYouCallYourProfile):

        <profile>
            <id>FunctionalTests</id>
            <activation>
                <activeByDefault>false</activeByDefault>
            </activation>
            <build>
                <plugins>
                    <plugin>
                        <groupId>org.apache.maven.plugins</groupId>
                        <artifactId>maven-failsafe-plugin</artifactId>
                        <configuration>
                            <includes>
                                <include>**/*FunctionalTest.java</include>
                            </includes>
                        </configuration>
                        <executions>
                            <execution>
                                <id>integration-test</id>
                                <goals>
                                    <goal>integration-test</goal>
                                </goals>
                            </execution>
                            <execution>
                                <id>verify</id>
                                <goals>
                                    <goal>verify</goal>
                                </goals>
                            </execution>
                        </executions>
                    </plugin>
                    <plugin>
                        <groupId>org.mortbay.jetty</groupId>
                        <artifactId>jetty-maven-plugin</artifactId>
                        <executions>
                            <execution>
                                <id>start-jetty</id>
                                <phase>pre-integration-test</phase>
                                <goals>
                                    <goal>run</goal>
                                </goals>
                                <configuration>
                                    <scanIntervalSeconds>0</scanIntervalSeconds>
                                    <daemon>true</daemon>
                                </configuration>
                            </execution>
                            <execution>
                                <id>stop-jetty</id>
                                <phase>post-integration-test</phase>
                                <goals>
                                    <goal>stop</goal>
                                </goals>
                            </execution>
                        </executions>
                    </plugin>
                </plugins>
            </build>
        </profile>

In this pro­file, we tell the jetty plu­gin to fire up our webapp just before the inte­gra­tion test phase starts (and to stop it once it is com­plete) and inform the fail­safe plu­gin that it should exe­cute tests that match the nam­ing con­ven­tion **/*FunctionalTest.java (you could, of course, avoid hav­ing to have this bit of con­fig­u­ra­tion by sim­ply accept­ing the default **/*IT.java con­ven­tion … but I have some­thing of a dis­like of “pub­lic abbre­vi­a­tions” like this). Note also that the scanIntervalSeconds prop­erty is set to 0: we do not want jetty acci­den­tally detect­ing some change on the class­path due to code gen­er­at­ing some resource there and restart­ing mid-test as a con­se­quence. Set­ting this prop­erty to 0 ensures this by turn­ing off the jetty plu­gin change scan. This over­rides the set­ting in the build plu­gin con­fig­u­ra­tion (where we set it to 10 sec­onds) which was intended to have the oppo­site effect: when we do “live cod­ing” against a run­ning jetty instance, we want it to pick-up and deploy our code changes regularly.

As usual with Maven, there is a bit of an excess of pointy brack­ets here … but once you have a use­ful project struc­ture you can always turn it into an arche­type, thus avoid­ing the need to have to recre­ate it by hand every time.

Cre­at­ing a “frame­work” for the func­tional tests

We now have a web appli­ca­tion (albeit with no actual code) that we can fire up and run from our build script and which will auto­mat­i­cally run cer­tains tests in-container dur­ing the build if and when we so choose. Before we plough ahead, we should stop and give a lit­tle thought to how we want to divide up our func­tional tests and what kind of sup­port infra­struc­ture they might ben­e­fit from.

The first thought that occurred to me at this point is “I’m using Spring already so surely I must be able to re-utilise it to make man­ag­ing my test code eas­ier?” This is indeed pos­si­ble but, if you are using annotation-based Spring con­text con­fig­u­ra­tion (as I am), then it is a very good idea to use a com­pletely sep­a­rate name­space for your func­tional test “con­text” to ensure there is no chance of it becom­ing mixed up with your real appli­ca­tion. In the case of Appo­site, my appli­ca­tion name­space is org.apposite. There­fore, rather than use org.apposite.bdd (or sim­i­lar), I opted for bdd.org.apposite: no chance of a con­flict, as it is not a sub­set of the appli­ca­tion name­space. I began by mak­ing a min­i­mal appli­ca­tion con­text con­fig that would pro­vide pretty much all the sup­port­ing infra­struc­ture I would need for my tests:

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xmlns:context="http://www.springframework.org/schema/context"
    xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
    http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-3.0.xsd">

    <context:property-placeholder location="classpath:application.properties,classpath:environment.properties" system-properties-mode="OVERRIDE" />

    <context:component-scan base-package="bdd.org.apposite" />

    <bean id="web" class="org.openqa.selenium.firefox.FirefoxDriver" scope="prototype" destroy-method="close" />

</beans>

This file achieves the following:

  1. Pro­vides access to “appli­ca­tion” and “envi­ron­ment” prop­er­ties (of the real appli­ca­tion) from src/main/resources/application.properties and src/test/resources/environment.properties, respec­tively, so that these val­ues can be utilised to con­struct test cases and asser­tions. Note that the environment.properties defines prop­er­ties spe­cific to the con­tainer or the envi­ron­ment and would nor­mally be pro­vided by the tar­get deploy­ment con­tainer. The ver­sion in src/test/resources there­fore repli­cates this require­ment for the test con­tainer whilst also serv­ing as a form of doc­u­men­ta­tion, thus help­ing me to keep these two dis­tinct areas of con­fig­u­ra­tion entirely seper­ate whilst main­tain­ing runnabil­ity “out-of-the-box”.
  2. Instan­ti­ates anno­tated com­po­nents within the test name­space only via context:component-scan.
  3. Pro­vides access to Web­Driver (I’m using the fire­fox dri­ver here). Notice that the scope is prototype so that each test will get it’s own new instance. Here I also spec­ify a destroy-method — that is a bit of “belt & braces” para­noia to try and ensure that we do not leave Fire­fox win­dows open on com­ple­tion of a test case (it doesn’t actu­ally achieve that due to the nature of the Spring bean life­cy­cle in rela­tion to the test exe­cu­tion, but out of pure super­sti­tion I felt it bet­ter defined than not, if you know what I mean).

Next, I started think­ing about how I wanted to divide up my tests. This is worth doing if you are using some­thing like JBe­have because one of the first things you will need to do is to tell it how to load “story files”. Con­se­quently, know­ing which story files to load is impor­tant. If you sim­ply load all your story files in one go (which is an option) you will have one mas­sive func­tional test. That may not nec­es­sar­ily be a prob­lem, but it does limit things some­what, espe­cially in terms of report­ing, and may quickly become unwieldy for any­thing but the small­est and most sim­ple of appli­ca­tions. In line with gen­eral BDD guide­lines, I wanted to split my tests up into “func­tional areas” within which one or more user sce­nar­ios could be encap­su­lated (for exam­ple, “reg­is­tra­tion”). I opted to go for a one-to-one rela­tion between a func­tional test class and story file because then, once a basic test exe­cu­tion mech­a­nism was in place, I would sim­ply be able to write the story file and cre­ate a sim­ple test class that spec­i­fied the story file to run. I would then see these as indi­vid­u­ally exe­cuted tests within both the Maven inte­gra­tion test phase and in the JBe­have report­ing. I felt the cost of hav­ing to pro­duce at least one “no code” class per story file was off­set by the flex­i­bil­ity this approach would pro­vide. After writ­ing a few tests, I extracted the fol­low­ing through a process of refactoring:

@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration("classpath:bdd/org/apposite/functional-tests.xml")
public abstract class AbstractFunctionalTest extends Embedder {

    @Autowired private GenericApplicationContext ctx;

    private final String includes;

    protected AbstractFunctionalTest(String storyFile) {
        includes = "bdd/org/apposite/" + storyFile;
    }

    @Test public void runStories() {
        useCandidateSteps(new InstanceStepsFactory(configuration().useStoryReporterBuilder(new StoryReporterBuilder().withCodeLocation(codeLocationFromClass(getClass())).withFormats(CONSOLE, TXT, XML, HTML)), ctx.getBeansWithAnnotation(Steps.class).values().toArray()).createCandidateSteps());
        runStoriesAsPaths(new StoryFinder().findPaths(codeLocationFromClass(getClass()), includes, ""));
    }

}

This class extends org.jbehave.core.embedder.Embedder, which is the key thing in enabling it to become an exe­cutable JUnit test that runs JBe­have sto­ries. (We also have to over­ride some of JBehave’s frankly ter­ri­ble report­ing defaults!) It utilises the Spring test­ing frame­work and the test con­text described above to allow autowiring of JBe­have “steps” (which are just POJOs anno­ta­tion with @Given, @When, and @Then meth­ods that are matched against the cor­re­spond­ing lines from a story file. I cre­ated a cus­tom Spring com­po­nent stereo­type called @Steps:

@Target({ElementType.TYPE})
@Retention(RetentionPolicy.RUNTIME)
@Component
public @interface Steps {
    String value() default "";
}

All Spring beans anno­tated with the @Steps anno­ta­tion are pre­sented to JBe­have as can­di­date steps for the exe­cu­tion of a story file which is spec­i­fied by a con­crete sub­class. At present, there is no clever fil­ter­ing of can­di­date steps and all story files are assumed at the very least to live within the bdd.org.apposite name­space. You could prob­a­bly be much more sophis­ti­cated if nec­es­sary, but this has proved suf­fi­cient for my require­ments so far.

Writ­ing the tests

Now we are in a posi­tion to write a basic story file. Let’s start with a “secu­rity” fea­ture, for exam­ple: “I should not be able to access the admin­is­tra­tion dash­board unless I am logged in and have suf­fi­cient priv­i­leges”. Whilst per­haps not the best exam­ple of a “func­tional area” (because “secu­rity” is in real­ity an con­stituent com­po­nent of other func­tional areas and cuts across them) it will nonethe­less suf­fice for now:

Scenario: I cannot access the administration dashboard unless I am logged in

Given I am not logged in
When I go to the administration dashboard
Then I am asked to login
Then I enter the administrator credentials
Then I am redirected to the administration dashboard

This is an extremely basic exam­ple of a story file. For a full descrip­tion of all the pos­si­ble fea­tures of story files in JBe­have, check out their doc­u­men­ta­tion (which, whilst it appears com­pre­hen­sive, does not win any prizes for clar­ity). Noneth­less, I’m sure you get the gist: each sce­nario con­sists of some pre­con­di­tions, an oper­a­tion, and some post­con­di­tions. You can have as many sce­nar­ios per “story file” as you like, sep­a­rated by scenario dec­la­ra­tions (this is why I opted to use the *.stories exten­sion rather than the more com­mon *.story — the for­mer made more gram­mat­i­cal sense to me).

Next, because I opted for a one-to-one rela­tion between story files and exe­cutable tests, you need to cre­ate a very sim­ple class that will indi­cate that this story file needs to be run:

public class SecurityFunctionalTest extends AbstractFunctionalTest {

    public SecurityFunctionalTest() {
        super("security.stories");
    }

}

Bingo! We have a behaviour-driven func­tional test. “But where the hell is all the logic?” I am sure you are ask­ing (if you are, it is, of course, the right ques­tion … and I shall move onto that now).

Organ­is­ing respon­si­bil­i­ties into page objects

The good peo­ple behind Web­Driver (and almost any­one else who has done this kind of test­ing) rightly rec­om­mend that you take the impor­tant step of describ­ing your web appli­ca­tion in terms of “page objects”. A page object should, loosely speak­ing, cor­re­spond to the response from a given URI and encap­su­late the “ser­vices” (forms, links, key infor­ma­tion) that it pro­vides. In the test case above, I am inter­ested in two pages:

  1. A “logout” page (this is the least obvi­ous but bear in mind that we need to encap­su­late access to a URI that will ensure that we are not logged in to com­plete the first step)
  2. The “admin­is­tra­tion dash­board” page
  3. The “login” page

Clearly, there are going to be many things that are com­mon to all pages (even if you have a very inco­her­ent user inter­face). For exam­ple, at the very least, you should be able to “visit” all pages. Also, for test­ing pur­poses, we should be able to assert what page we are cur­rently on. There­fore, I will cut to the chase and begin with an abstract super­class for them all that can be used to describe these com­mon features:

public abstract class AbstractPage {

    private static final int DEFAULT_PORT = 8080;

    private String url;

    private WebDriver web;

    protected AbstractPage(String uri, WebDriver web) {
        this.web = web;
        int port = System.getProperty("jetty.port") != null ? Integer.valueOf(System.getProperty("jetty.port")) : DEFAULT_PORT;
        url = "http://localhost:" + port + "/apposite" + uri;
    }

    public void assertIsCurrentPage() {
        assertThat(isCurrentPage(), is(true));
    }

    public abstract boolean isCurrentPage();

    public final void visit() {
        web.get(url);
    }

There is a bug in maven-surefire-plugin < 2.6 which will pre­vent the sys­tem prop­erty being avail­able to the test here, so will have to hard-code the port on which you run your func­tional tests or upgrade. See SUREFIRE-121.

Our abstract page is respon­si­ble for man­ag­ing the Web­Driver instance (which it requires for instan­ti­a­tion), and coor­di­nat­ing what port, host, and con­text path we are run­ning on (the lat­ter two hard­coded here for the sake of sim­plic­ity but eas­ily exter­nal­is­able through sys­tem prop­er­ties if nec­es­sary). This means that con­crete page instances only need to spec­ify what URI they have (with­out hav­ing to con­sider con­text paths etc) and the abstract super­class can per­form all the actual “get­ting” and so forth.

Next, we define our actual, con­crete pages:

public class AdministrationDashboardPage extends AbstractPage {

    @FindBy(css = "h2") private WebElement heading;

    public AdministrationDashboardPage(WebDriver web) {
        super("/admin", web);
    }

    @Override public boolean isCurrentPage() {
        return "Administration Dashboard".equals(heading.getText());
    }

}

This first page def­i­n­i­tion is extremely sim­ple. Using the “finder” anno­ta­tions pro­vided by the selenium-support arte­fact, we use the pres­ence of some head­ing text to deter­mine whether or not we are actu­ally on the admin dash­board page. I will come back to the @FindBy anno­ta­tion in a lit­tle while.

public class LogoutPage extends AbstractPage {

    public LogoutPage(WebDriver web) {
        super("/users/logout", web);
    }

    @Override public boolean isCurrentPage() {
        return false;
    }

}

The logout page is even more sim­ple because we should only ever need to “visit” it and we should never actu­ally be “on it”. It is just a URI.

public class LoginPage extends AbstractPage {

    @FindBy(css = "#j_username") private WebElement username;

    @FindBy(css = "#j_password") private WebElement password;

    public LoginPage(WebDriver web) {
        super("/users/login", web);
    }

    public void enterUsername(String username) {
        this.username.sendKeys(username);
    }

    public void enterPassword(String password) {
        this.password.sendKeys(password);
    }

    public void login() {
        password.submit();
    }

    @Override public boolean isCurrentPage() {
        return username != null && password != null;
    }

}

The login page shows a lit­tle more about how page objects are intended to be used in that it encap­su­lates access to crit­i­cal form ele­ments (again, injected using the Web­Driver anno­ta­tions) in what can be viewed as “ser­vice meth­ods”. This means that any change in the page should only ever require an update to one code loca­tion. How­ever, it does also place con­sid­er­able impor­tance on being able to accu­rately iden­tify what “a page” really is in your appli­ca­tion (this becomes more com­plex when you are deal­ing with asyn­chro­nous JavaScript mak­ing calls to HTTP “ser­vices” from within a supposed-page — these ser­vices are, in effect, pages them­selves even if the human end-user never sees them in-the-raw — so keep an open mind about the def­i­n­i­tion of a page there!)

Ora­gan­is­ing logic into steps objects

Finally, we need to cre­ate imple­men­ta­tions for the “steps” which are detailed in our “story file” (one of the nice fea­tures of JBe­have is that you can tell it not to fail tests where imple­men­ta­tions have not yet been done — mark­ing these as “pend­ing” in the reports — this way one bunch of peo­ple can get busy writ­ing sto­ries which do not have to be com­mit­ted at the same time as the imple­men­ta­tions, open­ing the pos­si­bil­ity of these two tasks being com­pleted by sep­a­rate groups; e.g. “busi­ness ana­lysts” on the one hand and soft­ware devel­op­ers on the other).

Again, there are cer­tainly going to be a num­ber of com­mons aspects to these steps objects, so you can begin with an abstract class. Mine cur­rently looks like this:

@Scope(BeanDefinition.SCOPE_PROTOTYPE) public abstract class AbstractSteps {

    protected final WebDriver web;

    protected AbstractSteps(WebDriver web) {
        this.web = web;
    }

    @AfterScenario public void closeWebDriver() {
        web.close();
    }

    protected final <T extends AbstractPage> T initPage(Class<T> pageClass) {
        return PageFactory.initElements(web, pageClass);
    }

}

Notice that, because my steps are going to be man­aged by the functional-tests.xml Spring con­text, I can define them as pro­to­type using @Scope so that we get a fresh instance wher­ever it is required with a cor­re­spond­ingly fresh instance of the pro­to­type Web­Driver bean, which is autowired in. I define only one com­mon method at present, which pro­vides a strongly-typed means to instan­ti­ate a Web­Driver page object (to use the @FindBy Web­Driver anno­ta­tions, you have to instan­ti­ate the page using the PageFactory.initElements(WebDriver, Object) method). Addi­tion­ally, I make absolutely sure that the fire­fox win­dow gets closed by using a JBe­have @AfterScenario method to close it (this is almost directly equiv­a­lent to JUnit’s @After anno­ta­tion). Now I am in a good posi­tion to start writ­ing steps classes that shouldn’t need to worry about too much extra­ne­ous fluff. Let’s take a look at the SecuritySteps that I wrote to imple­ment my basic story file described above:

@Steps public class SecuritySteps extends AbstractSteps {

    private String username;

    private String password;

    @Autowired public SecuritySteps(WebDriver web) {
        super(web);
    }

    @Given("I am not logged in")
    public void iAmNotLoggedIn() {
        initPage(LogoutPage.class).visit();
    }

    @When("I go to the administration dashboard")
    public void iGoToTheAdministrationDashboard() {
        initPage(AdministrationDashboardPage.class).visit();
    }

    @Then("I am asked to login")
    public void iAmAskedToLogin() {
        initPage(LoginPage.class).assertIsCurrentPage();
    }

    @Then("I enter the administrator credentials")
    public void iEnterTheAdministratorCredentials() {
        LoginPage p = initPage(LoginPage.class);
        p.enterUsername(username);
        p.enterPassword(password);
        p.login();
    }

    @Then("I am redirected to the administration dashboard")
    public void iAmRedirectedToTheAdministrationDashboard() {
        initPage(AdministrationDashboardPage.class).assertIsCurrentPage();
    }

    @Autowired public void setUsername(@Value("${apposite.security.root.user.name}") String username) {
        this.username = username;
    }

    @Autowired public void setPassword(@Value("${apposite.security.root.user.password}") String password) {
        this.password = password;
    }

}

Some things to notice about this class:

  1. It is anno­tated with my cus­tom @Steps anno­ta­tion so that the AbstractFunctionalTest will pick it up from the appli­ca­tion con­text and present it as can­di­date steps for the story file.
  2. Each method (exclud­ing the set­ters) cor­re­sponds to one of the state­ments from the story file: they are matched on the anno­ta­tion text. There are far more funky things one can do in terms of para­me­ter injec­tion here — this is just the most basic exam­ple possible.
  3. Because I decided to use Spring to man­age my steps classes, I am able not only to autowire any Web­Driver instance I choose to use, but also con­fig­u­ra­tion val­ues from the appli­ca­tion or envi­ron­ment prop­er­ties files (using @Value annotations)

Con­clu­sion

All of this is might seem like quite a lot of code to achieve the sim­ple goal of run­ning in-container func­tional tests. Nat­u­rally, there are ways of doing it with less code. How­ever, most of the above is sim­ply infra­struc­ture which can be extracted and shared between mul­ti­ple projects and is intended to pro­vide a har­ness that min­imises the amount of work required to add test cases, which, being the most numer­ous “code type” should ide­ally require the least amount of actual code indi­vid­u­ally. Both steps and page objects might fea­si­bly con­sid­ered as a form of “shared library”: re-use should most def­i­nitely be a goal and, once you begin to get a more com­pre­hen­sive col­lec­tion, you should not be required to always write new page objects or step imple­men­ta­tions in order to write new test cases (but don’t move them out of the project until you absolutely have to!)

Con­se­quently, with a lit­tle up-front work and thought, it is pos­si­ble to reduce the costs of adding test cases over time. This is often cited as a goal. Sadly, it is too often the case in prac­tice that a some­what lack­adaisi­cal approach at the out­set leads to this aim being con­founded: test har­nesses become brit­tle, test code becomes more com­plex than the code it tests and has bugs, tests start to blink, the whole thing becomes dis­or­gan­ised and confused.

My aim through­out has been to bal­ance flex­i­bil­ity with reg­u­lar­ity and pre­dictabil­ity: make the con­ven­tions clear and repeat­able but not too rigid. From the per­spec­tive of the build cycle, one of the keys to achiev­ing that pre­dictabil­ity is mak­ing sure that devel­op­ers can always run these kinds of test when­ever they need to (reduc­ing the time to feed­back). Hav­ing a build infra­struc­ture such as the one described here helps devel­op­ers to avoid not only “break­ing the build” but also “break­ing the functionality”.

Clearly, there is code to write in order to actu­ally make this test pass — which I shall hope­fully come on to in sub­se­quent posts — but that is one of the really nice things about BDD and func­tional test­ing of this kind: if you are using an agile method­ol­ogy (which you should be), then your JBe­have sto­ries, whilst not absolutely equiv­a­lent, should bear a close rela­tion to your iter­a­tion story cards. From this per­spec­tive, they can serve as a “def­i­n­i­tion of done” for any given ver­ti­cal slice of func­tion­al­ity: you can pro­ceed using TDD at a unit test level — the tests will fail & fail & fail — but, then, once all the var­i­ous units are com­plete, your end-to-end test will pass and you know you are done … or, I should say, you know you are “good enough” because you shouldn’t for­get to do a lit­tle refac­tor­ing and code-tidying before you con­sider it truly complete.

On May 14, 2011 christopher wrote: A Different Way to Integrate Velocity with Spring

I have often thought that the inte­gra­tion with Veloc­ity cur­rently pro­vided by Spring, whilst ade­quate, could be sig­nif­i­cantly improved so, recently, I spent a fun week­end cre­at­ing a cus­tom inte­gra­tion wrap­per, which I am call­ing spring-velocity for using Veloc­ity with Spring (sur­pris­ingly enough).

I had 4 main aims in this exercise:

  1. Sup­port the changes to Veloc­ity & Veloc­ity Tools (esp. tools) since 1.5 and 2.0 (new tool­box for­mat, no dep­re­ca­tion mes­sages etc)
  2. Pro­vide a stan­dard, Spring-style means to utilise Spring con­text sup­port to aug­ment veloc­ity tools infra­struc­ture (@ViewHelper anno­ta­tion as a com­po­nent stereo­type which can have a Spring @Scope spec­i­fied auto­mat­i­cally added to Veloc­ity context)
  3. Most impor­tantly: work nicely with ContentNegotiatingViewResolver so that a sin­gle velocity-based Spring View can be used to gen­er­ate mul­ti­ple text-based for­mats sim­ply by drop­ping in addi­tional tem­plates (+ spec­i­fy­ing in ContentNegotiatingViewResolver’s medi­aTypes map)
  4. Get all of the above to work trans­par­ently, out-of-the-box via anno­ta­tions, and cor­rectly deter­mine what sort of “con­text” the app is run­ning in (e.g. web app [access to servlet con­text] or plain token replacer with/without toolbox)

The source code and bina­ries can be down­loaded from my maven repos­i­tory.

It is cur­rently released under the GPL3 licence. How­ever, if any­one in the “real world” finds it use­ful and this licence becomes too restric­tive, I may switch to an Apache licence (in line with both Spring and Veloc­ity) in a future release, if there are any. Sim­i­larly, if it seems to other peo­ple that this is some­thing that may be gen­er­ally use­ful, I would prob­a­bly also move the source to github.

Any thoughts or feed­back would be most wel­come … and, of course, feel free to use.