What you should know:

• Metadata for scaffoling your GUI (described here).

What you need to compile/run the examples:

• java1.5 or newer
• opensource framework: agimatec-validation : download agimatec-validation.jar
• optional: javax.servlet.jar (servlet-api)
• optional: opensource frameworks/tools: agimatec-tools : download annomark-dist.zip and annogen.jar

1. Create model classes

We start with a plain java class, our model, used to transport attributes from the server to the GUI. (You can use DWR for marshaling object instances between Java and JavaScript/JDOM).

Example class “Business”:

import javax.persistence.*;

public class Business {
@Id
@Column(name=”business_id”, nullable=false, unique=true)
Long businessId;
@Column(name=”name”, nullable=false, unique=true, length=200)
String name;
@Column(name=”active”, nullable=false)
boolean active;
// getters, setters follow…
}

2. Write or generate xml meta data

You can write the xml meta data manually, or – if your classes have javax.persistence.* annotations – you can generate them using agimatec-tools/annomark.jar (described here):

<bean id=”com.agimatec.Business” impl=”com.agimatec.Business”>
<feature key=”mainKey”>
<value class=”string”>businessId</value>
</feature>
<property name=”businessId”>
<feature key=”uniqueKey”>
<value class=”boolean”>true</value>
</feature>
</property>
<property name=”name” mandatory=”true” maxLength=”200″>
<feature key=”uniqueKey”>
<value class=”boolean”>true</value>
</feature>
</property>
<property name=”active” mandatory=”true”/>
</bean>

Store the file as “beaninfos-default.xml” in the classpath.

2. Access the MetaBean

This requires agimatec-validation.jar to compile:

import com.agimatec.validation.MetaBeanManagerFactory;
import com.agimatec.validation.model.MetaBean;

MetaBeanManagerFactory.getRegistry().
addResourceLoader(“beaninfos-default.xml”);

MetaBean metabean = MetaBeanManagerFactory.getFinder().findForClass(Business.class);

3. Generate JSon for metabean(s)

We want to use the meta data in the AJAX layer, to have information about the properties, their types and additional features (mandatory, max-length, …) as a JSON formatted string. agimatec-validation offers a JSONGenerator, based on a freemarker template:

import com.agimatec.validation.json.JSONGenerator;

JSONGenerator generator = new JSONGenerator();
String json = generator.toJSON(metabean);
System.out.println(json);

This prints this JSON string:

agimatec.namespace(“agimatec.metadata”);

(function(){

var metaBean0 = {
“id” : “com.agimatec.Business”,
“beanClass” : “com.agimatec.Business”,
“name” : “Business”,
“features” :{   “mainKey” : “businessId”},
“properties” :{
“active”:{
“name” : “active”,
“type” : “boolean”,
“features” : {       “mandatory” : true       }},
“businessId”:{
“name” : “businessId”,
“type” : “long”,
“features” : {       “uniqueKey” : true       }},
“name”:{
“name” : “name”,
“type” : “java.lang.String”,
“features” : {       “maxLen” : 200,       “mandatory” : true,       “uniqueKey” : true       }},
“version”:{
“name” : “version”,
“type” : “int”,
“features” : {       }}}};

agimatec.metadata.metaBeans = {“com.agimatec.Business” : metaBean0};})();

4. Write a servlet to deliver JSON for the AJAX GUI

The java code for the servlet should not be a surprise, for it consists of what we have seen so far:

public void init() throws ServletException {
super.init();
MetaBeanManagerFactory.getRegistry().addResourceLoader(“beaninfos-default.xml”);
}

public void service(HttpServletRequest servletRequest, HttpServletResponse res)
throws ServletException, IOException {
Map<String, MetaBean> metaBeans = MetaBeanManagerFactory.getFinder().findAll();
// output JSON:
res.setContentType(“text/javascript”);
PrintWriter writer = res.getWriter();
String json = new JSONGenerator().toJSON(metaBeans.values());
writer.write(json);
}

Thats basically all! With this infrastructure your AJAX GUIs have access to meta information about the model classes.

5. Enrich meta data with information for the GUI-layer

You can enrich the xml meta data with additional hints (field sequence, formats, …) – whatever your GUI needs. If hhe additional information cannot be generated automatically, you should keep them in a separate xml-file:

<bean id=”com.agimatec.Business”>
<feature key=”DESCRIPTION”>
<value class=”string”>{name}</value>
</feature>
<feature key=”SORTING”>
<value class=”list”>
<string>name</string>
<string>active</string>
</value>
</feature>
<!–Visualisation as table–>
<feature key=”TABLE_COLUMNS”>
<value class=”list”>
<string>name</string>
<string>active</string>
</value>
</feature>
</bean>

Save this as “beaninfos-gui.xml ” in the classpath and register the file in the init() method of the servlet, too:

MetaBeanManagerFactory.getRegistry().addResourceLoader(“beaninfos-gui.xml”);

You see, that the JSON code will contain the additional information as well, because the MetaBeanManager merges them.

So far, we have the infrastructure to support meta data that allows us to build generic AJAX GUIs. The JSON string can be cached, because it will never change while the application is running.

6. User-specific meta data

We can go one step further: We can also support user-specific meta data! A usual requirement in a web application is, that – depending on the role of a user that is logged in – field-level permissions must be supported:
User A is allowed to change the “active”-state of our “Business” entity, while User B has read-only access to the field and the field is hidden for User C, …

This could also be useful, when a user can customize its GUI (disable fields) by himself or when the application is used by different companies with different settings for their users….

How does it work? – Use additional xml files and merge them in the JSON servlet. (You can also generate the meta beans during runtime programmatically).

beaninfos-role1.xml:

<bean id=”com.agimatec.Business”>
<property name=”active”>
<feature key=”readonly”>
<value class=”boolean”>true</value>
</feature>
</property>
</bean>

Extension of the servlet:

String customInfos = “beaninfos-role1.xml”; // determine this from session or request dynamically!
XMLMetaBeanInfos xmlInfos =
new XMLMetaBeanURLLoader(getClass().getResource(customInfos)).load();
Map<String, MetaBean> metaBeans =
MetaBeanManagerFactory.getEnricher().enrichCopies(xmlInfos);

The statements after “// output JSON” stay the same (see previous code fragment).

More information?

We already got some more posts in German about metadata and code generation, explaining the concepts and giving examples:

• TransferObjects generieren
• agimatec-tools als OpenSource auf google.code released!
• Statt Modellieren: Annotieren und generieren
• Klassen mit Mehrwert: Validierung und Metadaten

The open-source frameworks have a WIKI and additional examples or junit testcases, that help you.

If you have questions, do not hesitate to contact us!

This is the fifth article in our mini-series about developing highly ajaxified portlets. Other articles are:

• Generate Your UI Using Metadata
• Inter Portlet Communication
• Using YUI loader for your own javascript modules
• Portlet namespaces and javascript

This is the situation we are in. And here is how we ease the pain: we are generating the user interface.

Generating a Form

To generate a form using our framework, you need to tell it :

• what should be edited
• the type of what is edited

Say we are building a form to edit a user object that contains user ip addresses:

var user = {
	name:"Sebastian Schuth",
	ipAddress:"192.168.1.33",
	active:true
}

The form fields needed are described using object literals:

var fields = [
	{
		key:'name', //name of the edited property
		label:'User name', //label to use in the form
		type: 'string',//type information
		permissioning:{denied: false, readonly: false}
	},
	{
		key:'active',
		label:'User is active',
		type: 'boolean',
		permissioning:{denied: false, readonly: false}
	},
	{
		key:'ipAddress',
		label:'IP Address',
		type: 'ipAddress',//this is a custom type
		permissioning:{denied: false,readonly: false}
	}
];

The properties needed for each field are:

• key – name of the property that gets described by the entry
• label – description shown as label inside the form
• type – the type of the data stored in the property. This information is used by the framework to choose an editor implementation
• permissioning – data about what the user is allowed to do with the entry. This reflects what the server allows the user to do.

Using this information, we can create a form using the following API:

/**
 *

Adds a form with into the given container element using the given fieldDefinitions.

 *

The structure of the form's contents depend on the editor implementation inside fieldDefinitions. This
 * function creates just a form element as container for the elements the editors return and which it adds to the DOM.

 *

A Save button is added to the form as long as the addSaveBtn
 * config option is not set to false.
 * The fnOkCallback function must takes one parameter: it gets a new object filled with the values from the
 * created form.
 * 

 *
 * @param {object} namespace                The javascript portlet namespace (used to get I18N information)
 * @param {Element} container               HTML node to which the form is added
 * @param {object[]} fieldDefinitions       The definitions of the form fields
 * @param {function} fnOkCallback           Function which is called when the form's save button is clicked
 *                                          (if no save button is added (oConfig.addSaveBtn === false),
 *                                          this function is never called!)
 * @param {object} [oConfig]                Optional configuration.
 * @config {object} [formAttributes]        Attribute definitions for the form as key/value object.
 * @config {boolean} [addSaveBtn]           If false, no save button gets added to the form. Defaults to true.
 * @config {object[]} [additionalButtons]   Array of additional buttons that should get added left of the save button
 *                                          (if a save button is added) to the form.
 *                                          The contained objects need to members:
 *

 *
label {string} Label of the button.

 *
callback {function} Function called if the button gets clicked.

 *                                          

 *                                          Note: Make sure to stop the click event inside the callback function
 *                                          to make sure that the form does not get send!
 *
 * @return {function} An error handling function that can be called to check for errors and add error messages to the
 * form as needed.
 */
agimatec.scaffolding.createForm = function(namespace, container, fieldDefinitions, fnOkCallback, oConfig){
	// ...
}

(as you can see, we are using JSdoc toolkit to generate our API docs)

So, let’s create a form:

agimatec.scaffolding.createForm(this, formContainer, fields, null);

This call does two things: it creates a form inside formContainer and adds editor objects (read on to learn more about them!) to the field description objects given to it.

Now, we need to fill the form:

agimatec.scaffolding.fillFields(user, fields);

Now, we can see what we have constructed:

Form generated by the example code

Reading the values from the form is as easy as:

var objectFromForm = {};
agimatec.scaffolding.readValues(objectFromForm, fields, callbackWhenDone);

After this call, objectFromForm contains all data as edited by the user.

About Editors

Inside the framework, editor objects are used to create the form parts needed to edit object properties. All an editor does is create a fieldset which is added to the form created by agimatec.scaffolding.createForm(), which itself gets added to the DOM.
This means that if a special kind of data entry is needed, you will need to implement a special editor, which has to provide the following functions:

• getReadWriteElement() – returns the DOM element used to edit entries
• getReadElement() - returns the DOM element used to display
• setValue() – this is called by the framework when using agimatec.scaffolding.fillFields()

We have put together some editors (current count: 21), including ones to connect objects to other objects, upload files, entering IP addresses, weekdays, dates, times and so on. This makes maintaining our webapp much easier and guarantees the consistency our customers appreciate.

Now entering the stage: Metadata

Well, as easy defining fields using object literals is, this does not solve the issue of additions and changes to existing objects. This is where we added metadata to the mix. The metadata we get from the server tells us enough to create the field definitions we need to generate the forms!

So here we go:

	/**
	 * Gets the field definitions for a dynamic form element (@see scaffolding.js).
	 * This function returns the sorted field definitions for a specified object. These describe which
	 * editor is used and adds further information like key and label to it.
	 * If we have an enum type, it adds also the options to an feature object.
	 * Features:
	 *

	 *
sorting of field definitions using a bean's SORTING feature

	 *
AJF type matching for root type and all related beans (depends if the feature "include" is set)

	 *
adding type of modelObject AJF type as feature "refBeanId"

	 * 

	 *
	 * @param {object} namespace    The javascript portlet namespace (used to get I18N information)
	 * @param {string} beanInfoId   The bean info id (e.g. 'com.agimatec.connecta.model.XFireTemplate')
	 *
	 * @return {object[]} Field definitions for the given beanInfoId.
	 */
	agimatec.metadata.getFieldDefinitions = function(namespace, beanInfoId) {/*...*/};

All you need to pass is which kind of object you want to display, the rest is handled by the framework.

Whats next

In this post, i explained how our framework makes creating forms for data entry easy and how we use metadata to make things even more “magic”. But this is not the end of the story. Our metadata is generated in a multi-step process, which will be described in a future post. To wet you appetite: we are generating metadata from multiple sources: Java Beans and XML descriptors and are able to merge multiple descriptors and user-based permissions.

Of course, generated UIs always need to be highly configurable to meet your user’s needs. Our framework offers a lot of options for this, up to defining rules that can manipulate the form reflecting what the user entered (and it looks like: formManager.when("lifecycleStatus").isSetTo("ACTIVE").changeForm(/*...actions...*/)). This makes the forms more friendly to use and prevents the user from doing things that make no sense. If you are interested, i would like to hear from you, just leave a comment.

Another theme i did not mention in the example is client-side validation. This is built in, too. And it is not hard to use, but explaining it would have made this article even longer. Again: if you are interested, leave a comment. If there is interest, i will blog about it.

This is the fourth article in my mini-series about developing highly ajaxified portlets. In this blog, you can also find articles about:

In theory.

In practice, you will be faced with a problem that is obviously a blind spot of the portlet definition (at least, the “old” JSR-186): how to communicate between portlets. The ability to mix-and-match portlets is key to a truly pluggable application formed of portlets.

To solve the communication problem, we have developed a solution much like the one developed by the folks at liferay, but a bit more tuned and optimized for a fully pluggable solution.

Portlet Services and Invokers

Our implementation follows a simple pattern: Portlets are able to promote services to a page-central instance and can include interaction elements that are used to invoke services that match certain criteria:

• the type of object the service must be able to handle
• the type of operation the service offers

Promoting a service to other portlets includes:

• specifying what type of operation (CREATE/READ/UPDATE/DELETE) is offered
• a text telling the user what happens when invoking the service
• the function that should get called on service activation

Using these simple building blocks we can offer users a list of actions that are available for a specific object:

Completely discoupled

The major advantage of our solution is that we do not hard-wire which portlets offers what to other portlets. All you have to do is insert a service invoker into the UI and the framework does the work for you. All people working on the project can work in parallel, the communication overhead is minimal. All the developers must do is offering a portlets services using the framework and give services a name that speaks.

This makes turns our product into a completely customizable platform that fits exactly our users’s needs.

Maximum flexibility

The framework makes splitting up responsibilities between portlets easy and puzzling just the right application for the job together possible. After adding a new portlet to a page, all service menus are automatically updated to offer new services that may have been added by the new portlet.

A generated service menu

Service menu for a different set of portlets

What you see in these screenshots is that depending on which portlets are available, the portlet will offer different actions for an object (for Screenshot 2, the “Ping” portlet was added). All information about which services are available is calculated client-side and for the current setup. Adding a new portlet will add its services to the actions list. The text that appears inside the menu is set by the portlet offering the service.

Invoking services is not limited to the context menus. We also use buttons for services that enable the creation of new items.

Impacts on portlet design

All this makes it easy to develop portlets that are designed to fulfill small tasks the right way. Reusability of portlets is massively increased. And on top of all this you get a consistent usage pattern that your customers will be familiar with when you are adding more functionality.

This is the third article in my mini-series about developing highly ajaxified portlets. In this blog, you can also find articles about:

• Using YUI loader for your own javascript modules
  
• Portlet namespaces and javascript

In my next post in this series, i will give a little insight into how we use metadata generated by our server to make creating a fully ajax portlet a breeze. Meanwhile, check out our posts about testing our Ajax frontends using Selenium and the YUI Test framework.

The easiest method are unit tests. We have tried several JS unit test frameworks. Including JSUnit, Scriptaculous Unit Test Runner and YUI test. With a lot of YUI components like the YUI Loader and many YUI widgets we refactored our unit tests to use YUI Test. We like the Yahoo User Interface because of its documentation and also of its code quality.

YUI Test is not bundled with YUI. You can use it with YUI but we also use other frameworks like Prototype and DWR. To learn more about YUI Test you can watch the presentation of Nicholas C. Zakas.

Ok, this is great, now I can test my Javascript. But wait, what is if I have a lot of Javascript. I can group my tests in test cases and test cases in test suites. This is supported. But while the amount of Javascript is growing one file for all tests isn’t enough. We have a lot of different modules. Each consists of a lot of test cases. Sure, you can copy them all together in a file which is several thousand lines long. But every test run takes then a lot of time, debugging is more complex and it is more complicated to manage this file.

YUI TestManager

Wouldn’t it be great to have a test runner. A script which takes a lot of unit test files and run them one after another.  Our first approach was based on the Scriptaculous Unit Test Runner embedded in a frameset. We had a navigation frame and some Javascript included to run one unit test file in the main frame. One problem was to collect the results of all unit test files. We patched the Unit Test Runner and collected the data. It was ok and I wanted to refactor the solution to publish it. After starting I suddenly found an undocumented feature of YUI Test – The YUI TestManager. It is exactly what I wanted to do, define the unit test files, run through all of them and collect the results. I think Yahoo is using this feature internally and didn’t talk a lot about it. But it is included in the API, so please YUI team, include it to the documentation. It is a great piece of code.

Continuous Integration of Javascript with Selenium

Before I will show you how to use it, some more thoughts about this. We are coming from the agile Java world. So continuous integration is very important for us. Please search for “Continuous Integration Javascript” in your search machine. There are just a few hits and no ready to use solution. Yes, you can try to go back to the Java world and run tests with Rhino. There are some Java based unit test frameworks for Javascript. But there are two downsides. You are leaving the Javascript world and you are leaving the browser. It can be ok for you but I don’t trust the different browsers. I want to be able to run the tests fast in the browser when I’m developing and I want to test as many browsers as I can. We searched for a solution and what we have running is a suite of Selenium tests. They are at another level. Functional tests which run through use cases in the web frontend. But if we could give them a result of our unit tests we are able to include this in our setup. Then we can run the unit tests in IE, Firefox and Opera automatically.

YUI Test gives you the logger to view the results of your tests. It is not the best view for test results but it is ok. Also there are a lot of events which include the results of the tests after running them. We have implemented a little script which displays a nice summary which we test with Selenium.

Source code

Now let’s head to the source code.

At the beginning we need a unit test. Because I’m want to concentrate on the TestManager you will find a very very simple test.

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
        "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
    <title>String test</title>

    <!--CSS-->
	<link rel="stylesheet" type="text/css" href="http://yui.yahooapis.com/2.6.0/build/logger/assets/logger.css">
	<link rel="stylesheet" type="text/css" href="http://yui.yahooapis.com/2.6.0/build/yuitest/assets/testlogger.css">

	<!-- Dependencies -->
	<script type="text/javascript" src="http://yui.yahooapis.com/2.6.0/build/yahoo-dom-event/yahoo-dom-event.js"></script>
	<script type="text/javascript" src="http://yui.yahooapis.com/2.6.0/build/logger/logger-min.js"></script>

	<!-- Source File -->
	<script type="text/javascript" src="http://yui.yahooapis.com/2.6.0/build/yuitest/yuitest-min.js"></script>

    <script type="text/javascript">
    YAHOO.tool.TestRunner.add(new YAHOO.tool.TestCase({

        name: "String test",

        testEqualityAsserts : function () {
            var Assert = YAHOO.util.Assert;

            Assert.areEqual("hel"+"lo", "hello");     //passes
            Assert.areEqual("YUI test".substr(0,3), "YUI");     //passes
        }
    }));

    YAHOO.util.Event.onDOMReady(function (){

	    if (parent && parent.TestManager) {
            parent.TestManager.load();
        }
        else {
            var logger = new YAHOO.tool.TestLogger();

            //run the tests
            YAHOO.tool.TestRunner.run();
        }
  	});
  	</script>
</head>
<body>
</body>
</html>

In the first part we define our tests. In the second part you will see that the test will be called after the DOM is ready. It is a little bit different from the normal examples, because we call parent.TestManager.load() if it is available. Now you can copy this code, write your own tests, define test suites, … It starts to get interesting, if you have so many tests, that you want to group them into different files. With the code above you can go ahead and create multiple tests. In the example code I have created to test files calcTest.html and stringTest.html.

To run all tests in all your test files you need to call the TestManager:

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
        "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
    <title>JS Unit Tests</title>
    <meta http-equiv="content-type" content="text/html; charset=utf-8"/>

    <!--CSS-->
    <link rel="stylesheet" type="text/css" href="http://yui.yahooapis.com/2.6.0/build/logger/assets/logger.css">
    <link rel="stylesheet" type="text/css" href="http://yui.yahooapis.com/2.6.0/build/yuitest/assets/testlogger.css">

    <!-- Dependencies -->
    <script type="text/javascript" src="http://yui.yahooapis.com/2.6.0/build/yahoo-dom-event/yahoo-dom-event.js"></script>
    <script type="text/javascript" src="http://yui.yahooapis.com/2.6.0/build/logger/logger-min.js"></script>

    <!-- Source File -->
    <script type="text/javascript" src="http://yui.yahooapis.com/2.6.0/build/yuitest/yuitest-min.js"></script>

    <!-- Source file -->
    <script type="text/javascript">
    	var TestManager = YAHOO.tool.TestManager;

        TestManager.setPages([
            "./stringTest.html",
            "./calcTest.html"
            ]);

        YAHOO.util.Event.onDOMReady(function() {
        	TestManager.start();
        });
    </script>
</head>
<body>

<h1>JS Unit Tests</h1>

</body>
</html>

That’s simple, isn’t it?

That will give you the following output:

We wanted to give it more the feeling of a unit test suite. As I told you there is an event after test completion, which includes the test results. With our script we are building a summary table which gives you a nice overview for your test results.

With the overview table we also can track the results with a simple Selenium test:

package com.agimatec.ostium.portlets.jsunit;

import com.agimatec.ostium.portlets.util.ScreenshottingSelenium;
import com.thoughtworks.selenium.Selenium;
import org.testng.Assert;
import org.testng.Reporter;
import org.testng.annotations.*;

public class JsUnitTest {
    Selenium selenium;

    @BeforeMethod
    @Parameters({"selenium.host", "selenium.port", "browser", "baseUrl", "ostium.login",
            "ostium.password"})
    public void startSelenium(
            @Optional("localhost")String host,
            @Optional("4444")int port,
            @Optional("*firefox")String browser,
            @Optional("http://localhost:8080")String baseUrl
    ) throws Exception {
        Reporter.log("Running selenium");
        selenium = new ScreenshottingSelenium(host, port, browser, baseUrl);
        selenium.start();
    }

    @Test
    public void checkPassedJsUnitTests() throws InterruptedException {
        selenium.open("/portlets/ostium/utilities/ajf/src/test/javascript/testrunner.html");
        for (int second = 0; ; second++) {
            if (second >= 60) Assert.fail("timeout");
            try {
                if (selenium.isElementPresent("//div[@id='testcontainer']/div/h2")) break;
            } catch (Exception e) {
            }
            Thread.sleep(1000);
        }

        Assert.assertTrue(
                selenium.isTextPresent("exact:Result: Passed"),
                "The result of the JS tests should be 'PASSED'.");
    }

    @AfterMethod
    public void stopSelenium() {
        Reporter.log("Stopping selenium");
        selenium.stop();
    }
}

The only thing to do is to run the files on a local server. We use a Jetty with Maven to host the files because Selenium needs to include a proxy which didn’t work if you access the file directly.

Find more to Selenium on our other posts:

• Selenium testing of massive Ajax Apps
• Selenium CI tests with TeamCity
• Robust portlet testing

Download the script and some examples here. We are planning to release it with some other JS stuff as a Google code project, but we are still searching for a name.

If this post was helpful or you have additional ideas, please don’t hesitate to comment.

The setting

Let me explain the structure of our project a bit. The main constraint we are facing when developing rich user interfaces for the web platform is that we were up to do so using Java Portlets. Using the portlets standard (JSR-168), we are able to meet our customer’s needs in a way we never could have done when developing traditional, monolithic web applications. But this flexibility comes at a price: we do not know which apps are on a page when it gets loaded; further, we want to be able to add new portlets to each page at any time. From a Javascript perspective, this means that traditional “just put script tags into the header”-style Javascript coding will lead to a very bad user experience, since every page would be bloated with stuff the user will never use and – even more important- the user experience will degrade as we are adding new features to our product, even if our customer does not need or want the new features. As explained before, adding Javascript code to HTML/JSP Parts of the Portlets were no option and would have caused massive double-loading of script code. As our platform grew, we saw the need to put together a Javascript library to support code reuse up front. We clearly needed a mechanism to load our Javascript modules as they are needed. After some experiments (including the use of unholy synchronous XHR calls… shame on me!), we tried the YUI Loader.

Meet the YUI Loader Utility

Its no secret that we are using YUI in our product. As YUI comes with a special module for loading dependencies, the YUI Loader Utility, we explored if we could use YUI for our needs. The Loader’s feature list reads like our wish-list:

• Reliable, sorted loading of dependencies
• Safe, efficient mechanism for adding new components to a page on which YUI may already be present.
• Automatic use of rolled-up files.

So we’ve developed our own solution on top of the YUI loader for our Framework. It consists of two components: A Compile-time dependency analyzer and a configuration file generated by the dependency analyzer that tells YUI how to load our modules.

Analyzing JS dependencies

First of all, every JS module file starts with a set of comments telling the dependency analyzer how the current module is called and what modules are required by the declared module:

// @@module agimatec.topic
// @@requires event, agimatec.datatypes

// Module code goes here
// ...
// ...
// ...
// ...
// ...

// Tell the loader that this module has been loaded
YAHOO.register("agimatec.topic", agimatec.topic, {version:'1.0',build:'000'});

The analyzer (which is current implemented as stand-alone groovy script that is run by maven – we have plans to implement it as maven Mojo) parses the dependency information in the module files and generates a Javascript configuration file for the YUI loader from it:

// ******************************
// Generated Code - DO NOT EDIT
// ******************************
/*
  This script adds all AJF module definitions to the YUI Loader utility as noted by
  '// @@' - style comments inside the *.js files.
  If the global variable 'agimatec_config' is not defined, it is defined by this script.
*/
(function(){
    var arrModules =[

            //...

            {
             name:'agimatec.scaffolding',
             type:'js',
             skinnable:false,
             path: '../../agimatec/scaffolding-2.1.11-SNAPSHOT-min.js',
             requires:["yahoo","event","calendar","tabview","agimatec.topic","agimatec.util","agimatec.util.table","agimatec.util.element","agimatec.util.minibrowser","agimatec.util.datePicker","agimatec.util.hintedText","agimatec.validator","agimatec.metadata"]},

            {
             name:'agimatec.topic',
             type:'js',
             skinnable:false,
             path: '../../agimatec/topic-2.1.11-SNAPSHOT-min.js',
             requires:["event","agimatec.datatypes"]},

            {
             name:'agimatec.util.browser',
             type:'js',
             skinnable:false,
             path: '../../agimatec/util/browser-2.1.11-SNAPSHOT-min.js',
             requires:["yahoo","dom","event","animation","datatable","agimatec.topic","agimatec.util","agimatec.util.yui","agimatec.util.lucene","agimatec.util.element","agimatec.preferences"]},

            {
             name:'agimatec.util.connector',
             type:'js',
             skinnable:false,
             path: '../../agimatec/util/connector-2.1.11-SNAPSHOT-min.js',
             requires:["yahoo","dom","event","container","datatable","agimatec.util","agimatec.metadata","agimatec.util.table","agimatec.portletservice","agimatec.util.minibrowser"]},

            {
             name:'agimatec.util.context',
             type:'js',
             skinnable:false,
             path: '../../agimatec/util/context-2.1.11-SNAPSHOT-min.js',
             requires:["agimatec.metadata","agimatec.util"]},

            {
             name:'agimatec.util.datePicker',
             type:'js',
             skinnable:false,
             path: '../../agimatec/util/datePicker-2.1.11-SNAPSHOT-min.js',
             requires:["yahoo","dom","event","agimatec.util","agimatec.util.hintedText"]},

		 // ...
    ];

    agimatec_config = agimatec_config || {};
    agimatec_config._modules = arrModules;
    agimatec_config.getLoader = function(){
        var modulesLength=agimatec_config._modules.length, i;
        var configObject = {};
        configObject.base = agimatec_config.yuiBasePath;
        if(agimatec_config.moduleFilter){
            configObject.filter=agimatec_config.moduleFilter;
        }
        if(undefined !== agimatec_config.allowRollup){
            configObject.allowRollup = agimatec_config.allowRollup;
        }
        // configure the default skin
        configObject.skin = {
            'defaultSkin':agimatec_config.defaultSkinName,
            'base': 'assets/skins/',
            'path': 'skin.css',
            'rollup': 3
        };
        var loader = new YAHOO.util.YUILoader(configObject);
        for(i=0; i

This file is included inside the header of every portlet page. You see that thanks to the YUI Loader, we can mix YUI modules with our own ones without hassle, making development of a comprehensive Javascript Library much easier. (If your wondering why we add version numbers to the referenced module names, see this article)

Loading a module is as easy as:

// ...

var loader = agimatec_config.getLoader()
loader.require("agimatec.topic");
loader.insert(function(){
	// use agimatec.topic and ALL of its dependencies here !!!!
});

You see that using modules is very easy. All you have to do is to tell the loader what you need and provide a function that should get executed as soon as the dependencies are resolved by adding the noted Javascript files.
This solution served us very well in the last months, in fact our complete webapp development depends on this loading mechanism. It made growing a stable and feature-rich javascript library possible for us without destroying the user experience.

This post just scratches the features of our Javascript module tooling. We have also added support for a module description file format that allows us to describe Javascript modules using XML and generate the loading and dependency management code, along with support for “rollup” JS modules that allow us to reduce the number of requests needed to load portal pages.

If you are interested in this component, just tell us so in the comments and we will see how fast we can get this utility out of the door as Open Source.

This is part two of an article series about Ajax Portlet development. In my next post, i will tell you about how we handle inter-portlet communication in our framework and how our approach makes our application extremely flexible without hardcoding dependencies between portlets.

We have written the application as a demo in the first place and now wanting to test it in the wild. Because we use Cairngorm the application is well structured and contains as view layer only minimal parts of logic. Yes, we could include unit tests, but integration tests are far more interesting (from my point of view). In a large code base it could happen, that because of refactoring some things are broken. No deal, if you have tests and a CI system. But in the Flex app there are just tiny parts which I could test in a normal unit test.

Okay, so it should be integration test. Sounds good and not too complicated. There should exist a lot of frameworks because a lot of people start to use BlazeDS and other stuff in Java environments. Just something to start the app, run some tests and get the result back in the CI system.

I started to search on google and was surprised. Most of the blog posts I have found on that topic are just two month old. And there are not so many posts about that topic. With the combination of those posts I figured out, how to solve that problem. What problem?

• Integration tests of our Flex app
• Integrated in Maven
• Should work together with Cairngorm (would be more complicated with some unit tests frameworks)
• Could be Selenium (to start the app and run tests)
• Should be integrated in our continuous integration server TeamCity
• Could be written in Java

First thanks to the blog post of Julia at Blackpepper. She opened my eyes how to work with Java and Flex in combination. It is the combination of the SeleniumFlexAPI, which integrates as Flex to Javascript bridge in your Flex application and FlashSelenium.

SeleniumFlexApi is in one part a Flex library which needs to be included in your app and on the other part an extension to the Selenium IDE. I like the Flex part, because it makes it easy to dive over Javascript into the details of your Flex app. What I don’t really like is the Javascript extension. Yes, it is not only a Selenium IDE part and you can include it into the start of the Selenium Server, but it is far away from Java and Maven.

FlashSelenium is a piece of code which let’s you talk from Java over Selenium with your Flex app. The Java part is very handy, but you need to code into Flex, which methods you could call over your bridge. So I need to write special code, which opens the bridge. The SeleniumFlexAPI is much better in this part.

I liked the solution of the blog post and started to implement it.

First the pom.xml where you need to include the SeleniumFlexAPI (the scope cost me some time):

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">

   <parent>
        <groupId>agimatec</groupId>
        <artifactId>assessor</artifactId>
        <version>1.0-SNAPSHOT</version>
        <relativePath>../pom.xml</relativePath>
    </parent>

  <modelVersion>4.0.0</modelVersion>
  <groupId>agimatec</groupId>
  <artifactId>assessor-flexapp</artifactId>
  <packaging>swf</packaging>
  <version>${agimatec-assessor-version}</version>
  <name>Assessor Flex Application</name>
  <url>http://www.agimatec.de</url>

  <dependencies>
    		 <!--Flex SDK dependencies-->
		<dependency>
			<groupId>com.adobe.flex.sdk</groupId>
			<artifactId>playerglobal</artifactId>
			<version>3.0.0.477</version>
			<type>swc</type>
		</dependency>
		<dependency>
			<groupId>com.adobe.flex.sdk</groupId>
			<artifactId>flex</artifactId>
			<version>3.0.0.477</version>
			<type>swc</type>
		</dependency>
		<dependency>
			<groupId>com.adobe.flex.sdk</groupId>
			<artifactId>framework</artifactId>
			<version>3.0.0.477</version>
			<type>swc</type>
		</dependency>
		<dependency>
			<groupId>com.adobe.flex.sdk</groupId>
			<artifactId>framework</artifactId>
			<version>3.0.0.477</version>
			<type>resource-bundle</type>
			<classifier>en_US</classifier>
		</dependency>
		<dependency>
			<groupId>com.adobe.flex.sdk</groupId>
			<artifactId>rpc</artifactId>
			<version>3.0.0.477</version>
			<type>swc</type>
		</dependency>
		<dependency>
			<groupId>com.adobe.flex.sdk</groupId>
			<artifactId>rpc</artifactId>
			<version>3.0.0.477</version>
			<type>resource-bundle</type>
			<classifier>en_US</classifier>
		</dependency>
		<dependency>
			<groupId>com.adobe.flex.sdk</groupId>
			<artifactId>utilities</artifactId>
			<version>3.0.0.477</version>
			<type>swc</type>
		</dependency>
    <dependency>
      <groupId>com.adobe.cairngorm</groupId>
      <artifactId>cairngorm-bin</artifactId>
      <version>2.2.1</version>
      <type>swc</type>
    </dependency>

      <dependency>
        <groupId>org.openqa.selenium</groupId>
        <artifactId>flex-api</artifactId>
        <version>0.22</version>
        <type>swc</type>
          <scope>internal</scope>
      </dependency>

		 <!--FlexUnit dependencies-->
		<dependency>
			<groupId>flexunit</groupId>
			<artifactId>flexunit</artifactId>
			<version>0.85</version>
			<type>swc</type>
			<scope>test</scope>
		</dependency>
		<dependency>
			<groupId>flexunit.junit</groupId>
			<artifactId>flexunit-optional</artifactId>
			<version>0.85</version>
			<type>swc</type>
			<scope>test</scope>
		</dependency>
  </dependencies>

  <build>
    <sourceDirectory>src/main/flex</sourceDirectory>
    <plugins>
      <plugin>
        <groupId>info.rvin.mojo</groupId>
        <artifactId>flex-compiler-mojo</artifactId>
        <extensions>true</extensions>
        <configuration>
          <includeSources>
            <param>${project.build.sourceDirectory}</param>
          </includeSources>
          <locales>
            <param>en_US</param>
            <param>de_DE</param>
          </locales>
          <debug>true</debug>
        </configuration>
      </plugin>
      <plugin>
        <groupId>info.rvin.mojo</groupId>
        <artifactId>asdoc</artifactId>
      </plugin>
    </plugins>
  </build>
</project>

To get the Flex application compile with Maven you need to download the SeleniumFlexAPI and include the swc-file into your Maven repository, because it is (not yet) available on public repositories.

After compiling you can start the Selenium IDE and load the extension or go directly to FireBug and try to call the methods. Be aware, that the args-parameter is normally optional, but needs to be filled for the Javascript-Flex bridge (also cost me some time).

To call a method of your running application in Firefox fire up in FireBug’s console:

window.document['AppName'].getFlexExists('NameOfAnElement','');

To test your application you need to know, what the name of your application is (can be found in the HTML calling the Flash file) and you need to give ids to your elements, because there is just an id locator at the moment.

If you are at this point we can now switch to the Java part with Selenium. Selenium is just Javascript when you are in the browser. So all you do is to call the Javascript function which you called before over Firebug. FlashSelenium is a good wrapper for that call which means that you don’t have to write normal strings into your Selenium commands. I like a little bit more comfort and wanted to wrap the SeleniumFlexAPI a little bit closer. Also there are some methods in the FlashSelenium you don’t need with this solution. So I started to implement my own little wrapper. Maybe it is also interesting for the SeleniumFlexAPI (if completed with all methods).

Here is the beginning of the code:

package com.agimatec.assessor.selenium;

import com.thoughtworks.selenium.DefaultSelenium;
import com.thoughtworks.selenium.Selenium;

public class FlexSelenium {
    Selenium selenium;
    String flexApp;

    public FlexSelenium(String ip, int port, String browser, String url, String flexApp) {
        selenium = new DefaultSelenium(ip, port, browser, url);
        this.flexApp = flexApp;
    }

    public void start() {
        selenium.start();
    }

    public void open(String url) {
        selenium.open(url);
    }

    public void stop() {
        selenium.stop();
    }

    public String doFlexClick(String id) {
        return callFlexBridge("doFlexClick",id);
    }

    public boolean getFlexExists(String id) {
        return callFlexBridge("getFlexExists",id).equals("true");
    }

    private String callFlexBridge(String method, String id) {
        return callFlexBridge(method, id, null);
    }

    private String callFlexBridge(String method, String id, String args) {
        if(args == null || args.trim().equals("")) {
            args = "''";
        }
        return selenium.getEval("window.document['"+flexApp+"']."+method+"('"+id+"',"+args+");");
    }
}

And now the test case is the only thing which is missing. I used TestNG because it can be easier configured in a Selenium environment (when it started to become more complicated with several browsers, plattforms, test suites, test groups …):

package com.agimatec.assessor.selenium;

import org.testng.annotations.Test;

public class AssessorTest {
    private final static String URL = "http://192.168.1.175:8080/assessor-webapp";
    FlexSelenium selenium;

    @Test
    public void setUp() throws InterruptedException {
        selenium = new FlexSelenium("192.168.1.254", 4444, "*firefox", URL, "Terminal");
        selenium.start();
        selenium.open(URL);
        while(!selenium.getFlexExists("videoPlayer")) {
            Thread.sleep(1000);
        }

        selenium.doFlexClick("videoPlayer");
        Thread.sleep(300);
        selenium.doFlexClick("enterUserModeButton");
        selenium.stop();
    }
}

The easy way

When consulting the portlet books on the market or Sun’s own tips, you’ gonna find one easy solution to the problem: using the portlet namespace for your Javascript. The portlet namespace is a portlet-instance specific string that is suitable to identify assets of a portlet instance. So the easy way out is to use it as prefix or post fix for the functions you write:

<!-- Example: JSP page -->
<%@ taglib uri="http://java.sun.com/portlet" prefix="portlet" %>

Wow, that was easy. But there are some drawback when doing this:

• you have to stick to define your Javascript inside the portlets – no external Javascript files!
• being unable to define external Javascript files, there is no easy way of sharing Javascript code between portlets, which is not DRY
• Debugging gets even more painful with function names like “show_<insert some lengthy string here>()”

So, if your are up to make heavy use of Javascript and Ajax-based rich user interfaces, this is a no-go.

Using external Javascript files

So, loading Javascript form external file is the way to go. So here is our approach to the problem:

Use Javascript namespaces

We are putting all our stuff into one global variable, called “agimatec” (at the time we chose the name, we were not aware of Douglas Crockford‘s advice on making globals uppercase – otherwise we would haven named it “AGIMATEC”). This ensures that we’re leaving all third party scripts on the page untouched. Each portlet has its own (sub-)namespace inside this global.

Use a convention to locate the script files

We’re putting our javascript files into a java-like folder structure that reflects our namespace structure, so the portlet “Clientis Edit User” is put into the namespace “agimatec.portlet” and the corresponding file can be found under “agimatec/portlet/clientisEditUser.js”

Use a common initialization method for all portlets

All our portlet scripts must define the function “getPortlet()”, which returns a “raw” version of the portlet script.

Let a library create portlet script objects

Last but definitely not least we have written our own library to create portlet instances. The library contains a function that takes the portlet namespace string, along with the name of the portlet Javascript object, creates a new instance by calling “getPortlet()” on the Portlet object and puts the instance into its own namespace which is created using the portlet namespace:

/**
* Create a portlet namespace instance.
* @param {object} portletObject           A raw portlet namespace
* @param {string} portletNamespace    The Portlet ID as given by the portlet container
*/
agimatec.createPortletInstance(portletObject, portletNamespaceId){
    var rawNamespace = portletObject.getPortlet();
    portletObject[portletNamespaceId] = rawNamespace;
}

Using this technique, we are able to write clean, fully namespace-wrapped portlet scripts that have no problems with colliding function names and can co-exist even in multiple instances on one page without problems. As bonus, two instances of the same portlet on one page only load the external Javascript file once.

One major thing is not mentioned of this post: how to load the portlet scripts. We are using the YUI Loader for loading the portlet scripts, amoung other things. I’ll cover this in another post in the next weeks, so stay tuned.

If you have fought your way through the markup, you now get to another trap on your way to get your tests flying. The markup is changing, because you are using the Ajax technology. After the page is rendered there are asynchronous calls changing your DOM. You are asking: Where is the problem? The problem of testing is, that you need to teach your test tool, to look at the page in the same way you are looking at it. On the first look you want to get the third row of the table. But wait, it is still loading, even if there is just a minimal delay. If you do automatic tests they should run through all the tests as fast as possible. The tool is not to twiddle one’s thumbs to click on the button.

You need to write your tests in another way. The question isn’t only if the data could be found in the DOM, it is changing to if the surrounding element has loaded could the data be found then. And this is the essential point, where most recording tools like Selenium IDE are failing. It doesn’t give you a hint, what is loading after the DOM is ready the first time. Yes it is possible to add pause or a waitForElement-function before every field, but wouldn’t it be easier to have little helpers for the case.

First let’s have a look at how the waitForElement-function in Selenium is working in Selenium-RC:

for (int second = 0; ; second++) {
    if (second >= timeout) throw new ElementNotFoundException("Element not present: " + locator);
    try {
        if (this.isElementPresent(locator)) break;
    } catch (Exception e) {
    }
    Thread.sleep(1000);
}

It is polling every second, if the element specified be the locator could be found. That is better than adding a pause. Why?

How long does your pause needs to be? 500ms, 1s, 2s? You are not sure about it, I am sure. It is a guessing and it is really slowing down your tests. You have 20 elements in one test. And let’s say you’ve got  50-100 tests at the beginning. Well – you are pausing damn long.

That’s why you should start using the waitForX-functions of Selenium. Now nearly every line of your massive Ajax test has a previous waitForX-function. It doesn’t make your tests more readable, but more stable and as fast as possible.

Now we are in a Java world, writing our Selenium tests and are tired of writing waitForX-functions. The next step is to extend the DefaultSelenium class and adding some tiny helper functions:

• safeOpen
• safeGetText
• safeGetValue
• safeClick
• …

All these functions are equivalents of the normal Selenium functions, but having a waitForX-function build-in. Normally you are saying: getText for this element and giving a locator. Now use the locator to do a pre-test of the element. If the element could be found, get the text. It is just a little helper, but it makes your tests much cleaner and robust.

We have introduced another trick for the safeOpen function. Because we are fetching our dependend Javascript files asynchronously after the normal rendering has finished (check out the YUI Loader), we add a new element with an id after the Javascript has finished its loading process. Now we can poll if the element is in the DOM and start after all the Javascript has loaded.

Check out the following example and try out how it helps to speed up writing your Ajax tests and reduce the test to the test.

See the following example test. Even if you don’t understand which application is tested, you can read the test:

    @Test(dataProvider = "users")
    public void testUserDetails(User user) throws InterruptedException, ElementNotFoundException {
        selenium.safeOpen(URL);

        // fill in search form
        selenium.safeType(
                browserXPath().elementByClassName("searchForm").textField().toString(),
                user.getUserIdentification());

        selenium.safeClick(
                browserXPath().elementByClassName("searchForm").submitButton().toString());

        // action to detail portlet with service menu
        selenium.safeClick(
                browserXPath().table().tr(1).td(1).a().toString());

        // check if all fields are available
        String[] userDetails = {user.getUserIdentification(), user.getGender().toString(), user.getFirstName(), user.getLastName(), user.getEmail(), user.getMobilePrefix(), user.getMobileNumber()};
        for (int i = 1; i <= userDetails.length; i++) {
            selenium.waitForText(detailXPath().table().tr(i).td(2).toString(),
                    checkEmpty(userDetails[i]));
        }
    }

To see the “unbelievable magic” behind the scenes, check out the AjaxSelenium extension:

import com.thoughtworks.selenium.DefaultSelenium;

public class AjaxSelenium extends DefaultSelenium {
    public void safeClick(String locator) throws ElementNotFoundException, InterruptedException {
        this.waitForElement(locator);
        this.click(locator);
    }

    public void safeOpen(String pageUrl) throws InterruptedException, ElementNotFoundException {
        this.open(pageUrl);
        this.waitForPageToLoad("30000");
        this.waitForElement("//div[@id='javascript-loaded']");
    }

    public String safeGetText(String locator) throws ElementNotFoundException, InterruptedException {
        this.waitForElement(locator);
        return this.getText(locator);
    }

    public String safeGetValue(String locator) throws ElementNotFoundException, InterruptedException {
        this.waitForElement(locator);
        return this.getValue(locator);
    }

    public void safeType(String locator, String text) throws ElementNotFoundException, InterruptedException {
        this.waitForElement(locator);
        this.type(locator,text != null ? text : "");
    }

    public void waitForElement(String locator) throws ElementNotFoundException, InterruptedException {
        for (int second = 0; ; second++) {
            if (second >= timeout) throw new ElementNotFoundException("Element not present: " + locator);
            try {
                if (this.isElementPresent(locator)) break;
            } catch (Exception e) {
            }
            Thread.sleep(1000);
        }
    }

    public void waitForText(String locator, String text) throws InterruptedException, ElementNotFoundException {
        for (int second = 0;; second++) {
            if (second >= timeout) throw new ElementNotFoundException("Text '" + text + "' in element '"+ locator +"' not present.");
            try {
                if (text.trim().equals(this.getText(locator).trim())) break;
            } catch (Exception e) {}
            Thread.sleep(1000);
        }
    }

    public void waitForValue(String locator, String text) throws ElementNotFoundException, InterruptedException {
        for (int second = 0;; second++) {
            if (second >= timeout) throw new ElementNotFoundException("Text '" + text + "' in element '"+ locator +"' not present.");
            try {
                if (text.trim().equals(this.getValue(locator).trim())) break;
            } catch (Exception e) {}
            Thread.sleep(1000);
        }
    }
}

Robust and durable

Testing of web applications isn’t easy. If you are in a portlet environment things getting even more complicated. A portlet is a web application for itself and there could be a lot of different combinations on every page. There could also be many instances of each portlet on a page.

What are the consequences?
You couldn’t use normal ids on tags, because ids are unique on a page which conflicts when many instances of the portlet are on the same page.

A portal is themeable. The design and also the HTML structure depends on the user settings.

Portlets could be rearranged on a page, which means that the HTML structure is changing every time a new portlet is on a page or a portlet is moved to another place.

Lessons we learned with portlet testing

Nice but unstable

Try to write your tests as robust as possible. Nothing is as painful as seeing your test suite fallen to the ground.

You are developing a new theme or modifying your old one and all locators, in all tests, every field is broken.

You have a defined a set of portlets. Suddenly you need new portlets in this setup. You added the portlet at the top of the page and the tests are broken again.

Make portlet testing fun again.

Using XPath the right way

There are three tipps:

• Isolation
• Wildcards
• Abstraction

Isolation:

Normally there is a surrounding div-container with an id of the portlet instance. Try to use this element as entry point for your XPath. If you use the generated XPath expression of a tool like Selenium IDE, XPath Checker or other tools you will never get structured expressions which are easy maintainable.

//div[@id='agimatec_portlet_curatorArcaeEdit__CuratorArcaeEditPortlet_WAR_ostiumcuratorarcae__editParcel']

Wildcards:

I have never seen a portal source code with a really clean html structure. Because there are many columns, many layouts, many supported browsers you end in a kind of tag soup. Yes you can get an XPath expression with 20 elements. As mentioned there are tools which are generating the expression, but one change and you are doing the work again. Use a kind of wildcards in XPath. With the isolation you are at the portlet instance and right into your own code. Maybe there is just one table inside the portlet, but a lot of more div-layers between the table. With the // syntax you can navigate recursive to the next child element of that type.

//div[@id='agimatec_portlet_curatorArcaeEdit__CuratorArcaeEditPortlet_WAR_ostiumcuratorarcae__editParcel']//table/tr[1]/td[2]

Abstraction:

Don’t write the same code again and again. Write a helper which constructs the XPath for you. If you use a framework to build your forms, you can write a library which navigates directly to the form elements. XPath expression aren’t fun. It is better that it hurts one person than the hole team. Strive for abstraction.

detailPath().tab(1).ostiumTable().tr(1).td(2).toString())

Conclusion

There are a lot of tools for test recording. If you are using Selenium it is the Selenium IDE, if you use WebTest there is the WebTest Recorder. These tools are great for starting your web tests. But if your test suite is getting larger and you are working in an agile environment things are changing. Things are changing to often for maintaining recorded tests.

Learn with recorders but test with logic

The solution is great, we have reduced the requests to the Tomcat to a minimum (in our case less then 10%). Javascript is cached, CSS is cached, images are cached. But wait, we have do deal with updates and bugfixes. The files are now spread to hundred clients – deep burned into the brain with a far Expire HTTP header. This means if there is no manual refresh the files are there until Christmas, and will maybe survive the next Spring Break. We need a solution to get a clean update, on all client browsers.

What you need is a fresh URL for your files. The browser compares the URL and if there is a new URL inside the HTML it will be loaded. In our case the HTML is dynamic and won’t be cached, that’s why we don’t need to care about it. Our problem are the resource files (Javascript, CSS, images). To get a fresh URL for your files you need to answer the following question: What is the difference between two software version? Yes, you are right, the version number. So you need to include the version either in the path or in the file name. Because this is mainly a Java blog I hope you manage your files with a build tool like Ant or Maven. The first idea was to introduce the version number in the path and move folders with a Ant task to the new path before they are packed as WAR file. This worked for one folder but in our application we are dealing with a lot of portlets. Portlets are registered in the portal container and couldn’t be easily renamed. A solution would be to add the version number to all of folders inside the portlet but that’s sound like a lot of pain.

We need another solution, a solution that fits whenever a cached file is called which shouldn’t be reloaded. I followed a talk of Andrea Provaglio about AOP in web applications in the Java User Group Cologne two weeks ago. The main conclusion was: Whenever you say whenever - you should think about solving your problem in a kind of aspects. But how to solve our problem? The solution is a servlet filter. The files can still be deployed in the normal way but we can do a little bit of URL rewriting magic inside a filter. For the caching I’m using a servlet filter to set the Expire header. I extended this filter that there could be a version number given in the init-parameters. This version number is removed from the request and forwarded to the default file.

For this solution I need just a few steps:

1. The servlet filter with URL rewriting
2. The filter definition with version number as init-param in the web.xml
3. Resource calls with the version number included

Let’s have a look at the solution:

Before:

http://www.agimatec.de/blog/example.css

Now:

http://www.agimatec.de/blog/example-2.3.4-SNAPSHOT.css

Future:

http://www.agimatec.de/blog/example-8.9.10.css

The file has changed but is deployed to the same location /blog/example.css as before. The servlet filter has the version number (2.3.4-SNAPSHOT) and removes it from the call. What we get on the server side is the same call as before. With a new deployment we just need to increase the version number and wait for the download of the new version of the file.

Do you also need a solution for this problem? Maybe the following code could be a starting point for you:

import javax.servlet.*;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import java.io.IOException;
import java.util.Date;
import java.util.regex.Matcher;
import java.util.regex.Pattern;

public class CacheFilter implements Filter {
    private FilterConfig filterConfig;

    public void init(FilterConfig filterConfig) throws ServletException {
        this.filterConfig = filterConfig;
    }

    public void doFilter(ServletRequest servletRequest, ServletResponse servletResponse, FilterChain filterChain) throws IOException, ServletException {
        // get the http servlet request
        final HttpServletRequest httpServletRequest = (HttpServletRequest) servletRequest;
        final HttpServletResponse httpServletResponse = (HttpServletResponse) servletResponse;

        // set Expires header for optimized caching
        String cacheTime = filterConfig.getInitParameter("cache-time");
        if (cacheTime != null && !cacheTime.trim().equals("")) {
            httpServletResponse.setDateHeader("Expires", new Date().getTime() + Long.parseLong(cacheTime));
        }

        // get the url of the original request
        String url = httpServletRequest.getServletPath();

        String version = filterConfig.getInitParameter("version");

        // if there is a version info configured, try a forward
        if (version != null && !version.trim().equals("")) {
            // remove the version info
            String redirectUrl = removeVersionInfo(url, version);

            // if there is a difference in the url
            if (!url.equals(redirectUrl)) {
                // forward the call to the new url
                servletRequest.getRequestDispatcher(redirectUrl).forward(servletRequest, servletResponse);
                return;
            }
        }

        // normal call if there is no forward
        filterChain.doFilter(httpServletRequest, httpServletResponse);
    }

    public void destroy() {
        filterConfig = null;
    }

    public static String removeVersionInfo(String url, String version) {
        if (version != null && url != null && !url.trim().equals("")) {
            url = url.replaceAll(Pattern.quote(version), "");
        }
        return url;
    }
}

Questions, suggestions, errors? Write a comment.