RestFixture is now available

I have made the RestFixture (discussed in this post) available on GitHub here: http://code.google.com/p/rest-fixture/.

You can get it, use it and modify it under LGPL terms.

Feedback is more than welcome, of course.

Agile 2008 - part 1

Oh yes, Agile 2008... here I am, just landed from Toronto. I am just reshaping my notes. I'll be writing (as I have also done in the past) my comments on the sessions I have attended and presented, and about old friends I have talked to (Rick, Rachel, Angela, JB, Steve) and new people I have met. Stay tuned.

Get FitNesse with some Rest

UPDATE: code moved here: http://github.com/smartrics/RestFixture

UPDATE II: Thank you to Steven Haines for the clear and concise Howto giude on the RestFixture.

I am currently involved in building a REST API. Our direct customer proxy (the architecture team) needs to "understand" how we implement the agreed acceptance criteria and it has also asked to document the API.
FitNesse is our tool of choice for writing functional Customer Acceptance Tests; we currently use it to implement this type of tests for other parts of the system we're building. So it came natural to start using it for documenting the API.

At first we implemented the tests using essentially ActionFixtures. The approach adopted consisted of "wrapping" the REST API in more descriptive metods that could be pressed, checked and entered.

After the first dozen of tests it was apparent that this approach was not ideal. Tests were not clear enough and maintaining the fixtures was hard and time consuming as the amount of code duplication was high for any standard. New tests required us writing more code and, in fact, we were testing just the behaviour of the backend without giving enough exposure to the API itself - consequently, our tests were not descriptive enough to work as live documentation.

So I decided to write a new "type" of fixture, inspired by the ActionFixture, the RestFixture.

The core principles underpinning the decision to write a new fixture were the following:

• For documenting a REST API you need to show how the API looks like. For REST this means


• show what the resource URI looks like. For example
/resource-a/123/resource-b/234
• show what HTTP operation is being executed on that resource. Specifically which one fo the main HTTP verbs where under test (GET, POST, PUT, DELETE, HEAD, OPTIONS).


• have the ability to set headers and body in the request


• check expectations on the return code of the call in order to document the behaviour of the API


• check expectation on the HTTP headers and body in the response. Again, to document the behaviour


• I didn't want to maintain fixture code. If I could only write the tests...
• I wanted to be able to let the customer proxies to write the tests... And they understand Wiki syntax more than Java.

The RestFixture at a glance

The RestRixture is an ActionFixture, therefore all the ActionFixture goodies are available. On top of that it contains the following 7 methods:

• header: to be able to set the headers for the next request (a CRLF separated list of name:value pairs


• body: to allow request body input, essential for PUT and POST


• let: to allow data from the response headers and body to be extracted and assigned to a label that can then be passed around. For example our API specifies that when you create a resource using POST, the newly created resource URI is in the Location header. This URI is necessary in order to perform further operations on that resource (for example, DELETE it in a teardown).


• GET, POST, PUT, DELETE, to execute requests.
  

Each test is a rown on a RestFixture table and it has the following format:

|VERB|uri|?ret|?headers|?body|

• VERB is one of GET, POST, PUT, DELETE (at the moment we're not supporting HEAD and OPTIONS)


• uri is the resource URI


• ?ret is the expected return code of the request. it can be expressed as a regular expression, that is you can write 2\d\d if you expect any code between 200 and 299.


• ?headers is the expected list of headers. In fact, the expectation is checked by verifying that *all* the headers in here are present in the response. Each header must go in a newline and both name and value can be expressed as regular expressions to match in order to verify inclusion.


• ?body it's the expected body in the response. This is expressed as a list of XPath expressions to allow greater flexibility.

Some examples

The following pictures are snapshots taken from FitNesse with the aim of providing examples of usage of the RestFixture. The notes before each test explain the details of the fixture itself.








It's worth noticing that, generally, expectations are not matched by string comparing the expected value with the actual value. Expectations on headers are verified by checking that the expected set of headers is included in the actual set of headers, similarly with the body where expectations are matched by checking existences of nodes for a given path.

Conclusions

The RestFixture allows to write FitNesse tests without having to write any Java code to back up the fixtures. Tests are clear and easy to read/write and this obviously improves their readability. Therefore it works well for documenting the API and the behaviour of the system under test. Looking it from another angle, the fixture, essentially, implements a REST DSL, that allows customers to write tests on their own.

Groovy for XML transformation

I have been playing with groovy recently, specifically with the MarkupBuilder, the groovy native support for markup languages. It basically means writing XML using native groovy syntax. Pretty neat.

In my job I have more often than not, have to write software to integrate two or more systems. This, often, means writing code that reads an XML stream onto a Java object for manipulation and eventually writes the result into another XML stream.

This typically involves creating binding objects for the input XML and the output XML and then a sequence of calls to getters on the input object and setters on output object to implement the mapping.

This task is very error prone and tedious. A good solution is using the MarkupBuilder (and, yes, I know about XSLT, but this is not the point here!)

So, suppose that you have the following POJO

public class Contact {
  private String id;
  private String name;
  private String surname;
  // ...
}

and you want to serialize the instance

Contact c = new Contact();
c.setId("123");
c.setName("John");
c.setSurname("Bloggs");

into the following XML

<contact>
  <key>123</key>
  <firstname>John</firstname>
  <secondname>Bloggs</secondname>
</contact>

Note the difference between the POJO attribute names and the XML tag names

Using groovy and its MarkupBuilder, the first thing to do is to create a groovy converter, a class with a method that reads the data from the bean and produces the XML. Create a file src/groovy/ContactConverter.groovy with the following code:

class ContactConverter{
  def convert(bean){
     def writer = new StringWriter();
     def xml = new MarkupBuilder();
     xml.contact {
        firstname(bean.name)
        secondname(bean.surname)
     }
     writer.toString()
  }
}

If you execute the code

println new COntactConverter().convert(c)

in a groovy shell, you get the XML shown above.

The next step is then to make this code executable from your Java service.
You can use this class (an adaptation of the code available in the groovy documentation):

public class GroovyConverterInvoker {
  public String invoke(String fileName, Object bean){
    GroovyObject groovyObject = createObjectFromStreamName(strName);
    String result = (String)groovyObject.invokeMethod("convert", new Object[]{bean});
    return result;  
  }

  public GroovyObject createObjectFromStreamName(String fileName) {
    ClassLoader parent = getClass().getClassLoader();
    GroovyClassLoader loader = new GroovyClassLoader(parent);
    Class groovyClass;
    File f = new File(fileName);
    try {
      groovyClass = loader.parseClass(f);
      return (GroovyObject)groovyClass.newInstance();
    } catch (CompilationFailedException e) {
      throw new IllegalStateException("Unable to compile " + fileName, e);
    } catch (IOException e) {
      throw new IllegalStateException("Unable to open file " + fileName, e);
    } catch (InstantiationException e) {
      throw new IllegalStateException("Unable instantiate object for class in " + fileName, e);
    } catch (IllegalAccessException e) {
      throw new IllegalStateException("Unable access object of class in " + fileName, e);
    }
  }
}

You can then invoke the conversion:

public String invokeContactConverter(Object bean){
    String fName = "src/groovy/ContactConverter.groovy";
    return new GroovyConverterInvoker().invoke(fName, bean);
}

You can now use JAXB/XStream or XmlBeans to parse the result of invokeContactConverter and initialize a new POJO for further use, or send the XML on the wire.

The solution can be generalised and optimised to a point where you only need to write groovy converters and load/modify them dynamically when necessary.

Sicilian as a language

I am randomly browsing the web today. I just finished to read a Wikipedia entry on the Sicilian language. I always thought that Sicilian was an Italian dialect, although I was aware on several influences from French, Spanish, Latin and Greek, but I realised that scholars do consider it as a language in its own merit. That makes Sicilians bilingual at least.

UMLGaph for auto generating class diagrams from your source code

As part of improving the documentation of our software - a REST API - for the benefit of our customer (mainly an architect) and (future) users, we have investigated UMLGraph, a free application that is able to parse source code and (with the help of graphviz) generate class diagrams that can be embedded in the javadoc documentation.

The tool is very good and we're trying now to integrate it in our continuous integration build via ant.

BT Web21C SDK wins eWEEK Excellence Award

The BT Web21C SDK has been awarded with the eWEEK Excellence Award under the category "Application Development" as reported here. I am part of the team who built it and I feel proud that 2 years of hard work have been recognized.

Smartrics tag cloud

Go figure why the guys at blogspot have not provided a tag cloud widget. So, kudos to phydeaux3 for supplying the code for my tag cloud. Except for changing the font colors and types to make it look better it was a matter of copy and paste in my blog template. Nice and easy...

The (hyper)reality of social networking

I was reading about hyperreality when I started thinking of how it relates to the social networking phenomenon and communities built around it. Is there a connection between the two? Maybe. In fact someone else has already discussed this topic in this rather interesting post.

My favourite definition of hyperreality is Umberto Eco's one: "the authentic fake". As you would expect the web is full of information about hyperreality, for example this is fairly complete article with examples.

I'll try to write down my thoughts on how social networking community members do live in a hyperreal world. I should warn you though: I may sound cynical and destructive. Far from me: I think there's a place for social networking web sites and services (after all I am blogger too) and there's nothing wrong with being a consumer of these services. As far as I am concerned now I am just trying to understand the mechanics, why they work and why they exist the way they are now, and how they will transform in the future to come. So bear with me on this aspect.

At the core of the meaning of hyperreality is the ambiguity between "real" and "fake" reality that arises when facts and events or objects are filtered by a multitude of media.

Members of communities built around any of the latest and cutest Web2.0 website interact by the means and rules offered by the portal they use (Facebook, MySpace and the like). This medium filters the behaviour adopted by people. Actually, members of any community, in a way, always interact within an accepted framework: think of the rules, implicit (values, culture) or explicit (laws), that manage our relationship in a social environment and how they drive our behaviour.
The main difference, though, is that people meeting face to face have other means (environment, body language, time...) that contribute to maximise the possibility that facts and events are perceived and understood correctly and close to the reality.

Now back to the social networking. As said the web shapes the behaviour; there's also in the mix the fact that content and messages are exchanged using a different time pattern compared to normal face to face relationships. A message, a profile, a blog post, a twit is sent to the server and then delivered to the intender recipient(s). The recipient(s) reads it, digests it, formulates a response and eventually sends it back. This, to me, contributes to the creation of a world and of a reality that is transformed: what the recipient perceives is the autentically "fake" reality that the sender wants to transmit having lost its immediacy and being tailored to the media chosen to operate upon. Besides the fact that sender and recipient know that the content being generated is public.
I like to think that communities in Facebook (I am not picking on Facebook, it's just an example) are equivalent to the community of house mates in the BigBrother house (or, for argument's sake, to any other reality). The media, the place and being observed changes the behaviour allowing the community (participants and viewers) to create a fake real world.

So, it seems that social networking is son of its time: it has been made possible by new technologies - higher network speed, new web site capabilities, etc - and it's expression of our current time where appearance rules.

I have a memory from my infancy: if you're Italian or you watch Italian TV you know about the Mulino Bianco commercial (this is one of the many) which epitomises the (hyper)real world. We (the community of consumers) are let to believe that that is a beautiful real place with a real family where - obviously - plumcakes and brioches are soft and tasty.

Is there any difference between the Mulino Bianco ad and any of the profiles in Facebook? Let alone the style and the manufacture - Facebook users may not be professional advertisers - the communality is that both are hyperworld where the authors want the recipients to walk in and live the life they want them to live.

In this context let me fool around. If hyperreality is made of hyperreal facts and hyperreal events that exist in a hyperreal world, then Twitter allows users to exchange hyperreal facts; MySpace and Facebook as hyperreal multi-worlds; SecondLife as real hyperreal world.

The astute reader may now say that hyperreality is just a play on words, people make real money with this sites, businesses and enterprises back them up. It depends, it's a matter of prospective. Vegas and its casinos is extreme hyperreality made true. Punters entering a casino get in a manufactured hyperreal place where everything is obviously fake to let you believe that our money is fake too. Incidentally, those who think that your money isn't fake are casino owners who live in the real reality of gambling business.

So, who's making real money in this hyperreal world? I guess those who are able to spoil the user data by aggregating it and extract useful market information, for example. I suspect that biases will eliminate each other when aggregated, providing the clever data manager with information to use at his like (hopefully within the boundaries of the data protection acts).

A real example is available here: it describes how Amazon makes money using information extracted by user generated data.

Yes, clearly this is a win-win situation. Community members get value by being able to cultivate their social aspirations, voyeurism, need to be heard, trend and fashion, benefiting of other members' experience. Providers - on the other side - are able to target established communities by processing generating data, aggregating it with the purpose of extracting financially useful information (read the Facebook case study).

I am sure there's more to say. I stop now but I am interested to hear from you, dear reader, your view on this.

Covering the obvious

I was listening to Led Zeppelin whilst doing the restyling job at home and acknowledging how popular some of their songs are. Obviously Stairway to Heaven is amongst those... so I started wondering how many covers have been published in almost 40 years. I started digging the YouTube website, but, funny enough, at the same time I found that Ernesto Assante - a famous italian music critic - did it all for me by listing on his blog the most unusual covers of that masterpiece. The one I like amongst those he's listing is the version of Robyne Dunne, very 80s' and colorful. Here it is: