smartrics

Grammichele - Piazza C. M. Carafa

2011-08-07T17:57:00.003+01:00

I am re-organising the pictures in my hard drive and I have just came across a set of snaps I took 3y ago in Grammichele (the small town in Sicily where I grew up). After some stitching and cloning, this is the result

Piazza Carlo Maria Carafa, Grammichele (CT) Italy by Fabrizio Cannizzo is licensed under a Creative Commons Attribution 3.0 Unported License.

If you need/would like to use the large version of the file (over 200Mb for 14328x3091 pixels), by all means, get in contact.

UPDATE: Link to Google Maps

View Larger Map

Continuously integrate FitNesse and RestFixture

2011-05-11T21:54:00.008+01:00

I recently released the RestFIxture v2.0 with some additions and improvements. An effort, having worked on new features and code refactoring, that took about two months.

One thing I have always missed during the past two years of development is the lack of continuous integration with FitNesse; being the RestFixture a "plug-in" to FitNesse. Not a big deal maybe, given the size of the project and the narrow point of integration with FitNesse; but, in the true spirit of Agile software development, having being mildly burned in the past by the integration issues I decided to set up a CI build to gain confidence that both the latest versions of the tools work correctly at any point in time.
The problem was clearly where and how.

CI in the "cloud" with FaZend
I searched for a "cloud" provider of Continuous Integration solutions, and I soon discovered FaZend.com, a company offering hosted SVN, Trac and Hudson for small and medium projects. The service - at the time of writing - is still beta and free of charge.

I like their product; I am a user of such systems, but I also have contributed to the implementation a similar SaaS solution, for BT, two years ago, in our internal cloud and for our developers community. So I fully appreciate the value of a point and click hosted solution to sort out this curcial piece of the development environment.

FaZend is therefore a good fit: it's ideal for a Java based stack, runs Hudson - not Jenkins - one of the best CI servers out there, and it doesn't cost a penny. Having said that, as both RestFixture and FitNesse are hosted on GitHub, I don't need Trac and SVN (well, it turns out that I do need SVN, as I will explain later)

Few comments to summarise my experience (so far) with the service.
Reading the FaZend website I discovered that on Sign up, the CI features of the service are disabled and that it's at discrection of the company to enable users.
So I dropped an email to the support email address to explain my plan and check if it was possible to have CI. Straight after I got the thumbs up response from Yegor (the CTO) and an invite to sign up. And so I did.

Yegor enabled Hudson on my account and also bothered to create an initial project skeleton and so I was ready to go.

The project configuration is accessible via the "CI settings" page and offers the possibilty to set the essential information for the build to start. Preatty easy if edited alongside the tutorial available in the website.

The CI server offers SVN, GIT and Mercurial integration. It turns out, though, that the Git plugin used by Hudson isn't working very well and code can't be cloned from GitHub. Because that Hudson is very much locked down and only accessible via the CI settings (even the workspace is forbidden), it's hard to see what's going on. So I left with it to FaZend to sort it out (albeit I have offered help, if needed).

From the FaZend portal you can also set access control lists to setup everybody participating to the development of the project being build. As I am currently the sole developer of the ResatFixture, I didn't bother to go beyhond the default setup (allow r/w to the account and forbid everything to everybody else).

And there you go, here it is the RestFixture Integration build with logs and metrics published for reference.

Technicalities

At the time of writing the RestFixture-CI project - holding the build file and the necessary dependencies is available on GitHub, but also on the FaZend SVN, where the Hudson picks it from. It's not ideal, but given that the project isn't changing that often it's OK to double commit (to git and svn) every time.

The build strategy is very simple: a) clone the FitNesse master branch, b) clone the RestFixture master branch, c) build FitNesse, d) deploy the new fitnesse.jar in the RestFixture lib directory and then d) build and test the RestFixture. It's all in the RestFixture-CI build file in it's entirity.

And, the FaZend CI configuration that allows the build to run in Hudson, for everybody's benefit, is:

RestFixtureIntegr:
  people: *
  source:
    scm: svn
    url: svn://svn.fazend.com/RestFixtureIntegr/trunk
  builder:
    tool: ant
    target: test-integration
  publish:
    junit: build/restfixture-git-ro/build/reports/unit/html
    fitnesse: build/restfixture-git-ro/build/reports/fitnesse
    cpd: build/restfixture-git-ro/build/reports/metrics/cpd
    pmd: build/restfixture-git-ro/build/reports/metrics/pmd
    checkstyle: build/restfixture-git-ro/build/reports/metrics/checkstyle
    ckjm: build/restfixture-git-ro/build/reports/metrics/ckjm
    coverage: build/restfixture-git-ro/build/reports/metrics/coverage
    crap4j: build/restfixture-git-ro/build/reports/metrics/crap4j
    findbugs: build/restfixture-git-ro/build/reports/metrics/findbugs
    jdepend: build/restfixture-git-ro/build/reports/metrics/jdepend

More about the ant build

To implement the above strategy I had to - somehow - clone the two git master branches. Not being able to rely on accessing the underlying OS for the git command (albeit it turns out that it may be possible, but I haven't tried), I opted a full java solution.

I therefore implemented two simple Ant clone and pull tasks wrapping the jGit API. The project is called jgit-ant and it's available on GitHub. At the moment only cloning and pulling is implemented, which is just about what I need for this build to execute.

Having the 'clone' sorted, it was time to build.

FitNesse was a pain to build: the target "createUpdateList" simply doesn't work unless the build is started from the same directory as the build file. The (ugly) solution was to call the ant launcher via %lt;java>; importantly, it's necessary to set the attribute dir to the master branch directory for it to work.

After that an ant call to the RestFixture build completes the job.

The build overall takes less than 4 minutes, which is not bad. This, incidentally, suggests that the network connection between the hardware hosting Hudson and GitHub is pretty fast. (Running the build locally takes 30% more, having the clone task as the bottleneck).

Furthermore, after having properly crafted the build file to check for pull failures (that trigger a clone) and repository modifications (that trigger a build), now the build takes only 1 second if no mods/builds are executed, improving even further the feedback loop.

Conclusions
After releasing RestFixture v2.0 I wanted to add another safety net for spotting integration isssues with FitNesse, given that RestFixture and FitNesse development lifecycles are completely disjointed.
A Java based continuous integration hosted solution is what I was looking for. And FaZend ticks all the boxes (other than being the cheapest).
Very few issues along the way and in a total of half a day I set up the build to periodically integrate the two tools.
I am pretty happy with the result. I'll keep an eye on the solution to the Git Hudson plugin situation and possibly clean up the build process.

RestFixture v2.0

2011-05-09T23:09:00.001+01:00

I just completed the release of the RestFixture v2.0.
With spare time to dedicate to it (courtesy of National Railways who has increased the number of carriages in the train to London to 12, so I can sit!), I took the opportunity to finish off the unfinished business.

The new codebase includes few new features (plus some bug fixes and improvements to the fixture APIs). The RestFixture code is available on GitHub.

This is the list of the main additions. Further details are available in the generated documentation file.

SliM support
It's now possible to run the fixtures with the SLIM runner.
To do so, it's sufficient to specify in the root page of the suites to be run with SliM the following line:


!define TEST_SYSTEM {slim}

then, as each fixture is implemented as a TableTable, the first row of each fixture should be written as


| !-Table:smartrics.rest.fitnesse.fixture.RestFixture-! base_host_uri |

Setting custom content-type to adapter map
Up until version 1.1.1, the way of interpreting the expectations in each response body cell was dictated by the content type of the response.
Now, it's possible to override the default behaviour by specifying a config property that maps the content type of a response to a body handler. The name of the property is restfixture.content.handlers.map; and the available body handlers are

XML to interpret the body as an xml string and the expectations as a list of \n separated XPath expressions

JSON to interpret the body as a JSON string and the expectations as either a list of JavaScript lines to be evaluated in AND or as a block of JavaScript, if the first line of the cell is the string /* javascript */

TEXT to interpre the body as a string and expectations as a list of regular expressions to be matched against the body

The 'comment' command
A new command to inject comments in a fixture; useful to evaluate labels


|comment| the value of 'replaceme' is %replaceme% |

The LET handler
Let now can evaluate JavaScript strings and assign the result to labels. This allows greater expressiveness in extracting data from a REST response.

Other additions

labels containing a null value are rendered by default with the string 'null'. It's possible now to override this behaviour by defining the config 'restfixture.null.value.representation'

body cells with large content can be rendered in a folding tag. A button is provided to toggle the visibility of the content.

RestFixture with namespaces support

2011-01-17T22:42:00.009Z

I finally found some time to fix namespaces support on the RestFixture. Now it's possible to specify a namespace context where aliases are defined, and use such aliases in the XPaths that match the response body.

The namespace context is defined via the RestFixtureConfig fixture, defining the key restfixture.xml.namespace.context. The alias can then be used in any XPath subsequently.

The following two snippets are from the auto generated RestFixture documentation (section XPaths and namespaces). This shows the executed RestFixtureConfig with two aliases defined

whilst this shows the usage of one of the two aliases:

Note: the default namespace of a XML body must be explicitly defined as alias!

The latest release (v 1.1) of the Rest Fixture is downloadable from Google code here, and the source code is on GitHub here

My code has a new home

2010-07-31T15:43:00.005+01:00

I have just finished to migrate all my pet projects to GitHub: http://github.com/smartrics. In there you'll also find some of the forks I have made to stuff that I have modified.

I find GitHub easier to play with (albeit git is horribly more complicated than svn) and I like some of its features.

The projects in Google Code are now here:

* Rest Fixture: http://github.com/smartrics/RestFixture
* Pic Builder: http://github.com/smartrics/PicBuilder
* Rest Client: http://github.com/smartrics/RestClient
* JMeter Rest Sampler: http://github.com/smartrics/JMeterRestSampler

So, bye bye Google Code and thanks for the hospitality!

My daily newspaper

2010-06-16T13:42:00.001+01:00

http://paper.li/fab_can

Lost in metaprogramming

2010-06-03T22:41:00.004+01:00

More often than not, making heavy use of meta-programming, will get you into trouble. Especially when nasty side effects kick in and exceptions raise when they shouldn't. Clearly, analysis of the stack trace is rather pointless.

Then, sometimes, defining set_trace_func comes handy:


set_trace_func lambda { |event, file, line, id, binding, classname|
  if(event=="raise")
    puts "=" * 40
    puts "%8s :%-2d %5s %8s" % [event, line, id, classname]
  end
}

UPDATE
another interesting gem for this purpose is Unroller

Method chaining with Ruby

2010-04-22T00:15:00.014+01:00

Implementing method chaining in Ruby is a bit fiddly. Let's, for example and without loss of generality, look at plain data holders: objects whose sole existence in life is to hold some data.
Standard data holders, in Ruby, are easily defined using attr_accessors. For example:


class Person
 attr_accessor :name, :surname
end

person = Person.new
person.name = "Bob"
person.surname = "Smith"
p person.name
p person

produces


"Bob"
#<person:0xb78925a8 surname="Smith" name="Bob">

But it would be far better and more compact to be able to use method chaining, so to be able to write:


person = Person.new.name("Bob").surname("Smith")
person.name
person.surname

To achieve it, some Ruby metaprogramming is required. Jay Fields wrote an article some time ago providing an implementation that is not entirely satisfactory. His implementation differenciates between getters and setters and the example above would look like


person = Person.new.set_name("Bob").set_surname("Smith")
person.name
person.surname

It's possible to do better by doing:


module MethodChain
 def chained_attr_accessor(*names)
   names.each do |name|
     define_method :"#{name}" do | *args |
       return instance_variable_get(:"@#{name}") if args.length == 0
       if args.length == 1
           instance_variable_set(:"@#{name}", args[0])
           return self
       end
       raise ArgumentError.new("wrong number of arguments (#{args.length} for 1)")
     end
   end
 end
end

class Person
 extend MethodChain
 chained_attr_accessor :name, :surname
end

person = Person.new
person.name("Bob").surname("Smith")
p person.name
p person

which, undoubtedly, will produce the desired output:


"Bob"
#<Person:0xb776ff40 @surname="Smith", @name="Bob">

It could be neat, where appropriate, to add the chain_attr_accessor to the Class class, so that it's not even necessary to extend the class with the module:


class Class
 def chained_attr_accessor(*names)
  #...
 end
end

class Person
 # extend MethodChain
 chained_attr_accessor :name, :surname
end

Conclusion

Extending the code is trivial. What the module all does is defining a method on the class Person with the name of the attribute. Each implementation checks the length of the arguments list and decided if behaving like a setter or a getter.

More complicated logic can be implemented to allow more than one input or provide only getters for calculated attributes.

This technique can also be used to go beyond simple data holders, towards fluid interfaces.

But all that is left as an exercise to the reader.

Who owns user data?

2009-11-04T22:28:00.003Z

I have been involved several times on discussions around user data ownership. With the advent of cloud computing, though, some of the topics identified may take another shape and service/application providers may have a chance to fully leverage the capabilities exposed by cloud computing.

A bit of context first. If I subscribe to a service on the web, whatever it's nature is (Facebook, Twitter, you name it!), I generate data about me that the service stores, on my behalf, in it's premises. Examples are username, contact details, preferences, tags, ...
If for whatever reason I want to access such data, I typically can only do it via the user interface that the service provides. In any case I never - it seems to me - have access to the raw data; let alone being able to use some other access interface.

But think if such data is stored on the Cloud on an Amazon S3 bucket (or equivalent), which offers an easy to access interface over HTTP, user ownership, Access Control Policies and the possibility of managing such data via the plethora of applications part of the Amazon ecosystem. In such case I - the user - have access to the "master copy" of my data and I can do whatever I want with it (such as delete it, modify it, make it more or less public).

Clearly there are drawbacks:

it requires the service provider to implement and manage the mapping of it's internal user representation with that of Amazon (or whatever other cloud storage provider)

the a service provider looses the ability to control the stuctural consistency of the data. And this can cause the application to malfunction.

the service provider looses the possibility to hide such data from other service providers/application and consequently looses the power to forcibly keep the user using its servie (well, at least it doesn't facilitate it)

Point 1 is in reality a "non" issue from an implementation perspective. And it will be easily solved via agreements between the two interested parties. Agreements, moreover, are facilitated by the adoption of standard formats as and when appropriate.

Point 2 is the most interesting as it involves a different thinking on architecting applications (I refer here to the seven points illustrated here by Simone Brunozzi, that I discussed here): architect for failure (in this case data structures), loose couple application logic and data and so on.

Point 3 is probably the hardest to implement, not certainly because of technical issues. But this seems to me a false problem anyway as if the service offered degrades users will go away no matter what.

I want to see people making use of what the cloud brings to the table on top "glorified web hosting" or grid computing. So maybe this is a valid "use case" that will stimulate developers and architects to think more out of the box.

Things learned at AWS Cloud for the Enterprise: London

2009-11-04T00:34:00.007Z

The event was organised and funded by Amazon to market AWS to the enterprise and took place on Tuesday 3/11/09 in London. This is the list of points summarising what I learned. This list has been created by consolidating a bunch of Tweets fired off in the last hour or so.

Paul Nasrat, Lead System Integrator at Guardian mentioned some of the tools and techniques used to develop and deploy a website to distribute content related to PMs expenses. Technologies also includeed InnoDB and Puppet. Puppet seems definitely a tool I should be looking at for developing an idea I have been having for some time.

Bob Harris, CTO at Channel 4 did the best presentation of the day. The most interesting things he mentioned are:
- Amz roadmap isn't public. He hopes one day it will be to help enterprises to plan ahead
- Amz believes that Cloud computing is at the "peak of inflated expectations" http://is.gd/4Mo1F
- Cloud is no silver bullet: in fact he only deploys web sites for content distribution (one of which is ScienceOfScams)
- The way C4 managed legal issues around Data Protection is by having made an enterprise agreement with Amz covering all those issues
- Myths around cloud (lack of security, low throughput, low reliability, low availability) are just myths - to his experience, anyway.
- You need a techie and a credit card to get going on the Cloud. In fact, although CapEx is limited, he had to do initial investments to guarantee that AMIs being used for his deployments were up to his required standards (bottom line: it's not as simple as it seems)
- Hybrid clouds are next in his pipeline, with the intent of overcoming issues around deployment of enterprise applications in the public cloud.

There were, in total, 4 Amazon customers' presentations/case studies. Yet again all falling in one of the "number crunching" and "glorified web hosting" categories. (That is, I have yet to see a proper enterprise app making best use of the whole set of services and API access that Amazon uses)

Amazon took three years to push in anger the deployment of his retail stack (the book store and the reseller site) on the Cloud. This, to me, it's rather obvious as only recently AWS has reached a level of maturity and a set of functionalities that guarantees a certain success. They have currently migrated about 7% of their applications.

Simone Brunozzi (@simon on Twitter) - Technical Evangelist at AWS - presented an interesting summary of the things to take into account when architecting apps to be deployed on Cloud (see here). In a nutshell:
- Design for failure: apps should be written with failure in mind. Plan for when the app fails (not *if*)
- Loosed coupling sets you free: loose coupling improves resiliency (incidentally any new app developed at Amazon is exposed as a service
- Design for dynamism: leverage the cloud elasticity by making no assumptions on the location, health and configuration of the apps and it's dependencies should be made when the app is developed (note to self: should start making use of user data)
- Security is everywhere: think of security constraints at every level of the stack, OS, network, application (some of these are facilitated by the Amazon infrastructure)
- Don't fear constraints: think laterally on how to overcome constraints - does your app need huge amount of RAM? consider distribution across instances...
- Many storage options: one size doesn't fit all for storage, choose the one right for the type of application needed (see also One Size Doesn't Fit All, also via @simon)
- AWS ecosystem and community: re-use the expertise available in the community and ecosystem of applications both in the open-source and paid-for worlds.

PS: within the context of this post "the Cloud" is actually "the Amazon Cloud"

The real RestFixture and the copy

2009-07-26T23:09:00.004+01:00

I noticed, some time ago, that RestFixture has been implemented on GreenPepper a commercial clone of Fitnesse.
The concept has been taken on board and seamlessly re-implemented to the extent that the developers documentation of the fixture is pretty much identical.
I feel sooo proud!

JMeter to test robustness of system exposing REST APIs

2009-04-06T01:59:00.029+01:00

UPDATE: code moved here: http://github.com/smartrics/JMeterRestSampler

I have been involved on continuously testing systems for robustness. One of my toy projects is the JMeterRestSampler, a plugin for JMeter to ease creation of such tests and automate their execution on a continuous integration server.

Robustness testing
In this context, a robust system is such that if it is subject to a constant load for a given amount of time, its response is correct and its throughput is constant.

The aim of a robustness test is to identify design or development issues such as memory leaks or random/unfrequent bugs that over time may degrade or crash the system. It's worth pointing out that robustness tests differ from load or stress tests which aim more at identifying bottlenecks or actual system limits (responses to peaks or bursts).

Continuously test for robustness is hence a must to improve the quality of the system under construction.
I, Robbie and Raghav presented a session at Agile '08 and XPDay 08 on this very topic.

Specifically for robustness, a test is defined by running scenarios concurrently. A scenario is a set of operations on the system that emulate typical usage. Our examples at both the presentation were based on the robustness tests we built for Aloha, an application server for Voice over IP applications. In this context a scenario - typical usage of the system - is for example "Connect voice call between two participants" that translates into a sequence of simpler operations performed via the application server's public interface.

Robustness with JMeter
When preparing the XPDay presentation, I started musing with JMeter to see if it offered simple means to build robustness tests. The bare essentials i was looking for are

the possibility to build scenarios in relation to the system under test (given the above definition of scenario)
automate tests and run them from a continuous integration server and
the possibility to report on the outcome of the tests.

As I have been involved in building REST APIs for several months, I chose to experiment assuming that the system under test offered a REST exposure.

Scenarios

In JMeter robustness scenarios can be modeled as ThreadGroups. A ThreadGroup groups a set of sampling operations on the system under test. Each sampling operation is executed serially within the thread running the ThreadGroup and each ThreadGroup can be run in loop and in parallel with other ThreadGroups for as long as necessary, either terminating them after a fixed number of runs or after a given time.

All the goodies shipped with JMeter can be used like pre/post processing instructions, definitions of user variables, throttling, etc. (see JMeter manual for more).

The following picture shows a simple test plan for a system under test with a robustness test made of 2 scenarios: "Register customer" and "Place an order". For the "Register customer" scenario, the ThreadGroup will be run in 10 parallel threads and each thead will run in loop for 100 times (so, in toal, the system will be running 1000 scenarios).

Sampling REST resources

JMeter offers a wide selection of samplers for sampling systems and collecting results. It does not offer, though, a proper sampler for the definition of REST requests. In the spirit of readable tests for documentation I wanted each sampler (effectively a REST request) to look like a REST request with a URI, method, list of headers and a body (much in the spirit of the RestFixture). So I discarded the HTTP sampler as too bound to HTTP requests.

So the first stab was to build a new sampler: the REST Sampler. It allows to define a rest request by setting all the relevant parameters: the URI, the headers, the body, the verb. It also processes JMeter assertions and post processors (attached as chidren to the sampler node) to perform basic assertions on the response elements and, for example to extract data from the last response and store it for future reference.

The REST sampler

Asserting the response code of the last response

Extracting data from the response for future reference, via JMeter variables

Monitoring the remote system: JMX Sampler

As said, part of a robustness build is to observe the system under test for key indicators of the health of the system. If the system under test is built in Java, one key element is the amount of heap memory allocated by to the system at any point in time: the trend of the average value of the heap memory may spot memory leaks. To grab this data in the context of a robustness test via JMeter, I have added a JMX Sampler, which basically connects to the given JMX URL and retrieves Heap memory information.

Next, I wanted to present the information collected. If JMeter is run via GUI, a JMX listener can be added to gather and chart the memory data.

It's possible to add more than one JMX sampler pointing to different JMX servers in such cases when the REST application is distributed. Charts of each Heap memory usage are displayed in the same listener window. Also, the charts can be saved in one or more files if needed.

Test Automation

The REST and JMX samplers plus the JMX listener can be used to build robustness tests necessary to test systems exposing REST interfaces. The next step is automation.
Automation is achieved with Ant and the JMeter task, which allows to run the same test plan but from whitin an Ant task (hence invokable via any Continuous integration server). Documentation on how to use the JMeter ant task is available in the website.

I, though, thought to extend the task to generate charts using the data available in the JMeter result output file. The Charts generated are one for the time occurred for each of the rest requests (which gives an indication on how the performance of the system are degrading overtime) and one for the JMX sampling as detailed above.


<taskdef name="jmeter" 
   classname="smartrics.jmeter.ant.JMeterTaskExt" 
   classpathref="project.classpath" />

<target name="run" depends="clean, init, copy">
  <jmeter jmeterhome="${jmeter.home}" 
     resultlogdir="${artefacts}" 
     failure="false" 
     failureproperty="jmeter.failed"
     chartsoutputdir="${artefacts}" 
     succecssThresholdPerc="98" 
     jmeterproperties="${jmeter.home}/bin/user.properties">
     <testplans dir="${build}" includes="Robustness.jmx" />
  </jmeter>
</target>

<target name="report">
  <xslt in="${artefacts}/Robustness.jtl" 
      out="${artefacts}/Robustness.html" 
      style="${etc}/jmeter-results-detail-report_21.xsl" />
  <copy todir="${artefacts}">
    <fileset dir="${etc}" includes="*.jpg" />
  </copy>
</target>

The location where the files are generated is defined by the attribute chartsoutputdir="...", if omitted no chart will be generated.

Correctness and the failure threshold

In certain cases a build may be allowed to pass even if some of the assertions did not pass. Such behaviour is admissible in systems that implement, for example, optimistic locking or other mechanisms that trade correctness for performance. Each REST sampler flags the sample as sucess or failure depending on the result of each associated assertion. By counting such flag values it's easy to determine the percentage of successes.

If the Tester decides that failure is acceptable, she can specify the succecssThresholdPerc="..." attribute to define the minimum percentage of successes that need to happen in order for the build to pass.

Build and isntallation instructions

Build
The JMeterRestSampler is available here:

http://github.com/smartrics/JMeterRestSampler

~~currently hosted in the same SVN trunk as the RestFixture~~
After checkout, add a properties/${user.name}.properties file and define the jmeter.home property. Then run ant.
${user.name} is the username of the logged in user.

If the build passes the file build/JMeterRestSampler.jar is generated.

Install
Edit the ${jmeter.home}/bin/user.properties and set the searh_paths property to the location of the JMeterRestSampler.java file: search_paths=/path/to/JMeterRestSampler.jar. Alternatively, copy the JMeterRestSampler.jar in the directory ${jmeter.home}/lib/ext.

If installation is successful, when starting JMeter, the Rest Sampler and JMX Sampler should be available in the Add/Sampler ThreadGroup context popup menu.

Eclipse project
The .classpath and .project Eclipse metadata files are included. To successfully compile the project within Eclipse, once imported, you need to define two workspace variables JMETER_HOME and ANT_HOME to respectively point to your JMeter home and Ant home directories.

License
The software is licensed using a BSD license.

Disclaimer
I should point out that the JMeterRestSampler is no more than a toy to prove the concept of Robustness testing via JMeter. It started as a spike back in the days of XPDay 08 and evolved - slowly - to it's complete form. I am not intentioned to improve it in the foreseeable future.

Building Spring 3.0 M2 from behind a proxy

2009-03-24T14:42:00.003Z

It took me a while to find out where to put the proxy settings for allowing the ant/ivy build scripts to fetch the dependencies.


$> vi /spring-framework-3.0.0.CI-167/projects/spring-build/lib/ivy/jets3t.properties

then


#...
httpclient.proxy-autodetect=false
httpclient.proxy-host=<your-proxy-host>
httpclient.proxy-port=<your-proxy-port>
#...

This should proxy out any request (to s3). Note that proxy-autodetect is by default true.

You can now build the framework via ant:


/spring-framework-3.0.0.CI-167/projects/build-spring-framework$ ant

Words art: wordles

2009-01-27T01:00:00.006Z

I have found a cool website to create pictures from words, wordles that is. This is what I have come up with using the this blog posts as source.

New kids on the Blues block

2009-01-13T01:50:00.004Z

I just bought the debut CDs of Oli Brown and of Dani Wilde. I'll keep an eye on these two new kids on the block.

A taste of Blues from Oli Brown is here.

Dani Wilde instead can be heard here on a John Lee Hooker classic

Rotten apple meets the amazon

2009-01-10T23:03:00.005Z

I just found that I can't login any more into my iTunes account. "This Apple ID has not yet been used with the iTunes Store" I am told. Clearly it's not the case as I have already spent few bucks in the shop. I have few pounds left - I am hoping to spend as little as possible troubleshooting the issue (first clue is that I just updated to iTunes 8).

Anyway, I will not care for long: I have just relised - a bit late I know - that Amazon is selling DRM free MP3s cheaper than iTunes Store.

And, by the way, iTunes' interface ~~still sucks big time~~ hasn't improved at all.

Quote of the year

2009-01-05T10:45:00.003Z

It may seem obvious, but I believe this quote is pure truth:

"Pleasure in the job puts perfection in the work." (Aristotle)

RestFixture latest additions: JSON support and Sequence Diagrams Generation

2008-11-02T22:14:00.015Z

RestFixture has been enhanced with two new features:

basic support for JSON;

runtime generation of sequence diagrams showing the flow of http calls within a table

First of all, thanks to AndyO as he is the author of the JSON support enhancements. Now RestFixture allows testers to write and verify expectations for content returned in JSON.

As for the second feature, the generation of sequence diagrams from a RestFixture table, the idea has been growing since i found UMLGraph, a nice tool that generates class and sequence diagrams. The idea is that for describing complex REST API interactions... a picture is worth a thousand words.

Here follows two simple diagrams and associated tables to demonstrate the generation of the diagrams.
Note that the diagrams are generated at execution time, therefore they reflect the actual outcome of the run of a RestFixture test. This means that if the test fails the picture shows the flow of messages that failed the test.

The diagrams are pretty basic and they only show the objects (or resources) involved, the verb, the return code and, for POST the id, taken from the Location header.

Full instructions on how to install the required software and how to use the new feature are available in the RestFixture web site.

Support for generation of the diagrams is basic. It has been extracted out of the main RestFixture trunk as it's still in spike mode. The code, though is checked in the same SVN trunk (the project is called PicBuilder).

As always comments are very welcome!!

Getting immediate and accurate feedback from the IDE

2008-10-26T12:50:00.006Z

In Agile methodologies getting feedback to adjust future work is a valuable practice to follow. In reality this is nothing new, in any human activity information gathered from feedback received on the system in development is used to drive activities to improve the chances to get the desired output.

Agile methodologies typically address feedback from the customer perspective: the team produces the agreed artifacts, the customer provides feedback on the outcome, both together re-plan future activities to adjust appropriately to get maximum business value from new delivered artifacts.

Feedback is characterized by two properties: its “immediacy” and its “accuracy”. Immediacy is the perceived time elapsed between when feedback gathering starts and when it's actually obtained. Accuracy is related to the usefulness of the information extracted from feedback.

Bearing in mind these two properties, feedback can be extended to development activities by appropriately configuring development tools and refine development practices. The aim is to get feedback “now” (immediate) and “spot on” (accurate).

The source of ultimate feedback for a developer is the build process. Typically, though, “the build” is a resource intensive process that happens on a continuous build/integration server (for example Cruise Control). This process consists mainly of a sequence of steps that include checkout from source control repository, compilation of the code, execution of the unit tests, metrics gathering, deployment in some test environment, execution of the customer acceptance tests, documentation generation.
These steps are executed using the entire source tree and provide the ultimate feedback on whether the latest code changes have broken or not any of the existing functionalities. Clearly the larger the code base is the longer the build takes. In a short “write test – write code – check-in” loop (typical of a TDD approach) though it's usually unacceptable to wait even more than few seconds to understand if the code added breaks the build or not. So even for short builds (in the range of few minutes) it's advisable to use the IDE to help improving immediacy (by shortening feedback loop back time), albeit without diminishing accuracy.
One way to do that is to use the IDE to execute part of the whole build process locally, possibly in the IDE itself.
Taking Eclipse IDE as an example, the following is a list of tips that may be used to gather immediate and accurate feedback.

Use the JUnit plug-in to run the test just written until the code that passes it is complete (well, no new news here, I am sure most of us do this already). The caveat in this tip is to make sure that the result is accurate; way the plug-in runs the tests is equivalent to the way the CI executes them, that is via – typically – an Ant script. Whilst this is immediate for simple tests and code structures, it gets very hairy when frameworks are used (for example SpringFramework) which require configuration files in the classpath and/or external library dependencies. So the first thing to check is that the classpath in the Ant on which JUnit runs is the same as the one in which the test runs in Eclipse. One caveat to this approach is that Eclipse IDE is limited to a single classpath per project so it's less flexible than Ant, where order and content of a classpath can be more fine grained. So typically any “target” in an Ant build script should refer to a “project classpath” path definition that refers to the same set of directories and files in the Eclipse project classpath and in the same order.

Use code metrics plug-ins to run metrics on the fly on single files and incrementally on packages and on the entire source tree. The rationale is that in a short TDD loop the number of files touched is minimal, so running the metrics in that subset first and eventually run them on larger sets, improves the chances to find problems sooner. Eclipse IDE can be enriched with all sorts of plug-ins, including those for Checkstyle, FindBugs, JDepend. And to gather accurate feedback it's necessary that the configuration files for each one of these tools/plug-ins is shared between Eclipse IDE and the main build – it's usually good practice to check the configuration files in the same code tree so that they can be easily referred from the IDE and Ant.

Use the Ant plug-in in the IDE to execute single or set of targets in the IDE. Accuracy is guaranteed by having the Ant environments in the CI and in the IDE equivalent – this includes also redefining the Ant Home in the IDE to run the same version of Ant in all the environments. Running Ant in the IDE rather than in a shell offers the limited advantage of the IDE support on parsing the output from the build (context sensitive help and hyper-links to the source code).

There are plenty of other clever usages of the IDE features that can drastically improve feedback gathering; some of them are specifically for Eclipse, others exportable to any other IDE as the ones described above. In all examples, though the key message is that immediacy must not be achieved at expenses of accuracy (as my friend Paul would say "feedback is only valuable when it's accurate").

Cultural challenges

2008-09-11T14:34:00.004+01:00

Train Station, Coffee shop (I won't mention the name), 8am of a typical commuting day

Barista: Yes, please!
Me: Can I have a single espresso, please?
Barista: Sure!
(Barista goes to the coffee machine, but suddently comes back)
Barista: Sir, sure you want a single espresso? The Double espresso has the same price.
Me: Yes, I am sure, thanks.
Barista: (Baffled) Uh... Ok!

(Although, the most upsetting thing, after this conversation, is when the cup is 3/4 full)

Agile 2008 - Report

2008-08-27T23:31:00.004+01:00

As promised a few days ago, this is my report on the sessions I attended at Agile 2008.

Before the comments on the session I attended, I would like to mention that the whole conference did not blew me away. I mean, it was very good: the hotel, the location, the people, our active participation with two talks and all of that. But I had every day the feeling that all of that was too much. I am not used to group gatherings of this size so that may explain. Anyway, there were great sessions going on, some of which I could not attend for work commitments. And this time around I decided to attend different types of sessions, outside my comfort zone.
I have learned a lot from attending the sessions described below and - not surprisingly - I came back home with more questions than answers. Happily I am still in touch with some people I met at the conference following up ideas initially discussed there. So thumbs up for me over all to the whole experience. And Toronto is a great City.

Keynotes

"The Wisdom of Crowd" by James Surowiecki

Without a doubt this was an interesting opening session. It clearly was aimed to support the speaker's book The Wisdom of the Crowd, but non-the-less new news for me. I got interested in collective intelligence when I read Programming Collective Intelligence, aimed to a more technical audience. This keynote gave me the possibility of know of another aspect of the same topic: how to extract information from a group of people when they are identified as an entity on its own. The crowd, that is.

The common way to extract information from the crowd is by aggregating information coming from the single entities in the crowd. One crucial aspect - very well discussed in this speech - though is how to make sure that "crowd" is built to maximise the chances that the aggregation of ideas can generate new information: this is a case of "the total is bigger than the sum of all the single parts"

So, the key point is that under the right conditions a group of people can be effectively intelligent, contrary to the commonly perceived thinking that crowd are volatile and stupid. Obviously the catch is to understand what the right conditions are.

The right conditions are "diversity" and "independence".

Diversity has to be understood as cognitive diversity, not sociological: different approach to problem solving, different levels of skills and technical capabilities, as opposite to race, sex or social class. The speaker refers to different anecdotes (taken from the book) that prove its hypothesis, inferring also that diversity is even more important in small groups. In essence, homogeneous groups tent to hide flaws as ideas are always analysed in the same way (that is solutions are found by agreement rather than by conflict). In these cases an approach could be to have someone playing "Devil's advocate", with the aim to look at ideas and outcomes from a non standard perspective.

Independence is as much important as diversity. Looking at the crowd as the entity providing the most accurate and effective solution to a given problem, does not mean that the individuality is not important. Individuals must be independent and be able to express their best in any given situation. If this doesn't happen the aggregated results are biased and possibly wrong. This may happen because by nature individuals tent to imitate each other or because organizations punish diversity.

So overall an enlightening speech that I can relate to on my day to day job.

“Quintessence” by Robert "Uncle Bob" Martin

This session happened just after the Thursday's gala dinner. I must say that Bob Martin is a character. On the content of his session, the main points were related to how Agile is implemented in the real life and on how the developer community should behave professionally and push back to those who want them to deliver crap code on time.

About agility, the focus was on XP and Scrum and how - especially at the beginning - the two school of thoughts differed. XP more strict and guided, Scrum more relaxed about actual software development practices. He did an interesting experiment during the session. He asked the attendees to raise their hands. He then started to enunciate each of the XP practices asking people to put their hands down if they weren't following the specific practice.

I reckon we were over 1000 people (if not more) and at the end there must have been 10 people with hands up. I think the point o the experiment was to show that we are at a stage where people blend practices and principles of Scrum and XP to make what works for them.

The other crucial point made was on developers having to put courage on the line and never compromise on good code for new features to accommodate insane deadlines or scarce resources. "We value good code over crap" was suggested as another (unlikely) entry in the Agile Manifesto.

So a very interesting pitch full of truth and controversy. I can not disagree with most of the talk. I still have to find though a place where developers have such a discipline that they can draw a line between over-engineering, quality, speed and alignment with business goals.

"The Wisdom of Experience" by Alan Cooper

The closure keynote was nothing new to me having read The Inmates are running the asylum. Same message as Uncle Bob's for quality over quantity. The speech was full of anecdotes of teams and companies shipping "on time and on budget" bad and unsuccessful software. The key is "take your time and produce good and complete software" that people can enjoy using. The way to nirvana seems to complement agile principles. He's suggesting a typical lack of good software design (especially interfaces) and he's proposing to spend and invest on good and correct design even before a single line of code is written.

All the second part of the talk was dedicated to advertise "interaction design", the new speciality he is proposing and how this can bridge the gap between technical skills and business skills. An interaction designer knows what people want and how to make software usable and easy to understand. Interfaces should not be designed by developers. Users on the contrary should not be allowed to create stories directly to the developers. It is job of this new type of designers to filter and adjust to have the two groups get together for a better solution.

He's got a point: we all know of crappy software with poor user interfaces or with incomprehensible behaviour. Whether fixing this requires a full time job and a specific career path - the way he envisages - is still to be seen.

Sessions

Agile contracting - Rachel Weston, Chris Spagnuolo

I attended this session with the hope to know more about contracts for collaboration between a client and a contractor adopting agile methodologies.

I got a set of problem statements with some strategies on how to mitigate the risks and a bunch of statistics trying to show how agile is better than waterfall.
By the way, I have an issue when agile is sold using numbers like 93% increased productivity or 83% improved satisfaction versus a traditional waterfall approach that gives 35% successful projects. Firstly must be made clear - even more than the numbers - on what conditions these numbers are collected. Second, even thinking about statistics, saying that only X% of projects adopting waterfall is successful doesn't mean that the next project has the X% probability of being successful. The number of independent events that may occur and the people involved are completely different from project to project that the assumption is mathematically wrong. Hence using it as selling point is... incorrect.

Anyway, the most important takeaways from this session were an overall understanding of what problems contractor supporting agile do face. Mainly

having to compete with "non agile" competitors, who may offer a more sound sense of security in delivering quality software on time.
having to deal with customers who do not embrace agile or that are not fully supportive or that don't have the means to keep up with the fast agile pace
having to provide at bid time of predictability on scope and schedule
having to match customer expectations and internal financial department as for invoicing, payments and costs.

Strategies suggested, directly coming from the experience of the speakers were mainly focussed on

keeping the contract simple.
engaging and educating the customer to the agile practices and explain the technicalities.
clearly state the responsibilities of both parties (customer and contractor).
work out from past velocity charts what can be done, might be done won't be done.
agree with the customer on sharing the risks: loosely define scope when schedule and resource are fixed or instead of the classic time and material.
make sales people responsible of the performance of the contract, so they're focussed on knowing what they're selling.

Overall a good session, especially for beginners, with few pointers to think about. Surely, the devil is in the details, but overall the same message applies here: it's all about people collaborating and communicating effectively to achieve a shared goal.

Crafting User Stories – Four Experts and the audience weigh in

I ended up to this session with lots of hope. When talking of user stories it's always easy to come up with some techniques good on paper but not effectively reusable in real life. I must admit, I didn't get the answers I was looking for, but the big plus was to observe 5 experts in the subject matter (dis)agreeing on some concepts that maybe some time ago were considered heresy.
Each expert in the panel was asked to introduce himself and give an overview of their approach to user stories. The moderator then asked attendees to come up with questions for the panel to debate/answer.
I will here report my learning points.

it’s important for everybody in the team to understand where a story fits in the picture
the format in which stories are formulated is not important as long as few key parts are somehow present: what the story is all about, what it costs to have it implemented (whatever the unit of cost is), who’s benefiting from the implementation of the story and what benefit the story is going to provide, how can stakeholders know when it’s done and to what extent. Also important for me is to know “who” can accept it.
there’s no harm on having stories in the backlog specifically tailored to solve technical problems, as long as the points raised above are clear in the story.

Leading agile teams - Mike Griffith

This was a 90mins workshop for agile leaders aiming to learn leadership techniques. The speaker took the audience to a journey where all the aspects of leadership were analysed, especially those revolving around the team leadership.
One of the most important points raised was the importance of maintaining vision of what the team is building and why.
The speaker suggests using the “the product box” technique. The team is supposed to come up with a cardboard box (possibly to be available in the office) representing the product with a name, a logo and its main features. The intent is to maintain focus on what the team is building by means of a real physical object.
Techniques aside, the main learning point on this was that vision is so important that job of the leader is to make sure that it’s consistently updated and propagated to the team regularly.
It was also stressed the difference between managing and leading (the latter being build on the former) and – interestingly – the job of the leader is to build a team that eventually behaves like the “Orpheus orchestra” – which plays without a designated “Maestro” or like the Canadian geese, who make of self organisation and team work a matter of life and death.
For more information on the subject see http://www.apln.org and http://www.leadinganswers.com

Memorable mention for

Evolution of the Tools and Practices of a Large Distributed Agile Team

Pushing the boundaries of testing and Continuous Integration (for this talk you can find the slides here and a video of the talk here - thanks Robbie!)

RestFixture is now available

2008-08-20T03:47:00.005+01:00

I have made the RestFixture (discussed in this post) available on GitHub here: http://code.google.com/p/rest-fixture/.

You can get it, use it and modify it under LGPL terms.

Feedback is more than welcome, of course.

Agile 2008 - part 1

2008-08-10T15:32:00.006+01:00

Oh yes, Agile 2008... here I am, just landed from Toronto. I am just reshaping my notes. I'll be writing (as I have also done in the past) my comments on the sessions I have attended and presented, and about old friends I have talked to (Rick, Rachel, Angela, JB, Steve) and new people I have met. Stay tuned.

Get FitNesse with some Rest

2008-08-02T04:10:00.015+01:00

UPDATE: code moved here: http://github.com/smartrics/RestFixture

UPDATE II: Thank you to Steven Haines for the clear and concise Howto giude on the RestFixture.

I am currently involved in building a REST API. Our direct customer proxy (the architecture team) needs to "understand" how we implement the agreed acceptance criteria and it has also asked to document the API.
FitNesse is our tool of choice for writing functional Customer Acceptance Tests; we currently use it to implement this type of tests for other parts of the system we're building. So it came natural to start using it for documenting the API.

At first we implemented the tests using essentially ActionFixtures. The approach adopted consisted of "wrapping" the REST API in more descriptive metods that could be pressed, checked and entered.

After the first dozen of tests it was apparent that this approach was not ideal. Tests were not clear enough and maintaining the fixtures was hard and time consuming as the amount of code duplication was high for any standard. New tests required us writing more code and, in fact, we were testing just the behaviour of the backend without giving enough exposure to the API itself - consequently, our tests were not descriptive enough to work as live documentation.

So I decided to write a new "type" of fixture, inspired by the ActionFixture, the RestFixture.

The core principles underpinning the decision to write a new fixture were the following:

For documenting a REST API you need to show how the API looks like. For REST this means

show what the resource URI looks like. For example /resource-a/123/resource-b/234
show what HTTP operation is being executed on that resource. Specifically which one fo the main HTTP verbs where under test (GET, POST, PUT, DELETE, HEAD, OPTIONS).

have the ability to set headers and body in the request

check expectations on the return code of the call in order to document the behaviour of the API

check expectation on the HTTP headers and body in the response. Again, to document the behaviour

I didn't want to maintain fixture code. If I could only write the tests...
I wanted to be able to let the customer proxies to write the tests... And they understand Wiki syntax more than Java.

The RestFixture at a glance

The RestRixture is an ActionFixture, therefore all the ActionFixture goodies are available. On top of that it contains the following 7 methods:

header: to be able to set the headers for the next request (a CRLF separated list of name:value pairs

body: to allow request body input, essential for PUT and POST

let: to allow data from the response headers and body to be extracted and assigned to a label that can then be passed around. For example our API specifies that when you create a resource using POST, the newly created resource URI is in the Location header. This URI is necessary in order to perform further operations on that resource (for example, DELETE it in a teardown).

GET, POST, PUT, DELETE, to execute requests.

Each test is a rown on a RestFixture table and it has the following format:

|VERB|uri|?ret|?headers|?body|

VERB is one of GET, POST, PUT, DELETE (at the moment we're not supporting HEAD and OPTIONS)

uri is the resource URI

?ret is the expected return code of the request. it can be expressed as a regular expression, that is you can write 2\d\d if you expect any code between 200 and 299.

?headers is the expected list of headers. In fact, the expectation is checked by verifying that *all* the headers in here are present in the response. Each header must go in a newline and both name and value can be expressed as regular expressions to match in order to verify inclusion.

?body it's the expected body in the response. This is expressed as a list of XPath expressions to allow greater flexibility.

Some examples

The following pictures are snapshots taken from FitNesse with the aim of providing examples of usage of the RestFixture. The notes before each test explain the details of the fixture itself.

It's worth noticing that, generally, expectations are not matched by string comparing the expected value with the actual value. Expectations on headers are verified by checking that the expected set of headers is included in the actual set of headers, similarly with the body where expectations are matched by checking existences of nodes for a given path.

Conclusions

The RestFixture allows to write FitNesse tests without having to write any Java code to back up the fixtures. Tests are clear and easy to read/write and this obviously improves their readability. Therefore it works well for documenting the API and the behaviour of the system under test. Looking it from another angle, the fixture, essentially, implements a REST DSL, that allows customers to write tests on their own.

Groovy for XML transformation

2008-07-06T00:59:00.008+01:00

I have been playing with groovy recently, specifically with the MarkupBuilder, the groovy native support for markup languages. It basically means writing XML using native groovy syntax. Pretty neat.

In my job I have more often than not, have to write software to integrate two or more systems. This, often, means writing code that reads an XML stream onto a Java object for manipulation and eventually writes the result into another XML stream.

This typically involves creating binding objects for the input XML and the output XML and then a sequence of calls to getters on the input object and setters on output object to implement the mapping.

This task is very error prone and tedious. A good solution is using the MarkupBuilder (and, yes, I know about XSLT, but this is not the point here!)

So, suppose that you have the following POJO


public class Contact {
  private String id;
  private String name;
  private String surname;
  // ...
}

and you want to serialize the instance


Contact c = new Contact();
c.setId("123");
c.setName("John");
c.setSurname("Bloggs");

into the following XML


<contact>
  <key>123</key>
  <firstname>John</firstname>
  <secondname>Bloggs</secondname>
</contact>

Note the difference between the POJO attribute names and the XML tag names

Using groovy and its MarkupBuilder, the first thing to do is to create a groovy converter, a class with a method that reads the data from the bean and produces the XML. Create a file src/groovy/ContactConverter.groovy with the following code:

class ContactConverter{
  def convert(bean){
     def writer = new StringWriter();
     def xml = new MarkupBuilder();
     xml.contact {
        firstname(bean.name)
        secondname(bean.surname)
     }
     writer.toString()
  }
}

If you execute the code

println new COntactConverter().convert(c)

in a groovy shell, you get the XML shown above.

The next step is then to make this code executable from your Java service.
You can use this class (an adaptation of the code available in the groovy documentation):

public class GroovyConverterInvoker {
  public String invoke(String fileName, Object bean){
    GroovyObject groovyObject = createObjectFromStreamName(strName);
    String result = (String)groovyObject.invokeMethod("convert", new Object[]{bean});
    return result;  
  }

  public GroovyObject createObjectFromStreamName(String fileName) {
    ClassLoader parent = getClass().getClassLoader();
    GroovyClassLoader loader = new GroovyClassLoader(parent);
    Class groovyClass;
    File f = new File(fileName);
    try {
      groovyClass = loader.parseClass(f);
      return (GroovyObject)groovyClass.newInstance();
    } catch (CompilationFailedException e) {
      throw new IllegalStateException("Unable to compile " + fileName, e);
    } catch (IOException e) {
      throw new IllegalStateException("Unable to open file " + fileName, e);
    } catch (InstantiationException e) {
      throw new IllegalStateException("Unable instantiate object for class in " + fileName, e);
    } catch (IllegalAccessException e) {
      throw new IllegalStateException("Unable access object of class in " + fileName, e);
    }
  }
}

You can then invoke the conversion:

public String invokeContactConverter(Object bean){
    String fName = "src/groovy/ContactConverter.groovy";
    return new GroovyConverterInvoker().invoke(fName, bean);
}

You can now use JAXB/XStream or XmlBeans to parse the result of invokeContactConverter and initialize a new POJO for further use, or send the XML on the wire.

The solution can be generalised and optimised to a point where you only need to write groovy converters and load/modify them dynamically when necessary.

Sicilian as a language

2008-07-05T12:32:00.003+01:00

I am randomly browsing the web today. I just finished to read a Wikipedia entry on the Sicilian language. I always thought that Sicilian was an Italian dialect, although I was aware on several influences from French, Spanish, Latin and Greek, but I realised that scholars do consider it as a language in its own merit. That makes Sicilians bilingual at least.

UMLGaph for auto generating class diagrams from your source code

2008-07-01T00:43:00.003+01:00

As part of improving the documentation of our software - a REST API - for the benefit of our customer (mainly an architect) and (future) users, we have investigated UMLGraph, a free application that is able to parse source code and (with the help of graphviz) generate class diagrams that can be embedded in the javadoc documentation.

The tool is very good and we're trying now to integrate it in our continuous integration build via ant.

BT Web21C SDK wins eWEEK Excellence Award

2008-06-09T23:59:00.005+01:00

The BT Web21C SDK has been awarded with the eWEEK Excellence Award under the category "Application Development" as reported here. I am part of the team who built it and I feel proud that 2 years of hard work have been recognized.

Smartrics tag cloud

2008-06-08T13:24:00.004+01:00

Go figure why the guys at blogspot have not provided a tag cloud widget. So, kudos to phydeaux3 for supplying the code for my tag cloud. Except for changing the font colors and types to make it look better it was a matter of copy and paste in my blog template. Nice and easy...

The (hyper)reality of social networking

2008-06-07T12:23:00.005+01:00

I was reading about hyperreality when I started thinking of how it relates to the social networking phenomenon and communities built around it. Is there a connection between the two? Maybe. In fact someone else has already discussed this topic in this rather interesting post.

My favourite definition of hyperreality is Umberto Eco's one: "the authentic fake". As you would expect the web is full of information about hyperreality, for example this is fairly complete article with examples.

I'll try to write down my thoughts on how social networking community members do live in a hyperreal world. I should warn you though: I may sound cynical and destructive. Far from me: I think there's a place for social networking web sites and services (after all I am blogger too) and there's nothing wrong with being a consumer of these services. As far as I am concerned now I am just trying to understand the mechanics, why they work and why they exist the way they are now, and how they will transform in the future to come. So bear with me on this aspect.

At the core of the meaning of hyperreality is the ambiguity between "real" and "fake" reality that arises when facts and events or objects are filtered by a multitude of media.

Members of communities built around any of the latest and cutest Web2.0 website interact by the means and rules offered by the portal they use (Facebook, MySpace and the like). This medium filters the behaviour adopted by people. Actually, members of any community, in a way, always interact within an accepted framework: think of the rules, implicit (values, culture) or explicit (laws), that manage our relationship in a social environment and how they drive our behaviour.
The main difference, though, is that people meeting face to face have other means (environment, body language, time...) that contribute to maximise the possibility that facts and events are perceived and understood correctly and close to the reality.

Now back to the social networking. As said the web shapes the behaviour; there's also in the mix the fact that content and messages are exchanged using a different time pattern compared to normal face to face relationships. A message, a profile, a blog post, a twit is sent to the server and then delivered to the intender recipient(s). The recipient(s) reads it, digests it, formulates a response and eventually sends it back. This, to me, contributes to the creation of a world and of a reality that is transformed: what the recipient perceives is the autentically "fake" reality that the sender wants to transmit having lost its immediacy and being tailored to the media chosen to operate upon. Besides the fact that sender and recipient know that the content being generated is public.
I like to think that communities in Facebook (I am not picking on Facebook, it's just an example) are equivalent to the community of house mates in the BigBrother house (or, for argument's sake, to any other reality). The media, the place and being observed changes the behaviour allowing the community (participants and viewers) to create a fake real world.

So, it seems that social networking is son of its time: it has been made possible by new technologies - higher network speed, new web site capabilities, etc - and it's expression of our current time where appearance rules.

I have a memory from my infancy: if you're Italian or you watch Italian TV you know about the Mulino Bianco commercial (this is one of the many) which epitomises the (hyper)real world. We (the community of consumers) are let to believe that that is a beautiful real place with a real family where - obviously - plumcakes and brioches are soft and tasty.

Is there any difference between the Mulino Bianco ad and any of the profiles in Facebook? Let alone the style and the manufacture - Facebook users may not be professional advertisers - the communality is that both are hyperworld where the authors want the recipients to walk in and live the life they want them to live.

In this context let me fool around. If hyperreality is made of hyperreal facts and hyperreal events that exist in a hyperreal world, then Twitter allows users to exchange hyperreal facts; MySpace and Facebook as hyperreal multi-worlds; SecondLife as real hyperreal world.

The astute reader may now say that hyperreality is just a play on words, people make real money with this sites, businesses and enterprises back them up. It depends, it's a matter of prospective. Vegas and its casinos is extreme hyperreality made true. Punters entering a casino get in a manufactured hyperreal place where everything is obviously fake to let you believe that our money is fake too. Incidentally, those who think that your money isn't fake are casino owners who live in the real reality of gambling business.

So, who's making real money in this hyperreal world? I guess those who are able to spoil the user data by aggregating it and extract useful market information, for example. I suspect that biases will eliminate each other when aggregated, providing the clever data manager with information to use at his like (hopefully within the boundaries of the data protection acts).

A real example is available here: it describes how Amazon makes money using information extracted by user generated data.

Yes, clearly this is a win-win situation. Community members get value by being able to cultivate their social aspirations, voyeurism, need to be heard, trend and fashion, benefiting of other members' experience. Providers - on the other side - are able to target established communities by processing generating data, aggregating it with the purpose of extracting financially useful information (read the Facebook case study).

I am sure there's more to say. I stop now but I am interested to hear from you, dear reader, your view on this.

Covering the obvious

2008-06-07T11:34:00.009+01:00

I was listening to Led Zeppelin whilst doing the restyling job at home and acknowledging how popular some of their songs are. Obviously Stairway to Heaven is amongst those... so I started wondering how many covers have been published in almost 40 years. I started digging the YouTube website, but, funny enough, at the same time I found that Ernesto Assante - a famous italian music critic - did it all for me by listing on his blog the most unusual covers of that masterpiece. The one I like amongst those he's listing is the version of Robyne Dunne, very 80s' and colorful. Here it is:

Agile restyling

2008-05-31T21:37:00.003+01:00

Whilst my wife and son are away I took few days off at work and started to mess with the floor and the walls in the corridor of my house.
I am a brave man... with the help of a DIY book, some imagination and memories of a decorator who worked in my parents' house 20 odd years ago, I started taking off the horrible green carpet I found when I moved in. Surprise! Under the carpet there was an even more horrible black and red vinyl tiles floor stuck on the floor boards. Three days of hammering off the tiles inch by inch with a chisel didn't do the job: I had to hire a floor sander and use 7 P24 - amongst the coarsest - sand paper disks to get rid of the mixture of plastic and glue stuck in the 6 square mt area I am working on.
I took off the skirting boards and a portion of the wall plaster and redid all the necessary fixes to modernise the look and feel. I started four days ago and am now at the stage of painting all the surfaces i can pass a brush on.
Plenty to do and little time! No new news then.

I am learning though... a lot of things! I am practicing day by day (and mistake by mistake) new manual skills: lying undercoat and plaster, sanding, varnishing, painting and I haven't even started with the wood. But I am (re)discovering how to relax by doing manual activities: I am on my own, I am not under pressure to finish (yet!) and I can take my time whilst listening to my music. I am also fooling around by trying to apply agile techniques to my day to day activities. Ok, there's a conflict of interest (I am the customer and the development team) but I avoided a big upfront design and I am organising my activities to best respond to changing circumstances - this in fact has helped: I found on Friday that I have a meeting in London on Monday that I don't want to miss and this has forced a re-org of my activities.
In practice, I have a list of things I need to do and it's comprehensive to the best of my knowledge to date. I keep adding (or removing) to-do items as and when they come and re-prioritise the list every day: I pick the next to-do item from the list and do it, depending on what makes sense doing (I am the customer after all), what I fancy doing (as I am the developer too). I tend also to organise the list to minimise waste (of time mainly) taking into account task dependencies (take off the carpet before lying the new floor) and minimise the risks (paint the wall before lying the floor, because the paint may drip on it!)
The only visible side effect is that I need to go frequently to the nearby DIY shop to buy stuff. Not a big deal though, it's only 2 miles away, and it's also an excuse to see the cloudy sky.

PS: amazingly, when I took one of the skirting board off the wall, I found a little book published by Lloyds Bank with the list of cash points in the whole UK - all in 6 pages, mind you - and a 2p coin. Respectively dated 1973 and 1974.

Uncle Pino

2008-05-31T17:35:00.001+01:00

Sadly today my uncle Pino passed away. He was a fine musician who played guitar and bass and toured the world with many famous Italian artists including Domenico Modugno and Nini Rosso. He also published records: the one I can very well remember is titled Marmalade, back in the late 70's (copies are on sale on eBay.it at the time of writing).
I still remember the story that granma Teresa used to tell me, when I was a little boy, of him leaving Sicily when he turned 18 to fulfill his dream of being a musician.
And I'll be always grateful to him for having bought in Rome in March 1985, on behalf of my parents, a - very much desired - Commodore 64: my parents' present for my birthday. (That machine started my passion for computers and programming).

The Boss is in town!

2008-05-31T10:32:00.001+01:00

Great show! Probably the best I have seen in years. Bruce Springsteen and the E Street Band have played at the - almost full - Emirate Stadium in London a 3 hours gig with a mixture of old and new stuff. You won't believe how that 62 years old man was able to excite the crowd running up and down the stage shouting and playing Badlands, Born To Run, Thunder road (my favourite), Rosalita, Lonesome Day and so on.
He also had time to pass on to us a couple of (silly) jokes about English stereotypes: the weather - today was a sunny day after a miserable week of clouds and rain - and T time. Anyway... I took two pictures with my mobile, one before the start of the concert when the stadium was filling up, the other when it was dark already so the quality is pitiful, so you'll have to guess.

Fighting cot death and side effects

2008-05-30T00:13:00.001+01:00

It's today's news that some cases of cot death may be caused by bacterial infections.

If you have a child it's likely that a pediatrician (or equivalent) has told you about cot death - the death of an infant that is apparently inexplicable.

When my son Jacopo was born (19 months ago) we were told when we left the hospital just after he was born. We were also told few things to do to minimise the chances that this horrible situation may occur to us: don't smoke nearby your newborn, don't let him sleep in our bed and - above all - put him to sleep on his back (an not on his tummy, as commonly practiced until about 15 years ago).

All nice and easy. But...

But nobody told us of a naughty side effect of one of these simple practices. If your child always stays on his back (and most of his time he/she does as babies sleep more than 12h a day - also, our one actually disliked being put on his tummy even when he was awake) he/she may develop the flat head syndrome. This is a cosmetic syndrome whereby the soft skull of the baby flattens. It's - clearly - caused by the fact that the baby lies face up with the back of his head on a not so soft surface (soft pillows should not be used with newborns).

As said we weren't told and Jacopo developed a minor flat head on the right side of his skull. We were told by the pediatrician, after we realised it when he was 9 months old, not to worry, that it'll normalise by the time he's two (it hasn't yet, and we're 5 months away from his second birthday) and that this might have happened. We were told of tricks to play to encourage him to turn his head more frequently - move toys from one side of the bed to the other - and optionally to buy a specially made pillow. In the most extreme cases an helmet may be required but it wasn't our case.

Well, to the point... why nobody told us of what might have happened? Obviously preventing cot death is of highest priority, but making parents aware of side effects of having the baby sleep on his back is also important. So, if you're in the same situation we were a year and a half ago, invent something and have your baby move the head when he's awake. He'll be grateful when he'll grow up.

Agile 2008 - we'll be there!

2008-05-28T20:45:00.000+01:00

We are finalising our work for Agile 2008. I led two teams that submitted two sessions at the conference this August in Toronto, Canada. We had them accepted and the two papers will be published in the proceedings. We're now in the process of writing a summary of the sessions for the conference agenda. Here they follow

What's in the toolbox of a successful software craftsman?
Have you ever wanted to know which tools a big distributed team of successful software craftsman use to implement their user stories? How they configure them to support agile development based on XP and Scrum and deliver to the agreed plan? This session will answer these questions and more. Three representatives of this team will tell you what’s in their toolbox and how the toolbox supports four core agile practices that the team adopts to succeed: maximum project status visibility, effective communication, immediate feedback and ruthless automation.

Pushing the boundaries of testing and Continuous Integration
In this session, three representatives of an agile team will show how an automated build that executes robustness, scalability and performance tests helped them drastically improve the quality of their highly concurrent application server. They will also show how the team configured such builds in their continuous integration environment as well as what performance and robustness metrics they monitored. Finally, the team will show how valuable and effective this investment has been for capturing bugs and performance-related issues very early in their development process.

I look forward to get there: I'll hopefully meet people I already had the chance to talk to in the past (Kent Beck and JB) and people I only recently had the honour to collaborate with (Ron Jeffries and Manfred Lange - who reviewed our papers for the conference).

I'll keep you posted!

And now a public blog

2008-05-24T23:07:00.001+01:00

Subtitle: do I really need to do this?

Hello reader! Welcome to YAUB (yet another useless blog). I have decided to join the public blogsphere. The more obvious reasons are that I hope to practice my writing and use this tool to "remember" and "share" stuff (scripta manent, after all). Specifically things that I happen to be doing during my life. About the real motivation, don't ask, I am still looking for it: I suppose it all has to do with the fact that humans are social animals and that they achieve well being by participating to social life and the like. We'll see how it goes...

Actually, I have been publishing posts in a blog in the intranet of my company for few months; now that I am doing the leap I'll migrate my old posts soon so you can enjoy them too. So you'll eventually see posts that are older than this one once I finish my copy and paste job.

Cya!

Microsoft and Open Source

2008-04-29T12:46:00.001+01:00

It's very interesting to note the Microsoft's involvment in the Open Source (or open source) arena.
I came across this presentation held by Sam Ramji - director of the Open Source Lab at Microsoft - at the latest EclipseCon. It shows how Microsoft and Eclipse Foundation are working together on making Higgins and CardSpace interoperable and on building SWT on top of WPF (to let Eclipse shine on Vista).

One thought coming on my mind: how long is it going to take to have Visual Studio as an Eclipse plug-in.

Planning in Cork and Cyclomatic Complexity

2008-04-18T23:09:00.001+01:00

I am in charge of prepping the Day0 happening in Cork, for the next Release Planning of the Web21C SDK Team.

A bit of background: Day0 tradition started a year ago (at the Dublin Release Planning) as an opportunity for the whole team members (located between London, Ipswich, Denver in US) to meet and share knowledge. It happens the day before Release Planning - hence the name - and involves technical presentations and discussions on topics selected by a small number of members and presented to the whole team in - typically - two or three streams.

The process is quite interesting as we're organising the event as a mix of Open Space and Unconference. I have asked people to sign up for sessions to present and for topics to discuss at the Gold Fish Bowl at the end of the day. (Yeah, I know that a preset agenda is not truly Open Space, but it'll help me to manage the hotel resources and keep people focussed, whilst having the attendee to comment and provide feedback to the presenters - and whoever has something to say on the day can book his session at the start of the day on a properly prepared sheet)

I am confident that the event will be as successful as the one I organised in Glasgow last January.

I am even attempting to present two sessions although I have the feeling it's going to be too much, being now involved on preparing the two papers for Agile 2008 the release of the Messaging capability at the back of the Web21C SDK next 28th of April and the Brown Bag at the end of May.

Anyway, one of the sessions I was thinking to present is on software metrics and refactoring complexity out of code. The goods the bads and the evils of metrics and how they work as triggering alarms on spotting complexity in the design and implementation. I am thinking how to best make use of those numbers without being too mental on making them nice and round. Especially the Cyclomatic Complexity, which I find quite useful as an indication of the complexity of the code. In fact with Eclipse and the Checkstyle plugin, it's possible to get the number on the fly. Another useful application of the McCabe number (the other alias of the CC) is to get an indication on how many tests are required to fully cover untested code - useful clearly when you inherit untested code.

I'll think of writing a more extensive post once I finish the presentation

Euro and Pound signs in Java

2008-04-18T13:56:00.003+01:00

Never use € (euro) and £ (pound) symbols in a Java src file. If you do you're asking for trouble. Rather use their unicode prepresentation (and possibly a comment telling readers what that unicode represents), that is:

public static final char EURO = '\u20AC';

public static final char POUND = '\u00A3';

The main problem occurs when your src is meant to be managed in a windows platform (where encoding is by default Cp1252) and in a Linux platform (where encoding is by default UTF-8).
Unless you have to, then, don't use them. If you really have to, the option is to share between the platforms the same encoding.

You may need to consider

```
-Dfile.encoding
```
Use the same encoding in both platforms by passing to the ant the encoding attribute
Change the default workspace encoding in eclipse (windows/general/workspace) and set it to UTF-8 (bear in mind that then your € and £ wont be visible anyway)

Obviously this happens for any other outside US-ASCII char that is not represented in UTF-8

Sacrificing quality?

2008-03-18T23:26:00.000Z

Not compromising on quality is not only your professional obligation but it is also important for your own joy of work and is critical for the company. (Ken Schwaber)

My take on this is that compromising by skimming on test-dev-refactor loop (by eliminating refactor and/or test) is BAD. But working with the customer and negotiate a delivery of a Fiat 126 rather than an F430 at the end of the current release is something to encourage if resources and time are tight, in the spirit of the pure iterative approach. After all, if the business problem is "I need to drive from home to work", that is a perfectly valid solution to it.

This is what Jeff Patton was talking about at the last XPDay in London when he was talking about sacrificing quality if time/resources are scarce.

Babylon IT

2008-02-18T23:34:00.001Z

Interesting page on computer languages. It shows a (maybe) complete list of existing computer languages (alive and dead) and their main features. Quite unsurprisingly there are languages I have never heard of. And also some with amusing names, like Pizza. Also a good source for links to external websites.

Java top 5 technologies to learn during 2008

2008-02-11T23:39:00.000Z

I receive regularly emails from several technology sites. This time one carried an interesting link to a blog entry describing the top 5 technologies to learn in 2008. Some interesting ones, a side the usual web 2.0 ones, are OSGI (at #5) and Cloud computing (at #1). I have been exposed recently to cloud computing and grid computing doing scalability work for SpringRing (now Aloha). But, I must admit, I was not aware of OSGI.
Oh dear, something else on the stack.

Go apes with UltraMonkey

2007-12-28T02:43:00.000Z

I have been recently exposed to UltraMonkey as part of my daily job on SpringRing (now Aloha). UltraMonkey is a solution for load balancing and high availability of network services. We are using it to load balance the SimpleSipStack instances to experiment the stack scalability. It works quite well and it has - good! - native SIP support.

Well, UM is essentially a bunch of Linux tools that you configure to implement the topology you want (Load Balancer, Load Balancer + High Availability, ...).

Downloads are available tarballs or RedHat rpms or Debian packages, so installation in our Suse 9 was quite far from straightforward but in the end configuration is very easy, at least for the load balancer topology.

Pros: it's easy and quite powerful, it runs on Linux, it's cheap

Cons: you need to be more than a newbie on networking to really understand how it works and how to best make use of it. On the good side, it's a good opportunity to learn new stuff

Two links for the readers

2007-11-28T13:59:00.000Z

Today's browsing activity was quite interesting. I ended up reading an article on Spring Framework 2.5 on InfoQ and, digging around I came across this portal for web 2.0 stuff.

XPDay 2007

2007-11-23T22:41:00.004Z

XPDay 2007 - cool agile conference in London (19/20 Nov 07)
Organised also by those running the Extreme Tuesday Club, XPDay has reached his seventh birthday. From what people in the Agile world say, the conference is one of the best around for various reasons. Firstly, it's very small and organizers are very friendly, so there's a relaxed and friendly atmosphere. Secondly chairs are very good and competent and usually the level of the talks is higher than average. Thirdly the organisers have a good taste on choosing venue and pubs (for the meeting after the talks). Lastly, it's cheap :D

Therefore it's a nice place to be in, especially if you're involved on presenting as I was (with three other buddies from the Web21C SDK team).

Our session
Back in May (I think), when call for papers opened I had the idea to write a follow up to the presentation I and Paul Karsten did at XP2007 in Como. I felt that then we didn't tell the whole story. In Como we had a poster (and a paper) telling how the Web21C team was created and how it evolved. We concentrated on the practices followed and, marginally, touched the tools we used.

So I thought to prepare a session to complete the story of the journey I have been doing in the past two years by expanding on the tools.

Gabby, Nathan and Paul joined me to collaborate. It was great, as each one of us was involved in different parts of the project so we were sure to cover the whole spectrum of tools used within the team (Gabby has been involved on the Web21C SDK itself, Nathan on the Web21C SDK and on the build process, Paul on the Portal).

Slowly but thoroughly we started "What's in the toolbox of a successful software craftsman". The original idea was of a 45mins session where we were emulating a mini release showing how we were making use of our tools to support our agile development work. After talking to Angela Martin, one of the conference organizers, and having been allocated only half hour in total I changed the shape of the presentation. After few meetings with the team we decided to produce a standard slide driven presentation splitting it in 4 parts - to allow everybody to talk.

Being a presentation on tools, there was the risk of falling into a tutorial-like speech which would have defeated the point of presenting at an Agile conference.

The idea ultimately was to talk about tools, but in context of three agile principles that we, in Web21C - either explicitly or inadvertently - have been complying to. Those principles are: High project status visibility, Fast feedback and Complete automation of development tasks.

So we wrote the slides to show people how we were using a bunch of free and open source tools to achieve maximum results on each of these three principles. We picked four of the tools in the toolbox which we believed were the more representative (Eclipse IDE, CruiseControl, Selenium and ScrumWorks).

So... it took - i believe - four or five meeting to get to a definitive and presentable slidepack - it also involved Ivan Moore, our shepherd - assigned to us by the XPDay organisation, who had the role of looking at the format of the slides and helping us on balancing the contenent.

And a week before the day we started rehearsing. I must admit, I was quite nervous at those rehearsals - it has always happened to me.

Then at 3.30pm of Monday 19th of November we delivered. And it went very well. Ivan - who was chairing - and the audience gave us extremely positive feedback. Slides were clear and the message came across smoothly. Moreover we got questions at the Q&A who nicely filled in the available half hour.

On Tuesday, at the end of the conference, we were also awarded - with a bottle of champagne - of "Best newcomers session at XPDay 2007". From the words of Angela Martin: "...one of the reasons we selected your team's presentation for special mention this year, was not only it did receive good feedback ... but during the conversation's in the pub and during the breaks it was one of the presentations people were talking about and remembering, so well done."

How nice!

Our poster
The poster was a latest addition to the pack. The last day of the call for posters I thought that it would have been a good idea to present the Toolbox (in form of a mind map) in the poster session. The original idea for the session included an amount of time dedicated to experience exchange with the audience. Given that we were told to refactor our slidepack and cut to fill in 30mins and given that I really wanted to see what other people were doing out there, I decided to submit the poster. Yes, it was more work - as I did it on my own - but it was fun and well worth at the end.

I also thought that, in order to involve people more and to drive them to talk to us, it would have been a good idea to let them update the poster with their own fav tools. And so it went. As soon as Gabby sends me the piccies, I'll upload the mod poster here.

Sessions I went to

Keynote1: Jeff Patton - embrace uncertainty
A really good session. It opened my mind. Jeff is an authority on user stories writing. To cut it short, it explained very well why Agile projects can get to a mess and why this mess happens when Customer and Developers choose user stories badly.
Doing XP (and agile in general) is not about defining user stories, and then fully implement them iteration by iteration. It never happens like that [I heard somewhere else that this is Scrumfall, the tiny little cousin of RUP or any iterative waterfall-ish approach].
It's about rethinking what iterative means.
It made the example of a car. It is made of an engine, breaks, wheels, doors, seats and so on. There's no car if at least one of those parts are missing. So, rather than saying let's do the engine, then the wheels then the breaks and so on, without being able to figure out when the end product is going to be ready - there's also the chance to starve the stories at the bottom of the backlog - the right approach would be, lets to a cheap and working engine, breaks, doors and so on. Then iteratively improve the lot to get to the desired result.
The customer sees the end, which is not anymore uncertain. He has an end product. It may not be a Ferrari as he wished, but at least it's something he can drive.

Keynote2: Yvonne Rogers - Cheek to cheek: why co-located collaboration persists
Rather dull presentation on why colocation is better than remote meeting. Thanks, I hadn't figured it out!
Only thing worth mentioning is that there's some research in progress on how to make co-location even more effective. When a team pairs with one keyboard or brainstorms on a white board, there's one single point of contention - either the keyboard or the marker. This may not be effective as the one with the marker/keyboard is the one driving, possibly precluding the rest of the attendees to express their ideas. There are prototypes of tools (similar to a large flat touch screen that the working team can access and manipulate all together. Educational!

Refactoring databases: evolutionary database design
It didn't rock my world. DB and OO are two completely different beasts. OO is far more advanced in terms of methods and supporting tools for agile development.
The speaker seemed also to make a point of how culturally DBAs are different than developers.
Bottom line: cut cultural barriers and organise your sql scripst properly to be able to easly manage db changes. Mhhh! Oh well!
Lot's of techniques and tools he was using (ant/CC/subversion) weren't new to me. I can though imagine that they were new for most part of the audience. So after all a good session to remind me how badly I have done DB application in my remote past. It would have been more useful if I was a DBA or if i was spending 90% of my time coding oracle stored procs (Ouch!)

Jeff Patton's Master class: mapping user stories.
Interesting stuff. He did link to the keynote several times. Nothing particularly blowing. One thing though I learned: He introduced the concept of user activities as "high level" stories. Example "Manage emails" vs "send email", "receive email"... His point was that more often than not what really delivers money and value is delivering an activity of the system, made of user stories. Rather than the user story itself. And the way he suggests doing it is by iteratively incrementing each of teh stories in the activity by going cheap, then augmenting. Good stuff.
I missed the afternoon part of the master class as I did my preso.

Fighting the monster - agile development om top of a legacy database.
Interesing tutorial on DbFit and on how a bunch of developers made possible to wrap an old Db into a nice simple interface hiding the complexity of the db itself.

A Technical StoryHow a team of devs working for a financial company did agile without telling their customer and how they screwd up once when they decided not to implement user stories, rather do what they wanted, that is to implement a fully fledged configuration API. Good stuff - to a certain point.

But that's crazy! Cognitive Bias in decision making.
This session was supposed to teach me how to understand why someone (typically someone high up in the food chain) does make decisions sometimes taggable as crazy. In a way it did, or at least it explained me the tools i should use to do so... a bunch of cards with short sentences on ala "People usually takes big risks to avoid loss and does not take small risks to gain advantages"... not my soup but interesting enough as a change.

Fiat Lux: testing with the light on
Experience report of a company developing software for libraries using technologies not similar to ours. Specifically Concordion and Team CityConcordion looks like something interesting, although it suffers of the same problems that Fitnesse has: maintaining the fixture code :(

The secret backlog - behind every bug-report is a user story
Anthony Marcano telling us that from a bug theres a test, from a test there must be a story, from a story there's an idex card to stick to the wall. Who needs Jira anymore.

Specification by (executable) example: Prosperity of the FITtest
experience report on using Fitnesse!

Charts in Java

2007-11-05T23:51:00.001Z

Today I found this nice litter library that allows developers to produce charts out of data JFreeChart. I was up and running in 1/2 h - the issue was to generate a simple plot of some performance data we collect during our performance build.

Anyway - this is the graph produced out of some dummy data in my local database.

Scalability resources

2007-11-01T23:54:00.000Z

In my project (SpringRing, now Aloha) the metaphore for this cycle is "make it scalable". So, in order to document myself on the topic I have lurked the Internet and found these interesting websites/products

Especially the last one is a great source of information on the topic of scalability.

I bookmarked these and others in my del.icio.us archive

Please let me know if you have any other pointers.

XP2007

2007-06-26T22:20:00.000+01:00

A week in Lake Como at the XP2007 conference

The experience report

We were there to present our experience report, to attend some of the most interesting tutorial/workshops and to meet and discuss with people. The experience report title is "The creation of a distributed agile team" and talks about the story of building the Web21C SDK team from its inception to the current days.

The proceedings

The paper book is available to buy from the Springer website. Articles are downloadable as PDFs.
Overall an interesting book - also because our report is in there ;) - if you want theoretical studies behind agile and specifically XP methodologies.
Don't expect to find lots of insights or techniques other than those already best known.

The poster

We set up a poster in the hotel hall. We had few people talking to us to find out how about our experience and mostly they were interested on how hard it was to manage the team distributed all over the world and how hard it was to apply agile (XP) practices like stand ups, pair programming and so on.

A group was coming from the University of Cagliari (Sardinia).
They were some PhD students interested on real experience of big teams adopting agile for some of their research studies.
I think most of the interest on our experience was caused by my interventions to the first workshop, where I was part of the panel several times during the goldfish bowl.

After two days of standing by the poster, the hipe was gone but discussion continued at the agile cafes and at the bar during the coffee breaks.

Agile cafes

This was a brilliant idea. The hotel did not provide to us an open space where people could meet and talk. So someone thought to use the hotel patio and the round tables in there to sit and discuss of whatever topic anyone had in mind. with coffee or beer; anyone could join a table and dive into whatever the discussion was at that point.

This was a very good way to share experiences, pain and know each other.

It is surprising how people is separate parts of the globe, in projects of different nature, may share the same issues and problems.

Popular topics included: how to write user stories, how to manage a distributed agile team, is a small team more efficient than a big team no matter what the size of the project is, story of the waterfall methodology.

Meeting people

Most of the value i got out of the conference was by having met people of various type and experience.
Most notably I had the chance to talk and have dinner with Kent and Cinthia Beck and JB Reinsberger. Other than that i met people working for Philips, Ferrari, Exoftware, BT (!).
It was a good way of exchanging experiences on shared problems or get to know issues in different domains (for example test driving embedded code).

The place

Needless to say that Como and its Lake are beautiful. Probably you'll get the most of it for a long city break weekend (3/4 days). Food is faboulous - both fish and meat - and almost anywhere cheap.
I was not impressed by my Hotel, very expensive, not so good food and service not at the level of a four star rank.

Tutorials/Workshops

Can agile practices deliver high-quality large scale offshored projects?
This was the workshop i found most interesting, also because was in theme with our poster. It as arranged as a gold fish bowl and was set as an open discussion driven by the speakers (a couple of consultants from Sweden). There were five chairs in front of the audience and only four were supposed to be occupied. The four people in the panel had to discuss the topic in the agenda. If someone wanted to dibate or discuss an issue, s/he had to go on the last available chair and one of the panel had to leave.
We initiated the discussion with the following starting points:

process and tools to enable team members comms effectively;

impact of different time zones in productivity

what is the minimum amount of human comms

team to team comms

does the team know what the project goal is?

what are the additional tasks necessary in a big team?

how to avoid PM blind spots?

how to test control scope and cost?

are agile practices trustworthy?

I had chances to seat several times passing my experience in a big distributed agile team and had the opportunity to listen to someone else experience. I think we're on good shape, given also that i had been asked few questions (how do you do this and that sort of questions).

One thing i realised, though, we don't do at the moment is maintaining a continuous integration box where all the capabilities are deployed and the sdk tests run automatically.

Agile process anti-patterns
Interesting workshop too... it seems that the set of antipatterns for agile methodologies are not yet well defined. the author - Wayne Allen - is writing a book discussing the list of antipatterns that he's come across in his work as a consultant in the last five years. We were asked to remember the antipatterns we thought we came across in our project, then the audience was diveded in to two parts and each part had to pick an antipattern, discuss and elaborate it. The format for the description of the antipattern is the usual: name, poor solution, consequences, forces, better solution, exceptions.

My team picked the Scrummerfall antipattern, where by teams think to implement an agile project just because they do iterations but they end up with doing a waterfall approach in the sense that iterations dont produce valuable and testable stories, there's no goal defined for the release and so on.

Mapping XP

This was the tutorial from Kent Beck. It was about mind mappings applied to XP practices. The underlying idea was to use mind mappings to elaborate XP stories.
I had heard of mind maps but i never managed to use one. It seems quite a powerful concept expecially because it allows you to use more than one part of your brain (that for example associated with colors and maps). probably more effective than simple bullet point lists to elaborate on concepts and ideas.
So i did find it intellectually interesting but not very applicable to my day to day job - at least for the moment.
On the merit of XP much was stressed on Appreciative Inquiry: concentrate more on what went well rather than feeling depressed looking only to what went wrong

identify when things went well
what were the circumstances, who was involved what support was available and how did you feel?
how can you reuse the past positive experience in any new scenario?

Also Accountability and Honesty were discussed.

Thinking refining and communicating business perspective with executable documents
Another interesting workshop forcussed on executable documentation. The idea is to have documents written by customers to work as acceptance tests. This should lead to ATD development (acceptance tests driven).
Fitnesse is an implementation of a mechanism to write executable documentation. Although it was general consensus that Fitnesse is not very good for customers as it forces a strict syntax - based on tables - and it requires mapping between wiki and code.
The workshop ended with a brief discussion on fitnesse and then the audience split one part carrying on a q&a discussion of executable documentation, the other on advanced fitnesse topics. I picked the former and managed to have an in depth discussion on this matter with the speaker.
One thing i think is worth mentioning is that on ADT you focus more on the testability of the story and on explicitly understanding the value that a customer gets from having that story implemented.

Test driven enterprise code

JB Reisnberger's was an iteresting tutorial even if i - sort of - knew already how to proficiently use jUnit... He stresses the use of mock objects (jMock) all over the place and I tend to disagree with him... See his book jUnit recipes for more details.

Agile open source academy

Rather boring workshop on open source (and free) tools to use to implement agile projects. Most importantly i noticied that we're using most of the tools that are used by everyone else (in our domain).

Keynote 1: The grand prix starts at 2 o'clock: a race to race software development experience

This was more of an experience report from the IT managers of the Ferrari racing teams. Most notably they have very short iterations - one day tipically and very short releases - usually in conjuntion with the GP from march to november... that means every one or two weeks.

It was interesting to hear how they solved the problems of isolating the team from the rest of the world to minimise overhead and disturbance. They rotate a "driver" once a week which is the front door to the team. He responds to the emails, fixes immediate bugs/issues, does 1st/2nd/3rd line of support. They say it's a good way to introduce new members to the complexity of the projects (120ca) they have.

Keynote 2: Ease at work

This is the end key note from Kent Beck. It had the un-dubious ability to let people talk.
It was all about balancing the life at work with yourself by finding your path to feeling at ease during work time. Ease in the sense of feeling good with yourself and within your environment.
Ways suggested to feel at ease were doing a good job (eg being good at what you do), being honest with yourself and your team members, being accountable of what you do, being passionate on what you do.
I found his suggestions and his approach a bit out of context... not everybody can afford or has the courage (for personal or contingent reasons) to be honest, accountable or passionate. But I can see that Mr Beck is trying to bring XP (and himself) to the next level.

Conclusions
I am quete happy to have attended XP2007. It's not the best conference i have been to but i found answers to most of my questions.

The overall message i got from talking to people and attending to the tutorials is that it's not enough to apply the agile tecniques to be succesfull... it's up to us, the people working on a project, to buy in to the values of agility, expecially accountability, honesty and discipline.Then the tecnique once mastered can be effective. Otherwise it's again process over people... I asked around how does the percentage of failed projects adopting agile compares with the percentage of failed projects adopting waterfall (or in general non agile)...

Well you'll be surprised to know that they are pretty much the same. So it's not agile per se but its the values that it promotes when people really is engaged to make them work.

JavaOne 2007

2007-05-12T21:34:00.000+01:00

This is a brief report of last May's JavaOne week in San Francisco.

I arrived on Saturday May the 5th, I was pretty much nakered

Although the flight was smooth and easy, I didn't sleep much albeit the films offered by United Airlines were terribly boring (The Pursuit of Happiness and Music and Lyrics).

Anyway, had dinner at one restaurant in the Embarcadero and went to bed by 11pm.

Sunday was dedicated to an interesting walk around the city... I and my colleague Dan walked from the hotel to the Golden Gate Bridge. I don't know how far we walked but it was a lot (~10Km I reckon). We went through Chinatown, Little Italy down to the Embarcadero (nearby Pier 23) down to the Golden Gate Bridge and back (well, OK, half way through the return way we decided to catch a taxi). During the walk we came across several interesting places, including the Long Now Foundation shop and the Norway Day stand at Ft Mason Center (there was a lump of real snow directly from Norway - or maybe not?).

_{Escaped from Alcatraz}

Monday was dedicated to some shopping, catching up with the team and preparing the Booth at the Moscone Center for JavaOne, and attending the Sun University Class we were subscribed at (more on this later) . We then spent the evening visiting some of the most interesing house-brewed beer pubs in SF, most notably the Thirsty Bear

_{Team daily retrospectives at the pub}

Tuesday was an all day session at the booth. We were very busy showing people the Java SDK and giving away our shwags (and the little red phone box was a success). It was very pleasant to show people the features of the SDK, and the most succesful ones were - as expected - ThirdPartyCall and SMS. Most people registered through the portal (some for the sake of getting the usb flash memory IMHO) and promised to have a look at the product. We also realised that the booth was not as sexy as it should have been: several visitors said that the booth was not suggesting what we were then presenting. The day ended with almost 150 users registered.

_{The diamond: AT&T Park}

We then went to the AT&T Park stadium, where the local baseball team, San Francisco Giants were playing the New York Mets (and they won. Thanks to AndyO for explaining me (an old time footy lover) the rules of the game on the fly.

On Wednesday I attended to few other talks and spent most of the time at the booth. Attendance was more than expected and we were all busy almost all five hours of opening. The day ended with few more home brewed beers and Spiderman 3 at the local IMax.

Thursday again very busy (I think even more than Wednesday) and the last helicopters were awarded to those presenting an application built with the sdk (a nice blog on this).

Friday was definitely a day of shopping in and around Union Square... nice shops of all sorts and types.

Saturday we hired a bike (I, Adrian, Pawel and Dan) and we cycled the whole morning plus part of the afternoon around SF: through the Golden Gate Park to the Ocean and back along the Embarcadero. Very tiring but enjoyable experience.

Finally Sunday I flew back home... Flight was ok and not surprisingly movies offered by UA were utterly boring - even more than the outgoing flight (I can't even remember what I watched). But it took the chance to finish my book, Paul Graham's Hackers and Painters: a very interesting book that everybody in this industry should read.

_{The Golden Gate Bridge}

Sessions I attended

Using JavaEE 5 and SOA to architect and desing robust enterprise applications
Rather boring talk on the JEE technology stack and on how to use it to implement enterprise applications. The speaker, a JEE architect, spent half hour saying why architecture was important for the implementation of the non functional requirements and all the *ilities. Approach was based on a RUP like methodology - session ended with explaination of the core J2EE patterns.

Rapid web development with Groovy and GrailsVery interesting session on Groovy and Grails...
it's worth looking at these technologies as replacement of Ruby and Rails. The session included an hands on tutorial on how to implement a simple web site that exposes CRUD ops on a set of DB tables.

Distributed cachicg, using the JCACHE API and ehcache, including a case study of Wotif.com
Very interesting session on caching. The speaker (also author of ehcache) explained in details the issues aroung caching and how to use ehcache for distributed applications.

Non intrusive monitoring of java technology based applications with JMX, JConsole, and AOP
This is probably the most interesting talk i attended... the subject says it all: JMX is the way of managing applications in two ways. One is to allow easy runtime configuration of the behaviour of the application. The other is to collect stats on the various aspects of the apps (JVM, thread numbers, memory and so on). The nice thing is that this is all possible via spring JMX which integrates very well with the most popular jmx server implementations (including JBoss).

Java API for RESTful Web Services
JSR 311 is the way of standardise web service exposure through a Java API. Basically is a bunch of annotations that allow to specify how the methods and properties of a bean can be exposed the REST way. The JSR is still work in progress and the talk described the current state of play. Quite interesting talk as, when this API will be available, will encourage development to fit with the REST principles. Moreover services built with the API will be deployable by use of a variety of web container technologies and will benefit from built-in support for a variety of HTTP usage patterns and conventions.

Web algorithms
From the session abstract: "This session explains--without any needless math or boring proofs--several fun algorithms of interest to back-end web programmers.
Each algorithm was selected because it's really practical, really interesting, or both. The algorithms aren't always the same but can include public key cryptography, credit card checksum validation, TCP Slow Start, 2's complement, priority queues, the XOR swap, and the Google MapReduce function for massively distributed calculation."

Blueprints for Mashups: Practical Strategies, Tips, and Code for Designing and Building
This presentation highlights the issues around the development of mashups, specifically security concerns, protocols (soap vs REST), data format (xml vs json) design patterns and strategies (client vs service mashups). It gave me an overview of what JSon is and how to use it, and some in depth details on how to code mashups in \JavaScript.

Minimalist test techniques for enterprise java technology based applications.
The first part of the talk was about how testing is important and why people shold do it (kind of preaching to the convert), so after all quite boring. Second part described test strategies for each of the three layers of a typical JEE app (data, buisiness and presentation). Only interesting bit was a demo of Selenium. The rest was chitchatting and free publicity to the speaker's book (POJOs in Action).

Declarative Programming: Tighten Enterprise JavaBeans (EJB) 3.0 and JSR 303 Beans Validation
Probably the most boring presentation title so far. Essentially, the session was devoted to describe the ins and outs of the JSR 303, which, surprise surprise, is a way of standardising bean vaidations via Java5 annotations (I know something about this). The guy, expert member of the JSR 303 committee, showed how cool is to annotate beans and methods to validate input parameters. The JEE way will not use AOP though (the guy said AOP is useless and too complicated), buth rather with "interceptors" which basically are very similar (in fact, the same) as Spring AOP advices.
This was the only session I dared asking a question at. I was infact rather disappointed when he said that people should not use AOP at all and he did (not too much implicitly) say that such developers are masochists. The question i asked was: what's the difference and why Interceptors are better than Spring AOP. The answer was that Interceptors are standard and that I had been better off complaining to that standard, and that interceptors were easier to use. I only objected that he was confused with pure AspectJ (and not Spring AOP). Anyway it did not convince me. But there you go... we'll have a standard.

Was it worth?
Of course it was. The city and the experience at the booth was priceless. Some of the sessions interesting enough... some others rather dull. Also, you would expect JavaOne to be the showcase for Sun's product, and it was... I just thought it was too much!