Go apes with UltraMonkey

I have been recently exposed to UltraMonkey as part of my daily job on SpringRing (now Aloha). UltraMonkey is a solution for load balancing and high availability of network services. We are using it to load balance the SimpleSipStack instances to experiment the stack scalability. It works quite well and it has - good! - native SIP support.

Well, UM is essentially a bunch of Linux tools that you configure to implement the topology you want (Load Balancer, Load Balancer + High Availability, ...).

Downloads are available tarballs or RedHat rpms or Debian packages, so installation in our Suse 9 was quite far from straightforward but in the end configuration is very easy, at least for the load balancer topology.

Pros: it's easy and quite powerful, it runs on Linux, it's cheap

Cons: you need to be more than a newbie on networking to really understand how it works and how to best make use of it. On the good side, it's a good opportunity to learn new stuff

Two links for the readers

Today's browsing activity was quite interesting. I ended up reading an article on Spring Framework 2.5 on InfoQ and, digging around I came across this portal for web 2.0 stuff.

XPDay 2007

XPDay 2007 - cool agile conference in London (19/20 Nov 07)
Organised also by those running the Extreme Tuesday Club, XPDay has reached his seventh birthday. From what people in the Agile world say, the conference is one of the best around for various reasons. Firstly, it's very small and organizers are very friendly, so there's a relaxed and friendly atmosphere. Secondly chairs are very good and competent and usually the level of the talks is higher than average. Thirdly the organisers have a good taste on choosing venue and pubs (for the meeting after the talks). Lastly, it's cheap :D

Therefore it's a nice place to be in, especially if you're involved on presenting as I was (with three other buddies from the Web21C SDK team).

Our session
Back in May (I think), when call for papers opened I had the idea to write a follow up to the presentation  I and Paul Karsten did at XP2007 in Como. I felt that then we didn't tell the whole story. In Como we had a poster (and a paper) telling how the Web21C team was created and how it evolved. We concentrated on the practices followed and, marginally, touched the tools we used.

So I thought to prepare a session to complete the story of the journey I have been doing in the past two years by expanding on the tools.

Gabby, Nathan and Paul joined me to collaborate. It was great, as each one of us was involved in different parts of the project so we were sure to cover the whole spectrum of tools used within the team (Gabby has been involved on the Web21C SDK itself, Nathan on the Web21C SDK and on the build process, Paul on the Portal).

Slowly but thoroughly we started "What's in the toolbox of a successful software craftsman". The original idea was of a 45mins session where we were emulating a mini release showing how we were making use of our tools to support our agile development work. After talking to Angela Martin, one of the conference organizers, and having been allocated only half hour in total I changed the shape of the presentation. After few meetings with the team we decided to produce a standard slide driven presentation splitting it in 4 parts - to allow everybody to talk.

Being a presentation on tools, there was the risk of falling into a tutorial-like speech which would have defeated the point of presenting at an Agile conference.

The idea ultimately was to talk about tools, but in context of three agile principles that we, in Web21C - either explicitly or inadvertently - have been complying to. Those principles are: High project status visibility, Fast feedback and Complete automation of development tasks.

So we wrote the slides to show people how we were using a bunch of free and open source tools to achieve maximum results on each of these three principles. We picked four of the tools in the toolbox which we believed were the more representative (Eclipse IDE, CruiseControl, Selenium and ScrumWorks).

So... it took - i believe - four or five meeting to get to a definitive and presentable slidepack - it also involved Ivan Moore, our shepherd - assigned to us by the XPDay organisation, who had the role of looking at the format of the slides and helping us on balancing the contenent.

And a week before the day we started rehearsing. I must admit, I was quite nervous at those rehearsals - it has always happened to me.

Then at 3.30pm of Monday 19th of November we delivered. And it went very well. Ivan - who was chairing - and the audience gave us extremely positive feedback. Slides were clear and the message came across smoothly. Moreover we got questions at the Q&A who nicely filled in the available half hour.

On Tuesday, at the end of the conference, we were also awarded - with a bottle of champagne - of "Best newcomers session at XPDay 2007". From the words of Angela Martin: "...one of the reasons we selected your team's presentation for special mention this year, was not only it did receive good feedback ... but during the conversation's in the pub and during the breaks it was one of the presentations people were talking about and remembering, so well done."

How nice!

Our poster
The poster was a latest addition to the pack. The last day of the call for posters I thought that it would have been a good idea to present the Toolbox (in form of a mind map) in the poster session. The original idea for the session included an amount of time dedicated to experience exchange with the audience. Given that we were told to refactor our slidepack and cut to fill in 30mins and given that I really wanted to see what other people were doing out there, I decided to submit the poster. Yes, it was more work - as I did it on my own - but it was fun and well worth at the end.

I also thought that, in order to involve people more and to drive them to talk to us, it would have been a good idea to let them update the poster with their own fav tools. And so it went. As soon as Gabby sends me the piccies, I'll upload the mod poster here.

Sessions I went to

Keynote1: Jeff Patton - embrace uncertainty
A really good session. It opened my mind. Jeff is an authority on user stories writing. To cut it short, it explained very well why Agile projects can get to a mess and why this mess happens when Customer and Developers choose user stories badly.
Doing XP (and agile in general) is not about defining user stories, and then fully implement them iteration by iteration. It never happens like that [I heard somewhere else that this is Scrumfall, the tiny little cousin of RUP or any iterative waterfall-ish approach].
It's about rethinking what iterative means.
It made the example of a car. It is made of an engine, breaks, wheels, doors, seats and so on. There's no car if at least one of those parts are missing. So, rather than saying let's do the engine, then the wheels then the breaks and so on, without being able to figure out when the end product is going to be ready - there's also the chance to starve the stories at the bottom of the backlog - the right approach would be, lets to a cheap and working engine, breaks, doors and so on. Then iteratively improve the lot to get to the desired result.
The customer sees the end, which is not anymore uncertain. He has an end product. It may not be a Ferrari as he wished, but at least it's something he can drive.

Keynote2: Yvonne Rogers - Cheek to cheek: why co-located collaboration persists
Rather dull presentation on why colocation is better than remote meeting. Thanks, I hadn't figured it out!
Only thing worth mentioning is that there's some research in progress on how to make co-location even more effective. When a team pairs with one keyboard or brainstorms on a white board, there's one single point of contention - either the keyboard or the marker. This may not be effective as the one with the marker/keyboard is the one driving, possibly precluding the rest of the attendees to express their ideas. There are prototypes of tools (similar to a large flat touch screen that the working team can access and manipulate all together. Educational!

Refactoring databases: evolutionary database design
It didn't rock my world. DB and OO are two completely different beasts. OO is far more advanced in terms of methods and supporting tools for agile development.
The speaker seemed also to make a point of how culturally DBAs are different than developers.
Bottom line: cut cultural barriers and organise your sql scripst properly to be able to easly manage db changes. Mhhh! Oh well!
Lot's of techniques and tools he was using (ant/CC/subversion) weren't new to me. I can though imagine that they were new for most part of the audience. So after all a good session to remind me how badly I have done DB application in my remote past. It would have been more useful if I was a DBA or if i was spending 90% of my time coding oracle stored procs (Ouch!)

Jeff Patton's Master class: mapping user stories.
Interesting stuff. He did link to the keynote several times. Nothing particularly blowing. One thing though I learned: He introduced the concept of user activities as "high level" stories. Example "Manage emails" vs "send email", "receive email"... His point was that more often than not what really delivers money and value is delivering an activity of the system, made of user stories. Rather than the user story itself. And the way he suggests doing it is by iteratively incrementing each of teh stories in the activity by going cheap, then augmenting. Good stuff.
I missed the afternoon part of the master class as I did my preso.

Fighting the monster - agile development om top of a legacy database.
Interesing tutorial on DbFit and on how a bunch of developers made possible to wrap an old Db into a nice simple interface hiding the complexity of the db itself.

A Technical StoryHow a team of devs working for a financial company did agile without telling their customer and how they screwd up once when they decided not to implement user stories, rather do what they wanted, that is to implement a fully fledged configuration API. Good stuff - to a certain point.

But that's crazy! Cognitive Bias in decision making.
This session was supposed to teach me how to understand why someone (typically someone high up in the food chain) does make decisions sometimes taggable as crazy. In a way it did, or at least it explained me the tools i should use to do so... a bunch of cards with short sentences on ala "People usually takes big risks to avoid loss and does not take small risks to gain advantages"... not my soup but interesting enough as a change.

Fiat Lux: testing with the light on
Experience report of a company developing software for libraries using technologies not similar to ours. Specifically Concordion  and Team CityConcordion looks like something interesting, although it suffers of the same problems that Fitnesse has: maintaining the fixture code :(

The secret backlog - behind every bug-report is a user story
Anthony Marcano telling us that from a bug theres a test, from a test there must be a story, from a story there's an idex card to stick to the wall. Who needs Jira anymore.

Specification by (executable) example: Prosperity of the FITtest
experience report on using Fitnesse!  

Charts in Java

Today I found this nice litter library that allows developers to produce charts out of data JFreeChart. I was up and running in 1/2 h - the issue was to generate a simple plot of some performance data we collect during our performance build.

Anyway - this is the graph produced out of some dummy data in my local database.

Scalability resources

In my project (SpringRing, now Aloha) the metaphore for this cycle is "make it scalable". So, in order to document myself on the topic I have lurked the Internet and found these interesting websites/products

• Terracotta
  
• Open Spaces
  
• High scalability
  

Especially the last one is a great source of information on the topic of scalability.

I bookmarked these and others in my del.icio.us archive

Please let me know if you have any other pointers.

XP2007

A week in Lake Como at the XP2007 conference

The experience report

We were there to present our experience report, to attend some of the most interesting tutorial/workshops and to meet and discuss with people. The experience report title is "The creation of a distributed agile team" and talks about the story of building the Web21C SDK team from its inception to the current days.

The proceedings

The paper book is available to buy from the Springer website. Articles are downloadable as PDFs.
Overall an interesting book - also because our report is in there ;) - if you want theoretical studies behind agile and specifically XP methodologies.
Don't expect to find lots of insights or techniques other than those already best known.

The poster

We set up a poster in the hotel hall. We had few people talking to us to find out how about our experience and mostly they were interested on how hard it was to manage the team distributed all over the world and how hard it was to apply agile (XP) practices like stand ups, pair programming and so on.

A group was coming from the University of Cagliari (Sardinia).
They were some PhD students interested on real experience of big teams adopting agile for some of their research studies.
I think most of the interest on our experience was caused by my interventions to the first workshop, where I was part of the panel several times during the goldfish bowl.

After two days of standing by the poster, the hipe was gone but discussion continued at the agile cafes and at the bar during the coffee breaks.

Agile cafes

This was a brilliant idea. The hotel did not provide to us an open space where people could meet and talk. So someone thought to use the hotel patio and the round tables in there to sit and discuss of whatever topic anyone had in mind. with coffee or beer; anyone could join a table and dive into whatever the discussion was at that point.

This was a very good way to share experiences, pain and know each other.

It is surprising how people is separate parts of the globe, in projects of different nature, may share the same issues and problems.

Popular topics included: how to write user stories, how to manage a distributed agile team, is a small team more efficient than a big team no matter what the size of the project is, story of the waterfall methodology.

Meeting people

Most of the value i got out of the conference was by having met people of various type and experience.
Most notably I had the chance to talk and have dinner with Kent and Cinthia Beck and JB Reinsberger. Other than that i met people working for Philips, Ferrari, Exoftware, BT (!).
It was a good way of exchanging experiences on shared problems or get to know issues in different domains (for example test driving embedded code).

The place

Needless to say that Como and its Lake are beautiful. Probably you'll get the most of it for a long city break weekend (3/4 days). Food is faboulous - both fish and meat - and almost anywhere cheap.
I was not impressed by my Hotel, very expensive, not so good food and service not at the level of a four star rank.

Tutorials/Workshops

Can agile practices deliver high-quality large scale offshored projects?
This was the workshop i found most interesting, also because was in theme with our poster. It as arranged as a gold fish bowl and was set as an open discussion driven by the speakers (a couple of consultants from Sweden). There were five chairs in front of the audience and only four were supposed to be occupied. The four people in the panel had to discuss the topic in the agenda. If someone wanted to dibate or discuss an issue, s/he had to go on the last available chair and one of the panel had to leave.
We initiated the discussion with the following starting points:

• process and tools to enable team members comms effectively;


• impact of different time zones in productivity


• what is the minimum amount of human comms


• team to team comms


• does the team know what the project goal is?


• what are the additional tasks necessary in a big team?


• how to avoid PM blind spots?


• how to test control scope and cost?


• are agile practices trustworthy?

I had chances to seat several times passing my experience in a big distributed agile team and had the opportunity to listen to someone else experience. I think we're on good shape, given also that i had been asked few questions (how do you do this and that sort of questions).

One thing i realised, though, we don't do at the moment is maintaining a continuous integration box where all the capabilities are deployed and the sdk tests run automatically.

Agile process anti-patterns
Interesting workshop too... it seems that the set of antipatterns for agile methodologies are not yet well defined. the author - Wayne Allen - is writing a book discussing the list of antipatterns that he's come across in his work as a consultant in the last five years. We were asked to remember the antipatterns we thought we came across in our project, then the audience was diveded in to two parts and each part had to pick an antipattern, discuss and elaborate it. The format for the description of the antipattern is the usual: name, poor solution, consequences, forces, better solution, exceptions.

My team picked the Scrummerfall antipattern, where by teams think to implement an agile project just because they do iterations but they end up with doing a waterfall approach in the sense that iterations dont produce valuable and testable stories, there's no goal defined for the release and so on.

Mapping XP

This was the tutorial from Kent Beck. It was about mind mappings applied to XP practices. The underlying idea was to use mind mappings to elaborate XP stories.
I had heard of mind maps but i never managed to use one. It seems quite a powerful concept expecially because it allows you to use more than one part of your brain (that for example associated with colors and maps). probably more effective than simple bullet point lists to elaborate on concepts and ideas.
So i did find it intellectually interesting but not very applicable to my day to day job - at least for the moment.
On the merit of XP much was stressed on Appreciative Inquiry: concentrate more on what went well rather than feeling depressed looking only to what went wrong

• identify when things went well
• what were the circumstances, who was involved what support was available and how did you feel?
• how can you reuse the past positive experience in any new scenario?
  

Also Accountability and Honesty were discussed.

Thinking refining and communicating business perspective with executable documents
Another interesting workshop forcussed on executable documentation. The idea is to have documents written by customers to work as acceptance tests. This should lead to ATD development (acceptance tests driven).
Fitnesse is an implementation of a mechanism to write executable documentation. Although it was general consensus that Fitnesse is not very good for customers as it forces a strict syntax - based on tables - and it requires mapping between wiki and code.
The workshop ended with a brief discussion on fitnesse and then the audience split one part carrying on a q&a discussion of executable documentation, the other on advanced fitnesse topics. I picked the former and managed to have an in depth discussion on this matter with the speaker.
One thing i think is worth mentioning is that on ADT you focus more on the testability of the story and on explicitly understanding the value that a customer gets from having that story implemented.

Test driven enterprise code

JB Reisnberger's was an iteresting tutorial even if i - sort of - knew already how to proficiently use jUnit... He stresses the use of mock objects (jMock) all over the place and I tend to disagree with him... See his book jUnit recipes for more details.

Agile open source academy

Rather boring workshop on open source (and free) tools to use to implement agile projects. Most importantly i noticied that we're using most of the tools that are used by everyone else (in our domain).

Keynote 1: The grand prix starts at 2 o'clock: a race to race software development experience

This was more of an experience report from the IT managers of the Ferrari racing teams. Most notably they have very short iterations - one day tipically and very short releases - usually in conjuntion with the GP from march to november... that means every one or two weeks.

It was interesting to hear how they solved the problems of isolating the team from the rest of the world to minimise overhead and disturbance. They rotate a "driver" once a week which is the front door to the team. He responds to the emails, fixes immediate bugs/issues, does 1st/2nd/3rd line of support. They say it's a good way to introduce new members to the complexity of the projects (120ca) they have.

Keynote 2: Ease at work

This is the end key note from Kent Beck. It had the un-dubious ability to let people talk.
It was all about balancing the life at work with yourself by finding your path to feeling at ease during work time. Ease in the sense of feeling good with yourself and within your environment.
Ways suggested to feel at ease were doing a good job (eg being good at what you do), being honest with yourself and your team members, being accountable of what you do, being passionate on what you do.
I found his suggestions and his approach a bit out of context... not everybody can afford or has the courage (for personal or contingent reasons) to be honest, accountable or passionate. But I can see that Mr Beck is trying to bring XP (and himself) to the next level.

Conclusions
I am quete happy to have attended XP2007. It's not the best conference i have been to but i found answers to most of my questions.

The overall message i got from talking to people and attending to the tutorials is that it's not enough to apply the agile tecniques to be succesfull... it's up to us, the people working on a project, to buy in to the values of agility, expecially accountability, honesty and discipline.Then the tecnique once mastered can be effective. Otherwise it's again process over people... I asked around how does the percentage of failed projects adopting agile compares with the percentage of failed projects adopting waterfall (or in general non agile)...

Well you'll be surprised to know that they are pretty much the same. So it's not agile per se but its the values that it promotes when people really is engaged to make them work.

JavaOne 2007

This is a brief report of last May's JavaOne week in San Francisco.

I arrived on Saturday May the 5th, I was pretty much nakered

Although the flight was smooth and easy, I didn't sleep much albeit the films offered by United Airlines were terribly boring (The Pursuit of Happiness and Music and Lyrics).

Anyway, had dinner at one restaurant in the Embarcadero and went to bed by 11pm.

Sunday was dedicated to an interesting walk around the city... I and my colleague Dan walked from the hotel to the Golden Gate Bridge. I don't know how far we walked but it was a lot (~10Km I reckon). We went through Chinatown, Little Italy down to the Embarcadero (nearby Pier 23) down to the Golden Gate Bridge and back (well, OK, half way through the return way we decided to catch a taxi). During the walk we came across several interesting places, including the Long Now Foundation shop and the Norway Day stand at Ft Mason Center (there was a lump of real snow directly from Norway - or maybe not?).

_{Escaped from Alcatraz}

Monday was dedicated to some shopping, catching up with the team and preparing the Booth at the Moscone Center for JavaOne, and attending the Sun University Class we were subscribed at (more on this later) . We then spent the evening visiting some of the most interesing house-brewed beer pubs in SF, most notably the Thirsty Bear

_{Team daily retrospectives at the pub}

Tuesday was an all day session at the booth. We were very busy showing people the Java SDK and giving away our shwags (and the little red phone box was a success). It was very pleasant to show people the features of the SDK, and the most succesful ones were - as expected - ThirdPartyCall and SMS. Most people registered through the portal (some for the sake of getting the usb flash memory IMHO) and promised to have a look at the product. We also realised that the booth was not as sexy as it should have been: several visitors said that the booth was not suggesting what we were then presenting. The day ended with almost 150 users registered.

_{The diamond: AT&T Park}

We then went to the AT&T Park stadium, where the local baseball team, San Francisco Giants were playing the New York Mets (and they won. Thanks to AndyO for explaining me (an old time footy lover) the rules of the game on the fly.

On Wednesday I attended to few other talks and spent most of the time at the booth. Attendance was more than expected and we were all busy almost all five hours of opening. The day ended with few more home brewed beers and Spiderman 3 at the local IMax.

Thursday again very busy (I think even more than Wednesday) and the last helicopters were awarded to those presenting an application built with the sdk (a nice blog on this).

Friday was definitely a day of shopping in and around Union Square... nice shops of all sorts and types.

Saturday we hired a bike (I, Adrian, Pawel and Dan) and we cycled the whole morning plus part of the afternoon around SF: through the Golden Gate Park to the Ocean and back along the Embarcadero. Very tiring but enjoyable experience.

Finally Sunday I flew back home... Flight was ok and not surprisingly movies offered by UA were utterly boring - even more than the outgoing flight (I can't even remember what I watched). But it took the chance to finish my book, Paul Graham's Hackers and Painters: a very interesting book that everybody in this industry should read.

_{The Golden Gate Bridge}

Sessions I attended

Using JavaEE 5 and SOA to architect and desing robust enterprise applications
Rather boring talk on the JEE technology stack and on how to use it to implement enterprise applications. The speaker, a JEE architect, spent half hour saying why architecture was important for the implementation of the non functional requirements and all the *ilities. Approach was based on a RUP like methodology - session ended with explaination of the core J2EE patterns.

Rapid web development with Groovy and GrailsVery interesting session on Groovy and Grails...
it's worth looking at these technologies as replacement of Ruby and Rails. The session included an hands on tutorial on how to implement a simple web site that exposes CRUD ops on a set of DB tables.

Distributed cachicg, using the JCACHE API and ehcache, including a case study of Wotif.com
Very interesting session on caching. The speaker (also author of ehcache) explained in details the issues aroung caching and how to use ehcache for distributed applications.

Non intrusive monitoring of java technology based applications with JMX, JConsole, and AOP
This is probably the most interesting talk i attended... the subject says it all: JMX is the way of managing applications in two ways. One is to allow easy runtime configuration of the behaviour of the application. The other is to collect stats on the various aspects of the apps (JVM, thread numbers, memory and so on). The nice thing is that this is all possible via spring JMX which integrates very well with the most popular jmx server implementations (including JBoss).

Java API for RESTful Web Services
JSR 311 is the way of standardise web service exposure through a Java API. Basically is a bunch of annotations that allow to specify how the methods and properties of a bean can be exposed the REST way. The JSR is still work in progress and the talk described the current state of play. Quite interesting talk as, when this API will be available, will encourage development to fit with the REST principles. Moreover services built with the API will be deployable by use of a variety of web container technologies and will benefit from built-in support for a variety of HTTP usage patterns and conventions.

Web algorithms
From the session abstract: "This session explains--without any needless math or boring proofs--several fun algorithms of interest to back-end web programmers.
Each algorithm was selected because it's really practical, really interesting, or both. The algorithms aren't always the same but can include public key cryptography, credit card checksum validation, TCP Slow Start, 2's complement, priority queues, the XOR swap, and the Google MapReduce function for massively distributed calculation."

Blueprints for Mashups: Practical Strategies, Tips, and Code for Designing and Building
This presentation highlights the issues around the development of mashups, specifically security concerns, protocols (soap vs REST), data format (xml vs json) design patterns and strategies (client vs service mashups). It gave me an overview of what JSon is and how to use it, and some in depth details on how to code mashups in \JavaScript.

Minimalist test techniques for enterprise java technology based applications.
The first part of the talk was about how testing is important and why people shold do it (kind of preaching to the convert), so after all quite boring. Second part described test strategies for each of the three layers of a typical JEE app (data, buisiness and presentation). Only interesting bit was a demo of Selenium. The rest was chitchatting and free publicity to the speaker's book (POJOs in Action).

Declarative Programming: Tighten Enterprise JavaBeans (EJB) 3.0 and JSR 303 Beans Validation
Probably the most boring presentation title so far. Essentially, the session was devoted to describe the ins and outs of the JSR 303, which, surprise surprise, is a way of standardising bean vaidations via Java5 annotations (I know something about this). The guy, expert member of the JSR 303 committee, showed how cool is to annotate beans and methods to validate input parameters. The JEE way will not use AOP though (the guy said AOP is useless and too complicated), buth rather with "interceptors" which basically are very similar (in fact, the same) as Spring AOP advices.
This was the only session I dared asking a question at. I was infact rather disappointed when he said that people should not use AOP at all and he did (not too much implicitly) say that such developers are masochists. The question i asked was: what's the difference and why Interceptors are better than Spring AOP. The answer was that Interceptors are standard and that I had been better off complaining to that standard, and that interceptors were easier to use. I only objected that he was confused with pure AspectJ (and not Spring AOP). Anyway it did not convince me. But there you go... we'll have a standard.

Was it worth?
Of course it was. The city and the experience at the booth was priceless. Some of the sessions interesting enough... some others rather dull. Also, you would expect JavaOne to be the showcase for Sun's product, and it was... I just thought it was too much!