Getting immediate and accurate feedback from the IDE

In Agile methodologies getting feedback to adjust future work is a valuable practice to follow. In reality this is nothing new, in any human activity information gathered from feedback received on the system in development is used to drive activities to improve the chances to get the desired output.

Agile methodologies typically address feedback from the customer perspective: the team produces the agreed artifacts, the customer provides feedback on the outcome, both together re-plan future activities to adjust appropriately to get maximum business value from new delivered artifacts.

Feedback is characterized by two properties: its “immediacy” and its “accuracy”. Immediacy is the perceived time elapsed between when feedback gathering starts and when it's actually obtained. Accuracy is related to the usefulness of the information extracted from feedback.

Bearing in mind these two properties, feedback can be extended to development activities by appropriately configuring development tools and refine development practices. The aim is to get feedback “now” (immediate) and “spot on” (accurate).

The source of ultimate feedback for a developer is the build process. Typically, though, “the build” is a resource intensive process that happens on a continuous build/integration server (for example Cruise Control). This process consists mainly of a sequence of steps that include checkout from source control repository, compilation of the code, execution of the unit tests, metrics gathering, deployment in some test environment, execution of the customer acceptance tests, documentation generation.
These steps are executed using the entire source tree and provide the ultimate feedback on whether the latest code changes have broken or not any of the existing functionalities. Clearly the larger the code base is the longer the build takes. In a short “write test – write code – check-in” loop (typical of a TDD approach) though it's usually unacceptable to wait even more than few seconds to understand if the code added breaks the build or not. So even for short builds (in the range of few minutes) it's advisable to use the IDE to help improving immediacy (by shortening feedback loop back time), albeit without diminishing accuracy.
One way to do that is to use the IDE to execute part of the whole build process locally, possibly in the IDE itself.
Taking Eclipse IDE as an example, the following is a list of tips that may be used to gather immediate and accurate feedback.

• Use the JUnit plug-in to run the test just written until the code that passes it is complete (well, no new news here, I am sure most of us do this already). The caveat in this tip is to make sure that the result is accurate; way the plug-in runs the tests is equivalent to the way the CI executes them, that is via – typically – an Ant script. Whilst this is immediate for simple tests and code structures, it gets very hairy when frameworks are used (for example SpringFramework) which require configuration files in the classpath and/or external library dependencies. So the first thing to check is that the classpath in the Ant on which JUnit runs is the same as the one in which the test runs in Eclipse. One caveat to this approach is that Eclipse IDE is limited to a single classpath per project so it's less flexible than Ant, where order and content of a classpath can be more fine grained. So typically any “target” in an Ant build script should refer to a “project classpath” path definition that refers to the same set of directories and files in the Eclipse project classpath and in the same order.


• Use code metrics plug-ins to run metrics on the fly on single files and incrementally on packages and on the entire source tree. The rationale is that in a short TDD loop the number of files touched is minimal, so running the metrics in that subset first and eventually run them on larger sets, improves the chances to find problems sooner. Eclipse IDE can be enriched with all sorts of plug-ins, including those for Checkstyle, FindBugs, JDepend. And to gather accurate feedback it's necessary that the configuration files for each one of these tools/plug-ins is shared between Eclipse IDE and the main build – it's usually good practice to check the configuration files in the same code tree so that they can be easily referred from the IDE and Ant.


• Use the Ant plug-in in the IDE to execute single or set of targets in the IDE. Accuracy is guaranteed by having the Ant environments in the CI and in the IDE equivalent – this includes also redefining the Ant Home in the IDE to run the same version of Ant in all the environments. Running Ant in the IDE rather than in a shell offers the limited advantage of the IDE support on parsing the output from the build (context sensitive help and hyper-links to the source code).

There are plenty of other clever usages of the IDE features that can drastically improve feedback gathering; some of them are specifically for Eclipse, others exportable to any other IDE as the ones described above. In all examples, though the key message is that immediacy must not be achieved at expenses of accuracy (as my friend Paul would say "feedback is only valuable when it's accurate").