Experiences from the fields

Last week I blogged about the new release of DNN Events: version 05.02.03, which replaces the previous one (05.02.02) and is mainly a bug fix release.

Currently, we (Roger Selwyn and myself) have worked on 2 versions of Events: a bug fix release (05.02.03) and a new version (Events 6) for the DNN6 environment. And there is a story to go with it.

The start

The 05.02.03 was not planned, but came around when some small, but annoying, bugs popped up around the 05.02.02 release. About 3 weeks ago, Roger and I decided to do an intermediate, unplanned, bug fix release: 05.02.03. The problem was that we use CodePlex/TFS for our code repository and we have moved everything already to our alpha and published Events 6 release.

So we started again with the source pack of 05.02.02 and worked disconnected from CodePlex TFS for our sources (Note: we used DropBox to share our coding for this intermediate version). Roger did work on the bug fixes and also back ported some of the changes and enhancements from our Events 6 release back to 05.02.03.  I, in the mean time, worked on the SQL data scripts and made some enhancements for our automatic build and packaging tool to get it compliant for DNN5 installs. We have made a beta version of our bug fix release available on CodePlex, saw the download counter reach 40-50 download without any problematic feedback. So far so good.

We prepared our final Events 05.02.03 release and replaced the beta version on CodePlex. With confidence I changed the 05.02.03 version into a stable, preferred release of DNN Events.

The results: not a happy event

It did not take long before problems were reported: wrong DLL versioning. While Roger (who made the finale 05.02.03) was out, I created a new "stable" 05.02.03 and replaced the current  one on CodePlex. Again, not correct. Due to the Events 6 (DNN6 related) version in the works, some of the data script changes has slipped back into my "stable" version. Sh*t. Luckily, Roger was able to create a finale stable version of DNN Events 05.02.03 and also prepared a SQL repair script for the a few people around who suffer from our intermediate "work" (and do not do back-up's or testing before going live).

How did we do it before?

Release tracker - In the previous years, there was the DNN Release tracker process (which we did not like). Any core module release could take weeks if not months before it was formally released. However, the release tracker process would have saved us this time from the errors that we have made.

Special Events website - At the start of my involvement with the Events module (2007), we had a special website for distributing Events beta releases to users with real interest in the module. Users, who also gave us real feedback on any problems in the beta version. The current CodePlex beta downloaders do not give us feedback on beta version of the module.

Automated UI testing - At some point in time, we created automated test scripts with DejaClick, a FireFox AddIn with screen recording, replay and script editing facilities. This helped us to execute tenths of user scenario's automatically (creating events, editing settings, changing display formats, enrollments, approval, etc.). However, due to the dynamic nature of DotNetNuke (controls do not have fixed name but are dynamically generated) and the frequent releases of the core platform quickly made the management and editing of the tests scripts a daunting task. Back to manual testing which developers -and that includes us- do not really like to do over and over again.

My conclusion -  In Dutch (my native language) I would say: "Wet van behoud van ellende", which translates to something like "There is no such thing as a free lunch". We have not reached a level of getting enough feedback on the quality of our new version; if others test, it takes much too long before we can release; we are not able to execute easy maintainable automatic UI tests and we have unrealistic confidence in our own development quality (and/or do not like to repeatedly test the same things manually).

What do we want?

Roger and I discussed our experiences. We both are very committed to release high quality version of the Events module, and we have proven to be able to do so over the past years. The 120.000 downloads in total with an average of 50 downloads per day supports this.

We believe in the principle of automatic testing, so I will continue to look for testing tools that are really easy to work with, even in a dynamic environment like DotNetNuke. I am currently investigating Telerik Test Studio to see if this can work for us, and I am open for any experiences or suggestion from YOU on this point.

We also believe in the feedback of users (beta testers), but also realize that we have to trigger a response more actively. We are looking for a way of gathering names and email addresses of people who are really interested to test & provide feedback on new releases. The DNN Events forum? A Facebook page? Any suggestion?

What have we learned?

Maintaining high quality and bug free software is difficult and incorporates more work then you expect. And this reminds me on one of my statements in my PhD dissertation almost 30 years back: Making software and raising children have remarkable similarities: it take a lot of time and effort, only to find out in the end that there are bugs in the system that you have put in yourself.

Life has not changed much over the years.

Have fun with the new Events module. Looking forward hearing from you!