Gleanings From the Wet Coast - Agile and TDD

Mon, 21 Apr 2008 08:00:00 GMT

This Blog is the first in a series of Blogs that I intend to write about new concepts and ideas that I discover in my travels through .NET. These will not neccessarily be my ideas - in fact most of them won't be, but I hope you will find them interesting and useful for your own development projects.

A few weeks ago when the new codeplex-based DNN Forge was introduced, I created a project that I called "The Family Tree project". I have been working off-and-on for about 3 years on both a GEDCOM parser and a rich Object Model for managing Family Tree information. The problem I had with this project is that early on I had taken a Waterfall approach to the design, and as this was a hobby project - and therefore I had limited time to work on it - the progress was painfully slow.

On Friday, after returning from the MVP summit and seeing all the cool new stuff coming along as well as a new emphasis on Test Driven Development, I decided to take a new approach to this project.

I decided to start anew - using an Agile development process. Why an Agile process? My previous efforts resulted in poor progress as the tasks seemed so large to complete, and so I always found excuses to do something more interesting. Using an Agile process I can work on a single feature (eg List all Individuals) and get that feature working at all levels, before moving on to the next feature.

Andrew had recently downloaded Rob Conery's videos where he uses a TDD approach to rewiting the Commerce Starter Kit using MVC. So we decided to try using TDD (Test Driven Development) ourselves on this project. As I expect the project to take some time to complete, we also deccided to build it in ASP.NET 3.5 using the latest bits.

On Saturday, Andrew and I spent most of the day "Pair-Programming" the first "feature" - List Individuals. In about 6 hrs we were able to complete all the tests to test all layers of the application, from the View down to the database for this single feature.

So even before we actually browsed to the page we were pretty confident that everything would work - and it did. Next we added the tests for "View" a single Individual. As we had built much of the framework and hooks between the layers, this took less than an hour to complete, and again once we knew all the tests worked we were quite confident that browsing to the site would work and it did. These two features, which are essentially feature complete (although all we are returniing is a first name and last name for each individual) resulted in 28 tests. It really is cool to see all those green lights each time you make a change.

Although, you might think that TDD is more time-consuming as you have to write a whole bunch of test code - I think these results demonstrate that you can be productive - as you can be confident - if each test continues to pass - that new work does not break what you have already completed.

It is hard to convert an existing project like DNN to Test Driven Development - but I would urge you to consider this approach when starting new projects.

In future Blogs in this series I will share with you my "Gleanings" as we work through the project. Below is a list of some of the decisions we have made about what we will use for this project, but as we are being Agile this is just a first stab.

Use a TDD development approach to test ALL Layers of the Application.
Use MoQ as a Mocking Framework to help in the testing - we will also use a Web testing framework like WATIN or WebAii, so we can automate tests on the "browser" part of the equation.
Use an IoC (Inversion of Control) Container
Use all the new syntax sugar in C# 3.0 like Extension methods, Lambda Expressions, LINQ - sorry, I do all my non-dnn development in C# so I can keep up to date in both languages.
Use LINQ in the Data Layer (LINQ to SQL for now - but we may switch to LINQ to NHibernate or LINQ to Entities)
Build an MVC-based Website front end - Initially this part of the project is mainly designed as a learning experience, but this may become a full-blown application
Create an MVP (Module View Presenter) style Module for DNN (see Phil's blogs on this approach to Module Development)
Expose a Web Services Interface for Silverlight (RIA) and WPF (Windows) clients

Cambrian First Look - Extension Installer (Pt 3)

Mon, 07 Apr 2008 08:00:00 GMT

In my previous blog in this series I described how the new Extension Installer processes the manifest. In summary the Installer creates an instance of a PackageInstaller and passes the manifest to it. The PackageInstaller then parses the manifest and creates a ComponentInstaller for each component referenced in the manifest. we sthus end up with the PackageInstaller having a collection (SortedList(Of T)) of ComponentInstallers.

Installing the Package

In this blog I will describe the next step in the process - Installing the Package. As in my previous blog we will start in the Install wizard.

  292         Private Sub InstallPackage(ByVal e As System.Web.UI.WebControls.WizardNavigationEventArgs)
  293             CreateInstaller()
  294 
  295             If Installer.IsValid Then
  296                 'Reset Log
  297                 Installer.InstallerInfo.Log.Logs.Clear()
  298 
  299                 'Install
  300                 Installer.Install()
  301 
  302                 If Not Installer.IsValid Then
  303                     lblInstallMessage.Text = Localization.GetString("InstallError", LocalResourceFile)
  304                 End If
  305 
  306                 phInstallLogs.Controls.Add(Installer.InstallerInfo.Log.GetLogsTable)
  307             Else
  308                 'Error reading Manifest
  309                 Select Case e.CurrentStepIndex
  310                     Case 2
  311                         lblAcceptMessage.Text = Localization.GetString("InstallError", LocalResourceFile)
  312                         phAcceptLogs.Controls.Add(Installer.InstallerInfo.Log.GetLogsTable)
  313                     Case 3
  314                         lblInstallMessage.Text = Localization.GetString("InstallError", LocalResourceFile)
  315                         phInstallLogs.Controls.Add(Installer.InstallerInfo.Log.GetLogsTable)
  316                 End Select
  317                 e.Cancel = True
  318             End If
  319         End Sub

The InstallPackage method first creates the Installer (and parses the manfest). As long as the manifest is valid, it clears the Logs and call the Install method of the Installer. If the Install fails for any reason it displays a message to the user. In both cases (success or failure) the Logs are displayed so the user can see the results of the install.

  325         Public Function Install() As Boolean
  326             InstallerInfo.Log.StartJob(Util.INSTALL_Start)
  327             Dim bStatus As Boolean = True
  328             Try
  329                 InstallPackages()
  330             Catch ex As Exception
  331                 InstallerInfo.Log.AddFailure(ex)
  332                 bStatus = False
  333             End Try
  334 
  335             If InstallerInfo.Log.Valid Then
  336                 InstallerInfo.Log.EndJob(Util.INSTALL_Success)
  337             Else
  338                 InstallerInfo.Log.EndJob(Util.INSTALL_Failed)
  339             End If
  340 
  341             ' log installation event
  342             LogInstallEvent("Package")
  343 
  344             'Clear Host Cache
  345             DataCache.ClearHostCache(True)
  346 
  347             Return bStatus
  348         End Function

The Install method attempts to install the packages(line 329), loggeing any exception that might occur. The results of the installation are logged to the EventLog and the Host Cache is cleared.

  188         Private Sub InstallPackages()
  189             'Iterate through all the Packages
  190             For Each kvp As KeyValuePair(Of String, PackageInstaller) In Packages
  191                 'Check if package is valid
  192                 If kvp.Value.Package.IsValid Then
  193                     InstallerInfo.Log.AddInfo(Util.INSTALL_Start + " - " + kvp.Value.Package.Name)
  194                     kvp.Value.Install()
  195                     If InstallerInfo.Log.Valid Then
  196                         InstallerInfo.Log.AddInfo(Util.INSTALL_Success + " - " + kvp.Value.Package.Name)
  197                     Else
  198                         InstallerInfo.Log.AddInfo(Util.INSTALL_Failed + " - " + kvp.Value.Package.Name)
  199                     End If
  200                 Else
  201                     InstallerInfo.Log.AddFailure(Util.INSTALL_Aborted + " - " + kvp.Value.Package.Name)
  202                 End If
  203             Next
  204 
  205             'Delete Temp Folder
  206             If Not String.IsNullOrEmpty(TempInstallFolder) Then
  207                 Directory.Delete(TempInstallFolder, True)
  208             End If
  209             InstallerInfo.Log.AddInfo(Util.FOLDER_DeletedBackup)
  210         End Sub

The installPackages method iterates through the packages (Note: The Installer framework is setup to support the concept of multiple packages in a single manifest/zip. However, the initial implementation of the Wizard only supports one package per manifest/zip). For each Package, the method calls the PackageInstallers Install method.

  278         Public Overrides Sub Install()
  279             Dim isCompleted As Boolean = True
  280 
  281             Try
  282                 'Save the Package Information
  283                 If InstalledPackage IsNot Nothing Then
  284                     Package.PackageID = InstalledPackage.PackageID
  285                 End If
  286 
  287                 'Save Package
  288                 PackageController.SavePackage(Package)
  289 
  290                 'Iterate through all the Components
  291                 For index As Integer = 0 To ComponentInstallers.Count - 1
  292                     Dim compInstaller As ComponentInstallerBase = ComponentInstallers.Values(index)
  293                     If compInstaller.Version > Package.InstalledVersion Then
  294                         Log.AddInfo(Util.INSTALL_Start + " - " + compInstaller.Type)
  295                         compInstaller.Install()
  296                         If compInstaller.Completed Then
  297                             Log.AddInfo(Util.COMPONENT_Installed + " - " + compInstaller.Type)
  298                         Else
  299                             Log.AddFailure(Util.INSTALL_Failed + " - " + compInstaller.Type)
  300                             isCompleted = False
  301                             Exit For
  302                         End If
  303                     End If
  304                 Next
  305             Catch ex As Exception
  306                 Log.AddFailure(Util.INSTALL_Aborted + " - " + Package.Name)
  307             End Try
  308 
  309             If isCompleted Then
  310                 'All components successfully installed so Commit any pending changes
  311                 Commit()
  312             Else
  313                 'There has been a failure so Rollback
  314                 Rollback()
  315             End If
  316         End Sub

This method is the "hub" of the installation process. The first part of the method (lines 282-288) determine whether we are dealing with an upgrade or a new install and save the Package level settings to the database. Next, the method iterates through the ComponentInstallers collection. Remember, this collection is a SortedList, and the loop processes the ComponentInstallers in order. The loop checks if the componentInstaller's version is greater than the version of any previously installed package - we don't want to replace a version 2 component with a version 1 component. If the new version is greater than any existing version, the PackageInstaller calls the ComponentInstallers Install method. Once all the components have been installed, the isCompleted flag is checked to determine if any components failed to install. If there was a failure, then the Rollback method is called, while if the installs all completed successfully the Commit method is called. This provides a level of Transactional support. If one component fails, an attempt is made to "undo" the changes that were successfully made in other components.

If we look at the Commit method

  256         Public Overrides Sub Commit()
  257             For index As Integer = 0 To ComponentInstallers.Count - 1
  258                 Dim compInstaller As ComponentInstallerBase = ComponentInstallers.Values(index)
  259                 If compInstaller.Version > Package.InstalledVersion AndAlso compInstaller.Completed Then
  260                     compInstaller.Commit()
  261                 End If
  262             Next
  263             If Log.Valid Then
  264                 Log.AddInfo(Util.INSTALL_Committed)
  265             Else
  266                 Log.AddFailure(Util.INSTALL_Aborted)
  267             End If
  268         End Sub

and the Rollback method

  429         Public Overrides Sub Rollback()
  430             For index As Integer = 0 To ComponentInstallers.Count - 1
  431                 Dim compInstaller As ComponentInstallerBase = ComponentInstallers.Values(index)
  432                 If compInstaller.Version > Package.InstalledVersion AndAlso compInstaller.Completed Then
  433                     Log.AddInfo(Util.COMPONENT_RollingBack + " - " + compInstaller.Type)
  434                     compInstaller.Rollback()
  435                     Log.AddInfo(Util.COMPONENT_RolledBack + " - " + compInstaller.Type)
  436                 End If
  437             Next
  438 
  439             'If Previously Installed Package exists then we need to update the DataStore with this 
  440             If InstalledPackage Is Nothing Then
  441                 'No Previously Installed Package - Delete newly added Package
  442                 PackageController.DeletePackage(Package.PackageID)
  443             Else
  444                 'Previously Installed Package - Rollback to Previously Installed
  445                 PackageController.SavePackage(InstalledPackage)
  446             End If
  447         End Sub

we can see that both methods follow the same pattern as the Install method. They iterate through the ComponentInstallers calling the relevant methods in these classes.

The advantage of this process is that the installation of each component type is delgated to a custom installer that knows how to handle the component. Thus the install method of the AssemblyInstaller is very different from the Install method of a ConfigInstaller. As we saw in the previous blog, this approach is also extensible.

Uninstalling an Extension works in much the same way as Installing, so we will get a Log of the Uninstall, both if the uninstall is a success or a failure.

I hope these last two blogs have given you an overview of how the process works. Unless you actually need to create your own ComponentInstaller, you should not need to delve into this code in detail. However, if you would like to take advantage of some of the new aspects of the Extension Installer, you will need to understand how to build a version 5 manifest, and I will delve into that aspect in more detail in my next blog on the subject.

Charles Nurse

Gleanings From the Wet Coast - Agile and TDD

Cambrian First Look - Extension Installer (Pt 3)

Installing the Package