Continuous Integration and build engineering in Microsoft .NET

In this article will be introduction to Continuous Integration in .NET. I will use two cool book regarding to this topic: "Addison.Wesley.Continuous.Integration.Jun.2007" and "Deploying .NET applications. Learning MSBuild and ClickOnce" which you can download at the end of this article. I will do some kind of book review with best for my opinion phrases, questions and advices. Have you ever heard the term “integration hell”?

It is a situation where it takes more time to integrate the changes into the code base than to develop them. Normally the developer takes a repository snapshot to work with. While he is working on his copy the main code base changes because other developers submit their changes. But he has to eventually submit his work. That’s when all (integration) hell may break loose.

Continuous Integration (CI) is a process in software engineering aiming at minimizing this problem. It targets the best practices in software delivery: frequent integration, constant readiness, short and frequent build feedback, automated testing and so on. A full blown CI process will build, test, analyze and possibly deploy an application. This process should be triggered by every source code change. It should provide immediate feedback to the development team. The best way to accomplish all this is to set up a CI server.

Best practices in Continues Integration, CI servers and development process by Martin Fowler:

1. All developers run private builds2 on their own workstations before committing their code to the version control repository to ensure that their changes don’t break the integration build.
2. Developers commit their code to a version control repository at least once a day.
3. Integration builds occur several times a day on a separate build machine.
4. 100% of tests must pass for every build.
5. A product is generated (e.g., WAR, assembly, executable, etc.) that can be functionally tested.
6. Fixing broken builds is of the highest priority.
7. Some developers review reports generated by the build, such as coding standards and dependency analysis reports, to seek areas for improvement. 
8. Is it easy to use
9. Is it cool and hip

Regardless of platform, technology, or domain, deploying working software principally embodies six high-level steps.
1. Label a repository’s assets.
2. Produce a clean environment, free of assumptions.
3. Generate and label a build directly from the repository and install it on the target machine.
4. Successfully run tests at all levels in a clone of the production environment.
5. Create build feedback reports.
6. If necessary, you can roll back the release by using labels in your version control repository.

Peer-based code reviews are generally considered beneficial to the overall quality of a code base because they present opportunities for an objective analysis by a second pair of eyes. For this same reason, XP’s pair programming practice offers some of the same objective analysis benefits. Static source code analysis tools like Java’s PMD and .NET’s FxCop, which scan files for violations of predefined rules, offer some of the same analysis benefits.

These practices and tools also work very well in geographically distributed teams (i.e., some developers work from home, others at the office, and others in another state, country, continent, etc.). It helps mitigate any additional risks with people out of range for verbal collaboration.

Automate unit tests Automate your unit tests, preferably with a unit testing framework such as NUnit or JUnit. These unit tests should have no external dependencies such as a file system or database.  How reliable do you want your software to be? Source code is only as reliable as the test coverage, and tests are only as valuable as their execution frequency. By segregating tests into four automatable categories mapping to unit, component, system, and functional, a CI system can be configured to execute tests in an efficient manner. Unit tests can be run during checkins; component, system, and functional tests on some regular interval—such as with a secondary build.

Choosing the right CI server is not an easy task. You have to deal with the hardware and software aspects here. On the hardware side you have to determine if you have a separate physical machine for your CI server. Is it a full blown server with 99.9% uptime or only an old machine standing in the corner of your developers' room? If you don't have a physical machine can you get a virtualized server running somewhere where every team member has access? If you just want to check things out, it is all right to install it all together on your development machine but you have to be aware that it most likely wouldn't be a production setup.

Top 50 questions for your team regarding you build system:

1. On average, is everyone on your team committing code at least once a day? Are you employing techniques to make it easier to commit code often?
2. What percentage of each day’s integration builds is successful (that is, the most recent build run has passed)?
3. Is everyone on your team running a private build before committing to the repository so that integration errors are reduced?
4. Have you scripted your builds to fail if any of your tests or inspections fail?
5. Is a broken integration build a priority to fix on your projects?
6. Do you avoid getting the latest code from the version control system when there is a broken build?
7. How often do you consider adding automated processes to your build and CI system—on a continuous or even periodic basis?
8. When do you find the most defects on your project, in the beginning or in later parts of the lifecycle?
9. How do you determine the quality on your software projects? Are you able to measure it?
10. Which processes on your projects are manual? Have you determined which processes you can or should automate?
11. Do you have all of the scripts to rebuild your database and data in your version control repository? Are you able to rebuild your database and test data during the build process?
12. Are you able to perform regression testing whenever a change is made to the software? Are you able to run various types of regression tests, including functional, integration, load, and performance tests?
13. Do you have the capability to determine which source code does not have a corresponding test? Are you using test coverage tools?
14. What percentage of your software has duplicate code? Are you seeking to reduce this amount?
15. At any point in time, how do you verify that the current source code adheres to the software architecture?
16. How do you notify that the build or deployment is ready to test?
17. Which communication mechanisms on your project are manual, and which should be automated?
18. Are you able to view a current visual diagram of your software?
19. How do you communicate the software architecture to a new developer on the project?
20. How much money do you have to spend?
21. Do you want to pay the angle bracket tax (XML)?
22. Does it support the tools that you need?
23. How good is the documentation and support?
24. Does it do what you want it to do?
25. Does it do more than you need, not just now, but into the future?
26. Is your build automated? Are you able to run your build without your IDE?
27. Have you centralized all of your software assets into your version control repository? Are you able to perform a complete build by getting all necessary files from the version control repository?
28. Do you ensure that the build tasks that are more likely to fail are at the beginning of your build scripts so that you receive notification of a build failure quickly?
29. Do you have an “Integrate button” for your software build processes? Is your database integration automated? Testing? Inspection?
    
30. Deployment? Are you receiving and using feedback from the process?
31. Does your integration build process occur on a separate machine?
32. What is the duration of your integration builds? Are you seeking to shorten your build duration to improve feedback?
33. Are you using a CI server to integrate your software? Or do you have a disciplined process for manually integrating builds?
    
34. How often does your project perform integration builds: weekly, nightly, or hourly? Or is it at every change (continuously)?
35. Are you using a version control repository (or SCM tool)?
36. Is your project’s build process automated and repeatable? Does it occur entirely without intervention?
37. Are you writing and running automated tests?
38. Is the execution of your tests a part of your build process?
39. How do you enforce coding and design standards?
40. Which of your feedback mechanisms are automated?
41. Are you using a separate integration machine to build software?
42. Do you perform unit testing sporadically, periodically, or continuously?
43. How often do you run your full unit, component, and system test coverage review?
44. Are you monitoring code complexity?
45. Are you continuously performing automated design reviews with tools like JDepend and NDepend?
46. Are you automating code audits with tools like PMD, Checkstyle, or FxCop?
47. Are you monitoring code duplication?
48. Are you able to assess code coverage? How are you reacting to the data?
49. Do you know what percentage of your code has a corresponding test?
50. Is your build properly configured to produce coverage reports?

How to improve feedback from you build system?

References

Best books about continues integration:

continues integration.pdf (89.09 kb)

Addison.Wesley.Continuous.Integration.Jun.2007.pdf (4.05 mb)

Deploying .NET applications. Learning MSBuild and ClickOnce.pdf (8.89 mb)

Continues integration tools:

Bitten http://bitten.cmlenz.net/
BuildBeat www.timpanisoftware.com/
BuildBot http://buildbot.sourceforge.net/
CM Crossroads www.cmcrossroads.com/
CruiseControl.rb http://cruisecontrolrb.thoughtworks.com/
Gump http://gump.apache.org/
PerfectBuild www.codefast.com
Pragmatic Automation www.pragmaticautomation.com/
Pulse www.zutubi.com/products/pulse/
TeamCity www.jetbrains.com/teamcity/
Tinderbox www.mozilla.org/tinderbox.html

Best features into your build system in addition

Maven - A project management and build tool. Also see Appendix B.

NAnt - NAnt is the port of the Java-based Ant tool to the .NET platform.

Rake - Rake is the build-scripting tool for Ruby-based applications. If you are using Rake, you can also utilize the power of Ruby when scripting your builds.

NDepend - NDepend is a Visual Studio tool to manage complex .NET code and achieve high Code Quality. With NDepend, software quality can be measured using Code Metrics, visualized using Graphs and Treemaps, and enforced using standard and custom Rules.

Summary

You have to be well equipped if you want to fight integration hell. Continuous Integration is a prescription for how to win, but to do it, you will need good tools. One of them is a continuous integration server. A straightjacket you have to wear to achieve your goal. No matter if you are using a self made script, an Open Source tool or a complicated and expensive system, you need something that will get the constantly changing source code and integrate it according to a plan, to form working software. All the time and always - continuously!