In Continuous Integration Success Factors – Right environment , the essential question for the right environment is sitting on how to make my own Dev/QA integration testing environment has the right elements for CI and most integration testing coverage.
To make CI works, each Dev/QA integration testing environment should have continuous update of latest stable build for itself and their dependency components. In a big enterprise, that could get very complex in terms of getting the latest stable builds of the dependency components in an effective way. So this blog describes the first step on the CI practice – how to make each component or product ready for CI consumption by other team. Only when every component is CI ready, this practices can be executed effectively across the board.
Here are some check lists to make one component CI ready – Integration readiness criteria that we will be using to implement this practice:
1. Automatic build/deployment script
This is pretty straight-forward. Anything that involves manual inception will decrease CI efficiency, and in the end result in no real CI at all. It is ideal that when you integrate with some other software product or component, you start from a clean state which is source code. So by automatically building artifacts from source code and deploying the artifacts and necessary configurations including DB schema and server configuration, you are sure that you are integrated with all latest changes without any artificial noise. But in some case, team can choose to separate build and deployment process and only provide artifacts and deployment scripts for other team to consume.
The automation of those steps are not necessarily the hardest part. Powershell script or Maven (combined with Ant) in general provide powerful tools to automate those manual build and deployment steps. The tricky part is probably what is the defined “clean” or “appropriate” stated to deploy into each dev/QA testing environment.
This may involve the question what are the initial configuration to setup or data set to populate for different environment requirement, especially when multiple of software components are involved to make one type of configuration work. It could need some level of customization based on different team needs. So the build/deployment script should have that built-in flexibility to allow flexible customization of the data people may need, for example add configuration file for data or parametrize the data input.
2. Labeling of any stable build
Once you have automatic build/deployment script ready, you need to label any of your stable build so that other team can select which build to use for their own integration purpose. Also, labeling will help for any rollback as well once some blocking issues occurs.
This label can be made in source code level (p4 label with date information) or build artifacts level (for example versioning of jar or war file for a JEE application) depending on which scripts the product make it public to other team. Stable build in general is made available nightly after automation test cases run pass some defined criteria (see #3).
Once label is done, automatic build/deploy script should have the capability to pick up appropriate version of stable build according to each team’s need.
3. Stability criteria
Now you can automatic build/deploy your product and label it with appropriate versioning, but how to determine one build is “stable”? This is a very important question.
Most of time when team is hesitating to continuously integrate with its dependency in their own dev/QA environment, that is not because they dont want to or don’t know the values of CI, rather, it is because it is too expensive to integrate with “broken or unstable” products in daily routine. Once there is a broken link somewhere, this whole CI process could fall apart as there is chained reactions across the board.
It is probably easy to identify a stability criteria. General practice is to select a subset of high priority test cases and choose 100% pass as a stability criteria. Those high priority (P1) test cases should cover the majority of critical path or happy path of its software. The hard part is stability criteria can only works as well as how those high priority test automation cover. Identification of those P1 test cases are always a continuous work. The P1 test cases can be added or deleted based on feature changes, omission found during integration with other teams etc.
