Software Configuration Management
A Presentation to the Australian Delphi Users Group
by Steve Forbes, OzmoSys
21 October 1997
What is Software Configuration Management?
Software configuration management (SCM) is the process of organising and managing changes to the components of your software development efforts. Effective SCM helps members of a development team to retrieve and build any version of an application! in a consistent and repeatable manner. The attached documents provide an excellent overview of SCM and version control.
What is Version Control?
Even in the smallest development environments, accurately tracking and recording changes to project files is critical. Tracking these changes with version control gives you access to them at any time.
Version control is the process of managing and maintaining multiple revisions of the same file in an archive. An archive contains information about each revision of the file, allowing you or members of your development team to retrieve, modify, an! d return any version of a file in a safe, organised and consistent manner.
Software Version Control - Why Bother?
The traditional method of ensuring the security of computer information is through regular backups of changed files, however, additional protection is essential for software projects (frequently valued in terms of person-years). Version control was devel! oped specifically to provide source code security for software development projects and offer benefits which cannot be achieved by any other means..
Using a version control system requires a degree of discipline (just like performing backups does) but is well worth the effort. My use of version control has saved my hide on more than one occasion! Imagine the cost of having to redevelop an application! from scratch if the worst ever happened ...
Using your current code management procedures are you able to:
If you answered "no" to any of these questions, your coding investment may be at risk and some form of software version control is definitely worth considering.
With version control, you minimise the risk of overwriting changes made by yourself or another developer in your team. You don't have to worry about losing data, because all changes and information are archived and easily accessible. You can create a cop! y of a file at any point in its development history, and get specific information about each change made to your project files.
Although generally considered a team development tool, every developer should use version control. The table below lists a number of the version control products available, do yourself a favour and invest in one!
Table 1. Software Version Control Reference Sites
Intersolv PVCS Version Tracker
MKS Source Integrity
Microsoft Visual SourceSafe
Burton System Software Tlib
Quma Version Control System
The two attached documents, entitled Team Development Overview and How to Select the Right Team Development Product, are reproduced with the permission of their author Alex Lobba and were sourced from his web site at http://www.silcom.com/~alobba/
Team Development Overview
Version Control and Software Configuration Management: Past, Present, and Future
Version Control (VC) and Software Configuration Management (SCM) have greatly evolved in the last twenty years. Tools have gone from file oriented versioning utilities to full blown repository-based systems that manage projects and support team developme! nt environments, even across geographic locations.
This document presents a brief overview of team development, the evolution of this industry, and the trends that are going to shape the next generation of products. It also includes a list of typical product features.
The terms Version Control and Software Configuration Management describe some of the mechanisms employed by tools that manage software applications. Team Development indicates the end result that these tools should help us accomplish. Just controlling so! ftware versions and configurations is no longer enough. Today we are dealing with:
|Teams that are often distributed across different sites as a result of mergers, acquisitions, and market globalization.|
|"Massively" parallel development to meet time-to-market schedules, maintain prior releases, and support application variants for different platform, GUI, database, and national language flavors.|
Combine this with:
|Distributed and multi-platform computing environments.|
|Client/server applications and rapid application development tools.|
|Shared code between applications.|
...and the old days of monolithic applications for a single host look like a Hawaiian vacation. What is required today is a way of managing the complexity of software projects while empowering team members to work effectively together (as opposed to becomin! g very skilled at circumventing the system).
In a nutshell, a team development solution should:
|Track changes to all project components|
|Support parallel development (branches, differences, merges, distributed teams)|
|Control the entire project and its evolution in time (releases and variants)|
|Manage the approval of changes (promotion process)|
|Manage builds and dependencies|
It's not easy to classify VC and SCM tools. Most vendors claim very similar sets of functionality, therefore we find it easier to look at the evolution of this industry not in terms of supported features, but in terms of underlying architecture and metho! dological orientation. First we are going to outline the architectural evolution of the industry, then we are going to group tools by their approach to VC and SCM.
First generation tools provide what is commonly known as Version Control. These are tools that are file based in that they store changes to individual files in file archives. They support a check-out/check-in model, and provide branching. Archives store ! the contents and the meta data (user name, labels, comments, etc.) for each file version.
Public domain SCCS and RCS are the "grandparents" of version control. Many other tools evolved from their model, and many tools in different generations still use them as their file archiving mechanism. CVS is a public domain tool that enhances! RCS.
First generation commercial products include Sun Microsystems TeamWare that is basically a souped up SCCS with a GUI. Intersolv PVCS and MKS Source Integrity are also commercial products that have added, on top of file archives, a GUI, a project meta dat! a layer, a promotion model, and additional functionality.
Tools in the second generation are easy to identify because they store all project and file metadata in a database (repository) that is separate from their file archives. This architecture shifts the focus from the file level to the project level and pro! vides a central view of all project information. This results in better support for parallel development and team coordination, and provides the necessary infrastructure to implement process management.
Second generation products cover a wide range of functionality: from the more simplistic, but very intuitive, Microsoft Visual SourceSafe; to the fast command line P3 tool; to the change request based IBM CMVC; to the change-set based TrueSoftware Aide-d! e-Camp/Pro; to the process oriented Platinum CCC, and SQL Software PCMS.
SCCS, RCS, CVS, TeamWare, PVCS,
Project Repository Based
AdC/Pro, CCC, CMVC, P3, PCMS,
A challenge of second generation tools is that controlled files cannot be accessed directly from other tools unless copies are placed into working directories from the repository. This can result in the proliferation of local copies and the temptation to! overwrite the repository with changes made in the local workarea, potentially undoing changes made in the meantime by other users. To address this issue come third generation tools that add "file transparency" while retaining the benefits of a pr! oject repository.
Two third generation products are Atria ClearCase and Continuus/CM from Continuus. ClearCase accomplishes file transparency by means of a proprietary file system (MVFS) that intercepts system file access calls (open(), read(), etc.) and routes them to it! s repository. Continuus/CM provides direct file access by creating symbolic links from user workareas to its cache directories. Another unique aspect of third generation products is that the view of an application that users see is defined by configuration ! specs that automatically select the applicable file versions to accomplish things like "Give me release 5 of this product for Sun Solaris, plus all the changes made to fix bugs 678, 679, and 681 that have been approved by QA."
Here we group tools by their approach to version control and configuration management. Keep in mind that no tool fits perfectly in just one category, and that usually elements of different categories are mixed in each product. However, understanding the ! primary approach helps match products to the culture of an organization. We see two primary approaches in this industry today: flexibility and control.
Tools in this class put a stronger emphasis on catering to the needs of software engineers.
|Individual Developers - SCCS, RCS|
|Small Teams - Visual SourceSafe, Source Integrity, TeamWare, CVS|
|Scalable to Large Teams - ClearCase, P3|
Management needs for formalized procedures are at the forefront in these products.
|Change request based - CMVC|
|Change set based (logical change units) - AdC/Pro|
|Security based - PVCS|
|Process based - Continuus/CM, CCC, PCMS|
What are the current trends and what can we expect next? The last few years have seen a strong push to integrate configuration management with problem tracking. This has helped organizations close the gap between the recording and tracking of problems an! d their actual resolution. The metrics that become available from this type of integration go a long way in helping organizations improve the quality of their software and the effectiveness of their software processes.
The other new threshold that has been crossed is that of supporting teams distributed across different sites. This effort is still in its infancy, but is showing promising results with products like ClearCase and Continuus/CM. We expect this area to matu! re further in the next few years with complete wrappers of automation being built around what are now rudimentary mechanisms.
On another front, the mainstreaming of client/server applications and rapid application development tools, is pushing both RAD vendors and SCM vendors to provide good team development solutions for these environments that go beyond basic version control ! and manage projects and file sharing between applications.
And who can ignore the Internet and intranets? How is the "great equalizer" going to impact team development? We certainly hope to see soon Web interfaces to existing SCM products. But more importantly, the Internet is shifting the emphasis fro! m files and projects, to information and access. Therefore, we are tempted to predict that mechanisms to access project file over the Internet will soon make virtual teams a very workable reality. Having done that, these virtual teams will need to find an I! nternet equivalent to the brainstorming sessions that take place in front of the coffee maker, while grinding and brewing dark espresso beans.
Sounds like groupware, doesn't it? Team members will want to see what tasks are active for what release, and be able to browse project notes to understand the impact of another task on their own code before it becomes an integration nightmare. And when a! customer/user reports a problem, wouldn't it be nice to be able to look up the project notes to find out what tasks affected the area of functionality in question, get down to the individual problem reports or enhancements, and even the specific files. And! doing this from a UK office, accessing project information in the US while everybody there is asleep. If only browsers could support "sniff" files, we could even include the smell of brewing coffee in the project notes...
Will SCM vendors interface with groupware tools? Will they build groupware functionality into their tool? We will probably see a mixture of both. If we really wanted to get esoteric, we could also predict that the fourth generation of SCM products will s! upport truly fine grained objects, and will rely on embedded versioning services from object oriented operating and file systems. But this discussion requires too much coffee.
In this section we list the building blocks of functionality that should comprise team development solutions:
|Workspace and Release Management|
|Integrations and Tool Interfaces|
This is intended as a pool of information against which to match your requirements to determine what is relevant to you. It can also be used as a baseline against which to compare individual tools to determine how complete their solution is and to assess th! eir specific way of implementing functionality. Although all these features may look reasonable and necessary, no single tool includes them all, or implements them in a way that you would necessarily deem satisfactory. It is therefore important that you det! ermine a scale of priorities in order to find the best possible match that you can live with.
Version control deals with organizing projects and project components, tracking changes, and supporting parallel development.
|Version any type of file (source code, binaries, documentation, etc.)|
|Forward/reverse deltas (to store changes efficiently)|
|File compression (to version binaries and reduce disk requirements)|
|Branching (to support parallel development and project variants)|
|Automatic branch creation (when working on a given issue or view)|
|Version labels (to identify the reason for changes and group related changes)|
|Visual differences (to compare file and project versions)|
|Visual merging (to resolve parallel development efforts)|
|Parallel development (multiple checkouts, with lock, of same file version)|
|Support projects (in addition to managing individual files)|
|Support sub-projects (to share code between applications)|
|Unlimited directory hierarchy (to organize project files)|
|Recursive commands through hierarchy (to operate on a whole project)|
|Track changes to project structure (directories, deleted files, renames, etc.)|
|Project/file sharing between multiple applications (without physical duplication)|
Workspace and Release Management provides a working environment in which different team members can access and modify the specific version of project components they need.
|Identification of releases and baseline|
|Recreation of prior releases and baseline (better if accomplished without making copies of files)|
|Working directories/views (to make changes without being affected by other developers)|
|Rule-driven views (to specify the exact file versions required, such as "release 5 plus all changes to fix bug-777)|
|Transparent (direct) access to controlled files from tools (to support existing development tools)|
Build management automates the creation of deliverable products. This may not be a consideration when working with development tools that include and require their own build utility, but is an important issue when working with more traditional programmin! g languages such as C, C++, Fortran, etc.
|Make compatible (to leverage investment in existing Makefiles)|
|Parallel builds (to run multiple build steps concurrently)|
|Distributed builds (to distribute build steps on multiple machines)|
|Load balancing (to dynamically adjust build distribution to CPU loads)|
|Target builds (to direct builds to a different target platform)|
|Automated generation of dependencies (to ensure complete and accurate builds)|
|Bill-of-Materials (BoM) (to have a permanent record of all build constituents)|
|Footprinting (to include BoM in executables)|
|Shared binaries (to avoid unnecessary rebuilds of shared files)|
More formal organizations require the definition and enforcement of policies and procedures to bring discipline and standardization to their software development process.
|Triggers (pre- and post-event handlers to customize the product)|
|Lifecycle (workflow) stages and promotions (to standardize the process of releasing changes)|
|Type-based lifecycles (different file type, different lifecycle)|
|Customizable process model (to implement your process)|
|User groups/roles (to link sets of privileges to a class of users)|
|Lifecycle-and-role based security (to control who can do what, where, and when)|
|Change packages for group promotion (to ensure that all related changes are treated as a unit)|
|Notifications (to automatically inform team members of certain events)|
|Task management (to organize task related information and associated files)|
How users and other applications interface with the product.
|GUI (graphical user interface)|
|CLI (command line interface)|
|Same functions in GUI and CLI|
|Internet Web interface|
|API - Documented application programming interface|
How the product fits in your computing environment.
|Support multiple platforms|
|Interoperable across platforms|
|Three-tiered client/server (client, application server, database server)|
|Repository or file-based (all meta data in DBMS or in file archives?)|
|Standard or proprietary file system|
|Replication/synchronization across multiple sites|
|Backup and recovery|
Team development is not a stand-alone function. A good solution must work well with the other software development and maintenance tools that an organization employs.
|RAD (rapid application development)|
|Development frameworks and environments|
The rapid changes in computing and application architecture, and the wide-spread development of applications at the departmental level are forcing a new level of awareness of team development in software organizations, and are pushing vendors to make the! ir team development products more sophisticated and yet easier to use.
Version control and configuration management tools cover a wide range of approaches and functionality. A third generation tool is not necessarily "better" than a first generation tool. It all depends on which characteristics best match your nee! ds and environment. Buying the most advanced product could be a mistake if you are only trying to handle a very simple situation. Therefore, the issue is not which product is best, but which one is the best match for you.
How to Select the Right Team Development Product
Finding the Right Product
Management Support and User Acceptance
Determining Your Requirements
Selecting a Product
Implementing a Product
The objective of this document is to help organizations make an informed team development product evaluation and selection, and have a successful implementation that is hopefully not too painful.
Finding the Right Product
How is a development team to go about finding the right product for their needs? Given the fact that Software Configuration Management (SCM) provides a critical foundation for all development and maintenance activities, the selection process deserves som! e thought and time. On the other hand, nobody can afford to get stuck in "analysis paralysis" mode. Rest assured that whatever you choose is not going to be a perfect solution, and will require ongoing adjustments. But if planned properly it will ! produce valuable benefits in terms of quality and productivity.The secret to a successful SCM implementation lies in the combination of user acceptance and management support. Leave one of these ingredients behind and you are asking for a painful experience! .
Finding the right product is easy. Knowing what you are looking for is the tough part. The discussion below assumes that you are in an organization with 10 or more developers. If you are in a smaller team, some of these steps are going to be greatly stre! amlined, but should still be performed.
Management Support and User Acceptance
If management support is not there from the beginning, you need to do your homework and present your case in a compelling way leveraging the successes and horror stories of your organization and similar ones. Particularly useful are case studies in the s! ame industry. Network with other professionals and leverage the pool of knowledge in the configuration management discipline. A posting in the newsgroup comp.software.config-mgmt can get you in touch with excelle! nt resources.
After management support is secured, create a small and determined task force with representatives from each key functional area in the team (developers, quality engineers, testers, configuration managers, build meisters, operations, etc.). These individ! uals must be empowered to make decisions and ensure the buy-in of the group they represent.
Determining Your Requirements
Before you start looking at products, document your requirements. Requirements are not just a laundry list of features. Instead, requirements must include:
|A description of your computing environment, tools, and applications (current and planned).|
|A list of current project phases (i.e., middle of release cycle, between releases).|
|A description of the current structure of the team, and its culture (careful to acknowledge the politics, but not get sucked into them, if possible).|
|A description of how the software development and change process is performed today (tools and process). What works, what doesn't, and why.|
|A description of what policies and procedures you want to put in place that will ensure you meet the objectives of the organization.|
|A prioritized list of required features and capabilities. This should include a handful of must-have's (the things you can't live without), and a small list of highly desirable items. The rest is gravy.|
The thinner these documents are, the better (particularly the requirements section). The task force should not be paid by the pound of paper they produce. Leave that to consultants, like us.
Selecting a Product
Now you are ready to talk to vendors. You do it earlier, and you end up with one of those strange requirement documents that only one vendor could possibly fulfill. When you see one of them you can always tell which vendor an organization was "influ! enced" by. On the other hand, when you start talking to vendors be open to their suggestions. Many have a wealth of experience with a large number of installations and their input can be useful, if they really listen to what you want to accomplish, and! support you in your goals. In general vendors should be thrilled that you know what you are looking for. It lets them know that you have commitment and will not waste their time, which means that you will get their top notch resources to help you. It also ! makes it clear to vendors what it's going to take to get your business.
Do a first on-paper screening of products based on your must-have list. Those products that pass the test are candidates for evaluation. Take the time to do hands-on evaluations. No matter how good a product looks on paper, there is no way to be able to ! tell how it will really behave, until you play with it. Set up a simple case study with your code that quickly gives you an idea of what it would be like to go through a release cycle with each product.
In parallel with your evaluations, check references. Get names in your industry and with similar requirements. Just because you are starting to fall in love with a product doesn't mean that you should short cut your research. Check the vendor, their stab! ility, their support.
Implementing a Product
Before you throw the product you selected to the masses, take the time to strategize your rollout time-line. Make sure you build enough internal know-how to support the effort. And identify the potential risks (project deadlines, system capacity, etc.) t! o a successful implementation, and how you plan to address them.
Most important, remember that by now your original requirements are probably obsolete, and should not be treated like the bible.
Team development is a very dynamic discipline that affects a wide cross-section of software development functions. When implemented with management support and team buy-in, it can be extremely rewarding, and result in quality and productivity improvement! s. Otherwise, it can be extremely frustrating, and amplify political squabbling and finger-pointing.
It is important to take the time up front to build a foundation of understanding and communication that will create clear goals and requirements, simplify the tool selection process, and streamline its implementation. If you get external help in this pro! cess, make sure that you use it to help you find your answers and make your decisions, and not to make a decision for you.
Team development and developing a team are two sides of the same coin.