Difference between revisions of "Software Development (WG1) Version Control (T3)"

Revision as of 17:45, 22 November 2005

Setting up a VistA Development Environment

Overview

This is a description of how VistA development environments should be configured.

Monospaced text designates program names, directory names, command line parameters, etc., that are intended to be typed in at a Linux shell or a GT.M prompt, e.g. /bin/bash. Italic text is a descriptor of something whose actual value must be determined when the command is typed, e.g., gtmver.

Theory

A release is a collection of routines and global variables. The canonical packaging of a release, which is suitable for importation into any standard M distribution, is a collection of M source code routines and a database extract in ZWRite format. When installed on a system, to run with an M implementation, a release may have pre-compiled object files, database files, shell scripts, global directories, etc.

Below is a diagram of the “roll-up” by which a VistA release (“Release A”) gets converted into another VistA release (“Release B”) by a development team. A and B could be from different families, e.g., Release A could be a FOIA release and Release B could be a Leonardo da VistA release.

Once Release A is installed, it remains frozen and unchanged except for any essential patches that may become available independent of the development process for creating Release B and without which the release lacks some required functionality. This means, for example, that KIDS patches should be applied in an integration environment rather than a release environment.

At any given time, there can be multiple development projects underway based on Release A. In other words, there may well be a Release C that is also based on Release A, but which has its own integration and development areas.

To start the development process, an integration directory is created and initialized with a copy of the global variables from Release A, as well as shell scripts, global directories, etc. The integration directory may well have a different name from Release A and Release B. For example, Release A may be FOIAVistA_20051021, Release B may become LeonardoDaVistA_0.4 (at the time that the integration area is created, the name and version of Release B may not even be decided yet), and the integration and development areas may be named Greenbelt_200510. There is no need to copy routines, since the GT.M search path will be set up to look for routines in the integration area before looking in Release A; hence only routines that are different between Release A and the new release need to be duplicated.

Actual development does not take place in the integration area. Development occurs in development areas, typically in the directories of individual developers; however, there may be a shared development area if two developers are collaborating on making changes to the very same module (i.e., their code changes overlap). A development area is set up by creating a directory structure, shell scripts, etc. Note that development areas can be hierarchical so that development subtasks 1a and 1b can be developed separately and integrated into development task 1, prior to integration into the integration area, etc.

During this development process, developers develop code in individual sand boxes (development areas), but normally share the database (global variables) in the integration area – there may typically be no need to create a separate database. However, if some global variable changes are tied to code changes being made in a development area, a copy of those global variables can be placed in a separate database file in that development area, and a global directory for that development area can map those global variables to the database in that development area, with all other global variables mapped to the database in the integration area.

When a developer has code that is ready for integration, the new versions of the modified routines are promoted to the integration area. Any global variables tied to the code being promoted that are in a database in the development area must be merged, reconciled or otherwise integrated with the global variables in the database in the integration area and the global directory in the development area modified accordingly. Note that there needs to be a process, e.g., involving code reviews, by which a development task is considered complete enough to be promoted. These, and other quality gates, are not discussed herein.

Creating a release is minimally the process of creating the directory structure for Release B, copying the routines from Release A, overlying the routines with those from the integration area, and copying the global variables from the integration area.

Normally, Release A, the integration area and development areas would all use the same GT.M version. However, this is not a requirement.

It is strongly recommended that database files in integration and development environments be journaled, and backed up regularly. It is also strongly recommended that a version control system such as CVS or subversion be used for integration areas.

Practice

Shell

The standard shell for VistA community scripting is bash (installed as /bin/bash on Debian GNU/Linux systems).

GT.M Release Directories

Each release of GT.M is installed its own subdirectory of /opt (preferred) or /usr/local (acceptable). Each directory name starts with gtm followed by a release number separated from the release name by an underscore (_). Thus, for example, GT.M V5.0-000C would be installed in /opt/gtm_V5.0-000C. /opt/gtm is a relative symbolic link to the latest version, set up with a command sequence such as:

 cd /opt
 rm -f gtm
 ln -s gtm_V5.0-000C gtm

When installing GT.M, the file gtmprofile should be edited so that the line:

 EDITOR="/bin/vi"; export EDITOR

is modified to read:

 export EDITOR=${EDITOR:=/bin/vi}

VistA Release Directories

Each release is installed in its own sub-directory of a standard system location, such as /opt (preferred) or /usr/local (acceptable). Each directory has a release family name followed by a release / version number suffix separated from the release name by an underscore. Directories for releases without version numbers, such as FOIA releases, have a date suffix of the form yyyymmdd (and where appropriate just yyyymm). For example, an August 25, 2005 FOIA release would be in /opt/FOIAVistA_20050825, an October 21, 2005 release in /opt/FOIAVistA_20051021, an OpenVistA 0.3 release in /opt/OpenVistA_0.3 and a Leonardo da VistA 0.45 release in /opt/LeonardoDaVistA_0.45.

The normal protection on all directories and files in a release directory should be read-only. To install files in the p subdirectory, it should be made read-write, the source for the patch copied in, the object file(s) generated, and the directory again made read-only.

For each release family, the release directory contains subdirectories g, o, r and p, symbolic link gtm, and a script install. Other shell scripts may be created for various maintenance purposes (and ideally would be documented here).

• Subdirectory g contains a global directory file mumps.gld and a compressed database file mumps.dat.gz. The global directory maps all global variables to a single database file $vista_home/g/mumps.dat (i.e., the environment variable $vista_home is used at run-time to point a GT.M process to the location of the database file, so that different processes can actually use the same global directory to refer to entirely different database files).

• Subdirectories o and r correspond to directories for files containing routine object code and routine source code respectively. Each file in r ends in .m and has a corresponding file in o with a newer time stamp and which ends in .o.

• Subdirectory p contains source and object code for patches to a release (generated outside the scope of the work for creating Release B) and which are deemed to be essential for the proper functioning of the release.

• The symbolic link gtm points to the GT.M directory used for this VistA directory.

• The script install creates a new integration directory to be used in creating a new release from this release. Normally, the initial database file of the integration directory is created by uncompressing mumps.dat.gz. Option --newgtm gtmver, specifies that the integration directory is to be configured to run a different version of GT.M (as specified by the directory gtmver) from the one used by the release. If --newgtm gtmver is used, the default assumption is that the database will need to be extracted with the version of GT.M used by Release A, and loaded into a new database created with the version of GT.M to be used in the integration directory. When --newgtm gtmver is used, a an additional optional option --nonewdb is used to specify that the releases of GT.M have compatible database formats, and that database from Release A can be used unchanged in the integration directory. Note that the use of --newgtm gtmver will almost certainly result in a command that takes a long time, because the database may need to be extracted and reloaded, and the routines recompiled. Here is the script install:

VistA Integration Directories

A VistA integration directory is similar to a VistA release directory, with the following differences.

• Subdirectory g contains a database file, mumps.dat, rather than a compressed database file, mumps.dat.gz.

• The symbolic link parent points to the release directory from which this integration directory was created. A release directory does not have a symbolic link called parent. (The recommended way for a script to determine whether a directory is a release directory or an integration / development directory is to test for the existence of parent.)

• A script run which invokes and runs VistA in that integration environment. Note that a VistA release may physically include a run script, but it will not work because it is not possible to directly run VistA from a release directory.

• When the install script is executed, it will create a development directory.

• The normal protection on all directories and files in an integration directory should be read-only. When promoting code to the integration directory from a development directory, or to install externally generated patches in the p subdirectory, needed directories should be made read-write, sources copied in, object file(s) generated, and directories again made read-only.

VistA Development Directories

A development directory is like an integration directory, except that it does not normally have its own database file or global directory. However, as discussed above, it may be entirely appropriate for a development directory to have database files and global directories.

Indeed, since development directories are hierarchical – a development directory is an integration directory for development directories below it, and itself contributes to an integration directory above it, an integration directory is really nothing more than a development directory that does not itself contribute to an integration directory above it, but instead is used to create a release.

Note that although a GT.M database file can be opened by only one GT.M version at a time, it is quite easy to create a directory structure where Release A, the integration area, and each development area use different versions of GT.M. In the most common case, where the integration and development areas use a different version of GT.M from Release A, the only additional configuration requirement is to create a directory in the integration area for object files generated from the source files in Release A with the version of GT.M used for integration and development of Release B