Methodology Activities

In this activity, you will examine the package creation and extraction procedures and expose those steps via a Package object.

1.1 Analyze

There are two main procedures to understand for a given Package object: creating the package archive with the correct content and extracting that content to the desired location. Each of these steps is supported by Package type commands. To supply the needed information to drive the Package object answer these questions:

• How is the package archive obtained now? What build process creates it or is it a 3rd party dependency? If an in-house build process produces the package, identify the build script and where it runs. You will need to know that in Activity #2.
• What format is the package archive? Is it a compressed tar file or format an operating system facility handles (eg, rpm, deb, etc)? The base automation library that comes out of the box includes a number of common package formats which means you can use one directly. If not, you will need to subtype Package and implement two commands: create and extract.
• What location does the package get installed to and does it vary based on environment? The answer to this question is used by the installroot parameter to several of the Package object commands. If it is a fixed location, then just note it. If it varies, determine if there is a pattern to the path name. If there is no pattern, the variation can still be handled but it will be driven by the Service object discussed in Activity#3.
• What is the naming convention (if any) that can be used to identify the package archive file on the file system? Depending on the format, a package archive might be named with a base name, version and extension.
• Are there any special steps that should immediately precede or follow the extraction step? Sometimes it is necessary to incorporate steps such as creating a directory, fixing a permission or checking pre- and post-conditions. If so, note them and you will have to subtype Package (or another existing subtype) and implement prepare or finish accordingly.

With the answers to these questions in hand you have the needed information to proceed to the design step.

1.2 Design

The ControlTier base types include a number of package types (e.g., war, zip, tgz, rpm, jar, etc.) that can be used out of the box. If the package archive at hand is one of these formats, start by choosing the appropriate type.

If you must manage a new format, subtype Package and implement create and extract. The initial implementation of create might take a directory of content and produce an archive containing it. Likewise, the extract command's initial implementation takes an archive file and extracts it to the specified installroot.

You can choose to subtype existing Package types for no other reason but to distinguish them as your own. For example, you may build a WAR for your bookstore webapp so you might like to call this type of package, BookstoreWar, making it a subtype of the war type. In this case, there is no need to override any command.

Finally, you may not need to worry about the create command at all since the build process already produces it.

1.3 Implement

If you are subtyping a Package type, implement the create and extract commands.

Once the package archive exists, upload it either via Workbench or use the upload command. Then, use the register command to create an object to record its location and other pertinent installation data.

You have reached the first goal when the package archive resides in the repository and referenced by a new Package object.

The second goal is to install the package from the repository. That process can be handled by the install command. The install command takes parameters that should result in a number of subordinate commands to fire: prepare, get, extract and finish. If you are using an existing Package type you are done. If you are implementing a new Package subtype, ensure your extract command executed correctly, placing the extracted files into installroot.

1.4 Test

1.5 Sustain

N/A

With the activities complete for the package sub-process, you are ready to proceed to the deployment sub-process in activity #2.

In this activity, you will examine the existing build and packaging procedure and expose those steps via a Builder object.

The ProjectBuilder tutorial covers this topic in the Interfacing a build page.

2.1 Analyze

There are three main procedures that comprise the Builder's Build workflow command: checking out source code, running a build tool, loading the packaged build artifacts to the repository. Each of these steps is supported by a Builder command that uses a standard set of attributes to parameterize each step. To supply the needed values to drive the Builder commands begin by answering these questions:

• How is the source code accessed from the SCM repo? Are the source files in Subversion, CVS or something else? What module names and tagging/labeling/branching scheme are used? In the end, you'll want to know how to run the appropriate checkout command.
• What build tool is used? If it is a java project, is it Ant or Maven? If it is C-based, is it make or autoconf, etc.? Ultimately, you want to know what command and command parameters are necessary to execute a build on the checked out source code.
• How are builds versioned? Some groups version their build by a typical numbering scheme like major.minor.patch-level while others use some form of timestamp (e.g., yyyymmdd), and yet others by a release tag name. In the end, you'll want to establish a "buildstamp" a common version identifier that lets you find all the build artifacts from a given build execution.
• How are the build artifacts packaged? Java projects typically yield packaged artifacts like JAR, WAR, EAR files. Other builds yield executables and architecture specific binaries that may be packaged together in TAR, ZIP or operating specific package formats like RPMs. In any case, you need to know how to identify the target files that the build creates and in which directory to find them.

With the answers to these questions in hand you have the needed information to proceed to the design step.

2.2 Design

As noted earlier, the general build process is embodied by the Builder type. If you use CVS or Subversion, then there you should be covered by the standard Builder command, scmCheckout, to check out your source code. That really just leaves how to manage the invocation of your build tool. If you use Ant, then use AntBuilder. If you use Maven, use MavenBuilder. Both these Builder subtypes, override the runBuildscript command to invoke either tool with the needed parameters. If you use a different build tool, then you will subtype Builder and override runBuildscript.

All the parameters are maintained via an object model that allows default overriding. Take a look at the Builder attributes section for the dozen-odd settings that will configure your Builder.

2.3 Implement

If you are creating your own Builder subtype, then your primary command to implement is the overridden runBuildscript command. This will be a wrapper around the underlying build tool, and you will need to pass the necessary parameters and/or configuration to it.

Primarily, the implementation step is to review and possibly override the key attributes that are used by the individual Builder commands that are called in succession by the Build workflow command. The table below relates each command to its set of attributes:

If you are a workbench user you will create Setting objects and connect them as child dependencies to the Builder object. If you are a ProjectBuilder user, you will create an XML file that defines the objects and the dependency relationships.

2.4 Test

Testing a Builder is a simple procedure. Just run each of the commands that Build would run on its own and check the result of each one.

If each of the three commands completed correctly, then run the Build workflow command to see the whole build process in action. Note, all of these Builder commands take a -buildstamp <> argument that utilizes your build versioning convention discussed earlier.

2.5 Sustain

There are two aspects to sustaining the Builder process over time.

1. Object model maintenance. If you used Workbench to create the initial model, you might elect to take that information and maintain it in an XML file that can be used by ProjectBuilder.
2. Repository maintenance. After a while, obsolete packages will accumulate in the repository so you will want to clean them out. Builder provides a utility command, repoFind that takes parameters and can purge the repository. You can run this command from Job Center or a system facility like cron.

With the activities complete for the build sub-process, you are ready to proceed to the deployment sub-process in activity #3.

In this activity, you will examine the existing package deployment and server restart procedure and expose those steps via a Service object.

The ProjectBuilder tutorial covers this topic in the Managing deployment page.

3.1 Analyze

There are two primary concerns during the deployment process. First, knowledge about which packages to install (their version, location in the repository, etc.). Second, how to coordinate the package installation with the running application process. Each of these steps is supported by a Service command. To get started using a Service one must answer the following questions:

• What are the key directories? Find out where the packages will be extracted to and where the server binaries reside.
• What kind of package or packages are needed for a complete deployment? These will be the packages that were produced by the Builder in activity #2. Optionally, you might also deploy third party software that will be defined as its own Package object.
• Are there any customizations that must occur after the package(s) installation completes? Sometimes it is necessary to execute post-installation steps such as modifying configuration or setting permissions, etc.
• Does the application process need to be restarted and if so, when? Some applications support the "hot deploy" option while others must require the application process to be restarted. If there is a restart you should choose when the process will be restarted with respect to the package installation step.
• How do you know if the service is up or down and how do you stop it and start it? Checking if a service is up or down can be done by checking the process table, network port, or running a utility. Likewise, how a process is started and stopped will differ from application to application. In the end, you will invoke an executable to start it and communicate to the operating system how to stop it.

With the answers to these questions in hand you have the needed information to proceed to the design step.

3.2 Design

As noted earlier, the general deployment process is embodied by the Service type. The Service type provides a number of commands and workflow commands that provide a coordinated deploy and restart.

The workflow commands provided by Service are typically used as-is. These workflow commands create the general process but rely on hook commands that can be overridden to manage the runtime state of the application process. Of course, how a process is restarted varies widely depending on the application runtime requirements. While we at ControlTier do contribute new Service subtypes as times goes on, you may need to create your own Service subtype to meet your requirements. Luckily, it is a simple task to design and implement a Service as there are only four commands to worry about: assertServiceIsUp, assertServiceIsDown, startService and stopService.

Typically, these four commands are the only design consideration. If you find that your commands need parameters of their own, you might consider defining attributes for your subtype to contain them.

3.3 Implement

Implementation is done in these parts:

1. Choose or create a Service type
2. Create a Service object
3. Configure a set of package dependencies

3.3.1 Choose or create a Service type

Primarily, the implementation step includes: the review and possibly override the key attributes that are used by the individual Service commands that are called in succession by the Stop, Start, Status workflow commands. The table below relates each command to its set of attributes:

3.3.2 Create a Service object

Create an instance of your chosen Service subtype, giving it a name and setting the basedir and install-root values.

3.3.3 Configure a set of package dependencies

Identify the desired Package objects and make them child dependencies to Service object created in step 3.3.2.

If you are a workbench user you will create Setting objects and connect them as child dependencies to the Builder object. If you are a ProjectBuilder user, you will create an XML file that defines the objects and the dependency relationships.

3.4 Test

Testing a Service is a simple procedure. Just run each of the commands that Update would run on its own and check the result of each one.

If each of the three commands completed correctly, then run the Update workflow command to see the whole deployment process in action.

3.5 Sustain

The following aspects to sustaining the Service process over time include:

1. Object model maintenance. If you used Workbench to create the initial model, you might elect to take that information and maintain it in an XML file that can be used by ProjectBuilder.

With the activities complete for the deployment sub-process, you are ready to proceed to the deployment sub-process in activity #3.

In this activity, you will understand the coordination needs of build and deployment and expose these steps via the BuildAndUpdate workflow command. Along the way you will create and configure Site and Updater objects. This activity covers the "logical" process layer where you begin to coordinate the sub-processes in the "physical" layer.

The ProjectBuilder tutorial covers this in the Coordinating build and deployment page.

4.1 Analyze

There are three levels of coordination at work here. Firstly, there is coordinating all the Builders, invoking them to run their Build workflows. Secondly, there is coordinating all the Service objects to run their Update workflows. Finally, there is coordinating the two previous steps. Each of these steps are managed via the Updater and Site objects. To make use of these types begin by answering these questions:

• How will package version dependencies be coordinated across components ? The normal convention is to use a common buildstamp value to relate packages across a set of Builders for a single release.
• In what order will the Service objects be updated ? A multi-tier application is usually comprised of various layers of components. Higher layers normally are restarted after lower layers.
• What kind of logic is needed to update the set of interdependent application components ? Make a note if there are unique interleaving requirements to restarting application components.
• During the process are there other process management tools to tie in ? Sometimes there are ancillary facilities that must be coordinated during the process ( e.g., security, load balancing, monitoring, etc.).

With the answers to these questions in hand you have the needed information to proceed to the design step.

4.2 Design

Generally, the generic Site and Updater types are useful as they are and implement a basic coordinated build and deployment process. One can govern the order of Updates across Service objects by making use of the rank attribute. The Site object's commands use that value to sort in ascending and descending order depending on the command.

Of course, how a given set of application services that tie together to form an integrated service are managed during a broad deployment can vary. For this reason, you may find it necessary to eventually create a Site subtype that overrides the Update workflow command. The role of Site's Update workflow command is to coordinate deployment activity across an integrated stack of application service deployments. So, if you need to interleave application restarts in between software package installations in unique ways, you will likely need to:

• create specific commands in your Site subtype that manage tier-specific steps
• override the Update command and define a sequence of commands that presumably uses the commands defined above.

This topic is briefly treated in the Site reference documentation.

A final rule of thumb though is this: don't create Site or Updater subtypes initially, if you can avoid it. It is more important to have an uncomplicated end-to-end process to ensure the basic low-level steps exist and work correctly (i.e., actions in the Service and Builder level). You can always subtype to come up with your own coordination logic later.

4.3 Implement

If you are creating your own Site subtype, override the Update workflow command and implement the necessary supporting commands that will comprise its command sequence.

Once the Updater and Site types are chosen, implementation is done in these parts:

1. Create Site object
2. Create Updater object

4.3.1 Create Site object

Create an instance of your chosen Site type via Workbench or ProjectBuilder. Set the Site object's child dependencies to include the desired Site objects.

4.3.2 Create Updater object

Create an instance of your chosen Site type via Workbench or ProjectBuilder. Set the Updater object's child dependencies to include the Site object and desired Builder objects.

In between the Build and Update phases, Updater runs a command called runChangeDependencies to dynamically configure the Service object dependencies to use the new set of packages generated in the Build phase. This command uses an optional attribute, defaultpackagename, to pick among multiple packages of the same version to resolve ambiguity.

4.4 Test

Testing the end-to-end process is a simple procedure of running through each command on its own and check the result of each one.

If each of the three commands completed correctly, then run the BuildAndUpdate workflow command to see the whole deployment process in action.

4.5 Sustain

The following aspects to sustaining the Service process over time include:

1. Object model maintenance. If you used Workbench to create the initial model, you might elect to take that information and maintain it in an XML file that can be used by ProjectBuilder.

With the activities complete for the coordinated build and deploy sub-process, you are ready to proceed to define and run jobs in activity #4.

In this activity, you will expose the BuildAndUpdate workflow command to the Job Center tool to automate running it at timed intervals.

Because Job Center is a graphical tool that makes it simple to configure and run commands, the Analyze and Design steps are brief.

5.1 Analyze

There are really just a few considerations that must be made when exposing a command as a job in Job Center:

• What commands need to be run? Typically, you will run the Updater BuildAndUpdate command but you may also wish to run others like Site Status.
• Do the commands run on-demand or at scheduled intervals? Some commands you may wish to expose to Job Center operators that they can run any time. Others you will want to run automatically on a scheduled basis.
• What additional information do you want to communicate to the Job Center operator ? Each job entry, includes a section where you can type HTML to include anything useful the Job Center operator may benefit from.

5.2 Design

There is not much design consideration, except perhaps coming up with a naming convention for job entries. Job Center operators may be unfamiliar with the underlying type/object model so it might be better to express job names in plain english using business terms.

5.3 Implement

Implementing a job is a simple process of pushing the "Create new job ..." button in Job Center. In the new job form input the following:

1. Pick object and command (including needed options)
2. Name the job and give it a description. Include additional information if any.
3. If it is a scheduled job, configure the times and days

5.4 Test

Testing the job is a simple procedure of running the job from Job Center UI. It is easiest to run the command "on-demand" but if it is a schedule command you can temporarily edit the schedule to run it the next minute so you don't have to wait long to see it run.

5.5 Sustain

There is not much maintenance to perform for Job Center. The main consideration is thinking about what kind of reports you might like to share. Select a filter and report format and generate a report.