hsync Directory Synchronization

The hsync command updates a file system directory (reference directory) so it reflects the contents of a view path. The hsync command checks out the latest versions of items in the view and deletes files in the reference directory that no longer belong.
cahscm101
The hsync command updates a file system directory (reference directory) so it reflects the contents of a
CA Harvest SCM
view path. The hsync command checks out the latest versions of items in the
CA Harvest SCM
view and deletes files in the reference directory that no longer belong.
CA Harvest SCM
supports full view and active view approaches. The inventory of items that is checked out to each reference directory depends which approach you use. The approaches are described in the following sections.
2
The following scenario illustrates the basic process of refreshing reference directories. The diagram shows a
CA Harvest SCM
lifecycle with three states and their corresponding working views: DEV, TEST, and PROD. Promote and demote processes move packages from one state to the next state. Reference directories external to
CA Harvest SCM
keep copies of the files that
CA Harvest SCM
manages. You can have a single reference directory for each view, or multiple reference directories associated with each view. You can use multiple reference directories on different operating systems that support different build targets as shown in the diagram.
Multiple Reference Directories on Different Operating Systems
Multiple Reference Directories on Different Operating Systems
Full View Approach
In a full view approach, the latest trunk versions of all items in the associated view are checked out to the reference directory. The directory inventory can represent an entire repository tree, the recursive contents of a certain repository view path, or multiple repositories. When a process in
CA Harvest SCM
triggers the check-out of items, only one reference directory per host computer is processed. The hsync command selects the latest versions from the view and copies those versions to the target directory.
Active View Approach
In an active view approach,
CA Harvest SCM
manages the synchronization of source code versions based on package promotion and demotion. The hsync command locates files across multiple directories, and checks out the latest version of only those items that are associated with active packages based on their state location. Because files move from one reference directory to another,
CA Harvest SCM
must process multiple reference directories -- the directories that correspond to the initial and destination states -- when promotes and demotes take place.
hsync Item Selection Options
The hsync command creates an inventory of items and paths based on criteria specified by an item selection option. The item selection options are as follows:
  • All Items in View
    Selects all items and paths under a given view path. You can use this option to initialize or refresh a reference directory at any time. Use this option in a stand-alone UDP or script.
  • Item List
    Selects items and paths with names in the item list. Use this option after a
    CA Harvest SCM
    process uses a specific set of items.
  • Package List
    Selects the items and paths in packages with names in the package list. The packages do not need to be located in the state where hsync is executed. Use this option after a
    CA Harvest SCM
    process uses specific set of packages.
  • All Packages in State
    Selects items and paths in packages that are located in the state where hsync is executing. Use this option as an efficient alternative to the package list when the set of packages is large and all the packages are in the given state.
  • All Packages in View
    Selects items and paths in packages that are located in any state that shares a view with the packages' current state. This option is an alternative to use when the packages are in states that share a view.
Item selection searches are always recursive and start at a given item path.
Renaming an item or item path creates a version of the same item or item path. The inventory includes the names and paths of all versions of the selected items. Therefore, if an item has been renamed in the current project, both the old and the new item name are in the inventory. If an item has been moved in the current project, both the old and the new path of the item are in the inventory.
To the item list will be added items that are not covered by the item selection option, but have a version with the same name and path as an item that is already in the inventory.
Check-Out Version Selection
From the selected check-out items, hsync uses the No-tag filter to select the latest versions in View and other version selection filters that depend on reference directory type and the branch or trunk options as follows:
  • When the reference directory type is Full view, hsync selects the latest versions of all the items.
  • When the reference directory type is Active view, hsync selects the latest versions of all the items that are active.
  • When the Trunk option is enabled, hsync selects the latest versions on the trunk.
  • When the Branch option is enabled, hsync selects the latest versions on the branches based on timestamp.
  • When the Trunk and Branch option is enabled, hsync selects the latest version whether it is on a trunk or on a branch based on timestamp.
File Deletion
If the purge option is not enabled, hsync deletes files in the reference directory that meet the following conditions:
  • The check-out version list does not have the version with the same name and relative path as the file.
  • The file is read-only.
If an item is selected, but does not have a version in the check-out version list and a file exists with the same name in the reference directory and it has read-only permission, the file is deleted. Like check-out, hsync reports an error if the file has write access. For each item path in the inventory, but not in the check-out version list, if the directory has no files other than a signature file, the directory is deleted. After files are deleted from the reference directory, check-out is executed using the previously selected version list.
Files that do not correspond to items in
CA Harvest SCM
are not touched; therefore, the directory may contain files from some other source.
If a new item is demoted so that no version of the item remains in the current view, the hsync -iv option does not delete the corresponding file in the directory because the item is not in its inventory. For scenarios like this one, use the purge option.
When Subdirectories Are Deleted
After files are deleted, hsync determines the subdirectories to delete. The hsync command deletes subdirectories in the reference directory that meet the following conditions:
  • If the purge option is not enabled, the inventory includes a path that has the same name and relative path as the subdirectory.
  • The check-out version list does not have the version with the same name and relative path as the subdirectory.
  • The subdirectory is empty except for a signature file.
Purge Option and Exclusion Lists
You can use the hsync -iv (All Items in View) option to perform a refresh using a stand-alone UDP. Use the purge option (-purge) with -iv when a possibility exists that all versions of an item have disappeared from view after the last execution of hsync. When you enable the purge option, item inventory is not used to decide if a file or subdirectory should be deleted.
With the purge option, you can use exclusion lists to specify files and directories that should not be deleted. The hsync command provides two ways to specify file names:
  • -excl list of file or subdirectory names
  • -excls file name pattern
A file is deleted if the following conditions are met:
  • The file has not been excluded by either of the exclusion options.
  • The check-out version list does not have a version with the same name and relative path as the file.
  • The file is read-only.
A subdirectory is deleted if the following conditions are met:
  • The subdirectory has not been excluded by the -excl option.
  • The check-out version list does not have a version with the same name and relative path as the subdirectory.
  • The subdirectory is empty except for a signature file.
For details about the syntax for the exclusion list options, see the
Reference section
.
Examples: -excl option
To recursively synchronize all files with the .c file extension, use the following command:
hsync [options] -excl "*.c"
To synchronize *.c and *.exe and recursively synchronize the files one.c, two.c, and three.c, use the following command:
hsync file.txt *.c ... -excl one.c two.c three.c [option] *.exe
PurgeOpts and PurgeArg Tokens
Use the following format to specify hsync purge options (-purge, -excl, excls) using the PurgeOpts or PurgeArg tokens:
PurgeOpts: {command 1; command 2;…} PurgeArg: {command 1; command 2;…}
You specify the PurgeOpts and PurgeArg tokens in the same way that you specify the PreCmd and PostCmd, and the order of the PurgeOpts, PurgeArg, PreCmd, or PostCmd tokens is not significant; you can specify them in any order at the end of the line. Providing the purge options in a file allows for easier reuse of common exclusion lists, and keeps the hrefresh configuration file less cluttered.
Hsync does not allow the purge options to be used when package list or item list modes (-pl, -ps, -pv, and - il) are used. Hrefresh automatically excludes or applies the provided purge options based on the context. A certain record in the hrefresh configuration file can be used in an incremental refresh or a full view (-iv mode) refresh. Hrefresh supplies the purge options when they are legal or excludes them when they are not. The decision is logged in the job log file.
You can use the following methods to specify the PurgeOpts or PurgeArg tokens:
  • In the hrefresh configuration file record
    For example:
    Project1,Test,,\,c:\ref\p1\test,FULL,user03s,7001,harvest.dfo,user03.dfo, PurgeOpts: {-purge none -excl aaa bbb ccc}, PreCmds: {echoargs}
  • In an external argument file that an argument file name in the hrefresh configuration file record specifies
    For example:
    Project1,Release,,\,c:\ref\p1\rel ,FULL,user03s,7001,harvest.dfo,user03.dfo, PurgeArg: {purge.arg}
    In this example, purge.arg exists in the hrefresh home directory and contains any of the purge options. This argument file can be multiline, making it easier to read and maintain than traditional harvest argument files. For example, the file can include the following:
    -purge all
    -excl aaa bbb ccc
    -excls xxx yyy zzz
Synchronous/Asynchronous Execution Mode
The option, -execmode, changes the way hrefresh runs with respect to synchronous as opposed to asynchronous behavior.
  • In asynchronous mode, hrefresh initiates the refresh jobs and does not wait for any jobs to finish. Hrefresh does not detect nor report the exit status of any refresh job. This behavior is the default.
  • In synchronous mode, hrefresh initiates the refresh jobs and waits for all jobs to finish. Upon completion of the longest running job, a report displays the exit codes of all the refresh jobs. Additionally, if any refresh job exits with a failure (nonzero exit code), hrefresh also exits with a failure.
The - execmode option is optional and you can specify it in the following ways:
  • In the hrefresh.arg argument file
    To set the default execution mode for all hrefresh executions, use - execmode synchronous or - execmode asynchronous as an entry in hrefresh.arg. If hrefresh.arg does not contain this option, asynchronous mode is assumed.
  • On the hrefresh command line
    To set the default execution mode for a specific hrefresh execution, use - execmode=synchronous or - execmode=asynchronous as a command-line option. The command-line - execmode value has precedence over what is specified in hrefresh.arg.
Use Case-hsync Stand-Alone Execution
This setup example shows how you can use hsync to support incremental nightly builds.
The build setup is as follows:
  • All source code is in one reference directory named /Build/Src. The directory contains files that were checked out from
    CA Harvest SCM
    .
  • The
    CA Harvest SCM
    user has an encrypted username/password file (hsync.dfo) that executes hsync.
  • The Build state in
    CA Harvest SCM
    has a check-out process that allows Synchronize mode and does not preserve the original file time stamp. The hsync user has execute access to the check-out process.
    Check-out uses the following options to select versions: Latest in View, No Tag, Trunk Only, and Recursive. These versions are checked out in Synchronize mode.
The nightly build script executes the following command:
hsync -b Broker -en Project -st Build -vp \Repository\Src -cp “/Build/Src” -eh “/Build/Cred/hsync.dfo”
hsync refreshes the /Build/Src reference directory with the latest items in the view path \Repository\Src. If removed items entered the Build state's view that day, the files corresponding to those items are deleted. If a renamed item entered the view, the file with the old item name is deleted, and the latest version of the item with the new name is checked out.
hsync for Promotional File Management Example
This example demonstrates how you can use hsync for promotional file management (active view approach). This example uses the following setup:
  • The reference directory /Build/TEST/Src holds the latest version of active items in the TEST state working view. QA evaluates the changes in the TEST state before the changes are promoted for a final build out of the PROD state.
  • The PROD state has a working view and follows TEST in the project lifecycle. /Build/PROD/Src holds all the source files for release builds, and contains the latest version of all items in the PROD state working view.
  • /Build/TEST/Src and /Build/PROD/Src are in the search path that the TEST build procedure uses. When the build tool does not find a source file in /Build/TEST/Src, it uses the file in /Build/PROD/Src. The reference directories are not located on the same computer as the
    CA Harvest SCM
    server installation. Therefore, a remote file agent accesses the reference directories.
The following diagram shows the lifecycle for this example:
Use hsync for promotional file management (active view approach)
Use hsync for promotional file management (active view approach)
Promotional file management occurs as follows:
  1. A server UDP is post-linked to the promote process that promotes packages into TEST. This UDP uses the following command:
    hsync -b [broker] -en “[project]” -st TEST -vp \Repository\Src -cp /Build/TEST/Src -pl [“package”] -av -rm buildmachine  - rport agentport -eh /CAproduct/Cred/hsync.dfo -er /CAproduct/Cred/bldmach.dfo
  2. After packages are successfully promoted to TEST, the hsync command executes. hsync examines all items in the list of promoted packages. If there are removed or renamed items in the packages, hsync deletes corresponding files in /Build/TEST/Src. Latest versions of inventory items are checked out in Synchronize mode to /Build/TEST/Src when they are active in the TEST view, on the trunk, and have no tag.
  3. The Demote process in the TEST state has a post-linked server UDP with the same hsync command. The package list tells hsync what items must be processed. The demoted packages are not in TEST when hsync executes.
    • If an item is not in any package in TEST, it is no longer active; the corresponding file is deleted from the TEST reference directory.
    • If a demoted package contains a removed item, the item appears in TEST again and is checked out to the reference directory.
    • If a renamed item was demoted, the file with the new name is deleted from the reference directory and the previous version of the item with the old name is checked out.
  4. The Promote process in TEST that promotes packages to PROD has two post-linked UDPs. One UDP uses hsync to update the TEST reference directory; the other UDP uses hsync to update the PROD reference directory. After the Promote process executes successfully, items that were active in TEST are now active in PROD. The corresponding files must move from the TEST reference directory to the PROD reference directory.
    The hsync command to update the TEST reference directory is the same command as described previously:
    hsync -b [broker] -en “[project]” -st TEST -vp \Repository\Src -cp /Build/TEST/Src -pl [“package”] -av -rm buildmachine -rport 7001 -eh /CAproduct/Cred/hsync.dfo -er /CAproduct/Cred/bldmach.dfo
  5. After all packages are promoted from TEST to PROD, the items in the packages are still in the TEST working view, but they are no longer active. Therefore, files corresponding to those items are deleted from the TEST reference directory.
    The following hsync command updates the PROD reference directory:
    hsync -b [broker] -en “[project]” -st PROD -vp \Repository\Src -cp /Build/PROD/Src -pl [“package”] -rm buildmachine -rport 7001 -eh /CAproduct/Cred/hsync.dfo -er /CAproduct/Cred/bldmach.dfo
    You can use the default reference directory type, -fv, instead of -av because the PROD reference directory holds all source files in the PROD view.