Upgrade TDM Portal in Docker
From 4.7, TDM Portal is available as a collection of Docker images. You can use TDM Portal for Docker as stand-alone software, or you can use TDM Portal for Docker with the individual TDM components (Datamaker, Fast Data Masker, GT Subset etc).
Test Data Manager4.7, TDM Portal is available as a collection of Docker images. You can use TDM Portal for Docker as stand-alone software, or you can use TDM Portal for Docker with the individual TDM components (Datamaker, Fast Data Masker, GT Subset etc).
To upgrade to a new version of TDM Portal for Docker, it is only necessary to download the Docker images for that version, and use the
docker-compose.ymlfile appropriate to that version.
To use a full installation of Test Data Manager and an instance of TDM Portal in Docker with the same data, it is necessary that both versions refer to the same
Notes on Upgrade of gtrep
- From TDM 4.7, the TDM Portal upgrade procedure upgrades thegtreprepository. In previous versions, Datamaker manages the gtrep upgrade process. For more information, see Upgrade Product Components.
- From TDM 4.8, the first time TDM Portal (in Windows or Docker) runs, it upgrades thegtreprepository if gtrep already exists, or installs gtrep if it does not already exist.
- From TDM 4.8, Datamaker no longer upgradesgtrep.
If you upgrade the
gtreprepository, you must upgrade all other product components to the same version as the software that upgraded gtrep.
Upgrade paths for TDM Portal in Docker
Test Data Manager4.8, there are two possible paths to upgrade TDM Portal in Docker.
- You can still upgrade TDM Portal in Windows to v4.8.
Upgrade from TDM Portal in Windows to TDM Portal in Docker.
This upgrade path applies if you currently use TDM Portal in Windows (either with or without the individual Test Data Manager product components).
Follow these steps:
- (Optional)Upgrade Test Data Manager product components (Datamaker, GT Subset etc) to the target version.This applies to users who use TDM Portal in conjunction with TDM product components that the GT Server installer installs. The versionmustcorrespond with the version of TDM Portal for Docker that you want to run.The first execution of TDM Portal upgrades thegtreprepository. The TDM components will not function, until you upgrade the repository.For more information on the upgrade process, see Upgrade Product Components.
- Migrate OrientDB databases from TDM Portal for WindowsIt is necessary to transfer the databases that OrientDB uses, from their location in a Windows installation of TDM Portal (C:\programdata\CA\CA Test Data Manager Portal\orientdb\databasesby default), to the OrientDB container.TDM Portal uses the following OrientDB Databases:- ReservationDB- ModelingDB- StagingDBNot all of these Databases may exist on your system. The creation of these databases is dependent upon the usage of certain features.Follow these steps:
- Open a command prompt (cmd.exe) and navigate to the\orientdb\bindirectory in your TDM Portal installation folder (C:\Program Files\CA\CA Test Data Manager Portal\orientdb\binby default).
- Start the OrientDB console, with the commandorientdb.bat.It is necessary to expose port2424of the OrientDB container, to connect to it from the OrientDB console. To do this, add the following line to your docker-compose file:ports:- '2424:2424'
- For each of the databases that are present on your system, perform the following steps:
- Export the OrientDB database to the machine's local hard drive.
- To connect to the local OrientDB databases, type the following in the OrientDB console:connect remote:localhost/<DatabaseName> root <rootpassword>For example:connect remote:localhost/ReservationDB root myrootpwd
- To export a database to a location where you have write access, type the following in the OrientDB console:export database <local-path>For example:export database c:/users/username/ReservationDBThis command exports the contents of the database to the filec:/users/username/ReservationDB.json.gz
- Import the exported databases to the OrientDB Docker container.
- To connect to each database on the OrientDB container, type the following in the OrientDB console:connect remote:<dockerhost>/<DatabaseName> root <rootpassword>For example:connect remote:mydockerhost.com/ReservationDB root myrootpwd
- To import each database you exported in step i, type the following in the OrientDB console:import database <local-path-to-file.gz>For example:import database c:/users/username/ReservationDB.json.gzThis command imports the database to your OrientDB container. Repeat this process for all databases exported in step i.Your OrientDB databases are now available for your OrientDB Docker container to use.
- Download, setup and run the TDM Portal for Docker containers.You can download these images (or download Dockerfiles and source binaries to build these images yourself) from the files available from https://support.ca.com/. When you execute the docker-compose file that contains the TDMWeb container, TDM Portal upgrades the gtrep repository.For more information, see Install TDM Portal for Docker.
Upgrade from version 4.7 of TDM Portal in Docker, to version 4.8 of TDM Portal in Docker
This upgrade path applies if you currently use TDM Portal in Docker 4.7 (either with or without the individual Test Data Manager product components), and you want to upgrade to TDM Portal in Docker 4.8. The following process backs up the volumes that the TDM Portal and TDM Portal OrientDB containers need to operate.
This process requires the use of
restore-volumes-from-4.7-backup.sh. These are included in both the file packages that contain Docker images - for more information, see File Packages available.
Follow these steps:
- Identify TDM Portal containers on your Docker network.Do this with thedocker pscommand.$ docker psThis command lists all containers active in docker.Sample response:CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES9fc3226587b2 tdm.packages.ca.com/tdm/tdmweb:184.108.40.206 "/opt/tdm/bin/tdmweb…" 9 seconds ago Up 7 seconds 0.0.0.0:8080->8080/tcp, 0.0.0.0:8443->8443/tcp upgrade_tdmweb_10f1f4c0769bb tdm.packages.ca.com/tdm/orientdb:2.2.33 "/opt/tdm/bin/orient…" 10 seconds ago Up 8 seconds 2424/tcp, 2480/tcp upgrade_orientdb_1You may have other TDM Portal containers active, but you do not need to backup or restore these containers.If you execute yourdocker-compose.ymlfile from within a directory, Docker containers have the prefix "<directoryName>_" (e.g. "upgrade_" in the sample response above).You can see a list of all Docker volumes active on your Docker network, with the following command:$ docker volume lsIf you used adocker-compose*.ymlfile from TDM Portal 4.7, and did not create named volumes, the response to the above command is something similar to this:DRIVER VOLUME NAMElocal 1ca35ea30f627c3641ae04dc213295182637b31cae2438756b5da919848b392elocal 2a301be43b1ba2a797f0bfdb788e0403c9d615580365cfb7ff9e16285ce7bc52... ...
- Stop active TDM Portal Docker containersStop containers with the following command:$ docker-compose stopThis stops all active containers in your Docker network.Sample response:Stopping upgrade_tdmweb_1 ... doneStopping upgrade_orientdb_1 ... done
- Backup your TDM Portal containers' volumes.The bash scriptbackup-volumes-4.7.shis available for this purpose.Syntax:$ backup-volumes-4.7.sh <orientdb_container> <tdmweb_container> [file]Where
Example:$ backup-volumes-4.7.sh upgrade_orientdb_1 upgrade_tdmweb_1This creates the archivetdm-volumes-backup-4.7.tar.gz. This archive contains all volumes for the OrientDB and TDMweb containers.Sample response:INFO: 'tdm-volumes-backup-4.7.tar.gz' archive created
- <orientdb_container> / <tdmweb_container>Specifies the name of each of these containers (as in the response to thedocker pscommand).
- (Optional)fileDefines the relative or absolute path to a file, in which to store the output. Default:tdm-volumes-backup-4.7.tar.gz
- Create volumes on your Docker networkFor therestore-volumes-from-4.7-backup.shscript to function, it is necessary to create the following volumes on your Docker network:
The volumes you create must have the exact names above, for the script in Step 5 to function.You can create each volume with the following command (other methods are also possible):$ docker volume create <volume_name>At this stage, you may wish to add other attributes to your Docker volumes, for example with the--driverand--opt <option>flags.You are now ready to restore your backups into these new volumes.
- Restore your TDM volumesThe scriptrestore-volumes-from-4.7-backup.shis available for this purpose.Syntax:$ restore-volumes-from-4.7-backup.sh <archive>Where
Example:$ restore-volumes-from-4.7-backup.sh tdm-volumes-backup-4.7.tar.gzThis restores the volumes that you backed up in Step 3, to the volumes you created on your Docker network.
- archiveSpecifies the name of the tar.gz file, created by the backup-volumes-4.7.sh script.
- Rename your new TDM volumes (if necessary)If your container names from Step 1 include a prefix (e.g. "upgrade_"), it is necessary to rename the TDM volumes you created in Step 4, to include this prefix.This command removes the existing container<old_volume>, and recreates it with the new name<new_volume>:$ docker run --rm -it -v <old_volume>:/from -v <new_volume>:/to busybox ash -c "cd /from ; cp -av . /to"
- Modify your docker-compose file, to make the new volumes accessible to the TDM Portal and OrientDB containersFor each volume that you created (and renamed if necessary), add the attribute "external: true".The end ofdocker-compose.ymlshould look like this:volumes:upgrade_orientdb_backup:external: trueupgrade_orientdb_config:external: trueupgrade_orientdb_databases:external: trueupgrade_tdmweb_logs:external: trueupgrade_tdmweb_storage:external: trueupgrade_tdmweb_fdmconfig:external: trueThis prevents docker-compose from creating new volumes. Instead, it refers to your existing volumes.
You can now execute a TDM Portal 4.8 docker-compose.yml file, and TDMWeb and OrientDB services will use your data persisted from TDM Portal 4.7.