Work with Transformation Maps

As a test data engineer, you manage databases and file transformations in . Transformation maps are used to scramble data, condition data, age data, or as part of the data multiplier. You can create new transformation maps, copy existing maps to new maps or a new release, and can compare maps.
tdm44
As a test data engineer, you manage databases and file transformations in
Test Data Manager
. Transformation maps are used to scramble data, condition data, age data, or as part of the data multiplier. You can create new transformation maps, copy existing maps to new maps or a new release, and can compare maps.
Use this scenario to guide you through the process.
Create a Transformation Map
Use transformation maps to scramble, condition, or age data. To define data for use in databases or file transformations, create transformation maps.
Follow these steps:
  1. Select
    Projects
    ,
    Transformation Maps
    , and click the
    Plus
    button.
  2. The transformation map dialog appears.
  3. To create a map, click the
    Plus
    button and a new map is created.
  4. To specify the map name, double-click
    New Map
    .
  5. (Optional) To change the DBMS type, click the
    DBMS
    and select a type from the drop-down.
    Note:
    The default DBMS type is Oracle.
  6. If the content is ordered, enable the
    Ordered
    checkbox.
  7. To add a description, double-click the description field, type a description, and click
    Save
    .
    The transformation map is created.
Map Columns to Transformation Maps
To map columns to transformation maps,
Follow these steps:
  1. Select a registered object in the left panel, and click on the drop-down list to apply transformations to specific columns.
    Note:
    In the center panel, the first three boxes in the columns are for transformation map review. The three levels of transformation map review are;
    • Checked
    • Validated
    • Approved
If a previously checked and validated map has its Checked attribute removed, the Validated attribute is also removed.
You can map the following columns to the transformation map.
  • Transformation
Select the function from the drop-down list. To add extra functions, see Maintain Data Functions.
  • Keep Nulls
If the original value is NULL, the new value remains NULL. The option can be toggled On and Off for each row by selecting the check box.
  • Listcol No
The LIST function randomly picks up values from the seed table GTSRC_REFERENCE_DATA. The table allows you to store up to nine columns for one row. For example, PERSON NAME has four columns: Full Name, Title, First Name, and Last Name. The column allows you to pick from linked values so that First and Full name match.
  • Fixed Value
Any fixed value. Enclose any characters in inverted commas, for example, `test `.
  • Cross Ref
Cross Ref allows you to retain the same transformation for the same input value. For example, if the name Price is converted to Jones, all Prices become Jones.
  • Cross Ref Indent
Groups the rows in the cross-reference table, allowing you to have several columns which map to a single cross-reference map.
  • Substring Start
Enter the place value at the start of the substring to mask.
Note:
Positioning starts at the value 1, and not at the value 0
  • Sub-string Finish
Enter the place value at the start of the sub string to mask.
Note:
Positioning starts at the value 1, and not at the value 0
  • Notes
Use to add comments.
The following columns are only available when you select SDM as the RDBMS for a map:
Note:
SDM represents Fast Data Masker.
  • WHERE Clause
Double-click the cell to enter a WHERE clause. A pop-up window shows all the columns that you can use.
  • Date Format
Allows you to define the data format to use.
  • Preformat
Specifies the format of the original data before it is masked.
  • XPath Element
Specifies where in the XML data you want the masking to take place. For more information about mapping an XML file, see Defining Mapping for Multiple XML Files.
To set related tables with the same function for the column, click the tree icon to the right of the scramble function. You are guided through the model identifying any related columns.
Define Mapping for Multiple WHERE Clauses
To map multiple WHERE files within DataMaker, ensure that the Transformation Map selected is an SDM file.
Note:
If you do not have an SDM file, create one. For more information, see Create a Transformation Map.
Follow these steps:
  1. Select the SDM file, the column, and the transformation.
  2. Double-click the WHERE clause cell, and drag-and-drop available columns into the SQL window.
  3. Add text to define the WHERE clause.
    Note:
    We recommended that you validate your SQL against either the source or target connection.
  4. When the validation is finished, click the
    Disk
    icon to save.
  5. The WHERE clause is saved, and a new row opens to add another WHERE clause for the selected column.
Define Mapping for Multiple XML Files
To map multiple WHERE files within DataMaker, ensure that the Transformation Map selected is an SDM file.
Note:
If you do not have an SDM file, create one. For more information, see Create a Transformation Map.
Follow these steps:
  1. Select the SDM file, the column, and the transformation.
  2. Specify the location of the XML data to mask in the
    XPath Element
    cell.
  3. Click to save.
    A new row opens where you can enter another XML file location.
Copy and Delete Transformation Maps
Copy transformation maps to the current project, to another version within the project, or to an existing project. To copy, edit, or delete transformation maps,
Follow these steps:
  1. Select
    Projects
    ,
    Project Manager
    .
  2. In the
    Projects
    tree, expand a project and a version.
  3. Select
    Transformation Maps
    from the tree structure.
    A list of maps that are associated with that version appears.
  4. Right-click on the transformation map, and select a copy type:
    • Copy to the same project
      .
    • Copy to the different project
      .
  5. (Optional) To edit or delete a transformation map, follow the same steps, and select
    Edit
    or
    Delete
    from the menu.
  6. Specify the copy details, and click the
    Copy
    icon.
    The transformation map is copied.
Upgrade Transformation Maps Between Versions
When you upgrade a version, transformation maps are not copied to the new version by default. To copy a transformation map between versions,
Follow these steps:
  1. Right-click on the transformation map, and select
    Copy to the same project
    .
  2. Select the version, specify the name, and click the
    Copy
    icon.
    The transformation map is copied to the new version.
Reconcile and Merge Transformation Maps
If you have multiple transformation maps defined, you can compare and merge them between different versions. To compare and merge transformation maps between different versions;
Follow these steps:
  1. Go to
    Project Manager
    , expand the project and a version.
  2. Select
    Transformation Maps
    , right-click on the transformation map, and select
    Reconcile and Merg
    e.
  3. Select the two maps to compare from the drop-down lists.
    Note:
    Differences between the two maps are shown in blue.
  4. In the
    Action
    column, use the drop-down list to select one of the following merge action types.
    • Copy to Right
      Copies any changes from the left drop-down to the right
    • Copy to Left
      Copies any changes from the right drop-down to the left
    • No Action
      No action in either direction occurs.  
  5. Click the drop-down list in the top-right hand corner, and select one of the following actions;
    • Perform Action
      Performs all the right to left and left to right actions that are set.
    • Set Copy to Left
      Forces 'Copy to Left' the columns with differences
    • Set Copy to Right
      Forces 'Copy to Right' the columns with differences
    • Set All No Action
      Forces 'No Action' to the columns with differences
    • Set to Equalize Left
      Copies changes 'Left to Right' and equalizes the two lists.
    • Set to Equalize to Right
      Copies changes 'Right to Left' and equalizes the two lists.
    • Reset
      Returns selections back to their original state
Note:
To create seed data from cube views, define your definition or transactions using Data Subset. For more information, see the Data Subset Reference Guide.
To use the created Data Subset definition to extract a cube of data:
Follow these steps:
  1. Go to
    Project Manager
    , expand the project and a Version.
  2. Expand
    Subsets
    , right-click on the definition, and select
    Create Data from Cube
    .
  3. Specify the data subset information.
    Seed data is created from a cube.
    For more information about creating seed data from a data cube, see Create Seed Data from Cube. 
Mask Using Transformation Maps
You can also use transformation maps to mask your production data. You do this through transformation maps and Subset scripts. You first define a transformation map (Oracle or MSSQL) in Datamaker, create masking functions for the columns you want to mask. You then use Subset to create
masked
export scripts. These scripts perform masking as they export the source data to a dump file. You can then import the dump file (which contains the masked data) to the target database.
Furthermore, you can use transformation map files in Fast Data Masker to mask the data. In this case, you export your transformation map into a CSV file and use that CSV file in Fast Data Masker. For more information about how to use transformation map files in Fast Data Masker, see Use Transformation Map Files.
Note:
Review the Fast Data Masker and Transformation Maps section to understand the masking approach that you can follow depending on business requirements.
Import or Export Ordered Transformation Maps
When you export an ordered transformation map as a CSV file, the file content is not in the expected order by default.
To ensure that the order is reflected correctly, follow these steps:
  1. Set the transformation map to Ordered when you create it.
  2. Use the Order column in the transformation map window to specify the order manually.
  3. Export the transformation map to CSV.
Similarly, when you import SDM transformation maps from a CSV file, set the Ordered flag in the Import dialog.
The CSV output is based on the following criteria:
  1. Ordered by table name.
  2. Then ordered by the presence of a where clause. Rows without where clause are exported first.
  3. Then ordered by the order number. Rows without an order number are exported first, followed by low to high order value.
  4. Then ordered by an internal transformation column ID and where clause sequence number.