Use APIs to Audit and Mask PII Data

Test Data Engineers may Audit and Mask data sources either though the CA TDM Portal or via the auditing and masking APIs. For more information on the Audit PII workflow, see . 
4-8-1
Test Data Engineers may Audit and Mask data sources either though the CA TDM Portal or via the auditing and masking APIs. For more information on the Audit PII workflow, see PII Audit Using CA TDM Portal
This page refers to the following API Services:
 We recommend that for each application you want to discover and profile Personally Identifiable Information (PII) data in is part of a separate project.
Common parameters
The following parameters are common to all APIs:
  • <server>
    The system where the CA TDM Portal instance is available.
  •  
     
    <port>
     
    The address of the port through which you access your CA TDM installation.
Pagination parameters
APIs that incorporate pagination can take the following parameters:
  •  
     
    size=<size>
     
    For enumeration of long results. 
    size
     defines the number of elements you receive per page of results. The default page size is 10.
  •  
     
    page=<page>
     
    For enumeration of long results. Defines the page of your results that you want to receive in the body of your response. Page numbers start at 0.
    Divide the total number of elements by 
    size
     to determine the number of pages your results contain.
 
 
APIs for audit and masking
Import Classifiers 
As a Test Data Engineer, you can customize Classifiers as per your requirement and import classifiers into the CA TDM Portal. For more information about Classifiers, see Manage Data Classifiers
 During installation, CA TDM imports a standard set of Classifiers and SeedLists into CA TDM Portal. SeedLists are part of Classifier Packs.
 
Syntax
 
POST https://<server>:<port>/TDMModelService/api/ca/v1/profiler/classifiers
 
Parameters
 
  •  
    definitionFile
     (formData)
    Zip file containing classifier hierarchy definition to be imported.
  •  
    parentId
     (query, long)
    Container where the definitions should be imported.
  •  
    onduplicate
     (query, string)
    Defines behaviour during import, if a duplicate resource is found what to do. Possible values:
    •  
      IGNORE
       
    •  
      ABORT
       
    •  
      OVERWRITE
       
 
Example
 
POST https://host:8443/TDMModelService/api/ca/v1/profiler/classifiers?onduplicate=OVERWRITE
  •  
    FORM DATA
     
    Multipart/form-data
    Includes a file named 'definitionsFile' in a Zip file containing the Classifier Pack
  •  
    RESPONSE BODY
     
    {
    "classifiersCreated": 0,
    "classifiersUpdated": 2,
    "containersCreated": 0,
    "containersUpdated": 0,
    "seedListCreated": 0,
    "seedListUpdated": 1,
    "duplicateClassifiers": 2,
    "duplicateSeedlist": 1,
    "duplicateContainers": 3,
    "duplicateOption": "OVERWRITE"
    }
You have imported the required Classifiers and now you can continue to set up Audit PII.
Setup PII Data Scan
You as Test Data Engineer can setup PII Data scan by performing the following activities:
  1. Enumerate Data Sources:
    You as Test Data Engineer can perform PII Data scan on one or more Connection Profiles or an Environment.
    1.  enumerate_connection_profilesAs a Test Data Engineer you can determine the Connection Profiles on which you want to perform a PII Data scan job. 
    2.  Enumerate an Environment
      As a Test Data Engineer you can determine the Environment on which you want to perform a PII Data scan job.
  2.  Enumerate Classifiers
    You as Test Data Engineer determine the Classifiers against which you want to perform a PII Data scan job.
 If the required Connection Profiles and Data Classifiers are predefined in the CA TDM Portal, continue to Initiate?a Data Profiling Scan Job.
Enumerate Connection Profiles
Enumerating Connection Profiles enables you to identify the Connection Profiles that are defined for your CA TDM Portal instance. You can review this list of Connection Profiles to identify the Connection Profiles on which you want to Audit PII. If a Connection Profile does not exist to the required database, the Connection Profile will have to be defined through the CA TDM Portal or the Connection Profiles REST API.
For more information about how to create a Connection Profile using REST API, see Create a Connection Profile.
 We recommend that you make a note of the value of the 
name 
parameter value returned in the response. The 
name
 parameter value is used as an input for the Audit PII scan job.
 
Syntax
 
GET https://<server>:<port>/TDMConnectionProfileService/api/ca/v1/connectionProfiles
 
Parameters
 
  • (Optional) Filter criteria for results:
    •  
      groupOnly
       (query, Boolean)
 
Example
 
GET https://host:8443/TDMConnectionProfileService/api/ca/v1/connectionProfiles?groupOnly=true
  •  
    RESPONSE BODY
     
    [
    {
    "name": "Travel_E",
    "description": "none",
    "dbType": "sql server",
    "server": "host",
    "port": "",
    "instance": "",
    "service": "",
    "database": "Travel_E",
    "schema": "",
    "username": "sa",
    "password": "nwoUjTtlSBWRSbl9Z1i9p7ZP3iv42jgNimHwoNq+RNUNfnfO+DV-E55F",
    "datasourceUrl": "jdbc:sqlserver://host;database=Travel_E",
    "datasourceDriver": "com.microsoft.sqlserver.jdbc.SQLServerDriver",
    "created": 1513012472677,
    "modified": 1513012472677,
    "createdBy": 1,
    "additionalConnectionProperties": ""
    },
    {
    "name": "TravelDB",
    "description": "",
    "dbType": "sql server",
    "server": "host",
    "port": "",
    "instance": "",
    "service": "",
    "database": "travel",
    "schema": "",
    "username": "sa",
    "password": "taPAa1cuko63c9NCX32Y8DkKS3RkES8aWMKczIlgFFlq5wYyPk-yE24V",
    "datasourceUrl": "jdbc:sqlserver://host;database=travel",
    "datasourceDriver": "com.microsoft.sqlserver.jdbc.SQLServerDriver",
    "created": 1513063321730,
    "modified": 1513063321730,
    "createdBy": 1,
    "additionalConnectionProperties": ""
    }
    ]
You have enumerated the Connection Profiles and now you can continue to enumerate Classifiers. 
Enumerate Classifiers
 If you want to use all Classifier Packs for an Audit PII scan, you do not need to select Classifiers. Set 
classifierPacks
 parameter to 0 to initiate a Audit PII scan using all Classifier Packs.
Enumerating Classifiers returns a list of Classifiers and enables you to identify the specific Classifier against which you want to perform a Audit PII scan. You can use multiple Classifier Packs during Audit PII. 
You can use one of the following REST APIs to enumerate Classifiers:
Enumerate Classifiers in a Container (Classifier Pack)
 
Syntax
 
GET https://<server>:<port>/TDMModelService/api/ca/v1/profiler/classifiers/containers/<containerId>
 
Parameters
 
  •  
    containerId
     (path, long)
    Container of which to enumerate contents.
  • (Optional) Filter criteria for results:
    •  
      recursive
       (query, long)
    •  
      classifierType
       (query, string)
    •  
      classifierClass 
      (query, string)
    •  
      classifierOrigin
       (query, string)
    •  
      tags 
      (query, string)
 
Example
 
GET https://host:8443/TDMModelService/api/ca/v1/profiler/classifiers/containers/0
The response for this REST API includes the Classifier Pack 
name
 and each Classifier Pack includes a specific 
id
 that you can use in the request to initiate a Audit PII scan.
For example, to use the Common Classifier Pack, find the Classifier Pack name ("name":"
Common
") in the response and use the associated id parameter ("id":"
19668
") when you perform an Audit PII scan.
  •  
    RESPONSE
    200 OK
  •  
    RESPONSE BODY
     
    {
    "name": "ROOT",
    "names": [
    {
    "id": 1,
    "lang": "en",
    "value": "ROOT"
    }
    ],
    "description": "The root classifier container",
    "descriptions": [
    {
    "id": 2,
    "lang": "en",
    "value": "The root classifier container"
    }
    ],
    "root": true,
    "created": 1513012309553,
    "updated": null,
    "createdBy": "SYSTEM",
    "updatedBy": null,
    "containedContainers": [
    {
    "name": "Common"
    ,
    "names": [
    {
    "id": 19668
    ,
    "lang": "en",
    "value": "Common"
    }
    ],
    "description": null,
    "descriptions": [],
    "root": null,
    "created": 1513012385037,
    "updated": null,
    "createdBy": "SYSTEM",
    "updatedBy": null,
    "containedContainers": [],
    "containedClassifiers": [],
    "id": 19667
    },
    {
    "name": "Germany"
    ,
    "names": [
    {
    "id": 19950
    ,
    "lang": "en",
    "value": "Germany"
    }
    ],
    "description": null,
    "descriptions": [],
    "root": null,
    "created": 1513012387117,
    "updated": null,
    "createdBy": "SYSTEM",
    "updatedBy": null,
    "containedContainers": [],
    "containedClassifiers": [],
    "id": 19949
    },
    {
    "name": "Japan"
    ,
    "names": [
    {
    "id": 29690
    ,
    "lang": "en",
    "value": "Japan"
    }
    ],
    "description": null,
    "descriptions": [],
    "root": null,
    "created": 1513012413520,
    "updated": null,
    "createdBy": "SYSTEM",
    "updatedBy": null,
    "containedContainers": [],
    "containedClassifiers": [],
    "id": 29689
    },
    {
    "name": "Sweden"
    ,
    "names": [
    {
    "id": 29698
    ,
    "lang": "en",
    "value": "Sweden"
    }
    ],
    "description": null,
    "descriptions": [],
    "root": null,
    "created": 1513012414340,
    "updated": null,
    "createdBy": "SYSTEM",
    "updatedBy": null,
    "containedContainers": [],
    "containedClassifiers": [],
    "id": 29697
    },
    {
    "name": "UK"
    ,
    "names": [
    {
    "id": 29722
    ,
    "lang": "en",
    "value": "UK"
    }
    ],
    "description": null,
    "descriptions": [],
    "root": null,
    "created": 1513012415370,
    "updated": null,
    "createdBy": "SYSTEM",
    "updatedBy": null,
    "containedContainers": [],
    "containedClassifiers": [],
    "id": 29721
    },
    {
    "name": "USA"
    ,
    "names": [
    {
    "id": 39536
    ,
    "lang": "en",
    "value": "USA"
    }
    ],
    "description": null,
    "descriptions": [],
    "root": null,
    "created": 1513012443263,
    "updated": null,
    "createdBy": "SYSTEM",
    "updatedBy": null,
    "containedContainers": [],
    "containedClassifiers": [],
    "id": 39535
    }
    ],
    "containedClassifiers": [],
    "parentId": -1,
    "id": 0
    }
Enumerate classifiers and containers (classifier packs) which are children of a container (classifier pack)
 
Syntax
 
GET https://<server>:<port>/TDMModelService/api/ca/v1/profiler/classifiers/classifiers/{classifierId}
 
Parameters
 
  •  
    classifierId
     (path, long)
    Container (classifier pack) of which to enumerate contents.
 
Example
 
GET https://host:8443/TDMModelService/api/ca/v1/profiler/classifiers/classifiers/0
The response for this REST API includes a list of Classifiers and Classifier Pack names for a specific Classifier id. The response code also includes the masking function associated with the tag of each Classifier.
  •  
    RESPONSE:
     
    200 OK
     
  •  
    RESPONSE BODY:
     
    {
    "classifierClass": "string",
    "classifierOrigin": "string",
    "classifierType": "string",
    "config":
    [
    {
    "id": 0,
    "name": "string",
    "value": "string"
    }
    ],
    "created": "2018-07-02T14:11:00.070Z",
    "createdBy": "string",
    "description": "string",
    "descriptions":
    [
    {
    "id": 0,
    "lang": "string",
    "value": "string"
    }
    ],
    "id": 0,
    "name": "string",
    "names":
    [
    {
    "id": 0,
    "lang": "string",
    "value": "string"
    }
    ],
    "parentId": 0,
    "tags": "string",
    "updated": "2018-07-02T14:11:00.070Z",
    "updatedBy": "string"
    }
You have enumerated the required Classifiers and now you can continue to initiate a Audit PII scan job.
Initiate an Audit PII Scan Job
You as a Test Data Engineer can start a Audit PII scan job using the Connection profiles and Classifier Packs. In the request body you can specify the classifier packs and connection profiles you want to use.
 
Syntax
 
POST https://<server>:<port>/TDMJobService/api/ca/v1/jobs
 
Parameters
 
  •  
    Request body
     (JSON, body)
 
Example
 
POST https://host:8443/TDMJobService/api/ca/v1/jobs
  •  
    REQUEST 
     
    {
    "name": "Profiling Scan for PII [ 1.0 ]",
    "description": "Profiling Scan - project id: 2352",
    "projectId": 2352,
    "versionId": 2353,
    "type": "PIISCAN",
    "origin": "profiling",
    "scheduledTime": 1513063791088,
    "parameters": {
    "connProfiles"
    : [
    "TravelDB"
    ],
    "classifierPacks"
    : [
    19667,
    19949,
    39535
    ],
    "scanType": "FAST",
    "refreshToken": null
    }
    }
  •  
    RESPONSE: 
    201 Created
  •  
    RESPONSE :
     
    {
    "message": "Job successfully submitted with name: Profiling Scan for PII [ 1.0 ], id: 2",
    "name": null,
    "jobId": 2
    ,
    "description": null,
    "projectName": null,
    "projectId": 0,
    "versionId": null,
    "createdBy": null,
    "email": null,
    "scheduledTime": null,
    "startTime": null,
    "endTime": null,
    "status": null,
    "type": null,
    "parentId": 0,
    "jobs": null,
    "duration": null,
    "artifactLocation": null,
    "origin": null,
    "parameters": null,
    "statusMessage": null,
    "runningStatus": null,
    "created": null
    }
You have started a Audit PII scan job and now you can continue to check the job status with the value of the 
jobId
 attribute.
Check Job Status
You as a Test Data Engineer can obtain the status of a specific Job to identify if the scan results are ready for review. You can obtain the Job ID from the response provided in Initiate a Audit PII Scan Job. 
You can use one of the following REST APIs to obtain the status of a specific Job:
  1.  
    Syntax
     
    GET https://<server>:<port>/TDMJobService/api/ca/v1/jobs/{jobId}
     
     
     
    Parameters
     
    •  
      jobId
       (path, long)
       
     
    Example
     
    GET https://host:8443/TDMJobService/api/ca/v1/jobs/
    2
    •  
      RESPONSE:
      200 OK
    •  
      RESPONSE :
      The 
      runningStatus
       parameter in the response identifies the job status. 
      {
      "name": "Profiling Scan for PII-Example [ 1.0 ]",
      "jobId": 2
      ,
      "description": "Profiling Scan - project id: 2362",
      "projectName": "PII-Example",
      "projectId": 0,
      "versionId": null,
      "createdBy": "Administrator",
      "email": null,
      "scheduledTime": "2017-12-13T09:36:53Z",
      "startTime": "2017-12-13T09:41:19Z",
      "endTime": "2017-12-13T09:41:36Z",
      "status": "Completed",
      "type": "PIISCAN",
      "parentId": 0,
      "jobs": [],
      "duration": 16630,
      "artifactLocation": null,
      "origin": "profiling",
      "parameters": {},
      "statusMessage": "",
      "runningStatus": "Completed"
      ,
      "created": null
      }
  2. Use this REST API to get more detailed status of a job. The response includes the number of tables and columns scanned for PII data.
    Syntax
     
    GET https://<server>:<port>/TDMModelService/api/ca/v1/profiler/jobs/{jobId}
     
    Parameters
     
    •  
      jobId
       
     
    Example
     
    GET https://host:8443/TDMModelService/api/ca/v1/profiler/jobs/
    2
    The 
    state
     parameter in the response identifies the job status and also provides more information about the Profiling job progress.
    •  
      RESPONSE :
       
      {
      "jobID": 2,
      "jobName": "Profiling Scan for PII-Example [ 1.0 ]",
      "projectID": 1180,
      "projectVersionID": 1181,
      "projectName": "QTP - Example Project",
      "state": "SCAN_COMPLETE",
      "startDate": 1513412020743,
      "stopDate": 1513412055037,
      "completeDate": null,
      "signOffRequestedDate": null,
      "setup": null,
      "submittedBy": "Administrator",
      "approvedBy": null,
      "approved": null,
      "reason": null,
      "severity": 0.0,
      "totalPii": 57,
      "contentClassifierHash": -1693587654,
      "contentSeedlistHash": 1009518500,
      "columnClassifierHash": -1072720395,
      "columnSeedlistHash": null,
      "totalTables": 40,
      "totalColumns": 298,
      "columnsClassified": 46,
      "tablesClassified": 23,
      "tablesScanned": 40,
      "columnsScanned": 298,
      "tablesReviewed": 0,
      "totalReviewers": 0,
      "totalApprovers": 0,
      "listApprovers": [],
      "listReviewers": [],
      "warnings": ""
      }
After the job status changes to Completed or SCAN_COMPLETE state, you can continue to query through the tables to obtain table details.
Query Through the Tables
As a Test Data Engineer, you can query the tables to identify the specific table names, column names, schema names, connection profile names, or tags. Depending on the content of the tables you can mark the table as Not PII.
 
Syntax
 
GET https://<server>:<host>/TDMModelService/api/ca/v1/profiler/jobs/{jobId}/piidata?
 
Parameters
 
  •  
    jobId
     (JSON, body)
    ID of the PII Profiling job
  •  
    hasTags
     (boolean)
    True = Only include columns with PII tag(s)
  •  
    history
     (boolean)
    True = Include tag history
  •  
    (query, string)
    Filter query results. You can perform a search on the relevant data source based on one or more of the following key fields:
    •  
      table=
      Matches a table name. 
    •  
      column=
      Matches a column name.
    •  
      schema=
      Matches a schema name.
    •  
      profile=
      Matches a connection profile name.
    •  
      tag=
      Matches a tag name.
       If no key field is specified then all tables, columns, schema names, connection profiles, and tags are searched.
      This REST API supports the basic wild card characters such as * (used to match one or more characters) and ? (used to match a single character). Supplied search terms are treated with ANY logic (i.e. condition A=true OR condition B=true). For example, when you perform a search for "tag=Surname tag=Title", the response includes all tables that have either a Surname tag or a Title tag.
 
Example
Use the following API call to search for tables with tags equal to Surname, or tags that end with the word "Code", such as Post Code, Zip Code.
GET https://host:8443/TDMModelService/api/ca/v1/profiler/jobs/2/piidata?page=1&size=5&q="tag=Surname tag=*Code"
  •  
    RESPONSE
     
{
"elements": [
{
"databaseName": "travel",
"schemaName": "dbo",
"tableName": "ACCESS_CONTROLS",
"profileName": "travel",
"tableId": 601,
"rowCount": 418,
"columnCount": 4,
"tagHistory": null,
"piiTags": [
"
Surname
"
],
"confirmed": false,
"reason": null,
"reviewer": null,
"severity": 2,
"matchedSamples": 3,
"dateReviewed": null,
"notPII": false
},
{
"databaseName": "travel",
"schemaName": "dbo",
"tableName": "ACCOUNT_PERIODS",
"profileName": "travel",
"tableId": 602,
"rowCount": 789,
"columnCount": 18,
"tagHistory": null,
"piiTags": [
"
Surname
"
],
"confirmed": false,
"reason": null,
"reviewer": null,
"severity": 2,
"matchedSamples": 2,
"dateReviewed": null,
"notPII": false
},
{
"databaseName": "travel",
"schemaName": "dbo",
"tableName": "ADDRESS",
"profileName": "travel",
"tableId": 603,
"rowCount": 10,
"columnCount": 9,
"tagHistory": null,
"piiTags": [
"Given Name",
"
Post Code
",
"
Surname
",
"Towns"
],
"confirmed": false,
"reason": null,
"reviewer": null,
"severity": 2,
"matchedSamples": 21,
"dateReviewed": null,
"notPII": false
},
{
"databaseName": "travel",
"schemaName": "dbo",
"tableName": "AIRCRAFT_TYPES",
"profileName": "travel",
"tableId": 605,
"rowCount": 11,
"columnCount": 4,
"tagHistory": null,
"piiTags": [
"
Post Code
"
],
"confirmed": false,
"reason": null,
"reviewer": null,
"severity": 2,
"matchedSamples": 2,
"dateReviewed": null,
"notPII": false
},
{
"databaseName": "travel",
"schemaName": "dbo",
"tableName": "AIRCRAFT_LAYOUTS",
"profileName": "travel",
"tableId": 604,
"rowCount": 1307,
"columnCount": 12,
"tagHistory": null,
"piiTags": [
"
ZIP Code
",
"Account Number"
],
"confirmed": false,
"reason": null,
"reviewer": null,
"severity": 1,
"matchedSamples": 0,
"dateReviewed": null,
"notPII": false
}
],
"numberOfElements": 5,
"totalElements": 40
}
You have queried through all the tables and now you can identify and mark tables as Not PII.
Mark Empty/Known/Unmatched Tables as Not PII
After querying through the tables and retrieving relationships for a table, as a Test Data Engineer you can identify if a table can be marked as Not PII. For example, you can mark a table as Not PII based on the following conditions:
  • Table is empty
    where, 
    rowCount
     parameter is 0
  • Table does not match any PII tags
    where, 
    piiTags
     parameter is empty
  • Known table that does not contain any PII data
 
Syntax
 
PATCH https://<server>:<host>/TDMModelService/api/ca/v1/profiler/jobs/{jobId}/piidata/{tableId}
 
Parameters
 
  •  
    jobId
     (JSON, body)
    ID of the PII Profiling job
  •  
    tableId
     (JSON, body)
    ID of the table on which to change tag data
  •  
    history
     (boolean)
    True = Include tag history
  •  
    patch
     (JSON, body)
    Properties to update within the table object.
 
Example
 
Use the following REST API to mark a table as Not PII, and to confirm that a table is not using any PII data.
PATCH https://host:8443/TDMModelService/api/ca/v1/profiler/jobs/
2
/piidata/
604
  •  
    REQUEST BODY
    :
    {
    "confirmed": true,
    "notPII": true,
    "reason": "Not PII Data"
    }
  •  
    RESPONSE
    :
200 OK
  •  
    RESPONSE BODY:
     
    {
    "databaseName": "travel",
    "schemaName": "dbo",
    "tableName": "ACCESS_CONTROLS",
    "profileName": "TravelDB",
    "tableId": 604,
    "rowCount": 418,
    "columnCount": 4,
    "tagHistory": null,
    "piiTags": [],
    "confirmed": true,
    "reason": "Not PII Data",
    "reviewer": "Administrator",
    "severity": 0.0,
    "matchedSamples": 0,
    "dateReviewed": null,
    "notPII": true
    }
After marking empty/known/unmatched tables as Not PII, login to the CA TDM Portal and review the remaining tables and create a report. For more information, see End-to-End Scenario for PII Audit.
Audit Log Extraction
Use this REST API to get an audit on all operations that have been performed on a Audit PII scan job. The response of this REST API includes details about who initiated a scan, who reviewed and confirmed the tables and so on. You can filter results with the query. See available filter fields below.
 
Syntax
 
GET https://<server>:<host>/TestDataManager/api/ca/v1/auditlogs
 
Parameters
 
  •  
    q
     (query)
    Filter query results. You can filter the audit logs you receive with the following key fields (use wildcard % to match zero, one or multiple characters in this position):
    •  
      format
       (string)
      Format in which results returned. JSON (default) or ZIP-CSV.
    •  
      origin
       (string)
    •  
      description
       (string)
    •  
      type
       (string)
    •  
      status
       (string)
    •  
      user_name
       (string)
    •  
      link_id
       (string)
      This corresponds with the jobId.
    •  
      proj_id
       (string)
    •  
      proj_version_id
       (string)
    •  
      timestamp_start
       (string)
    •  
      timestamp_end
       (string)
    •  
      sort
       (string)
    •  
      order
       (string)
    •  
       Pagination parameters 
      (integers)
      parameter
       pagesize
       replaces size
      .
       
    •  
      cachesize 
      (integer in range 1 to 1,000,000)
      Performance tuning parameter. Default = 1,000.
 
Example
 
Filter by Job ID (
link_id
 parameter)
GET https://host:8443/TestDataManager/api/ca/v1/auditlogs?link_id=2
  •  
    RESPONSE BODY:
     
    {"elements":[
    {
    "id":11,
    "user_name":"Administrator",
    "link_id":2,
    "origin":"PII Data Scan",
    "type":"MODELPIISCAN",
    "status":"CREATED",
    "description":"STARTED",
    "timestamp":"2018-07-27 08:48:26.83",
    "proj_id":2346,
    "proj_version_id":2347
    },
    {
    "id":12,
    "user_name":"Administrator",
    "link_id":2,
    "origin":"PII Data Scan",
    "type":"MODELPIISCAN",
    "status":"STARTED",
    "description":"STARTED",
    "timestamp":"2018-07-27 08:48:27.027",
    "proj_id":2346,
    "proj_version_id":2347
    },
    {
    "id":13,
    "user_name":"Administrator",
    "link_id":2,
    "origin":"PII Data Scan",
    "type":"MODELPIISCAN",
    "status":"COMPLETED",
    "description":"COMPLETED",
    "timestamp":"2018-07-27 08:48:56.107",
    "proj_id":2346,
    "proj_version_id":2347
    },
    {"id":14,
    "user_name":"Administrator",
    "link_id":2,
    "origin":"PII Data Scan",
    "type":"MODELPIISCAN",
    "status":"COMPLETED",
    "description":"COMPLETED",
    "timestamp":"2018-07-27 08:48:56.357",
    "proj_id":2346,
    "proj_version_id":2347}
    ],
    "numberOfElements":4,
    "totalElements":4,
    "totalPages":1}
    You have obtained the audit log and identified all operations that have been performed on a Audit PII scan job. 
Retrieve a List of Tags
A Test Data Engineer (TDE) can enumerate a list of tags that are available in the CA TDM Portal. The search criteria follows the RSQL format and allows the query to be filtered on any of the resource's field values, such as 'name', 'id', 'whoCreated', etc. For more information about the RSQL format, see https://github.com/jirutka/rsql-parser.
This REST API supports the wild card characters * (used to match one or more characters) and ? (used to match a single character). Search terms are treated with ANY logic (i.e. condition A=true OR condition B=true). For example, when you perform a search for "q=name==co*", the response includes all tags with Country or County names. The query parameter in this REST API is case insensitive.
 If no search criteria is included in this REST API, a list of all tags is returned.
 
Syntax
 
GET https://<server>:<host>/TDMModelService/api/ca/v1/profiler/tags
 
Parameters
 
  •  Pagination parameters
  •  
    q
     (query)
    Filter criteria.
 
Example:
 
The following example only returns tags whose name starts 'co' (not case-sensitive). It returns the first page of results, with up to 20 entries per page.
GET https://host:8443/TDMModelService/api/ca/v1/profiler/tags?page=0&size=20&q=name==co*
  •  
    RESPONSE:
    200 OK
  •  
    RESPONSE BODY:
     
    {
    "elements": [
    {
    "id": 1271674,
    "name": "Country",
    "whoCreated": "Administrator",
    "programCreated": "TDMApi"
    },
    {
    "id": 1281739,
    "name": "County",
    "whoCreated": "Administrator",
    "programCreated": "TDMApi"
    }
    ],
    "numberOfElements": 2,
    "totalElements": 2
    }
Retrieve Tag Details for a Specific Tag
A Test Data Engineer (TDE) can retrieve details about a specific tag based on a tagId. This helps a TDE to understand how the data identified by a tag can be masked. 
 
Syntax
 
GET https://<server>:<host>/TDMModelService/api/ca/v1/profiler/tags/{tagId}
 
Parameters
 
  •  
    tagId
     (integer, long)
    ID of the tag for which to return details.
 
Example:
 
GET https://host:8443/TDMModelService/api/ca/v1/profiler/tags/1271674
  •  
    RESPONSE:
    200 OK
  •  
    RESPONSE BODY:
     
    {
    "id": 1271674,
    "name": "Country",
    "dateCreated": 1524242459710,
    "whoCreated": "Administrator",
    "programCreated": "TDMApi"
    }
Retrieve a list of Matched Classifiers for all Columns in a Table
As a Test Data Engineer (TDE), you can retrieve a list of counts of classifiers that matched a column in a table. These results are organized by column. This helps a TDE to understand how the data identified by a tag can be masked. 
In the response code:
  • parameter 
    clsMatches
     provides a list of all classifier IDs that have matched to the column together with the count of matched values from the column.
  • parameter 
    clsMaxMatch
     provides the Classifier Id with the highest match count.  
 
Syntax
 
GET https://<server>:<host>/TDMModelService/api/ca/v1/profiler/jobs/{jobId}/piidata/{tableId}/columns
 
Parameters
 
  •  
    tagId
     (integer, long)
    ID of the tag for which to return details.
  •  
    tableId
     (integer, long)
    ID of the tag for which to return details.
  •  Pagination parameters
  •  
    hasTags
     (boolean)
    True = Only include columns with PII tag(s)
  •  
    history
     (boolean)
    True = Include tag history
 
Example
 
GET https://host:8443/TDMModelService/api/ca/v1/profiler/jobs/18/piidata/1644/columns
  •  
    RESPONSE:
    200 OK
     
  •  
    RESPONSE BODY:
     
    {"elements":
    [
    {
    "columnName":"ID",
    "columnId":39207,
    "dataType":"numeric",
    "piiTags":[],
    "clsMatches":{},
    "clsMaxMatch":null,
    "tagsSet":false,
    "reason":null,
    "dateReviewed":null,
    "reviewer":null,
    "severity":null,
    "tagHistory":[]
    },
    {
    "columnName":"NAME",
    "columnId":39208,
    "dataType":"varchar",
    "piiTags":["Country","County","Ethnicity","Gender","Given Name","Towns"],
    "clsMatches":{
    "108064":1,
    "108992":2,
    "98326":4,
    "108969":1,
    "30009":1,
    "108120":17,
    "98042":7,
    "109002":9
    },
    "clsMaxMatch":108120,
    "tagsSet":true,
    "reason":null,
    "dateReviewed":null,
    "reviewer":null,
    "severity":null,
    "tagHistory":[]
    },
    {
    "columnName":"STATE_PROVINCE",
    "columnId":39209,
    "dataType":"varchar",
    "piiTags":["Country","Ethnicity","Given Name","Surname","Title"],
    "clsMatches":{
    "109842":2,
    "117845":2,
    "98681":1,
    "108969":6,
    "98042":10,
    "109002":1
    },
    "clsMaxMatch":98042,
    "tagsSet":true,
    "reason":null,
    "dateReviewed":null,
    "reviewer":null,
    "severity":null,
    "tagHistory":[]
    },
    {
    "columnName":"TIME_ZONE_CITY",
    "columnId":39210,
    "dataType":"varchar",
    "piiTags":[],
    "clsMatches":{},
    "clsMaxMatch":null,
    "tagsSet":false,
    "reason":null,
    "dateReviewed":null,
    "reviewer":null,
    "severity":null,
    "tagHistory":[]
    },
    {
    "columnName":"TIME_ZONE_TO_GMT",
    "columnId":39211,
    "dataType":"numeric",
    "piiTags":[],
    "clsMatches":{},
    "clsMaxMatch":null,
    "tagsSet":false,
    "reason":null,
    "dateReviewed":null,
    "reviewer":null,
    "severity":null,
    "tagHistory":[]
    },
    {
    "columnName":"CITY_MAP_LEVEL",
    "columnId":39212,
    "dataType":"numeric",
    "piiTags":[],
    "clsMatches":{},
    "clsMaxMatch":null,
    "tagsSet":false,
    "reason":null,
    "dateReviewed":null,
    "reviewer":null,
    "severity":null,
    "tagHistory":[]
    },
    ],
    "numberOfElements":6,
    "totalElements":6
    }
download fast data masker configuration
A Test Data Engineer (TDE) can download a PII data scan job in the Fast Data Masker (FDM) configuration format. You can import this FDM configuration file into FDM to mask PII data, or run .bat file provided in the zip file to mask PII data.
The FDM configuration is downloaded as a zip file and includes the following files for each database connection:
  •  
    Text file
    Specifies the connection details for FDM to connect to the database.
  •  
    CSV file
    Specifies the masking rules for tables and columns for a particular database.
  •  
    .BAT file
    The script file can be used on a Windows machine with FDM installed to run FDM in batch mode to perform the masking operation.
 
Syntax
 
GET https://<server>:<host>/TDMModelService/api/ca/v1/datamodel/profiler/fdm
 
Parameters
 
  •  
    projectId
    Specifies the project ID.
  •  
    versionId
    Specifies the version ID.
  •  
    environmentId
    Specifies the environment ID to mask.
  • (Optional)
     confirmedOnly
    Includes all confirmed tables.
    Values: true or false (default)
  • (Optional) 
    fileName
    Specifies the name of the downloaded FDM configuration file.
  • (Optional) 
    excNotPii
    Excludes all tables marked as Not PII.Values: true or false (default)
  • (Optional)
     dataSources
    Filter for data sources to mask.
  • (Optional)
     options
    Filter for masking options.
 
Example
 
GET https://host:8443/TDMModelService/api/ca/v1/datamodel/profiler/fdm?projectId=1180&versionId=1181&confirmedOnly=true&excNotPii=true&environmentId=6
  •  
    RESPONSE:
    200 OK
You have downloaded the FDM configuration and now you can use this with FDM to mask PII data. You can run the appropriate .bat file to perform the masking directly. Alternatively, to import a masking configuration into FDM, copy the required text file(s) into your Connection Files directory, and select the appropriate connection in FDM. Choose 
Open Saved Mask
 and open a .csv file from the zip and perform masking using FDM. For more information about how to perform masking using the FDM configuration file in FDM, see Mask Profiled Data in a Data Model.
Initiate Masking for a Data Model
A Test Data Engineer (TDE) can initiate a masking job for an entire Data Model. This helps the user to mask any sensitive data that should not be used for purposes such as development and testing.
 
Syntax
 
POST http://<server>:<host>/TDMMaskingService/api/ca/v1/masking/jobs/start
 
Parameters
 
  •  
    Masking job
     (JSON, body)
 
Example
 
POST http://host:8443/TDMMaskingService/api/ca/v1/masking/jobs/start
  •  
    REQUEST BODY:
     
    {
    "projId" : 2378,
    "pverId" : 2379,
    "environmentId" : 704
    }
  •  
    RESPONSE:
    200 OK
  •  
    RESPONSE BODY:
     
    {
    "jobId": 507
    ,
    "projId": 2378,
    "pverId": 2379,
    "environmentId": 704
    }
The masking job is initiated. You can query the masking job ID to retrieve status of the masking job.
Initiate a custom masking task from a file
As a Test Data Engineer, you can initiate a masking task with a custom configuration file that you download from Fast Data Masker. With this endpoint, you can execute masking tasks you create in FDM, with TDM Portal's scalable masking architecture.
 Files that you download from FDM are in plain text. You can encode plain text to BASE64 with a number of text editors, for example Notepad++.
 
Syntax
 
POST http://<server>:<host>/TDMMaskingService/api/ca/v1/masking/jobs/startCustom
 
Parameters
 
  •  
    projId
    Specifies the project ID on which to execute the masking task.
  •  
    pverId
    Specifies the project version ID on which to execute the masking task.
  •  
    customConnectionFile
    Connection file, encoded to BASE64. For more information on Connection Files, see Use, Create, and Manage Connection Files. You can download this file with the download_fdm_config endpoint.
     
    Sample customConnectionFile
    datasource=10.0.2.5
    username=sa
    epassword=XnKS5U=
    DBMS=SQLSERVER
    defaultschema=dbo
    database=fdm
    port=1433
    host=host_name
    forcedencryption=Y
  •  
    customConfigFile
    Masking configuration CSV file, encoded to BASE64. You can download this file with the download_fdm_configendpoint.
     
    Sample customConfigFile
     
     
    Table,Column,Function,Parm1,Parm2,Parm3,Parm4,Keep Nulls,Date Format,Cross Reference,Override SQL,Unique Columns,XPath Element,Substr start,Substr length,Notes,Preformat,Update,Use Masked Values,Restart Column,From Occurance,To Occurance,Parm5
    address,street,HASHLOV,VEGETABLE PRODUCE,1,,,Y,,,,,,,,,,,,,,,
  •  
    customSeedConnectionFile (optional)
    Seedtable connection file, encoded to BASE64.
     
    Sample SeedConnectionFile
    datasource=win10-sql14
    username=sa
    epassword=Z+Lcfuij=
    DBMS=SQLSERVER
    defaultschema=dbo
    database=Scramble
    port=1433
    forcedencryption=Y
  •  
    customOptionsFile (optional)
    Options file, encoded to BASE64.
     
    Sample customOptionsFile
    SEEDTABLE=gtsrc_reference_lov1
    SEEDTABLECOLUMNS=rl_ref_id,rl_ref_value,rl_ref_value2,rl_ref_value3,rl_ref_value4,rl_ref_value5,rl_ref_value6,rl_ref_value7,rl_ref_value8,rl_ref_value9,rl_ref_value10,rl_ref_value11,rl_ref_value12,rl_ref_value13,rl_ref_value14,rl_ref_value15,rl_ref_value16,rl_ref_value17,rl_ref_value18,rl_ref_value19,rl_ref_value20,rl_ref_value21,rl_ref_value22,rl_ref_value23,rl_ref_value24,rl_ref_value25,rl_ref_value26,rl_ref_value27,rl_ref_value28,rl_ref_value29,rl_ref_value30,rl_rn,rl_rn1
 
Example
 
POST http://host:8443/TDMMaskingService/api/ca/v1/masking/jobs/startCustom
  •  
    REQUEST BODY:
     
    {
    "projId" : 2346,
    "pverId" : 2347,
    "customConnectionFile": "<customConnectionFile encoded into BASE64>",
    "customConfigFile": "<customConfigFile encoded into BASE64>",
    "customSeedConnectionFile": "<customSeedConnectionFile encoded into BASE64>",
    "customOptionsFile":"<customOptionsFile encoded into BASE64>"
    }
  •  
    RESPONSE:
    200 OK
  •  
    RESPONSE BODY:
     
    {
    "jobId": 229,
    "projId": 2346,
    "pverId": 2347,
    "jobName": "Custom masking for FDM [ 1.2 ]",
    "environmentId": 0,
    "previewMode": false,
    "storePreSamples": false,
    "autoHandleConstraints": false,
    "confirmedOnly": false,
    "excNotPii": false,
    "customConnectionFile": "<customConnectionFile encoded into BASE64>",
    "customConfigFile": "<customConfigFile encoded into BASE64>",
    "customSeedConnectionFile": "<customSeedConnectionFile encoded into BASE64>",
    "customOptionsFile":"<customOptionsFile encoded into BASE64>",
    "customMasking": true
    }
Retrieve Status of a Job 
After initiating a job, a Test Data Engineer (TDE) can query the job ID to retrieve status of the job. The "status" parameter indicates the status of a job.
 
Syntax
 
GET http://<server>:<host>/TDMMaskingService/api/ca/v1/masking/jobs/{jobId}
 
Parameters
 
  •  
    jobId
     (path, long)
     
 
Example
 
GET http://host:8443/TDMMaskingService/api/ca/v1/masking/jobs/
507
  •  
    RESPONSE:
    200 OK
  •  
    RESPONSE BODY:
     
    {
    "name": "PII Data Scan",
    "jobId":
    507
    ,
    "description": "Masksing Project: 2378 Version: 2379",
    "projectName": "fresh",
    "projectId": 0,
    "versionId": null,
    "createdBy": "Administrator",
    "email": null,
    "scheduledTime": "2018-05-09T15:07:57Z",
    "startTime": "2018-05-09T15:07:58Z",
    "endTime": null,
    "status": "Running"
    ,
    "type": "PIIMASK",
    "parentId": 0,
    "jobs": [],
    "duration": null,
    "artifactLocation": null,
    "origin": "masking",
    "parameters": {},
    "statusMessage": null,
    "runningStatus": "Running",
    "created": null
    }
APIs related to Mask Function Groups
Retrieve a List of Mask Function Groups for a Project Version
As a TDE, you can retrieve a list of all mask function groups in a project version.
 
Syntax
 
GET http://<server>:<host>/TDMModelService/api/ca/v1/datamodel/maskConfigurations
 
Parameters
 
  •  
    projectId 
    (query, long)Project ID.
  •  
    versionId 
    (query, long)Version ID.
  • (Optional) 
    attributeId 
    (query, long)Attribute ID.If attributeId is provided, the API only returns mask function groups that have a tag the same as the attribute's primary tag. 
  • (Optional) 
    (query, long)RSQL format (see https://github.com/jirutka/rsql-parser ). Filter the query on 'tagName', 'maskGroupId', 'maskGroupLabel', 'maskGroupShared', 'classifierBased'.
     
 
Example
 
GET http://host:8443/TDMModelService/api/ca/v1/datamodel/maskConfigurations?projectId=1&versionId=1
  •  
    RESPONSE BODY:
     
{ "elements":
[
{
"maskGroupId": 20720235,
"maskGroupLabel": "Bank Account Number (Germany)",
"tagName": "Bank Account Number",
"maskGroupShared": true,
"classifierBased": true
},
{
"maskGroupId": 20729988,
"maskGroupLabel": "Bank Account Number (Sweden)",
"tagName": "Bank Account Number",
"maskGroupShared": true,
"classifierBased": true
},
{
"maskGroupId": 20730908,
"maskGroupLabel": "Sort Code (UK)",
"tagName": "Bank Sort Code",
"maskGroupShared": true,
"classifierBased": true
}
],
"numberOfElements": 3,
"totalElements": 3
}
Add Mask Function Group to a Project Version
As a TDE, you can add a mask function group to a project version.
 
Syntax
 
POST http://<server>:<host>/TDMModelService/api/ca/v1/datamodel/maskConfigurations
 
Parameters
 
  •  
    projectId 
    (query, long)Project ID.
  •  
    versionId 
    (query, long)Version ID.
  •  
    maskFunctionGroup 
    (body, maskFunctionGroup)Mask Function Group to add.
 
Example
 
POST http://host:8443/TDMModelService/api/ca/v1/datamodel/maskConfigurations?projectId=1&versionId=1&maskFunctionGroup={"maskGroupLabel": "My Surname Mask Function Group","maskGroupShared": false,"tagName": "Surname","configuration": [ {"maskFunctionName": "HASHLOV","displayName": "Surname UK","maskFunctionParams": [{"pos": "1","value": "uknames.txt"}]}]}
  •  
    RESPONSE BODY:
     
{
"maskGroupId": 20755001,
"maskGroupLabel": "My Surname Mask Function Group",
"tagName": "Surname", "maskGroupShared": false,
"classifierBased": false,
"configuration": [
{
"maskFunctionId": 20755002,
"maskFunctionLabel": "HASHLOV (uknames.txt)",
"maskFunctionName": "HASHLOV",
"displayName": "Surname UK",
"maskFunctionParams": [
{
"pos": 1,
"value": "uknames.txt"
}
]
}
]
}
Get Details of a Mask Function Group
As a TDE, you can retrieve details of a mask function group.
 
Syntax
 
GET http://<server>:<host>/TDMModelService/api/ca/v1/datamodel/maskConfigurations/{maskFunctionGroupId}
 
Parameters
 
  •  
    maskFunctionGroupId 
    (path, long)Id of Mask Function Group for which to get details.
     
     
 
Example
 
GET http://host:8443/TDMModelService/api/ca/v1/datamodel/maskConfigurations/20755001
  •  
    RESPONSE BODY:
     
{
"maskGroupId": 20755001,
"maskGroupLabel": "My Surname Mask Function Group",
"tagName": "Surname",
"maskGroupShared": false,
"classifierBased": false,
"configuration": [
{
"maskFunctionId": 20755002
"maskFunctionLabel": "HASHLOV (uknames.txt)",
"maskFunctionName": "HASHLOV",
"displayName": "Surname UK",
"maskFunctionParams": [
{
"pos": 1,
"value": "uknames.txt"
}
]
}
]
}
Update Details of a Mask Function Group
As a TDE, you can update details of a mask function group. You submit a new maskFunctionGroup object as the parameter.
 
Syntax
 
PATCH http://<server>:<host>/TDMModelService/api/ca/v1/datamodel/maskConfigurations/{maskFunctionGroupId}
 
Parameters
 
  •  
    maskFunctionGroupId 
    (path, long)Id of Mask Function Group for which to get details.
     
     
  •  
    maskFunctionGroup 
    (body, maskFunctionGroup)Mask Function Group to update to maskFunctionGroupId.
 
Example
 
PATCH http://host:8443/TDMModelService/api/ca/v1/datamodel/maskConfigurations/20755001?maskFunctionGroup={"maskGroupLabel": "My Updated Surname Mask Function Group","maskGroupShared": true,"tagName": "Surname","configuration": [ {"maskFunctionName": "HASHLOV","displayName": "Surname France","maskFunctionParams": [{"pos": "1","value": "frenchnames.txt"}]}]}
  •  
    RESPONSE BODY:
     
{
"maskGroupId": 20755001,
"maskGroupLabel": "My Updated Surname Mask Function Group",
"tagName": "Surname",
"maskGroupShared": true,
"classifierBased": false,
"configuration": [
{
"maskFunctionId": 20755004,
"maskFunctionLabel": "HASHLOV (frenchnames.txt)",
"maskFunctionName": "HASHLOV",
"displayName": "Surname France",
"maskFunctionParams": [
{
"pos": 1,
"value": "frenchnames.txt"
}
]
}
]
}
Delete a Mask Function Group
As a TDE, you can delete a user-defined mask function group. If you try to delete a mask function group from a classifier, the API returns a 'Forbidden' error.
 
Syntax
 
DELETE http://<server>:<host>/TDMModelService/api/ca/v1/datamodel/maskConfigurations/{maskFunctionGroupId}
 
Parameters
 
  •  
    maskFunctionGroupId 
    (path, long)Id of Mask Function Group to delete.
     
     
 
Example
 
DELETE http://host:8443/TDMModelService/api/ca/v1/datamodel/maskConfigurations/20755001
  •  
    RESPONSE:
     
 
true
 
Masking configurations by entities (tables) and attributes (columns)
Get list of attributes linked to a masking configuration in a project version
As a TDE, you can retrieve a list of attributes (columns) in a project version, that use the same masking configuration.
 
Syntax
 
GET http://host:8443/TDMModelService/api/ca/v1/datamodel/maskConfigurations/{maskFunctionGroupId}/attributes
 
Parameters
 
  •  
    maskFunctionGroupId 
    (path, long)Id of Mask Function Group for which to get list of linked attributes.
     
     
     
     
  •  
    projectId 
    (query, long)Project ID.
     
     
  •  
    versionId 
    (query, long)Version ID.
 
Example
 
GET http://host:8443/TDMModelService/api/ca/v1/datamodel/maskConfigurations/20755001/attributes?projectId=1&versionId=1
  •  
    RESPONSE BODY:
     
[
{
"attributeId": 91280,
"attributeName": "WEB_USER_PASSWORD",
"primaryTag": "Surname",
"entityName": "ACCESS_CONTROLS",
"dataSource": "ds2",
"databaseName": "travel2",
"schemaName": "dbo"
},
{
"attributeId": 91353,
"attributeName": "MONTH_NAME",
"primaryTag": "Surname",
"entityName": "ACCOUNT_PERIODS",
"dataSource": "ds2",
"databaseName": "travel2",
"schemaName": "dbo"
},
{
"attributeId": 91460,
"attributeName": "DESCRIPTION",
"primaryTag": "Surname",
"entityName": "AIRPORTS",
"dataSource": "ds2",
"databaseName": "travel2",
"schemaName": "dbo"
}
]
Get a list of masking configurations for tables that contain PII, for a project version
As a TDE, you can retrieve a list of masking configurations for all tables that contain PII (entities) in a project version.
 
Syntax
 
GET http://<server>:<host>/TDMModelService/api/ca/v1/datamodel/entities/maskConfigurations
 
Parameters
 
  •  
    projectId 
    (query, long)Project ID.
     
     
  •  
    versionId 
    (query, long)Version ID.
 
Example
 
GET http://host:8443/TDMModelService/api/ca/v1/datamodel/entities/maskConfigurations?projectId=1&versionId=
  •  
    RESPONSE BODY:
     
[
{
"attributeId": 91748,
"attributeName": "NAME",
"primaryTag": "Country",
"numOtherTags": 1,
"maskGroupId": 20719929,
"maskGroupLabel": "Country",
"hasWhereClause": false,
"maskGroupShared": true,
"classifierBased": true,
"dataType": "char"
},
{
"attributeId": 91753,
"attributeName": "CURRENCY_DESCRIPTION",
"primaryTag": "Country",
"numOtherTags": 2,
"maskGroupId": 20719929,
"maskGroupLabel": "Country",
"hasWhereClause": false,
"maskGroupShared": true,
"classifierBased": true,
"dataType": "char"
},
{
"attributeId": 91752,
"attributeName": "FLAG_PHOTO_FILENAME",
"primaryTag": "Country",
"numOtherTags": 2,
"maskGroupId": 20719929,
"maskGroupLabel": "Country",
"hasWhereClause": false,
"maskGroupShared": true,
"classifierBased": true,
"dataType": "char"
}
]
Get a list of attributes and their associated masking configurations, for tables that contain PII
As a TDE, you can retrieve a list of attributes (columns) that contain PII, for a table that contains PII (an entity).
 
Syntax
 
GET http://<server>:<host>/TDMModelService/api/ca/v1/datamodel/entities/{entityId}/maskConfigurations
 
Parameters
 
  •  
    entityId 
    (path, long)
  •  
    projectId 
    (query, long)
  •  
    versionId 
    (query, long)
 
Example
 
GET http://host:8443/TDMModelService/api/ca/v1/datamodel/entities/1234/maskConfigurations?projectId=1&versionId=1
  •  
    RESPONSE BODY:
     
[
{
"attributeId": 91748,
"attributeName": "NAME",
"primaryTag": "Country",
"numOtherTags": 1,
"maskGroupId": 20719929,
"maskGroupLabel": "Country",
"hasWhereClause": false,
"maskGroupShared": true,
"classifierBased": true,
"dataType": "char"
},
{
"attributeId": 91753,
"attributeName": "CURRENCY_DESCRIPTION",
"primaryTag": "Country",
"numOtherTags": 2,
"maskGroupId": 20719929,
"maskGroupLabel": "Country",
"hasWhereClause": false,
"maskGroupShared": true,
"classifierBased": true,
"dataType": "char"
},
{
"attributeId": 91752,
"attributeName": "FLAG_PHOTO_FILENAME",
"primaryTag": "Country",
"numOtherTags": 2,
"maskGroupId": 20719929,
"maskGroupLabel": "Country",
"hasWhereClause": false,
"maskGroupShared": true,
"classifierBased": true,
"dataType": "char"
}
]
Get the current masking configuration for an attribute, for a project version
As a TDE, you can retrieve the masking configuration for an atttribute (column), in a project version.
 
Syntax
 
GET http://<server>:<host>/TDMModelService/api/ca/v1/datamodel/entities/{entityId}/attributes/{attributeId}/maskConfigurations
 
Parameters
 
  •  
    attributeId
     (path, long)
  •  
    entityId 
    (path, long)
  •  
    projectId 
    (query, long)
  •  
    versionId 
    (query, long)
 
Example
 
GET http://host:8443/TDMModelService/api/ca/v1/datamodel/entities/1234/attributes/91748/maskConfigurations?projectId=1&versionId=1
  •  
    RESPONSE BODY:
     
{
"maskGroupId": 20719929,
"maskGroupLabel": "Country",
"tagName": "Country",
"maskGroupShared": true,
"classifierBased": true,
"notes": "Created from classifier import",
"configuration":
[
{
"maskFunctionId": 20719930,
"maskFunctionLabel": "HASHLOV (country.txt)",
"maskFunctionName": "HASHLOV",
"displayName": "Countries",
"maskFunctionParams":
[
{
"pos": 1,
"value": "country.txt"
}
],
"notes": "Countries derived from a hashed index into a lookup-table"
}
]
}
Set the mask function group of an attribute for a project version
As a TDE, you can set the masking function group for an attribute (column), in a project version.
 
Syntax
 
POST http://<server>:<host>/TDMModelService/api/ca/v1/datamodel/entities/{entityId}/attributes/{attributeId}/maskConfigurations
 
Parameters
 
  •  
    maskFunctionGroupId 
    (body, JSON)Id of Mask Function Group to assign to an attribute (attributeId).
  •  
    attributeId
     (path, long)
  •  
    entityId 
    (path, long)
  •  
    projectId 
    (query, long)
  •  
    versionId 
    (query, long)
 
Example
 
POST http://host:8443/TDMModelService/api/ca/v1/datamodel/entities/1234/attributes/91748/maskConfigurations?projectId=1&versionId=1
  •  
    BODY TEXT:
     
{"maskGroupId" : 20719929}
  •  
    RESPONSE
     
    BODY:
     
{
"maskGroupId": 20719929,
"maskGroupLabel": "Country",
"tagName": "Country",
"maskGroupShared": true,
"classifierBased": true,
"notes": "Created from classifier import",
"configuration":
[
{
"maskFunctionId": 20719930,
"maskFunctionLabel": "HASHLOV (country.txt)",
"maskFunctionName": "HASHLOV",
"displayName": "Countries",
"maskFunctionParams":
[
{
"pos": 1,
"value": "country.txt"
}
],
"notes": "Countries derived from a hashed index into a lookup-table"
}
]
}
Update the mask function group of an attribute, for a project version
As a TDE, you can update the mask function group of an attribute (column), in a project version.
 If the previous mask function group of the attribute is not from a classifier, not marked as 'shared', and not linked to any other attributes, this API deletes it.
 
Syntax
 
PATCH http://<server>:<host>/TDMModelService/api/ca/v1/datamodel/entities/{entityId}/attributes/{attributeId}/maskConfigurations
 
Parameters
 
  •  
    maskFunctionGroupId 
    (body, JSON)Id of Mask Function Group with which to update attribute (attributeId).
  •  
    attributeId
     (path, long)
  •  
    entityId 
    (path, long)
  •  
    projectId 
    (query, long)
  •  
    versionId 
    (query, long)
 
Example
 
PATCH http://host:8443/TDMModelService/api/ca/v1/datamodel/entities/1234/attributes/91748/maskConfigurations?projectId=1&versionId=1
  •  
    BODY TEXT:
     
{"maskGroupId" : 20719929}
  •  
    RESPONSE BODY:
     
{
"maskGroupId": 20719929,
"maskGroupLabel": "Country",
"tagName": "Country",
"maskGroupShared": true,
"classifierBased": true,
"notes": "Created from classifier import",
"configuration":
[
{
"maskFunctionId": 20719930,
"maskFunctionLabel": "HASHLOV (country.txt)",
"maskFunctionName": "HASHLOV",
"displayName": "Countries",
"maskFunctionParams":
[
{
"pos": 1,
"value": "country.txt"
}
],
"notes": "Countries derived from a hashed index into a lookup-table"
}
]
}
Remove current mask function group from an attribute, for a project version
As a TDE, you can remove the mask function group that an attribute (column) uses, in a project version.
 This action has the same effect as changing the mask function group to 'Do Not Mask' for a column in the Configure Data Masking section of the Portal UI.
 
Syntax
 
DELETE http://<server>:<host>/TDMModelService/api/ca/v1/datamodel/entities/{entityId}/attributes/{attributeId}/maskConfigurations
 
Parameters
 
  •  
    attributeId
     (path, long)
  •  
    entityId 
    (path, long)
  •  
    projectId 
    (query, long)
  •  
    versionId 
    (query, long)
 
Example
 
DELETE http://host:8443/TDMModelService/api/ca/v1/datamodel/entities/1234/attributes/91748/maskConfigurations
  •  
    RESPONSE
    :
 
true
 
Masking configurations by tag
Get a list of mask function groups and attributes
As a TDE, you can get a hierarchical view on the current masking function groups sorted by tag. Each tag that is in the discovered PII is returned in the top-level object. Nested in this object is the current masking group list and their associated attributes, a list of attributes with this tag assigned but not configured to be masked and a list of all mask function groups associated with this tag.
If all mask function groups for the contained masking group list are the same, the effective mask group name and ID are populated into the top-level object.
 
Syntax
 
GET http://<server>:<host>/TDMModelService/api/ca/v1/datamodel/tags/maskConfigurations
 
Parameters
 
  •  
    projectId 
    (query, long)
  •  
    versionId 
    (query, long)
 
Example
 
GET http://host:8443/TDMModelService/api/ca/v1/datamodel/tags/maskConfigurations?projectId=1&versionId=1
  •  
    RESPONSE BODY:
     
For brevity, the full response body is not shown.
{
"elements":
[
{
"tagId": 5021,
"tagName": "Country",
"effectiveMaskGroup": "Country",
"effectiveMaskGroupID": 5022,
"maskingGroupCount": 2,
"currentMaskingGroupList": [ ],
"unmaskedAttributes": [],
"knownMaskingGroupsList": []
},
<further tags listed with the same format>
]
}
Get a list of mask function groups and attributes for a tag
As a TDE, you can get a hierarchical view on the current masking function groups for a single tag.
 
Syntax
 
GET http://<server>:<host>/TDMModelService/api/ca/v1/datamodel/tags/{tagId}/maskConfigurations
 
Parameters
 
  •  
    tagId
     (path, long)
  •  
    projectId 
    (query, long)
  •  
    versionId 
    (query, long)
 
Example
 
GET http://host:8443/TDMModelService/api/ca/v1/datamodel/tags/5021/maskConfigurations?projectId=1&versionId=1
  •  
    RESPONSE BODY:
     
{
"tagId": 5021,
"tagName": "Country",
"effectiveMaskGroup": "Country",
"effectiveMaskGroupID": 5022,
"maskingGroupCount": 2,
"currentMaskingGroupList": [ ],
"unmaskedAttributes": [],
"knownMaskingGroupsList": []
}
Remove mask function groups for all attributes with a tag
As a TDE, you can remove all masking functions associated with a tag.
 
Syntax
 
DELETE http://<server>:<host>/TDMModelService/api/ca/v1/datamodel/tags/{tagId}/maskConfigurations
 
Parameters
 
  •  
    tagId
     (path, long)
  •  
    projectId 
    (query, long)
  •  
    versionId 
    (query, long)
 
Example
 
DELETE http://host:8443/TDMModelService/api/ca/v1/datamodel/tags/5021/maskConfigurations?projectId=1&versionId=1
  •  
    RESPONSE BODY:
     
 
true
 (Response Code 204)
Add existing mask function group to a tag
As a TDE, you can set an existing mask function group to all attributes associated with a tag. This replaces any existing mask function group associated with any of these attributes.
 
Syntax
 
POST http://<server>:<host>/TDMModelService/api/ca/v1/datamodel/tags/{tagId}/maskConfigurations
 
Parameters
 
  •  
    maskFunctionGroupId 
    (body, maskFunctionGroup)Id of Mask Function Group with which to update tag (tagId), as maskFunctionGroup object.
     
  •  
    tagId
     (path, long)
  •  
    projectId 
    (query, long)
  •  
    versionId 
    (query, long)
 
Example
 
POST http://host:8443/TDMModelService/api/ca/v1/datamodel/tags/5021/maskConfigurations?projectId=1&versionId=1
  •  
    Body text:
     
{"maskGroupId" : 20719929}
  •  
    RESPONSE BODY:
     
 
true
 (Response Code 204)
Add new mask function group to a tag
As a TDE, you can create a new mask function group and assign it to all attributes associated with a tag.
 
Syntax
 
PUT http://<server>:<host>/TDMModelService/api/ca/v1/datamodel/tags/{tagId}/maskConfigurations
 
Parameters
 
  •  
    maskFunctionGroup 
    (body, maskFunctionGroup)
    Mask Function Group with which to update tag (tagId), as a maskFunctionGroup pbject.
     
  •  
    tagId
     (path, long)
  •  
    projectId 
    (query, long)
  •  
    versionId 
    (query, long)
 
Example
 
PUT http://host:8443/TDMModelService/api/ca/v1/datamodel/tags/5021/maskConfigurations?projectId=1&versionId=1
  •  
    Body text:
     
{
"maskGroupLabel":"test label",
"tagName":"Country",
"maskGroupShared":true,
"classifierBased":false,
"configuration":
[
{
"maskFunctionName":"HASHLOV",
"displayName":"TestMaskFunction",
"maskFunctionParams":
[
{
"pos":1,
"value":"uktowns.txt"
}
]
}
]
}
  •  
    RESPONSE BODY:
     
{
"maskGroupId": 20719955,
"maskGroupLabel": "test label",
"tagName": "Country",
"maskGroupShared": true,
"classifierBased": false,
"notes": "",
"configuration":
[
{
"maskFunctionId": 20719930,
"maskFunctionLabel": "HASHLOV (uktowns.txt)",
"maskFunctionName": "HASHLOV",
"displayName": "TestMaskFunction",
"maskFunctionParams":
[
{
"pos": 1,
"value": "uktowns.txt"
}
],
"notes": ""
}
]
}
Remove specific mask function groups from attributes with a specific tag
As a TDE, you can remove specific mask function groups from a tag. Use this API to remove masking for all attributes with the specified tag ID that are masked with the specified mask function group.
 
Syntax
 
DELETE http://<server>:<host>/TDMModelService/api/ca/v1/datamodel/tags/{tagId}/maskConfigurations/{maskFunctionGroupId}
 
Parameters
 
  •  
    maskFunctionGroupId 
    (path, long)
    Id of Mask Function Group to remove from attributes associated with tagId.
     
  •  
    tagId
     (path, long)
  •  
    projectId 
    (query, long)
  •  
    versionId 
    (query, long)
 
Example
 
DELETE http://<server>:<host>/TDMModelService/api/ca/v1/datamodel/tags/5021/maskConfigurations/25001?projectVersion=1&versionId=1
  •  
    RESPONSE:
     
 
true
 (Response Code 200)
Get a mask function group
As a TDE, you can get the description of a specific mask function group associated with a tag.
 
Syntax
 
GET http://<server>:<host>/TDMModelService/api/ca/v1/datamodel/tags/{tagId}/maskConfigurations/{maskFunctionGroupId}
 
Parameters
 
  •  
    maskFunctionGroupId 
    (path, long)
    Id of Mask Function Group for which to retrieve details.
     
  •  
    tagId
     (path, long)
  •  
    projectId 
    (query, long)
  •  
    versionId 
    (query, long)
 
Example
 
GET http://host:8443/TDMModelService/api/ca/v1/datamodel/tags/5021/maskConfigurations/25001?projectVersion=1&versionId=1
  •  
    RESPONSE BODY:
     
{
"maskGroupId": 25001,
"maskGroupLabel": "test label",
"tagName": "Country",
"maskGroupShared": true,
"classifierBased": false,
"notes": "",
"configuration":
[
{
"maskFunctionId": 20719930,
"maskFunctionLabel": "HASHLOV (uktowns.txt)",
"maskFunctionName": "HASHLOV",
"displayName": "TestMaskFunction",
"maskFunctionParams":
[
{
"pos": 1,
"value": "uktowns.txt"
}
],
"notes": ""
}
]
}
Make changes to a mask function group
As a TDE, you can update a mask function group associated with a tag.
 
Syntax
 
PATCH http://<server>:<host>/TDMModelService/api/ca/v1/datamodel/tags/{tagId}/maskConfigurations/{maskFunctionGroupId}
 
Parameters
 
  •  
    maskFunctionGroupId 
    (path, long)
    Id of Mask Function Group to which to make changes based on maskFunctionGroup object.
     
  •  
    maskFunctionGroup
    (body, object)
    A maskFunctionGroup object (can be incomplete). CA TDM amends attributes of the Mask Function Group defined by maskFunctionGroupId, with whichever attributes you supply in this object.
  •  
    tagId
     (path, long)
  •  
    projectId 
    (query, long)
  •  
    versionId 
    (query, long)
 
Example
 
PATCH http://host:8443/TDMModelService/api/ca/v1/datamodel/tags/5021/maskConfigurations/25001?projectVersion=1&versionId=1
  •  
    BODY TEXT:
     
{"maskGroupLabel" : "new label for function"}
  •  
    RESPONSE BODY:
     
{
"maskGroupId": 25001,
"maskGroupLabel": "new label for function"
,
"tagName": "Country",
"maskGroupShared": true,
"classifierBased": false,
"notes": "",
"configuration":
[
{
"maskFunctionId": 20719930,
"maskFunctionLabel": "HASHLOV (uktowns.txt)",
"maskFunctionName": "HASHLOV",
"displayName": "TestMaskFunction",
"maskFunctionParams":
[
{
"pos": 1,
"value": "uktowns.txt"
}
],
"notes": ""
}
]
}
  •  
    RESPONSE:
     
 
true
 (Response Code 200)
Replace the mask function group assigned with an existing group
As a TDE, you can replace the mask function group to all attributes associated with a tag and an existing group ID.
 
Syntax
 
PUT http://<server>:<host>/TDMModelService/api/ca/v1/datamodel/tags/{tagId}/maskConfigurations/{groupId}
Parameters
  •  
    projectId
     (query, long)
  •  
    versionId
     (query, long)
  •  
    tagId
     (path, long)
  •  
    groupId
     (path,long)
    Mask function group ID
  •  
    maskFunctionGroup
     (body, object)
    The new mask function group object
 
Example:
 
PUT http://host:8443/TDMModelService/api/ca/v1/datamodel/tags/5021/maskConfigurations/25001?projectVersion=1&versionId=1
  •  
    BODY TEXT:
     
{
"maskGroupLabel":"new group label",
"tagName":"Country",
"maskGroupShared":true,
"classifierBased":false,
"configuration":
[
{
"maskFunctionName":"HASHLOV",
"displayName":"hashlov uk towns",
"maskFunctionParams":
[
{
"pos":1,
"value":"uktowns.txt"
}
]
}
]
}
  •  
    RESPONSE BODY:
     
{
"maskGroupId": 25001,
"maskGroupLabel": "new group label",
"tagName": "Country",
"maskGroupShared": true,
"classifierBased": false,
"notes": "",
"configuration":
[
{
"maskFunctionId": 20719930,
"maskFunctionLabel": "HASHLOV (uktowns.txt)",
"maskFunctionName": "HASHLOV",
"displayName": "hashlov uk towns",
"maskFunctionParams":
[
{
"pos": 1,
"value": "uktowns.txt"
}
],
"notes": ""
}
]
}
Retrieve a List of Masking Functions
A Test Data Engineer (TDE) can retrieve a list of available masking functions provided by Fast Data Masker.
 
Syntax
 
GET https://<server>:<host>/TDMMaskingService/api/ca/v1/masking/functions
 
Parameters
 
  • (Optional)
     dataType
    Filters the masking functions based on data type.
    Values: char, number, date, char_date
  • (Optional)
     functionName
    Filters the masking functions based on full or partial function names.
    For example, 'ACCT' will match function name 'ACCT_01'
  •  Pagination parameters.
 
Example
 
GET https://host:8443/TDMMaskingService/api/ca/v1/masking/functions?page=0&size=2
  •  
    RESPONSE:
    200 OK
  •  
    RESPONSE BODY:
     
    {
    "elements":{
    "ACCT_01":{
    "id":"1",
    "name":"ACCT_01",
    "description":"ACCT_01 - replace digits 0..9 in original",
    "parm1":"Digits to replace",
    "parm2":"",
    "parm3":"",
    "parm4":"",
    "char":"true",
    "number":"true",
    "date":"false",
    "char_date":"false"
    },
    "ADD":{
    "id":"2",
    "name":"ADD",
    "description":"ADD - Add a fixed value",
    "parm1":"Fixed Value",
    "parm2":"",
    "parm3":"",
    "parm4":"",
    "char":"true",
    "number":"true",
    "date":"true",
    "char_date":"true"
    }
    },
    "numberOfElements":2,
    "totalElements":101
    }
Retrieve Details of a Masking Function 
A Test Data Engineer (TDE) can retrieve details about a specific masking function. 
 
Syntax
 
GET https://<server>:<host>/TDMMaskingService/api/ca/v1/masking/functions/{functionId}
 
Parameters
 
  •  
    functionId
     
 
Example
 
GET https://host:8443/TDMMaskingService/api/ca/v1/masking/functions/
1
  •  
    RESPONSE:
    200 OK
  •  
    RESPONSE BODY:
     
    "ACCT_01":{
    "id":"
    1
    ",
    "name":"ACCT_01",
    "description":"ACCT_01 - replace digits 0..9 in original",
    "parm1":"Digits to replace",
    "parm2":"",
    "parm3":"",
    "parm4":"",
    "char":"true",
    "number":"true",
    "date":"false",
    "char_date":"false"
    },
Retrieve a List of Masking Seed Lists
As a Test Data Engineer (TDE), you can retrieve a list of masking seed lists available. This helps you in choosing the appropriate seed list for the masking function to be used.
 
Syntax
 
GET https://<server>:<host>/TDMMaskingService/api/ca/v1/masking/seedlists
 
Parameters
 
 
Example
 
GET https://host:8443/TDMMaskingService/api/ca/v1/masking/seedlists?size=2&page=6
  •  
    RESPONSE:
    200 OK
  •  
    RESPONSE BODY:
     
    {
    "elements": [
    {
    "id": 13,
    "name": "firstnamemaleamerican.txt",
    "description": "American Male First Name"
    },
    {
    "id": 14,
    "name": "firstnames.txt",
    "description": "First Names"
    }
    ],
    "numberOfElements": 2,
    "totalElements": 66
    }
Retrieve Details of a Seed List
As a Test Data Engineer (TDE), you can retrieve details about a single seed list. 
 
Syntax
 
GET https://<server>:<host>/TDMMaskingService/api/ca/v1/masking/seedlists/{seedlistId}
 
Parameters
 
  •  
    seedlistId
    ID of the seedlist for which to return details
  •  Pagination parameters
 
Example
 
GET https://host:8443/TDMMaskingService/api/ca/v1/masking/seedlists/
13
  •  
    RESPONSE:
    200 OK
  •  
    RESPONSE BODY:
     
    {
    "id":
    13
    ,
    "name": "firstnamemaleamerican.txt",
    "description": "American Male First Name"
    }
Download Masking Audit Files 
A Test Data Engineer (TDE) can download masking audit details to see what data has been masked and view samples of the masked data. The masking audit file is downloaded as a zip file and includes the project version, environment Id, when the masking job was started, when the masking job was completed and so on.
 
Syntax
 
GET https://<server>:<host>/TDMMaskingService/api/ca/v1/masking/jobs/{jobId}/audit
 
Parameters
 
  •  
    jobId
    ID of the job for which to return masking audit details
 
Example
 
GET https://host:8443/TDMMaskingService/api/ca/v1/masking/jobs/1/audit
  •  
    RESPONSE:
    200 OK
You have downloaded the masking audit file. You can use this to understand the data that is masked and view samples of the masked data.