Digital Operational Intelligence Plugin (DOI Plugin) Connector Guide 10.7 SP3 HF35

You can use the DOI Plugin to get a comprehensive insight into the performance of your infrastructure and applications in your environment.DOI Plugin integrates CA Application Performance Management (CA APM) with CA Digital Operational Intelligence (CA DOI). The DOI Plugin enables you to automatically ingest data from CA APM to CA DOI. The ingested data spans across your performance monitoring tools and any third-party sources into a single extensive data lake.
apmdevops106
Overview
You can use the DOI Plugin to get a comprehensive insight into the performance of your infrastructure and applications in your environment.
DOI Plugin integrates CA Application Performance Management (CA APM) with CA Digital Operational Intelligence (CA DOI). The DOI Plugin enables you to automatically ingest data from CA APM to CA DOI. The ingested data spans across your performance monitoring tools and any third-party sources into a single extensive data lake.
 
 
3
 
 
How CA APM interacts with CA DOI?
DOI Plugin helps you push the following details from CA APM to CA JARVIS - metrics, alerts, and inventory data, which includes information pertaining to vertices. Further, the DOI Plugin enables CA DOI to consume the ingested CA APM data from CA Jarvis and displays the CA APM ingested data in CA DOI user interface. The CA DOI user interface displays the following data: metrics and alerts. CA DOI uses the alerts to provide various notification notifications such as email notifications, service alarms, and so on. CA DOI uses the inventory data for root cause analysis.
Installation and Configuration
To use the DOI Plugin, you need to perform the following prerequisite tasks.
Install CA APM 10.7 SP3
You need to install CA APM 10.7 SP3. When you install CA APM 10.7 SP3, the DOI Plugin is installed along with the build. For more information about how to install CA APM 10.7 SP3, see APM Installation.
Install CA DOI 1.3
You need to install CA DOI 1.3. For more information about how to install CA DOI 1.3, see Installing.
Onboard Transaction Traces Index
Display of the Transaction Traces in ATC is not
 
supported in a Hybrid deployment for CA APM on-premise and CA DOI on SaaS.
You can create the transaction traces, 
tt index
 (
ao_apm_tt_1_1
), on CA Jarvis by running the following command on the host on which EM is installed:
curl -v -H "Content-Type: application/json" -X POST "http://<<jarvis-Host>>:<<jarvis-port>>/onboarding/doc_type" -d @<<EM_HOM>>/jarvis/doc-type-traces.json
Configure DOI Plugin in CA APM 10.7 SP3
You can configure the DOI plugin in CA APM 10.7 SP3 by performing the following tasks. In the Enterprise Manager of CA APM 10.7 SP3, perform the following tasks:
  1. Navigate to the 
    Introscope folder/Config/IntroscopeEM.Properties
     file.
  2. Under AO Platform Integration Configuration section, uncomment and edit the details of the following properties:
    1. The default value of the following property is the tenantId:
      #apm.doi.tenantId=default
      For example, DOI99GB-USERSTORE
    2. Provide the details of the CA Jarvis host and port for ingestion as follows:
      #apm.doi.jarvis.ingestion.url=http://<apiHost>:<apiPort>/ingestion
      For example, 
      http://eajarvis/. sales05-osetest-vm07.devfco.com:8080/ingestion
       
    3. Provide the details of the CA Jarvis host and port for onboarding as follows:
      #apm.doi.jarvis.onboarding.url=http://<apiHost>:<apiPort>/onboarding
      For example, 
      http://eajarvis/. sales05-osetest-vm07.devfco.com:8080/onboarding
       
  3. Under APM OI Properties section, edit the details of the following properties:
    • (Hot Property) The default value of the following property is 
      false
      :
      introscope.enterprisemanager.doi.enabled=true
      You can edit the property value to true to push the metrics data from CA APM to CA DOI through CA Jarvis.
  4. Under OI Alert Configuration section, edit the details of the following hot properties:
    • The default value of the following property is 
      false
      introscope.enterprisemanager.doi.alert.enabled=true
    You can edit the property value to true to push alerts from CA APM to CA DOI through CA Jarvis.
    The alerts from CA APM are translated to alarms in CA DOI.
  5. Under the APM Inventory Properties section, edit the following details:
    • (Hot Property) The default value of the following property is 
      false
      :
    introscope.enterprisemanager.doi.inventory.enabled=true
    You can edit the property value to true to push the inventory data from CA APM to CA DOI through CA Jarvis.
    • Provide the CA APM WebView Host and Port details as follows:
      introscope.enterprisemanager.webview.url =http://<CA APM WebView Host>:<CA APM WebView Port>
    • (Hot Property) Provide the following inventory token:
      You need to provide a token to fetch the inventory data from CA APM.
      Follow these steps:
       
    1. Login CA APM as an Administrator
    2. From the menu, navigate to 
      Security
       
    3. In the Security page, click 
      Generate New Token
       
    4. In the New API Security Token page, provide an appropriate label in the 
      Label
       field
    5. In the New API Security Token page, ensure that the 
      Type
       is 
      Public API
       
    6. In the New API Security Token page, ensure that you the check the option, 
      Never Expires
       
    7. Click 
      Generate Token
      A new token is generated. 
    8. You copy the generated token and provide the newly generated token as a value in the following property:
      introscope.enterprisemanager.doi.inventory.token=
  6. Save and Close the 
    IntroscopeEM.Properties
     file.
    The metric grouping of the CA APM OI Plugin is pre-configured and provided as a part of the Default Management Module as a jar file, DefaultMM.jar. Ensure that the CA APM OI Metric Regex is configured and available as a part of the Default Management Module (DefaultMM.jar) in all Collectors and Mangers of Managers (MOM) in your environment. Consider that you modify the CA APM DOI Metric Regex. Then, ensure that you manually modify the same details in the Management Modules of all Collectors and MOM in your environment.
  7. Restart Enterprise Manager.
View CA APM Data in DOI
You can view the CA APM data in CA DOI in the form of metrics, alerts, and service alarms. For more information about how to view the data, see Using 
Troubleshooting DOI Plugin
Troubleshooting DOI Plugin section helps you solve problematic symptoms in your implementation.
Defining Non-Numeric Metric Names in CA APM
In CA APM WebView, when you click the Management tab, you can view the Management Modules in a tree structure in the left-hand side pane. The Management Modules displays metrics, metric groupings, alerts, and so on.
Navigate to the Management Modules, Default, Metric Groupings, APM OI. APM OI is a default metric grouping name. After clicking APM OI, the details are displayed in the Preview page. Under the Expressions section, the Metric Expressions displays a list of specifiers DOI Plugin provides out-of-the-box.
You can edit the metric expressions that are listed under the Expressions section. Typically, to view all CA APM metrics, in CA APM you can edit the specifier such as 
 
(.*)
 
. DOI contains an engine named as CA Jarvis. Note that CA Jarvis has a limitation to identify numeric metric expression and stops sending numeric metric names to be displayed in CA DOI. To view all CA APM metrics in CA DOI, replace 
 
(.*)
 
 with 
 
.*[^0-9]
 
.
Document Contains at least One Immense Term in Field
Problem:
 
If you find the error messages in the Jarvis verifier logs
 
Issue:Document contains at least one immense term in field, original message: bytes can be at most 32766 in length; got 33303","caused_by":{"type":"max_bytes_length_exceeded_exception","reason":"bytes can be at most 32766 in length; got 33303"}}
Here are the Jarvis verifier logs that represent the issue:
{"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"Document contains at least one immense term in field=\"avro_schema\" (whose UTF8 encoding is longer than the max length 32766), all of which were skipped. Please correct the analyzer to not produce such terms. The prefix of the first immense term is: '[123, 34, 116, 121, 112, 101, 34, 58, 34, 114, 101, 99, 111, 114, 100, 34, 44, 34, 110, 97, 109, 101, 34, 58, 34, 67, 65, 65, 95, 68]...', original message: bytes can be at most 32766 in length; got 33303"}],"type":"illegal_argument_exception","reason":"Document contains at least one immense term in field=\"avro_schema\" (whose UTF8 encoding is longer than the max length 32766), all of which were skipped. Please correct the analyzer to not produce such terms. The prefix of the first immense term is: '[123, 34, 116, 121, 112, 101, 34, 58, 34, 114, 101, 99, 111, 114, 100, 34, 44, 34, 110, 97, 109, 101, 34, 58, 34, 67, 65, 65, 95, 68]...', original message: bytes can be at most 32766 in length; got 33303","caused_by":{"type":"max_bytes_length_exceeded_exception","reason":"bytes can be at most 32766 in length; got 33303"}},"status":400} at org.elasticsearch.client.RestClient$1.completed(RestClient.java:357) ~[verifier.jar:?] at org.elasticsearch.client.RestClient$1.completed(RestClient.java:346) ~[verifier.jar:?] at org.apache.http.concurrent.BasicFuture.completed(BasicFuture.java:119) ~[verifier.jar:?] at org.apache.http.impl.nio.client.DefaultClientExchangeHandlerImpl.responseCompleted(DefaultClientExchangeHandlerImpl.java:177) ~[verifier.jar:?] at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.processResponse(HttpAsyncRequestExecutor.java:436) ~[verifier.jar:?] at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.inputReady(HttpAsyncRequestExecutor.java:326) ~[verifier.jar:?] at org.apache.http.impl.nio.client.InternalRequestExecutor.inputReady(InternalRequestExecutor.java:83) ~[verifier.jar:?] at org.apache.http.impl.nio.DefaultNHttpClientConnection.consumeInput(DefaultNHttpClientConnection.java:265) ~[verifier.jar:?] at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:81) ~[verifier.jar:?] at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:39) ~[verifier.jar:?] at org.apache.http.impl.nio.reactor.AbstractIODispatch.inputReady(AbstractIODispatch.java:114) ~[verifier.jar:?] at org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.java:162) ~[verifier.jar:?] at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractIOReactor.java:337) ~[verifier.jar:?] at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(AbstractIOReactor.java:315) ~[verifier.jar:?] at org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:276) ~[verifier.jar:?] at org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:104) ~[verifier.jar:?] at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:588) ~[verifier.jar
Workaround:
 
You need to run the following REST command:
curl -XPUT http://<esHost>/jarvis_metadata/_mapping/md -d '{ "properties": { "avro_schema": { "type": "keyword", "ignore_above": 1000 } } }'
Limitation of Total Number Fields in Elasticsearch Index
Problem:
 
By default, Elasticsearch supports the maximum total number fields in Elasticsearch index as 1000. Typically, when you try to instrument the CA APM Metrics with CA DOI using the DOI Plugin, where the CA APM Metrics exceeds 1000, then you get the following error in the Jarvis verifier logs:
[4498]: index [ao_itoa_metrics_apm_1_1], type [data], id [7338db72-ac0f-4e7e-aa6b-a1ed26d55179], message [java.lang.IllegalArgumentException: Limit of total fields [1000] in index [ao_itoa_metrics_apm_1_1] has been exceeded
Workaround:
 
Consider that you want to instrument the CA APM Metrics of count 2000 with CA DOI using the DOI Plugin. In the Elasticsearch host, run the following REST command:
 In the REST command, 
<esHost>
 represents the Elasticsearch Host.
curl -XPUT --header "Content-Type: application/json" 'http://<esHost>/<index_name>/_settings' -d '{ "index.mapping.total_fields.limit": 2000 }'