Integrate with DX Operational Intelligence

2
casp1042
2
Overview - What's New in 10.4.2!
The integration between
DX NetOps Spectrum
and DX Operational Intelligence allows you to use data from
DX NetOps Spectrum
and helps to analyze, correlate and proactively resolve the network issues. The SpectrumDataPublisher is a utility/service in
DX NetOps Spectrum
that publishes the
DX NetOps Spectrum
data to the analytics platform.
The 10.4.2 release supports sending Topology inventory and its relations from
DX NetOps Spectrum
to Topology Analytics Service (TAS) using
DX NetOps Spectrum
Data Publisher (SPUB).
SpectrumDataPublisher 10.4.2 with DX Operational Intelligence 20.2.x Overview

The following data is synchronized from
DX NetOps Spectrum
to DX Operational Intelligence:
  • Alarms
    Reconciliation does not happen when elastic has a large number of alarms. A limitation of 10000 alarms from DX Operational Intelligence exists.
  • Network Configuration Manager- Change events (Disabled by default, enable if necessary)
  • Topology
The following data are not supported in this release as the related data is pushed as part of Topology (TAS Integration):
  • Inventory
  • Services
  • Groups (Global Collection and Container)
  • Metrics: Device count and Device availability (based on Model Type and Group)
  • VNAInventory (Disabled by default, enable if necessary)
Integration Architecture
The following diagram explains how the
DX NetOps Spectrum
-Operational Intelligence integration works:
Spectrum-CA Digital Operational Intelligence
Compatibility Matrix
For more information about the integration of
DX NetOps Spectrum
with other CA products, see the following links:
Prerequisites for SpectrumDataPublisher
Ensure that the following prerequisites are met on the machine where you want to install the SpectrumDataPublisher.
  • JRE (v8)
Ensure that the JRE version is 8. The SpectrumDataPublisher is compatible with JRE v8 only.
  • For Linux machines, set JAVA_HOME to the JRE installed location. For example, "/usr/java/jre.8.66"
  • For Windows machines, the installer/service takes the JRE installed location or JAVA_HOME path (if already set). If Java Home is not set, make sure Java is in your path.
Installing the SpectrumDataPublisher
The
DX NetOps Spectrum
OneClick page allows you to download and install the SpectrumDataPublisher JAR file. The
DX NetOps Spectrum
Data Publisher version should be the same as the version of the
DX NetOps Spectrum
.
Follow these steps:
  1. Log in to the OneClick client.
  2. Navigate to the
    OneClick Administration
    page.
  3. Select the
    Analytics Configuration
    option from the panel on the left.
    The
    Analytics Configuration
    page opens in a separate window.
  4. Under the
    DX NetOps Spectrum
    - DX OI Connector
    section, select the
    DX NetOps Spectrum
    - DX OI Connector
    link to download the installer.
    The SpectrumDataPublisher.jar file is downloaded to the default Downloads folder on your computer.
  5. In
    Windows:
    Double-click on the jar to open the install wizard and enter the install directory. By default, the path is:
    C:\win32app\
    The
    DX NetOps Spectrum
    Data Publisher install wizard opens.
  6. In
    Linux
    perform one of the following tasks to start the installation:
    • Execute the following command for the silent installation:
      Java -jar SpectrumDataPublisher.jar -i silent -DUSER_INSTALL_DIR="<install directory>"
      For example:
      Java -jar SpectrumDataPublisher.jar -i silent -DUSER_INSTALL_DIR="/opt"
    • Execute the following command to export the DISPLAY to a Windows Server and start GUI-based installation:
      Export DISPLAY=<machine-name>:0 and then run the following command:
      Java -jar SpectrumDataPublisher.jar
  7. Select
    Next
    to install the folder location.
  8. Select Install for the
    DX NetOps Spectrum
    Data Publisher
    to be installed.
  9. Select
    Done
    to close the Install wizard.
    After the installation is complete, the SpectrumDataPublisher folder is created in the install folder location.
Install
SpectrumDataPublisher 10.4.2.x with DX Operational Intelligence 20.2.x DEMO

Upgrade
DX NetOps Spectrum
Data Publisher
From the 10.3.1 release, there are considerable changes that are done to the
DX NetOps Spectrum
Data Publisher. We recommend upgrading the
DX NetOps Spectrum
Data Publisher version to verify that the
DX NetOps Spectrum
Data Publisher is running the same version of
DX NetOps Spectrum
.
During the DX Operational Intelligence Connector upgrade to the current release, the installer upgrades the necessary files with new parameters.
Follow these steps:
  1. Log in to the OneClick client.
  2. Navigate to the OneClick Administration page and select
    Analytics Configuration
    in the left panel.
    The
    Analytics Configuration
    page opens.
  3. Under the
    DX NetOps Spectrum
    - DX OI Connector
    section, select the
    DX NetOps Spectrum
    - DX OI Connector
    link to download the installer.
    The SpectrumDataPublisher.jar file is downloaded to the default Downloads folder on your computer.
  4. Double-click the JAR file.
    The
    DX NetOps Spectrum
    Data Publisher
    install wizard opens.
  5. Select 
    Next
    to install the folder location.
  6. Select
    Install
    .
    A pop-up window appears to display the current version and the version you are upgrading to.
  7. Select
    OK
    to proceed.
    The
    DX NetOps Spectrum
    Data Publisher is upgraded.
  8. Select
    Done
    to close the installation wizard.
Configuring the SpectrumDataPublisher
To synchronize data from
DX NetOps Spectrum
to Operational Intelligence, configure the SpectrumDataPublisher.
Follow these steps:
  1. Navigate to the
    DX NetOps Spectrum
    Data Publisher install location.
  2. Open the 'config' folder.
  3. Open the ConnectorConfig.xml file, and configure the following parameters:
    1. SpectrumConfiguration
      section: Enter the
      DX NetOps Spectrum
      OneClick host server details in this section.
      For example:
      <SpectrumConfiguration> <OneClickHostname>DX_NetOps_Spectrum_OneClick_host_server></OneClickHostname> <OneClickPort>8080</OneClickPort> <OneClickProtocol>http</OneClickProtocol> <FilterUIMContent>true</FilterUIMContent> <ConfigFile>SpectrumConfig.xml</ConfigFile> </SpectrumConfiguration>
      If you want to use the Secure Sockets Layer (SSL) protocol to encrypt communications between TAS and the SpectrumDataPublisher, you must import SSL (https) Certificate to TAS endpoint.
      If the value for 'FilterUIMContent' is set to 'True' (default), the
      DX NetOps Spectrum
      Data Publisher does not send the UIM alarms and inventory information to DX Operational Intelligence.
      Launch OneClick Console or WebApp from DX Operational Intelligence Alarms
      (Introduced in 10.4.2.1)
      When
      WebappLaunchUrl
      property is empty OneClick console is launched and when you provide the WebApp URL, OneClick WebApp is launched.
    2. Destinations
      section: Modify the destinations in this section to provide the DX Operational Intelligence connection details. The table displays the required parameters and explains how to obtain their values in the different DX Operational Intelligence environments:
      The Connector Parameters section in the DX SAAS Settings page provides all the mandatory parameters that are required to configure SpectrumDataPublisher. For more information, see Connector Parameters.
      1. DestinationDefaultTenant:
        Parameter Name
        Description
        DX Operational Intelligence On-Prem
        DX Operational Intelligence SaaS
        DestinationDefaultTenant
        Specifies the tenant ID or Cohort ID.
        1. Obtain the Elastic endpoint. Depending on your environment, run the appropriate command.
          • Openshift:
            oc get routes -n<your-namespace> | egrep jarvis-es
          • Kubernetes:
            kubectl get ingress -n<your-namespace> | egrep jarvis-es
        2. Run a query against the elastic endpoint to list all tenants.
          http(s)://<elastic-endpoint>/ao_dxi_tenants_1_1/_search?size=200&pretty
        3. Locate your tenant name and extract the value of the associated "tenant_id" parameter.
        1. Navigate to the DX SaaS, Settings,  Connector Parameter page
        2. Note down the tenat_id which is equal to the
          Cohort ID
      2. DestinationUrl:
        Parameter Name
        Description
        DX Operational Intelligence On-Prem
        DX Operational Intelligence SaaS
        DestinationUrl
        Specifies Jarvis or
        nginx
        endpoint.
        • Openshift:
          oc get routes -n<your-namespace> | egrep nginx
        • Kubernetes:
          kubectl get ingress -n<your-namespace> | egrep nginx
        1. Navigate to the DX SaaS, Settings, Connector Parameter page
        2. Note down the
          Jarvis endpoint
      3. AlarmReconcileConfiguration
        : Use the AlarmReconcileConfiguration parameter to clear the stale alarms from the DX Operational Intelligence. Modify the AlarmReconcileConfiguration parameters as shown in the following table:
        Parameter Name
        Description
        DX Operational Intelligence On-Prem
        DX Operational Intelligence SaaS
        Enable
        Set the parameter to
        true
        to clear the stale alarms during service startup.
        OIUrl
        Specifies the DX Operational Intelligence host URL.
        • Openshift:
          oc get routes -n<your-namespace> | egrep dxi-adminui
        • Kubernetes:
          kubectl get ingress -n<your-namespace> | egrep dxi-adminui
        • SaaS NA= https://axa.dxi-na1.saas.broadcom.com
        • Saas EU= https://axa.dxi-eu1.saas.broadcom.com
        BearerToken
        Specifies the bearer or also known as DX Auth token.
        Reconciliation does not happen when elastic has a large number of alarms. There is a limitation of 10000 alarms from DX Operational Intelligence.
    3. TasConfiguration
      section: Set the following mandatory parameters in this section and other parameters as per your requirement:
      Parameter Name
      Description
      DX Operational Intelligence On-Prem
      DX Operational Intelligence SaaS
      EnableIntegration
      Set the parameter to
      true
      to enable TAS configuration.
      TasUrl
      Specifies TAS endpoint.
      • Openshift:
        oc get routes -n<your-namespace> | egrep gateway
      • Kubernetes:
        kubectl get ingress -n<your-namespace> | egrep gateway
      1. Navigate to the DX SaaS, Settings, Connector Parameter page
      2. Note down the
        TAS endpoint
      For example, FullSyncInterval - Specifies the interval between two full synchronizations in hours. By default, full synchronization is set to 720 hours. To synchronize the TAS inventory more frequently, you can, for example, set it to 24 hours.
    4. Save and close the file.
      After the configuration is complete, start the SpectrumDataPublisher service.
Start the SpectrumDataPublisher service
To start the SpectrumDataPublisher service, follow these steps:
  1. Perform one of the following step to start the SpectrumDataPublisher service:
    • In Windows, from the command line, run the
      run.bat start
      command.
    • In Unix, from the console, run the
      run.sh start
      command.
  2. When you run the 'run.bat start' or 'run.sh' for the first time, it asks to create a log in password for SpectrumDataPublisher.
    You will need this login password for next time when you run 'run.sh' or 'run.bat start' commands.
  3. After creating the login password for SpectrumDataPublisher, it asks for the following login details that must be encrypted/decrypted with the password.
    1. Enter New password:
      Enter the password of the
      DX NetOps Spectrum
      Data Publisher service.
    2. Confirm Password:
      Re-enter the password of the
      DX NetOps Spectrum
      Data Publisher service.
    3. Enter OneClick username:
      Enter the OneClick username
    4. Enter OneClick Password:
      Enter the OneClick password
    5. Enter DOI username:
      DX Operational Intelligence username
    6. Enter DOI password:
      DX Operational Intelligence user password
    7. Enter the TAS Bearer Token
      : The following table shows how to get this token.
      Parameter Name
      Description
      DX Operational Intelligence On-Prem
      DX Operational Intelligence SaaS
      TAS Bearer Token
      Specifies TAS Bearer Token.
      1. Generate a Tenant or TAS Bearer token from APM. For more information, see Generate Security Token page in APM documentation.
      2. If APM is not available, generate a Tenant token from Cluster Management using the master admin account.
      1. Navigate to the DX SaaS, Settings, Connector Parameter page.
      2. Generate a new Ingestion Token.
    8. Enter username for destination proxy -- if applicable:
      Enter a proxy server user name
    9. Enter password for destination proxy -- if applicable:
      Enter the password of the proxy server
    The SpectrumDataPublisher service starts.
Post-installation
To avoid a manual restart of the SpectrumDataPublisher service, change the 'Startup Type' from 'Manual' to 'Automatic' in Windows Services.
Stop the SpectrumDataPublisher service
If you want to stop the SpectrumDataPublisher-service, run the following command:
Windows:
run.bat stop (Windows)
Linux:
run.sh stop (Linux)
Restart the SpectrumDataPublisher service
If you want to restart the SpectrumDataPublisher-service, run the following command:
Windows:
run.bat restart (Windows)
Linux:
run.sh restart (Linux)
On Windows machines, the SpectrumDataPublisher-service can be stopped and restarted through the Windows Services Console.
Import SSL (https) Certificate into Spectrum Data Publisher
If you want to use the encrypted communication (https protocol) between
DX NetOps Spectrum
DataPublisher and integrated products (
DX NetOps Spectrum
and CA Jarvis server), you must import the SSL/https certificate from the respective products into the SpectrumDataPublisher.
Follow these steps to import the certificate:
  1. Ensure that the 'https' certificate is exported and copied to the server where the SpectrumDataPublisher is installed. For export instructions, refer to the notes provided at the end of this section.
  2. Perform one of the following steps to download the SSL/HTTPS certificate of the server where you got the SSL error:
    • Run the appropriate command:
      openssl s_client -connect Jarvis_Endpoint:<port> < /dev/null | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' > jarvisServer.cer openssl s_client -connect TAS_Endpoint:<port> < /dev/null | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' > tas.cer openssl s_client -connect Spectrum_OneClick_Hostname:<port> < /dev/null | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' > ocservercerts.cer
      OR
    • Access the Jarvis URL using HTTPS in a browser and save the certificate from site information.
  3. Run the following command to import the certificate into the Keystore of SpectrumDataPublisher.
    keytool -importcert -alias <certificate_alias> -file /<PATH>/<FILENAME.cer> -keystore /<PATH>
    For example:
    keytool -importcert -alias jarvisssl -file jarvisServer.cer -keystore /<SpectrumDataPublisher-HOME>/Security/cacerts keytool -importcert -alias tas -file tas.cer -keystore /<SpectrumDataPublisher-HOME>/Security/cacerts keytool -importcert -alias tomcatssl -file OCServer.cer -keystore /<SpectrumDataPublisher-HOME>/Security/ocservercerts
    Ignore the warning about migrating to PKCS12 format.
  4. When prompts, provide the Keystore 'changeit' as a password.
  5. After the certificate is imported, stop the SpectrumDataPublisher service.
  6. Make sure the hostname, port, and protocol details are proper in the /SpectrumDataPublisher/config/ConnectorConfig.xml
To export the ssl/https certificate of
DX NetOps Spectrum
, run the following command from
DX NetOps Spectrum
OneClick server:
keytool -export -keystore /usr/<PATH> -alias <certificate_alias> -file <FILENAME.cer>
For example:
keytool -export -keystore /usr/Spectrum/custom/keystore/cacerts -alias tomcatssl -file OCServer.cer
Reset the Login Details of SpectrumDataPublisher service
In case you forgot the login password for the SpectrumDataPublisher service, you can reset the same by using the following commands. Note that, when you reset the login details, the stored configuration details are deleted and you should provide the details again in the ConnectorConfig.xml file.
Windows:
run.bat reset (Windows)
Linux:
run.sh reset (Linux)
Logging Configuration for SpectrumDataPublisher
The SpectrumDataPublisher logs are stored in the SpectrumDataPublisher/logs/SpectrumDataPublisher.log file.
  • By default, 'trace', 'debug', 'info', 'warn', 'error' and 'fatal' information is logged in the 'SpectrumDataPublisher/config/log4j2.properties' file.
  • To see the log for the number of alarms and inventory being synced, set the rootLogger.level to 'info' in the log4j2.properties file. Any changes to the log4j2.properties file requires a restart of the SpectrumDataPublisher.
  • The maximum size of the log file is 128 MB. When the file reaches the maximum size, a backup file is created. The name of the backup file is SpectrumDataPublisher.log prefixed with a timestamp. The normal logging is continued in the SpectrumDataPublisher.log file.
Get the DX Auth Token Used in Alarm Reconcile Configuration
You need a DX Auth token when you configure the SpectrumDataPublisher. Note this bearer token for future use.
The token expires periodically, hence you must fetch the token afresh while starting the service.
Follow these steps:
  1. Refresh the DOI page after login.
  2. Press CTRL+SHIFT+R on the keyboard.
  3. Navigate to the Network.
  4. Refresh the page again.
  5. Click on any API in the
    Network
    tab.
  6. Scroll down and note the token of the authorization: Bearer.
    Spectrum-DX Operational Intelligence
Troubleshooting SpectrumDataPublisher
Troubleshooting information for the problems that are encountered with SpectrumDataPublisher.
DX NetOps Spectrum
Data does not sync to Digital Operational Intelligence
Symptom:
SpectrumDataPublisher.log shows the exception: com.ca.spectrum.spub.common.ConnectorException.
DX NetOps Spectrum
Data does not sync to Digital Operational Intelligence.
Resolution:
Provide correct details of the Jarvis server under the DestinationConfiguration section of ConnectorConfig.xml and ensure that the DestinationHostname is resolved to a valid IP address.
SpectrumDataPublisher Troubleshooting