Monitoring Virtual Appliance

The Virtual Appliance does not support installation of the third-party monitoring tools, which require root access. Virtual Appliance can be monitored remotely by SNMP and Log Forwarding.
cislp143
The Identity Manager, Identity Governance, Identity Portal and other Java servers in Virtual Appliance can be monitored remotely using SNMP, Log Forwarding and Java Profilers (Example: DX Application Performance Management).
Note:
Virtual Appliance does not support installation of the third-party monitoring tools which require root access.
Monitoring with SNMP
The
config
user has the permission to configure and enable the
snmpd
and
snmptrapd
services. By default, these services are disabled for security reasons and come with a minimal snmp view configuration for the default
public
community.
A user can configure the
snmpd
and
snmptrapd
services by modifying the configuration files under
/etc/snmp/
.
To enable the services, run the following commands:
chkconfig snmpd on
chkconfig snmptrapd on
service snmpd start
service snmptrapd start
Example 1: Monitoring Free Disk Space
  1. Add the following lines to the
    /etc/snmp/snmpd.conf
    file:
    disk / 5%
    view systemview included .1.3.6.1.4.1.2021.9
  2. Restart the
    snmpd
    service:
    service snmpd restart
  3. Run the following command locally on the vApp to query the snmp:
    snmpwalk -v 1 localhost -c public .1.3.6.1.4.1.2021.9
    Result:
    UCD-SNMP-MIB::dskIndex.1 = INTEGER: 1
    UCD-SNMP-MIB::dskPath.1 = STRING: /
    UCD-SNMP-MIB::dskDevice.1 = STRING: /dev/mapper/vg_ca--vapp-lv_root
    UCD-SNMP-MIB::dskMinimum.1 = INTEGER: -1
    UCD-SNMP-MIB::dskMinPercent.1 = INTEGER: 99
    UCD-SNMP-MIB::dskTotal.1 = INTEGER: 49099068
    UCD-SNMP-MIB::dskAvail.1 = INTEGER: 41297772
    UCD-SNMP-MIB::dskUsed.1 = INTEGER: 5310496
    UCD-SNMP-MIB::dskPercent.1 = INTEGER: 11
    UCD-SNMP-MIB::dskPercentNode.1 = INTEGER: 2
    UCD-SNMP-MIB::dskTotalLow.1 = Gauge32: 49099068
    UCD-SNMP-MIB::dskTotalHigh.1 = Gauge32: 0
    UCD-SNMP-MIB::dskAvailLow.1 = Gauge32: 41297772
    UCD-SNMP-MIB::dskAvailHigh.1 = Gauge32: 0
    UCD-SNMP-MIB::dskUsedLow.1 = Gauge32: 5310496
    UCD-SNMP-MIB::dskUsedHigh.1 = Gauge32: 0
    UCD-SNMP-MIB::dskErrorFlag.1 = INTEGER: error(1)
    UCD-SNMP-MIB::dskErrorMsg.1 = STRING: /: less than 5% free
Example 2: Monitoring the vApp Dashboard Alarm Log
The alarms that are shown on the Virtual Appliance dashboard are exposed by SNMP.
  1. Add the following lines to the
    /etc/snmp/snmpd.conf
    file:
    extend test /bin/cat /opt/CA/VirtualAppliance/logs/alarm.log
    view systemview included .1.3.6.1.4.1.8072.1
  2. Restart the
    snmpd
    service:
    service snmpd restart
  3. Run the following command locally on the vApp to query the snmp:
    snmpwalk -v2c -On -c public 127.0.0.1 nsExtendOutput2Entry
Monitoring with Log Forwarding
The Virtual Appliance comes with a central log server service which can be deployed from the Virtual Appliance service deployment screen.
The CA Identity Manager, CA Identity Governance, CA Identity Portal, and JCS services write logs to the central log service called
Syslog
.
Once deployed, the central log service merges all application logs (from the entire cluster) to a single log file: /opt/CA/VirtualAppliance/centralLogs/vapp_central.log
Note:
In a cluster environment, any node (not only the central log server node) can forward logs to the central log service.
For monitoring or log analysis purposes, you can forward the central log to a remote Syslog server by following the given two steps:
  1. Modify the following Syslog configuration file:
    /etc/rsyslog.d/rsyslog-custom.conf
  2. Enable a forwarding rule.
    Examples:
    • Forwarding to a remote syslog server using UDP:
      *.* @<Remote_Syslog_IP_Address>:514 
    • Forwarding to a remote syslog server using TCP:
      *.* @@<ip_address.of.remote.syslog>:514
After the central log is forwarded to a remote Syslog server, you can use applications such as
Splunk
to monitor and analyze the log.
Monitoring with Java Profilers
The Identity Manager, Identity Governance, Identity Portal and other Java servers in Virtual Appliance can be monitored using Java Profilers which use JVM argument based instrumentation.
The general procedure to monitor the Virtual Appliance Java Servers using Java Profilers is as follows:
  1. On the Virtual Appliance, create a folder (for example,
    tools
    ) under
    /opt/CA/VirtualAppliance/custom/profiler
    . Ensure that the user who spawns the Java server has access to the newly created folder. The actual access permissions depend on the Profiler. To monitor Wildfly-Servers on Virtual Appliance, configure folder permissions for the wildfly user.
  2. In the newly created folder (
    /opt/CA/VirtualAppliance/custom/profiler/tools)
    , copy the Profiler code and configuration.
  3. Configure the instrumentation hook in custom JVM arguments for the corresponding server by following the Custom JVM Arguments document.
  4. If the Profiler requires to open a port, ensure that you specify a port which is not already in use. For the ports used by Virtual Appliance, see Ports.
  5. Restart the Java servers for the changes to take effect.
Note:
With
wildfly
user as the owner, Java profilers can create logs and other artifacts. To modify or delete these files, you need appropriate permissions. Run the following command to reset permissions to /opt/CA/VirtualAppliance/custom/profiler folder.
$> resetProfilerPerms.sh
Example 1: Profiling Identity Manager with DX Application Performance Management (DX APM) (Formerly CA Application Performance Management)
To monitor Identity Manager server with DX APM, follow the given steps:
  1. Create the JBoss agent home directory (Example:
    apmim)
    under
    /opt/CA/VirtualAppliance/custom/profiler/.
  2. Copy the contents of the extracted wily JBoss agent folder to
    /opt/CA/VirtualAppliance/custom/profiler/apmim
    .
    Example:
    /opt/CA/VirtualAppliance/custom/profiler/apmim |_common |-connectors |-core |_deploy |-examples |-extensions |-logs |-tools |-Agent.jar
  3. Update the agent profile
    (/opt/CA/VirtualAppliance/custom/profiler/apmim/core/config/IntroscopeAgent.profile)
    configuration as needed to connect to the DX APM Server.
  4. Grant access permission to the
    wildfly
    user to the apmim folder.
    $> chmod -R 777 /opt/CA/VirtualAppliance/custom/profiler/apmim/
  5. Configure the agent instrumentation hook in the custom JVM parameters in
    /opt/CA/VirtualAppliance/custom/IdentityManager/jvm-args.conf.
    JAVA_OPTS=-Xms512m -Xmx1512m -XX:+UseG1GC -XX:+UseStringDeduplication -XX:+UseCompressedOops ="http://djava.net/">Djava.net.preferIPv4Stack=true -Djava.awt.headless=true -XX:+UseLargePages -Djava.security.egd="http://file/dev/urandom">file:/dev/./urandom -Djboss.modules.system.pkgs=org.jboss.byteman,com.wily,com.wily.* -javaagent:/opt/CA/VirtualAppliance/custom/profiler/apmim/Agent.jar -DagentProfile=/opt/CA/VirtualAppliance/custom/profiler/apmim/core/config/IntroscopeAgent.profile
  6. Restart the Identity Manager server.
    $> restart_im
  7. Check the IntroscopeAgent log file (../profiler/apmim/logs/) to verify that the agent is connected to the DX APM server.
    [INFO] [IntroscopeAgent.IsengardServerConnectionManager] Connected controllable Agent to the Introscope Enterprise Manager at IDSAPM:5001,com.wily.isengard.postofficehub.link.net.DefaultSocketFactory. Host = "apmvapp", Process = "IPJBossProc", Agent Name = "IPJBossAgent", Active = "true".
    [INFO] [IntroscopeAgent.ConnectionThread] New list {}@1564821312119 downloaded from IDSAPM:5001,com.wily.isengard.postofficehub.link.net.DefaultSocketFactory
    [INFO] [IntroscopeAgent.Agent] New list accepted
    [INFO] [IntroscopeAgent.ConnectionThread] Connected to IDSAPM:5001,com.wily.isengard.postofficehub.link.net.DefaultSocketFactory in allowed mode.
    [INFO] [IntroscopeAgent.Instrumentation Manager] Agent Dynamic Instrumentation bean has been deployed.
    Notes:
    • To monitor Identity Governance and Identity Portal with DX APM, follow the same procedure as above. Ensure that you create a different JBoss Agent home directory for Identity Manager, Identity Governance and Identity Portal. You can update the JVM parameters for Identity Governance in
      /opt/CA/VirtualAppliance/custom/IdentityGovernance/jvm-args.conf
      ,
      and for Identity Portal in
      /opt/CA/VirtualAppliance/custom/IdentityPortal/jvm-args.conf
      .
    • For information about DX APM Agent configuration, refer to the DX APM documentation.
Example 2: Profiling Identity Portal Server with JProfiler
To monitor Identity Portal server with JProfiler, follow the given steps:
  1. Create a folder
    jprofiler
    under
    /opt/CA/VirtualAppliance/custom/profiler.
  2. Download and extract the
    JProfiler agent
    and
    jprofiler_linux_<version>.tar
    under
    /opt/CA/VirtualAppliance/custom/profiler/jprofiler
    .
  3. Rename
    /opt/CA/VirtualAppliance/custom/profiler/jprofiler/jprofiler_linux_<version>
    to
    /opt/CA/VirtualAppliance/custom/profiler/jprofiler/jprofiler.
  4. Grant access permission to the wildfly user to the
    jprofiler
    folder.
    $> chmod -R 777 /opt/CA/VirtualAppliance/custom/profiler
  5. Configure the agent instrumentation hook in the custom JVM parameters in
    /opt/CA/VirtualAppliance/custom/IdentityPortal/jvm-args.conf
    .
    JAVA_OPTS=-Xms512m -Xmx1512m -XX:+UseG1GC -XX:+UseStringDeduplication -XX:+UseCompressedOops -Djava.net.preferIPv4Stack=true -Djava.awt.headless=true -XX:+UseLargePages -Djava.security.egd=file:/dev/./urandom -agentpath:/opt/CA/VirtualAppliance/custom/profiler/jprofiler/jprofiler/bin/linux-x64/libjprofilerti.so=port=8851,nowait
  6. Restart the Identity Portal server.
    $>restart_ip
  7. Open JProfiler Console on another machine and connect to VirtualAppliance_machine:8851 address.
Background Monitoring Frequency for an External Oracle Database
By default, Virtual Appliance monitors the availability of an external Oracle database every 5 minutes.
To customize the frequency at which an external database can be monitored by Virtual Appliance, follow these steps:
  1. Navigate to
    /opt/CA/VirtualAppliance/custom/monitor
    location.
  2. Open
    monitor.conf
    file for editing.
  3. Change the value of
    db_interval_minutes
    parameter to a desired value in minutes.
  4. Reboot the machine for the changes to take effect.