Poll Sensitive and Critical Devices Without a Performance Impact

Critical devices are sensitive to too many polls, which can lead to performance problems. To throttle the SNMP poll requests and avoid overwhelming your sensitive devices, configure the SNMP polling controls.
capm250
Critical devices are sensitive to too many polls, which can lead to performance problems. To throttle the SNMP poll requests and avoid overwhelming your sensitive devices, configure the SNMP polling controls.
By default, SNMP polling is controlled in three ways:
  • SNMP traffic threshold
    No more than 15 SNMP requests can be sent to a device at a time. Poll and discovery SNMP requests over 15 are queued and sent to a device when possible during the polling cycle. Up to 600 requests can be queued.
  • SNMP timeouts threshold
    When 15 or more SNMP requests timeout, polling is suspended for the remainder of the current polling cycle. An event is generated, informing you of the situation.
    Polling resumes at the beginning of each poll cycle. When the timeouts do not exceed the 15 threshold, a "clear" event is generated.
  • SNMP PDU Segmentation
    SNMP packets can be limited to contain a maximum number of varbinds by splitting a larger SNMP request into multiple smaller requests and reassembling the responses.
These thresholds are designed to prevent overwhelming a device with too many poll requests or PDUs that are too large. You can override these SNMP polling thresholds defaults by modifying the following parameters:
  • TimeoutFailSafeThrottleDefault
    Specifies the default maximum number of timeouts that trigger the fail-safe throttle
    .
  • MaxOutstandingRequestsDefault
    Specifies the default maximum number of outstanding requests.
  • MaxRequestSizeDefault
    Specifies the default maximum number of varbind that a single SNMP request may contain.
These thresholds can also be modified per device so that a specific device can be assigned individual threshold values.
For example, your older router is exceptionally sensitive to polling. But, this router is critical and must be polled as frequently as possible. You already adjusted your monitoring profile to remove unnecessary metric families from polling. You also applied a filter in your monitoring profile to reduce the number of polled interfaces. However, polling still causes this router to crash. Therefore, your only option is to adjust the default SNMP polling parameters for your sensitive router.
You can add any of the following parameters to the policy for individual IPs or IP ranges in an IPRange section within the IPRangeList:
  • TimeoutFailSafeThrottle
    The maximum number of timeouts that are applied on the devices within this IP range
  • MaxOutstandingRequests
    The maximum number of outstanding requests that are sent to the devices within the indicated IP range
  • MaxRequestSize
    Limits the number of varbinds in an outgoing SNMP request
    If the number of varbinds in the SNMP request exceeds the value of
    MaxRequestSize
    , the outgoing request is split into two or more smaller requests.
    Some IP ranges are not covered in the IPRange sections. For global settings, use the
    MaxRequestSizeDefault
    parameter to set the varbind limit.
    If
    MaxRequestSize
    is 0, the original request is sent regardless of its size.
For related troubleshooting information, see Gaps in Data Appear during Throttling and Polling Stopped Event Message.
Follow these steps:
  1. Find the ID for your IP Domain (that contains your sensitive router) by opening:
    http://
    DA_host
    :port/rest/ipdomains
    • DA_host
      :
      port
      Specifies the Data Aggregator hostname and the port number where you are accessing the REST web services from.
  2. Locate your IP Domain ID in the following SNMP throttle policy list, and note the corresponding policy ID:
    http://
    DA_host:port
    /rest/snmpthrottlepolicies
  3. Determine the number of varbinds that you want to include in a single outgoing SNMP request. Some devices ignore requests that are too large without sending an error. As a result, the SNMP poller cannot reach the device. Use the
    MaxRequestSize
    value to allow the Data Collector to monitor these devices.
    Example
    If the interface SNMP request has 27 varbinds and
    MaxRequestSizeDefault
    is set to 15, the outgoing request is split into two smaller requests. One request contains 14 varbinds, and the other contains 13 varbinds.
    Example:
    The following example from an SNMP throttle policy shows that the policy ID is "601" for IP Domain "2" with no limit on the number of varbinds:
    <SnmpThrottlePolicy version="1.0.0">
    <ID>601</ID>
    <MaxOutstandingRequestsDefault>15</MaxOutstandingRequestsDefault>
    <QueueLength>600</QueueLength>
    <TimeoutFailSafeThrottleDefault>15</TimeoutFailSafeThrottleDefault>
    <MaxRequestSizeDefault>0</MaxRequestSizeDefault>
    <IPDomainID>2</IPDomainID>
    </SnmpThrottlePolicy>
  4. Open a REST client editor or HTTP tool that sends requests and gets responses, and set the Content-type to application/xml.
  5. Open and edit the SNMP throttle policy for your IP Domain by entering the following criteria:
    • URL:
      http://
      DA_host
      :
      port
      /rest/snmpthrottlepolicies/
      policyID
      • policyID
        Specifies a unique identification number that is assigned to the SNMP throttle policy for the IP Domain that contains your sensitive device.
      Example:
      http://
      DA_host:port
      /rest/snmpthrottlepolicies/601
    • HTTP method = PUT
    • Adjust the following values for your IP Range on the Body tab in the HTTP Request pane:
      • <MaxOutstandingRequests>
        SNMP traffic threshold
      • <TimeoutFailSafeThrottle>
        SNMP timeouts threshold
      Both values are required for every IP Range entry. You can disable either parameter by setting the value to "0."
    • Remove the following lines:
      • <ID>
      • <IPDomainID>
    Results are returned in the Body tab of the HTTP Response pane.
    Example:
    In this example, the thresholds are lowered to "10" for device 10.231.41.7 only. For this device, the number of varbinds is limited to 50. The default thresholds and other IP Range thresholds continue using the default value of "15." For devices 10.231.41.1-10.231.41.255, SNMP requests are limited to 30 varbinds.
    <SnmpThrottlePolicy version="1.0.0">
    <IPRangeList>
    <IPRange>
    <IPRangeText>10.231.41.7</IPRangeText>
    <MaxOutstandingRequests>10</MaxOutstandingRequests>
    <TimeoutFailSafeThrottle>10</TimeoutFailSafeThrottle>
    <MaxRequestSize>50</MaxRequestSize>
    </IPRange>
    <IPRange>
    <IPRangeText>10.231.41.1-10.231.41.255</IPRangeText>
    <MaxOutstandingRequests>15</MaxOutstandingRequests>
    <TimeoutFailSafeThrottle>15</TimeoutFailSafeThrottle>
    <MaxRequestSize>30</MaxRequestSize>
    </IPRange>
    </IPRangeList>
    <MaxRequestSizeDefault>0</MaxRequestSizeDefault>
    <MaxOutstandingRequestsDefault>15</MaxOutstandingRequestsDefault>
    <QueueLength>600</QueueLength>
    <TimeoutFailSafeThrottleDefault>15</TimeoutFailSafeThrottleDefault>
    </SnmpThrottlePolicy>
    You can adjust the thresholds for a single device or a range of devices. The IP Range definition and the IP Range order determine which threshold applies. The IP Ranges are listed in priority order. That is, the first IP Range that applies to a device determines the threshold value to apply.
  6. Always include the
    MaxOutstandingRequestsDefault
    ,
    MaxRequestSizeDefault
    , TimeoutFailSafeThrottleDefault
    ,
    and
    QueueLength
    parameters in the update/POST XML at the root level. Include the parameters even if the values do not differ from the default.
    Example:
    This PUT command generates the policy that follows.
    Update XML: PUT on URL DA-HOST:8581/rest/snmpthrottlepolicies/21
    <SnmpThrottlePolicy version="1.0.0">
    <IPRangeList>
    <IPRange>
    <IPRangeText>130.119.103.8</IPRangeText>
    <MaxOutstandingRequests>10</MaxOutstandingRequests>
    <TimeoutFailSafeThrottle>10</TimeoutFailSafeThrottle>
    <MaxRequestSize>20</MaxRequestSize>
    </IPRange>
    </IPRangeList>
    <MaxRequestSizeDefault>0</MaxRequestSizeDefault>
    <MaxOutstandingRequestsDefault>15</MaxOutstandingRequestsDefault>
    <TimeoutFailSafeThrottleDefault>15</TimeoutFailSafeThrottleDefault>
    <QueueLength>600</QueueLength>
    </SnmpThrottlePolicy>
    This command generates the following policy:
    <SnmpThrottlePolicy version="1.0.0">
    <ID>21</ID>
    <QueueLength>600</QueueLength>
    <TimeoutFailSafeThrottleDefault>15</TimeoutFailSafeThrottleDefault>
    <IPDomainID>2</IPDomainID>
    <IPRangeList>
    <IPRange>
    <IPRangeText>130.119.103.8</IPRangeText>
    <MaxOutstandingRequests>10</MaxOutstandingRequests>
    <TimeoutFailSafeThrottle>10</TimeoutFailSafeThrottle>
    <MaxRequestSize>20</MaxRequestSize>
    </IPRange>
    </IPRangeList>
    <MaxRequestSize>0</MaxRequestSize>
    <MaxOutstandingRequestsDefault>15</MaxOutstandingRequestsDefault>
    </SnmpThrottlePolicy>