Delete Components That Are Not Present

Data Aggregator includes a script to delete components that are not present. These components are components that no longer exist on physical devices. The presence of excessive numbers these components can impact user interface performance.
capm250
Data Aggregator includes a script to delete components that are not present. These components are components that no longer exist on physical devices. The presence of excessive numbers these components can impact user interface performance.
 
Follow these steps:
 
  1. Open a command prompt and access the /opt/IMDataAggregator/scripts directory.
  2. Type the following command:
    ./remove_not_present_items.sh
    The script usage is described.
  • If you filter the components by IP domain name or IP domain ID, also specify a specific IP address to return correct results.
  • If your filter criteria return too many components, the REST interface does not return a response. Use other filtering options to narrow the results. More filter criteria are available at http://
    hostname
    :
    port
    /rest/retired/xsd/filterselect.xsd.
Automate the Removal of Components That Are Not Present
As an administrator, you can automate the removal of components that are not present from your network. For example, you can set up a weekly cron job to delete the components that are a month old.
The remove_not_present_items script that is included with Data Aggregator is comprised of two parts. The first part of the script identifies and returns data about the components, which is based on the filter that you set. The second part of the script issues the delete of the component list. To automate the process, understand how this script was built.
 
Example: Filter the List of Components By a Device IP Address
 
In this example, you want to find all of the components for a device that has a primary IP address of 10.252.1.1. Filtering by IP address is a two-step process because no direct component filter by IP address is available. To filter the components, first make note of the IP address for the device that the components are associated with. With the IP address information, you will determine the device item ID for the device. Then, using the device item ID, you will determine what the components are. Finally, you delete the components.
 This example uses the curl command, but you can use any command that you are familiar with.
  1. Create the filterDeviceIP.xml file. You will use this file to return the device item ID for the device that has a primary IP address of 10.252.1.1. The file must look like the following example:
    <FilterSelect xsi:noNamespaceSchemaLocation="filter.xsd" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
    <Filter>
    <And>
    <Device.PrimaryIPAddress type="EQUAL">10.252.1.1</Device.PrimaryIPAddress>
    </And>
    </Filter>
    </FilterSelect>
  2. Run the following command:
    curl -X http://
    hostname
    :
    port
    /rest/devices/filtered
     
    -H "Content-Type: application/xml" -T "filterDeviceIP.xml" > returnedDeviceID.xml
    •  
      -X
      Creates the filter that you indicate.
    •  
       
      hostname
      :
      port
       
      Specifies the Data Aggregator host name and the port number.
      Default port:
       8581
    •  
      -H
      Indicates the content type of the file that you are posting.
    •  
      -T
      Indicates the file that you are posting.
    The following result is returned as an HTTP response:
    <?xml version="1.0"?>
    <DeviceList>
    <Device version="1.0.0">
    <ID>
    107881
    </ID>
    <PrimaryIPAddress>10.252.1.1</PrimaryIPAddress>
    <supportsOnDemandMFDiscovery>true</supportsOnDemandMFDiscovery>
    <SupportedProtocolsList>
    <SupportedProtocols>ICMP</SupportedProtocols>
    </SupportedProtocolsList>
    <DiscProfileID>107503</DiscProfileID>
    <HostName>rtp003723rts.ca.com</HostName>
    <RelatesTo>
    <MonitoredGroupIDList relatesURL="relatesto/monitoredgroups" rootURL="monitoredgroups">
    <ID>509</ID>
    </MonitoredGroupIDList>
    <GroupIDList relatesURL="relatesto/groups" rootURL="groups">
    <ID>547</ID>
    <ID>530</ID>
    <ID>509</ID>
    </GroupIDList>
    </RelatesTo>
    <IsAlso>
    <IsA name="MetricFamilyDiscoveryHistory" rootURL="devices/mfdiscoveryhistory"/>
    <IsA name="AccessibleDevice" rootURL="devices/accessible"/>
    <IsA name="Syncable" rootURL="syncable"/>
    <IsA name="IPDomainMember" rootURL="ipdomainmember"/>
    </IsAlso>
    <DataCoectionMgrId version="1.0.0">
    <DcmID>dcname.ca.com:8f53bc55-f442-42fc-9bd5-a907d0261421</DcmID>
    </DataCollectionMgrId>
    <Syncable version="1.0.0">
    <SyncID>-1</SyncID>
    </Syncable>
    <Item version="1.0.0">
    <DisplayName>router.ca.com</DisplayName>
    <CreateTime>Wed Feb 05 10:20:26 EST 2014</CreateTime>
    <Name>router.ca.com</Name>
    </Item>
    <IPDomainMember version="1.0.0">
    <IPDomainID>2</IPDomainID>
    </IPDomainMember>
    <DeviceMonitoringProfile version="1.0.0">
    <ConsolidatedMonitoringProfile>2509</ConsolidatedMonitoringProfile>
    </DeviceMonitoringProfile>
    </Device>
    </DeviceList>
    Device item ID 107881 is returned. The results also display detailed information about the device.
  3. Create the filterNotPresent.xml file. You will use this file to return the components that are not present and are associated with the device whose device item ID is 107881. This file must look like the following example:
    <FilterSelect xsi:noNamespaceSchemaLocation="filter.xsd" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
    <Filter>
    <And>
    <DeviceComponent.DeviceItemID type="EQUAL">107881</DeviceComponent.DeviceItemID>
    </And>
    </Filter>
    <Select use="exclude">
    <Item use="exclude">
    <DisplayName use="include"/>
    </Item>
    </Select>
    </FilterSelect>
  4. Run the following command:
    curl -X post http://
    hostname
    :
    port
    /rest/retired/filtered
     
    -H "Content-Type: application/xml" -T "filterNotPresent.xml" > returnedNotPresentItems.xml
    The following result is returned as an HTTP response:
    <?xml version="1.0"?>
    <RetiredList>
    <Retired version="1.0.0">
    <ID>
    128452
    </ID>
    <Item version="1.0.0">
    <DisplayName>GigabitEthernet0/239 - GigabitEthernet0/239</DisplayName>
    </Item>
    </Retired>
    <Retired version="1.0.0">
    <ID>
    128451
    </ID>
    <Item version="1.0.0">
    <DisplayName>GigabitEthernet0/238 - GigabitEthernet0/238</DisplayName>
    </Item>
    </Retired>
    </RetiredList>
    Two components that fit the filter criteria are returned. The item IDs for the components are 128452 and 128451.
  5. Create the deleteNotPresentList.xml file. You will use this file to delete the returned list of components. The file must look like the following example:
    <DeleteList>
    <ID>128452</ID>
    <ID>128451</ID>
    </DeleteList>
  6. Run the following command:
    curl -X post http://
    hostname
    :
    port
    /rest/retired/deletelist -H "Content-Type: application/xml" -T "deleteRetiredList.xml" > deletelistreponse.xml
    The following result is returned as an HTTP response:
    <?xml version="1.0"?>
    <DeleteListResult>
    <DeleteResult>
    <ID>128452</ID>
    <Error>SUCCESS</Error>
    </DeleteResult>
    <DeleteResult>
    <ID>128451</ID>
    <Error>SUCCESS</Error>
    </DeleteResult>
    </DeleteListResult>
    The components are successfully removed.
 
Example: Delete a Large Number of Retired Components
 
You can easily delete all of the retired components where the total exceeds 100,000:
  • To review all of the retired components where the total exceeds 100,000, type the following command:
    ./remove_not_present_items.sh -h host_name -o ouputfile
    •  
      -o 
      outputfile
       
      Is the output of all of the retired components. The output is a .csv file.
    For example, the following command outputs a list of all of the retired components. The .csv file format includes the device item ID, the device display name, the retired component ID, and the retired component display name:
    ./remove_not_present_items.sh -h my_host_name -o myretired.csv
  • To delete all of the retired components where the total exceeds 100,000, and to log the information to a .csv file, type the following command:
    ./remove_not_present_items.sh -h host_name -o ouputfile -c Yes
    •  
      -o 
      outputfile
       
      Is the output of all of the retired components. The output is a .csv file.
    •  
       -c Yes
      Confirms the deletion of all retired components.
    For example, the following command deletes all of the retired components:
    ./remove_not_present_items.sh -h my_host_name -o myretired.csv -c Yes