Perform Initial Load and Refresh Index and Data on the Search Server

This article contains the following topics:
casm173
This article contains the following topics:
Configure the Objects
Before you perform initial load or refresh index and data on the search server, you can configure the objects that you want to load or refresh. Search currently supports the following objects:
  • cnt
    (users)
  • kd
    (knowledge document)
  • nr
    (configuration items)
  • cr
    (incidents and requests)
  • kdlinks
    (suggested solutions)
  • pcat
    (category)
  • service_offering
  • service_request
By default, the
dataload_limit
is set to one year. You can use this parameter to limit the initial load data for number of years. 
By default, all these objects are configured.
You cannot add new or custom objects other than the eight objects that the Search currently supports.
Follow these steps:
  1. As an administrator, log in to the search server and navigate to the
    <es_install>\config
    folder and edit the
    config.yml
    file.
  2. In the sdm_index property, remove the object you don't want to configure:
    • cnt
    • kd
    • nr
    • cr
    • kdlinks
    • pcat
    For example, if you want to perform initial load only for users (cnt) and tickets (cr), define the value for
    sdm_index
    in the following way:
    objects:
    sdm_index: cnt, cr
    #Limit the initial load to number of years
    dataload_limit: 1y
  3. In the catalog_index property, remove the object you don't want to configure:
    • serive_offering
    • service_request
  4. By default, the
    dataload_limit
    is set to one year. You can use this parameter to limit the initial load data for number of years. 
    dataload_limit: 1y
  5. Save and close the file and perform initial load.
Data Verification
Execute the query against the MDB database to verify data:
  • Oracle
    :
    select * from usm_offering where (date_available <= to_date('1901-12-13','yyyy-mm-dd')
    or date_available >= to_date('2038-01-19','yyyy-mm-dd'))
    or (date_unavailable <= to_date('1901-12-13','yyyy-mm-dd')
    or date_unavailable >= to_date('2038-01-19','yyyy-mm-dd'))
    or (date_cancelled <= to_date('1901-12-13','yyyy-mm-dd')
    or date_cancelled >= to_date('2038-01-19','yyyy-mm-dd'))
  • SQL
    :
    select * from usm_offering where (date_available <= CONVERT(DATETIME, '1901-12-13', 102)
    or date_available >= CONVERT(DATETIME, '2038-01-19', 102))
    or (date_unavailable <= CONVERT(DATETIME, '1901-12-13', 102)
    or date_unavailable >= CONVERT(DATETIME, '2038-01-19', 102))
    or (date_cancelled <= CONVERT(DATETIME, '1901-12-13', 102)
    or date_cancelled >= CONVERT(DATETIME, '2038-01-19', 102))
    We recommend you update the data and run the rebuild index, if the query returns any results.
Perform Initial Load on the Search Server
Initial load is an activity that is typically performed when you install search server using the installer. Initial load prepares the search server with the index and data that are imported from the CA SDM and CA Service Catalog database that is used by search in
xFlow Interface
.
We recommend that you perform the initial load task during off peak hours as it affects the CA SDM server performance.
If you perform the initial load action after installation, execute one of the files based on your operating system:
  • $ES_HOME/search/bin/pdm_es_initial_load.bat (Windows)
  • $ES_HOME/search/bin/pdm_es_initial_load.sh (Linux)
This file creates an index and loads the data into the search server.
When you execute the file from the command prompt, you can use the following options:
  • $ES_HOME/search/bin/pdm_es_initial_load.bat [v, V] [h, help] (Windows)
  • $ES_HOME/search/bin/pdm_es_initial_load.sh [v, V] [h, help] (Linux)
Where,
v, V
: This option displays the data load version. You can use any option with the batch file.
h, help
: This option displays the help on the batch file. You can use any option with the batch file.
To perform initial load on a specific index on Windows, execute the command as
$ES_HOME/search/bin/pdm_es_initial_load.bat index=<index_name>.
For example, to perform initial load on
sdm_index
, execute the command as
$ES_HOME/search/bin/pdm_es_initial_load.bat index=sdm_index
Refresh Index and Data on the Search Server
Typical scenarios to refresh the index and data:
  • When you execute the
    pdm_load
    command on the users, assets, knowledge base, or tickets (incidents and requests) data.
  • To refresh the changes that are done to the dependent objects, such as changes to the location or asset class name.
  • When you enable or disable tenancy.
If you selected to run initial load while configuring the Search Server details from the common installer, pdm_es_rebuild is run by default. when initial load is not checked while configuring the Search Server Details, you must manually execute the
pdm_es_rebuild
command.
  • $ES_HOME/search/bin/pdm_es_rebuild_index.bat (Windows)
  • $ES_HOME/search/bin/pdm_es_rebuild_index.sh (Linux)
When you execute the batch file from the command prompt, you can use the following options:
  • $ES_HOME/search/bin/pdm_es_rebuild_index.bat [v, V] [h, help] (Windows)
  • $ES_HOME/search/bin/pdm_es_rebuild_index.sh [v, V] [h, help] (Linux)
Where,
v, V
: You can use any option with the file. This option displays the data load version.
h, help
: You can use any option with the file. This option displays the help on the file.
To refresh a specific index on Windows, execute the command as
$ES_HOME/search/bin/pdm_es_rebuild_index.bat index
=<index_name>.
For example, to refresh the
catalog_index
index, execute the command as
$ES_HOME/search/bin/pdm_es_rebuild_index.bat index=catalog_index
To load the local specific service offerings, execute the command as:
$ES_HOME/search/bin/pdm_es_rebuild_index locale=local_name
For example, to load the service offering data in Japanese, execute the command as
$ES_HOME/search/bin/pdm_es_rebuild_index locale=ja_JP
Refresh the Smart Words for Command Bar in the Search Server
Smart Words that are created in CA SDM are not automatically updated in the Search Server. Therefore, when you type a new smart word in the Command Bar, the associated categories and templates do not display because the Search Server is not updated with the latest Smart Words.
: We recommend that you schedule a reindex at specific intervals to maintain the latest smart words and associated categories and templates in the Search Server.
Schedule a Refresh on Windows
The example in this procedure schedules the refresh once every week.
Follow these steps:
  1. From the
    Start
    menu on the
    CA Search Server
    , open the
    Windows Task Scheduler
    .
  2. For example, on Windows Server 2012, go to
    Control Panel
    ,
    System and Security
    ,
    Administrative Tools
    ,
    Task Scheduler
    .
  3. Click
    Create Task
    .
  4. On the
    General
    tab, type a name for the task.
    • Select the check box for
      Run
      whether user is logged in or not.
  5. Navigate to the
    Actions
    tab and click
    New
    .
    • In the
      Action
      field, select
      Start
      a program.
    • In the
      Program/script
      field, type the execution script name as
      pdm_es_rebuild_index.bat index=sdm_autocat_index
      .
    • In the
      Start in (Optional)
      field, set the root path of
      pdm_es_rebuild_index.bat index=sdm_autocat_index
      script and click
      OK
      .
      For Example: C:\es\bin\
  6. Navigate to the
    Triggers
    tab and click
    New
    .
    • In the
      Settings
      field, select
      Weekly
      .
    • In the
      Start
      field, select
      12:00:00 AM
      .
    • In
      Recur every
      field, type
      1
      , and select a day from the list, and click
      OK
      .
  7. On the
    Create Task
    dialog, click
    OK
    .
    You have completed scheduling the Windows task to refresh the auto-categorization index.
Schedule a Reindex on Non-Windows
Define a cron job to schedule a reindex on Linux.
Prerequisite
Enable the execute permission for the user executing the reindex script.
Follow these steps:
  1. As an administrator, log in to the search server.
  2. Navigate to the
    <search_server_install>/search/bin
    folder.
  3. Execute the following command to open the cron job file for editing.
    crontab –e
  4. Define the cron job schedule.
    Crontab job scheduling pattern:
    # Minute Hour Day of Month Month Day of Week Command
    # (0-59) (0-23) (1-31) (1-12 or Jan-Dec) (0-6 or Sun-Sat) path_to_script
    Cron job command examples:
    # execute every minute
    * * * * * cd /root/<search_server>/search/bin && ./pdm_es_rebuild_index.sh index=sdm_autocat_index
    # execute every 5 minutes
    */5 * * * * cd /root/<search_server>/search/bin && ./pdm_es_rebuild_index.sh index=sdm_autocat_index
    # execute every Friday 1AM
    0 1 * * 5 cd /root/<search_server>/search/bin && ./pdm_es_rebuild_index.sh index=sdm_autocat_index
    # execute on workdays 1AM
    0 1 * * 1-5 cd /root/<search_server>/search/bin && ./pdm_es_rebuild_index.sh index=sdm_autocat_index
    # execute 10 past after every hour on the 1st of every month
    10 * 1 * * cd /root/<search_server>/search/bin && ./pdm_es_rebuild_index.sh index=sdm_autocat_index
    You can define a cron job that is based on the scheduling pattern or can enable the cron job from the examples.
  5. Save and close the file.
    To view the cron job, execute the following command:
    $ crontab -l
select * from usm_offering where (date_available <= to_date('1901-12-13','yyyy-mm-dd') or date_available >= to_date('2038-01-19','yyyy-mm-dd')) or (date_unavailable <= to_date('1901-12-13','yyyy-mm-dd') or date_unavailable >= to_date('2038-01-19','yyyy-mm-dd')) or (date_cancelled <= to_date('1901-12-13','yyyy-mm-dd') or date_cancelled >= to_date('2038-01-19','yyyy-mm-dd'))