Manage Data Source Providers

Add a data source provider when you require a connection to a data source that  does not support. Data source providers are sets of JavaScript code that implement the extensible data source framework.
lac52
As a system administrator (
sa
), you can add data source providers when you require a connection to a data source that 
CA Live API Creator
 does not support. Data source providers are sets of JavaScript code that implement the extensible data source framework. They are system-level objects that are available to all TeamSpaces. API developers can connect to the underlying database using the data source providers that you add.
This framework includes interfaces where you can create functions. Functions establish database connectivity, discover the database objects, implement the required operations, and expose the operations in 
CA Live API Creator
.
CA Live API Creator
 includes the 
MongoDB
 and 
JDBCExample
 data source provider samples that you can use as a starting point to adding your data source provider. The 
MongoDB
 data source provider demonstrates connecting to a NoSQL database. The 
JDBCExample
 data source provider demonstrates connecting to a SQL database.
For more information about these examples, see the "Data Source Provider Examples" section.
In this article:
The Lifecycle of a Data Source Provider
The following diagram shows the lifecycle for adding a data source provider and adding a data source using the data source provider:
DataSourceProviders2Col
API developers use the following process to add a data source provider, and then add a data source using the data source provider:
  1. In their local development environment, logs in as a system administrator (
    sa
    ) user, adds a data source provider for a specific data source (for example, 
    data source x
    ) using the extensible data source framework or imports the data source provider.
    The data source provider is now available in all TeamSpaces.
  2. From an existing API, completes the following:
    1. Creates a data source, selecting 
      Data source provider
       from the 
      Create data source using
       drop-down.
    2. Selects the 
      data source x
       data source provider from the 
      Data source provider
       drop-down. The drop-down displays the data source providers that use the extensible data source framework.
      The 
      Connection
       tab for the data source displays.
      For more information about how to create a data source, see Database Connectivity.
    3. Enters the parameter values for the data source. The parameters that are displayed are specific to the data source provider.
      The 
      getConfigInfo
       function specifies which parameters are listed on this tab. For more information about this function, see The getConfigInfo Function.
    4. Activates the data source by selecting the 
      Active
       checkbox, and then clicks 
      Save
      .
 
CA Live API Creator
 does the following:
  1. Tests and verifies that the data source is configured correctly and ready to service data requests by calling the 
    testConnection
     function.
    For more information about this function, see The testConnection function.
  2. If the connection is successful, configures the data source by calling the 
    configure
     function.
    For more information about this function, see The configure Function.
  3. Retrieves the definition of the structure of the underlying data source by calling the 
    getStructure
     function. 
    CA Live API Creator
     calls this function when you reload the schema.
    For more information:
 
CA Live API Creator
 makes calls to the data source provider whenever interactions with the data are required.
Add a Data Source Provider
You can add a data source provider using API Creator or by importing it into 
CA Live API Creator
. The following procedure explains how to add a data source provider using API Creator.
Prerequisites:
 
  • You have some programming expertise.
  • You understand the API for the data source (Java in most cases) and how to interact with it from the JavaScript code.
  • You have added the JDBC driver that you want to implement and call from the data source provider's JavaScript code. 
  • You have installed the JDBC driver and restarted API Server.
    • If you have installed the self-contained, single-user version of 
      CA Live API Creator
       that is based on Jetty, copy the JDBC driver into the 
      %JETTY_HOME%/caliveapicreator/lib/ext
       directory.
    • If you have installed 
      CA Live API Creator
       on Tomcat, copy the JDBC driver (JAR file) into the 
      %{CATALINA_HOME}/lib
       directory.
  • You have system administrator (
    sa
    ) user privileges in 
    CA Live API Creator
    .
Follow these steps:
 
  1. Log in to API Creator as a system administrator (
    sa
    ) user.
  2. Click the 
    Data Source Providers
     tab. 
  3. Click 
    Add
    .
  4. Enter the data source provider name and comment for the data source provider, and then save your changes. 
  5. (Recommended) Include a logger entry for the data source provider log type in the logging configuration file while configuring external logging. Use this entry to log the events to an external logger.
    For more information about how to configure external logging, see External Logging.
  6. Select the various functions that the data source must implement and enter the JavaScript code for these functions.
    For more information, see the "Define the Data Source Provider" section.
Define the Data Source Provider
Define the data source provider by exposing capabilities. For example, allow only some queries or accept only certain data types. You expose capabilities by way of implementing functions. A minimal data source provider can consist of a few functions, whereas a full-fledged data source provider can require many hundreds of lines of code to implement the full interface.
The functions that the data source provider must implement are noted as required in the following table. The functions are grouped by behaviors that 
CA Live API Creator
 implements for the data source provider:
  • Initialization:
     Code that 
    CA Live API Creator
     calls to initialize the data source provider and test the connection to the data source. 
  • Data Retrieval:
     Code that specifies how 
    CA Live API Creator
     executes queries. 
  • Data Manipulation:
     Code that specifies how API developers can manipulate the data in a data source by way of insertion, update, or deletion of data. 
    CA Live API Creator
     translates the structure of the connected data store into a schema so that API users can perform REST operations, such as POST, PUT, and DELETE, on the data by way of REST URLs.
  • Data Transaction:
     Code that supports transactions.
The following table lists the functions. You must implement the required functions and at least one optional function. For example, if you want the ability to create objects, the data source provider must implement the 
insert
 function.
Data Source Provider Function Group
Function Name
Description
Initialization
Determines what configuration parameters are required when a system administrator (
sa
) user adds a data source provider.
Required:
 Yes
Initialization
Creates a connection to a database. 
CA Live API Creator
 hands the data source provider the configuration values that the API developer enters, and the function does whatever is required to prepare for interacting with the data source. For most data source providers, this means opening a connection object.
Required:
 Yes
Initialization
Retrieves the schema of the data source. The value that this function returns can be detailed, such as from SQL databases, or sparse, such as from NoSQL databases.
Required:
 Yes
Initialization
Verifies that you have defined the parameter values for the data source properly, that you have activated it, and that it is ready to service requests.
Required:
 Yes
Initialization
 close 
Closes the data source when you no longer require it.
Required:
 No
Data Retrieval
Gets an object from the data source. For SQL-type databases, the identifier is a representation of the primary key. For other types of databases, the identifier is database-specific. For example, MongoDB uses the
 _id
 attribute to identify documents.
This function is a commonly implemented function.
Required:
 No
Data Retrieval
Implements the general query capabilities of the provider. 
CA Live API Creator
 passes in query parameters, and the function translates these to the underlying database's query mechanisms.
This function is a commonly implemented function.
Required:
 No
Data Retrieval
Retrieves the value of a specific attribute from a specific object. 
CA Live API Creator
 uses this function to access the value of attributes that are difficult to interact with otherwise, such as binary values or large text values.
Required:
 No
Data Manipulation
 insert 
Inserts into the data source with the payload, consisting of the objects that 
CA Live API Creator
 must insert.
Required:
 No
Data Manipulation
 update 
Updates the data source with the payload, consisting of the objects 
CA Live API Creator
 must update.
Required:
 No
Data Manipulation
 delete 
Deletes an object in the data source using an identifier, which designates the object that 
CA Live API Creator
 must delete.
Required:
 No
Data Manipulation
 invoke 
Invokes a stored procedure, or whatever the equivalent is for the given data source. For typical SQL databases, the function usually maps to stored procedures. For MongoDB, the function maps to server-side MongoDB functions.
Data Transaction
 commit 
Commit all changes after other calls that specify that the underlying data source.
If the underlying data source is not transactional, then you can leave this function unimplemented.
Data Transaction
 
CA Live API Creator
 calls this function after other calls, and specifies that any changes to the data in the data source should be undone and forgotten.
If the underlying data source is transactional, then this function must instruct it to save all changes permanently. If the underlying data source is not transactional, then you can leave this function unimplemented.
Required:
 No
While defining the JavaScript code for each group of data source provider functions, you can:
  • Test by calling the data source endpoints in the REST Lab (in another browser window). This process can be iterative.
  • Debug by printing out debug statements to the Java console. 
Data Source Provider Examples
The following data source provider examples are included as examples that you can use:
  • JDBCExample:
     Use this example if you are adding a data source provider by connecting to a SQL database. This example is included with 
    CA Live API Creator
    . Duplicate this example and modify the functions for the data source provider. 
  • MongoDB: 
    Use this example to connect to MongoDB as a data source. It is included with 
    CA Live API Creator
    . If you are adding a data source provider by connecting to a NoSQL database, duplicate this example and modify the functions for the data source provider.
  • Amazon DynamoDB: 
    Use this data source provider example to connect to Amazon DynamoDB. You can add this example to 
    CA Live API Creator
     and modify the functions for the data source provider.
    For more information about this example, see GitHub.
Data Source Provider Limitations
The following are known limitations for the data sources that you add that use the extensible data source framework:
  • You can use all request/response formats except XML and CSV.
  • You can only read resources that contain a table that is based on the data source provider. To insert, update, and delete data from a data source provider, POST, PUT, or DELETE on the table.
  • You can filter the results of endpoints for data source connections that use the data source provider only using system filters.
  • If you have defined system filters on subresources that are connected to data sources that use a data source provider, you can apply filters 
    only
     using the following syntax:
    sysfilter.<resourcename>.<subresourcename>
    For example:
    sysfilter.CustomerBusObject.Orders=equal(paid:true)
  • You can add only early event rule types, event rule types, and validation rule types.
  • For data source providers that connect to NoSQL databases (for example, the 
    MongoDB
     data source provider), 
    CA Live API Creator
     cannot discover the columns within the database and returns limited schema. However, can define aliases or exclude attributes for the collections that the database returns using extended properties.
    For more information about how to define the extended properties for table-based resources, see Manage the Extended Properties for Resources.
  •  
    CA Live API Creator
     enforces security only at the REST endpoint. You cannot define specific role-level security for the entities that are exposed using data sources that use the framework.
  • When creating table-based resources and you are connected to a NoSQL database (you are using the the 
    MongoDB
     data source provider as a data source), 
    CA Live API Creator
     cannot discover the columns within the database. You can exclude attributes for the collections that the MongoDB database returns using extended properties.
    For more information, see Manage the Extended Properties for Resources.
Next Steps
Now that you have added and defined the data source provider, you can do the following:
  • Add a data source that uses this data source provider.
    For more information, see Database Connectivity.
  • Debug the data source provider. For example, in the REST Lab, you can test the data source by retrieving data that uses the data source provider.
    For more information about the REST Lab, see Test your API Using the REST Lab.
  • Iterate and define the JavaScript code for the functions in the data source provider until they fulfill your implementation requirements.
  • Deploy the data source provider as part of deploying the API Server-level configuration files.
    For more information about this step in the DevOps workflow, see Team Development.