Known Issues

This topic describes known issues in version 9.3 of the gateway, with workarounds if available.
gateway93
This topic describes known issues in version 9.3 of the
Layer7 API Gateway
, with workarounds if available.
Gateway Now Uses SunJSSE TLS Provider: Increased Time for Authentication Bind Operation
Issue:
Impacts all Gateway form factors that perform LDAP authentication. Beginning with Gateway version 9.2, SunJSSE became the primary TLS provider, which is used by LDAP user authentication when the Gateway communicates with an LDAP provider over a secure network. Unlike previous TLS providers (i.e., the deprecated FIPS/BSAFEJSSE), LDAP binds performed by SunJSSE require a full TLS handshake for each LDAP user. This process adds approximately
100 ms to the LDAP authentication bind for each user
when compared to the same process for Gateway version 9.1.
Workaround:
The Gateway provides authorization caching of previously completed or successful binds.To reduce the performance impact of the full TLS handshake for repeat binds against the same user, the authorization cache size and storage of successful binds may be increased:
  • authCache.successCacheSize (default is 2000)
  • authCache.maxSuccessTime (default is 60000 ms)
Both properties can be configured as Gateway Cluster Wide Properties - see Credential Caching Cluster Properties for more information.
The optimal size of the cache is dependent on the size of your LDAP user base. Note that the cache success will not take into account of users changing passwords or being disabled on the LDAP server while success is cached.
Cluster Inter-Node Connection Issue
Issue:
In a cluster, when a Gateway node makes an inter-node connections (to retrieve logs), the opened connections remain in close_wait indefinitely. The current maximum connections per route is 20 per node. (DE331350)
Connection Timeout During SSH Routing
Issue:
When the "SFTP" protocol is selected in the Route via SSH2 Assertion, the connection timeout is twice as long as the value specified in the assertion. (DE215650)
Workaround:
Reduce the timeout to one half the desired value.
Context Variable ${request.time} Returns Incorrect Time When Used in Encapsulated Assertions
Issue:
When
${request.time}
is used in the underlying ("backing") policy of an encapsulated assertion, it returns an incorrect value.
(DE305223)
Context Variables with ${service.*} Prefix Have a Global Scope
Issue:
Context variable that have the prefix "service" results in a globally scoped variable, instead of the local scope as expected. (DE293047)
Examples:
  • The value for context variable ${service.AAA} set in a global policy can be retrieved from the service policy. The expected result is that the variable is empty when access from the service policy.
  • The value for context variable ${service.BBB} set in the underlying policy fragment for an encapsulated assertion can be retrieved from the service policy. Assuming the variable is not configured to be returned as a parameter, the variable should be empty when accessed from the service policy.
Workaround:
Use a prefix other than "service" if you do not want the context variable to have a global scope.
Denial of Access During Database Upgrade After Patch Installation on AMI
Issue:
While upgrading the database after a Gateway patch installation, the
gateway
user is denied access to 'ssg_testUpgrade' database on AMI image, and therefore unable to upgrade. (DE287665)
Workaround:
Add the following command to grant all privileges for 'ssg_testUpgrade':
GRANT ALL privileges on ssg_testUpgrade.* to 'gateway'@'%' identified by '7layer7layer';
Dependency Errors When Deleting OAuth Folder
Issue:
Deleting an OAuth policy that contains encapsulated assertions trigger dependency errors in the Policy Manager. These errors cannot be acknowledged in bulk nor can the deletion be canceled. (DE222685)
Workaround:
To avoid this, see "Problem: Deleting OAuth policy creates dependency errors" in Troubleshoot.
False Warning Messages in Gateway Log
Issue:
The Gateway log contains a warning similar to the following:
WARNING 1 com.l7tech.server.policy.PolicyCacheImpl: 3255: Policy "Google Auth Code Extension" (#d095d9a4c56c975b34c64db5a6741dd0) contains an unlicensed assertion: Unknown assertion: RetrieveOAuth2Token
The RetrieveOAuth2Token assertion is part of the OAuth Toolkit, which is installed as part of the CA Mobile API Gateway.  (DE317426)
Workaround:
Ensure that the license for the CA Mobile API Gateway is installed. If it is, the RetrieveOAuth2Token assertion is indeed present and functioning normally. The erroneous log message is caused by internal timing issues during the license check stage of Gateway startup and you may safely ignore this log message.
File Permissions After a Restore
Issue:
Some file permissions may not be correctly restored after a Gateway is restored using the
ssgrestore.sh
command. This results in the Policy Manager being unable to connect to the Gateway. (DE221036)
Workaround:
See "Problem: Gateway is not running properly after a restore" in Troubleshoot.
Gateway Does Not Connect to Azure MySQL Database (fixed in 9.3 CR4)
Issue:
An error occurs when you try to connect the
Layer7 API Gateway
to an external Azure MySQL database. Microsoft requires usernames in the format "[email protected]_name", but the Gateway does not allow special characters in usernames. This affects the Appliance Gateway and the Container Gateway. (DE371781)
Workaround:
  1. Create a database on the localhost.
  2. Dump the database to a file using
    mysqldump
    .
  3. Create an empty database in the Azure Database for the MySQL instance.
  4. Create the user on the AzureSQL for MySQL instance with the same rights as on the local instance.
  5. Load the dump using this command:
    mysql ssg -h [host] -u [rootuser] -p < dumpfile.sql
Gateway Fails to Start with SafeNet Luna HSM Is Present
Issue:
When a SafeNet Luna HSM is in FIPS 140-2 approved configuration mode, the following error is logged: (DE243553)
WARNING 203 com.l7tech.server.util.UnsupportedExceptionsThrowsAdvice: An exception during a remote invocation is not supported by the client.java.security.KeyStoreException: Luna provider not configured
Workaround:
  1. Open the
    /opt/SecureSpan/JDK/jre/lib/security/java.security
    file in a text editor.
  2. Add the following line to the file:
    security.provider.10=com.safenetinc.luna.provider.LunaProvider
  3. Save and exit the file.
  4. Restart the Gateway.
Gateway Restart Needed for WebSocket Connection Changes to Take Effect (fixed in 9.3.00 CR1)
Issue:
When the WebSocket Connection configuration is changed for other nodes in a cluster via the Policy Manager, only the nodes where the Policy Manager is connected to pick up the changes. A restart is required for the other nodes. (DE303135)
Gateway Unable to Start Due to Unsupported Certificates (fixed in 9.3.00 CR1)
Issue:
Using the Manage Private Keys >
Import
option to import a private key results in this error:
Import failed: java.util.concurrent.ExecutionException: java.security.KeyStoreException: Unable to load software database keystore named Software DB: java.io.IOException: Only named ECParameters supported
Rather than failing, the erroneous key is actually imported. The next time you attempt to restart the Gateway, it is unable to start. (DE288220)
Importing certain unsupported keys into Gateway's Java KeyStore (JKS) results in the keystore not being able to be loaded at the next Gateway startup. Known unsupported keys are Elliptic-curve cryptography (ECC) keys without a named curve.  
Workaround:
Verify that the Gateway failure to start is due to the unsupported Elliptic-curve keys. To do this, look for the following string within the
/opt/SecureSpan/Gateway/node/default/var/logs/ssg_0_0.log
file:
java.security.KeyStoreException: Unable to load software database keystore named Software DB: java.io.IOException: Only named ECParameters supported
If this string is found restore the Gateway from a backup. If this string is not found, the Gateway inability to start could be caused by another issue. Contact CA Support for possible resolutions that do not require restoring from a backup.
Gateway Unable to Start if Syslog Server is Unreachable
Issue:
The
API Gateway
fails to start when a Syslog server is unreachable during the startup process.
Workaround:
Refer to Knowledge Base article KB000074318 for instructions on disabling the Syslog server before the Gateway starts.
Incorrect Cookie String When Using UseHTTPOnlyCookies ACO Parameter (fixed in 9.3 CR2)
Issue:
Authenticate Against CA Single Sign-on Assertion supports composing a session cookie string based on ACO parameters. After successful authentication, the generated cookie string is made available from ATTR_SESSION_COOKIE_STRING. The UseHTTPOnlyCookies ACO parameter does not reflect in the cookie string as HttpOnly when it is set to 'yes'. (DE343232)
Workaround:
Follow these steps:
  1. Define a custom parameter (for example, HttpOnlyFlag) for an Agent Configuration Object in CA SSO Policy Server and assign the value similar to that of UseHTTPOnlyCookies ACO parameter.
  2. Provide the Agent Configuration Object name in the
    CA Single Sign-on Check Protected Resource Properties
    dialog. After successful execution, the ACO parameter details are available through
    ${<prefix>.smcontext.attributes.ATTR_ACO_*}
    .
  3. After successful authentication against CA SSO, the SESSION COOKIE string is available through
    ${<prefix>.smcontext.attributes.ATTR_SESSION_COOKIE_STRING}
    .
  4. Append HttpOnly to the generated SESSION COOKIE string if the custom parameter is set to 'yes'. This custom parameter is reflected in the exposed SMCONTEXT attributes like
    ${<prefix>.smcontext.attributes.ATTR_ACO_HttpOnlyFlag}
    .
Hazelcast Node Continuously Logs WrongTargetException
Issue:
There is a known Hazlecast issue (3395) that causes WrongTargetException errors to be logged when using the Protect Against Message Replay Assertion. Additionally, some Gateways may have TargetNotMemberException logged.(DE425025)
Workaround:
Upgrade to Gateway 9.4. Gateway 9.4 installs with Hazelcast 3.10.2. The Hazelcast issue was fixed in Hazlecast 3.7.
Intermediate Certificate Not Updated for Existing Signed Private Key
Issue:
There is a known issue with replacing an Intermediate certificate in a trusted chain for an existing signed private key. (DE330274)
Workaround:
Do the following to ensure successful replacement of your Intermediate certificate:
  1. Create a new key.
  2. Generate a CSR against this key and have it signed by the new CA certificate.
The new key will have the updated certificate chain (specifically, the updated validity and expiry of the Intermediate certificate).
Intermittent JMS Error When Migrating from Gateway Version 8.1 (fixed in 9.3 CR1)
Issue:
There is a known issue with the Saxon library that causes intermittent JMS errors when migrating from Gateway v8.1 to a newer release. (DE308073)
Issues Creating a New Database Using Older Oracle Servers
Issue:
This issue applies only to servers earlier than Oracle X5-2. If you use version 9.3 to create a new database on an older Oracle server, the following error occurs: (DE333430)
Configuration Results
---------------------
An error occurred during configuration.
Unexpected error saving configuration 'Could not send Message.'
Would you like to re-configure? [Yes]:
Workaround:
When the error message is displayed, enter
No
when prompted to reconfigure. Next, select option
3
(Configure the CA API Gateway) from the menu and then complete the settings for options
1
to
4
to configure the Gateway.
JDBC Connection Test False Positives
The [Test] button in the JDBC Connection Properties may fail to detect invalid JDBC URLs. This is caused by a problem in a third-party library used by the Gateway.
Workaround:
For MySQL, ensure that the URL is in this format:
jdbc:mysql://[host1][:port1][,[host2][:port2]] ...[/[database]]
Note that if a database is defined, it must be preceded by the '/' character.
Key Generation Fails with Luna HSM
Issue:
When Private Key Properties). (DE305070, DE314879)
Workaround:
Use a key type other than the "Elliptic Curve" key types.
Manage Roles Performance Issues
Issue:
Performance and speed of Manage Roles may be impacted when many (> 500) custom roles are created, and each role has numerous (200 to 300) permissions. (DE326081)
Workaround:
Avoid excessive number of roles and permissions.
Migration Issue Between Gateways
Issue:
Migration issues may occur after upgrading. Release 9.1 introduces a new "gateway-hazelcast" FIREWALL_RULE. This rule is automatically added to the Gateway upon restart, if it does not already exist. This entity is created with a unique ID per Gateway. If you attempt a bundle import, the mapping for the "gateway-hazelcast" FIREWALL_RULE must be "MapBy:name" to prevent a 409 conflict. (DE211367)
Workaround:
Modify the bundle file that is created during bundle export as follows:
Locate this line:
<l7:Mapping action="NewOrExisting" srcId="
<unique_ID>
" srcUri="http://
<IP_address>
:8080/restman/1.0/firewallRules/
<identifier>
" type="FIREWALL_RULE"/>
And then modify it as follows:
<l7:Mapping action="NewOrExisting" srcId="
<unique_ID>
" srcUri="http://
<IP_address>
:8080/restman/1.0/firewallRules/
<identifier>
" type="FIREWALL_RULE">
<l7:Properties>
<l7:Property key="
MapBy
">
<l7:StringValue>
name
</l7:StringValue>
</l7:Property>
</l7:Properties>
</l7:Mapping>
Multi-Valued Variable Error with Look Up Item by Value Assertion
Issue:
The Look Up Item by Value Assertion fails if a multi-valued variable contains the same value multiple times. (DE305234)
For example:
Set Context Variable fave as String to: banana
Set Context Variable fruits as String to: apple,banana,pear,banana
Split variable fruits into fruitsalad on ","
Look Up Item by Value: find item ${fave} within ${fruitsalad}; output index to ${index}
Return Template Response to Requestor -- ${index}
This fails to return "1,3". However, in the ssg_0_0.log file, a similar INFO entry to the following is logged:
com.l7tech.external.assertions.xmlsec.server.ServerIndexLookupByItemAssertion: 6: More than one value was matched. Exception caught!
Workaround:
Use the following sample workaround policy:
To use the workaround policy:
  1. Download the .xml file to your computer.
  2. Click
    Import Policy
    on the Policy Tool Bar to import the policy into the Policy Manager. For more information, see Importing a Policy from a File.
No Audit Event Shown with "Show Audits Events" Option
Issue:
On the Gateway Dashboard, when you right-click a service and then select "Show Audits Events", no audit events are shown on the View Gateway Audit Events window. This occurs when two services with similar names are consumed in parallel. If "all services" are selected, the audit event is shown. (DE326622)
Occasional Inbound XMPP Connections and Sockets Issue (fixed in 9.3 CR1)
Issue:
The Gateway occasionally logs events stating that XMPP assertions are reporting an Apache Mina failure. This may result in failure of inbound XMPP connections and also prevents the Gateway from reopening inbound XMPP sockets. (DE306944)
Occasional Outbound JMS Request Processing Error
Issue:
There is a known issue with NamingException where outbound JMS messages are not processed correctly and returned the following error:
com.l7tech.server.policy.assertion.ServerJmsRoutingAssertion: 4: Error in outbound JMS request processing: Something already bound at (queue-name). Exception caught!
However when the messages are reprocessed, they behave as expected. (DE304792)
Password Expiry Issue When Running Gateway in Azure Cloud
Issue:
When using Azure to access the Gateway via SSH using a public key-based authentication as the primary method, the OS forces local users to a standard 60-days password expiration. Once expired, the session is terminated if user does not provide a current password and update it with a new one. (DE265171)
Workaround:
Once expired, the password needs to be reset via Azure Portal Web UI. See Run CA API Gateway in Microsoft Azure Cloud for more information.
Account Lock Outs When Running Gateway in Azure Cloud
Issue:
You are locked out when attempting to connect to a Gateway running in the Microsoft Azure Cloud. This is caused by the Gateway incorrectly counting successful login attempts as failure attempts. (DE368072)
Workaround:
Edit the
/etc/pam.d/password-auth
file and move the line "account required pam_tally2.so" to immediately below the "auth required pam_env.so" line. The file should resemble this:
auth required pam_tally2.so deny=5 even_deny_root_account onerr=fail unlock_time=1200 root_unlock_time=1200
auth required pam_env.so
account required pam_tally2.so
auth sufficient pam_unix.so try_first_pass
auth requisite pam_succeed_if.so uid >= 500 quiet
auth required pam_deny.so
Perform JDBC Query Assertion Error When Using Hyphen in Procedure Name
Issue:
When setting up a Perform JDBC Query Assertion, using hyphen in the stored procedure name will fail the query, as it does not comply with JDBC's naming conventions. (DE309730)
Workaround:
Do not use hyphen in the procedure name.
Performance Issue When Using Apache Libraries (fixed in 9.3 CR1)
Issue:
There is a known issue with certain Apache libraries such as Xerces performing synchronous operations in the background, which reduces performance. (DE306924)
Workaround:
To disable this behavior, pass
-Dorg.apache.xml.dtm.DTMManager=org.apache.xml.dtm.ref.DTMManagerDefault
option to the Java VM by adding it to the
/opt/SecureSpan/Gateway/runtime/etc/profile.d/appliancedefs.sh
file in the NODE_OPTS variable.
Policy Manager Browser Client Error When Using Internet Explorer
Issue:
An error occurs when logging out and then back in to the browser client version of the Policy Manager as another user. This issue only occurs in Internet Explorer. (DE211542)
Workaround:
Choose one of the following workarounds:
  • Retry the logon
  • Use Mozilla Firefox instead
Policy Manager Startup Takes A Long Time for New User
Issue:
When a new user is created and assigned various different roles, it may take up to 30 minutes for the User to log in to the Policy Manager, compared to the usual startup time for an Admin User (around a couple of minutes). (DE222762)
Workaround:
The following procedure can be used to reduce the roles items of a user, which may improve the speed and performance of permission validation:
  1. Create a new security zone. Two new roles will become available:
    • View X Zone
    • Manage X Zone
  2. Assign all the entities which should be available for a group of users (for example, developers) to this security zone.
  3. Do one of the following:
    • Only assign View/Manage X Zone roles to the user role.
    • Create a new group, assign View/Manage X Zone roles as well as other necessary predefined roles to this group. Only assign this new group to the user role.
  4. Disable auto-assign roles by setting these cluster-wide properties, to ensure the number of roles for a user does not automatically increase:
    rbac.autoRole.managePolicy.autoAssign=false
    rbac.autoRole.manageProvider.autoAssign=false
    rbac.autoRole.manageService.autoAssign=false
Policy Manager Does Not Display Correctly
Issue:
The Policy Manager may not display correctly on some high resolution monitors. If you experience display problems such as small fonts and illegible lines, try overriding the high DPI scaling behavior of the executable file. (DE211450)
Workaround:
See procedure to override high DPI scaling behavior in the Troubleshooting section on Start the Policy Manager.
Private Key is Created without Special Purpose When Loaded from Bootstrap/Bundle Folder or Sent to RESTman Endpoint
Issue:
When trying to create a private key with a defined special purpose in a RESTman bundle via the bootstrap/bundle folder of the Container Gateway, that private key is created but without the special purpose (e.g. Default CA). (DE320421)
Published Service Properties Dialog Opens Slowly Using Java Web Start Policy Manager
Issue:
When using the Java Web Start version of the Policy Manager Published Service Properties dialog for the first time takes much longer than expected. (DE326621)
Workaround:
None. The browser client needs to download additional .jar files upon first invocation of the Published Service Properties dialog. Subsequent opening returns to normal speed.
Response Processing Fails Due to Invalid Characters in Request URL (fixed in 9.3 CR2)
Issue:
Response processing fails if the request URL contains "unwise" characters that violate RFC 2396.(DE347523)
Workaround:
In 9.3 CR2, you can enable a system property that allows "unwise" characters in the request URL. For instructions on enabling this property, see DE347523 in Resolved Issues.
RESTman Bundle Import Issue (fixed in 9.3 CR1)
Issue:
Importing a bundle containing a security zone predicate does not work unless the folder or security zone already exists in the target Gateway. (DE271778)
Retrieving Certificate When Gateway in FIPS Mode
Issue:
The Add Certificate Wizard fails to retrieve a third-party certificate using the “Retrieve via SSL Connection (HTTPS or LDAPS URL)" option if the Gateway is in FIPS mode (cluster property
security.fips.enabled
= true). (DE211158)
Workaround:
  1. Retrieve or download the certificate file using external means. For example, you can use this OpenSSL command:
    # echo | openssl s_client -connect host.host:port 2>/dev/null | openssl x509 -outform PEM > cert.pem
  2. Import the certificate to the Gateway by navigating to
    Tasks > Certificates, Keys, and Secrets > Manage Certificates > Add > Import from a file
Route via HTTP(S) Assertion Unexpected Failure
Issue:
The [Other] tab in the HTTP(S) Routing Properties contains the control "Never fail as long as target returns an answer". There is a known issue that can still cause the assertion to fail even when an answer is returned under the following scenario:
  • In the [Security] tab, the Service Authentication is “Use HTTP Credentials from Request”.
  • The backend services return a 401 HTTP error. (DE215522)
Workaround:
To prevent assertion failure in this scenario, use “Specify HTTP Credentials” as the Service Authentication method instead. Specify the username and password as context variables from the request (for example,
${request.username}
,
${request.password}
).
Send Email Alert Assertion Fails with Office365
Issue:
The Send Email Alert Assertion fails when using Office365 as the SMTP server. This is not a defect and occurs due to SSL renegotiation and root certificate validation issue with Office365. (DE245778)
Workaround:
Follow the guidelines set out in Send Email Alert Assertion to configure around this issue.
Service Callback is Not Invoked for Non-HTTP Traffic with CA PAM
Issue:
When the CA API Precision API Monitoring (PAM) application is using non-HTTP protocols on the
Layer7 API Gateway
, the 'serviceFinished' callback is not invoked as expected, except for the JMS Inbound Service. (DE329295)
Broken Links in Gateway Online Documentation
CA Technologies continually improves its user documentation throughout the year. If you maintain bookmarks to topics in techdocs.broadcom.com, there is a slight chance a link may be broken due to a topic being moved or renamed. To resolve, simply search for the topic title and then update your bookmark.
(9.3 CR4) LDAP Group Query in Gateway is not Showing Results
Issue:
When running a group LDAP query, no results are found. (DE413457)
You can see a log entry in the SSG logs indicating failure to return results:
INFO com.l7tech.external.assertions.ldapquery.server.ServerLDAPQueryAssertion: 9027: The search filter (&(objectClass=user)(sAMAccountName=<name>)(memberof:1.2.840.113556.1.4.1941:=CN=<CN>,OU=groups,OU=<OU>,DC=<DC>,DC=com)) did not return any ldap entry
Workaround:
Upgrade to CA API Gateway 9.4 CR2.