Release Risk Score

You use the release score to quickly assess release readiness for production deployments. In this way, you can prevent low-quality code from being delivered and exposed to your customers.
You can determine release health, risks, and quality at a glance, based on metrics collected during the development and delivery lifecycle.
You view the release score in the
RELEASES
page. The release score appears as a scale on the right in a release row and is graded from A (highest) to E (lowest) in increments of 20%:


A: 0-20%
B: 21-40%
C: 41-60%
D:61-80%
E:81-100%

An
Continuous Delivery Director
release can consist of multiple applications to be delivered to production. The release score that appears at the release level is based on the metrics for all release applications. The release score is automatically updated whenever one of the underlying metrics changes.
Release score updates can occur as a result of a change in the source control (such as new commits, or the addition to or removal from the release of applications and application versions).
You can choose how
Continuous Delivery Director
calculates a release score:
  • Continuous Delivery Director
    calculates the average of all the metrics
  • Continuous Delivery Director
    calculates the release score based on the worst-case metric score
You configure the calculation method by editing the following properties in
settings.properties
:
settings.properties ======================================= # algorithm can be worstMetric or averageOfMetrics} cdd.release.score.algorithm={worstMetric,averageOfMetrics} =======================================
To view the underlying metrics, click the A-E scale in the release row.
Release score in release row
Release score in release row
The metrics that are shown are:
Test Coverage
- Test coverage of source files that have changed during the release.
  • Calculation: Percentage of the number of changed source files that were covered by tests*100 / total number of changed source files during the release. The higher the score, the better.
  • No data available: Possible causes - None of the application source files were changed during the release; no continuous testing agent reports for this application.
  • Example: 1 (files with test coverage) / 2 (total number of files that have changed during the release) = 50% (C)
Flaky Tests
- Test suites with inconsistent/unstable results during the release.
  • Calculation: Percentage of the number of flaky test suites*100 / total number of test suites that were executed during the release. The lower the score, the better.
  • No data available: Possible causes - Test module (CDI) is inactive.
  • Example: 4 (number of flaky test suites) / 14 (total number of test suites) = 29% (B)
Code Churn
- The number of commits in the last week. A large number of commits indicates code volatility. Measuring the rate of code change is especially relevant in the week before the end of the release. At this point in time, the codebase should be stable.
  • Calculation: The number of commits / the number of days. The higher the score, the better.
  • No data available: Possible causes - No commit source (source control connection) is configured.
  • Example: If 3 files changed, 3 are counted; if the same file changes 3 times, 3 are counted.
    (A) = 0-5 files have changed in the week before release end
    (B) = 6-10 files have changed in the week before release end
    (C) = 11-15 files have changed in the week before release end
    (D) = 16-20 files have changed in the week before release end
    (E) = More than 21 files have changed in the week before release end
You can change the code churn configuration in
settings.properties
:
cdd.release.score.code_churn.period_in_days=7 cdd.release.score.code_churn.0.risk=5 cdd.release.score.code_churn.1.risk=10 cdd.release.score.code_churn.2.risk=15 cdd.release.score.code_churn.3.risk=20
Incomplete Work Items
- The percentage of the planned work items (such as user stories, defects) in the ALM tool (such as Rally, Jira) for the release applications that are not marked as completed/accepted.
  • Calculation: Percentage of the number of incomplete work items*100 / total number of work items for the release. The lower the score, the better.
  • No data available: Possible causes - No work items have been imported from the ALM tool into
    Continuous Delivery Director
    .
  • Example: 38 (number of incomplete work items) / 50 (total number of work items) = 76% (B)
Metrics for release scores
Metrics for release scores
The release score, in this example, C, is the average of the percentage score of each of the following three metrics:
Test Coverage, Flaky Tests, and Incomplete Work Items
:



29 + 50 + 76 = 155
155 / 3 = 51.66% (C)

You can filter to see only the top issues affecting the release applications that require urgent attention:
Top issues affecting release score
Top issues affecting release score
You can also view the scores for all release applications:
Release scores for all applications
Release scores for all applications
You can also see the release complexity which refers to the number of applications in the release. This metric does not affect the release score. Additionally, you can see the number of builds and whether this release is assigned to a track.
Release complexity metric
Release complexity metric
Additionally, you can see the release score when you open a release.
Release score display when release opens
Release score display when release opens
Watch Video