To both assess the efficacy of the SRDP capabilities, the usage patterns, and drive decision making based on informed data we will rely on metrics throughout the project.
Note: Unless otherwise noted collected metrics should be available on weekly, monthly, semester and cumulative timescales.
Status Progression
| Status | ||||||
|---|---|---|---|---|---|---|
|
| Status | ||||||||
|---|---|---|---|---|---|---|---|---|
|
| Status | ||||||||
|---|---|---|---|---|---|---|---|---|
|
| Status | ||||||||
|---|---|---|---|---|---|---|---|---|
|
| Table of Contents | ||
|---|---|---|
|
Archive Content Metrics
| Description | Purpose | Traceability | Status | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Report the number and volume of calibration results ingested to the archive | Track usage of space and coverage of SRDP product in the archive. | TUC02.5.1 |
| ||||||||||
Archive Usage Metrics
Unless otherwise noted all requirements should be reported both as the number of primary products and the total data volume.
| Description | Purpose | Traceability | Status | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Number of registered users that download each type of product. | This metric as different from the total download volume allows us to determine how widespread the SRDP user community is. | TUC12.5.1 |
| ||||||||||
| Total number of product downloads of each type (including anonymous users). This should be both a total number and a number per collection. | Total volume of data and products going out is one of the key performance indicators for the project. | TUC12.5.1 |
| ||||||||||
Number of products downloaded by a member of the observing team, both during and after the product proprietary period | Understand data reuse and other usage patterns. | TUC12.5.1 |
| ||||||||||
| Method of Data Delivery | Understand how users prefer to access data, deprecate unused methods. | TUC12.5.1 |
| ||||||||||
Capability Usage Metrics
| Description | Purpose | Traceability | Status | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Report the number of times the standard calibration pipeline was triggered. | Determine overall usage of the pipeline. Normalize error reporting and other pipeline related reports. | TUC02.5.1 |
| ||||||||||
| Number and type of customization choices made by the user as part of the download from the archive. | Understand usage patterns, gauge interest for various options Understand resource implications of capabilities | TUC12.5.1 |
| ||||||||||
Workflow Metrics
| Description | Purpose | Traceability | Status | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Report the number of times (and reason) that the standard products required intervention before passing QA. | Gauge efficienciency and maturity of pipeline. Estimate effort required for QA in future performance periods. | TUC02.5.1 |
| ||||||||||
| The number of times that a pipeline failed to produce a valid product and needed to switched to manual. | Understand the success rate of the pipeline to mediate between additional modes and increased robustness. | New |
| ||||||||||
Product Quality Metrics
| Description | Purpose | Traceability | Status | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Report the number of times that calibrations are reported by the PI as deficient. | Gauge efficienciency and maturity of pipeline. Estimate effort required for QA in future performance periods. Monitor the rate of bad data released by the SRDP project. | TUC02.5.1 |
| ||||||||||
Deployment Metrics
| Description | Purpose | Traceability | Status |
|---|---|---|---|