Review Purpose
The TTA Tools System Description introduces multiple new concepts and requirements to the NRAO Proposal and Review process. Version 0.2 is the implementation of proposal submission and review that permits users to experience them in practice and evaluation efficacy. Please provide the following feedback broken down by Review Stages 2–4 (see below):
- Predictability. Is the user experience predictable?
- Process. Is the process reasonable?
- Usability. Too many clicks, tool tips, missing info?
- Bugs. Report any issues.
- Documentation. Are there any gaps in the documentation?
Logistics
The different review stages are listed below. Review stages 1–4 will primarily be done offline (i.e., we do not need to organize a Zoom meeting). For select individuals, however, we want to have our software front-end developer watch how the user interacts with the software. These cases are listed below. During the Consensus review we will all meet via Zoom.
- Release Proposals for Review
- TTA member configures SRP/vets proposal science category.
- TTA member sends emails.
- Conflict Declaration
- SRP members declare conflicts of interest.
- Zoom Meeting: Emily (1 hr)
- SRP Chair Assignment
- SRP chair assigns reviewers to proposals.
- Zoom Meeting: Loreto (1 hr)
- Individual Science Review
- SRP members perform Individual Science Reviews.
- Zoom Meeting: Emily (1 hr)
- Consensus Review
- Panel meets to perform Consensus Reviews.
- Zoom Meeting: All (2 hr)
Testing Plan and Roles:
We plan to simulate the review process for one Science Review Panel (SRP). To do this a TTA Member will generate about 25 mock proposals to be considered for review by one SRP. To be realistic the SRP will be comprised on one chair and five members. Here are the roles:
- TTA member (admin): Dana
- SRP chair: Loreto
- SRP members: Allie, Jeff, Amy, Robert, Emily