TTA Tools Sprint Feedback Page

Instructions	View TAC UI	View Resources	Publish to Directors Review
User Role: TTA Member Navigation: → Testing Steps: Select "Generate" enter a Solicitation Name, Proposal Code Prefix, and select Proposal Process select submit
User Role: TTA Member Navigation: → Testing → Select a Solicitation → Simulated Proposal List Steps: enter a Proposal Count less than 50 select "generate proposals" wait
User Role: TTA Member Navigation: → Testing → Select a Solicitation Steps: Select "Close" to close the call period of the Solicitation
User Role: TTA Member Navigation: → Testing → Select a Solicitation → Review Process Steps: select "Advance to end of Consensus Phase" pause
User Role: TTA Member Navigation: → Reviews → Select a Solicitation → Configuration → TAC Members Steps: select "Assign N SRP Chairs to TAC" and/or search for users to assign to the TAC (optional) select one or more chair icons to assign TAC Chair (optional) toggle "Active" to allow TAC member to access the TAC UI	TAC Membership does not need to be configured for TTA member to inspect TAC UI
User Role: TTA Member Navigation: → Reviews → Select a Solicitation → Scheduling Tools Steps: for each Facility in list, select "Create Version from Allocation Requests"
User Role: TTA Member Navigation: → Reviews → Select a Solicitation → Scheduling Tools Steps: for each Facility in list: For an Allocation Version, select the version number (e.g., 1) on the left side (known bug: hard refresh required to show list of proposals) select a proposal select an Allocation Disposition (known bug: sometimes trying to navigate back up a level from within an Allocation Disposition or Observation Specification Disposition boots the UI back to "Reviews")
User Role: TTA Member Navigation: → Reviews → Select a Solicitation → Scheduling Tools Steps: for each Facility in list, select "TAC" to publish Allocation Dispositions and Pressure Plots to the Resources UI and TAC UI
User Role: TTA Member; non-TTA member must be assigned to TAC and "Active" Navigation: → Reviews → Select a Solicitation → Time Allocation Review Steps: Select a Proposal to view Proposal, Feasibility Review, Science Review, and (published to TAC) Allocation Dispositions (Known refresh bug)
User Role: TTA Member Navigation: → Reviews → Select a Solicitation → Resources Steps: Select a Facility from subtab Select Time Model (if more than 1 exists) (optional) download pressure plot data (optional) right-click, save as on chart to download png (optional) download proposal summary data; filters apply to downloaded data		Allocation Dispositions must be published to populate TAC, Directors Review, or Disposition Letters resources
User Role: TTA Member Navigation: → Reviews → Select a Solicitation → Scheduling Tools Steps: for each Facility in list, select "Clone a version" or "create new version from Allocation Requests", as a version can only be published once select "Directors Review" to publish Allocation Dispositions and Pressure Plots to the Resources UI and TAC UI
User Role: TTA Member Navigation: → Reviews → Select a Solicitation → Scheduling Tools Steps: for each Facility in list, select "Clone a version" or "create new version from Allocation Requests", as a version can only be published once ensure the Allocation Disposition States are all set to "Approved" (not a supported feature in v0.2.3) select "Disposition Letters" to publish Allocation Dispositions and Pressure Plots to the Resources UI and TAC UI			Cannot actually publish to Disposition Letters as the Allocation Disposition States cannot be modified. All Allocation Disposition States must equal Approved to publish.

Issue

Solution

Notes

When trying to generate a Solicitation, the option to select a "Proposal Process" isn't displaying.
- A "Check the form for errors" message appears.

Log out of the user.

Refresh the page.

Log back in as a user.

This is a caching issue due to not implementing real logins yet. Once we enforce logins, this should not occur anymore.

Generating Proposals generates an error:

A backend error

occurred at 'api/testdata/generate_proposals' : <!DOCTYPE HTML

PUBLIC "-//IETF//DTD HTML 2.0//EN"> <html><head> <title>502 Proxy

Error</title> </head><body> <h1>Proxy Error</h1> <p>The proxy

server received an invalid response from an upstream server.<br />

The proxy server could not handle the request<p>Reason:

<strong>Error reading from remote server</strong></p></p>

</body></html>"

Wait for the proposals to populate - they are being created even though the error is displayed.

The FE is timing out after 60 seconds even though the backend does not.

Jira	QA	Comments/Testing Notes
STT-1967 - Getting issue details... STATUS
STT-1975 - Getting issue details... STATUS	DONE	cannot test until stt-1976; which is FE
STT-1976 - Getting issue details... STATUS	DONE	I expect to be able to set a seed when generating a test or demo solicitation and it produce the same data sequence each time a solicitation is generated with the same seed. 7.23.24 - This works as expected.
STT-1563 - Getting issue details... STATUS
STT-1981 - Getting issue details... STATUS	DONE	I expect that when I try to publish an AV and go to select an ATMV, I will be prevented from publishing if no ATM exists on the ATMV. 7.19
STT-1982 - Getting issue details... STATUS	DONE	I except the read_only flag to be respected on ATMVs so I should not be able to add/delete/modify any ATM or Time Reservation. I should be able to change the comments, clone, and send for publishing with any AV. Test- The BE respects boolean on ATM updates - yes 7.19 The FE respects boolean on ATM and prevents modifications (disabled text field) - no 7.19 (split to STT-1962) The BE prevents updates to Time Reservations - no 7.19 - yes 7.24 The FE prevents modification to TRs - no 7.19 (split to STT-1962)
STT-1950 - Getting issue details... STATUS	DONE	Works as expected 7.26 improvements for future work identified
STT-1979 - Getting issue details... STATUS	DONE/DEVTEST	Not that I can test it
STT-1983 - Getting issue details... STATUS	TO TEST/DEVTEST	No UI component, dependent on later work 1978
STT-1984 - Getting issue details... STATUS	TO TEST/DEVTEST	No UI component, dependent on later work 1978
STT-1985 - Getting issue details... STATUS	TO TEST/DEVTEST	No UI component, dependent on later work 1978
STT-1869 - Getting issue details... STATUS

Jira	QA	Comments/Testing Notes
STT-1661 - Getting issue details... STATUS	DONE

Jira	QA	Comments/Testing Notes
STT-1894 - Getting issue details... STATUS	DONETO TEST
STT-1919 - Getting issue details... STATUS
STT-717 - Getting issue details... STATUS
STT-1924 - Getting issue details... STATUS
STT-1925 - Getting issue details... STATUS
STT-1916 - Getting issue details... STATUS
STT-1907 - Getting issue details... STATUS
STT-1888 - Getting issue details... STATUS	DONE	5/24/24 I can make multiple allocation requests now.
STT-1883 - Getting issue details... STATUS
STT-1914 - Getting issue details... STATUS	DONE	5/24/24 This seems to work now.
STT-1913 - Getting issue details... STATUS

Jira	QA	Comments/Testing Notes
	DONETO TEST
STT-1562 - Getting issue details... STATUS	DONE	5/16/24 Cloning isn't setting the parent version to read-only cannot test version history of parent version is updated with a time stamp of when it was cloned (can we add the allocation version number of cloned?) version history of cloned version is updated to reflect what parent version spawned it and a timestamp Daniel says the log history should work but it doesn't seem to be. It now works, though there's an extra item being displayed on the AV row. I made a ticket for respecting the read only. Also found that a hard refresh is required to show the read only badge.
STT-1886 - Getting issue details... STATUS	DONE	until the new publishing endpoint is connected with the FE, this ticket cannot be further tested. Luckily it worked when I could test it. 5/15
STT-1846 - Getting issue details... STATUS	DONE	until the new publishing endpoint is connected with the FE, this ticket cannot be further tested. Luckily it worked when I could test it. 5/15
STT-1847 - Getting issue details... STATUS	DONE	until the new publishing endpoint is connected with the FE, this ticket cannot be further tested. Luckily it worked when I could test it. 5/15
STT-1893 - Getting issue details... STATUS	DONE	until the new publishing endpoint is connected with the FE, this ticket cannot be further tested. Luckily it worked when I could test it. 5/15
STT-1755 - Getting issue details... STATUS	DONE	I see the history of publication now. And when I publish another version, I see that the log history is updated.
STT-1823 - Getting issue details... STATUS	DONE	this "works" but the data it is comparing is from solicitations, not allocation versions - Reid says that needs a refactor. STT-1917 pending.
STT-1876 - Getting issue details... STATUS	DONE	There's some obvious refinement needed but this works. It doesn't apply to the pressure plot or facility reports yet. that is another story.
STT-1891 - Getting issue details... STATUS	DONE	N/A
STT-1889 - Getting issue details... STATUS	DONE	This still appears when adding a new ATM but that is outside the scope of the ticket as we need to delink pressure plots and facility reports.
STT-1887 - Getting issue details... STATUS	DONE	this was done as part of STT-1823
STT-1778 - Getting issue details... STATUS	DONE	this was done as part of STT-1755 well, you can publish an AV to one place now. I think that's okay though. I don't think we have to let a single AV go to all three places.

Dana: Redhat 8.9; Firefox 115.10.0esr (64-bit)

-----------------------
User: Dana Balser_TTA |
-----------------------

Testing
-------

1. Click Generate button.

2. Enter Solicitation Name = "dsbTest76" and Proposal Code Prefix =
"dsb76". No Demo. Click Submit.

Got an error: "Check the form for errors". I noticed the Proposal
Process was not listed which is probably the issue. Disucssion
with Allie revealed that there is sometimes an cache issue and you
need to logout and then login.

3. Logout and then Login. Try again, selecting PPR.

Simulated Proposing List
------------------------

4. Enter "25" proposals and click "Generate Proposals".

After some time I get the following error: "A backend error
occurred at 'api/testdata/generate_proposals' : <!DOCTYPE HTML
PUBLIC "-//IETF//DTD HTML 2.0//EN"> <html><head> <title>502 Proxy
Error</title> </head><body> <h1>Proxy Error</h1> <p>The proxy
server received an invalid response from an upstream server.<br />
The proxy server could not handle the request<p>Reason:
<strong>Error reading from remote server</strong></p></p>
</body></html>"

Proposals are eventually created after about 10 minutes. I receive
the email notifications and they show up in the dsbTest76
Solicitation.

5. Click "Close" to close Solicitation. Seems to work (badge changes
to closed).

Review Process
--------------

6. Click "Advance to end of Consensus Phase". After a few seconds the
review process appears to have advanced properly.

Reviews ... ---> TAC Members
----------------------------

7. Click "Assign N SRP Chairs to TAC" and then assing "Allie Costa" as
the TAC chair. Seems to work.

Reviews ... ---> Scheduling Tools
----------------------------------

8. Select "VLA" Facility and then click "Create Version from
Allocation Requests". Do the same thing for the "GBT" Facility.

9. Select "VLA" and then "1". After a refresh I can see the list of
Allocation Dispositions and do some inspection. Little clunky but
okay. Same thing for "GBT".

10. Select "VLA" and then click "TAC". Same for "GBT".

Reviews ... ---> Time Allocation Review
---------------------------------------

What is the purpose of this page?

The flow of the TAC meeting is: for each Facility the TAC chair goes
through each proposal ordered by linear rank. This is often done for
different Proposal Types (TTAT Proposal Class and Triggered Flag).
The TAC chair scans the PDF Summaries to provide a very concise
summary of the proposal to the TAC and then asks the SPR chair of the
proposal for any comments. A discussion may follow and other TAC
members may chime in. Here is an example of the concise summary given
by the TAC chair with [] indicating information that is parsed from
the PDF Summary.

"Proposal [VLA/24B-999] with a linear rank of [3.2] in the [ISM] SPR
is to perform [A Survey of hyper-compact HII regions]. The proposal
is joint with [HST] requesting [4 orbits]. There are [10 hrs]
requested in semester [24B] for [A-configuration] with priority [A],
and [5 hrs] requested in a future semester [25A] for [D-configuration]
with default priority [N]. The observations are for multiple bands:
[L,S,C,X]."

It is important to have a list of all authors so we can manage the
conflicts.

Common issues that arise needing information that is currently in the
PDF Summary:

- This is a thesis proposal.
- The SRP Comments to the TAC say do not approve of the HST time.
- The Triggering Criteria note that they need a 24 hr response.
- The SRP Comments to the PI have limited weaknesses based on our
discussion.
- The Tech comment to the TAC say the overhead has been underestimated
by a factor of two.

Less common issues that arise needing information that is not
currently in the PDF Summary (need to use the TAC Meeting page or
proposal PDF):

- The Triggering criteria are confusing, do the authors say anything
in the Scientific Justification? Technical Justification?
- I will look at the Scientific Justification to make sure that this
is indeed the authors intent.
- I will look at the Technical Review to check those details again.

I do not think that the TTAT Time Allocation Review page can serve
this general purpose. The information is present but not without some
clicking/scrolling. It could be used for the less common issues,
which require some drilling down anyhow. This is similar to the
function of the PST TAC Meeting page, but with more information and a
better interface. One positive, however, is that the TTAT Time
Allocation Review page has any proposed TAC Comments to the PI or
Internal Comments to the TAC (essentially what the GBT does with the
big table in the GBT Booklet). Any comments here may or may not need
to be discussed at this point and thus is probably best initiated by
the scheduler. Is the plan for the scheduler to be using this page?

The PST TAC Meeting page is also used for the TAC chair to check the TAC
Comments to the PI after the Directors' Review. These comments are
for each proposal and thus the TTAT Time Allocation Review page would
not be optimal since the proposal list is organized by Facility.
Also, the interface is a bit clunky for this purpose since if you
select a proposal the other proposals disappear and then you have to
hit the "x" to get the list back and then figure out where you are in
the list. Better to have the information expand and collapse.

Reviews ... ---> Resources
--------------------------

The purpose of this page seems clear: a one stop shopping for the TTA
Group member to prepare for the TAC meeting, Directors' Review, and
sending of Disposition Letters.

Pressure plot. Not clear how the various filters are suppose to apply
to the plot. Things might get complicated when you add in the
different frequency ranges to accommodate weather conditions. Too,
the pop-up with the key is annoying, blocks the view, and is not
needed (the key is above). We should probably flesh out the detailed
requirements for each Facility and find the common features.

Table. Seems like the information produced by the Download Data
button is not the same as the table in the UI. If we wanted to
produce the PDF Summaries is the plan for the TTA Group member to do
this using the Download Data button?

TO TEST

DONE

Jira	QA	Comments/Testing Notes
STT-1864 - Getting issue details... STATUS	DONE
STT-1829 - Getting issue details... STATUS	DONE	The Active TAC Members cannot see all proposals. Non SRP members that are Active TAC members cannot see any proposals. Test: Active TAC + SRP Chairs Tony - can see proposal list in TAC Active TAC + SRP member Toney - can see proposal list in TAC Active TAC Test User 2 - can see proposal list in TAC and not SRP UI Not Active TAC Test user 3 - Solicitation not listed on review UI Amy (SRP Chair) Can see SRP UI but not TAC UI Allie (SRP Chair/TAC Chair) Can See SRP UI but not TAC UI
STT-1777 - Getting issue details... STATUS
STT-1716 - Getting issue details... STATUS	DONE
STT-1663 - Getting issue details... STATUS	DONE	In UI, text field should be inaccessible 4.23 - you cannot type in the field but it still looks like you can.
STT-1852 - Getting issue details... STATUS	DONE	tested 4.16 could download the proposal summary. opened it successfully in Numbers and Excel. Filtered the proposal summary and the downloaded data reflected the filters as expected. pressure plot image can be right click and download pressure plot data downloads as a csv+json where the data bins are in json. I think this is okay once you know the data structure.
STT-1686 - Getting issue details... STATUS	DONE
STT-1831 - Getting issue details... STATUS	DONE
STT-1853 - Getting issue details... STATUS	DONE	only Large and Regular proposals are populating proposal class list as expected
STT-1854 - Getting issue details... STATUS	DONE	This is fixed but I uncovered a different bug in the is Triggered criteria here. STT-1873 - Getting issue details... STATUS created
STT-1868 - Getting issue details... STATUS	DONE	For joint proposals, the comments to pI show up on both facility tabs, which I think makes sense. The save button is global for any of the comments. I am worried about that behavior but we can revisit it.

TO TEST

DONE

Jira	QA	Comments/Testig Notes
STT-1816 - Getting issue details... STATUS	DONE
STT-1815 - Getting issue details... STATUS	DONE
STT-1829 - Getting issue details... STATUS		once a TAC is configured and a TAC member is active, they should be able to see the TAC UI if they are not active, they should no longer be able to see the TAC UI Same behavior for chairs
STT-1800 - Getting issue details... STATUS	DONE	pressure plot in scheduler tools and resources page should be live but the data is fake Eventually, we will revisit colorscheme, hatches hook up to live ADs is next version
STT-1801 - Getting issue details... STATUS		download button for pressure plot and pressure plot data available in resources page
STT-1784 - Getting issue details... STATUS	DONE	download for proposal summary information in resources page work for FE hook in another story
STT-1821 - Getting issue details... STATUS		we believe a bespoke structure is needed for a scheduler tool implementation that processes one proposal at a time versus a bulk processor An Allocation Disposition and ATM need to know what flavor of Allocation Version created them. One option would be to have flavor IDs on the ADs and ATM: one ID per flavor even if the ID isn't filled
STT-1820 - Getting issue details... STATUS	DONE	Identified a refactoring in how data is redacted for roles. preliminary work + stories written (e.g., STT-1829)
STT-1842 - Getting issue details... STATUS		many uses cases not working as expect, commented on ticket
STT-1830 - Getting issue details... STATUS		cannot test until 1842 resolved
		In Proposal Class, what are the extra classes? Joint Filtering doesn't seem to work correctly. Is it not implemented? stories created for future work

Configure TAC
- Independent from SRP Chair definition
- One or more TAC Chairs (not sure if ever used)
- Add, delete
- Persists always and not locked behind a Finalization of SRP
Allocation Version
- Create using Allocation Requests
- Per facility (in flight)
- Shows who created it
- Has publish to TAC/DR/DL buttons
  - Immutable is enforced in the BE but not FE (expected)
  - Should only be able to publish an AV to once (True)
    - A nice warning message appears to inform the user why it cannot be published
    - Expected buttons to be inactive if readonly
  - Only one AV can be uniquely published to TAC or DR or DL (False - e.g., can currently publish 2 AVs to TAC)
  - Publishing to DL checks for all(AD states == Approved) (in flight)
- has Clone Version
Allocation Dispositions
- Have high level information in roll up (demo only)
- They are called Allocation Requests 1 → should rename to Allocation Dispositions and show what Allocation Request spawned it
- Can change Allocation Disposition State per AD → doesn't persist yet
OSDs
- Each AD has 1 or more OSDs
- Set Scheduling Priority for OSDs
  - bulk (demo)
  - per OSD (demo)
- Overview
  - high level information about OSDs (demo only)
  - no longer have ability to expand OSDs in this view? That might be okay
- Manual Edit OSDs
  - List of OSDs
  - Expand OSDs (doesn't work)
  - Edit (not implemented)
- Effective Science Target List
  - Not implemented
ATM
- View Pressure Plot (demo)
- Create a new ATM (demo)
- Configure (demo)
- Make Time reservations (demo)
  - Weather
Misc Scheduler Tools
- View Observatory Copy
- View Proposal Review (in flight)
- Add Comments for TAC (demo)

Dana: Redhat 8.9; Firefox 115.5.0esr (64-bit)

---------------------------------
User: Dana Balser_TTA |
---------------------------------

Testing
----------

1. Click Generate button.

2. Enter Solicitation Name = "dsbTest68", Proposal Process = "PPR",
and Proposal Code Prefix = "dsb68". No Demo. Click Submit.
Nothing appears to happen but when I scroll down I can see the new
Solicitation.

3. Click "dsbTest68". I can now see additional info.

Simulated Proposal List
---------------------------------

1. Enter Proposal Count = "75". Click "Generate Proposals". Takes
about 25 seconds to diplay proposals. After a few minutes all of
the 75 email notifications arrive in my inbox. Format on rhs of
this page is a bit sloppy (not aligned).

2. Click "Close" to close the Solicitation. I can see the "close"
badge and the "Generate Proposals" button is grayed out. Good.

Review Process
----------------------

There are four buttons: (in green) "Vet", "Generate Panels", "Finalize
Configuration", and (in blue) "Advance to end of Consensus Phase". So the green
buttons are more specific actions, whereas the blue buttons are more general?

1. Click "Vet". Within a second or so I see the proposals vetted
counter go from 0 to 75 and the "Vet" button is grayed out. The
Reviews-->Configuration-->Vetting tab for this Solicitation looks
correct.

2. Click "Generate Panels". With a second or so I can see the SRP
panels go from 0 to 3. The Generate Panels button is still active,
presumably since I can create more panels. Also, I can see the
list of panels on this page: GWT (21 proposals), NGA (31
proposals), and SFM (23 proposals). Looks okay on
Reviews-->Configuration-->Panels.

For a given panel there are two tabs: ISR and Consensus. Under ISR
there are four buttons which are all grayed out: Set & Certify
Conflicts, Generate ISR comments, scores, and set Review Types, Set
ISR Review State to Finalized, and Start Consensus Review. Under
Consensus there are three buttons which are all grayed out:
Generate Consensus Comments, Set Consensus Review State to
Completed, and Set Consensus Review Type to Finalized. Presumably,
I need to finalize the configuration before these buttons are
active.

3. Click "Finalize Configuration". Panel Config Finalized goes to
"Yes". Both the Generate Panels and Finalize Configuration buttons
are now grayed out. I can now see 126 ISRs under the GWT panel. I
get the 18 (6 reviewers times 3 panels) email notifications letting
the reviewer know we are open for business. The buttons under ISR
are now active, except for "Start Consensus Review". So we seem to
be allowing the ISR review state to be finalized before we set and
certify conflicts. Is this correct?

GWT
Individual Science Reviews
-------------------------------------

1. Click "Set & Certify Conflicts" button. The GWT tab is closed.
When I open the tab I can see the conflicts have been certified.
Not sure why the tab closed. I confirm this on the Reviews tab.
Perform the same with the other panels.

2. Click "Generate ISR comments, scores, and set Review Types" button.
The button did not gray out. I can see the review types are set
and the review is in the saved state. The tab did not close this
time. I confirm this on the Reviews tab. Seems like we assign all
reviews that are possible. Probably okay but not typical. Perform
the same with the other panels.

3. Click "Set ISR Review State to Finalized" button. Review state
goes to finalized. Now the second and third buttons are grayed out
(everything except "Start Consensus Review"). Seems okay. Perform
the same with the other panels. Looks okay on the Reviews page. I get
the 3 email notifications.

4. Click "Start Consensus Review" button. The button is grayed out.
I can see the proposals and reviews.

GWT
Consensus
----------------

1. Click "Generate Consensus Comments" button. State goes to saved
and the button is grayed out. On Reviews I see that some comments
have been added. Perform the same with other panels.

2. Click "Set Consensus Review State to Completed" button. State goes
to completed and the button is grayed out. Perform the same with
other panels.

3. Click "Set Consensus Review Type to Finalized" button. States goes
to finalized and the button is grayed out. Perform the same with
other panels. The blue button "Advance to end of Consensus Phase"
is now grayed out. I receive 3 email notifications that the
consensus reviews have been finalized.

TO TEST

DONE

Jira	QA	Comments
STT-1630 - Getting issue details... STATUS	DONE	would be nice if the badges has consistent spacing in proposal list
STT-1631 - Getting issue details... STATUS	DONE	would be nice if the badges has consistent spacing in proposal list
STT-1616 - Getting issue details... STATUS
STT-1617 - Getting issue details... STATUS
STT-1618 - Getting issue details... STATUS
STT-1672 - Getting issue details... STATUS	DONE
STT-1671 - Getting issue details... STATUS	TO TEST
STT-1674 - Getting issue details... STATUS	DONE
STT-1678 - Getting issue details... STATUS
STT-1675 - Getting issue details... STATUS	DONE

Dana: Redhat 8.9; Firefox 115.5.0esr (64-bit)

---------------------------------
User: Dana Balser_TTA |
---------------------------------

Testing
----------

1. Click Generate button.

2. Enter Solicitation Name = "dsbTest67", Proposal Process = "PPR",
and Proposal Code Prefix = "dsb67". No Demo. Click Submit.
Nothing appears to happen but when I scroll down I can see the new
Solicitation.

3. Click "dsbTest67". I can now see additional info. Seems to take a
while to load since the summary of info does not populate right
away. I also see the interface to generate the Solicitation which
is confusing. This goes away if I leave this page and come back.

Simulated Proposal List
---------------------------------

1. Enter Proposal Count = "75". Click "Generate Proposals". Takes
maybe 10 seconds and then displays the 75 proposals. I can see
these on the Proposal tab. I get 75 email notifications. Appears
to spread these between three science categories: NGA, SFM, GWT.
Appears you cannot specify the number of science categories. Will
this be important for testing?

2. Click "Close" to close the Solicitation.

Review Process
---------------------

There are four buttons: "Vet", "Generate Panels", "Finalize
Configuration", and (in blue) "Advance to end of Consensus Phase".

1. Click "Vet". Within 1 second I see the display change from 0 to 75
vetted. I go to Reviews->Configuration->Vetting and can see that they
indeed have been vetted.

2. Click "Generate Panels". Within 1 second I see the display change
from 0 to 3 SRP panels. I do not see the list of panels on the
current page. I go to Reviews->Configuration->Panels and can see
that there are three SRPs: SFM (21 proposals); GWT (25 proposals);
and NGA (29 proposals). There are 6 SRP member per panel and one
chair. When I return to Testing->Review Process I can now see the
list of panels below and can open each SRP to view more buttons.
Appears you cannot specify the number of panels. Will this be
important for testing?

At this stage there are two layers of actions/buttons. Under
Review Process I can still click "Vet" and "Generate Panels"; that
is, they are not grayed out. Seems odd since I should not be able
to generate anymore proposals since the Solicitation is closed.

3. Click "Finalize Configuration". I can now see a list of the
proposals under "Individual Science Review". The "Finalize
Configuration" button is grayed out but not the "Vet" and "Generate
Panels" buttons. Odd. I do get 6x3 email notifications letting
the reviewers know that the panel is open for business.

Individual Science Reviews
--------------------------------------

There are four buttons: "Set & Certify Conflicts", "Generate ISR
Comments, scores, and set Review Type", "Set ISR Review State to
Finalize", and "Start Consensus Review". The last button is grayed
out at the start which makes sense. Should the second and third
buttons be grayed out? Seems like we should separate "Set Review
Type" from "Generate ISR Comments and Scores".

1. Click "Set & Certify Conflicts" in SFM panel. Nothing appears to
happen. I still see the Not Certified badge. When I go to Reviews
it appears the conflicts have been set but not certified.

***N.B., I stopped testing at this point and resumed in about 24 hours.***

I still noticed the SFM conflicts are not certified but after going
back and forth between Reviews and Testing tabs the reviews appear to
be certified. That is, the badges say certified now.

2. Click "Generate ISR Comments, scores, and set Review Type" in SFM
panel. Not clear if anything is happening. I go to the Reviews tab
to check there. At some point I get the following error:

A backend error occurred at 'api//testdata/set_isr_comments_scores/28' : <!DOCTYPE html> <html> <head> <title>Error</title> <style> body { width: 35em; margin: 0 auto; font-family: Tahoma, Verdana, Arial, sans-serif; } </style> </head> <body> <h1>An error occurred.</h1> <p>Sorry, the page you are looking for is currently unavailable.<br/> Please try again later.</p> <p>If you are the system administrator of this resource then you should check the error log for details.</p> <p><em>Faithfully yours, nginx.</em></p> </body> </html>

I try to reload the browser. Takes tens of seconds to reload data;
that is, when you click on a menu it expands and quickly collapses.
Maybe try another panel.

3. Click "Set & Certify Conflicts" in NGA panel. I notice the
certified badge turn from gray to green but the text says "Not
Certified". Go to Reviews tab and back. No change. Wait for
maybe 10 minutes. No change, I still see "Not Certified". Reload
browser. Again need to wait...wait...wait...okay now the badges
say "Certified" and "Set & Certify Conflicts" button is grayed out.

4. Click "Generate ISR Comments, scores, and set Review Type" in NGA
panel. Again, not clear anything is happening. The badges have
Blank and None. If I go to Reviews->Summary I do not get the
purple "? Conflict" in the matrix; I can see the "PSTN" cells, but
they are all set to N. And the reviews are blank.

At this point here are two general comments and a question:

(i) Similar to the Reviews->Summary page I am a bit saturated with
information. Several layers and lots of buttons. But I can see
the general flow which is reasonable.

(ii) I do not quite get the logic of the graying buttons. Sometimes I
expected the button to be grayed and it was not and vice versa.

(iii) Can we use this functionality to advance a working Solicitation?
For example, if there is some issue in a current working
Solicitation and we wanted to see how this issue would propagate
downstream, could we copy this Solicitation to test and use this
functionality to do advance the working Solicitation? Or would
we have to generate a test Solicitation instead?

TO TEST

DONE

Work	QA	Comments
STT-1620 - Getting issue details... STATUS	DONE	Generate Solicitation Change "Use Realistic Data" to "Set as Demo" Change it so the check box is not set by default Solicitation -> Proposals Change tab name from "Proposals" to "Simulated Proposal List" Remove "vetted" counter in favor of the update in STT-1594 ~~Timeout "Generate Proposals" button until the proposals are actually generated. Otherwise it is too easy to multiclick it~~ ~~Add feedback that proposals are generating~~ ~~e.g., I generated 60 proposals and it took some time, which is fine. But there wasn't any feedback in the UI so I could have clicked the button right away again which would have exasperated the issue~~ Solicitation -> Reviews Change name of tab from "Reviews" to "Review Process" ~~Change "Run All Steps" to "Advance to end of Consensus Phase" (completed)~~ ~~Move the former "Run All Steps" button to the far right of the list. (completed)~~ Make the individual step buttons a different color to differentiate it from "Closed", "Generate Proposals", and the former "Run All Steps" actions Vet, Generate Panels, Finalize Configuration (ISRs) Set & Certify Conflicts, Complete, Finalize ISRs, Initiate Consensus (Consensus) Generate Comments, Complete, Finalize Move the individual step buttons to be left justified and in this order Vet, Generate Panels, Finalize Configuration (ISRs) Set & Certify Conflicts, Complete, Finalize ISRs, Initiate Consensus (Consensus) Generate Comments, Complete, Finalize Solicitation -> Reviews -> Panel's ISR Add a badge per row to show that the reviewer is certified (there is not UI indication that the action is completed right now) Change "Complete #" to "Generate ISR comments, scores, and set Review Types" Do not remove the former "Complete #" button once the reviews are finalized; just make it inactive Change "Finalize ISRs" to "Set ISR Review State to Finalized" Make "Initiate Consensus" button always shown but not active unless it is supposed to be Change "Initiate Consensus" to "Start Consensus Review" to make it consistent with wording in the real Reviews tab Solicitation -> Reviews -> Panel's Consensus Change "Generate Comments" to "Generate Consensus comments" Change "Complete #" to "Set Consensus Review State to Completed" Change "Finalize" to "Set Consensus Review State to Finalized" Do not remove "Complete #" button once finalized Remove Mean, St.Dev. and SRP Score from rows
STT-1622 - Getting issue details... STATUS	DONE	12.6.23
STT-1581 - Getting issue details... STATUS
STT-1621 - Getting issue details... STATUS	DONE	Clicking on a user's name in a simulated panel's ISR directs to a deadpage of headers that doesn't even load. I suspect it is trying to load as if I were logged in as that user. Copying the URL and then logging in as that user does show their list of ISRs. Since the TTA member is the only one who can access the simulation page, change this behavior so the TTA member gets the list of the user's ISRs a PPR "Run all Steps" should not work if the Solicitation is open have to close solicitation and then all steps (vetting, srp finalize, ...) can then be run as part of Run all.
STT-1545 - Getting issue details... STATUS	DONE	All three SC defined solicitation have proposals and each has a panel; each panel has people in it. Run all button working on all panels Works without one of the SCs having a proposal so therefore it isn't created
STT-1466 - Getting issue details... STATUS	DONE	12.11.23
STT-1551 - Getting issue details... STATUS	DONE	12.6.23
STT-1594 - Getting issue details... STATUS		it's done but doesn't quite follow the requested info in the ticket.
STT-1619 - Getting issue details... STATUS	DONE

Work	QA	Comments
STT-1473 - Getting issue details... STATUS	TO TEST	I think done but does it respect mid process changes? Make panel, do some ISRs and some assignments. hit Complete ISRs It does respect previous changes but it will assign multiple Primary or Secondaries without error. E.g., If I assign a Primary by hand, then hit the Complete testing button, it will assign an additional primary as well.
STT-1476 - Getting issue details... STATUS
STT-1477 - Getting issue details... STATUS	DONE	I think done but does it respect mid process changes? Make panel, do some ISRs and some assignments. hit Complete ISRs Yes is does respect changes. It does not autofill comments (previous ticket not yet done). I'm surprised it allows the state change. This must mean it is bypassing validation.
STT-1478 - Getting issue details... STATUS	DONE
STT-1546 - Getting issue details... STATUS
STT-1547 - Getting issue details... STATUS

Work	QA	Comments
STT-1548 - Getting issue details... STATUS	NA
STT-1313 - Getting issue details... STATUS		need to refresh to get the chair and the reviewers in Reviews Tab
STT-1312 - Getting issue details... STATUS	DONE
STT-1471 - Getting issue details... STATUS	DONE	need refresh in testing UI to see ISRs once finalized config doesn't give feed back and "turn off"
STT-1472 - Getting issue details... STATUS	NOT COMPLETED	need indicator on testing UI that set & Certify is done doesn't respect previously modified ones
STT-1537 - Getting issue details... STATUS		skip
STT-1542 - Getting issue details... STATUS	DONE	prototype
STT-1475 - Getting issue details... STATUS	DONE
STT-1473 - Getting issue details... STATUS		not done - isr comments
STT-1474 - Getting issue details... STATUS	DONE	respects previously finalized ones. error shown when invalid ISRs for finalization (need more verbose statement) does not finalize as TTA member

Work	QA	Comments
STT-1532 - Getting issue details... STATUS	DONE	10.20.23
STT-1463 - Getting issue details... STATUS	DONE	10.24.23
STT-1533 - Getting issue details... STATUS	DONE	10.20.23 - Done for Consensus, for ISR 10.24.23
STT-1541 - Getting issue details... STATUS	DONE	10.24.23
STT-1540 - Getting issue details... STATUS	DONE	10.24.23
STT-1487 - Getting issue details... STATUS	DONE	10.31.23
STT-1539 - Getting issue details... STATUS	DONE	10.24.23
STT-1544 - Getting issue details... STATUS	DONE

Work	QA	Comments
STT-1498 - Getting issue details... STATUS	DONE	10.6.23 bug found; comment left on ticket. Fixed 10.10.23
STT-1493 - Getting issue details... STATUS	DONE	10.6.23
STT-1494 - Getting issue details... STATUS	DONE	10.10.23 Works even with external reviewers, multiple conflicted, multiple Tertiary
STT-1481 - Getting issue details... STATUS	DONE	10.6.23
STT-1483 - Getting issue details... STATUS	DONE	10.6.23
STT-1431 - Getting issue details... STATUS	DONE	10.6.23
STT-1495 - Getting issue details... STATUS	DONE	looks as expected but needs a space between the badge and the preceding text; SRP members cannot see conflicted badges unless they are conflicted then they only see themselves, which is duplicate information at that point. Conflicted status should have not badges for roles; SRP members do not need list of conflicted badges
STT-1522 - Getting issue details... STATUS	DONE	it's fine but will iterate with more feedback: hover-over text? collapse titles+info for each/all/none? make column headers float? cell shader toggler to hide radial buttons and only display reviewtype not equal to none roles?
STT-1501 - Getting issue details... STATUS	DONE	10.10.23
STT-1525 - Getting issue details... STATUS	DONE	10.10.23
STT-1427 - Getting issue details... STATUS	DONE	10.12.23
STT-1502 - Getting issue details... STATUS	DONE	10.12.23
STT-1482 - Getting issue details... STATUS	DONE	10.17.23
STT-1499 - Getting issue details... STATUS	DONE	10.12.23
STT-1496 - Getting issue details... STATUS	DONE	10.16.2023
STT-1484 - Getting issue details... STATUS	DONE	10.13.23
STT-1527 - Getting issue details... STATUS	TO TEST
STT-1497 - Getting issue details... STATUS	DONE	10.13.23
STT-1524 - Getting issue details... STATUS	DONE	10.13.23
STT-1531 - Getting issue details... STATUS	DONE	10.16.2023
STT-1530 - Getting issue details... STATUS	DONE	10.16.2023
STT-1529 - Getting issue details... STATUS	DONE	10.16.2023
STT-1500 - Getting issue details... STATUS	DONE	the badges aren't different colors in the ISR nor Consensus list. have screengrab - fixed 10.17.23
STT-1486 - Getting issue details... STATUS	TO TEST	the guidance text on the conflict declaration page shouldn't be there

Just a note - when someone is an external reviewer, they see the "select a panel" screen instead of being taken directly to their SRP

Work	QA	Comments
STT-1481 - Getting issue details... STATUS
STT-1483 - Getting issue details... STATUS
STT-1431 - Getting issue details... STATUS
STT-1470 - Getting issue details... STATUS	DONE
STT-1480 - Getting issue details... STATUS	DONE
STT-1465 - Getting issue details... STATUS	DONE
STT-1440 - Getting issue details... STATUS
Conflict Declaration Make the "Certify Conflict Declarations" button always present on the Conflict Declaration page but only active once the correct trigger conditions are met (currently the latter is true but not the former, as an inactive button only appears once a single conflict has been modified) Update "Certify Conflict Declarations" to "Submit & Certify Conflict Declarations" Update color palette such that conflict state Unknown is the most visible state. The important aspect is that they change from unknown, not so much that the state is available or conflicted. They also do not spend a lot of time on the page or see it again, so it is not necessary to highlight which proposals they are available on in this UI.	DONE DONE DONE
If a reviewer is only on one panel, let their selection of a solicitation put them directly into the panel instead of needing to select the panel.	DONE
ISRs Add expandable instruction widget (see text below) Update "Mark Complete" button to "Save & Complete" Update color palette so save and complete are different colors or at least different shades of blue (only this UI, leave summary UI alone) Add instruction text inside of "Upload Reviews" widget, (see below) Hide Blank and None badges in ISR list Update default order of proposals to be by conflict state, then Review Type (e.g., P, S, T, Conflicted) Track number of (P,S,T + Blank) proposals vs (P,S,T + not Blank) in counter Make Finalize button always "there" but in inactive until conditions are met to use. Text for Upload Reviews You may upload a csv file with comments and scores for one or more proposals for which you are not conflicted. A template with the necessary column headers is provided via the "Download Reviews" feature. Note, you should edit the downloaded template to only include the proposals for which you want to modify the score and comment. Text for ISR instructions. can we make [primary], [secondary], or [tertiary]as stylized versions to represent the UI? The SRP Chair may assign you a specific role for each proposal: Primary, Secondary, or Tertiary. As a Primary reviewer, you will guide the discussion on the proposal's strengths and weaknesses during the Consensus meeting. Once roles are designated, they will be annotated next to the corresponding proposals as [primary], [secondary], or [tertiary]. If you do not have an assigned role, no review is expected from your end. For each proposal assigned to you, you will review the proposal for its scientific merit and provide comments and a score. The comments remain confidential amongst the panel members. Each assigned proposal requires a score between 0.1 and 9.9, where the lower number denotes a better-ranked proposal. You should use the whole range to score your proposals, as the scores will be normalized for the Consensus meeting. Once each assigned proposals has both a score and comment, you can finalize your reviews, which will prevent further edits. Note, marking them as complete is a convenience feature and alterations can still be made if needed. All assigned reviews are required to be finalized prior to the Consensus meeting.	DONE DONE SKIPPED DONE REVIEW MODIFIED REVIEW DONE	Will do 3 as a whole system update later on 5, the Review State and Review Type are now misaligned because not every row has both; e.g., Review Type None Review State Saved on 6, we have a sort "Assigned to Me" option it is not the default though on 7, the Assigned Blank is working as I expect but the Saved/Completed is tracking None review types. I think it should only do Assigned review types. Not sure if it useful to have a counter for blank or total reviews
Consensus Conflicted badge in Consensus list of proposals should line up with review type badges Update color palette so save and complete are different colors or at least different shades of blue, similar to ISR Hide Blank badges Add expandable instruction widget (see text below) Make Finalize button always "there" but in inactive until conditions are met to use (chair and tta member only) Text for Consensus instructions (TBD) Only Primary and Secondary reviewers may save or edit the Consensus Comments to the PI. Once a Consensus comment is marked as Complete, only the SRP Chair can edit it further if they are not conflicted.	DONE SKIPPED DONE REVIEW DONE	on 4, can the text have the stylized badges of (P) and (S) after the words "Primary and Secondary" so.. "Primary (P) and Secondary (S)"

Dana (Firefox, Fedora)

Goal: One last run through the system to make sure we are ready for
the V0.2 Internal Testing.

-----------------------
User: Dana Balser_TTA |
-----------------------

Solicitations
-------------

1. Generate new Solicitation: "dsbTest61", "dsb61", "PPR", and "25".
Looks good.

2. Close Solicitation. Looks good. Got the 25 emails.

Reviews
-------

1. Configure SRPs. Add panel "GWT Panel" with "GWT" as the science
category. Select reviewers: Lorant (chair), Allie, Mark, Amy,
Ryan, and Emmanuel. Looks good.

2. Vet Science Categories. Looks good.

3. Save and Finalize. Seems okay. I am brought back to the Summary
page. Received the 6 emails, one for each SRP memeber, that the
panel was open for business.

-------------------------
User: GWT Panel Members |
-------------------------

1. Declare conflicts. UX is better. I can see the certified badge
right away along with the new page to start my reviews.

------------------------
User: Lorant Sjouwerman | Chair
-------------------------

1. Assign Reviewers. Overall okay. Still a bit hard to absorb.

-------------------------
User: GWT Panel Members |
-------------------------

4. Individual Reviews. Login as each reviewer and use the file
download/upload functionality to enter reviews. We probably want
to add the proposal title to the file since this will help to
recall the proposal in question. Once all members have finalized
their reviews I get the email notification.

-----------------------
User: Dana Balser_TTA |
-----------------------

Reviews
-------

1. Start Consensus.

-------------------------
User: Lorant Sjouwerman | Chair
-------------------------

Reviews
-------

1. Consensus Reviews. The "T's" now show up for the chair and I can
see the two Tertiary reviewers.

2. Revote. Seems like this is working now. When I change an
individual normalized score the mean and SRP Score are updated.
The orignial individual normalized score is displayed. We do not
see the orignial mean normalized score but that is probably okay.
If I change the SRP score directly I can still revote and update
the mean, but not the SRP score. Looks good.

3. Download Reviews. This is now gone as requested.

------------------
User: Allie Costa:
------------------

Reviews
-------

1. Concensus Reviews. This all seems to work okay. I can edit
Primary and Secondary text only. I cannot change Individual
Normalzied score or SRP score. If I complete a review then I can
no longer make changes but the chair can do so.

-------------------------
User: Lorant Sjouwerman | Chair
-------------------------

Reviews
-------

1. Complete and Finalize. I go through and check that even when a
review is completed I can change the review as chair. Once all of
the reviews are completed I see the Finalize button. Once I click
this button then I cannot make any changes as chair. I also receive
an email notification.

Conclusion: Looks good to go for internal review.

	Dev	QA
STT-1365 - Getting issue details... STATUS	DONE
STT-1349 - Getting issue details... STATUS	DONE	~~Non-TTA can still see all Solicitations, even if they do not have a role on any panel associated with the solicitation. 8.22.23~~ Fixed on dev 8.28.23
STT-1358 - Getting issue details... STATUS	DONE	Not sure how this is different? 8.28.23
STT-1355 - Getting issue details... STATUS	DONE
STT-1397 - Getting issue details... STATUS	BUG	the Mean of the FNISs and SRP scores are still consistent in their display see ticket with new image. 8.22.23
STT-1393 - Getting issue details... STATUS	DONE
STT-1355 - Getting issue details... STATUS	DONE	9.1.23
STT-1407 - Getting issue details... STATUS	DONE
STT-1205 - Getting issue details... STATUS	DONE
STT-1403 - Getting issue details... STATUS	DONE
STT-1204 - Getting issue details... STATUS	DONE
STT-1408 - Getting issue details... STATUS	DONE
STT-1385 - Getting issue details... STATUS	DONE	9.1.23
STT-1359 - Getting issue details... STATUS	DONE	I think wait for external reviewers to comment.
STT-1347 - Getting issue details... STATUS	DONE
STT-1400 - Getting issue details... STATUS	DONE
STT-1346 - Getting issue details... STATUS	DONE
STT-1384 - Getting issue details... STATUS	DONE
STT-1416 - Getting issue details... STATUS	DONE	9.13.23
STT-1413 - Getting issue details... STATUS	DONE
STT-1412 - Getting issue details... STATUS	DONE
STT-1396 - Getting issue details... STATUS	DONE
STT-1415 - Getting issue details... STATUS	DONE
STT-1414 - Getting issue details... STATUS	DONE
STT-1402 - Getting issue details... STATUS	DONE
STT-1375 - Getting issue details... STATUS	DONE
STT-1423 - Getting issue details... STATUS	DONE	9.1.23
STT-1419 - Getting issue details... STATUS	DONE
STT-1422 - Getting issue details... STATUS	DONE	9.1.23
STT-1364 - Getting issue details... STATUS	DONE	9.1.23
STT-1374 - Getting issue details... STATUS	BUG	9.5.23
STT-1421 - Getting issue details... STATUS	DONE	9.5.23
STT-1425 - Getting issue details... STATUS	DONE	9.5.23
STT-1429 - Getting issue details... STATUS	DONE	9.5.23
STT-1430 - Getting issue details... STATUS	DONE	9.5.23

STT-1386 - Getting issue details... STATUS	DONE	9.1.23
STT-1401 - Getting issue details... STATUS	DONE
STT-1407 - Getting issue details... STATUS	DONE

Dana (Firefox, Fedora)

Goal: Test run to see if we are ready for the V0.2 Internal Testing.

---------------------------------
User: Dana Balser_TTA |
----------------------------------

Solicitations
------------------

1. Generate new Solicitation: "dsbTest60", "dsb60", "PPR", and "25".
Looks good.

2. Close Solicitation. Looks good.

Reviews
------------

1. Configure SRPs. Add panel "GWT Panel" with "GWT" as the science
category. Select reviewers: Lorant (chair), Allie, Mark, Amy,
Ryan, and Emmanuel. Looks good.

2. Vet Science Categories. Looks good.

3. Save and Finalize. Okay, but not clear that I have finalized
unless I leave the page and come back.

-------------------------------------
User: Lorant Sjouwerman | Chair
--------------------------------------

1. View Summary page. First impression. Not bad. I can view all 6
reviewers on one page. Still a bit busy but without much effort I
can see the status: the folks that have not declared their conflict
yet and some that have been automatically conflicted (see here).

2. Declare conflicts. Once I hit the button the badge still says
uncertified. I go to the Summary page and still see that Lorant's
reviews are uncertified. When I refresh the browser I can now see
that Lorant's reviews are certified but the proposal list is
missing (see here). The proposals are also missing on Individual Reviews
tab. If I login as another reviewer (Allie Costa) I cannot see the
list of proposals to perform conflict declaration (see here). If I go to the
Proposals tab as a TTA Member (Dana Balser_TTA) I do not see the
list of proposals. After refreshing the browser a few times I can
see the proposals. Then if I go back to the Reviews tab I can see
the list of proposals on the Summary page and the proposal-reviewer
matrix on the Reviews->Reviews->Summary page. If I login as the
chair (Lorant Sjouwerman) I can now see the proposal-reviewer
matrix on the Summary page. Next login as each SRP member and
declare conflicts. Again I do not initially see the certified badge
after clicking the button but otherwise it seems to work.

3. Assign Reviewers. The UX is okay but the number of P,S,T, and N
values are hard to absorb. Partly because the numbers are small
and hard to read. Also, the overall matrix is complex once the
P,S,T, and N boxes are shown. Lots of small boxes with different
colors.

4. Individual Reviews. Login as each reviewer and use the file
download/upload functionality to enter reviews. The Individual
Reviews page is improved. Conflicts are clear and I can filter. I
like the UX when finalizing the ISR---the popup with high/low
score. Once the reviews are finalized for a given reviewer the UX
on the Summary page becomes less cluttered as the P,S,T, and N
boxes go away and the cell is labeled as Primary, Secondary, etc.
This is not the case for proposals with review type N. At this
point, however, there are just too many colors (see here). Maybe just remove
all colors associate with P,S,T, and N.

---------------------------------
User: Dana Balser_TTA |
---------------------------------

Reviews
------------

1. Start Consensus. As discussed the location of this functionality
will change as having a button on the Summary page is not what we
want. When the button is clicked the matrix changes a bit. The
P,S,T, and N buttons go away for those cells with N and the words
None are listed. This is because once consensus is started you can
no longer change the review type. Okay.

-------------------------------------
User: Lorant Sjouwerman | Chair
-------------------------------------

Reviews
-------

1. Consensus Reviews. Overall the layout is pretty good. Couple of
issues. Lorant's Tertiary reviews are not listed in the column
next to the proposal ID. I have "P", "S", and "Conflict Symbol",
but no "T" for SRP chair. Too, there is only one Tertiary reviewer
name listed under that proposal title.

2. Download Reviews. I get the consensus reviews for all
Solicitations instead of just "dsbTest60". We will want additional
info here (e.g., mean, std., etc.).

------------------------
User: Allie Costa:
-------------------------

Reviews
------------

1. Consensus Reviews. For an SRP member I do get "T" in the column
next the proposal ID. No Primary, Secondary, Tertiary names under
the Title. This seems reasonable since the chair is running the
meeting. I notice that I cannot see Consensus comments or
Individual comments for proposals for which I am conflicted. I
cannot edit Consensus comments if T or N but can view them. This
appears correct.

2. Enter Consensus Comments. Enter draft comments manually for
Primaries. I have to enter Internal comment to save which should
not be a constraint. Perform the same task for the other SRP
members.

-------------------------------------
User: Lorant Sjouwerman | Chair
-------------------------------------

Reviews
-----------

1. Enter Consensus Comments. Enter draft comments manually for
Primaries.

2. Edit SRP Score. This seems straight forward. I am not able to
edit scores for which Lorant is conflicted which is the correct
behavior.

3. Re-vote. The interface is a bit clunky. Not much real estate, and
when I click save I am put back in Summary view. In practice I
would need to edit several of these at a time. Too, this does not
seem to work. When edit the Individual score for a reviewer and
click save the SRP Score does not update. When I go back to see
the Individual Score it is now blank. If I do not click Save then
the Individual Score persists but the SRP score is still not
updated.

4. Complete/Finalize. I have to complete all reviews before I can see
the Finalize button. I had to login as a TTA Member to Complete
proposals for which the chair (Lorant) was conflicted.

Conclusion: Close to being ready for internal testing. Would be good
to investigate the missing proposals. Or at least to try to reproduce
it. We should probably fix the re-vote functionality unless I am
doing something wrong.

	Dev	QA
STT-1379 - Getting issue details... STATUS	DONE	This is working properly. If a TTA member goes to finalize, any Blank + P/S/T will have a score of 0, but those will be closed.
STT-1351 - Getting issue details... STATUS	DONE	BE only
STT-1202 - Getting issue details... STATUS	DONE	BE only
STT-1381 - Getting issue details... STATUS	DONE
STT-1376 - Getting issue details... STATUS	DONE	8.23.22
STT-1395 - Getting issue details... STATUS	DONE	An external reviewer can now see the newly assigned ISR, regardless if consensus has started on their panel or not.
STT-1398 - Getting issue details... STATUS	DONE	8.18.23
STT-1384 - Getting issue details... STATUS	TO TEST	now i can pseudo-finalize with only 1 ISR completed/saved - 8.22.23
STT-1387 - Getting issue details... STATUS	DONE
STT-1394 - Getting issue details... STATUS	DONE
STT-1388 - Getting issue details... STATUS	DONE	~~I can still edit (though not save) Closed ISRs. I can no longer edit any ISR as an SRP member except those that are Closed~~
STT-1338 - Getting issue details... STATUS	DONE	this does work; still unclear of full behavior though
STT-1358 - Getting issue details... STATUS	TO TEST
STT-1349 - Getting issue details... STATUS	BUG	Non-TTA can still see all Solicitations, even if they do not have a role on any panel associated with the solicitation. 8.22.23
STT-1358 - Getting issue details... STATUS		moved to sprint 61
STT-1404 - Getting issue details... STATUS	TO TEST
STT-1405 - Getting issue details... STATUS		moved to sprint 61
STT-1406 - Getting issue details... STATUS	DONE
STT-1341 - Getting issue details... STATUS	TO TEST
STT-1378 - Getting issue details... STATUS	DONE	8.18.23
STT-1360 - Getting issue details... STATUS	DONE	8.18.23
STT-1369 - Getting issue details... STATUS	DONE	8.18.23
STT-1324 - Getting issue details... STATUS	DONE	8.18.23
STT-1355 - Getting issue details... STATUS		moved to sprint 61
STT-1397 - Getting issue details... STATUS	BUG	the Mean of the FNISs and SRP scores are still consistent in their display see ticket with new image. 8.22.23
STT-1407 - Getting issue details... STATUS		moved to sprint 61
STT-1205 - Getting issue details... STATUS		moved to sprint 61
STT-1399 - Getting issue details... STATUS		moved to sprint 61

STT-1401 - Getting issue details... STATUS
STT-1199 - Getting issue details... STATUS	DONE
STT-1382 - Getting issue details... STATUS	DONE	Very nice.
STT-1357 - Getting issue details... STATUS	DONE	Works as expected and then some: for a chair it does not give the prompt but for the SRP reviewer it does. I think this is a nice functionality.
STT-1201 - Getting issue details... STATUS	DONE
STT-1200 - Getting issue details... STATUS	DONE
STT-1390 - Getting issue details... STATUS	DONE

	DEV	QA
STT-1202 A SRP member should not be able to enter an ISR for proposals for which they are conflicted	IN PROGRESS
STT-1327 notification unique email when consensus reviews are completed	DONE
STT-1328 add external reviewer in the UI	IN PROGRESS
STT-1329 do not redact mean and stdev scores for conflicted reviewer in consensus	DONE	DONE
STT-1330 new UI in ISR phase	IN PROGRESS
STT-1339 Conflict declaration fails when insufficient reviewer notification is triggered	IN PROGRESS
STT-1343 Chair should not be able to edit Review Type if ISR is finalized	DONE	DONE
STT-1348 removed conflicted badge for tta members in consensus ui	DONE	DONE
STT-1349 reviewer only sees solicitation in reviews tab for which they have an active role	TO DO
STT-1350 Conflicted reviewer should not see details in print view of proposal	DONE	DONE
STT-1351 conflicted reviewer should not be able to see isr comments or scores in consensus	IN PROGRESS	DONE in FE
STT-1352 non conflicted srp members are not able to see consensus comments	DONE	DONE
STT-1353 calculation of mean finalized normalized scores and stdev are incorrect in consensus	DONE	DONE well, the mean and std are correct for the FNIS scores listed in consensus new bug ticket for FNIS calculation when Closed ISR in mix STT-1341 updating FNIS requires refresh.
STT-1354 SRP score should be able to be 0.00 precision on save and complete/finalize consensus reviews (and display consistently)	DONE	DONE
STT-1356 TTA need to edit PPR proposal reviews after finalized	DONE	DONE
STT-1373 ISR conflict state should be displayed in a list of ISRs for an SRP reviewer to they understand why they cannot interact with it	DONE	DONE on Individual Reviews, the word "Conflict" is clearly displayed on the list of proposals instead of the review state and the review type. This clearly tells the reviewer they are not responsible for it. I can only see the abstract, title, and proposal ID for the proposals for which the reviewer is conflicted.
STT-1376 error in consensus when review type is not assigned to a proposal or missing primary/secondary	DONE	BUG i still cannot do this if it is missing primary it fails except for tta member. it it has no one on it, the chair cannot interact with it.
STT-1379 Missing prompt when user finalizes their ISRs to confirm their choices	DONE	BUG prompt sometimes? incorrectly identifies Blank, None proposals with scores = 0 as top ranked.
STT-1381 cannot delete reviewer from a panel during configuration	IN PROGRESS
STT-1340 Review type None and review state of blank prevents finalization of ISRs	DONE	DONE
STT-1345 badges showing number of proposals and reviewers on panel correctly displays for chairs, tta, and srp members	DONE	DONE

STT-1330
- new Chair UI in ISR phase
  - Conflicted status is displayed, regardless of auto or user declared
  - message saying "Awaiting Conflict Declaration" is shown when not certified. Prevents chair from assigning a type.
  - updates once reviewer has done their certifications. Shows certified badge.
  - reorders list of proposals base do who logged in
  - still shows review state of proposals in addition to the assignment of the review type. it is less cluttered though.
    - I assume that if I were to change the RT of a proposal i would need to hit finalization. I need instructions written on how I will test this.
  - I can save just one person's RT even if the others are conflicted,
  - Once i finalize, in the summary view I can no longer change the review type of the reviewer as the chair.
  - It just shows a static badge of review type and the review state of the proposal
    - need to test closed, blank, None, completed.
  - Review Types None with any RS of blank, saved,completed can continue to be edited by srp reviewer and the chair can assign a RT still. Hopefully these are both closed when consensus starts.
  - As a TTA member, I can go in still and change the review types even though they are finalized.
  - When Consensus started, the UI changed for TTA and Chairs
    - Now all RT modifications are gone. Just badges showing the Review Type, the Review State are available, conflict state.
The number of reviewer badge is now correctly populating for non chairs and non tta members.
the chair name is not.
Finalize reviews is not showing up the moment I have access to the ISRs. All ISRs are Blank, None so it could and used to. It shows up once there is the right states now I think.

Consensus

As a conflicted reviewer, I cannot see the proposal anymore except for the abstract, title and ID. that's good.
As the chair, I cannot edit a conflicted consensus review
I can see all the reviewers (As chair) and their roles on an proposal.
- as a reviewer i only see my role.
- the filtering on RT is gone.
As a tertiary, i cannot interact with a proposal but i can see everything.

STT-1175 New OSR UI

Dana (Firefox, Fedora)

Goal: Check entire path from Solicitation creation to Finalize
Consensus Reviews. I am not going to worry about trying to break the
system but just see if I get go from A to B and the overall look/feel
and functionality. Let's setup just one SRP with a typical proposal
load (25) and panel membership (5 reviewers + 1 chair).

TTA member sets things up
========================

-----------------------
User: Dana Balser_TTA |
-----------------------

Solicitations
-------------

1. Click "Generate Test Solicitation" button. Enter "dsbTest58",
"dsb58", "PPR", and "25". Click "Submit". Seemed to work. I did
not have to refresh browser to see the Solicitation show up in the
list.

2. Click "Close". Seemed to work. Close icon disappeared.

Proposals
---------

I can see the 25 proposal in the dsbTest58 Solicitation. I also
received 25 email notifications. Good.

Reviews
-------

1. Select "dsbTest58". I can now see the list of 25 proposals with
subtabs: Summary, Configuration, Reviews, and Allocation Disposition.

2. Configuration->Panels. Enter "GWT Panel" and click "New Science
Review Panel". Click GWT Panel. I can see the "delete" button to
delete the panel. Select "GWT" science category. Search/Select
reviewers: Lorant (chair), Allie, Mark, Amy, Ryan, and Emmanuel.
Lorant has one conflict and Allie has no conflicts.

3. Configuration->Vetting. Click "auto assign panels". The "GWT
Panel" was selected for all propsoals as expected. Click "Vet
All". All the vetted checkboxes are checked. Click "Save
Proposals" button. Nothing seems to happen. When I do a
browswer refresh the "Save Proposals" button is gone and I can see
a "Finalize" button. Click "Finalize" button. I am sent back to
the the Summary page. If I go back to Configuration I can see the
"Finalized" icon (button is gone). Also, I received 6 email
notifications, presumably sent to each reviewer, indicating that
the SRP is open for business.

Declare conflicts of interest
=========================

-------------------------
User: Lorant Sjouwerman | Chair
-------------------------

Review
------

1. Needed to click the following to get to the conflict declaration:
Reviews, dsbTest58, GWT panel, Individual Reviews, Lorant
Sjouwerman. Lots of clicks but the flow was logical. I then added
one conflict for proposal 20.

2. Click "Certify Conflict Declarations" button. I now see the
certify badge but I do not see any conflicts in the list which is
not correct. I see two column of badges to the right. The first
column are all blank, presumably indicating the review is blank.
The second column are all None, presumably indicating that I am
assigned the review type of None (the default). But from this view
I do not know which proposal I am conflicted on.

3. On the Summary page, however, I can see that Lorant is certified
and see the conflicts.

-------------------
User: Allie Costa |
-------------------

Review
------

1. Declared no conflicts. Since Allie is not a chair she only sees
the Individual Reviews and Consensus Reviews tabs which makes
sense. I also only see Allie Costa and not the list of other
reviewers which is reasonable.

---------------------
User: Mark Claussen |
---------------------

Review
------

1. Declared no conflicts.

------------------------
User: Amy Mioduszewski |
------------------------

Review
------

1. Declared a conflict for dsb58-10.

------------------
User: Ryan Lynch |
------------------

Review
------

1. Declared no conflicts.

------------------------
User: Emmanuel Momjian |
------------------------

Review
------

1. Declared no conflicts.

GWT Chair assigns reviewers to proposals
====================================

-------------------------
User: Lorant Sjouwerman | Chair
-------------------------

Review
------

As already discussed the summary page is too busy but the details look
correct before the chair begins assigning reviews.

1. Summary tab: assign reviewers to proposals: P, S, or T. Then click
"Save". I get the following error:

"A backend error occurred at 'api/individual_science_reviews' : Expected individual science review conflict declaration to be available. Found a conflict state of Conflicted for ISR id 172"

2. Upon inspection I had assigned P,S, or T to a reviewer on a
proposal for which they were conflicted. After changing these
assignments I clicked the "Save" button and get the following
error:

"A backend error occurred at 'api/individual_science_reviews' : Expected individual science review conflict declaration to be available. Found a conflict state of AutomaticallyConflicted for ISR id 246"

3. Found one more assignment with a conflict. Fixed this and then
click the "Save" button. Works this time.

Perform Individual Reviews
========================

-------------------------
User: Lorant Sjouwerman | Chair
-------------------------

Review
------

Inspection:

- The individual review tab for the chair list the reviewers with some
info. The tab is highlighted which is good. When I click on
"Lorant Sjouwerman" I go to his reviews which is fine but the
"Individual Reviews" tab is no longer highlighted.

- It would be useful to filter/order the proposal list.

1. Individual review. Use the file download/upload to enter reviews.
The download seemed to work fine. I cannot recall the exact
requirements of the file format. No field to complete or an
indication of conflicts. When I try to upload I get the following
error (appears I have to include a review for all proposals which I
do not think is the behavior we want):

A backend error occurred at 'api/science_reviewer/43/import_isrs' : CSV errors found when attempting to import Individual Science Reviews: Row 2: No comment for the SRP found, Row 15: No comment for the SRP found, Row 10: No comment for the SRP found, Row 9: invalid individual score found - score must be between 0.1 and 9.9 in increments of a tenth, Row 15: invalid individual score found - score must be between 0.1 and 9.9 in increments of a tenth, Row 5: No comment for the SRP found, Row 18: No comment for the SRP found, Row 10: invalid individual score found - score must be between 0.1 and 9.9 in increments of a tenth, Row 3: No comment for the SRP found, Row 9: No comment for the SRP found, Row 18: invalid individual score found - score must be between 0.1 and 9.9 in increments of a tenth, Row 19: No comment for the SRP found, Row 8: invalid individual score found - score must be between 0.1 and 9.9 in increments of a tenth, Row 19: invalid individual score found - score must be between 0.1 and 9.9 in increments of a tenth, Row 8: No comment for the SRP found, Row 3: invalid individual score found - score must be between 0.1 and 9.9 in increments of a tenth, Row 2: invalid individual score found - score must be between 0.1 and 9.9 in increments of a tenth, Row 5: invalid individual score found - score must be between 0.1 and 9.9 in increments of a tenth

2. Enter reviews manually. I can enter reviews for proposals for
which I have not been assigned or are conflicted which is not
correct. Once I complete all of the assigned reviews there is no
initial indication that I am done. If I go to the Individual
Review tab with list of reviewers, however, I can see the finalized
badge, but this needs to be more prominent. (I do get an email
notification). Would be useful to allow one to order the proposals
(e.g., proposal ID, review type, status). Need to see which
proposals I am conflicted on.

-------------------
User: Allie Costa |
-------------------

Review
------

1. Enter reviews manually. I do not receive an email
notification. If I login as the GWT chair, however, I can see
that Allie's reviews have been finalized on the Individual Review
summary view.

---------------------
User: Mark Claussen |
---------------------

Review
------

1. Enter reviews manually.I do not receive an email
notification.

------------------------
User: Amy Mioduszewski |
------------------------

Review
------

1. Enter reviews manually. I do not receive an email
notification.

------------------
User: Ryan Lynch |
------------------

Review
------

1. Enter reviews manually. I do not receive an email
notification.

------------------------
User: Emmanuel Momjian |
------------------------

Review
------

1. Enter reviews manually. I do not receive an email
notification.

Inspection:

- The only indication in the UI that all reviews have been finalized
is for the chair to go to the Individual Reviews tab in the summary
view. Maybe when we rework the Summary tab this will be more clear.
But there was also no email notification.

- After discussion with Allie and Reid the issue is that finalize is
checking against a state of Blank without cross-checking the Review
Type. So all reviews must be completed. This may explain the
behavior of the file import.

-------------------------
User: Lorant Sjouwerman | Chair
-------------------------

Review
------

1. Click "Dowload Reviews" button. Edit file offline. Add a review
(text + score) for every proposal.

2. Click "Upload Reviews" button, select file, and then click
"Submit". This appears to work. Interesting, for the proposals
which had a state of "Completed" from the previous work are now
"Saved" after the upload. The state field does not exist in the
file format. I now, however, see the "Finalize Reviews" button.

3. Click "Finalize Reviews" button. I see the state change to
"Finalize" only for those proposals for which the Review Type is
primary, secondary, or tertiary.

-------------------
User: Allie Costa |
-------------------

Review
------

1. Use file Download/Upload to perform the remaining reviews. I was
able to include commas in the review text, which were enclosed in
quotes, and this worked fine.

2. Click "Finalize Reviews" Button. It would be good to have some
indication on this page that all reviews are finalized. I did not
receive an email notification, but maybe this is the requirement.

---------------------
User: Mark Claussen |
---------------------

Review
------

1. Use file Download/Upload to perform the remaining reviews.

2. Click "Finalize Reviews" Button.

------------------------
User: Amy Mioduszewski |
------------------------

Review
------

1. Use file Download/Upload to perform the remaining reviews.

2. Click "Finalize Reviews" Button.

------------------
User: Ryan Lynch |
------------------

Review
------

1. Use file Download/Upload to perform the remaining reviews.

2. Click "Finalize Reviews" Button.

------------------------
User: Emmanuel Momjian |
------------------------

Review
------

1. Use file Download/Upload to perform the remaining reviews.

2. Click "Finalize Reviews" Button.

Inspection:

- After all reviewers reviews were finalized I did receive an email
notification to this effect. Good. But not easy to tell in the UI
that all reviews are finalized.

- But when I login as the SRP chair and go to the Consensus tab I do not
see any proposals listed, even after a browser refresh. After some
discussion with Allie I need to login as a TTA member and click the
"Start Consensus Review" button. Once I do this I can see proposals on
the Consensus tab.

Perform Consensus Reviews
=========================

-------------------------
User: Lorant Sjouwerman | Chair
-------------------------

Review
------

Inspection:

- Consensus Review tab: Not all reviewers are listed under the
title. For example, Allie is not included for dsb58-006. Also, the
mean/std is zero for proposals for which Lorant is conflicted but
these numbers should be listed. The SRP score does not match the
mean. Hard to distinguish between different states (e.g., blank vs
saved). Different colors would be helpful.

- Consensus Review tab-->Proposal: For conflicted proposals the author
is listed. Should be anonymous. Should not be able to see
scientific justification or any Allocation Requests.

- Consensus Review tab-->Reviews: For conflicted proposals the author
should not be able to see the Individual reviews (score or text).
Not sure why the UI real estate is so large for the score. The
normalized scores look okay. Here is what I get for Allie where the
UI has a normalized score of 3.4 for dsb58-014.

Proposal ID Raw Norm
dsb58-001 4.0000 3.4495
dsb58-006 7.0000 6.4397
dsb58-003 2.0000 1.4560
dsb58-022 7.0000 6.4397
dsb58-004 8.0000 7.4365
dsb58-021 5.0000 4.4463
dsb58-005 4.0000 3.4495
dsb58-020 3.0000 2.4528
dsb58-009 6.0000 5.4430
dsb58-010 4.0000 3.4495
dsb58-018 6.0000 5.4430
dsb58-011 8.0000 7.4365
dsb58-012 7.0000 6.4397
dsb58-016 7.0000 6.4397
dsb58-013 9.0000 8.4332
dsb58-014 4.0000 3.4495
dsb58-002 6.0000 5.4430
dsb58-019 3.0000 2.4528

1. Enter Consensus comments for all proposals for which Lorant is
Primary. In order to save comments I have to include Internal
Comments which is not the desired behavior.

-------------------
User: Allie Costa |
-------------------

Review
------

1. Enter Consensus comments for all Primary proposals.

---------------------
User: Mark Claussen |
---------------------

Review
------

1. Enter Consensus comments for all Primary proposals.

------------------------
User: Amy Mioduszewski |
------------------------

Review
------

1. Enter Consensus comments for all Primary proposals.

------------------
User: Ryan Lynch |
------------------

Review
------

1. Enter Consensus comments for all Primary proposals.

------------------------
User: Emmanuel Momjian |
------------------------

Review
------

1. Enter Consensus comments for all Primary proposals.

2. I was unable to see Consensus comments for proposals for
which Emmanuel was conflicted.

3. I was able to see but not enter Consensus comments for proposals for
which Emmanuel was Tertiary or None

4. I was able to edit Consensus comments for which Emmanuel was
Secondary.

5. I was never able to edit the SRP score.

-------------------------
User: Lorant Sjouwerman | Chair
-------------------------

Review
------

1. Modify the SRP Score and click "Save". Seems to work.

2. Change the Normalized Individual score for a reviewer. Allows the
chair to change the value and displays the original value (called
"As Finalized"). The Mean value or SPR Score is not updated.

-----------------------
User: Dana Balser_TTA |
-----------------------

Review
------

Inspection:

- Interestingly Dana, that's me, is listed as conflicted on all
proposals. This is not correct.

- I am able to change the SPR Score and do the re-voting (change the
Normalized Score) with the same behavior as the chair above.

-------------------------
User: Lorant Sjouwerman | Chair
-------------------------

Review
------

1. Mark as Complete all reviews for proposals for which Lorant is not
conflicted. Indeed, I cannot complete proposals for which Lorant
is conflicted. (As a regular SRP member I can only Complete
proposals for which I am either Primary or Secondary as expected.)

2. After the last review is Completed I see the "Finalize Consensus
Reviews" button. When I click the button the state goes to
"Finalized". I can also now see the Normalized Linear Rank Score.
This is not really a score so I would call it "Normalized Linear
Rank". The linear ranks are correct based on the SPR Scores. I received an email notification.

General Issues:

1. Need to redo the Summary page.

2. Need to be able to export Summary page info. This is one of the most
useful tools for the chair when running the SRP meeting.

3. Need more functionality on the Individual Review page.
Filter/Order features and an indication of conflicted proposals.

4. Need to sort out the behavior of the File Download/Upload process.

Notes for Dana (no testing spoilers) -

FE and BE has not been hooked up for Consensus Reviews. This work is being done now, so you can start consensus but not save comments or change scores yet.

Allie

Refresh Bugs:
- When the Summary tab first loads in a Solicitation's Proposal Process, it needs a refresh the very first time it opens. After that, it is fine and I see a list of proposals. Might consider what information could go here
  - Number of proposals? Large vs regular?
  - Facility, Joint, External?
  - Metrics landing page?
- When I first went into Reviews→ Reviews→GWT, it shows all of the proposals in the solicitation instead of just the ones assigned to GWT. It did not show the Review Type assignment matrix for them. I had to refresh and then only the proposals assigned to the panel are available for TTA member or Reviewer.
Review Configuration – not testing because of overhaul in UI in sprint 58
Vetting
- Changing panel assignment resets (collapses) all of the rows. If there are unsaved SC changes or "Vetted" changes, they revert.
- Clicking Auto Assign Panels again gives an error: Object Object
- Clicking Finalize sends me back to home page
  - Got 6 emails about panels being open (3 per panel)
As a TTA member I should not be able to modify the configuration, vetting, or panel assignment of proposals after the configuration is Finalized.
As a TTA member I need to be able to add an external reviewer.
Reviews (Chair)
- Summary
  - See proposal matrix (skip for now)
  - Return after all conflicts are certified
  - Can assign to conflicted, story not done yet.
- Individual Reviews
  - First conflicts – as expected
  - Redirects to ISRs; certified icon changes.
  - RS and RT shows Blank/None until they are updated, as expected
  - Can mark Completed on ISRs without comments (I think we said that was okay?)
  - Error thrown when trying to complete/save a ISR score of 0
  - Used TTA powers to do conflict declaration for 1 reviewer – got an email about it
  - got MANY emails about insufficient reviewers
  - On list of reviewers, the Finalized icon is showing as Finalized before I have actually finalized. Then it isn't showing Finalized when the ISRs are finalized.
    - When I look at the list of reviewers in ISRs, I want the Finalized icon to display the Review State of Finalized accurately so it is clear if reviewers have finalized.
  - As a SRP member I do no want to be able to enter scores or comments into ISRs for which I am conflicted nor should I see the proposal.
  - When I use the Finalize Review as a TTA member,
    - There isn't visual feedback that the finalization has worked without a refresh.
      - When ISRs are Finalized, I want feedback in the UI so I know the action was successful.
      - As a reviewer, I only want the Finalize button to be available when the condition is correct so I don't use it inappropriately.
        Finalized as the Chair and it is still letting me hit the finalize button. – It's because there is a RT of None here. There isn't the condition
        I can finalize with no RT assigned when all of the nonconflicted proposals have RS of saved/completed. This isn't the requirement.
        Bigger issue: I made 2 reviews with the same score and the other ones I left blank. Then I went in as a TTA member and finalized them, which set the blank ones with RT nq None to Closed. Then finalization was allowed initially, I presume, because of the 4 with RT nq None, they did not all have the same score but since the two non unique ones were Closed, they not included in future finalization calculations. This is expect but now the onle two reviews have the same score. I suspect it will NaN out in the next phase. What are exact conditions for finalizing even as a TTA member. This is protected against as a normal user already, but a TTA member could mess it up.
        Should close out first and then calculate the finalized normalized individual score.
    - Any RT not equal to None and RS equal to Blank get state changed to closed.
    - I can still use the Finalize button as a reviewer, which gives an error.
    - I can use the Finalize button as a TTA member (as it should be) but it also gives an error.
    - Finalizing sends all ISRs with RT of None to RS of Closed. Changes Completed or Saved to Finalized if RT nq None.
  - As a TTA member, I need to be able to edit Finalized ISRs until the start of consensus.
  - As a Chair, I do not want to be able to edit the Review Type of an ISR if it Finalized.
    - Once the ISRs are finalized, I can still access the buttons but then the save buttons hangs. Presumably, it is because I cannot edit the RT. This needs more UX consideration
    - (STT-1200 for tta powers)
  - As a reviewer, I do not want to be able to edit ISRs with Review States of Closed.
    - Note, a TTA member can edit Closed and saving/completing takes it out of closed. There isn't a way to make it closed again though. I can change the RT so it won't go into consensus I suppose.
  - As a TTA member, I would like to be able to Close a single ISR until the start of Consensus.
  - As a reviewer, I want the name of the chair to be displayed for my panel.
  - As a reviewer, I want the icon that tracks how many reviewers to either display correctly or be taken out.
  - As a reviewer, I would like filtering and sorting on my ISRs.
- Consensus Review (Only partial as it isn't hooked up)
  - As a Chair, I should not be able to see the button that starts Consensus.
    - The triggering criteria for consensus phase is correct, in that it only appears once all of the members have finalized their ISRs.
    - I also got an email saying that all ISRs for GWT panel are finalized.
    - Only the TTA member using the button actually starts consensus I think though.
    - Even after consensus was started, the button is still available.
  - As a TTA member, I do not want to see the start consensus button once consensus has started for a panel so I will not be confused by the progress of the panel.
  - As a TTA member I would like the Reviews/Reviews page to show what subphase the panel is at (ISR or Consensus).
  - As a reviewer, I would like the Reviews/Consensus tab to display an informational message until Consensus is actually started.
  - As a TTA member, I would like to be able to edit the message displayed in the consensus tab at will.
  - What is the purpose of the State filter in Reviews/Consensus? It doesn't seem to work and is populated by the Proposal State model instead of the Consensus Review State model?
    - As a reviewer, I want the state filter to be the consensus review state model.
  - When a TTA member I can see the names of who is primary/ secondary per proposal, which is good. As a reviewer though, it only shows "my" name.
    - As a reviewer, I want to see the name of the Primary and Secondary so I know who is leading the discussion.
    - As a reviewer, I would like the filtering on primary to display more than just my RT primary proposals.
  - Generally, the Chair view is quite right as compared to the TTA member and reviewer. It should be close to the TTA member than the reviewer at this point I think.
    - As a chair I need more tta member like visibility into consensus reviews so I can finalize the consensus reviews.
Misc
- When I came in as Toney Minter, I can suddenly see in the list that he is not assigned to. Clicking on them, however, gives an error.
- Got an error certifying conflicts once.
- Mixed Conflicted states on TM now.

Demoable Stories:
- STT-861: Hidden & Submit
  - Can submit hidden proposals when a solicitation deadline has passed.
- STT-1188: Restrictions on withdrawing a proposal.
  - The default behavior is a proposal with In Review state cannot be withdrawn (e.g., OSR proposals), but PPRs are treated slightly differently. Once you Finalize the Review Configuration in a PPR proposal process, you cannot withdraw. Note, a new Proposal Process, should we make one, will default to only allowing withdrawing when Proposal State = Submitted. If we want this behavior modified for a new Proposal Process, a case statement needs to be added to the software.
- STT-1319: OSR works again
- STT-1318: ISR subphase works again
  - Note, the "chair" icon that shows the chair name is being hidden now because of anonymous data. Will address later as a cosmetic issue.
Carryover
- STT-1234:
  - Still need more time to wrap up the refactor.
- STT-918:
  - Notifications are not working on test. Will fix for next sprint

Notes:

we should report any "refresh" bugs, as it should be fixed now.
In sprint 58, the review configuration UI will be overhauled so that portion of the review process likely does not need close scrutiny until that happens.
Part of the Review config overhaul is changing the behavior of the panel creation drop down tool, which currently resets the collapsed panel with each edit.

Allie on Dev

Solicitation generation worked after a refresh to populate the proposal process field.

Did not creat a proposal but went to reviews tab.

Reviews *prior to close

*Summary

No changes - just a list of proposals in the solicitation

*Configuration

**Panels

Able to add panels, assign SC, assign and delete reviewers, assign 1 chair.

I need to refresh all the time though to see my changes. Also, every assignment of a reviewer to a panel collapses the panel view which is very obnoxious.

We still want a reviewer focused interface here.

The save changes button only appears when I make changes not related to the reviewers. Not sure if the changes to reviewers persists yet without the save. It looks like it does though. It is strange that there is different behavior between the fields (e.g., SC assignemnt) and the reviewer assignment.

Vetting doesn't look like it is persisting.

I can still go to the reviews tab though.

As a Chair, I click Reviews → solicitation → mypanel and see three tabs: Summary, individual reviews, consensus reviews. I cannot assign RT or declare conflicts.

Proposals are not populating for the chair though.

As a reviewer, I click Reviews → Solictation → mypanel and see two tabs: Indiviual Reviews, Consensus Reviews. By default, a list of proposals is shown but I cannot interact with it. When I click IR or CR, it shows nothing. Hitting a refresh shows some of the conflict declaration page. I cannot actually declare conflicts though.

I think this is all I can test now.

Allie on test

There was a stale log in apparently, that was only "fixed" by logging in as a normal user and then returning as a TTA member. Refreshing and logging out did not help.

Solicitation

**Generation**

Successfully make a Solicitation with PPR and 10 proposals.

Received 10 email notifications about their submission.

**Prior to Close**

Reviews

Can access the Reviews Tab. It lists by solicitation. There are error messages that comes up when I hit the reviews tab

"a backend error occurred at api/science_reviewers': <html> <head> <title> 404 not found </title> </head> </body> <h1> 404 not found </h1> the resource could not be found. <br/> <br/> \science_reviewers </body></html>"

"a backened error occured at api/ppr_proposal_reviews: 404 not found .. the resource could not be found... "

"a backend error occured at api/osr_proposal_reviews: 404 not found ... the resource could not be found..."

In the ahc-1 solicitation proposal process, there are four tabs: Summary, Configuration, Reviews, and Allocation Disposition. I should be able to configure the review panels even prior to the close of the solicitation.

*Summary

Lists the proposals by proposal ID and Title. No interaction beyond that though at this point.

*Configuration

Generates an error: "A backend error occured at api/individual_science_reviews: <html> ... 404 Note found .... the resource could not be found.

Generates an error: object Object

View shows two subtabs: panels and vetting. Neither are really functional beyond this. The vetting is fine as the solicitation is open.

**Close the Solicitation**

Panels and Vetting still not functional.

Proposal Creation

Created proposal for Sem25A.

As an author, I would like to only specify the Solictation when I first create a proposal so I only have to specify solicitation specific information.

As an author, I would like to have information about the available Solicitations when I create a proposal so I know which one to select.

As an author, I would like a visual clue as to why I can't access the observatory copy for a proposal in the draft state.

I have to refresh a lot but generally it working as I expect. I can resubmit.

As an author, I would like to be able to access the observatory copy after I submit without needings to first navigate out of the proposal creation screen.

As an author, I would like the view of the observatory and author copy to populate correctly when not in edit mode.

Dana (Firefox, Fedora)

Testing for issues after the refactoring. In scope is Solicitations and Proposals. Review is not ready. Know issues are here: Sprint 56 - Known Issues. Per request, if an issue can be fixed by a refresh then there is no need to note this here.

----------------------------------
User: Dana Balser_TTA |
---------------------------------

Solicitations
-----------------

First create a new Solicitation.

1. Click "Generate Test Solicitation" button. Enter "dsbTest56",
"dsb56", "PPR", and "10". I can view the Solicitation. I received
the 10 email notifications. I can view the 10 proposals.

-------------------------------------
User: Lorant Sjouwerman |
--------------------------------------

Proposals
--------------

Begin by creating a new proposal in Solicitation "dsbTest56".

1. Click "Add". Enter Title, Solicitation, and science category.

Basic Information
-------------------------

2. Add abstract and then click "Save". So far so good.

Scientific Justification
------------------------------

3. Upload a PDF file. No problems.
4. Click view. I can see the PDF.

Allocation Requests
----------------------------

---> Capability Requests

5. Click "Add" under Allocation Requests.
6. Edit Allocation Request name; click "Save."
7. Click "Add" under Capability Requests.
8. Field Sources. Import from a file. Seems to work okay.
9. Spectral Specifications. Import from a file. Seems to work okay.
10. Performance Parameters. Enter values for Angular Resolution, LAS, and rms.
11. Calibration Parameters. Select "yes" for Polarization.
12. Click "Save".

---> Observation Specifications

I can see the Observation Specification. Overall looks okay. I can
see the polarization calibration, together with the standard flux,
gain, and bandpass calibrations.

Let's go back to the Capability request and adjust the rms noise since
to increase the integration time. Currently rms = 0.001 Jy/beam and
integration time = 1 sec. Change rms to 0.00025 Jy/beam.
When I do this I still get the same integration time.

Maybe some limit, so let's reduce the rms to 0.000001 Jy/beam.
Hmmm...I get the same result.

One more time. Change the rms to 0.00000000001 Jy/beam. Okay, now I
get a big number. Probably just needed to refresh which I did this
time.

Try to get back to rms = 0.000001 Jy/beam which should yield a more
sensible result. Indeed, I get about 4 hours on source. Okay.

13. Click "submit". Seems to work. I get an email notification.

Dana (Firefox, Fedora)

Testing mostly focused on looking at the new structure where the
Reviews tab now contains the Proposal Process.

---------------------------------
User: Dana Balser_TTA |
----------------------------------

Solicitations
-----------------

1. Generate test Solicitation with name="dsbTest51", proposal prefix =
"dsb51", and count = 10. Received notifications.
2. Click "Close" button.

Reviews (effectively the proposal process)
-----------

Initially see a list of the Solicitations (name and review process).

1. Click "dsbTest51". See "Summary", "Configuration", "Reviews", and
"Allocation Disposition" tabs. Nothing is listed under Summary.

- Sub-tab: Configuration-->Panels

2. Configure panels: "GWT Panel" with one chair and five SRP memebers.
Click "Save" button.

- Sub-tab: Configuration-->Vetting

3. Vet proposals: click "auto assign panels" button (all GWT).
4. Click "vet all" button.
5. Click "Save Proposals" button. So far so good.
6. Click "Finalize" button. Moved to Home tab.

If I now go back to the Reviews tab for this Solicitation I do see a
list of the proposals in the Summary sub-tab.

- Sub-tab: Reviews-->Summary

I can now see the matrix: proposals by reviewers.

- Sub-tab: Reviews-->Individual Reviews

I see a list of the reviewers with some summary info. If I click on a
reviewer I see either the conflict declaration page or the ISRs.

Login as each reviewer and declare conflicts. Notice that only the
chair has the Reviews-->Summary sub-tab and can also see the list of
other reviewers in the Reviews-->Individual Reviews sub-tab. If I
login as chair I can see in the Reviews-->Summary sub-tab that all SRP
members are certified.

Next login as each reviewer and perform ISR, mark as completed, and
then finalize. One reviewer had only on ISR assigned so when I finalized
I got the following error:

"A backend error occurred at
'api/individual_science_reviews/finalize/20' : Expected normalizable
individual scores not to all be the same, found instead [3.0] for
ScienceReviewer with User ID 6 "

So I login as the chair and assign another review and then the SRP
member in question is able to finalize their reviews. Once this was done
I got the email notification that all ISRs are finalized.

I can now see the "Start Consensus Review" button. Still on the
Reviews-->Summary sub-tab but this is probably out of scope here.
Missing, of course, is the TTA Member page that allow the process to
move from one review stage to the next, but overall the structure here
is reasonable.

When I click the "Start Consensus Review" button I now see the
Reviews-->Consensus Review sub-tab and can see a summary of each
proposal.

Dana (Firefox, Fedora)

---------------------------------
User: Dana Balser_TTA |
---------------------------------

Setup stuff.

Solicitations
-----------------

1. Generate test Solicitation with name="dsbTest50", proposal prefix =
"dsb50", and count = 10. Received 10 email notifications. Good.
2. Configure panels: "GWT Panel" and "SFM Panel". Each panel has one
chair and two SPR members. Click "Save" button.
3. Vet proposals: 7 as GWT and 3 as SFM (changed). Click "Save
Proposals" button.
4. Click "Finalize" button.

Forgot to close Solicitation. F#$K! Circle wagons.

5. Generate test Solicitation with name="dsbTest50_2", proposal prefix =
"dsb50_2", and count = 10. Received 10 email notifications. Good.
6. Click "Close" button.
7. Configure panels: "GWT Panel" and "SFM Panel". Each panel has one
chair and two SPR members. Click "Save" button.
8. Vet proposals: 7 as GWT and 3 as SFM (changed). Click "Save
Proposals" button.
9. Click "Finalize" button. Okay, worked this time. Received an
email notification for each reviewer.

Check email notification when less than 3 reviewers. Test by adding
conflicts to some proposals. Since there are some automatic conflicts
(proposal 1 and 7) and only three reviewers this would happen
regardless, but the notification should list them.

--------------------------------------
User: Lorant Sjouwerman |
-------------------------------------

Reviews
------------

1. Select "Available" for proposals 1-6; auto conflict on 7.

2. Click "Certify Conflict Declarations" button. Got one email
notification that insufficient reviewers on dsb50_2-001 and
dsb50_2-007. These were both due to automatic conflicts.

--------------------------------------
User: Emmanuel Momjian |
--------------------------------------

Reviews
-------

1. Select "Available" conflict on proposal 2; available on rest.
2. Click "Certify Conflict Declarations" button. Got one email
notification with insufficient reviewers on dsb50_2-001,
dsb50_2-002, and dsb50_2-007. Seems correct.

------------------------------------
User: Amy Mioduszewski | Chair
------------------------------------

Reviews
-----------

1. Select "Available" for proposals 2-7; auto conflict on 7.
2. Click "Certify Conflict Declarations" button. Got one email
notification with insufficient reviewers on dsb50_2-001,
dsb50_2-002, and dsb50_2-007. Seems correct.

3. Assign reviewers to proposals.
4. Click Save.

--------------------------------------
User: Lorant Sjouwerman |
-------------------------------------

Reviews
-------

1. Enter ISRs for all proposals. I was able to do this for the
conflicted proposal, which should not be allowed, but this has None
so it should not count. Mark these as complete.
2. Click "Finalize Reviews".

--------------------------------------
User: Emmanuel Momjian |
--------------------------------------

Reviews
------------

1. Enter ISRs for all proposals. Mark these as complete.
2. Click "Finalize Reviews".

-------------------------------------
User: Amy Mioduszewski | Chair
-------------------------------------

Reviews
-----------

1. Inspect reviews summary. The state is finalized for all proposals,
even the one with a self-declared conflict. For the automatically
conflicted proposals the state is completed.
2. Enter ISRs for all proposals. I was able to do this for the
conflicted proposal, which should not be allowed, but this has None
so it should not count. Mark these as complete.

3. Click "Finalize Reviews". I do receive the email notification that
all ISRs have been finalized.

----------------------------------
User: Dana Balser_TTA |
---------------------------------

Reviews
------------

Start consensus review.

1. Click "Start Consensus Review" button. I get the following message:

"A backend error occurred at 'api/science_review_panels/5/launch_consensus' : [object Object]"

Allie (Firefox, Redhat)

General Impressions

I think that the buttons like Finalize, Save should always be there but just grayed out if I can't use them. This would make is obvious when I first arrive that my end goal is.
The Conflict Declaration page has some weirdness with it. It will commonly override my inputs and go back to unknown.
Refreshing a page either helps a broken page or breaks a good page.
Making a new Solicitation threw the error or "Science Review Panel's IndividualScienceReviews could not be assigned: object Object"
To finalize a Review Panel, the set of vetted Science Categories must map to a panel's Science Category. e.g., if there are 2 vetted science categories, there must be those 2 science categories assigned to any number of panels.
If proposals are assigned to a panel and then the panel is modified, all of the proposals will be deleted from the UI.
- we can have a story that this work could go into.

As TTA member,

generate new test solicitation ahcSP49_1 with 23 proposals
23 emails received that the proposals were submitted
the test solicitation generator keeps the fields filled even after I have generated the solicitation.
- - As a TTA member I would like fields to not keep the previously generated information after a successful submission.
  - As a TTA member I would like the ordering of the Solicitations to have a consistent behavior and for filtering to be available.
Look at Reviews tab
- - As a TTA member, I would like the reviews tab to load optimally.
  - As A TTA member, I would like the ordering of the panels in the reviews tab to have a consistent behavior and to have filtering on the panels.
- I have no closed my solicitation, so I do not see it on the Reviews tab, which is good. Even after hitting refresh, it does not show up. Good.
- I logged in as a non tta member and can see the Reviews tab. Initially, it was cached such that I saw it as a TTA member. Once I hit refresh though, then I could only see the panels for which I am a member of.
  - As a product owner, I want to protect against caching issues that would allow users to see content they do not have permission to view.
- The same happens initially when I log back in as a TTA member. I only see the panels of the regular user at first, and then it populates more panels.

In Solicitation, Panel Configuration now.

Panels and Vetting are separate. Vetting is hit by default.
Can change SCs. Can save. Can vet individually. Can vet all.
- - As a TTA member I would like summary information about the proposals on the vetting page e.g., number of proposals per SC.
Changing to the Panels Tab does not prompt a warning about losing unsaved content. It looks like the vetting info is preserved despite this but I bet when I make a save to panels, it will revert.
- It did not. The SC I changed reverted back when I saved on the Panels page.
- Similarly, when I did not save on the Panel page, went back to vetted and saved, the panels I had created but not saved disappeared.
  - As a TTA member, I would like notification when I am navigating away from unsaved changes in Vetting and Panel Configuration.
There are now 13 GWT, 5 SFM, and 5 NGA proposals.
I created a GWT panel, a SFM panel, and a NGA panel.
- I assigned each their appropriate SC and now the counter says "3/3 Science Categories" as I would expect.
- What does the icon next to the word "Panels" mean? It shows "2" currently.
All proposals have been vetted. Panels have science categories, and each expected SC belongs to a panel. Since I have panels, I can auto assign proposals to the panels.
- Auto assign looks at the vetted Science Category.
- I should test if it needs to be saved first before it will do that.
Now that all proposals have been assigned and all of the SC are attached to a panel, the Finalize button appears.
- Error - Could not Finalize because solicitation is open. The system message gives the useful error message. The banner just says it cannot be finalized.
Before I go to close the Solicitation, I looked at the Panels page again. The icon that displays how many proposals are associated with that panel is not populating until a refresh.
Close Solicitation. Get a message asking if I'm sure. good.
Go back to Panel Configuration and the page doesn't load correctly. Have to hit refresh. I can now successfully hit finalize and it boots me out to the home page.
- I receive a number of notifications via email that the panels are open.

Go to Reviews page

In GWT panel. I see everyone that I assigned to it. As a TTA member I have full access.
- When I go to look at a SRP member, I can see ISRs and Conflict. At the bottom of the Conflict page, there's a banner that says Science Review. I don't think that should be there.

GWT Chair: Amy

On the Reviews tab, I can see the summary view of all the panels associated with the Solicitation because I am the chair of one of the panels. When I click on them, it gives an error that says "You do not appear on the requested panel". They don't have access but they shouldn't see the other panels. Wait - now I see only the panels I am on. This is not consistent!
- - As a SRP member I want the Reviews tab to only show the panels I am on.
In the GWT panel, as chair I can see the matrix view. I cannot see the ISRs or conflicts of the other members like the TTA member can (good), I can only do my own.
First it forces conflict declaration
- I can see the title, proposal id, and abstract of the proposals. I can mark my status. There are 2 that I am auto conflicted on.
  - An error message warns if I do not give a reason for my conflict.
- I mark all but 1 as available. The certification button appears and I can use it. It leaves me on this screen. I can click the "back to panel" button, but it isn't obvious.
- I am now certified.
- I go back to the panel and then click my name again so I can see my ISRs. It takes me back to the certify conflicts page and I can change my conflicts but it gives an error that it cannot be certified.
  - Bug - After certification, a srp member can only see their conflict declaration page instead of the ISR page.
- I log in as a TTA member and I can change the conflict and certify without issue.
  - I get notifications stating that the changes were made. There is one for each proposal and not just the ones I changed.
  - As a TTA member I would like to only receive one email per certification change (bulk or only for the one that changed).
The matrix screen shows that I am available for 10 proposals.

Log in as GWT panel SRP member

I go to the Reviews page and now only see the panels I am actually on.
Clicking on the GWT panel takes me to conflict declaration. I do so and certify. It auto puts me to the ISRs, which is good and now I can review them. !! If I refresh the page, I get the conflict certification back !!
- I see the review type and the review state. There is not marker for conflicted and I can see conflicted proposals.
- I can save and mark as completed.
- Errors are appropriately given if my score is not between 0.1 and 9.99.
- I can complete and save if no comments are made.
- The Review States change appropriately.
  - As a SRP member, I would like the view to not collapse on a save.
  - As a SRP member, I would like the "Mark Complete" button to not look like it was already used before I use it.
I certify and start saving ISRs before the Review Type is assigned. The Finalize button does not appear for me? I would think it does.

Log in as GWT Chair

Wanted to go assign types
It says everyone is uncertified but I went through and certified!
- - As a SRP Chair, I do not want to be able to assign a Review Type to a Conflicted reviewer
  - As a SRP Chair, I want to be prevented from assigned multiple Primary Review Types per Proposal. Same for Secondary.
- The conflicted cells should just be red or something.
I really can't test any more because I cannot see ISRs.
When I saved the Review Types, it suddenly populated the review types for all of the other ISRs that I hadn't done?
I tried to save and got a weird error.
- A backend error occured at 'api/individual_science_reivews': all isrs for the science review panel must be submitted together. 52 ISRs submitted, 2 expected.

Dana (Firefox, Fedora)

---------------------------------
User: Dana Balser_TTA |
---------------------------------

Solicitations
-----------------

First create a new Solicitation.

1. Click "Generate Test Solicitation" button. Enter "dsbTest49",
"dsb49", "PPR", and "10". I received the 10 email notifications.
I could see the Solicitation and proposals without refreshing the
browser. The PIs now do not include TTA Group members (good). All
science categories are still GWT.
2. Click "Review Configuration" button for dsbTest49. I now see two
tabs: Panels and Vetting. I have not yet closed the Solicitation.
3. In Panels tab: configure GWT Panel and SFM Panel. At the top I see
"Panels 2 ... 0/1 Science Categories". Not sure what 0/1 means.
4. Click "Save" button. I now see at the top:
"Panels 2 ... 2/1 Science Categories". Still confused.
5. Before vetting let's close the Solicitation. This is what we would
do in practice. Need to click "Solicitation" tab to get back to
the list view. Then for dsbTest49 click the "Close" button. This
seemed to wipe everything out: I can no longer see the two panels
that were created.

Start over. Redo steps 1-4 with "dsbTest49_2" and "dsb49_2". Let's
first close the Solicitation and then start to configure, etc.

6. Click "Close" button. This does not appear to work either. There
does not appear to be any proposals in the configuration step.

Start over again. Redo steps 1-4 with "dsbTest49_3" and "dsb49_3".
This time do not close the Solicitation at all and see what happens.

7. Start vetting. Change the last three proposals from science
category GWT to SFM. I still need to select the Panel for each
proposal. (I notice later there is an Auto Assign Panels button.)
Click vet all button which check all of the boxes. At the top I
see: "Proposals 10 ... 0 vetted 0 assigned".
8. Click "Save Proposals" button. I now see at the top:
"Proposals 10 ... 10 vetted 10 assinged". I also see
the "Finalize" button appear. If I go back to the Panels tab I see:
"Panels 2 ... 2/2 science categories".
9. Click "Finalize" button. I get an error message that says:
"Configuration could not be finalized". Looking at the system
messages it appears this is causing problems because the
Solicitation is open. Let's try closing the Solicitation.
10. For dsbTest49_3 click the "Close" button. When I go back to the
Configure Review view I do not see any proposals but the Finalize
button is still present. Click "Finalize" button. Seems to work
but when I look at the system messages it appears to fail. I do,
however, get 8 email notifications which probably correspond to
the 6 reviewers on GWT panel plus the 2 reviewers on the SFM
panel.

Reviews
------------

I do see dsbTest49_3 listed so maybe this worked. The summary has 6
reviewers/7 proposals for GWT and 2 reviewers/3 proposals for SFM,
which is correct. Login as one of the reviewers (who has a conflict).

-------------------------------------
User: Lorant Sjouwerman |
--------------------------------------

Reviews
-----------

For dsbTest49_3, I can see both GWT Panel and SFM Panel. Seems like I
should only see GWT since Lorant Sjouwerman is not a reviewer on SFM.
If I click on SFM, however, I do get a system messaging saying that I
am not on SFM Panel.

1. Click on GWT Panel. This gives me the Conflict Declarations page.
2. I am automatically conficted on 001. I select available for the rest
except for 007 where I declare a conflict.
3. Click "Certify Conflict Declarations". I now see the certified
badge. I should be conflicted on 001 and 007 but it seems I can
enter an ISR. I might put the proposal info on the left and the
review stuff on the right (in English we read/write from
left-to-right).
4. Enter ISRs for 002-006 (save all but only complete 005).

-------------------------------------
User: Amy Mioduszewski |
-------------------------------------

Reviews
------------

Login as chair to inspect the status of the reviews.

1. Select GWT Panel. I can see the matrix. Indicates that Lorant
Sjouwermant is not certified but should indeed be certified.

Oops! I should have assinged reviews first. Well...a good test
anyhow. The reviewer should not be able to input ISRs until they are
assigned. Let's assign reviewers.

2. Select either P,S,T, or N for each proposal/reviewer. Click
"Save". Seems to work but the row/column stats do not make sense.

-------------------------------------
User: Lorant Sjouwerman |
--------------------------------------

Reviews
------------

Login as a reviewer again to have a look. I see that the conflict
declarations were saved but I am now uncertified and there is no
button to save conflict declarations. If I make a change the button
appears but then I get an error message that I have already certified.

---------------------------
User: Dana Balser |
---------------------------

Reviews
-----------

1. Declare conflicts: select available for all but 007 which is auto
conflict.
2. Click "Certify Conflict Declarations". I can now see the review
assignments. I only see proposal details for 007, the proposal for
which I am conflicted, which is the opposite of what I expected.

Dana (Firefox, Fedora)

---------------------------------
User: Dana Balser_TTA |
---------------------------------

Solicitations
-----------------

First create a solicitation and generate proposals.

1. Click "Generate Test Solicitation" button.
2. Enter Solicitation Name = "dsbTest48", Proposal Code Prefix =
"dsb48", Proposal Count = 10, PPR. I see the Solicitation is
created and receive the 10 proposal submission emails.
3. Click "Close" button.

Proposals
--------------

1. Select "dsbTest48" Solicitation filter. I can see the proposals
went from "Submitted" to "In Review" from step (2) to step (3)
above. Looks good.

Reviews
------------

Now configure the reviews / vet science categories. I notice here
that the Solicitation name is now all in caps; we should probably
retain case.

1. Click "Configure" button.
2. Need to refresh browser to see the proposals, etc.
3. Enter "GWT Panel" and click "New Science Review Panel" button.
4. Enter "SFM Panel" and click "New Science Review Panel" button.
5. Associate Panels to Science Categories and add members.
6. Click "Save" button. So far so good.
7. Perform the vetting: change the last three proposals from GWT to
SFM, connect the panels to categories, and click the vetted checkbox.
8. Click "Save Proposals" button. Looks good. The numbers at the top
look sensible: 10 vetted, 10 assigned, 2 panels, and 2/2 science
categories. The Finalize button appears.
9. Click "Finalize" button. I am now brought back to the list view.

10. Select "Gravitation Waves and Energetic Transients".
11. Click on "Lorant Sjouwerman" and save link (just in case):
https://tta-test.nrao.edu/reviews/solicitation/28/science-review-panel/38/reviewer/4

--------------------------------------
User: Lorant Sjouwerman |
--------------------------------------

Reviews
------------

I can only see the page to declare reviews. The user is automatically conflicted on one proposal.

1. Select available for all but one (add conflict and reason).
2. Click "Certify Conflict Declarations" button. I see the "certify"
badge in top right corner. It appears I can now enter reviews for
all proposals, including the two for which I am conflicted; maybe
not hooked up yet.

---------------------------------
User: Dana Balser_TTA |
---------------------------------

Reviews
-----------

Now try to change the conflict status.

1. Click "Lorant Sjouwerman". I can view his conflict declarations.
These appear to be what was saved above.
2. Change the conflict status for dsb48-001 from conflicted to available.
3. Click "Certify Conflict Declarations" button.

In the matrix view I see that user Lorant Sjouwerman has been
certified. The conflicts counter (rows and columns) does not appear
to be working. For example, I see 6 conflicts for user Lorant
Sjouwerman where there should be just one.

I cannot check that this worked for user Lorant Sjouwerman since that
part does not appear to be hooked up; that is, ISR stage.

----------------------------------
User: Dana Balser_TTA |
---------------------------------

Check out consensus review prototype. Click "Consensus Reviews"
button on the matrix summary page for dsbTest48. This gives me access
to the prototype.

The summary view looks good, but in addition to filtering we will need
to be able to sort on the different columns (e.g., mean , std, etc.).
Also, we will want some additional information: PI, proposal class,
hours, thesis proposal?, Joint proposal (time), multi-config. It
might also be nice to know the state of the SRP score: initial value,
updated value via adjustment of normalized score(s), or updated value
via direct adjustment. Might be useful, from a metrics point of view,
to do some statistics on this (I think Fred Schwab did this early on
when we went to panel reviews to see how often SRPs changed the SRP
score and by how much).

The detailed view looks good, but I would swap the information
(left<->right). That is, list the ISRs on the left and the SRP
score/comments on the right. We read/write from left to right and the
order of operation is to view the ISRs first and then start work on
the SRPs.

In the ISR, we should note these are "normalized" scores. Reviewers
will be confused since they will recall their "raw" scores. We should
have two columns for the normalized scores: initial and final.

Generate a new Solicitation for PPR (QA Sprint 48 1) with 6 proposals. I get 6 notifications the proposals have been submitted.
Do not close, so have to refresh the Reviews tab to see the configuration option.
Able to make panels, vet, and assign proposals to panels manually. Save all.
Finalize buttons appears when proposals have been vetted and panels assigned to all available SCs. But it gives an error that says "Configuration could not be finalized" appears as a banner but if I look in the system log, then I see "A backend error occur at 'api/panel_configuration_complete/22': validation errors found when completing ScienceReviewPanel configuration on Solicitation: One or more Science ReviewPanels lack a chair (28,29,30) Solicitation is Open"
- Ah, forgot to assign a chair. Okay. I assign a chair but the save button doesn't appear because the FE doesn't think anything has changed. So I have to work a little to make it think a change has occurred because the just changing a panel's SC to a different one and back doesn't do it. The FE thinks that's the "same". A different chair should really count.
  - As a TTA member, I would like changes to Chair Assignment to trigger the Save option in the panel.
- Use finalize button. Still get an error that "Configuration could not be finalized" in the banner. Looking in the system messages, I see that it is because the Solicitation is open. Let me go check that it indeed did not finalize though.
  - As a TTA member, I would like the reason configuration could not be finalized to display as a banner.
- The panels and their members are visible now in the Reviews page but there isn't anything to do because the configuration isn't finalized e.g., not conflicts can be declared, not assigns of Review Types can be viewed or done.
  - Does this auto happen on a save?
    - Make new solicitation QA Sprint 48 3 with 6 proposals. Do not close. Go to reviews page, refresh to see list.
      - As a user, I would like to not have to refresh a page to ensure I am seeing the correct content (story already exists though).
    - It happens when both vetting and panel creation is saved.
  - If I change a srp member does it propagate correctly?
    - Yes, it appears to propagate correctly.
  - This is a choice if we want to lock the view until Configuration is Finalized.
If I do not change any SCs during vetting, such that all the Requested SCs are the same (GWT), and instead just distribute the proposals across the panels, the finalize button will not appear until I have at least 1 Vetted SCs per expected Solicitation SC.
- I can change the Vetted SCs to match the expected and then only give 2 of the three panels proposals and the finalize button works. It looks like the criteria is that at least one proposal per SC is required to be finalized. I thought the requirement was
  - All proposals must be vetted
  - Each Science Category, as specified in the Solicitation, must be assigned to a panel.
  - The Solicitation is Closed.
  - As a product owner, I would like the criteria for Finalization to be clear and correct.
- While trying to understand this SC assignment, 2 of 6 proposals just disappeared from the list. They happened to be the ones assigned to a panel that I changed the SC of and saved.
  - Replicated in Sprint 49. Any edit to a panel that has proposals assigned to it will "delete" those proposals from the UI permentantly.
Finally Finalize QA Sprint 48 2. No error appears. List of Solicitations has a marker that says "finalized" now.
- I can go into the configuration panel but changes are not saved. An error is thrown saying it cannot be saved.
  - The Configuration panel should be available to TTA member because that is where the vetting comments are stored (for now) and maybe the TTA member wants to double check assignments, though they cannot change them.
  - The save button should just be grayed out I think. Much like the Author copy isn't available on proposals after the close of solicitation. A banner display with "Cannot modify" would also be nice.
    - As a TTA member, I would an indicator that I can no longer make modifications to Vetting or Panels.
Finally inside the Panel View of SRPs as a TTA member.
- Copy URL and log in a chair for NGA. Paste URL so I can see the Chair's view of the matrix.
- There's a state marker for Consensus as well as the ISRs states and review types.
- This is very cluttered now.
Chair Conflicts: Go to Chair's conflicts so I can certify as the chair.
- Set both to available. Click Certify, which shows up but isn't available until I have set both from Unknown. After certify, I am on the ISR screen. Seems to work correctly.
  - Marker that I am certified.
  - Navigation button "back to panel".
  - Shows Review State (default None).
  - No Review Type, which isn't expected
  - Only clicking on the drop down arrow (very small) expands the proposal.
  - Have place for comments, have a place for score. Can see where proposal will go eventually.
  - Ordering of Reviewers has changed for some reason.
    - As a TTA member and SRP Chair, I would like the ordering of the reviewers to not change on the Panel View.
  - I cannot assign multi primaries or secondaries without a warning message. It does not require me to assign any primary or secondary role. I can save if all are None. I can save if all are Tertiary.
  - As a Chair, I cannot see the other reviewer's conflicts or ISRs from the panel but! I found later that I can just change the URL and then see them.
SRP Reviewer: Log in as TTA member, navigate to srp member's conflicts. Copy URL, log in as the member. Paste URL. Can see their view now.SRP
- Conflict management:
  - I can see Title, Proposal ID, Abstract
  - Setting to Conflicted will make a Reason text field appear. Give a reason.
- Can Certify before a response is given in the text field but a str must be given or else an error is thrown.
- After certify, I can see the ISRs.
- Once certified, I cannot see the conflict management (good).
- Conflict state not shown on proposal list.
- I can enter in ISRs for the proposal for which I marked Conflicted.
  - As a SRP Member, I should not be able to enter comments or a score for proposals for which I am Conflicted (+AutoConflict)
- There is no back to panel option which is fine because this is the SRP reviewer's view and they don't get a panel view.
- There are no filters/sorting on the list of Proposals when viewing ISRs
  - ~~As a SRP Member, I would like filtering on the Proposal List by Review Type, Review State, Conflict State~~. (maybe this is overkill to have both)
  - As a SRP Member, I would like sorting on the Proposal List by Review Type, Review State, Proposal ID, Conflict State
TTA Member:
- I can see the Conflict Declarations of any reviewer at any time, so I have access to the reason.
- I can change their conflict states and add a reason if needed. Certify again.
- Change a state from Available to Conflicted.
  - On the panel view, it shows up correctly, but I know that reviewers can still see Conflicted Proposals so it isn't changing things downstream.
Chair: Log in as chair.
- If i paste the URL from the SRP member, I can see their ISRs.
  - As a product owner, I would like the Reviewers to not be able to access another reviewer's ISRs or conflict via pasting the URL.
- Everyone has certified conflicts, I can see the marker that says they are certified now. I can see one member is conflicted. The marker is small though.
- I can still give the conflicted reviewer a Review Type.
  - As a Chair, I would like to be prevented from assigning Review Types to a Conflicted ISR
- I can save. No confirmation on a save
  - As a Chair, I would like confirmation of a successful save when saving Review Types in the Panel View.
- As the chair, I go to do my ISRs. I can see the the Review Type in the ISRs that reflect the assignments I made.
- I entered comments into a ISR but left the score as a default of 0. I tried to save. It rightly threw an error that I didn't have an acceptable score but it looks like it still saved. The Review State is showing Saved now.
  - As a SRP Member, I would like the list of Proposals to not reorder after I have modified by ISR.
- It would be nice to have a message saying I will lose my work if I don't save.
  - As an SRP member, I would like a warning before I navigate away from unsaved work in Consensus and ISRs.
- There isn't an option to Complete or Finalize my reviews.
- I went back into reviews to move them to Saved at least and suddenly there were pregen Comments?
  - Saving doesn't clear the review changes pending. Maybe saved isn't hooked up yet.
- Going to Consensus.
  - I can already navigate into Consensus but it is not connected so is just there for show.
  - Feasibility tab kicks me out of the review (doh).
  - On list of Proposals
    - Mean Finalized Normalized Individual Score
    - Std of ^
    - SRP Score
    - Title, Proposal ID
    - Who is Primary and Secondary
    - (Consensus) Review State
    - Missing Completed/Finalized options still
  - In expanded Proposal view:
    - SRP Score and place to modify if chair.
    - Comments for PI (not yet prefilled prompts)
    - Internal Comments (not yet pregen prompts)
    - In Reviews tab:
      - a "Score" which is presumably the Finalized Normalized Individual Score per ISR
        The chair can change these
      - Comments per ISR
      - Missing - Normalized Individual Score
  - You can sort and filter on the list of proposals in Consensus. I don't think we need both; now that I look at it, I like sorting more than filtering here.

STT-863 - Getting issue details... STATUS

OSR proposal - Submitted

I could save an edit to Author and Observatory Copy while Proposal State is In Review.
Though it says it saved to the Observatory Copy, it actually applied them to the Author Copy instead.
The changes did not show the changes in OSR UI in any Review State.

PPR does not show many details yet, but it looks like it doesn't work there either.

I tested changing the Science Category on the Obs Copy and saving before the Solicitation closes and it did not affect the title I see in the Vetting phase.
Change the Author Copy title and Science Category of a Submitted Proposal. Resubmit:
- the title change is shown properly in the Vetted UI.
- However, the Vetted Science Category shows up as the original Science category. The requested science category shows up as the one I changed it to in the Author Copy. The "unlinked" icon is shown already, though I haven't made any vetting changes.
I changed the title of an In Review Obs Copy. The save was applied to the Author Copy only apparently. The change was not shown in the title in Vetting, though I could navigate into the proposal from vetting and see that it was applied to the Author Copy.

It seems like the "save" function is meaningless on the Obs Copy. Any changes I committed to a Submitted or In Review proposal applied instead to the Author Copy. Even as a TTA member, I could not resubmit the proposal to force the change (not that it would have worked anyway because no changes could be saved the Observatory Copy).

STT-861 - Getting issue details... STATUS

As author, I had a proposal for a solicitation +ppr that was open. I saved but did not submit it. Only author copy exists. I delete it. It is still on my list of proposals as an Author but I cannot edit it.

As a TTA member, I can see the Hidden proposal. I can get into it an edit it. I can save changes. I cannot submit it though, even though the CfP hasn't closed.

I can withdrawl it though? I thought only Submitted proposals could be withdrawn.
I now appears in my proposal list as the author and I cannot edit it when it is withdrawn.
There is not an observatory copy though, so I cannot see anything about the proposal. This needs to be reconciled with withdrawn state model transitions.

There is another hidden proposal that is not withdrawn, just deleted. I close the solicitation.

As a TTA member, there are not Observatory Copies to edit or view for either proposals because they were never submitted. I can view the Author Copies though.
There is no submit button. I can make edits and save it, but I am never given a submit button.

At least neither the Withdrawn or Submitted ones show up on the Reviews Page but only because, I suspect, they were never submitted.

What is weird is when I navigate into a Proposal with Proposal State of Submitted, I can see a submit button. It isn't available but it at least shows up in the UI. It doesn't even show up in the UI for the case I was describing prior to this one.

I withdrew a Proposal after the Close and it still appears in the Vetting UI.

I do not understand the logic behind the ordering of the filtering in the Proposals List.

Dana (Firefox, Fedora)

---------------------------------
User: Dana Balser_TTA |
---------------------------------

First generate test solicitation.

Solicitations
-----------------

1. Click Generate Test Solicitation button.
2. Enter "sprint47Test" for name; enter "demo47" for proposal code
prefix; enter "10" for proposal count; and select PPR.
3. Click Submit button. Did not seem to take; needed to refresh
browser. I did receive 10 notification emails.

Proposals
--------------

I see the 10 proposals for this Solicitation. Looks okay but they are
all in GWT which is odd.

Reviews
------------

Initially I do not see the generated Solicitation. Need to refresh
browser.

1. Click Configure.
2. Enter "GWT" as name for new panel; click "New Science Review Panel" button.
3. Select GWT science category.
4. Add Amy (chair), Mark, Toney, Jeff, and Ryan as panel members.
5. Click Save button.
6. Check vetted buttons for all 10 proposals.
7. Select GWT SRP for all 10 proposals.
8. Click "Save Proposals" button.
9. Click Finalize button. Got error message that "Configuration could
not be finalized". Refreshed browser. Click Finalize button again
with the same result. Go to Proposals page and back. Refresh browser
to see Solicitation, and then click on bar. Now I see the GWT panel.
I did not receive a notification email.
10. Click on GWT panel. I now see a summary view of panels (I think),
as I have only configured one SRP. But I cannot do anything. I
do not see the full matrix of reviewers vs proposals, just a
summary at the top of the page.

Start over!

Solicitation: dsbTest47
Proposal prefix: dsbTest
Proposal count: 10

When configuring the panels I messed up since I created a PCO SRP,
connected to the PCO science category, but then realized the PCO
science category cannot be selected when vetting. Anyhow, I tried
to save proposals but could not ever get finalize button.

Try again!

Solicitation: dsbTest47_2
Proposal prefix: dsbTest_2
Proposal count: 10

When configuring the panels I kind of maybe messed up again. I did
not select the chairs until after I clicked Save. Anyhow, after
vetting the proposals I clicked Save Proposals. This time I was more
patient and waited until the Finalize button appeared. Took more than
10 seconds. Then clicked the Finalize button. Get same error message.
Get same result. No notification email.

Try one more time (next day)!

Solicitation: dsbTest47_3
Proposal prefix: dsbTest_3
Proposal count: 10

Same result. I also saved the url:

https://tta-test.nrao.edu/reviews/solicitation/18/science-review-panel/19

and then logged in as the chair (Lorant Sjouwerman), cut/paste the url,
and get the same result. Saved two screen shots: one of the configuration page
just before clicking the Finalize button, and one of the review assignment page.

After talking with Allie she had similar problems but recommended that
I close the Solicitation. After this, and a few browser refreshes, I
was able to see the matrix. Moreover, I received some notification emails
about the panel now being open for business.

-----------------------------
Lorant Sjouwerman |
------------------------------

Next start assigning reviewers to proposals.

1. Assign reviewers to proposals. Automatic conflicts not working but
I think that was expected. That is, I can assign someone who is conflicted
as Primary.
2. Click Save. Not yet implemented as expected.

Overall, I think this page is too busy. Too much information to get
an overall view of the status. Breaking each cell in the grid into
sub-cells (P,S,T,N), together with the conflict and review state, adds
a lot of information. I think having text (review type) and color
(conflict/review state) is more compact and contains most of the
required information.

Including a summary of the conflict/review state and review type
assignments along the rows and columns, however, is very useful. It
does add to the amount of information to the page but is worth it.

Next start the conflict declarations.

1. Click on my name.
2. View Instructions. Nice.
3. Declare if I am available or conflicted. For a lot of proposals it
might get tedious clicking available for most of the proposals, but that
might not be a bad thing. Forces folks to decide.
4. Click "Certify Conflict Declaration". Not yet implemented.

---------------------------------
User: Dana Balser_TTA |
---------------------------------

Next login as a TTA Group member and override a conflict assignment.

1. Click Lorant Sjouwerman.
2. I can override a conflict status. Not sure how to save. I do not
receive a notification email; I recall Sam was able to get a
notification in the sprint review.

As a TTA Group member I can also see the Individual Science Review
structure. Overall this structure looks good to me.

From discussions with Reid, I know there are some changes coming to the review configuration panel in that it is likely that the configuration and the vetting will be separated. That being said:
- I vetted proposals. Saved vetting side. I made panels and updated the SRP members. Saved panel side. Added proposals to panels on vetting side. Saved. Panels did not update their icons about how many proposals were assigned to them without a refresh.
- "Configuration could not be finalized" error". Tried again. "A backend error occurred at api/panel_configuration_complete/9: internal server error the server encountered an unexpected internal error (generated by waitress)" error.
I added 6 reviewers (SRP Member + Chair) and now the chair view doesn't populate.
Now only 1 of the 3 panels populates. I don't know how I broke it.
- Tried a new solicitation. At first, I couldn't see the 6 person panel. Then I could for some reason. It wasn't tied to a refresh though.
Cannot see the reviews tab as a non tta member, even if on the panel (as chair or member) – not yet implemented i think.
Functions not hooked up to the backend yet:
- Finalize Review Configuration without an error (tta member)
- view the conflicts or ISRs as a SRP Member or Chair (unless also a TTA member)
- Save an ISR (tta)
- Certify Conflicts (tta)
- Save the Review Type assignment (chair/tta member)

Dana (Firefox, Fedora)

---------------------------------
User: Dana Balser_TTA |
---------------------------------

Solicitations
-----------------

Attempt to generate test Solicitation with proposals.

1. Click "Generate Test Solicitation" button. Form appears.
2. Enter Solicitation Name (Sprint 46 Testing), Proposal Code Prefix
(S46Test), Proposal Count (10); select PPR Proposal Process; click
submit. I see the new Solicitation. Good.

Proposals
--------------

Check that the test proposals exist and are reasonable.

1. Filter on "Sprint 46 Testing". I can there are 10 proposals. They
have the correct proposal id prefix. They are all submitted and
have the same science category (GWT). Probably want some variation
here in the science category.

Reviews
------------

Perform vetting of the science categories.

1. Initially I do not see the Solicitation I created, but then
remembered I have to do a refresh. Now I see "Sprint 46 Testing".
2. Click Configure. I can see the interface. Proposals (left) and
Panels (right). Little confused at first. Not sure what
"assigned" means under Proposals section. The Panels section must
be to create SRPs and to connect them to science categories.
3. Let's create SRPs first. Enter GWT, click New Science Review Panel
button. Continue for SFM and NGA. The number next to Panels is 2
but I have created 3 SRPs. This number does not change.
4. Select the obvious science category to the SRP. I see that more
than one science category can be connected to the SRP (okay).
Across from Panels I see 0/3 Science Categories; expected this to
update.
5. Add reviewers. As you start to enter a name the interface shows
possible options which can be selected (okay). Once selected a new
search is opened with the previous name. Probably better if this
was blank. Select chairs by clicking icon (okay). Only one chair
can be selected but the requirement is to have one or more chairs.
6. Refreshed the browser and this cleared everything. Oops.
7. Start over. Redo 3-5. Appears that actions in the Panel section
are not changing anything in the Proposals section.
8. Move to Proposals and start vetting. Seems straight forward, but
again the numbers at the top (0 vetted 0 assigned) are not
updating. The panel selector in the top right of each proposal is
none and grayed out. I was expecting this to be updated once I
connected the SRP to the science category.
9. Filter on Science Category. When I select GWT the form clears.
All science categories go back to GWT and none are vetted (checkbox
cleared). The filters do not appear to be working. I am assuming
what is meant by assigned is when a science category is connected to
and SRP.

I can potentially see the value of having the functionality of
assigning SRPs to science categories and the vetting of science
categories to be on the same page. But hard to tell since the
interface did not appear to be updating in some places. Actions on
the Panels section had no effect on the Proposals section.
Regardless, I do not think we will be doing these at the same time (or
even the same TTA Group member). So I would keep them separate.

Is the search on science category the requested or vetted? Seems like
it would be useful to have both. Or maybe just the vetted (which
initially defaults to the requested).

Ah, just realized that under Panels I had to click the Save button.
When I do this I see 3/3 Science Categories (got updated). Still 2
next to Panels. Too, no changes to the Proposals section, except that
I can now select the Panel (no longer grayed out). I would have
thought this would automatically update since I have already connected
each panel to a science category.

And when I click Save Proposals under Proposal I see 10 vetted now.
Next, for each proposal I select the panel. If I click Save Proposals
again the number updates: 10 Assigned. Okay good. As noted, this
really does not make sense because I have already connected the panel
with the science category under Panels. So for proposal S46Test-001,
which has a vetted science category of Gavitational Waves and
Energetic Transients, I can select the SFM panel. But under Panels
the SFM panel is connected to Star Formation.

After saving things I see that the filters seem to work. Okay, I
guess this makes sense. But if I do not want to loose work I have to
keep clicking save.

---------------------------------

Allie Costa

---------------------------------

Solicitations

Can generate a demo solicitations and proposals easily enough.
When I hit the save button though, it leaves the information in the UI that I just asked to be generated. There isn't feedback to show that the solicitation is actually generated.
It would be nice if in the list of solicitations, the text were aligned within the list.

PPR Review

The landing page for the Review Tab shows a nice summary of each of the solicitations+proposal process.
I can only access the review configuration page for a Solicitation that is closed. This is not the desired behavior. We want to be able to configure the panels before the solicitation closes. The vetting will only occur after the close really. I think this is more evidence that the vetting and panel configuration can indeed be separated.
Clicking anywhere on the line for a panel, takes me to an overview of the panel (if there are people are assigned to it). That's neat.
There's no "easy" way to back out of panel view without clicking on the Reviews Tab, which takes me back to the landing page.
(Bug) You have to refresh the page when entering the Review Configuration UI, otherwise the proposals are populated and the review panels are not displayed when added.
The Finalize Button is briefly displayed when I first enter the page, even when it should not be. It triggers when all proposals are vetted, all science categories are assigned to a panel, and all proposals are assigned to a panel. I think this is okay. Maybe it would be better if it was always there but grey'd out when not viable? Like the submit button on proposals so the user knows there's another step.
There does not seem to yet be a way to auto distribute proposals to panels.
Proposal Vetting
- Can navigate into the proposal pretty easily from the vetting side.
- Can change the Science Category and enter in comments.
- Note! The save button on the vetting side is independent of the same button on the panel side. So if there are unsaved changed on one or the other, it won't save them. This isn't clear in the UI that is will happen though. It is nice seeing the proposal information next to the panels but really I think these could be separate tabs. Since most cases we will NOT be assigning by hand, it isn't that important to have both views at once.
- After save, order of list changes, which is confusing at first (known already).
Panel Creation
- Note! The save button on the vetting side is independent of the same button on the panel side. So if there are unsaved changed on one or the other, it won't save them. This isn't clear in the UI and maybe suggests that these really should be separate tabs for clarity.
- We still want a more reviewer focused view. As we add more people and have to search for them, we want to be sure that the reviewer we are adding is the correct one or that they indeed have a user account.
- How do I delete a panel or rename it?
- I tried to have identical names for panels and it correctly gave me an error. But now I can't delete or rename it without hitting refresh.
- The number next to Panels is (2) but the number next to Proposals is (5). If these are showing how many of each, then Panels was showing (2) when I only had 1 panel created and never updated when I eventually made 3 panels.
- I do think the drop down on the panel creation side should be restricted to the SC in the solicitation.
- How do you see the proposals that are assigned to the panel in the panel side? Maybe it isn't supposed to show them there and you use the proposal vetting side to inspect that. I am still concerned about the save relationship between the left and right sides of the screen then.
Refresh Bug Again: I created two solicitations with different names.
- I configured the review for one of them and saved.
- I configured the review for the other one and saved.
- I went back to the first to look at it and it had all of the panels from the other one in addition to the ones that were supposed to be there (see image). The proposals are not shared though.
- I can modify all of them. Saving throws the error about duplicate names.
- I cannot delete panels however.
- If I refresh, then it populates the panels properly.

Dana (Firefox, Fedora)

---------------------------------
User: Dana Balser_TTA |
---------------------------------

Solicitations
-----------------

Inspect the Solicitation created by Sam during the sprint review using curl and
the ability to Close Solicitations.

1. Click on "Sprint 45 Demo." Looks okay.
2. Click Close button on Solicitation "Sem_25A". I now see this is indicated as closed.
3. Go to Home page. I now see that this Solicitation is listed under Recently Closed Solicitations.
4. Go to Proposals. Under the list view there is no indication that Sem_25A is closed. It might
be useful to indicate this Solicitation is closed.
5. Click Add. When I try to select a Solicitation I notice that Sem_25A is not listed. Good.

Proposals
--------------

Inspected the Demo Proposals. We probably should have dummy authors
since we do not want to confuse our users or TTA Group members. Might
not want to use a Beasley memo for the scientific justification. In
some cases we will need (I think) to include a viable Allocation
Request.

Reviews
-----------

Inspect SRP UI prototype which is under construction.

1. Click SEM_25A under mocks. Need to click twice as indicated at the sprint review.
2. Click on Science Panel 54.
3. Matrix view is okay. Might be better to list reviewer across the
top (columns) and have the proposals in the left (rows). But seems
reasonable otherwise.
4. Enter some conflicts. Not connected to a given user yet but this
was expected. The Certify Conflict Declarations button will highlight
once they are all entered.

Dana (Firefox, Fedora)

---------------------------
User: Dana Balser |
---------------------------

Proposals
---------------

Create a proposal and check if you can add and view the science category.

1. Click Add.
2. Enter title.
3. Select Solicitation (Sem_25A). After selected there is an option to select the science category.
4. Click Create. Error message and below the science category there is red text "Select a Science Category".
5. Select Science Category (Star Formation).
6. Click Create. We discussed this at the Sprint Review but it is not clear what fields should be
created at this stage. We probably need some unique identifier for the proposal at this stage since
in practice the title could be duplicated. So maybe the solicitation is the only field necessary. But
it seems that science category should not be listed at this stage but later.
7. Enter abstract and save.
8. Click List. I can view the science category. Looks good. No filtering option yet.
9. Create another proposal and select a different science category. Looks okay.
10. In Basic Information, select and difference science category, save, and check. Looks okay.

---------------------------------
User: Dana Balser_TTA |
---------------------------------

Reviews
------------

Check that a TTA Group member can perform the vetting of science categories.

1. Click on Configure for SC_GBT_24B Solicitation.
2. Click on Vetting tab. The vetting and review configuration operations are on the same page.
Maybe better to have this separate but I could be convinced otherwise.
3. Click binocular icon. Opens new browser tab to the Proposal List with error message
"Proposal 'SEM25A-001' cannot be viewed". Maybe this is not yet implemented. Best to
just open a PDF of the proposal.
4. Click > icon for SC_GBT_24A-1 to expand.
5. Select PCO and add text to Notes.
6. Click Vetted check box.
7. Click Save Vetting Info button. Not sure this does anything.
7. Went to Proposal tab to check something and then back to Review-->Vetting. Changes do not
persist but I recall that is expected at this stage.
8. Vetted a few more proposals. Looks okay.
9. Click Vet All Proposals button. The remaining proposals Vetted check box were checked.

Overall this looks reasonable. In the slim view I notice that the
science category is the vetted category which is good. There is also
an icon next to the science category that indicates if the vetted
science category is different than the requested science category. If
you make a change and then go back to the original the icon is not
updated.

We probably want to be able to filter on the science category
(requested and vetted).

Allie (Firefox,redhat)

Proposal Creation

I do not like that the Science Category is part of the landing page for a new proposal. I think it should be on the next screen instead only.
I can select a SC and it persists.
I made a proposal and submitted it. The Proposal State updated to "submitted" but all of the information in the proposal is now shown until I refresh.
Issues from previous proposal creation (e.g., sprint 42, 41) persist
The Withdraw button is only available to TTA members. It is also only available if I am editing a proposal. This button should be at a higher level that the edit view.

Reviews

Review a newly created DDT proposal
- Sam said that the backend wants the boolean for the score but the front end is not changed, so it can accept any number.
- The Review State correctly changed from "blank" to "saved" when I save the OSR. It then goes to Finalized once I submit the review and I can no longer make edits to the review.
- It is not obvious how to back out of this page though. Really I have to hit one of the top navigational tabs. There should be an "x" or close button.
- I wish the acknowledgement bar (NRAO, AUI, etc) were lower because the proposal preview is very squished.
- The Edit button should not be there (Sprint 43 comment)
Review Configuration
- I like the vetting page, but I do not have experience vetting proposals to know if this checks all the boxes.
- New panel configuration page but it isn't complete.
- Not sure of the interaction of how vetting will work and the panels, considering that a change to the vetted SC will affect where the proposal goes. Much of the panel configuration is still very much so under development, so likely this will become more clear later.

Allie (Firefox,redhat)

Review Configuration

Cannot assign two chair to a panel, which is a requirement.
want to consider UI for the 90% useage – automatic assignment of proposals to panels.
- tta member is mainly focused on reviewers
- information about the people (verify that it is the person we think it is)
- separate feasibility from panel entirely.
  - feasibility review concept isn't correct
protect again reviewers being on multiple panels
adding an external reviewer to an ISR would be a separate interface because it happens after review configuration is finished
OSR have review state (blank, saved, completed/finalized)

Dana (Firefox, Fedora)

Observatory Site Review

---------------------------
User: Dana Balser |
---------------------------

Proposals
--------------

Created a proposal

1. Click Add, enter title, select Solicitation SC_GBT_24B), save.
2. Enter General info, save.
3. Upload scientific justification.
4. Enter Capability Request, save.
5. Check Observation Specification (looks good).
6. Submit. Acts like a DDT as the proposal state goes to "In Review".
Proposal ID is SEM24B-005 (but okay probably not implemented yet).

Reviews
------------

I am able to see SEM24B-005 but no other reviews. A bit odd. Only
the TTA Group member should see proposals for review in the OSR.

----------------------------------
User: Dana Balser_TTA |
---------------------------------

Reviews
-----------

When I enter the Reviews tab I can see the proposals that are in the
"In Review" state.

Perform a review

1. Click on the Review button for SEM24B-005. Layout is okay. I can
see details about the proposal below but no proposal ID. Might be
nice to have some info at the top of the page: proposal ID, PI, and
title. The most important info to see here is the technical justification,
which does not exist yet.
2. Wanted to cancel but could not find a cancel button or icon; clicking
the Review tab did work, however, and also gave me a popup message.

3. Entered a score (1), clicked save. Status when from blank to saved.
4. Added some science comments for PI.
5. Added some technical comments for PI and internal.
6. Clicked Save button.
7. Clicked Submit Review button. Review is finalized.
8. The review persists; I can leave page and come back.

Science Panel Review Process Configuration

---------------------------------
User: Dana Balser_TTA |
---------------------------------

Reviews
-----------

Under Mocks, clicked the Configure button for Solicitation Sem_25A.
First impression is that the interface is a bit confusing. For
example, separation of science and feasibility reviews is not at first
clear. After staring at this for a while I see the logic. The panel
on the left show the items that are to be selected: science
categories, proposals, and review groups. The panel on the right
makes the connections.

Tried to configure panels:

1. Added a panel: enter "SSP" and clicked "New Science Review Panel".
2. Selected science category "SSP" and two proposals: Sem_25A-1 and Sem_25A-2.
3. Selected Amy Mioduszewski (chair), Mark Claussen, and Emmanuel
Momjian. Odd that the name persists in the search box after selected.
4. Clicked Save button. I now see the Finalize button.
5. Perform 1-4 to add SFM and PCO. I was able to add the same person
to two SRPs which I do not think is allowed. But we do need a way
to assign a reviewer in panel 1 to a proposal in panel 2.

Tried to configure feasibility reviewers:

1. Add Review Group: enter "HSA" and click "New Review Group".
2. Remove John Doe. This does not work, but there is a note that this
is only partially implement so okay.
3. Add Mark Claussen. Again this does not work, but okay.
4. Perform assignment: select proposals to HSA group: Sem_25A-1,
Sem_25A-2, Sem_25A-6. I do not think this is what we want, however,
since we changed the requirements to add users to proposals not
groups. The groups would act as an alias.

I would prefer we separate these configurations. For example, have a
tab that configures the science reviews and another tab that
configures feasibility reviews (which may be further divided into
technical and data management).

Moreover, I think the focus should be on people. We are assigning
people to do things. So for the science review configuration I would
have a list of the panels with the members listed below. Something like:

SSP SRP SSP Science Category v Proposals v
==================================================================
Name       Username    Email                          Role
--------------------- ------------------------ -----------------------------------------   -------------
Lynn Mathews lynndmatthews    lmatthew@haystack.mit.edu chair
Paula Benaglia pbenaglia       pben.radio@gmail.com    member
Jose Guirado    guirado          jose.c.guirado@uv.es           member
etc.

PCO SRP PCO Science Category v Proposals v
==================================================================
Name Username Email Role
------------------- --------------------------- ----------------------------------------- ------
etc.

The 90% use case will have one science category per SRP and the
proposals can be automatically assigned. In practice I suspect we
will almost always want to upload a file. The username is useful to
trouble shoot on the test server and the email helps to make sure we
have the right person (we have lots of duplicate accounts and I do not
think the new account system will "fix" this issue).

I am less sure about the feasibility review configuration. Here
people are directly connected to proposals (not groups). Need to
think more about this.

DDT proposal created as Allie_Costa
- hitting save does not kick the view over to Observation Specifications and the little icon on OS doesn't update until I go there myself.
- when I hit submit, it leaves me on the author copy even though I should be in the Observatory Copy view only mode. I have to go back to proposal list to get to the Observatory Copy.
- PID assigned, and datetime in the correct timezone available in the UI.
- going to list of proposals shows the DDT is now "In Review", only observatory copy is available.
  - cannot see the Author Copy anymore and a nice tooltip shows that I cannot go to it either.
  - Still can't see the OSs
- from a general user's POV, I'm done.
Sem_25A proposal created without issue, submitted as Allie_Costa
- Proposal view correctly shows the Submitted Proposal with a View and Edit button
- View takes me to Observatory copy, where I can toggle directly to the Author copy and edit it there
- Observatory copy does not have an edit option for me
- when I am in the author copy edit mode, i cannot access the observatory copy.
I can access the reviews page as a non tta member. I can't review a DDT or configure a panel though
Logging in as TTA Allie
- I can see all the proposals
- I can edit observatory or author copy of proposal 1.
- I can access the Reviews tab
  - The SC_GBT_24B has a configure panel. As a DDT, I'm not sure that is what we would want.
  - Currently the list of proposals will expand and it pushes the configuration panel down. This might be annoying when we have 300 proposals.
- Reviewing a DDT
  - The panels are very squashed together when the review tab opens. I have to scroll to see most of the proposal information even though there is plenty of real-estate on the page.
  - The Observatory Copy is only shown in the review page but it has an edit button that redirects to the proposal list with an error message that says 'cannot edited this version" which is good that it doesn't allow me to edit but there shouldn't be an edit button there to start with.
  - I cannot see the Observation Specifications still.
  - I can navigate between the comment tabs without saving. yay.
  - The score should be a boolean or 0/1 for a DDT not a higher integer or otherwise.
  - Saved and submitted review for Proposal 1.
- Reviewing Proposal 2 now.
  - Without any edits, the banner of Review Changes Pending is showing up.
  - Submitting the review changes the review state to Finalized.
  - Who did the review is not displayed anywhere that I can tell. STT-925 was that is was stored, not displayed I suppose
  - It is not clear in the review list what DDT proposal have a finalized review. Don't have a story for this yet.
  - No prompt for finalizing?
Configuring a review panel
- Saving a review panel didn't actually save it. Maybe for the better right now because we're experimenting with it.
- I saved and then made someone a chair. It did not show the save button again.
- Finalize doesn't do anything yet, as expected as we haven't done this story STT-899
- I can add science categories to a panel
- I can add proposals to a panel manually. fine until we get a lot of them ( upcoming work in STT-1078)
- I can add science reviewers
  - adding via the search function is great but it also pre populates the same name into the search bar when i go to add another one.
- I like the little icons on the side to track how many science categories, proposals, and chairs there are in a panel.
- I like the tracking on the panel on the left which shows which proposals have been assigned. It would be nice to filter on that once we get a lot of proposals.
Ignoring the feasibility reviews, as they are a place holder

Allie (Firefox, redhat)

User: Allie Costa

video of bug walkthrough

Proposal - VLA Continuum SEM_25A

I created a new proposal, entered the title and hit save. Then I could enter the remaining basic information.
- I cannot interact with the observatory copy, as expected.
- With just the basic information provided, I tried to hit the submit button and an error displayed saying that it was missing the SJ, AR, CR, etc.
- Now though, the "Proposal Changes Pending" banner will not disappear even when I hit save. Nothing has changed though from when I last saved it and when I try to navigate away, it brings up the warning that I will lose unsaved changes. Going to a different tab (SJ) clears it though.
I can submit my proposal and the proposal ID assigned to it, the proposal state changes to submitted, and the time of submission is displayed. It is displayed in MST though?
Now I should be able access the Observatory Copy but I cannot click on it. I have to go back to the list of proposals and only when I use the "view" button, can I go to the observatory copy. I can access the author copy from there. I suppose it makes sense because I cannot edit the Observatory Copy and I am in the edit mode for the author copy. This is not intuitive though for a proposer. On a submission, it needs to redirect to the observatory copy/view mode.
Why does the banner say "Author Version" instead of "Author Copy"?
There is a noticeable lag when navigating.
In the observatory copy, It does not display information about the capability request(s). It says there are no observation specifications either, even though there are.
- when I revisit it, now I can see the CR? Not sure why it didn't trigger it the first time. I can replicate this. I have to go to the Author copy first, and then the ARs will show up but then I lose the OSs.
In the view only mode of the author copy (not what we initially discussed for the author copy but might be okay)
- I recall that the display of these fields hasn't been addressed yet. All the information appears to the be there except the Calibration Parameters are missing (VLA) and the Advanced tab is missing (not that we necessarily need in it in the view only mode).
- The observation specification for the AR looks okay and matches my expectations mostly.
I like that the view only modes are summary-like.
I tried to add another AR to my submitted proposal. I cannot save it because the save button is gone, so I cannot resubmit it.
- Why isn't the save button appearing when I add a new AR or field source?
- It only seems to appear when I edit a previously submitted field. I still cannot resubmit though, even though I have a whole new AR that wasn't submitted.
- When I go to view the Observatory copy, it correctly does not show the new AR and the author copy does show it.
- I had to back out of the proposal completely and then go back into it to be able to re-submit the proposal.
- I could edit the existing fields, save, but again, the resubmit button is not available.
- Deleting a FS or AR doesn't always work either

Proposal - GBT Special call

I created a new proposal under a different solicitation and it pre-populated everything from the VLA continuum proposal until I hit refresh.
2 ARs, 1 CR each --> 4 OSs, one with all calibration parameters
When I submit, it goes directly to In Review as expected.
For the CR that has no calibration parameter specified, the tab is emtpy. For the one with the CP, then it lists them.

General

When unsubmitted changes are pending between the author copy and observatory copy, the list view of the proposals correctly displays the banner. However there isn't any indication in the view only modes or edit mode that there are unsubmitted changes. This needs to be clearer to a proposer.
I can delete a proposal in the draft state but it still shows up in my proposal list. the state is correctly "Hidden" at least.
- I can edit it even though it is hidden.
When I go in and edit a submitted proposal Observatory Copy as a TTA member, it immediately shows a banner saying "allocation request changes pending" even though I haven't done anything yet.
- There is a save button at the Allocation Request level. Not sure what it does. Why I click it, it gives an error saying "check form for errors"
  - I can edit the CR and it has a different save button that is the normal one. It works as expected.
- on the list of proposals, it does not show that there is a difference between the observatory copy and the author copy, which I think is okay.
As a TTA member, the toggle between author and observatory copy in the edit mode works just fine.
As a TTA member, I have to go into the edit mode to access the withdraw button. I think it would be available on the view modes too.
If a proposal is withdrawn, the author should not be able to see the author copy, let alone edit it.
Things are starting to feel "off" - likely because we haven't discussed optimizing really. For example, the list view of the proposals has different looks per proposal depending on the state. it just looks a bit weird, particularly for the tta member view that can see all the proposals. the lack of consistency is off putting to me.

Dana (Firefox, Fedora)

---------------------------
User: Dana Balser |
--------------------------

Proposals
--------------

- Create a proposal. Throughout the process I can see in the top
right the version: author (orange highlight), observatory (gray).
If I click "observatory" nothing happens. I can also see orange
horizontal bar labeled "AUTHOR VERSION".

1. Click Add button.
2. Enter title, select solicitation (Sem_25A), and save.
3. Basic Information tab: enter abstract and save.
4. Allocation Requests tab: click Add button, create Capability, and save.
5. Confirmed that an Observation Specification is created.

- Submit proposal.

1. Click Submit button. I can see in the top left that the proposal is
now submitted and a proposal ID. In the top right the I now see the
date of submission and a grayed out Re-Submit button.
2. Click Observatory button. Nothing happens.
3. Click List button. I can see the one proposal listed with
submitted displayed (twice) and the proposal ID.
4. Click View button. I now see the observatory version. Purple banner with
label "OBSERVATORY VERSION". The tabs to toggle between the
Capability Request and the Observation Specification are missing.
5. Click Author button. I can now see the author version with an edit
button to the far right. Here I do see tabs to toggle between
Capability Request and Observation Specification. Later I did this for
a proposal that was in the draft state and as expected I only saw the
Author version.
6. Click List button.
7. Click Edit button. Now I see the Author version. Observatory
version is not active. I guess this makes sense since I cannot modify
the Observatory version.

- Re-Submit a proposal.

1. Click Edit button (in list view).
2. Edit abstract and save.
3. I now see the Re-Submit button is not grayed out.
4. Click List button. I can now see "Unsubmitted Changes Pending".
5. Click Edit button.
6. Click Re-Submit button. Re-Submit button is now grayed out again.
7. Click List button. Comment about "Unsubmitted Changes Pending" is gone.

- Filters. Create three proposals for different solicitations and in
different states. Tried several different combinations of filters
for state and solicitation. Seemed to work fine.

---------------------------------
User: Dana Balser_TTA |
---------------------------------

Proposals
--------------

- Withdraw Proposal.

1. Click Proposals.
2. Click Edit Button.
3. Click Withdraw Button. I now see the state is withdrawn. I can
still edit this proposal which is correct for TTA group member.
4. Click Edit Button.
5. Edit abstract. I can do this since I am a TTA Group member. This
is only reflected in the Author copy and not the Observatory copy. I
guess this makes sense. I do not think the TTA Group member would
really ever need to edit a withdrawn proposal but that is in the
requirements.

----------------------------
User: Dana Balser |
---------------------------

Proposals
---------------

- Withdrawn Proposal. I can now see that the proposal above has been
withdrawn. No Edit button which makes sense.

- List View. It would be helpful to align the different fields that
are listed: ID/State; Title/Solicitation; Submitted; PI; View/Edit
Buttons (see image).

- Draft ID. We need some proposal ID for Draft proposals just so we
can reference the proposal when problems arise (e.g., helpdesk
ticket during an open Solicitation---maybe out of scope here).

Dana (Firefox, Fedora)

-------------------
Dana Balser |
-------------------

Proposals
--------------

- I created a new proposal and filled in the Basic Information and created one Allocation Request. I specifically did not import a Scientific Justification. While viewing the Observation Specification I clicked the Submit button. I got an error (validation). I then imported a scientific justification PDF and while on the Scientific Justification page clicked the Submit button. Seemed to work. I can see the following at the top of the page. Date appears to be the correct UT.

"SEM25A-004 Test submit 1 ..........................."
"Sem_25A 11/28/22, 3:18 PM Submitted"

- I changed the abstract and then clicked the Save button. This took a few seconds to save but seemed to work. I then clicked the resbumit button. Seemed to work. I can see the following at the top of the page. Date appears to have been updated and the proposal ID is the same.

"SEM25A-004 Test submit 1 ..........................."
"Sem_25A 11/28/22, 3:26 PM Submitted"

- I changed the Calibration Parameters for the VLA Capability and then clicked the Save button. After this I clicked the Resubmit button and this seemed to work.

- I am able to resubmit even if there have been no changes to the proposal. I thought we agreed to gray out the resubmit button until there were changes but maybe this was not included for Sprint 41.

Allie(Firefox, linux)

Specifically testing Submission/Resubmission

proposal already exists (QA sprint 40 proposal 2 ddt)
Submitted from the Observation Specification tab.. On submit, the observations specification disappeared and instead the message of ``No Observation Specifications..'' is displayed. going to Capability Request and back to OS fixes this. Or a refresh fixes this.
Proposal State changed to submitted and a date and time is displayed now. It displays UTC until a refresh, when it changes to local time. (Known)
Changed the abstract, saved, and resubmitted. I believe that it worked because the date time changed. A warning popped up warning no history is kept - good.
On proposals' tab, I see the state change and the updated datetime. I can filter on state.
Changing the basic information, no save, resubmit → change not persisted. (Known)
Changing capability request, no save, can resubmit → change not persisted. I would not be able to make a change to CR, and submit with an invalid OS because it would, presumably submit the one before from before the unsaved changes. It would be annoying for a user though to make all of the changes and then have them blown away on a resubmit because they did not save.
Resubmitting on the CR editor briefly displays the Changes Pending banner and then reverts. I did not make any changes though beyond just resubmitted.
There's no system message confirming the resubmission. (Add story)
No validation still - I submitted an empty CR (not implemented)
We apparently did not write the story to put the proposal ID into the UI. (Add story)
I thought that the Resubmit button is available only if the Author Copy is out of sync with the Observatory Copy. A Save is required to make that check, I believe. The resubmit button seems to always be available, as long as the solicitation is open. (Add Story?)

Thoughts on save/resubmit

Use case 1
- I have a valid and submitted proposal. I change a CR detail. I would only like to submit a valid OS, so one must be generated.
  - Currently, the only way to generate an OS is with the save button.
  - If a Resubmit and Save happens together with the Resubmit button, then I would not have the chance to review the OS before submission. If I made a change that invalidated the OS, then the resubmit would throw back an error saying that validation failed but maybe it is a valid OS that asks for 100 hours when really I only need 1 hr because e.g., I flubbed the units on the RMS!
- Opinion - the save must be separate from the resubmit and the resubmit should be visible but grayed out until a save is made.
Use case 2
- I have a valid and submitted proposal. I make a change to the CR, do not save, and go to the Basic Information to change the title. I am greeted with a pop up window stating I will lose my changes to the CR if I go away. I now know I need to Save the CR - thus generating a new OS (or getting an error message ). I have the chance to review the contents of the OSs.
- Now when I go to the Basic Information and make a change, Saving before a Resubmit is an extra step. The OS contents are known to me (in theory) so I cannot mistakenly be asking for way too much time.
Opinion - the resubmit or submit button should only be available on the Basic Information, Science Justification, and Observation Specifications editors. It should not be available on the Capability Request editor. Or, it should be moved higher than all of those and system messages could indicate that a submission or resubmit is available.

Dana (Firefox, Fedora)

User: Dana Balser

Proposals
---------------

- Create Proposal. I had problems creating a proposal. Basically I
was tying to create several proposals quickly (without much detail)
to fill in the summary list of proposals. Here is the procedure:

1. Click Add button under Proposals.
2. Enter Title.
3. Select Solicitation.
4. Click Save (so far so good).
5. Click Proposals (go to summary list).
6. Click Add button under Proposals (create another proposal).
7. I see the form to create a new proposal (title/solicitation) but
the previous proposal is listed; that is, the form is not cleared.
The system hangs (get the spinning cursor).
8. Need to click Home to get out.

- Filtering. This does not work as expected on the summary list view.

State Filter: I can filter on All, Draft or Submitted okay but not
the other fields (e.g., Withdrawn). Maybe the other fields are not
yet supported.

Solicitation Filter: At first this appears not to work at all but maybe
the internet is slow to respond. Selecting "20D" does not work, but maybe
because this solicitation is not included.

Combination Filter: Sometimes combinations do not work. For
example, selecting State="Submitted" and Solicitation="SC_GBT_24B"
list a submitted proposal (good) that is for the Sem_25A
solicitation (not good). See attached screen shot.

Proposals-->Allocation Requests
---------------------------------------------

- Navigation. I created two Allocation Requests (ARs), each with one
Capability Request (CR). I saved them generating Observations
Specifications (OSs). I wanted to compare the OS for each AR. To
do this is a bit of work (see below). This is very logical but time
consuming. Will folks want to compare OSs between ARs? I did only
because I wanted to compare the integration times that were
calculated.

1. Click List button under AR.
2. Click AR1.
3. Click OS.
4. To see the OS for AR2 I have to:
a. Click List button under AR.
b. Click AR2.
c. Click OS.

Proposals-->Allocation Requests-->Capability Request
---------------------------------------------------------------------------

- Ordering. Normally the ordering of the CR is: Field Sources,
Spectral Specifications, Performance Parameters, Calibration
Parameters, Advanced. I performed the following operation:

1. Deleted one source in Field Sources.
2. Click Home (leave Proposals).
3. Dialog box asking if I want to leave: yes (should not save edits).
4. Click Proposals.
5. Click Proposal in question.
6. Click AR in question.
7. Click CR in question. I now see that the ordering of the CR
is: Spectral Specifications, Performance Parameters, Calibration
Parameters, Field Sources, Advanced. (See attached screen shot.)

- Deleting. If I create a single field source (single spectral spec) I
cannot delete it. I have to create another field source (spectral
spec) before I see the menu with the delete option.

Proposals-->Allocation Requests-->Observation Specification
-------------------------------------------------------------------------------------

- Missing Sources. See "Test Proposal E". For one AR I created two
CRs. The first CR "GBT Spectral Line C-band" has one field source
and one spectral spec. After this is saved the Observation
Specification is created and looks okay. I then created the second
CR "GBT Spectral Line W43 L-band" which also has one field source
and one spectral spec. After this is saved the Observation
Specification does not include the new source, etc. I thought this
worked before. N.B., I did not (ever) see the popup message
indicating the Observation Specification was created which was in
v0.1.

Allie (Firefox,redhat)

Home page

+Proposals gives error but does take me to the proposal page. I screen capped it.
the "view" button on the proposals also takes me to the proposal page

Proposal Page

New "List" and "Add" Button left side
Filtering on State doesn't work
Filtering on solicitation works.
Created a Proposal with a title and solicitation specified, went back to list and can see it in the List mode.
Added another Proposal. It pre-populated with the same title and semester selection as the first one instead of being blank. Save button isn't available. The form is hanging. On refresh, the Proposal is gone, which is expected as it didn't save. I added a new one again and it now has a blank Title and Solicitation, which is desired. Not sure why the first time it got hung up.

Proposal Creation (Basic Information, Sci Just)

When first creating a proposal, it just wants the Solicitation and a Title.
Once the formed is saved, it will expand to Abstract entry and author widget. The Sci Just and Allocation Request Tabs are also now available and actually look like tabs. The layout is nice so far.
If I enter in an abstract but do not save it and try to go to the Sci Just tab, a warning appears prompting me to confirm to discard my unsaved changes. nice.
I can upload a pdf and delete it. I do see the preview and "view" of the pdf, as I am on linux with firefox.
When I uploaded a different pdf after deleting the first, I no longer have all the buttons. The system Messages say it was uploaded, but the interface looks like it is not uploaded. Refreshing solves the issue and I can see my pdf. I can upload a new pdf as well and it seems to work fine.
When creating a new proposal that hasn't been saved, the only way to "back out" of the process is to hit the Home tab. Selecting the Proposal tab does nothing.

Allocation Requests

Entering the AR tab for the first time appropriately says "No ARs". The add button is the same as before. It does not give the List button because there aren't any ARs I suppose.
The List Proposal Button IS available however, as well as the add proposal button. That may trip someone up at some point.
Adding an AR shows a view with just the Facility selector - as usual.
Once the Capability Request panel and add button are available, the Capability Request vs Observation Specification views are also available. They are the familiar square tabs on the right of the screen.
I like that the Basic Information, Sci Just, AR are clearly tabs and the CR vs OS is a different style. It makes more sense to me because we are toggling to the OS view inside the AR tab. While the CR has tabs as well.
For the DDT solicitation, the GBT Spectral Line capability is the only one available, as expected.
For the Semester solicitation, VLA and GBT facilities are available, as expected.
I can rename the Allocation Request
I can delete ARs

Capability Requests (DDT, GBT)

I can make a crazy long name for a field source. Doing so will cause the tabs on the left with the different field sources to expanded until finally it wraps at the page edge. As it expands, it forces the field source, spectral spec (Etc) tabs to compress to the right. At the line wrap, it forces those tabs down a row and any newly added field source create those tabs above the form entry. I screen capped this.
Field Source, SS, CP, PP do not appear to have different behavior. I can navigate between them without hitting save and I do not lose information.
Saving gives the familiar prompt. Now the CR view tab has a "1" next to it immediately, indicating that there is 1 CR. In v0.1, it would show 0 even though there were 1+ CRs. I wasn't ever sure how to trigger the change. The OS doesn't now have a number next to it. Not sure if it should or not.
Saving make the tab view switch to the first tab, regardless of where I hit the save button at. In this case, it is field source. However, it has been noted that the tab order changes and the trigger is not known.
Go to (A) in Observation Specifications
I can delete CRs
If I am in the capability request and I select "Add AR", it does add it but it leaves me at the page I am on, which might make someone think it hasn't been added. You have to navigate to the list of ARs to see that it has been added. This is not the case when I add a CR: it navigates to the empty CR right away. Same with Proposals.

Observation Specifications (DDT, GBT)

(A) A 2 FS, 1 spectral spec CR has generated 2 Observation specifications. The GBT has partitioning in RA. I put Hen 2-10 in for one source in ICRS and Hen 2-10 in Galactic Coordinates for another, so there shouldn't be two OS and really only 1 target source, as they are identical sources except for the coordinate system. The system doesn't handle this yet though.
The time on source is not correct. I cannot update it either through the Performance Parameters nor the Advanced Tab. — It requires a refresh to see the updated OS.
Filtering on field source works mostly. It doesn't work on the 0th subscan target.
Filtering by band doesn't work
Filtering by intent doesn't work
The long name on the field source causes the filtering to completely push the scan list down beneath the filter panel instead of just wrapping the field source name.
The Scans tab has a nice numerical in the tab showing how many scans are in it.
I can add a scan as previously. There's a save button when I do that and the Status changes from System Generated to User Generated. Nice
Adding a band and field source is the same.

Capability Requests (Semester)

This all looks familiar
As soon as I save an Advanced option, the Performance Parameter no longer works at all.
Navigating to the list of CRs or otherwise prompts the warning about discarding unsaved changes.

Observation Specifications (Semester)

If I save, I get the warning about generating a new OS
When I go to the OS tab, it almost always prompts me again saying there are unsaved changes. But there aren't unsaved changes.
Filtering doesn't work at all here? Had to refresh and now it works just fine, even band and intent.
Submitting generates an error. The button is available though

Dana (Firefox, fedora)

-------------------------
Dana Balser_TTA |
-------------------------

Home
--------

- The solicitation are both open and recently closed. This makes no
sense. Maybe this is just an example of what the TTA member will
see.

Solicitations
------------------

- Looks good. Sem_25A has both VLA and GBT Facilities and all bands
(L-Q), whereas SC_GBT_24B has only the GBT and L,S,C bands.

-------------------
Dana Balser |
-------------------

Home
---------

- I only see the open solicitations.

Proposals
--------------

- A bit slow going from Home to Proposals.

Proposals-->Basic Information
------------------------------------------

- Looks good.

Proposals-->Scientific Justification
-----------------------------------------------

- This looks okay. Might alter text (e.g., replace "5" with "x" since
this will depend on the Proposal Class).

"Each proposal should include a Scientific Justification as a pdf file.

The Scientific Justification should outline the scientific merit
expected if this proposal is awarded time on it's requested facility.

The pdf file should be no more than 5 pages."

- Functionality appears good. Able to upload/view/delete pdf as
expected.

Proposals-->Allocation Requests
---------------------------------------------

- This looks okay. Might alter text.

"Create at least one Allocation Request for each facility you would
like to use.

Time is awarded on a facility per Allocation Request. Awarded time on
a Proposal may only be applied to some of it's Allocation Requests.

If there are multiple acceptable configurations that can meet your
science goals (e.g. minimum required configuration, ideal
configuration), create an Allocation Request for each one to increase
the chances of getting awarded time."

Proposals-->Allocation Requests-->Capability Request
--------------------------------------------------------------------------

o Proposal: Probing Molecular Cloud-HII Region Dynamics using the OH
Satellite Line Flip (Sem_25A)

- This proposal was a very simple setup. Just edited everything by
hand. One Capability Request (VLA continuum) that included just one
field source and one spectral spec. Edited everything first and
then hit the save button. Pop-up message was nice to let me know
that the observation specification was created.

o Proposal: Multi-Configuration, Multi-Band VLA Proposal (Sem_25A)

- This proposal is more complex. Two ARs (e.g., different array
configs for TAC). Each AR had the same setup of two capabilities
(different bands: X, Ka) and fields sources that should be divided
into two groups on the sky.

- Noticed that the spectral spec for VLA continuum has bandwidth and
spectral resolution. This is a bit confusing since this is not
spectral line. Presumably the time exposure calculation uses
bandwidth since this is continuum?

- When manually entering 40 (in arcsec) for the LAS in the spectral
spec the following was displayed: 39.999999599999995. If I leave
the page and come back the units are degrees. I think this is
expected since we agreed for v0.1 that we would show the precision
necessary for GMVA at W-band and angles are stored in degrees.

- Explored the Advanced tab a bit. Interface seems reasonable. I can
edit any single field source/performance parameter or there is a
utility to update the performance parameter for all field sources.
I looked at the input file format but did not experiment.

o Probing the Warm Ionized Medium toward the Inner Galaxy (SC_GBT_24B)

- This proposal is for the special GBT call. I only see the GBT
listed as a facility as expected.

- When I enter "-11:29:18.9808" for the one field source Dec it
displays -12° 30' 41.01920". This was repeatable. But when I
entered in other numbers for other field sources it seemed to work
fine. Odd.

Proposals-->Allocation Requests-->Observation Specification
-------------------------------------------------------------------------------------

o Proposal: Probing Molecular Cloud-HII Region Dynamics using the OH
Satellite Line Flip (Sem_25A)

- I specified NO polarization calibration yet there exists a
Calibrate_Pol_Angle scan. This appears to be a bug.

- The integration times appear to be fixed. Changing the rms
sensitivity does not change the time on source. I recall that we
set these to a "reasonable" fixed number when the calculated time
was really small. Indeed this appears to be the case. If I reduce
the rms sensitivity I can see the time change.

o Proposal: Multi-Configuration, Multi-Band VLA Proposal (Sem_25A)

- Okay, for each AR I get four different Observation
Specifications. They are organized into two source groups for each
band. This is reasonable.

- Integration times are not quite what I expect. If I use a frequency
of 9 GHz, a bandwidth of 4 GHz, and rms sensitivity of 10 microJy
then I get an time of 2m40s. From the ECT I get 4m27s. Factor of
1.7 more time on science target. Probably okay for v0.1. The time
does integrate as expected.

- The science target scans tend to have rather small integration times
(e.g., 3 minutes). So if the total time needed on science target is
large (e.g., few hours) there are many scans and thus the overhead
is also much larger than needed. I recall, as noted above, that we
fixed this under certain circumstances but not sure why that is the
case here.

- Explored the effects of changing the sensitivity for one field
source in the advanced tab of the Capability request. This does
change the time on source in the Observation Specification but it
changes all of the field sources instead of the single one that I
altered. This seems to be a bug.

- Explored changing the Observations Specification. Not particularly
intuitive but will probably server the purposes of V0.1.

o Probing the Warm Ionized Medium toward the Inner Galaxy (SC_GBT_24B)

- Not convinced the integration times are correct for the GBT. I have
entered an rms sensitivity of 0.001 Jy/beam (1 mJy), spectral
resolution of 1 km/s, and a frequency of 6 GHz. I estimate about 3
hours on science target from the Python script I wrote, but get 0.1
sec from the tool. My guess is that the bandwidth is being used to
calculate the time instead of the spectral resolution, which is what
I would expect for spectral line. The bandwidth is 100000 km/s
which yields about 0.1 sec, consistent with the tool. So this is
the issue.

Allie - redhat, linux

Copy Edit Changes:
- Home Page, when logged in as TTA member - Recently closed solictations mirror open solicitations currently
  - Fake text or Add a solicitation that is closed to show the purpose of the region but isn't mirroring the open soliciations
- Scientific Justification - yellow instruction box should read:
  - This region can be customized to contain instructions for the proposers. An example dialog is ".pdf only; font size no less than 11pt; no more than 4 pages (including figures, tables, and references). Maximum file size: 16 MB
    The science review panel members are instructed to reduce the score of any proposal that does not abide by these constraints."
- New Allocation Request- yellow instruction box should read:
  - This region can be customized to contain instructions for the proposers. An example dialog is "A proposal requesting resources from multiple facilities requires at least one Allocation Request per Facility. See the online guide for suggestions and guidelines on how multiple Allocation Requests per Facility can convey your top priority concerning the requested resources."
Proposals Tab
- When logged in as TTA member, I do not have "My" proposals. I understand the profiles are not linked. Noting to remember to put into document to not expect that behavior.

Allie QA v0.1.1 on prod

STT-1001 updated how time is distributed in the scan list because there is no concept yet of a repeat count on the OS.
- For the GBT, all of the Requested Time is held in 1 scan/subscan. The default setup time is 6 s. This is the general behavior I see.
  - When I put in a really large BW and large RMS, the displayed time in the scan list and the Time per Science Target Tab show 0s. This is because the real time is quite small I believe. This should either default to a minimum of some value (TBD) or display the time, even if it is small
  - Has the Calibrate Pointing and Calibrate Focus Target always been called "0th Subscan Target?". The source should the same as the first Science Target (STT-612)
- For the VLA, we are showcasing a simple interleaving of the phase calibrator. However, we have limited the number of scans generated per Science Target because the scan list was getting unwieldily without the repeat count concept.
  - The behavior is
    - If the Requested Time is less than 90s, all of the time is held in a single scan/subscan.
    - If the Requested Time is > 90s, then three scans are created and the time is split evenly between them.
  - Similar to the GBT case, if the time is very small, the acquisition time on a subscan shows as 0.
  - The default setup time appears to be 1m 40s - not sure why it is different than the GBT case.
  - The Requested Time seems to match the sensitivity calculators, though the GBT is using bandwidth instead of the spectral res to calculate time.
  - The behavior is not as I expect:
    - Either the setup time is being factored into the decision to create 1 or 3 scans or three scans are always created regardless of the requested time.
    - For example, a source has a requested time of < 90s, so I would expect 1 scan in the Observation Spec. However, 3 scans are made instead, each with a very small amount of time and 1m40s of setup time. I have only tried a few iterations but I'm having trouble getting just 1 scan to be created instead of three, so something is up.

Allie (Firefox, redhat)

Note! 20D Solicitation only has specific hardware for the facilities.

VLA has Ka- and L-band. GBT has P- and K-band.

Landing page
- Can edit a message of the day as a ttamember and save it. The save feature doesn't give useful feedback: it would be nice if a system message were displayed and the button changed somehow to reflect that a successful save had occurred
  - Logging in as a different non ttamember does show the message I wanted to save.
For the iteration that goes in front of the TTA group, it would be good to remove the notification on the review tab, since that doesn't go anywhere right now.
If I log out from a profile, it doesn't redirect anywhere. It should probably redirect to the landing page. It just sat and tried to load the proposals, which is the page I was at when I hit the log out button.
Solicitation Tab should not be available for non TTAmember or it should be different as non ttamembers are not modifying the solicitation.
Proposal with a realistic looking solicitation
- Basic Information works as expected.
- Scientific Justification
  - Tried to upload a 3.1 MB pdf. Got error messages of " A backend error occurred. Request Entity too large at api/proposal/6/sciencetific_justification" and "http failure response for https://tta-test.nrao.edu/api/proposals/6/scientific_justification:413 request entity too large" but then I also got the message that says "file uploaded" and I can see the file in the scientific justification view. If I exit the proposal and load it up back, the Scientific justification does not persist.
  - The view button works
  - The delete button does not actually delete it. It also gives an error of "http failure response for https://tta-test.nrao.edu/api/proposals/6/scientific_justification: 401 unauthorized"
  - Replace the file does work but I get the same errors as above. this file is 2.2 MB in size.
  - I uploaded a png just to see if it would allow me to do so. It does allow a png to be uploaded.
- Allocation Request
  - Selected GBT and GBT Spectral Line; reloaded after creating the allocation request to try to avoid the error discussed on Tuesday.
    - Maybe I didn't hit it in the right order, as I was not able to save the Field Source I created. I had to refresh first and then I could reenter the field source information and save.
  - I could not delete the only Capability Request in the Allocation Request.
    - I got the error "Could not remove Request from Proposal"
  - Field Source
    - Mostly works as expected except there are red boxes, indicating problems, around negative numbers like for radial velocity, declination.
    - The hexadecimal view of the RA gives the minor error of "only enter numbers" when the hour, minutes, seconds are used to differentiate the inputs. It saves despite this error though.
    - The precision on the conversion looks off.
    - RA should only have options of Degree & HMS
    - Dec should only have options of Degree & DMS
    - Parallax should only have degree, arcmin, arcsec
    - The units on the Flux Density fields are should not be per beam if they are actually meant to be flux densities and not intensities.
      - It would be nice to other Jy as an option for input to these fields too
    - FOV Shape
      - It's really nice that the field for FOV in RA/ FOV in Dec will change with the coordinate system.
      - The units here should only have degree, arcmin, arcsec and we need to be clear if we really do mean arcmin and not minutes, even for the RA field.
    - Proper Motion
      - The proper motion titles also change with the coordinate system. We need to be very clear about this I think or restrict this ability because proper motion in Galactic is not equal to that in RA. There's a transformation between them I do not think we want to be responsible for it. Did we decide if we were responsible for coordinate transformations in general? I think ALMA puts that on the observe and I recall that's what we decided.
    - I don't think it is useful to have the up and down arrows to change the numbers for any fields here because the precision on that function is an issue and will change between fields. Plus, right now you can adjust it to be a negative number for fields like FOV, which should never be allowed to be negative.
    - There should be a better error message if a negative number has been entered into a field that cannot be negative instead of the current error, which says "only enter numbers".
    - It would be nice if there was a mass select or delete on the field sources.
    - Importing field sources somewhat works
      - The import widget is nice and does a good job at assigning the columns
      - The RA and DEC entries were in hexadecimal format and did not propagate to the fields properly at import.
      - The Proper Motion, parallax values in the file are 0 and the widget shows no values for them, so it is unsurprising that there are no values in the Field Source entries. However, 0 is a valid entry.
      - I appreciate that on a new CR, there isn't a "dummy" field source that is created right away and I need to delete, particularly after the import. That being said, I would think I would be able to delete all the Field Sources to start anew without having to delete the CR.
      - The units on the import file are not matched to the fields. I imported in Jy and of course, the field for continuum flux density is microJy. This is also a problem in other fields.
      - Importing Galactic Coordinates seem to work correctly.
      - Import of different field of views work
      - Uncertainty in Ra and Dec fields should be removed from the import list
      - I suspect coordinate epoch should also be removed from the list.
      - Importing seems to handle mixed coordinate systems well.
  - Spectral Specifications
    - It would be nice if it were possible to enter in center frequencies with units of MHz too; likewise, the bandwidth in and resolution could be entered in as Hz, MHz, GHz
    - These fields are highlighted in red when a negative number is entered, which is good as they aren't allowed to be negative. It would be helpful if a message were also displayed or the bang that appears with the red highlighting had a hover over tool tip to explain the error.
      - Despite having negative numbers here, I can still save. That isn't a good behavior.
    - Importing a spectral spec works as expected.
  - Performance
    - Performance should really be Performance Parameters, which is what it is called in the documentation and consistent with how we call Calibration Parameters.
    - The Units on Angular Resolution should be deg, arcmin, arcsec
    - It would be nice if the rms sensitivity also had Jy/beam or mJy/beam options
  - Advanced
    - There should be units on Angular Resolution and RMS Sensitivity here
    - Eventually it would be nice to have a tool tip explaining what the eye is, as it may not be clear right away. Also, it isn't centered under the word observe.
    - The filters aren't filtering the list view; partially works for the 20D solicitation
      - What do the question marks in the filter view mean?
        I think it is trying to fill the position but cannot do so here. In the OS, it has the position of the source in this area of the widget.
      - the filters only work if i select a FS and SS, so I can't just select all of a SS or all of the FS.
    - After modifying in the Advanced view, I can save.
  - First pass at generating OS (Sam says realistic looking proposal is broken)
    - with a seemingly acceptable proposal gave an error of "could not find of generate observation specification"
    - STL is not available. has system error of "a backend error occurred. internal server error at api/science_target_list/5". I tried refreshing and still was not able to view it or the OS
  - Science Target List
    - Should only be viewable to a TTA member
  - Future testing thoughts:
    - Future question for testing: if the Obs Strategy is going to try to merge FS+SS that are similar, what happens when two entries with the same coordinates have other attributes that are different like FOV, peak Flux Density, line width?
  - General:
    - I caused the system to hang after uploading a spectral spec and filling in performance parameters. The FS entries weren't all correct so I refreshed and tried again. I fixed the field sources entries, didn't hit save, went to SS and saw the system was hanging again. When I went back to FS, it hadn't persisted the parallax field but did persist the proper motion fields.
      - Despite refreshing, the system still continued to hang on a save.
      - I deleted the allocation request and begin a new. I clicked to add a CR and got the system error "a backend error occurred. internal server error at api/capability_requests" then quickly followed by "Could not create a new Allocation Request"
      - This proposal seems to be stale now and I cannot add allocation requests to it anymore. Even when I back out and come back to the proposal, it will not let me add AR.
      - I was uploading and deleting a lot of Field Sources before this happened. Maybe it didn't like that?
      - I made a new proposal and still cannot seem to make AR. I can upload Sci Just though. I had to logout of the user and log in again.
    - At some point when I was editing FS, the CR had a banner that said Changes Pending, which I expected. But then the Allocation Request also got a banner with Changes Pending that didn't go away until I hit the save button associated with the Allocation Request. Typically, I only see a banner for the CR when I am editing FS/SS/etc and not one for the Allocation Request. I have no idea how I triggered it.
    - I am not convinced that Proper Motion and Parallax fields are persisting properly. There is no predictable behavior for when the fields are turned to null but it happens frequently, despite not being given values of 0.
GBT proposal with a 20D solicitation
- Sci Justification has similar errors
- Allocation Requests
  - So far all issues I found in the realistic-looking solicitation apply here
  - Calibration parameters
    - Polarization Calibration is missing its button. The text is still here but there is no button.
  - I still cannot generate an OS or STL – it is because I was requesting SS that aren't available to the Solicitation. They did say this, I just didn't intake it apparently.
VLA proposal with 20D Solicitation
- Sci Justification has similar errors
- Allocation Requests
  - Field Source
    - - Importing Sources
    - Radial Velocity was imported NaN when I mistakenly assigned the wrong column to it. It tried to give is a string instead of a number. So that rather works intuitively.
  - Spectral Spec
    - Import works
      - Units on spectral resolution and bandwidth should be defaulted to MHz or at least the option of changing to that. Will we be responsible for the transformation? I think that's simple enough.
  - Performance Parameters
    - There is not field entry for Angular Resolution and there should be.
  - Calibration Parameters
    - Polarization Calibration is missing its button here too.
  - Advanced
    - Angular Resolution doesn't show up as an option to be changed in this view, nor is there a display for it.
    - The upload here works as I expect. Though I added 2 "ringer" sources and it did not give the prompt that it couldn't match them to anything. It did bring that up for two sources that I had deleted.
  - Observation Specification
    - If I request the right bands (Ka-, L-band) then this works
    - Filters
      - Filtering doesn't work properly in all the tabs. I think it works correctly in the Science Targets and Observing Target tab but not in the scan tab.
      - I think Target should be changed to Source in the filter
      - The "Bands" filter reflect the Hardware Configurations the system decided; should the name be the system's name for the band or how the user's SS was mapped to the band?
        Otherwise, there is no inclusion of the user's name for the SS anywhere in the OS.
      - Instead of, or in addition to, the position in the filter, it might be more useful to have the pointing pattern that the system is going to use. For example, if the user enters a FS in the CR that is clearly meant to be a mosaic, that information would be more useful than just the position. That would align with our definition of a Source, which has a name, a nominal position, pointing pattern.
      - The positions attached to the filter cards need attention in their precision. Also, they directly reflect the FS position so they are a mix of coordinate systems but it doesn't actually specify it. They should also have units.
    - It created 2 OBs split in frequency. I didn't think it could, or should, do that yet.
    - Total Time on Observing Targets is reporting NaN s.
    - Adding to the OS
      - I can add a scan and assign the new FS to it. This very different but i think I get it and it works as expected
        The exception in the setup time. The user would not specify this because it depends on the slew. We don't want live updates on this though I think. Instead there should be a way to validate to get the updated slew times. Maybe as soon as an edit occurs, any time estimate except the acquisition time and the total time on science targets should be voided until the system is called? Is that a behavior we're going to have is the first question I suppose.
      - The STL doesn't reflect the addition of the ringer, which is consistent with the behavior I expect. The STL is not to be regenerated from the OS.
      - My added target did not persist in the OS when I moved away from the OS.
      - I cannot add a SS
      - Once I modify the OS, it should change the status on the OS from system generated. It should be more evident that this is no longer a purely system generated OS.
        Toggling off the OS page should also warn me that the CR is out of sync now with the OS.
        The manual edits to the OS should persist.
        There should be a warning in the CR if they are out of sync that I will rewrite the OS.
        There should be a button that clearly indicates I am generating an OS or something. Otherwise, how do I force a new OS to be generated from the CR? It cannot be just toggling back and forth between the OS and CR tabs, as I may want to do that without overwriting the edits I made to the OS.
    - The coordinate transformations are not correct in the OS: the conversion between Equatorial Coordinate System (RA,DEC) and Galactic coordinate is not correct for example. The RA coordinates of the calibrators do not match the JIRA ticket. Something went wrong there.

Dana (Firefox, fedora)

User: Jane Doe. Solicitation: 20D. Proposal: Direct Measurement of the Expansion of UCHII Regions.

Allocation Request 1 (Facility Real VLA)
- We should probably change Facility "Real VLA" to "VLA"
- Capability Request
  - We should probably change "Real VLA Continuum" to "VLA Continuum".
  - Field Sources: Keyword names have changed but that is okay. Epoch can be input via file but is not displayed.
  - Spectral Specifications: Looks okay.
  - Performance: No textbox is present to enter data for Angular Resolution.
  - Calibration Parameters: No yes/no switch for Polarization Calibration.
  - When I click "Save" I get the following message: A backend error occurred. 'Not Found' at 'api/capability_requests/64. I get spinning icon. Have to go to Home and then back to Proposals. Started from scratch and was able to save the Capability Request. Still had same issues above. Created another Capability Request and had to go through the same steps.
  - Tried to change the Capability Request name. Ran into the bug Reid highlighted at the Sprint Review. Go to home and back and you can see the new name.
- Observation Specification.
  - Two Observation Specifications are created. One for each Capability Request. Looks okay.
Allocation Request 2 (Facility GBT)
- Capability Request
  - Field Sources: okay. Now we see "Peak Line Flux Density" and "Line Width" in the file import.
  - Spectral Specifications: Looks okay.
  - Calibration Parameters: No yes/no switch for Polarization Calibration.
  - Performance: okay.
  - Note that the tab order is different between the VLA and GBT (Calibration Parameters and Performance are swapped).
- Observation Specification.
  - One Observation Specification is created. Looks okay. The calibration is done at Ka-band and not K-band.

Dana (Firefox, fedora)

General
- The message of the day feature seemed to work as advertised. Overall, I did not see any significant lags. I then created a new proposal as SallyScientist called "The WISE Extension of the HRDS".
Basic Information.
- Seemed to work okay.
Scientific Justification
- When importing the sj PDF (2.8 MB) I get the following error message: "A backend error occurred. 'Request Entity Too Large' at 'api/proposals/37/scientific_justification". I was able to import a smaller sj PDF (0.445 MB).
Allocation Request
- Capability Request
  - Setup 200 field sources and 1 spectral spec. When saving I got the following error message: "A backend error occurred. 'Request Entity Too Large' at 'api/allocation_requests/68". When I reduced this to 50 field sources the Allocation Request was saved.
- Observation Specification
  - In general this looked okay. I could see the hardware and OI names, etc. Need to decide better names for all of this but okay. But I noticed that there were only 51 targets. I expected the 50 science targets plus 3C286 and test A for a total of 52. Field source 12 was missing (HII0012). Closer inspection revealed that since I did not have sufficient precision in my input file for the RA/Dec that field source HII0011 and HII0012 had the same coordinates. A bit surprised that the software would recognize this fact and decide not to observe HII0012. Probably best to not do this but warn the user that two field sources have the same coordinates.

Allie (Firefox,redhat)

General
- Toggling between basic information and allocation request has a lag of about 1~2 seconds before the information is shown. It almost looks like a blank proposal at first before it loads. I have 50 FS and 2 SS and I generated the OS before trying to toggle between them.
- (Feature) there should be a warning when switching between these views if the input data isn't saved.
- Message of the day is editable, saving, and persisting. There isn't an indication that it is saved or edited though when a ttamember.
Allocation Request
- Capability Request
  - Import of FS and SS mostly works as expected except when the value is 0, then nothing is populated into the FS.
  - (Feature) there should be units attached to the TOM inputs.
  - (Feature) it would be nice to have an easier way to delete FSs en mass.
- STL
  - The name of the SS is not propagating to the STL. Instead only the mapped Hardware Configuration name is shown.
- Observation Specification
  - Second Dana's comment.
  - The name of the SS isn't propagating to here either.

Dana (Firefox, fedora)

Allocation Requests
- Overall the layout is better organized and therefore easier to navigate.
Capability Requests
- Some of the functionality here is a bit different than for Allocation Requests. For example, (1) the way to list the different Allocation Requests or Capability Requests; and (2) the functionality to rename the Allocation Requests and Capability Requests. I prefer consistent functionality but understand there are trade-offs.
Target Overrides
- I think the functionality of this tab should be incorporated into the Capability Request tab; effectively we are fine tuning the Capability Request. Need to make the flow more clear; e.g., user has to intuitively know that they have to mash the Generate Observation Specification button.
- The labels here are numerical (1, 2, etc.) but under Capability Request they have names so the mapping is not clear.
Observations Specifications
- Not sure why the calibration scans are at Ka-band, whereas the science targets are at C-band and L-band. Maybe legacy stuff from the VLA?
- Not clear to me we want the scan and subscan intents to be so prominent in the display. I tend to want to locate the source name and found this a bit hidden at first. Need to think about this since the intents are important and I may just be biased here from past experience.
- The subscan intents are a bit confusing to me but are implemented as described (STT-441). Need to discuss this with the product owners.
- Not clear we should include the test source as a science target (e.g., as displayed under Total Time on Science Targets). But again, this is implemented as described (STT-441).
- I suspect we will eventually want the observing procedure, what I think we are calling the pointing pattern, displayed. But we have not gotten to this yet. Anyhow, just something to think about vis-a-vis the real-estate as the displays are getting crowded.
- It would be useful to be able to collapse each Observation Specification; again similar functionality that exists for Allocation Requests and Capability Requests.

Allie (Firefox, redhat)

General
- If a CR has generated an Observation Specification and then I delete the CR, the observation specification (TOM+STL) are still available. There should also be a warning that you are deleting the CR, which will modify the OS. Generating a new CR then overwrites the previous OS, I think.
- I think 50 FS and 1 SS has caused enough of a load on the system that the navigation is noticeably slowed down.
- For science side - I think it would be good to have a tooltip that has more detail about the scans. For example, we thought the OS was wrong because a calibrate flux scan had two on_source subscans. If it were a OnOFF OI, then we would have on_source/off_source. But this is a GBT Nod OI, where both subscans are on_source. You wouldn't be able to tell that though from the scan list. To have transparency in the scan list creation, because the user doesn't see the algorithm work, we need more information on the OS. That way, the user knows where to refer to in the algorithm document.
Capability Request
- When I first open the CR, it has an empty FS and SS in the CR. When I upload a file, I keep forgetting I need to delete that first entry. Not deleting the first (and empty) entry causes an error and my inputs are wiped out.
- Is there a way for persisting the entries even if an error occurs on the save?
- I think the Calibration Parameters should have defaults. Maybe the first entry, instead of being empty could have a working example filled in?
- Field Source Parameters in the GBT capability are not persisting even after a save. I thought they should in the GBT case.
Observation Specification
- I recreated the 50 Megamaser's usecase as a CR via upload. I expected the partitioning function to partition the 50 FS into 2 OS's. It does indeed do this. It isn't obvious when multiple OS are created though, as they are down at the bottom (future task to revisit this).
- The RA for the calibrators is a decimal hour but the RA for the Science Targets is in Degrees. These should have the same units of degrees. Even worse! the same calibrator (e.g., 3C286) has decimal hours in the calibrate flux scan but decimal degrees in other scans.
- The peak scan is there but is called calibrate pointing. This is a discussion for scan intents with GBT.
- The scans numbers are showing an incorrect previous state, even when I have not moved scans around. I think I triggered it by filtering by intent. All the previous states say "1".
- Band and Target Filtering doesn't seem to work. Filtering by intent almost works but it thinks the science targets are calibrate_pointing scans and observe_target scans and vice versa.
- I believe the total overhead calculation is correct.
  - The Setup Time should eventually have the Slew Time +Hardware Overhead Configuration
- I don't quite understand why Ka band is on the calibrators when the usecase hard coded K. The uploaded file asked K and the Science Targets have K instead.
- An Undo button would be nice. Particularly when deleting scans.

Live notes from Sprint Review

Bugs:
- Numbering on sorting in Obs Spec for Scans
- Some Calibration Parameters are not propagating to ObsSpec STT-729 - Getting issue details... STATUS
- Firefox doesn't display the SciJust quite right STT-721 - Getting issue details... STATUS
VLA and GBT should have similar flows
Disabled making new Targets in the Obs Spec tab for now.
Can make subscans and filter on them.
Will create drop down menus for the subscan intents and similar items STT-725 - Getting issue details... STATUS .
Currently VLA calibration strategy service looks like the GBT calibration strategy service for now because VLA cal plan service needs updating to the level of GBT.
Cannot delete field sources/ spec spec if it is the only entry.
Saving a bad capability request no longer breaks everything. It can be updated now and you do not have to start over completely.

Allie (Firefox, redhat)

The Field source input for RA in HMS should not be h ' '' but h m s, as min != arcmin and sec != arcsec

Allie (Firefox,redhat

Expectations
- A realistic-looking solicitation can have Real VLA or GBT proposals.
- It should be possible to upload a config file for Solicitations, Fields Sources, and Spectral Specs
- There should be minimal differences between the CR→ TOM→ STL→ ObsSpec between VLA and GBT now
- Unit conversions for angles should work but coordinate conversions do not (and perhaps are never expected to)
- I should be able to deleted FS and SS in a CR without deleting the whole CR
- All Field Source parameters are on display because the superset was implemented but not the masking based on capability
- Entering in Long/Lat is finicky and under review STT-714 - Getting issue details... STATUS
Solicitation
- System is hanging when i am trying to upload the supplied json file (but with a new name)
Capability Requests (JaneDoe Sprint30 Proposal1)
- VLA Continuum
  - Something made the system hang after I entered in one field source and hit save Allocation Request. I had yet to enter in Spectral Spec, so maybe that's the problem? I would like some type of confirmation that it's saving my progress regularly or on demand so I don't enter 100 field sources and then lose it all.
  - All but the coordinates and name are being cleared from the FS entry. The coordinate system is also being reset back to ICRS it seems.
  - Sometimes the TOM is automatically adding back a source that I am trying to uncheck the Observe button for. Sometimes the TOM won't let me add a source back in. The system is hanging somewhere.
  - If the input is ICRS, degrees
    - The TOM is displaying in DMS for both long/lat coordinates
    - The STL is displaying hourangle (Decimal)/degree
    - The OS is displaying HMS/DMS
    - The conversions appear to be correct
    - Declination doesn't like a negative number
    - RA shouldn't allow a negative number.
  - Editing the Sources in the OS
    - Only the name is propagated to the fields in this edit widget
    - Editing the fields seems to collapse the field entry such that I can't see what I put in
    - Whatever format I enter in the edit menu for the coordinates of the field source, the OS directly reflects that i.e., there are now dissimilar display units in the OS.
  - The filtering appears to work in the OS as expected
  - The movement of the scans appears to work (with mouse, with select and move button
  - The ability to add a scan works until I try to edit it and save the edits.
  - Editing the OS directly causes a backend error occurred. ;internal server error' at 'api/observation_specifications'. System seems to hang when I try to save an edited OS
  - I can delete field sources and spectral specs, though the interaction point is completely different than how to delete a CR/allocation request
  - The upload of FS and SS appears to work
    - FS clears all fields when after save except for name and coordinates.
    - If the first FS or SS is empty, it does not override it and leaves an empty FS/SS in the list.
  - STL doesn't always generate properly from TOM. The OS seems to reflect the STL.
  - If duplicate SS are entered/uploaded to a CR, the TOM display duplicate entries. It's hard to tell if the STL would also do this because it doesn't always generate properly anyway.
- GBT SL
  - The FS and SS entries are not deleted when I navigate around after saving, as they are in the VLA Continuum. Why is this behavior so different between the two CR?
  - Performance has angular resolution here; VLA doesn't? FS has a superset but SS/PP/CP aren't yet a superset perhaps?
  - Not all of the FS are propagating to the STL
  - Something seems to be weird about the requested time calculation. The Display shows 19h19h5m50s so I'm not quite sure how to interpret that.

Dana (Firefox, Fedora)

Solicitation
- I was able to import a solicitation. Not much one can do at this stage but I was able to change the Solicitation name, for example, and this worked.
Allocation Request
- I would move "Allocation Request Name" above the tabs (Capabilities, Target Overrides, Observation Specification, Science Target List).
- When trying to create a second Allocation Request I got the following error: "Could not find or generate the requested item".
- Capabilities
  - Entering coordinates in HMS format, for example, is a bit funky as discussed at the sprint review.
  - Entering in zero (e.g., parallax) does not persist.
  - Importing a file worked okay for both Field Sources and Spectral Specs.
  - I was able to delete a Capability, Field Source, and Spectral Spec.
- Target Overrides
  - I had 22 field sources and 2 spectral specs. I altered the sensitivity for one row (first field source/spectral spec). In the observations specification, however, all of the scans that included the spectral spec were altered. Also I reduced the sensitivity by a factor of 2 and was therefore expecting the integration time increase by a factor of 4, but the time increased by a factor of ~20. ***N.B., I may not have hit the save button.***
  - I tried this again. Simple case with one field source and one spectral spec. Here is the procedure:
    1. Target Overrides tab: modified the rms sensitivity from 500 to 1000.
    2. Target Overrides tab: Clicked "Save" button.
    3. Target Overrides tab: Clicked "Generate Observation Specification" button.
    4. Observation Specification tab: view duration. I expected the time to decrease by a factor of four. The duration went from 11m34s to 2m58s. Not quite a factor of four.
    5. Science Target List tab: The time on source, presumably in seconds is not quite the same as being displayed in the Observation Specification tab. For example, 171.98560615411375 seconds is 2m51.99s. But the duration here did change by exactly a factor of 4 (from 687.942424616455 to 171.98560615411375) .
  - I used the exact same procedure as above except on a more complex setup: 22 field sources and 2 spectra specs. I modified the rms sensitivity from 500 to 1000 for only one field source/spectral spec. Indeed, on the Observation Specification tab and the Science Target List tab only the modified field source/spectral spec duration changed. Had the same issue with the slight difference in time as noted above.
- Observation Specification
  - Not sure why the calibrator is at X-band when the target is L-band.
- Science Target List
  - Using the VLA hardwired values but I think this is TBD. The coordinates were not converted correctly from front-end to back-end. Started in Galactic but the same numbers are listed as RA/Dec.

Allie: (Firefox, redhat)

My expectations are
- A realistic-looking solicitation can have a Real VLA or GBT proposal. The test solicitation is no good for testing?
- In a Real VLA proposal, I can see the TOM, the ScienceTargetList, and the OS.
  - The capability request can update the TOM, the OS, and STL.
  - Updates to the TOM do not affect the OS.
  - Updates to the TOM should affect the STL.
  - I can edit the OS directly.
- In a GBT Spectral Line,
  - a more detailed capability request now exists
  - can save a capability request but there is no TOM/STL/OS generated yet.
Home page
- Clicking on "create proposal" on a solicitation directs to the new proposal page as expected. I have to then specify what solicitation I want again on this page. It would be nice if it auto filled this if I'm navigating from the solicitation on the home screen.
- delete the trailing quote+period on the Global Millimeter VLBI Array (GMVA)". line (sprint 24 comment)
Trying to edit an already created proposal
- Home → My Proposals Card, blue edit button once I've found the proposal → edit mode activated
  - Difficulty in finding proposal but easy once I do
- Home → Open Solicitations Card → View Proposals button → click my proposal card → doesn't take me to the proposal. it just takes me to a list view of all the proposals and my proposal is expanded into a new card. The expanded card as the grey edit button → edit mode activated
  - Many screens and scrolling required. The edit button is subtle here.
  - In a previous version, the expanded card would show up next to a list of all the filtered or unfiltered proposals. It tries to do that briefly but then gets shunted to the bottom of the page in my browser.
- Home → My Proposals Card → View All Proposals Link → click my proposal card → doesn't take me to the proposal. it just takes me to a list view of all the proposals and my proposal is expanded into a new card. The expanded card as the grey edit button → edit mode activated
  - Same issues as previous path.
Capability Request for a realistic looking solicitation
- Specifications
  - The + button for field source almost looks like it is for the spectral spec instead of adding a field source. Some space between the two might help. Same is true for the + for spectral spec and performance
- Spectral Spec
  - The units of bandwidth and spectral resolution are wrong
- Field Source
  - Trying to specify negative values for FOV, Parallax, etc does throw an error once I save.
  - A + in the radial velocity also throws the error "this.answer is null"
  - Adding a new field source but not filling it in throws the same error.
    - How do I delete a field source or a spectral spec?
  - When I collapse the capability request and expand it later, there is a truncation of the wording on any of the check boxes (e.g., coordinate system, epoch) that are checked.
  - I can't save with the null errors and so my changes don't persist if I collapse the capability but the checked boxes do persist? This is expected?
- Unclear what delete button deletes what. Which one deletes the capability request vs the entire allocation request when I have them expanded?
- In a GBT proposal for TommyttaMember, I've filled in all the fields; it is returning a null error but not highlighting any of the fields. I did not have this experience when I was in the SallyScientist proposal.
  - I tried a new proposal and it worked this time. I'm not sure why it didn't work before. I was trying to see what would break it at first and I got the null. I changed it all to "good" inputs and was still getting the null error. I fresh allocation request rectified this.
- VLA Capability
  - In capabilities tab, if I hit save, I believe that updates the TOM, ScienceTargetList.
  - In the Target overrides matrix, do I need to hit save to update the override or does it do it automatically? - I think it's the save button but at first I thought it was the save button from the capability request.
  - I was unsure about how these are saved/updated, so it might also confuse a user, leading them to update or not update inadvertently.
  - When is the ScienceTargetList generated/updated - only when I hit save? when I edit the TOM?
    - No system message is generated when I hit save on the TOM.
    - My edits to the TOM do not seem to be propagating to the ScienceTargetList. E.g., deselecting a source or changing the RMS sensitivity.
  - Is the band name that I specify in the capability request only in the TOM?
    - Should it eventually be in the ScienceTargetList and OS?
  - The source name isn't in the ScienceTargetList. I think it should be.
  - OS
    - When I add a scan, how does it decide how to populate the data? It seems like it's just the first ScienceTarget being filled in and being put at the bottom.
    - It would be nice to highlight this scan as being a change pending, as the scan lists can get really long and it might not be clear that this was the scan added.

Dana: (Firefox, Linux)

Solicitations
- I tried to load my own solicitation (gbtSC.json) but got the following error: "A backend error occurred. 'Bad Request' at 'api/solicitations/configure 5/6/22, 3:11:23 PM". I then copied "a realistic-looking solicitation" and changed only the name (real2.json) and got the same error message. The system seemed to hang (got the rotating circular arrows).
Allocation Request
- Capabilities
  - I tried to create a GBT Spectral Line Capability Request. I was able to fill out all of the parameters (two field sources, one spectral spec, performance parameters) but when I clicked saved I got the following error: "A backend error occurred. 'Internal Server Error' at 'api/allocation_requests/21 5/6/22, 5:23:23 PM". The only thing I noticed is a red highlight around one of the Declination textboxes that had a negative value (I recall Sam said this was a known issue but should be okay to save to the database). Again the system seemed to hang. I had to leave the Allocation Request tab and come back. Nothing was saved.

Allie: (Firefox, redhat)

Allocation Requests
- Creating a second Allocation Request works. At some point though, the ordering of the Allocation Request tabs change. It's happened a couple of times where Allocation Request and Allocation Request 2 will flip their order on the top bar. Spooky. I think it's putting whatever tab I'm currently working on first instead of just leaving the order alone.
- Capability Request (the tab is called Capability though?)
  - When the specification first loads, I can see all the fields but if I click on a particular parameter, it gives a filtered view. Performance doesn't currently have more fields, so I wasn't sure what to do there. It was a bit unclear how to navigate the RMSSentitivty/performance/spatial/spectral filters from there then. I think you can hit enter, which brings up the macro list of parameters (or cycles through the parameters) or you can click again on a parameter to return to the macro list.
  - The save button does save but it also redirects to the TOM right away, regardless if I was done entering in all the fields. For example, I put in one field source and no spectral spec, and it redirected me. As someone who hits the save button as a soothing mechanism/ thought-clearing instinct, this is a bit disruptive. Happily the information in the parameters persists as I navigate around the capability tab. They do not propagate to the observation specification unless I hit the save button.
    - If I enter a parameter in the Capability, go to the Observation Specification, and back without hitting save, the newly entered parameter does persist in the Capability even though I did not save it. I think that's a good thing.
  - the spectral specification
    - the Center frequency field has a helpful tip suggesting I enter in an RA. Bandwidth similarly suggests that a Declination is needed.
  - Should the side bar instead be named performance, calibration, field source, spectral (spec)? RMS is a performance parameter. The other performance parameters are missing.
  - The calibration parameters are missing.
  - (bookkeeping) the Advance Button still exists
  - I can delete a Capability Request. It seemingly still exists in the Observation Specification though and I'm not sure how to get rid of it.
  - If I act like I'm going to delete a Capability Request but then hit cancel when it prompts me to delete it, sometimes a weird error shows up at the top of the screen in red saying that that item can't be found. It's hard to replicate but I have seen it 3 ish times.
- Observation Specification
  - I think the filters are working as I expect.
  - If I hold down shift to try to multi select, it opens a new window of firefox instead. Crtl causes tabs with the title of "javascript:void(0)" to appeared in my browser. Maybe it's a linux problem?
  - Will there be a way to clear all filters?
  - I can Save the Observation Specification with my changes. It needs a big notice telling the user that the capability request is now invalid though.
  - I'm able to generate multiple Observation Specifications (see TOM comment). I don't have an OS with a lot of scans, but I think it has a nice layout.
    - Since there can be multiple Observation Specifications, should the name in the tab be "Observation Specification" still?
  - The name I put into the spectral spec doesn't seem to show up anywhere in the Observation Specification. It might be good to think more about why we have this field and its purpose, as the observation specification should probably display the actual names of the bands, as it currently does.
  - (bookkeeping) there are two save buttons: the one at the bottom of the observation spec and the one in the edit mode of a scan.
  - Adding a second Capability Request (and saving) did not immediately add it to the Observation Specification. I had to click around a bit and then it showed up finally. I'm having trouble replicating this behavior though.
  - I like that when I'm editing a Observation Specification, it says changing pending for that one and not all of them.
  - I can delete all the scans in an Observation Specification but I cannot delete the OS itself.
  - The column headings on the scan list are not lining up properly.
- TOM
  - Clicking the button generates a new Observation Specification in addition to the one(s) already in the Observation Specification tab. I know the TOM isn't user facing but thought I'd note it.
  - Once a Observation Specification exists, how do you update it from the Capability Request. Clicking the Generate Observation Specification button in this tab will create a second one that reflects the state of the capability request. (I think this is in Sprint 28: STT-682)

Dana (Firefox; Fedora):

Allocation Request
- Capabilities
  - I think RMSSensitivity should be under Performance.
  - I would add "Calibration" to the list (even if just a place holder now).
  - I would order these: Spatial, Spectral, Performance, and Calibration. This is more in line with how a scientist would think about the flow of information.
  - After creating a VLA Continuum Capability I tried to create a GBT Spectral Line Capability within the same Allocation Request. I should not be able to do this since an AR can only have one Facility. But I was able to do this initially until I saved the Capability Request and got an internal server error.
  - When I save a Capability I am sent to the Target Overrides tab which is a bit odd but okay for now.
- Target Override
  - Little confused by this tab but I think we discussed this at one of the meetings.
  - Each time I click the "Generate Observation Specification" Button I can see a new Observation Specification, even if I have not changed the Capability Request.
- Observation Specification
  - Overall this looks pretty good.
  - I could not find a way to delete an entire Observation Specification.

Dana:

Allocation Requests
- Not sure of the purpose of Target Overrides.
- Only one Field Source and one Spectral Spec is saved.
- Advanced button does not seem to do anything.

Jeff:

Why are there no Specifications under the VLA Continuum (is this a configuration issue for the solicitation or something else)?

Dana:

Home
- I can now see the Solicitations without logging in first.
Allocation Request
- When in edit mode, if I click the Allocation Request tab (on left) I see "Allocation Request +Add". My intuition was that I had to click the "+Add" to create an Allocation Request, but if I do this it creates a second Allocation Request. What I should have done was to click "Allocation Request" to start filling in the first one.
- Not sure if there was suppose to be any way to input information for the VLA Continuum Capability.
- I still cannot save an Allocation Request, but I can delete one.
- When in edit mode, if I cancel (click the "x" in the top right-hand corner), I am brought back to the view with the 2-D layout of cards. I would have expected to see the view with the 1-D column of cards on the left with the proposal that I just cancelled on the right.

Allie:

Home page
- delete the trailing quote+period on the Global Millimeter VLBI Array (GMVA)". line
- is it necessary to have two different create proposal buttons? For example, would the button next to the open solicitations automatically have that solicitation checked?
- How is the priority set for the list of proposals in "My Proposals"? The newest created proposal is not at the top of the list as I would expect.
- TonyattaMember and TommyttaMember have different home pages. Is this expected behavior because they have different profiles?
Editing a Proposal
- Allocation Request
  - When I add a Capability to an Allocation Request, the changes pending bar behaves as expected: the bar does appear until I make changes. It does not go away when I hit Save though. The save button doesn't appear to work. Maybe that isn't implemented yet.
  - When I make multiple Allocation Requests and then try to delete one, it sometimes deletes all of them instead. I'm having trouble consistently replicating this behavior but it did happen at least 4 times.
  - Is this expected behavior?
    - Create Allocation Request and Allocation Request 2
    - Delete Allocation Request
    - Add an Allocation Request makes a new allocation request with the name of "Allocation Request 2" such that now there are two Allocation Request 2s.
    - When adding more Allocation Requests (AR 3, AR4), the tabs reorder themselves as
      - AR3, AR4, AR2, AR2
  - Capabilities do not persist when switching between allocation requests (maybe the aforementioned save issue?).
  - Cannot rename the AR or the Capability
- Clicking between "Basic Information", "Scientific Justification", "Allocation Request" very briefly changes the wording at the top left from "Edit Proposal" to "New Proposal" and back again to "Edit Proposal".
- When clicking back to "Basic Information" from "Scientific Justification" or "Allocation Request", the check box with the "test solicitation" sometimes briefly displays the words "[object Object] " instead.

Dana:

Home Tab
- Looks better. Need to think what we really want here. If we list Open Solicitation maybe we should list Draft Proposals instead of My Proposals. That is, these are the things that I am currently working on. Then again, Submitted Proposals can be edited before the deadline and re-submitted for Semester Solicitations. So maybe only list proposals that can be modified.
Proposals Tab
- Added a bunch of proposals to see how this looks when viewing proposals (either in the 2-D layout of cards or the 1-D column of cards when viewing a selected proposal). I think this will work as long as we are able to use filters to keep the number of proposals down to a reasonable number.
- Basic Information. Cleaned up the "Changes Pending" feedback. Works well. Still cannot leave to another section without loosing my edits (if I do not click Save first), but okay.
- Scientific Justification. Known issues with persistence; otherwise okay.
- Allocation Request. Known issues with persistence; otherwise okay. Layout seems reasonable. This will probably evolve as we include details so we probably do not want to fine tune this at this stage. But since each Allocation Request has only one Facility we might want to include the Facility when showing the list of Allocation Requests.

Consensus Feedback

Really nice overall
Before the user is logged in the open solicitations are not shown. During the sprint review this was demonstrated to be a 401 error.
- STT-591 - Getting issue details... STATUS
Want to discuss the overall navigation (like the top w/ icons, lower ones are confusing)
- Try to isolate functional roles using UX
Consistent & Persistant Filtering
Changes pending is not working correctly. At least one of us didn't notice the changes pending notification.
- STT-593 - Getting issue details... STATUS
Note: Track OS and Browser
Note: Need to do the only authors can edit a proposal story at some point.
- STT-592 - Getting issue details... STATUS
Is delete supposed to work?
Edit button on detailed view took a while to find,

Jeff:
- For the solicitation I am not sure we want to display capabilities (there may be too many), I suspect we want to display the facilities for each solicitation though (maybe the capabilities as a redirect?)
- What is the purpose of having the proposals section under the home page, I agree with Dana that this is confusing.
- I think there is a question of if we want to be able to select the role we are logged in as?
- Filter should persist when going to a detail view.
Dana:

o Initial View
------------

- Overall the interface is very clean. Easy to read, easy to
navigate, and fast.

- Before login I cannot view the Solicitations.

- Maybe

"NRAO + GBO Telescope Time" --> "NRAO/GBO Telescope Time"
"Greenbank Observatory (GBT)" --> "Green Bank Telescope (GBT)"

and add:

"High Sensitivity Array (HSA)"
"Global Millimeter VLBI Array (GMVA)".

- The purpose of JWT at the top right is not clear. If I click
on it nothing happens.

o Login as SallyScientist
-----------------------

- Once I login I can view the Solicitations only after I reload the
page.

- On Home there are two buttons to create a proposal: one at the top
and the other under the "test solicitation". I was expecting the
latter to have already selected "test solicitation" once the proposal
was created.

- If I click on "View Past Awarded Proposals" I should only see those
proposals that were awarded time (no draft proposals). I realize
that this concept is probably not yet in the system.

- There are two ways to view Proposals. One is by clicking the
Proposal tab next to Home (where one can filter on the proposal), and
the other is by clicking the My Proposals tab next to Open
Solicitations. I can see the value of different displays of this
information but the functionality is a bit confusing. If I edit the
Proposal under My Proposal I am taken to the Proposal tab.

- Proposal View Mode

Maybe reduce the size of the left section that lists the proposals.

If I click on View under Scientific Justification I am placed back on
the Home page. I am not able to see the scientific justification
(which was already uploaded).

- Proposal Edit Mode

I cannot re-size the abstract window; seems like there is a little
widget for this but it does not work.

After I have clicked Save on the Basic Information page the there is a
temporary window that indicates the proposal is saved, but I still see
the blue bar at the top indicating "Changes Pending". This is
confusing since I have not made any changes yet.

If I edit the abstract on the Basic section and then move to the
scientific justification section without saving I loose the edit. It
would be useful to have some warning or indication of the edit status
and if information will be lost.

After I upload a scientific justification PDF, which works fine, if I
leave the proposal and come back I can no longer see (or can view) the
uploaded scientific justification. There is just a button "Select New
PDF".

o Login as TonyaTtamember
-----------------------

- Proposals Tab

Seems like the filter settings should persist. For example, if I select
SHOW->My, then go to the Home tab and back to the Proposal tab the filter
is set to SHOW->All.

What determines the placement of proposals cards?

Need to include a search to the filtering of proposals.

- Solicitation

Clicking on "Create a Solicitation" does not do anything.
Presumably not implemented yet.

If I click on "View Proposals" I was expect to see
only the proposals from the specified Solicitation but
all were listed.

- During the session multiple browser tabs were opened; not clear
when this happened and why.

Allie

This may not have real application but it might indicate an unwanted behavior
1. I login with JohnDoe and edit test proposaleeeeeee
2. While on that page, I login as Sally Scientist. I can now edit and save edits to this proposal. Even after a refresh
3. I can do this with any other log in.
4. The edits show up in test proposaleeeeeee when I login again as JohnDoe.
5. I can do the same to proposal My test proposal 2 W4 HMXB, which only has JohnDoe as an author
6. Sometimes I can see in the list of proposals test proposaleeeeeee in SallyScientists' list
7. Returning to item (b), if I refresh, I see SallyScientist's proposals listed but I can still edit JohnDoe's
  1. This image shows John Doe's proposal but logged in as sally without refreshing first
    1. prefresh_as_Sally
  2. This image shows the screen after the refresh
    1. postrefresh_as_sally
8. Finally, logging out leaves the information open (e.g., the edit proposal page) but I can no longer edit it, so that's good.
Again this may not be a practical issue, but I had multiple tabs open editing proposals as different users and it seemed to behavior well.
1. Having the same person logged in editing the same proposal in different tabs creates problems. If I save an edit in one tab and do not refresh in the other, what ever I save in the "other" tab will overwrite the first. A refresh shows the first tab's update. This is a fair behavior perhaps.
2. The same behavior happens when I am trying to edit the same proposal with two logged in accounts, which may be a problem. There is tandem editing in the current tool (to what ever degree it does or doesn't exhibit this problem).
Is it expected behavior that Tommy Ttamember and TonyaTtamember have different items in the "Review tab"? It looks like TonyaTtamember is a reviewer while Tommy Ttamember is admin?
It seems that there is a proposal listed as "My proposal" for Tommy Ttamember that has Tammy Ttamember as the primary author. Are these supposed to be the same person?
1. If not, is Tommy ttamember exercising admin-powers to edit/create a proposal for Tammy?
2. If Tommy ttamember has no intention of being on this proposal, should it show up under "my proposals"?
3. Can users create a proposal without being on it? That seems like a policy question, and I know the author portion has some work to do on it still.
I had the same experience as Dana with the abstract not saving upon navigation
1. Happily though, I can't seem to accidentally navigate backwards using the mac track pad short cut.
On the landing page once i log in as anyone, I find it a bit unsightly to see duplicate tabs for navigation at the top.
Hitting the Review tab takes me back to home, not a separate interface for reviews and not even to the review tab at the bottom.
I can't seem to delete a proposal.

===========

The second set of tabs on the home page were only intended to get some "quick start" areas there - likely actions a person wants to take e.g. working on the most proposals edited or viewing the responses numbers for recently closed solicitations. We do need to do some UX work to really flesh out what should be here, but for now the tabs were removed so they are less confusing.

Multiple tabs: local storage is used to save the logged in user's JWT (identifies who is logged in). Local storage in a browser is organized by domain name and stores simple key/value pairs. Having multiple tabs open in one browser and logging in as different people won't work so well because the domain name is the same, so whatever tab logs in last overwrites who is logged in for all tabs.

The "JWT" button: This is for developers, so it doesn't look like does anything for users and it will go away when we have "real" authentication.

Clicking the "create proposal" button on the home page next to an open solicitation is going to have that solicitation already selected when the creation form opens - but that isn't hooked up yet so it doesn't work. The assessment of what should happen was correct - it just isn't implemented yet. Perhaps we make that a story so it doesn't get lost. The same is true for clicking "view proposal" on a solicitation card - that is also not hooked up yet.

Switching between sections of proposal editing does lose the changes on that page without warning. We should make a story to address that.

the scientific justification pdf upload is set up in the UI, but isn't set up in the backend, so saving doesn't actually work right now. We have stories for this already.

Filtering of proposal cards is handled by the back end, so any search or sorting will need to happen there as well in case we need to introduce pagination (which we will at some point). We will need stories for this.

Filters do not persist, but should. This is all front end work and we should make a story for it.

Anyone with a role of TTAT member can see all proposals, but the "my" filter should still work to restrict what is shown. Because authorization is fake, we have a different list of users in the frontend than the backend, so sometimes things look funny. We can fix this or ignore it until we get to more default test data or real authorization.

Space shortcuts

Page tree

Consensus Feedback