Versions Compared

Old Version 4

changes.mady.by.user John Tobin

Saved on

compared with

New Version Current

changes.mady.by.user Anthony Sowinski

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Below is a list of items that are priorities for Archive implementation, some of these are Legacy AAT retirement tickets, others are new development. The order in the tables is in priority order.


Main topics (click link to jump to section in page):

Data Delivery

Search/Data Selection

VLBA-specific

VLA(VLBA) Metadata presentation

Legacy VLA Data

Data Integrity

DA Tools

Missing Data

Other

ALMA

Product Versioning



Data Delivery

Anchor
DataDelivery
DataDelivery

The goal of this page is to compile both issues with the New Archive (AAT) and features the Legacy Archive (LA) has but not the AAT. The issues posted will be reviewed by John Tobin and Anna Kapinska (archive sub-system scientists) for submission as Jira tickets or submission of a requested development project for SSA planning. See the example for an issue posting below, we ask that whomever records an issue put their name and date along side it such that we can reach out if further information is needed.

Example:

Real recorded issues start here:

...

Priority/Done?NotesJiRA ticket /Confluence page
CriticalDownloads via Scriptable query interface

Scriptable Query Interface to NRAO Archive  

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
key

...

WS-

...

1395


criticalA friendly error is needed when a users is not authorized to access proprietary data.

Jira

...

Jira
serverDMS JIRA
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySSA-7421

...

serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySSA-

...

7474

confusing to staff as well

-need to kill RH, WS 3.0+

criticalRestore data using same version of CASA as calibrated

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keyWS-1405


mediumPut warning that only one CMS at a time can be requested in AAT
Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySSA-7106


could be a flyover text (currently has a flyover circle with slash)
high (WS 2.8/2.9)

Clean up the nested directory data delivery mess


legacy VLA data deliveries are one level deeper in directories than an SDM

*May need revisions to account for VLBA differences.

Delivery Directory Improvements

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySSA-7000

WS 2.7 or 3.0?
planned 4.2.2

Don't restrict download number limit.

New ticket to remove the '/100 string from webpage'

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySSA-7352
Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySSA-7436

We would like no limit on the number of files, but keep the 10TB for the total downloaded volume.
4.2.1

Fix select-all functionality

  • Always have select all available (only displayed files/SDM or from all pages)


  • make select-all work properly with select/deselect

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySSA-7573


mediumDownload weblogs directly (and/or serve them as well!)

Direct weblog download from archive search return interface 

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keyWS-1388
Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keyWS-1389
!


mediumBulk download of Calibration tarballs

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySSA-7353


lowarchive should ask for confirmation to continue if the download or processing job is considered largeTo be requestedThe front end can show size estimates for various products.
lowData Delivery times out if tar file(s) are too large/time consuming


Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySSA-6083
will not be an issue with taring not being the default.
lowarchive-new delivery not chmod'd/chowned


Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySSA-5096

√ 2022.2 pipelineoption to return UNTAR'd file does not work 

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySSA-6495
https://open-confluence.nrao.edu/display/SRDP/Product+lists+for+Basic+and+Calibrated+Measurement+Set+downloads

pipeline topic
√ (4.2.0)The email sent by the archive tool will include the wget command in the near future.

 

Jira
serverDMS JIRA
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySSA-7421


√open up local delivery restrictions (to /home and /lustre). 

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySSA-7179

The part related to opening the download destination to nm-### accounts on lustre is

Jira
serverDMS JIRA
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySSA-7345

This should cover users with visitor accounts wanting to download data to lustre, and local staff wanting to download data to their local machines. We have a problem with the quota of the observer accounts (I downloaded 8 TB of data to my observer account!); CIS remains irresponsive!
√disable checksums by default

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySSA-7343


√make tar downloads default to 'off'

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySSA-7344



Search interface/Data Selection

Anchor
Search
Search

Priority/Done?Notes

high

Fix Frequency-dependent FOV and position searches

  • exact same feature as legacy archive, must select to do the freq. dependent FOV
  • legacy archive has this feature

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySSA-4915
Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySSA-7497


high (4.2.2)

Position searches provide distance from searched position and sort by distance

  • legacy archive feature

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySSA-7387

User request, but is a legacy AAT feature
medium

Treat wild cards consistently in AAT search interface

  • account for * in source names

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySSA-7496


lowAllow search return of observation scans?

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySSA-7388

Helpdesk ticket for a search return we do not have in AAT, but legacy archive provides.

high

Allow search filtering based on total time on source.

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySSA-

...

7424

Based on legacy archive helpdisk question from user. UC feedback as well.
mediumFix mosaic position searches for VLASS and ALMA mosaics (subscan vs. scan)

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
key

...

WS-

...

1365



Rearrange boxes in 'Search Inputs'Ticket to be created.
√ (4.2.0)Scriptable query interface

Scriptable Query Interface to NRAO Archive

Jira
serverDMS JIRA
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySSA-7395


√

Download number limit: 100 data sets, but download data volume limit to 10TB per request.


Doesn't fully solve problem see also

...

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySSA-

...

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySSA-7352

...

7352


This change has already been made by Stephan, but 100 files is too limiting for various cases (take BM360 as one example). We would more likely want no limit on the number of files, but keep the 10TB for the total downloaded volume.


VLBA Stuff

Anchor
VLBA
VLBA

...

option to return UNTAR'd file does not work 
Priority/Done?NotesJiRA ticket /Confluence page
High

VLBA: DA archive tools (note specifically the limitation of USNO data not having PI)

  • CHECK IF ALL REQUIREMENTS ON CONFLUENCE ARE SATISFIED
  • Current tools should be able to add PI for USNO

https://open-confluence.nrao.edu/display/SPR/VLBA%3A+DA+archive+tools

proprietary period setter: 

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySSA-

...

7350

The VLBA DA tools for changing PI and setting proprietary period have been complete for some time. The confluence page also mentions a tool for annotations; maybe that part should be moved to the Data Integrity section?

HighVLBA+EVLA projects without metadata and sometimes also mismatched PI

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345

...

-36eee8a98f01
keySSA-7423


High (4.2.1)Proxy issue for huge VLBA projects

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySSA-

...

7436


High (4.2.1)Slow VLBA indexing

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySSA-7563

Issues already in JIRA/Confluence:

Done?TitleJiRA ticket /Confluence pageDetails
Tool for setting Proprietary Period of science observations on the VLA and the VLBAhttps://open-confluence.nrao.edu/pages/viewpage.action?spaceKey=SPR&title=Tool+for+setting+Proprietary+Period+of+science+observations+on+the+VLA+and+the+VLBADA annotation tools/Archive data annotation toolshttps://open-confluence.nrao.edu/display/SRDP/Archive+Data+Annotation+Tools

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySSA-

6495https://open-confluence.nrao.edu/display/SRDP/Product+lists+for+Basic+and+Calibrated+Measurement+Set+downloads

7541


MedWrong band designation in VLBA data (old data only?)
pipeline topicOther tickets (so they are not forgotten)

https://open-jira.nrao.edu/issues/?filter=11464

Proposal Contact Author not added to data.nrao.edu databaseWeb pageWeb linkJIRA ticket (if applicable)my.nrao.edu dashboard (Under My Information - My Data)https://my.nrao.edu/nrao-2.0/secure/Home.htm?index=1 JiraserverDMS JIRA

Jira
serverDMS JIRA
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySSA-

7465Bug when a project uses different PI and Contact Author

For completeness, in the table below we give web pages that will not work once the legacy archive becomes internal only (we can move this to another confluence page if needed)

7420

This is in both the legacy archive and the new archive tool. Issue needs more research on how prevalent it is; unclear.
Med
  • For observing VLBA projects that incorporate other dishes (e.g. +Y1, +Y27 etc), the VLA files should be associated with the relevant VLBA segment and not considered as "mixed" projects. This has been descoped from ticket 
    Jira
    serverDMS JIRA
    columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
    columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
  • serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
    keySSA-
7317
  • 7185
    but needs to be picked up in future AAT releases

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345

-36eee8a98f01keySSA-7322VLA plone pageshttps://science.nrao.edu/facilities/vla/archive/indexSW provided a wget example to be added to the science webpage. We need to add this example to the plone page, see the ticket SSA-7421 for the example. The email sent by the archive tool will include the wget command in the near future. JiraserverDMS JIRA

-36eee8a98f01
keySSA-7609

Ticket created Anna Kapinska, Emmanuel Momjian, and Anthony Sowinski should check it to make sure it's correct and specifies all requirements.


MedDefault Instrument of VLBA projects in AAT/PPI

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution

serverIdeb2e750b-a83a-387e-8345-36eee8a98f01

keySSA-7421VLBA plone pageshttps://science.nrao.edu/facilities/vlba/facilities/data-archive/index

keySSA-7208

And a new ticket created:

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution

SW provided a wget example to be added to the science webpages. We should include the example (see SSA-7421) in the plone pages (VLA, and VLBA). The email sent by the archive tool will include the wget command in the near future. JiraserverDMS JIRA

serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySSA-

7421Computing guidehttps://info.nrao.edu/computing/guide/cluster-processing/data-storage-and-retrievalNumerous CASA Guides referencing the archivehttps://casaguides.nrao.edu/index.php?title=Main_PagePut warning that only one CMS at a time can be requested in AAT7106Data Delivery times out if tar file(s) are too large/time consuming6083open up local delivery restrictions (to /home and /lustre). 7179The part related to opening the download destination to nm-### accounts on lustre is DMS JIRA7345disable checksums by defaultkey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution7344archive-new delivery not chmod'd/chowned5096Bulk download of Calibration tarballs7353legacy VLA data deliveries are one level deeper in directories than an SDM7000A friendly error is needed when a users is not authorized to access proprietary data.7474Frequency-dependent FOV searches4915Position searches provide distance from searched position7387Allow search return of observation scans?73887424import FITSAIPS files without matching projects (51 files)7426Add FITSAIPS files to 7328old VLBA project metadata issuesresolutionDMS JIRA7420Remove Legacy Archive label and link from new archive7375The new archive has a link to the legacy archive at the top. This needs to be removed since the legacy archive will not be accessible from outside.

7619

Confluence req: VLBA: Default Instrument of VLBA projects in AAT/PPI


MedAdd PFT (proposal finder tool) link in Project View for VLBA projectsData Delivery

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySSA-

could be a flyover text

7461


LowImport remaining 8 GMVA files

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySSA-

7438


LowMark4 files with 0 length getting added to new archive metadatawill not be an issue with taring not being the default.attempted for 4.2.0 but is deferredDon't restrict download number limit.

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySSA-
7352
7297

Small We would like no limit on the number of files , but keep the 10TB for the total downloaded volume.4.1.1affected.
LowIncomplete VLBA data of observations crossing midnight (open in case more cases show up)

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySSA-6916


HighCreate USNO download acknowledgement

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution

Jira
server

serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySSA-

This should cover users with visitor accounts wanting to download data to lustre, and local staff wanting to download data to their local machines. We have a problem with the quota of the observer accounts (I downloaded 8 TB of data to my observer account!); CIS remains irresponsive!4.1.1

6952


√(reopened) Low?
Remove Legacy Archive label and link from new archive

Jira
serverDMS JIRA
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySSA-7375

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01

keySSA-7343
4.1.1make tar downloads default to 'off'
Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columns

keySSA-7627

Update (26-Sep-2022, Anna): the link is still present on some of the AAT subpages, especially the download status page when request for data is submitted.

The new archive has a link to the legacy archive at the top. This needs to be removed since the legacy archive will not be accessible from outside.

√ 4.2.0Add wget command in emails generated by the AAT upon downloading data

Jira
serverDMS JIRA
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySSA-

7421


√ 4.1.1

import FITSAIPS files without matching projects (51 files) and project GP0024

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySSA-

7426


√ 4.1.1

Add FITSAIPS files to new archive and segment association

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySSA-

7328


√ 4.1.1old VLBA project metadata issues

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySSA-

Will happen with workspaces later

7379


√ 4.1.1VLBA data structure in the new archive - improved interface

Jira
serverDMS JIRA
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySSA-7185

Confluence: GUI (segment hierarchy) for VLBA data in the new archive


√ (4.1.1)Database has zeros for min/max frequencies for some archive filesarchive should ask for confirmation to continue if the download or processing job is considered largeTo be requestedThe front end can show size estimates for various products.

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySSA-

7277


√877 VLBA correlations with missing files entries

Jira
serverDMS JIRA
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySSA-7471


√proprietary clock for VLBA dataSearch  Interface

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySSA-

6552

done
√VLBA Reingestion Refinements

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySSA-

User request, but is a legacy AAT feature

5917

ongoing → done
√VLBA Polarization Duplication

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySSA-

5640

done
√String Terminators in VLBA fields

Helpdesk ticket for a search return we do not have in AAT, but legacy archive provides.

Allow search filtering based on integration time.

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySSA-

Based on legacy archive helpdisk question from user.VLBA Stuff

5641

done
√BL473 unlock and missing segments

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySSA-

7148

done
√VLBA data structure in the new archive

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySSA-

7076
Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,

serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySSA-7379
Wrong band designation in VLBA data (old data only?)
Jira
serverthis is in both the legacy archive and the new archive tool.Add wget command in emails generated by the AAT upon downloading data

resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySSA-

7182

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
key

SSA-7421√

SSA-7183

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySSA-

7184

done
√

Download number limit: 100 data sets, but download data volume limit to 10TB per request.

This change has already been made by Stephan, but 100 files is too limiting for various cases (take BM360 as one example). We would more likely want no limit on the number of files, but keep the 10TB for the total downloaded volume.USNO Proprietary Download System/A way to deliver data from projects that are not archived

VLBASYS 1008

done





VLA Metadata Presentation 

Anchor
VLAMetadata
VLAMetadata

VLBA Reingestion Refinements5917
Priority/Done?NotesJiRA ticket /Confluence page
critical

Scan list does not show coordinates between dec= 00 and dec = -1 correctly

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution

877 VLBA correlations with missing files entries

Jira
serverDMS JIRA
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySSA-7471

√

7620


Critical (4.3.0)Make scanlist information useful VLBA data structure in the new archive

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySSA-7076
Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySSA-7182
7188
 https://open-confluence.nrao.edu/display/SRDP/VLA+Metadata+Display+Updates

JT: revise to capture what we want about the frequency/setup information

This ticket has multiple points to make the new tool match the Legacy archive and to have the displayed data be much more useful.
High (4.3/4.4)VLA Metadata display updates. 

https://open-confluence.nrao.edu/display/SRDP/VLA+Metadata+Display+Updates 

Jira
SSA-7183
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keyWS-1390

This has many items including 'telescope observing logs'.
Medium

Incorrect metadata sometimes displayed for projects with duplicate proposal IDs between VLA and VLBA

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySSA-7184

doneVLBA data structure in the new archive - improved interface

Jira
serverDMS JIRA
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySSA-7185

Confluence: GUI (segment hierarchy) for VLBA data in the new archive

7318


MediumVLA/VLBA proposal code degeneracyDefault Instrument of VLBA projects in AAT/PPI
Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySSA
-7208

Confluence req: VLBA: Default Instrument of VLBA projects in AAT/PPI

VLBA: DA archive tools (note specifically the limitation of USNO data not having PI)

https://open-confluence.nrao.edu/display/SPR/VLBA%3A+DA+archive+tools

proprietary period setter: 
-5538

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySSA-
7350
7318


MediumIndexing of VLA low band projectsneeds follow up, somethings are inconsistent (especially the topic of the tool's readiness to change the proprietary period of VLBA data).proprietary clock for VLBA data

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySSA-6552

should have been done for 4.1.0. Stephan will follow up to confirm.

7187


√

Subscans table holding both Radian and Degree positions

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySSA-

5275

doneongoing
√archive search returning incorrect projectsMark4 files with 0 length getting added to new archive metadata
Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySSA-
7297		Small number of files affected.
6192


done





Legacy VLA Data

Anchor
Legacy
Legacy

6952-7192Incorrect metadata sometimes displayed for projects with duplicate proposal IDs between VLA and VLBA
Priority/Done?NotesJiRA ticket /Confluence page
Criticalcorrupt legacy VLA data in the AAT√VLBA Polarization Duplication

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySSA-56407628

done
HighMetadata Issues in Legacy VLA dataIncomplete VLBA data of observations crossing midnight (open in case more cases show up)

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySSA-69167192

Needs input from Bryan
HighLegacy VLA data missing error codes in the new archive tool√String Terminators in VLBA fields

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySSA-5641

done√

7191

Mapping of quality errors were lost. Stephan will continue following up with Bryan. Blocker for LA retirement.
Highmetadata errors in VLBA (BD114) and old VLA data in new archiveBL473 unlock and missing segments
Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySSA-
7148		doneCreate USNO download acknowledgement
7128

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySSA-

√USNO Proprietary Download System/A way to deliver data from projects that are not archived

VLBASYS 1008

done

7129

This has been split into two tickets: one VLBA and one VLA
Mediumlegacy VLA metadata error: EBs that claim to have run for 2+ daysVLA Metadata PresentationMetadata Issues in Legacy VLA data

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySSA

Needs input from BryanVLA Metadata display updates. https://open-confluence.nrao.edu/display/SRDP/VLA+Metadata+Display+UpdatesThis has many items including 'telescope observing logs'.

-7129

requires re-ingestion


Data Integrity

Anchor
Integrity
Integrity

Subscans table holding both Radian and Degree positions5275donearchive search returning incorrect projects-36eee8a98f01SSA-5538
Priority/Done?NotesJiRA ticket /Confluence page
BlockerGap Analysis Between Legacy and New Archive
Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySSA-6222


can be done once RLSW finishes the AIPSFITS files
CriticalAutomated annotations for missing scans/BDF, Automated Error Recognition of Ingested Data7318Legacy VLA data missing error codes in the new archive tool

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySSA-71917190
https://open-confluence.nrao.edu/display/SRDP/Archive+Data+Annotation+Tools


CriticalDisplay of archive ingestion/observation issuesMapping of quality errors were lost. Stephan will continue following up with Bryan. Blocker for LA retirement.Make scanlist information useful 
Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySSA-
7188 https://open-confluence.nrao.edu/display/SRDP/VLA+Metadata+Display+Updates		This ticket has multiple points to make the new tool match the Legacy archive and to have the displayed data be much more useful.
7189


related to SSA-7190
CriticalIdentification of ingestion problems and automatically fixlegacy VLA metadata error: EBs that claim to have run for 2+ days

Example ticket of a silent failure: 

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySSA-7129

requires re-ingestionDatabase has zeros for min/max frequencies for some archive files

7566

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySSA-72777665


√create missing db entries for Flag.xml
Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySSA-
6273


Done
√fix ASDM.xml files missing FlagTable entities
Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345
keySSA-6192
doneVLA/VLBA proposal code degeneracy
Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
key
-36eee8a98f01
keySSA-6274


Done


DA Tools for Data Editing

Anchor
DATools
DATools

SSA7318SSA-7128
Priority/Done?NotesJiRA ticket /Confluence page
√Tool for setting Proprietary Period of science observations on the VLA and the VLBAhttps://open-confluence.nrao.edu/pages/viewpage.action?spaceKey=SPR&title=Tool+for+setting+Proprietary+Period+of+science+observations+on+the+VLA+and+the+VLBA
CriticalDA annotation tools/Archive data annotation tools

https://open-confluence.nrao.edu/display/SRDP/Archive+Data+Annotation+Tools 

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
key

WS-

metadata errors in VLBA (BD114) and old VLA data in new archive

1392
Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
key

WS-1393
WS



Missing Data

Anchor
Missing
Missing

fix ASDM.xml files missing FlagTable entities
Priority/Done?NotesJiRA ticket /Confluence page
Critical

Tix to be created 

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keyWS-1394

May require discussion about how new archive will display images and the uvfits datasets.


HighALMA Standard images into AAT

Jira
serverDMS JIRA
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySSA-71297278


HighLarge project support - ALFALFA + Other collection ingestThis has been split into two tickets: one VLBA and one VLAIndexing of VLA low band projects

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySSA-7187

Data Integrity

SSA-7437

√create missing db entries for Flag.xml

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySSA-

6273		Done√

5712
ALFALFA: SRAO and Collection Support



Other

Anchor
Other
Other

key,summary,type,created,updated,due,assignee,reporter,priority,status,resolution6274Gap Analysis Between Legacy and New Archive6222
Priority/Done?NotesJiRA ticket /Confluence pageDetails

Other tickets (so they are not forgotten)

https://open-jira.nrao.edu/issues/?filter=11464

Tickets remaining are listed throughout their respective categories.
Critical

Proposal Contact Author not added to data.nrao.edu database

Jira
serverDMS JIRA

columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columns

serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySSA-

Done

7465

Bug when a project uses different PI and Contact Author
MediumPreserve login between AAT and my.nrao.edu

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySSA-

7322


√ (4.2.0)Remove pop-up page when accessing data.nrao.educan be done once RL finishes the AIPSFITS filesAutomated annotations for missing scans/BDF, Automated Error Recognition of Ingested Data

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySSA-7190
https://open-confluence.nrao.edu/display/SRDP/Archive+Data+Annotation+Tools
7502



ALMA

Anchor
ALMA
ALMA

Priority/Done?NotesJiRA ticket /Confluence pageDetails
CriticalALMA MOUS/calibration ingest unreliable; ALMA butler implementation into productionDisplay of archive ingestion/observation issues

Jira
serverDMS JIRA
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySSA-

7189

7482



Product Versioning

Anchor
Versioning
Versioning

Priority/Done?NotesJiRA ticket /Confluence pageDetails
CriticalNeed to allow multiple versions of products (images/calibrations) in the archive

Product Versioning and Replacement Design

Jira
serverDMS JIRA
serverIdeb2e750b-a83a-387e-8345-36eee8a98f01
keySSA-6476

related to SSA-7190