You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 76 Next »

Below is a list of items that are priorities for Archive implementation, some of these are Legacy AAT retirement tickets, others are new development. The order in the tables is in priority order.


Main topics (click link to jump to section in page):

Data Delivery

Search/Data Selection

VLBA-specific

VLA(VLBA) Metadata presentation

Legacy VLA Data

Data Integrity

DA Tools

Missing Data

Other

ALMA

Product Versioning



Data Delivery

Priority/Done?NotesJiRA ticket /Confluence page
CriticalDownloads via Scriptable query interface

Scriptable Query Interface to NRAO Archive  


criticalA friendly error is needed when a users is not authorized to access proprietary data.

SSA-7474 - Getting issue details... STATUS

confusing to staff as well

-need to kill RH, WS 3.0+

mediumPut warning that only one CMS at a time can be requested in AAT SSA-7106 - Getting issue details... STATUS


could be a flyover text (currently has a flyover circle with slash)
high (WS 2.8/2.9)

Clean up the nested directory data delivery mess


legacy VLA data deliveries are one level deeper in directories than an SDM

*May need revisions to account for VLBA differences.

Delivery Directory Improvements

SSA-7000 - Getting issue details... STATUS

WS 2.7 or 3.0?
attempted for 4.2.0 but is deferred

Don't restrict download number limit.

New ticket to remove the '/100 string from webpage'

SSA-7352 - Getting issue details... STATUS SSA-7436 - Getting issue details... STATUS

We would like no limit on the number of files, but keep the 10TB for the total downloaded volume.
4.2.1

Fix select-all functionality

  • Always have select all available (only displayed files/SDM or from all pages)


  • make select-all work properly with select/deselect

SSA-7573 - Getting issue details... STATUS


mediumDownload weblogs directly (and/or serve them as well!)Direct weblog download from archive search return interface
mediumBulk download of Calibration tarballs

SSA-7353 - Getting issue details... STATUS


lowarchive should ask for confirmation to continue if the download or processing job is considered largeTo be requestedThe front end can show size estimates for various products.
lowData Delivery times out if tar file(s) are too large/time consuming


SSA-6083 - Getting issue details... STATUS 		will not be an issue with taring not being the default.
lowarchive-new delivery not chmod'd/chowned


SSA-5096 - Getting issue details... STATUS

√ 2022.2 pipelineoption to return UNTAR'd file does not work 

SSA-6495 - Getting issue details... STATUS https://open-confluence.nrao.edu/display/SRDP/Product+lists+for+Basic+and+Calibrated+Measurement+Set+downloads

pipeline topic
√ (4.2.0)The email sent by the archive tool will include the wget command in the near future.

  SSA-7421 - Getting issue details... STATUS


√open up local delivery restrictions (to /home and /lustre). 

SSA-7179 - Getting issue details... STATUS

The part related to opening the download destination to nm-### accounts on lustre is SSA-7345 - Getting issue details... STATUS

This should cover users with visitor accounts wanting to download data to lustre, and local staff wanting to download data to their local machines. We have a problem with the quota of the observer accounts (I downloaded 8 TB of data to my observer account!); CIS remains irresponsive!
√disable checksums by default

SSA-7343 - Getting issue details... STATUS


√make tar downloads default to 'off'

SSA-7344 - Getting issue details... STATUS



Search interface/Data Selection

Priority/Done?Notes

high

Fix Frequency-dependent FOV and position searches

  • exact same feature as legacy archive, must select to do the freq. dependent FOV
  • legacy archive has this feature

SSA-4915 - Getting issue details... STATUS SSA-7497 - Getting issue details... STATUS


high (4.2.2)

Position searches provide distance from searched position and sort by distance

  • legacy archive feature

SSA-7387 - Getting issue details... STATUS

User request, but is a legacy AAT feature
medium

Treat wild cards consistently in AAT search interface

  • account for * in source names

SSA-7496 - Getting issue details... STATUS


lowAllow search return of observation scans?

SSA-7388 - Getting issue details... STATUS

Helpdesk ticket for a search return we do not have in AAT, but legacy archive provides.

high

Allow search filtering based on total time on source.

SSA-7424 - Getting issue details... STATUS

Based on legacy archive helpdisk question from user. UC feedback as well.
mediumFix mosaic position searches for VLASS and ALMA mosaics (subscan vs. scan)


Rearrange boxes in 'Search Inputs'Ticket to be created.
√ (4.2.0)Scriptable query interface

Scriptable Query Interface to NRAO Archive SSA-7395 - Getting issue details... STATUS


√

Download number limit: 100 data sets, but download data volume limit to 10TB per request.


Doesn't fully solve problem see also

SSA-7352 - Getting issue details... STATUS


This change has already been made by Stephan, but 100 files is too limiting for various cases (take BM360 as one example). We would more likely want no limit on the number of files, but keep the 10TB for the total downloaded volume.


VLBA Stuff

Priority/Done?NotesJiRA ticket /Confluence page
High

VLBA: DA archive tools (note specifically the limitation of USNO data not having PI)

  • CHECK IF ALL REQUIREMENTS ON CONFLUENCE ARE SATISFIED
  • Current tools should be able to add PI for USNO

https://open-confluence.nrao.edu/display/SPR/VLBA%3A+DA+archive+tools

proprietary period setter:  SSA-7350 - Getting issue details... STATUS

needs follow up, some things are inconsistent (especially the topic of the tool's readiness to change the proprietary period of VLBA data).
HighVLBA+EVLA projects without metadata and sometimes also mismatched PI

SSA-7423 - Getting issue details... STATUS


High (4.2.1)Proxy issue for huge VLBA projects

SSA-7436 - Getting issue details... STATUS


High (4.2.1)Slow VLBA indexing

SSA-7563 - Getting issue details... STATUS SSA-7541 - Getting issue details... STATUS


MedWrong band designation in VLBA data (old data only?)

SSA-7420 - Getting issue details... STATUS

This is in both the legacy archive and the new archive tool. Issue needs more research on how prevalent it is; unclear.
Med
  • For observing VLBA projects that incorporate other dishes (e.g. +Y1, +Y27 etc), the VLA files should be associated with the relevant VLBA segment and not considered as "mixed" projects. This has been descoped from ticket  SSA-7185 - Getting issue details... STATUS but needs to be picked up in future AAT releases

SSA-7609 - Getting issue details... STATUS

Ticket created Anna Kapinska, Emmanuel Momjian, and Anthony Sowinski should check it to make sure it's correct and specifies all requirements.


MedDefault Instrument of VLBA projects in AAT/PPI

SSA-7208 - Getting issue details... STATUS

And a new ticket created:

SSA-7619 - Getting issue details... STATUS

Confluence req: VLBA: Default Instrument of VLBA projects in AAT/PPI


MedAdd PFT (proposal finder tool) link in Project View for VLBA projects

SSA-7461 - Getting issue details... STATUS


LowImport remaining 8 GMVA files

SSA-7438 - Getting issue details... STATUS


LowMark4 files with 0 length getting added to new archive metadata

SSA-7297 - Getting issue details... STATUS

Small number of files affected.
LowIncomplete VLBA data of observations crossing midnight (open in case more cases show up)

SSA-6916 - Getting issue details... STATUS


HighCreate USNO download acknowledgement

SSA-6952 - Getting issue details... STATUS


√(reopened) Low?
Remove Legacy Archive label and link from new archive

SSA-7375 - Getting issue details... STATUS

SSA-7627 - Getting issue details... STATUS

Update (26-Sep-2022, Anna): the link is still present on some of the AAT subpages, especially the download status page when request for data is submitted.

The new archive has a link to the legacy archive at the top. This needs to be removed since the legacy archive will not be accessible from outside.

√ 4.2.0Add wget command in emails generated by the AAT upon downloading data

SSA-7421 - Getting issue details... STATUS


√ 4.1.1

import FITSAIPS files without matching projects (51 files) and project GP0024

SSA-7426 - Getting issue details... STATUS


√ 4.1.1

Add FITSAIPS files to new archive and segment association

SSA-7328 - Getting issue details... STATUS


√ 4.1.1old VLBA project metadata issues

SSA-7379 - Getting issue details... STATUS


√ 4.1.1VLBA data structure in the new archive - improved interface

SSA-7185 - Getting issue details... STATUS

Confluence: GUI (segment hierarchy) for VLBA data in the new archive


√ (4.1.1)Database has zeros for min/max frequencies for some archive files

SSA-7277 - Getting issue details... STATUS


√877 VLBA correlations with missing files entries

SSA-7471 - Getting issue details... STATUS


√proprietary clock for VLBA data

SSA-6552 - Getting issue details... STATUS

done
√VLBA Reingestion Refinements

SSA-5917 - Getting issue details... STATUS

ongoing → done
√VLBA Polarization Duplication

SSA-5640 - Getting issue details... STATUS

done
√String Terminators in VLBA fields

SSA-5641 - Getting issue details... STATUS

done
√BL473 unlock and missing segments

SSA-7148 - Getting issue details... STATUS

done
√VLBA data structure in the new archive

SSA-7076 - Getting issue details... STATUS SSA-7182 - Getting issue details... STATUS SSA-7183 - Getting issue details... STATUS SSA-7184 - Getting issue details... STATUS

done
√USNO Proprietary Download System/A way to deliver data from projects that are not archived

VLBASYS 1008

done





VLA Metadata Presentation 

Priority/Done?NotesJiRA ticket /Confluence page
critical

Scan list does not show coordinates between dec= 00 and dec = -1 correctly

SSA-7620 - Getting issue details... STATUS


Critical (4.3.0)Make scanlist information useful 

SSA-7188 - Getting issue details... STATUS  https://open-confluence.nrao.edu/display/SRDP/VLA+Metadata+Display+Updates

JT: revise to capture what we want about the frequency/setup information

This ticket has multiple points to make the new tool match the Legacy archive and to have the displayed data be much more useful.
High (4.3/4.4)VLA Metadata display updates. https://open-confluence.nrao.edu/display/SRDP/VLA+Metadata+Display+UpdatesThis has many items including 'telescope observing logs'.
Medium

Incorrect metadata sometimes displayed for projects with duplicate proposal IDs between VLA and VLBA

SSA-7318 - Getting issue details... STATUS


MediumVLA/VLBA proposal code degeneracy SSA-5538 - Getting issue details... STATUS

SSA-7318 - Getting issue details... STATUS


MediumIndexing of VLA low band projects

SSA-7187 - Getting issue details... STATUS


√

Subscans table holding both Radian and Degree positions

SSA-5275 - Getting issue details... STATUS

done
√archive search returning incorrect projects SSA-6192 - Getting issue details... STATUS


done





Legacy VLA Data

Priority/Done?NotesJiRA ticket /Confluence page
Criticalcorrupt legacy VLA data in the AAT

SSA-7628 - Getting issue details... STATUS


HighMetadata Issues in Legacy VLA data

SSA-7192 - Getting issue details... STATUS

Needs input from Bryan
HighLegacy VLA data missing error codes in the new archive tool

SSA-7191 - Getting issue details... STATUS

Mapping of quality errors were lost. Stephan will continue following up with Bryan. Blocker for LA retirement.
Highmetadata errors in VLBA (BD114) and old VLA data in new archive SSA-7128 - Getting issue details... STATUS

SSA-7129 - Getting issue details... STATUS

This has been split into two tickets: one VLBA and one VLA
Mediumlegacy VLA metadata error: EBs that claim to have run for 2+ days

SSA-7129 - Getting issue details... STATUS

requires re-ingestion


Data Integrity

Priority/Done?NotesJiRA ticket /Confluence page
BlockerGap Analysis Between Legacy and New Archive SSA-6222 - Getting issue details... STATUS


can be done once RLSW finishes the AIPSFITS files
CriticalAutomated annotations for missing scans/BDF, Automated Error Recognition of Ingested Data

SSA-7190 - Getting issue details... STATUS https://open-confluence.nrao.edu/display/SRDP/Archive+Data+Annotation+Tools


CriticalDisplay of archive ingestion/observation issues SSA-7189 - Getting issue details... STATUS


related to SSA-7190
CriticalIdentification of ingestion problems and automaticallyfix

Example ticket of a silent failure:  SSA-7566 - Getting issue details... STATUS


√create missing db entries for Flag.xml SSA-6273 - Getting issue details... STATUS


Done
√fix ASDM.xml files missing FlagTable entities SSA-6274 - Getting issue details... STATUS


Done


DA Tools for Data Editing

Priority/Done?NotesJiRA ticket /Confluence page
√Tool for setting Proprietary Period of science observations on the VLA and the VLBAhttps://open-confluence.nrao.edu/pages/viewpage.action?spaceKey=SPR&title=Tool+for+setting+Proprietary+Period+of+science+observations+on+the+VLA+and+the+VLBA
CriticalDA annotation tools/Archive data annotation toolshttps://open-confluence.nrao.edu/display/SRDP/Archive+Data+Annotation+Tools


Missing Data

Priority/Done?NotesJiRA ticket /Confluence page
Critical

Tix to be created

May require discussion about how new archive will display images and the uvfits datasets.


HighALMA Standard images into AAT

SSA-7278 - Getting issue details... STATUS


HighLarge project support - ALFALFA + Other collection ingest

SSA-7437 - Getting issue details... STATUS SSA-5712 - Getting issue details... STATUS ALFALFA: SRAO and Collection Support



Other

Priority/Done?NotesJiRA ticket /Confluence pageDetails

Other tickets (so they are not forgotten)

https://open-jira.nrao.edu/issues/?filter=11464

Tickets remaining are listed throughout their respective categories.
Critical

Proposal Contact Author not added to data.nrao.edu database

SSA-7465 - Getting issue details... STATUS

Bug when a project uses different PI and Contact Author
MediumPreserve login between AAT and my.nrao.edu

SSA-7322 - Getting issue details... STATUS


√ (4.2.0)Remove pop-up page when accessing data.nrao.edu

SSA-7502 - Getting issue details... STATUS



ALMA

Priority/Done?NotesJiRA ticket /Confluence pageDetails
CriticalALMA MOUS/calibration ingest unreliable; ALMA butler implementation into production

SSA-7482 - Getting issue details... STATUS



Product Versioning

Priority/Done?NotesJiRA ticket /Confluence pageDetails
CriticalNeed to allow multiple versions of products (images/calibrations) in the archive

Product Versioning and Replacement Design

SSA-6476 - Getting issue details... STATUS
















  • No labels