Imaging weblog is not untar/zipped in products directory
Issue | Name | Description | Who | Ticket(s) | Status | Due |
---|---|---|---|---|---|---|
10 | Proprietary icon | Icons for proprietary data not displaying properly (though data is properly protected) | Stephan | SSA-5628 - Minor issue SSA-5559 - Minor issue 8/20 - Address with RH retirement? Need a service? RH message error is ambiguous, could lead to confusion; 10/09 - This is a more long term issue we've agreed to address later, Stephan owns it not Richard | ||
M6 | Scripts | SSA-5599, VLA implemented, ALMA in progress, 1) Script to notify Operations manager and DAs when imaging pipeline job is completed and ready for review: status - already exists 2) Script to rerun the imaging pipeline after edits to the PPR and/or extra flagging and/or changes to continuum subtraction: status - not yet done, but could be adapted from existing VLA recaldirs script. 3) Script to ingest image products into the archive and clean up the pipeline directory: status - not yet done (needs product system) | Stephan/Jim | SSA-5393? | 7/29 - Are there critical tasks in this epic, or do they all need to be done? 8/6 - 14 tix, 3 closed/resolved, 4 ready for validation, 2 ready/under verification, 4 scheduled Cube has ancillary prod, need to ingest cube with these products, have to hook this together and ingest a cube, check that DF works, notification - 8/8 re-run - 8/8 ingest - later (when?), SSA-5632? 9/17 - Stephan to find ticket number... 9/25 - 10/2 - Jim - thumbnail in hand, in the thick of it; for testing: Mark, L back on the 15th; discussing full deploment to Test next week, DMS will do intitial testing | |
13 | Uncaught error | Workflows for Optimized imaging and ALMA/VLA restores need to parse CASA log for unreported errors: mismatch on number of rows is not caught after restore, leading to errors. | Rick, Janet | SSA-5846 (B) | 9/12 - The ticket involves parsing the CASA command logs to look for an error condition that doesn't cause CASA to fail but should, my guess is that is a day or two of work. We'll try to fold that in next week. 9/25 - don't think it will be difficult; buys some time on #14 10/2 - John will provide a reproducable test case, Further discussion in AOB. 10/23 - any progress? - part of the Nov 1 release (will be there) 10/30 - ticket status unchanged... 11/13 - Plan to deploy software to test and fix this while testing. 12/9 - User is not being notified of AUDI success or failure. - to be in 3.7.1 bug fix release | |
14 | ALMA calibrations, CASA versions | Need to know which version of CASA should be used when restoring. | Stephan | (Hold for Wave 2?) | 9/12 - This issue about tracking the version of CASA used to make a calibration and making sure we use that version when we do a restore. We danced around that before, tried to put it off, but it sounds like we can't do that now. It is less trivial, the issue denoted is a research request on how to get the metadata we need, that ticket should be closed out now: we know there is no clever way to do it, so that leaves the un-clever ways. We don't trust the data in the ALMA metadata database, meaning we'll need to crack open the calibrations and harvest what we need from the manifest and/or the logs. 9/25 - Workaround? Need some stakeholder guidance, decompose problem into parts, cost estimate 10/2 - No action this week due to integration - Do #13, move this as "metadata improvements" to Wave 2 | |
15 | Missing ASDM binary | "In further exploration of SSA-5846 and SSA-5519 I discovered that the failing restores nearly always have missing ASDM binary files." | Daniel | SSA-5853 (B) | 9/17 - looks like this was related to a space issue on the machine? Need to catch error when it happens.... Short term - catch and report error, long term - get away from using asdmexport_lite 9/25 - looking for a reliable test case (real missing file in the archive)? - ask Catarina 10/2 - Didn't find an example, testing will need to work around lack of an example by hook or crook; #13 will mitigate this one also. Need to figure out how to create ".missing" file. 10/3 - Example provided 10/23 - Under verification, ETA? - part of Nove 1 release 10/30, 11/13, 12/11 - ticket status hasn't changed.... | |
17 | Proprietary Identification (Thumbnails) | Public/proprietary data state not properly tracked for ALMA | Stephan | SSA-5925 (B) | 10/23 - Discussions in progress, could display data release date for MOUS 10/30 - In verification, on webtest 11/13 - Status? 12/11 - was there a workaround, did this get moved to Wave 2? | |
18 | 19A-306 - Unable to download data | Permission error when trying to download this dataset. | Daniel | SSA-5934 (B) | 11/13, 12/11 - Status? | |
19 | Weblog format (ALMA imaging) | Imaging weblog is not untar/zipped in products directory for ALMA user initiated imaging (as it is for VLA calibration) | Rick | SSA-5918 | ||
Issue | Name | Description | Who | Ticket(s) | Status | Due |
---|---|---|---|---|---|---|
Closed issues: | ||||||
1 | spw | Pipeline needs to know what the archive knows about spw metadata | John Jim Joe | Found virtual spw ID in ALMA db, carrying it in archive 4/16 - use spw as param to editimlist 4/23 - PIPE-46 under validation, SSA-5483 closed | 4/24 | |
2 | special characters | some characters in field name need quotes, e.g. contain “.” or similar characters. | Joe John | PIPE-321 | "The stated requirements have been implemented (but require some minor updates to the PPR template to be complete). However, further testing by the pipeline group as found a potential issue with the double quotes. Awaiting results from the pipeline group before closing the ticket." decision - handle in PL code, workflow will put (appropriately encoded) XML in field, pass to PL 4/16 - will handle inside PL, Joe will implement handling of special char, Kana will take triple bracket out of recipe and verify 4/23 - SSA-5482 closed, PIPE-2 under validation, PIPE-321 created for special char, resolved | 4/30 |
3 | spw for CLI | Do we have alignment on the spw parameter for the CLI? | Jim | 1) Which of the parameters will be allowed to be blank? 2) Will these cubes span multiple SPWs, or will the interface (and you for the CLI) limit things to a single one? 4/16 - Morgan to talk with Jim to get an update 4/23 - CLI close to ready to test, need doc | ||
4 | exporting metadata | export metadata from PL to outside | Kana | PIPE-100 | Need to confirm requirements, where to put the md. 4/9 - all in manifest; Remy has a question on data processing date 4/16 - from cal PL, info should come from SSA workflow system - check with Stephan and Jeff, may be able to close. 4/23 - Resolved, data will come from the workflow system | 4/23 |
8 | Estimate for remaining pilot work | Need estimates for the remaining pilot scope | Morgan Stephan Kana | 6/4 - Need to review this with SSA quarterly planning 8/20 - See SRDP-410 | ||
5 | RMS in AQUA report | John Kana | PIPE-44 | 4/9 - RMS should be in the AQUA report, representative source and spw resolution for user specified bandwidth; if SRDP needs something different, we will need to negotiate with ALMA; need to specify which RMS 4/22 - Kana working on it | 8/20 - Closed | |
6 | Priority of follow up tickets | John requested priority focus on PIPE-99 and SRDP-275 | John Kana Morgan | 4/15 - CASR ticket and new PIPE ticket to be opened for PIPE-99. (CASR-430) This (heuristics development) work won't be done before start of pilot, but probably by start of Wave 1 4/23 - PIPE-322 created for fluxboot2 hueristic implementation (PIPE-99 follow-up). Sched in May. SRDP-275, "Reference antenna prioritization should be recalculated after hifv_flagbaddef/hifv_checkflag," PIPE-68? 6/4 - PIPE-322 actively testing (John/Brian) 8/20 - PIPE-99 Done, PIPE-322 Closed, CASR-430 Done; other issues are Wave 2 | 8/20 - Done | |
7 | Timing for scripts | We need to develop scripts as a workaround, as the workspace is not ready yet. When will these be available | Stephan | 6/4 - The first two scripts necessary for pilot have been delivered, on has been tested; for the other, testing is in progress. 8/20 - See M6 | 8/20 - Closing issue (duplicate) | |
9 | Missing EB's | Stephan John | 6/4 - See "Missing EB's" page 8/20 - Wrapped up for the moment, check for recurrance | 8/20 - | ||
12 | SPW selection | Spectral window id selection is incorrect in ALMA archive. | Kana | PIPE-441 | 9/3 - Spw selections in ALMA cubes - ALMA archive table is incorrect, algorithm to define spw id in archive and sdm different -Options: 1) ALMA Archive suggested another table, could change, not confirmed yet 2) Support spw selection by name, not supported by hif_editimlist -
9/17 - Resolved | |
Proprietary identification | Public/proprietary data state not properly tracked for ALMA | SSA-5561, etc. | ||||
16 | Spectral Window Name | Due to inaccurate metadata in the ALMA database, the ALMA Optimized imaging must switch from using SPW number to SPW NAME. | Jim | SSA-5837 (B) | 9/17 - Related to PIPE-441, broken into subtasks, changes just in, we'll see what breaks, mid-process 9/25 - Verifying, still need to do E2E test? 10/2 - Pending other testing? "UI to generating products" ready for testing, full deployment to test next week (A) Stephan to ping John for version of CASA and path when doing the deploy 10/23 - Resolved | |
11 | Planning | Plan work for next quarter | Stephan | SRDP-410 | 8/23 - The tickets are in place up through the design of the workspace, once we decompose that we’ll make the rest of them 9/12 - Reviewed with Jeff, workspace planning TBD 10/2 - Scheduling Workspace review for 10/24 or 10/25, workspace task plan will be included; Nov-Jan planning - SSA has all the info needed for planning, decomposing workspace, may need input from stakeholders the last week of October. 10/23 - Closed | 8/10 |