Imaging weblog is not untar/zipped in products directory

Issue


NameDescriptionWhoTicket(s)StatusDue

10

Proprietary iconIcons for proprietary data not displaying properly (though data is properly protected)Stephan

SSA-5628

SSA-5559

SSA-5628 - Minor issue

SSA-5559 - Minor issue

8/20 - Address with RH retirement?  Need a service?  RH message error is ambiguous, could lead to confusion;

10/09 - This is a more long term issue we've agreed to address later, Stephan owns it not Richard



M6Scripts

SSA-5599, VLA implemented, ALMA in progress,

1) Script to notify Operations manager and DAs when imaging pipeline job is completed and ready for review: status - already exists

2) Script to rerun the imaging pipeline after edits to the PPR and/or extra flagging and/or changes to continuum subtraction: status - not yet done, but could be adapted from existing VLA recaldirs script.

3) Script to ingest image products into the archive and clean up the pipeline directory: status - not yet done (needs product system)

Stephan/Jim

SSA-5599

dependent on SSA-5330

SSA-5393?

7/29 - Are there critical tasks in this epic, or do they all need to be done?

8/6 - 14 tix, 3 closed/resolved, 4 ready for validation, 2 ready/under verification, 4 scheduled

Cube has ancillary prod, need to ingest cube with these products, have to hook this together and ingest a cube, check that DF works,

   notification - 8/8

   re-run - 8/8

   ingest - later (when?), SSA-5632?

9/17 - Stephan to find ticket number...

9/25 -

10/2 - Jim - thumbnail in hand, in the thick of it;  for testing: Mark, L back on the 15th; discussing full deploment to Test next week, DMS will do intitial testing


13Uncaught errorWorkflows for Optimized imaging and ALMA/VLA restores need to parse CASA log for unreported errors:  mismatch on number of rows is not caught after restore, leading to errors.Rick, JanetSSA-5846 (B)

9/12 - The ticket involves parsing the CASA command logs to look for an error condition that doesn't cause CASA to fail but should, my guess is that is a day or two of work. We'll try to fold that in next week.

9/25 - don't think it will be difficult; buys some time on #14

10/2 - John will provide a reproducable test case, Further discussion in AOB.

10/23 - any progress? - part of the Nov 1 release (will be there)

10/30 - ticket status unchanged...

11/13 - Plan to deploy software to test and fix this while testing.

12/9 - User is not being notified of AUDI success or failure. - to be in 3.7.1 bug fix release


14ALMA calibrations, CASA versionsNeed to know which version of CASA should be used when restoring.Stephan

SSA-5519

 (Hold for Wave 2?)

9/12 - This issue about tracking the version of CASA used to make a calibration and making sure we use that version when we do a restore.  We danced around that before, tried to put it off, but it sounds like we can't do that now. It is less trivial, the issue denoted is a research request on how to get the metadata we need, that ticket should be closed out now: we know there is no clever way to do it, so that leaves the un-clever ways. We don't trust the data in the ALMA metadata database, meaning we'll need to crack open the calibrations and harvest what we need from the manifest and/or the logs.

9/25 - Workaround?  Need some stakeholder guidance, decompose problem into parts, cost estimate

10/2 - No action this week due to integration - Do #13, move this as "metadata improvements" to Wave 2



15Missing ASDM binary"In further exploration of SSA-5846 and SSA-5519 I discovered that the failing restores nearly always have missing ASDM binary files."DanielSSA-5853 (B)

9/17 - looks like this was related to a space issue on the machine?  Need to catch error when it happens.... Short term - catch and report error, long term - get away from using asdmexport_lite

9/25 - looking for a reliable test case (real missing file in the archive)? - ask Catarina

10/2 - Didn't find an example, testing will need to work around lack of an example by hook or crook; #13 will mitigate this one also.  Need to figure out how to create ".missing" file.

10/3 - Example provided

10/23 - Under verification, ETA? - part of Nove 1 release

10/30, 11/13, 12/11 - ticket status hasn't changed....




17Proprietary Identification (Thumbnails)Public/proprietary data state not properly tracked for ALMAStephanSSA-5925 (B)

10/23 - Discussions in progress, could display data release date for MOUS

10/30 - In verification, on webtest

11/13 - Status?

12/11 - was there a workaround, did this get moved to Wave 2?



1819A-306 - Unable to download dataPermission error when trying to download this dataset.DanielSSA-5934 (B)11/13, 12/11 - Status?


19Weblog format (ALMA imaging)

Imaging weblog is not untar/zipped in products directory for ALMA user initiated imaging (as it is for VLA calibration)

RickSSA-5918







Issue


NameDescriptionWhoTicket(s)StatusDue
Closed issues:





1spwPipeline needs to know what the archive knows about spw metadata

John

Jim

Joe

SSA-5483

PIPE-46

Found virtual spw ID in ALMA db, carrying it in archive

4/16 - use spw as param to editimlist

4/23 - PIPE-46 under validation, SSA-5483 closed

4/24
2special characterssome characters in field name need quotes, e.g. contain “.” or similar characters.

Joe

John

PIPE-2

SSA-5482

PIPE-321

"The stated requirements have been implemented (but require some minor updates to the PPR template to be complete). However, further testing by the pipeline group as found a potential issue with the double quotes.  Awaiting results from the pipeline group before closing the ticket."

decision - handle in PL code, workflow will put (appropriately encoded) XML in field, pass to PL

4/16 - will handle inside PL, Joe will implement handling of special char, Kana will take triple bracket out of recipe and verify

4/23 - SSA-5482 closed, PIPE-2 under validation, PIPE-321 created for special char, resolved

4/30
3spw for CLIDo we have alignment on the spw parameter for the CLI?Jim

SSA-5175

SSA-5507

1) Which of the parameters will be allowed to be blank? 2) Will these cubes span multiple SPWs, or will the interface (and you for the CLI) limit things to a single one?

4/16 - Morgan to talk with Jim to get an update

4/23 - CLI close to ready to test, need doc



4exporting metadataexport metadata from PL to outsideKanaPIPE-100

Need to confirm requirements, where to put the md.

4/9 - all in manifest;  Remy has a question on data processing date

4/16 - from cal PL, info should come from SSA workflow system - check with Stephan and Jeff, may be able to close.

4/23 - Resolved, data will come from the workflow system

4/23
8Estimate for remaining pilot workNeed estimates for the remaining pilot scope

Morgan

Stephan

Kana


6/4 - Need to review this with SSA quarterly planning

8/20 - See SRDP-410




5RMS in AQUA report

John

Kana

PIPE-44

4/9 - RMS should be in the AQUA report, representative source and spw resolution for user specified bandwidth;  if SRDP needs something different, we will need to negotiate with ALMA;  need to specify which RMS

4/22 - Kana working on it

8/20 - Closed
6Priority of follow up ticketsJohn requested priority focus on PIPE-99 and SRDP-275

John

Kana

Morgan

PIPE-99

SRDP-275

CASR-430

PIPE-322


4/15 - CASR ticket and new PIPE ticket to be opened for PIPE-99.  (CASR-430)  This (heuristics development) work won't be done before start of pilot, but probably by start of Wave 1

4/23 - PIPE-322 created for fluxboot2 hueristic implementation (PIPE-99 follow-up).  Sched in May.

SRDP-275, "Reference antenna prioritization should be recalculated after hifv_flagbaddef/hifv_checkflag,"  PIPE-68?

6/4 - PIPE-322 actively testing (John/Brian)

8/20 - PIPE-99 Done, PIPE-322 Closed, CASR-430 Done; other issues are Wave 2

8/20 -

Done


7Timing for scriptsWe need to develop scripts as a workaround, as the workspace is not ready yet.  When will these be availableStephan

6/4 - The first two scripts necessary for pilot have been delivered, on has been tested; for the other, testing is in progress.

8/20 - See M6

8/20 -

Closing issue (duplicate)

9Missing EB's

Stephan

John


6/4 - See "Missing EB's" page

8/20 - Wrapped up for the moment, check for recurrance

8/20 -


12SPW selectionSpectral window id selection is incorrect in ALMA archive.KanaPIPE-441

9/3 - Spw selections in ALMA cubes - ALMA archive table is incorrect, algorithm to define spw id in archive and sdm different -Options:

1) ALMA Archive suggested another table, could change, not confirmed yet

2)  Support spw selection by name, not supported by hif_editimlist -

    1. If name vs id, changes needed to workflow also

9/17 - Resolved



Proprietary identificationPublic/proprietary data state not properly tracked for ALMA
SSA-5561, etc.

16Spectral Window NameDue to inaccurate metadata in the ALMA database, the ALMA Optimized imaging must switch from using SPW number to SPW NAME.Jim

SSA-5837 (B)


9/17 - Related to PIPE-441, broken into subtasks, changes just in, we'll see what breaks, mid-process

9/25 - Verifying, still need to do E2E test?

10/2 - Pending other testing?  "UI to generating products" ready for testing, full deployment to test next week

(A) Stephan to ping John for version of CASA and path when doing the deploy

10/23 - Resolved



11PlanningPlan work for next quarterStephanSRDP-410

8/23 - The tickets are in place up through the design of the workspace, once we decompose that we’ll make the rest of them

9/12 - Reviewed with Jeff, workspace planning TBD

10/2 - Scheduling Workspace review for 10/24 or 10/25, workspace task plan will be included;  Nov-Jan planning - SSA has all the info needed for planning, decomposing workspace, may need input from stakeholders the last week of October.

10/23 - Closed


8/10
  • No labels