Date: Fri, 29 Mar 2024 05:50:22 -0400 (EDT)
Message-ID: <2017448906.30294.1711705822956@atl2.cv.nrao.edu>
Subject: Exported From Confluence
MIME-Version: 1.0
Content-Type: multipart/related;
boundary="----=_Part_30293_86266525.1711705822953"
------=_Part_30293_86266525.1711705822953
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
Content-Location: file:///C:/exported.html
About the NRAO Archive Access Tool
Introduction
The AAT/PPI (Archive Access Tool/Pipeline Processing Interface) is desig=
ned to be a replacement for the NRAO's current archive tool (https://arc=
hive.nrao.edu) that gives astronomers the ability to utilize the NRAO's=
processing clusters to manipulate observation data using CASA: this prelim=
inary release provides the capability to run EVLA and ALMA observations thr=
ough their respective calibration pipelines, additional capabilities will b=
e forthcoming.
Additional features include:
- Two interfaces, basic and advanced.
- Responsive interface that works on mobile devices.
- Natural language (text based) searches of the observation's title, abst=
ract, source names, authors and more, e.g. search on 'nova'.
- The 'my data' feature: login, press a button and see a list of your obs=
ervations.
- If you link your ALMA and NRAO accounts, you can use either to access y=
our observations.
- For ALMA and VLA you can download your SDM as a measurement set and fil=
ter by scan intent.
Testing and Feed=
back
We'd like testers to focus on authentication and authorization: before w=
e allow outside access to this tool we'd like a high degree of confidence w=
e won't be exposing proprietary data through it: we have tested it as well =
as we could but feel it could benefit from broader testing.
Any request for data during the proprietary period requires authenticati=
on and authorization: you need to log in and be attached to the observation=
, this goes for both download and reprocessing requests.
Any re-processing request requires authentication (we want to keep anony=
mous users from tying up cluster resources), if the data is in the propriet=
ary period it also requires authorization. ALMA re-processing requires the =
user have an ALMA account, VLA reprocessing requires the user have an NRAO =
account
Successful requests result in the files being staged for download and th=
e URL for them given to the user: this URL can't be guessed by other users,=
it is the key to the files. To share the files with other users, give them=
the URL. The URL will be active for at least five days after the request, =
though we will have to adjust that in the future based on storage space and=
demand.
- Users without special privileges should try to access sets they shouldn=
't be able to.
- Users should try to make sure they can access things they should be abl=
e to.
Please report your feedback to Mark Lacy (mlacy@nrao.edu), Claire Chandler=
(cchandle@nrao.edu) and Stephan Witz (switz@nrao.edu).
Known Issues
- Stale search indexes: the ALMA search index is re-built daily, so it sh=
ould never be more than a day out of date: the VLA/VLBA/GBT search index is=
built less frequently and can be out of date by a week or more. We are wor=
king on it.
- Missing fields: the NRAO metadata database which we use to populate the=
search index spans four decades of observations, some of the observations =
have missing fields like array configuration or start and stop dates. We ha=
ve no evidence of incorrect fields, just missing fields. There is no easy f=
ix for the issue, we will have to extract each affected observation and upd=
ate the database from the metadata in the files, we will start that after t=
he software is released.
- Poor error feedback: there are many kinds of failures possible (observa=
tions that can't run through the calibration pipeline, observations that ar=
e missing some files, permissions errors writing files to user specified un=
-writable locations and so on), the AAT/PPI doesn't currently let the user =
know why something failed, and sometimes it won't let the user know it fail=
ed at all, requests never seem to complete. This is something we are active=
ly working on.
- At this time only the calibration and flagging tables are downloadable =
after reprocessing, not the full calibrated measurement set.
- ALMA Cycle 0 and Cycle 1 observations are known to fail reprocessing, a=
s are VLA observations before 2013, there is currently no feedback to the u=
ser that is the case. Users are advised to avoid reprocessing on these sets=
.
- GBT data delivery format: the SDFITS format for GBT data is not yet sup=
ported. GBT data can only be downloaded as a set of GBT-FITS files.
- The distributed applications that make up the AAT/PPI are fragile to da=
tabase downtime, they don't recover well from it yet and have to be restart=
ed to resume functioning: this is exacerbated by the fact that the system r=
elies on several databases to work, not all of which are under the team's c=
ontrol. We are working to reduce the number of databases and to also make t=
he system more robust.
Relea=
se Notes - 2.5.2 - 2017-05-04
Feature
Bug
Relea=
se Notes - 2.5.3 - 2017-05-12
Engineering Task
- [SSA-401=
0] - Improve reliability of workflow-job signalling
Bug
- [SSA-400=
6] - problems under load/maui kills jobs
Release Notes - 2.5.4 - 2017-05-26
Feature
- [SSA-387=
8] - missing feature: NED/SIMBAD name resolution
- [SSA-387=
9] - missing feature: different formats for source position input
- [SSA-400=
3] - input position as HH:MM:SS.SSS
Bug
------=_Part_30293_86266525.1711705822953--