Metadata files for the previous day's GBT observing are
     are created (Melinda) and copied into a single directory
     by the GBT.

     The metadata files are in /data1/gbt-archive/AAT (vegas-1) or
     may be accessed as archmgr@zuul01.cv.nrao.edu at /home/gbt-archive/AAT.

     The daily metadata directories look like this :
     GBTArchive_2015-07-21T06:30:01.csv_aatexport..

     And contain files whose format nearly matches the Archive DB table
     schema.. The daily files are written at about 06:30 ET.

     A cronjob at archmgr@aocngas-master executes
     /home/archive.nrao.edu/bin/autocopyGBTmetadatafiles at 0600 MT

     It copies the new metadatafile sets from CV to the AOC.  It
     cross checks the CV daily file sets against the AOC copies..
     Generally it only finds that yesterday's filesets are missing in AOC.
     The copies are executed like so (archmgr@aocngas-master):
     scp -r archmgr@zuul01.cv.nrao.edu:/home/gbt-archive/AAT/$cv_dirname
/home/ngas_staging/gbt_metadata;

    Now after the metadata files are copied from CV to the AOC:

     a cronjob at e2emgr@aocngas-master at 0630 MT ..
    /home/archive.nrao.edu/bin/autoingestGBTintoArchive_aocngas-master
    finds the new filesets loaded in /home/ngas_staging/gbt_metadata
    at AOC

    /home/archive.nrao.edu/bin/autoingestGBTintoArchive_aocngas-master
    creates sql insert files to load the GBT metadata into the Archive DB.
    The sql files are written into :
    /home/archive/e2e/archive/catalogs/NRAO/sqlscripts like
    GBTAGBT14B_339_71.sql

    GBT Metadata Ingestion Schedule :

    06:30am ET  - daily metadata -> /data1/gbt-archive/AAT  (CV)

    08:00am MT  - copy daily metadata frpm CV to AOC
    06:00am MT  -> /home/ngas_staging/gbt_metadata

    06:30am MT  - ingest new metadata into Archive DB
                  sql inserts executed


    GBT Data File Ingestion into NGAS Schedule :

    12:00am ET  - start autoingestGBTintoNGAS_hostvegas1
    10:00am MT


  • No labels