Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  • It seems the transfer mechanism won't transfer symlinks to directories (e.g. data/vlass.ms → /lustre/aoc/...) Is there a way around this?
    • ANSWER: there is no flag to chase symlinks at the moment.  The top level dir (e.g. data) could be a symlink  may work if transfer_input_files=data/
    • If data is a symlink (e.g. data ../ data ) and transfer_input_files=data then symlink I get the error about won't transfer Transfer of symlinks to directories is not supported.
    • if data is a symlink (e.g. data → ../data ) and transfer_input_files=datasymlink/ then it transfers the contents not the directory.  In other words I don't have a data directory in scratch I have a VLASS... directory.
    • If data/VLASS → /some/path/VLASS and transfer_input_files=data/VLASS/

  • DAG log time stamps,  is there a way to differentiate data import/export time and process run time.
    • Look in the job log file not the dag log file
    • 040 (150.000.000) 2020-06-15 13:05:45 Started transferring input files
              Transferring to host: <10.64.10.172:9618?addrs=10.64.10.172-9618&alias=nmpost072.aoc.nrao.edu&noUDP&sock=slot1_1_72656_7984_60>
      ...
      040 (150.000.000) 2020-06-15 13:06:04 Finished transferring input files

...