Introduction

JACAL integrates Yandasoft (previously known as ASKAPSoft) and the execution framework DALiuGE. A shared library offers a calling convention supported by DALiuGE and internally links and reuses Yandasoft code. JACAL is freely available in GitLab under a variation of the open source BSD 3-Clause [License](LICENSE). The repository contains the following:

  • The C/C++ code of the shared library libjacal.so described above.

  • A number of tests running the different components inside DALiuGE graphs.

  • A standalone utility for library testing independent of DALiuGE.

DALIuGE apps

The way JACAL integrates Yandasoft into DALiuGE is by wrapping individual pieces of functionality into DALiuGE-compatible applications that can then be deployed on a DALiuGE graph.

DALiuGE is an execution framework where programs are expressed as directed acyclic graphs, with nodes representing not only the different computations performed on the data as it flows through the graph, but also the data itself. Both types of nodes are termed drops. Computation drops (in DALiuGE, application drops) read or receive data from their input data drops, and write the results into their output data drops. Data drops on the other hand are storage-agnostic and host-agnostic, meaning that regardless of underlying storage and location application drops can work with their inputs and outputs in the same way.

Although application drops can be implemented in many ways, DALiuGE offers out-of-the-box support for certain type of applications. Among those, shared libraries can be written by users to implement application drops. This capability allows reusing code written in C, C++ or other low-level languages to work as application drops in a DALiuGE graph.

Using Yandasoft in DALiuGE

Before JACAL, the only way to use the Yandasoft functionality was to invoke the binaries it generates (e.g., cimager, cbpcalibrator, etc.); composition was only possible by arranging pipelines using shell scripts and similar techniques, and with data having to touch disk between each invocation of the binaries.

JACAL on the other hand implements a shared library (i.e., libjacal.so) wrapping different parts of Yandasoft as DALiuGE-ready application drops. This makes it possible to reuse finer-grained pieces of functionality from the Yandasoft code base, and with data not having to be necessarily written to disk between these steps.