Coverage for python/lsst/images/ndf/__init_

1# This file is part of lsst-images.

3# Developed for the LSST Data Management System.

4# This product includes software developed by the LSST Project

5# (https://www.lsst.org).

6# See the COPYRIGHT file at the top-level directory of this distribution

7# for details of code ownership.

9# Use of this source code is governed by a 3-clause BSD-style

10# license that can be found in the LICENSE file.

13"""Archive implementations that write HDS-on-HDF5 files compatible with the

14Starlink N-Dimensional Data Format (NDF) data model.

16This is an *experimental* HDF5 serialization backend to demonstrate that the

17package can be used to write files in formats other than FITS.

18It does write files in HDF5 format that can be read by the

19`Starlink software <https://starlink.eao.hawaii.edu/starlink>`__.

20We do not recommend you use it in a production environment at this time to

21create archival products.

23Files written by this archive are valid NDF files readable by

24applications and libraries from the

25`Starlink Software Collection <https://starlink.eao.hawaii.edu>`_

26(KAPPA, ``hdstrace``, CUPID, etc.) although the LSST data model adds

27extensions that are not known to the Starlink tooling.

29The NDF data model is described in

30`Jenness et al (2015) <https://doi.org/10.1016/j.ascom.2014.11.001>`_.

32For a `lsst.images.MaskedImage` the data and variance arrays map to the

33standard NDF equivalents.

34Any FITS headers are stored in the standard ``.MORE.FITS`` extension and

35the WCS is stored as a serialized AST FrameSet, modified to include a ``GRID``

36frame in addition to a ``PIXEL`` frame, in the ``.WCS`` component.

37Starlink tools only allow 8-bit masks so the mask is stored directly as a

38``QUALITY`` component when only 8 bits are needed but if more bits are

39required the full mask is stored in the ``.MORE.LSST.MASK`` extension as

40a 3-D unsigned byte NDF where the third dimension matches the internal

41representation of the 3-D mask. The ``QUALITY`` component includes a collapsed

42version of the full mask to enable Starlink software to apply a coarse mask.

43In the future the collapsing is intending to be more granular to allow mask

44planes to be combined based on their related concepts.

45Since the NDF data model has a subset that is compatible with the

46`~lsst.images.MaskedImage` data model, an NDF can be read as a

47`~lsst.images.MaskedImage` with full data, variance, mask, WCS, and FITS

48header support, although of course any extensions will be ignored.

50For more complex image types such as `lsst.images.VisitImage` the LSST

51components are stored in the ``.MORE.LSST`` extension, including the ``JSON``

52component containing the Pydantic models. Data arrays are represented as

53NDFs in the ``.MORE.LSST`` extension and referenced by path from the JSON.

54Tabular data is not supported by the NDF data model and is not currently

55representable. It is expected that they would become individual columns

56in an extension, all with the same length.

58These files use HDF5 format and can be read by any HDF5 tooling such as

59``h5dump`` or the Python ``h5py``, although the data model includes

60group attributes to allow it to be read by the Starlink libraries

61that add additional semantic meaning to data structures.

62The HDS-on-HDF5 format is described in

63`Jenness (2015) <https://doi.org/10.1016/j.ascom.2015.02.003>`_.

65HDF5 files are not generally usable for remote access of components through

66cloud storage. There is no facility to store byte offsets to components in

67any of the data structures.

68"""

70try:

71 import h5py # noqa: F401

72except ImportError as e:

73 raise ImportError(

74 "lsst.images.ndf requires the optional 'h5py' package. "

75 "Install it directly or via 'pip install lsst-images[ndf]'."

76 ) from e

78from ._common import *

79from ._input_archive import *

80from ._output_archive import *

Coverage for python/lsst/images/ndf/init.py: 71%

7 statements