lsst.obs.base  16.0-14-g71e547a+1
Public Member Functions | Public Attributes | Static Public Attributes | List of all members
lsst.obs.base.gen3.ingest.VisitInfoRawIngestTask Class Reference
Inheritance diagram for lsst.obs.base.gen3.ingest.VisitInfoRawIngestTask:
lsst.obs.base.gen3.ingest.RawIngestTask

Public Member Functions

def makeVisitInfo (self, headers, exposureId)
 
def extractVisitEntry (self, file, headers, dataId, associated)
 
def extractExposureEntry (self, file, headers, dataId, associated)
 
def getDatasetType (cls)
 
def run (self, files)
 
def readHeaders (self, file)
 
def ensureDataUnits (self, file)
 
def ingestFile (self, file, headers, dataId, run=None)
 
def processFile (self, file)
 
def extractDataId (self, file, headers)
 
def getFormatter (self, file, headers, dataId)
 

Public Attributes

 butler
 
 datasetType
 
 units
 
 unitEntryCache
 
 stashRun
 

Static Public Attributes

 ConfigClass = RawIngestConfig
 

Detailed Description

An intermediate base class of RawIngestTask for cameras that already
implement constructing a `afw.image.VisitInfo` object from raw data.

Subclasses must provide (at least) implementations of `extractDataId` and
the new `makeVisitInfo` method; the latter is used to provide concrete
implementations of `extractVisitEntry` and `extractExposureEntry`.

Definition at line 435 of file ingest.py.

Member Function Documentation

◆ ensureDataUnits()

def lsst.obs.base.gen3.ingest.RawIngestTask.ensureDataUnits (   self,
  file 
)
inherited
Extract metadata from a raw file and add Exposure and Visit
DataUnit entries.

Any needed Camera, Sensor, and PhysicalFilter DataUnit entries must
exist in the Registry before `run` is called.

Parameters
----------
file : `str` or path-like object
    Absolute path to the file to be ingested.

Returns
-------
headers : `list` of `~lsst.daf.base.PropertyList`
    Result of calling `readHeaders`.
dataId : `dict`
    Data ID dictionary, as returned by `extractDataId`.

Definition at line 197 of file ingest.py.

◆ extractDataId()

def lsst.obs.base.gen3.ingest.RawIngestTask.extractDataId (   self,
  file,
  headers 
)
inherited
Return the Data ID dictionary that should be used to label a file.

Parameters
----------
file : `str` or path-like object
    Absolute path to the file being ingested (prior to any transfers).
headers : `list` of `~lsst.daf.base.PropertyList`
    All headers returned by `readHeaders()`.

Returns
-------
dataId : `dict`
    Must include "camera", "sensor", and "exposure" keys. If the
    Exposure is associated with a PhysicalFilter and/or Visit,
    "physical_filter" and "visit" keys should be provided as well
    (respectively).

Definition at line 343 of file ingest.py.

◆ extractExposureEntry()

def lsst.obs.base.gen3.ingest.VisitInfoRawIngestTask.extractExposureEntry (   self,
  file,
  headers,
  dataId,
  associated 
)
Create an Exposure DataUnit entry from raw file metadata.

Parameters
----------
file : `str` or path-like object
    Absolute path to the file being ingested (prior to any transfers).
headers : `list` of `~lsst.daf.base.PropertyList`
    All headers returned by `readHeaders()`.
dataId : `dict`
    The data ID for this file.  Implementations are permitted to
    modify this dictionary (generally by stripping off "sensor" and
    adding new metadata key-value pairs) and return it.
associated : `dict`
    A dictionary containing other associated DataUnit entries.
    Guaranteed to have "Camera", "Sensor", "PhysicalFilter", and
    "Visit" keys, but the latter two may map to ``None`` if
    `extractDataId` did not contain keys for these or mapped them to
    ``None``.  May also contain additional keys added by
    `extractVisitEntry`.

Returns
-------
entry : `dict`
    Dictionary corresponding to an Exposure database table row.
    Must have all non-null columns in the Exposure table as keys.

Definition at line 490 of file ingest.py.

◆ extractVisitEntry()

def lsst.obs.base.gen3.ingest.VisitInfoRawIngestTask.extractVisitEntry (   self,
  file,
  headers,
  dataId,
  associated 
)
Create a Visit DataUnit entry from raw file metadata.

Parameters
----------
file : `str` or path-like object
    Absolute path to the file being ingested (prior to any transfers).
headers : `list` of `~lsst.daf.base.PropertyList`
    All headers returned by `readHeaders()`.
dataId : `dict`
    The data ID for this file.  Implementations are permitted to
    modify this dictionary (generally by stripping off "sensor" and
    "exposure" and adding new metadata key-value pairs) and return it.
associated : `dict`
    A dictionary containing other associated DataUnit entries.
    Guaranteed to have "Camera", "Sensor",  and "PhysicalFilter" keys,
    but the last may map to ``None`` if `extractDataId` either did not
    contain a "physical_filter" key or mapped it to ``None``.
    Also adds a "VisitInfo" key containing an `afw.image.VisitInfo`
    object for use by `extractExposureEntry`.

Returns
-------
entry : `dict`
    Dictionary corresponding to an Visit database table row.
    Must have all non-null columns in the Visit table as keys.

Definition at line 457 of file ingest.py.

◆ getDatasetType()

def lsst.obs.base.gen3.ingest.RawIngestTask.getDatasetType (   cls)
inherited
Return the DatasetType of the Datasets ingested by this Task.

Definition at line 123 of file ingest.py.

◆ getFormatter()

def lsst.obs.base.gen3.ingest.RawIngestTask.getFormatter (   self,
  file,
  headers,
  dataId 
)
inherited
Return the Formatter that should be used to read this file after
ingestion.

The default implementation returns None, which uses the formatter
configured for this DatasetType/StorageClass in the Butler.

Definition at line 425 of file ingest.py.

◆ ingestFile()

def lsst.obs.base.gen3.ingest.RawIngestTask.ingestFile (   self,
  file,
  headers,
  dataId,
  run = None 
)
inherited
Ingest a single raw file into the repository.

All necessary DataUnit entres must already be present.

This method is not transactional; it must be wrapped in a
``with self.butler.transaction` block to make per-file ingest
atomic.

Parameters
----------
file : `str` or path-like object
    Absolute path to the file to be ingested.
headers : `list` of `~lsst.daf.base.PropertyList`
    Result of calling `readHeaders`.
dataId : `dict`
    Data ID dictionary, as returned by `extractDataId`.
run : `~lsst.daf.butler.Run`, optional
    Run to add the Dataset to; defaults to ``self.butler.run``.

Definition at line 266 of file ingest.py.

◆ makeVisitInfo()

def lsst.obs.base.gen3.ingest.VisitInfoRawIngestTask.makeVisitInfo (   self,
  headers,
  exposureId 
)
Return an `afw.image.VisitInfo` object from the given header and ID.

Parameters
----------
headers : `list` of `~lsst.daf.base.PropertyList`
    All headers returned by `readHeaders()`.
exposureId : `int`
    Integer ID to pass to the `VisitInfo` constructor.

Definition at line 445 of file ingest.py.

◆ processFile()

def lsst.obs.base.gen3.ingest.RawIngestTask.processFile (   self,
  file 
)
inherited
Ingest a single raw data file after extacting metadata.

This creates any new Exposure or Visit DataUnit entries needed to
identify the ingest file, creates a new Dataset entry in the
Registry and finally ingests the file itself into the Datastore.
Any needed Camera, Sensor, and PhysicalFilter DataUnit entries must
exist in the Registry before `run` is called.

Parameters
----------
file : `str` or path-like object
    Absolute path to the file to be ingested.

Definition at line 307 of file ingest.py.

◆ readHeaders()

def lsst.obs.base.gen3.ingest.RawIngestTask.readHeaders (   self,
  file 
)
inherited
Read and return any relevant headers from the given file.

The default implementation simply reads the header of the first
non-empty HDU, so it always returns a single-element list.

Parameters
----------
file : `str` or path-like object
    Absolute path to the file to be ingested.

Returns
-------
headers : `list` of `~lsst.daf.base.PropertyList`
    Single-element list containing the header of the first
    non-empty HDU.

Definition at line 178 of file ingest.py.

◆ run()

def lsst.obs.base.gen3.ingest.RawIngestTask.run (   self,
  files 
)
inherited
Ingest files into a Butler data repository.

This creates any new Exposure or Visit DataUnit entries needed to
identify the ingested files, creates new Dataset entries in the
Registry and finally ingests the files themselves into the Datastore.
Any needed Camera, Sensor, and PhysicalFilter DataUnit entries must
exist in the Registry before `run` is called.

Parameters
----------
files : iterable over `str` or path-like objects
    Paths to the files to be ingested.  Will be made absolute
    if they are not already.

Definition at line 148 of file ingest.py.

Member Data Documentation

◆ butler

lsst.obs.base.gen3.ingest.RawIngestTask.butler
inherited

Definition at line 131 of file ingest.py.

◆ ConfigClass

lsst.obs.base.gen3.ingest.RawIngestTask.ConfigClass = RawIngestConfig
staticinherited

Definition at line 118 of file ingest.py.

◆ datasetType

lsst.obs.base.gen3.ingest.RawIngestTask.datasetType
inherited

Definition at line 132 of file ingest.py.

◆ stashRun

lsst.obs.base.gen3.ingest.RawIngestTask.stashRun
inherited

Definition at line 146 of file ingest.py.

◆ unitEntryCache

lsst.obs.base.gen3.ingest.RawIngestTask.unitEntryCache
inherited

Definition at line 142 of file ingest.py.

◆ units

lsst.obs.base.gen3.ingest.RawIngestTask.units
inherited

Definition at line 133 of file ingest.py.


The documentation for this class was generated from the following file: