lsst.obs.base  21.0.0-12-ge5f8019+e3fa360c9c
Public Member Functions | Public Attributes | List of all members
lsst.obs.base.gen2to3.calibRepoConverter.CalibRepoConverter Class Reference
Inheritance diagram for lsst.obs.base.gen2to3.calibRepoConverter.CalibRepoConverter:
lsst.obs.base.gen2to3.repoConverter.RepoConverter

Public Member Functions

def __init__ (self, *CameraMapper mapper, Sequence[str] labels=(), **kwargs)
 
bool isDatasetTypeSpecial (self, str datasetTypeName)
 
Iterator[Tuple[str, CameraMapperMapping]] iterMappings (self)
 
RepoWalker.Target makeRepoWalkerTarget (self, str datasetTypeName, str template, Dict[str, type] keys, StorageClass storageClass, FormatterParameter formatter=None, Optional[PathElementHandler] targetHandler=None)
 
str getRun (self, str datasetTypeName, Optional[str] calibDate=None)
 
List[str] getSpecialDirectories (self)
 
def prep (self)
 
Iterator[FileDataset] iterDatasets (self)
 
def findDatasets (self)
 
def expandDataIds (self)
 
def ingest (self)
 
None finish (self)
 

Public Attributes

 mapper
 
 collection
 
 task
 
 root
 
 instrument
 
 subset
 

Detailed Description

A specialization of `RepoConverter` for calibration repositories.

Parameters
----------
mapper : `CameraMapper`
    Gen2 mapper for the data repository.  The root associated with the
    mapper is ignored and need not match the root of the repository.
labels : `Sequence` [ `str` ]
    Strings injected into the names of the collections that calibration
    datasets are written and certified into (forwarded as the ``extra``
    argument to `Instrument` methods that generate collection names and
    write curated calibrations).
**kwargs
    Additional keyword arguments are forwarded to (and required by)
    `RepoConverter`.

Definition at line 44 of file calibRepoConverter.py.

Constructor & Destructor Documentation

◆ __init__()

def lsst.obs.base.gen2to3.calibRepoConverter.CalibRepoConverter.__init__ (   self,
*CameraMapper  mapper,
Sequence[str]   labels = (),
**  kwargs 
)

Definition at line 62 of file calibRepoConverter.py.

Member Function Documentation

◆ expandDataIds()

def lsst.obs.base.gen2to3.repoConverter.RepoConverter.expandDataIds (   self)
inherited
Expand the data IDs for all datasets to be inserted.

Subclasses may override this method, but must delegate to the base
class implementation if they do.

This involves queries to the registry, but not writes.  It is
guaranteed to be called between `findDatasets` and `ingest`.

Definition at line 441 of file repoConverter.py.

◆ findDatasets()

def lsst.obs.base.gen2to3.repoConverter.RepoConverter.findDatasets (   self)
inherited

Definition at line 424 of file repoConverter.py.

◆ finish()

None lsst.obs.base.gen2to3.repoConverter.RepoConverter.finish (   self)
inherited
Finish conversion of a repository.

This is run after ``ingest``, and delegates to `_finish`, which should
be overridden by derived classes instead of this method.

Definition at line 511 of file repoConverter.py.

◆ getRun()

str lsst.obs.base.gen2to3.calibRepoConverter.CalibRepoConverter.getRun (   self,
str  datasetTypeName,
Optional[str]   calibDate = None 
)
Return the name of the run to insert instances of the given dataset
type into in this collection.

Parameters
----------
datasetTypeName : `str`
    Name of the dataset type.
calibDate : `str`, optional
    If not `None`, the "CALIBDATE" associated with this (calibration)
    dataset in the Gen2 data repository.

Returns
-------
run : `str`
    Name of the `~lsst.daf.butler.CollectionType.RUN` collection.

Reimplemented from lsst.obs.base.gen2to3.repoConverter.RepoConverter.

Definition at line 294 of file calibRepoConverter.py.

◆ getSpecialDirectories()

List[str] lsst.obs.base.gen2to3.repoConverter.RepoConverter.getSpecialDirectories (   self)
inherited
Return a list of directory paths that should not be searched for
files.

These may be directories that simply do not contain datasets (or
contain datasets in another repository), or directories whose datasets
are handled specially by a subclass.

Returns
-------
directories : `list` [`str`]
    The full paths of directories to skip, relative to the repository
    root.

Reimplemented in lsst.obs.base.gen2to3.rootRepoConverter.RootRepoConverter.

Definition at line 292 of file repoConverter.py.

◆ ingest()

def lsst.obs.base.gen2to3.repoConverter.RepoConverter.ingest (   self)
inherited
Insert converted datasets into the Gen3 repository.

Subclasses may override this method, but must delegate to the base
class implementation at some point in their own logic.

This method is guaranteed to be called after `expandDataIds`.

Definition at line 483 of file repoConverter.py.

◆ isDatasetTypeSpecial()

bool lsst.obs.base.gen2to3.calibRepoConverter.CalibRepoConverter.isDatasetTypeSpecial (   self,
str  datasetTypeName 
)
Test whether the given dataset is handled specially by this
converter and hence should be ignored by generic base-class logic that
searches for dataset types to convert.

Parameters
----------
datasetTypeName : `str`
    Name of the dataset type to test.

Returns
-------
special : `bool`
    `True` if the dataset type is special.

Reimplemented from lsst.obs.base.gen2to3.repoConverter.RepoConverter.

Definition at line 69 of file calibRepoConverter.py.

◆ iterDatasets()

Iterator[FileDataset] lsst.obs.base.gen2to3.repoConverter.RepoConverter.iterDatasets (   self)
inherited
Iterate over datasets in the repository that should be ingested into
the Gen3 repository.

The base class implementation yields nothing; the datasets handled by
the `RepoConverter` base class itself are read directly in
`findDatasets`.

Subclasses should override this method if they support additional
datasets that are handled some other way.

Yields
------
dataset : `FileDataset`
    Structures representing datasets to be ingested.  Paths should be
    absolute.

Reimplemented in lsst.obs.base.gen2to3.standardRepoConverter.StandardRepoConverter, and lsst.obs.base.gen2to3.rootRepoConverter.RootRepoConverter.

Definition at line 405 of file repoConverter.py.

◆ iterMappings()

Iterator[Tuple[str, CameraMapperMapping]] lsst.obs.base.gen2to3.calibRepoConverter.CalibRepoConverter.iterMappings (   self)
Iterate over all `CameraMapper` `Mapping` objects that should be
considered for conversion by this repository.

This this should include any datasets that may appear in the
repository, including those that are special (see
`isDatasetTypeSpecial`) and those that are being ignored (see
`ConvertRepoTask.isDatasetTypeIncluded`); this allows the converter
to identify and hence skip these datasets quietly instead of warning
about them as unrecognized.

Yields
------
datasetTypeName: `str`
    Name of the dataset type.
mapping : `lsst.obs.base.mapping.Mapping`
    Mapping object used by the Gen2 `CameraMapper` to describe the
    dataset type.

Reimplemented from lsst.obs.base.gen2to3.repoConverter.RepoConverter.

Definition at line 73 of file calibRepoConverter.py.

◆ makeRepoWalkerTarget()

RepoWalker.Target lsst.obs.base.gen2to3.calibRepoConverter.CalibRepoConverter.makeRepoWalkerTarget (   self,
str  datasetTypeName,
str  template,
Dict[str, type]  keys,
StorageClass  storageClass,
FormatterParameter   formatter = None,
Optional[PathElementHandler]   targetHandler = None 
)
Make a struct that identifies a dataset type to be extracted by
walking the repo directory structure.

Parameters
----------
datasetTypeName : `str`
    Name of the dataset type (the same in both Gen2 and Gen3).
template : `str`
    The full Gen2 filename template.
keys : `dict` [`str`, `type`]
    A dictionary mapping Gen2 data ID key to the type of its value.
storageClass : `lsst.daf.butler.StorageClass`
    Gen3 storage class for this dataset type.
formatter : `lsst.daf.butler.Formatter` or `str`, optional
    A Gen 3 formatter class or fully-qualified name.
targetHandler : `PathElementHandler`, optional
    Specialist target handler to use for this dataset type.

Returns
-------
target : `RepoWalker.Target`
    A struct containing information about the target dataset (much of
    it simplify forwarded from the arguments).

Reimplemented from lsst.obs.base.gen2to3.repoConverter.RepoConverter.

Definition at line 77 of file calibRepoConverter.py.

◆ prep()

def lsst.obs.base.gen2to3.repoConverter.RepoConverter.prep (   self)
inherited
Perform preparatory work associated with the dataset types to be
converted from this repository (but not the datasets themselves).

Notes
-----
This should be a relatively fast operation that should not depend on
the size of the repository.

Subclasses may override this method, but must delegate to the base
class implementation at some point in their own logic.
More often, subclasses will specialize the behavior of `prep` by
overriding other methods to which the base class implementation
delegates.  These include:
 - `iterMappings`
 - `isDatasetTypeSpecial`
 - `getSpecialDirectories`
 - `makeRepoWalkerTarget`

This should not perform any write operations to the Gen3 repository.
It is guaranteed to be called before `ingest`.

Reimplemented in lsst.obs.base.gen2to3.rootRepoConverter.RootRepoConverter, and lsst.obs.base.gen2to3.standardRepoConverter.StandardRepoConverter.

Definition at line 308 of file repoConverter.py.

Member Data Documentation

◆ collection

lsst.obs.base.gen2to3.calibRepoConverter.CalibRepoConverter.collection

Definition at line 65 of file calibRepoConverter.py.

◆ instrument

lsst.obs.base.gen2to3.repoConverter.RepoConverter.instrument
inherited

Definition at line 212 of file repoConverter.py.

◆ mapper

lsst.obs.base.gen2to3.calibRepoConverter.CalibRepoConverter.mapper

Definition at line 64 of file calibRepoConverter.py.

◆ root

lsst.obs.base.gen2to3.repoConverter.RepoConverter.root
inherited

Definition at line 211 of file repoConverter.py.

◆ subset

lsst.obs.base.gen2to3.repoConverter.RepoConverter.subset
inherited

Definition at line 213 of file repoConverter.py.

◆ task

lsst.obs.base.gen2to3.repoConverter.RepoConverter.task
inherited

Definition at line 210 of file repoConverter.py.


The documentation for this class was generated from the following file: