lsst.obs.base  tickets.DM-23835-g2f59a1585e
Public Member Functions | Public Attributes | List of all members
lsst.obs.base.gen2to3.rootRepoConverter.RootRepoConverter Class Reference
Inheritance diagram for lsst.obs.base.gen2to3.rootRepoConverter.RootRepoConverter:
lsst.obs.base.gen2to3.standardRepoConverter.StandardRepoConverter lsst.obs.base.gen2to3.repoConverter.RepoConverter

Public Member Functions

def __init__ (self, **kwds)
 
bool isDatasetTypeSpecial (self, str datasetTypeName)
 
List[str] getSpecialDirectories (self)
 
Tuple[Optional[BaseSkyMap], Optional[str]] findMatchingSkyMap (self, str datasetTypeName)
 
def prep (self)
 
def insertDimensionData (self)
 
Iterator[FileDataset] iterDatasets (self)
 
def ingest (self)
 
Iterator[Tuple[str, CameraMapperMapping]] iterMappings (self)
 
RepoWalker.Target makeRepoWalkerTarget (self, str datasetTypeName, str template, Dict[str, type] keys, StorageClass storageClass)
 
def findDatasets (self)
 
def handleDataIdExpansionFailure (self, FileDataset dataset, LookupError err)
 
def expandDataIds (self)
 
List[str] getCollections (self, str datasetTypeName)
 

Public Attributes

 butler2
 
 mapper
 
 task
 
 root
 
 subset
 

Detailed Description

A specialization of `RepoConverter` for root data repositories.

`RootRepoConverter` adds support for raw images (mostly delegated to the
parent task's `RawIngestTask` subtask) and reference catalogs.

Parameters
----------
kwds
    Keyword arguments are forwarded to (and required by) `RepoConverter`.

Definition at line 44 of file rootRepoConverter.py.

Constructor & Destructor Documentation

◆ __init__()

def lsst.obs.base.gen2to3.rootRepoConverter.RootRepoConverter.__init__ (   self,
**  kwds 
)

Member Function Documentation

◆ expandDataIds()

def lsst.obs.base.gen2to3.repoConverter.RepoConverter.expandDataIds (   self)
inherited
Expand the data IDs for all datasets to be inserted.

Subclasses may override this method, but must delegate to the base
class implementation if they do.  If they wish to handle expected
failures in data ID expansion, they should override
`handleDataIdExpansionFailure` instead.

This involves queries to the registry, but not writes.  It is
guaranteed to be called between `insertDimensionData` and `ingest`.

Definition at line 428 of file repoConverter.py.

◆ findDatasets()

def lsst.obs.base.gen2to3.repoConverter.RepoConverter.findDatasets (   self)
inherited

Definition at line 393 of file repoConverter.py.

◆ findMatchingSkyMap()

Tuple[Optional[BaseSkyMap], Optional[str]] lsst.obs.base.gen2to3.rootRepoConverter.RootRepoConverter.findMatchingSkyMap (   self,
str  datasetTypeName 
)
Return the appropriate SkyMap for the given dataset type.

Parameters
----------
datasetTypeName : `str`
    Name of the dataset type for which a skymap is sought.

Returns
-------
skyMap : `BaseSkyMap` or `None`
    The `BaseSkyMap` instance, or `None` if there was no match.
skyMapName : `str` or `None`
    The Gen3 name for the SkyMap, or `None` if there was no match.

Reimplemented from lsst.obs.base.gen2to3.standardRepoConverter.StandardRepoConverter.

Definition at line 78 of file rootRepoConverter.py.

◆ getCollections()

List[str] lsst.obs.base.gen2to3.repoConverter.RepoConverter.getCollections (   self,
str  datasetTypeName 
)
inherited
Return the set of collections a particular dataset type should be
associated with.

Parameters
----------
datasetTypeName : `str`
    Name of the dataset type.

Returns
-------
collections : `list` of `str`
    Collections the dataset should be associated with.  The first
    item in the list is the run the dataset should be added to
    initially.

Definition at line 480 of file repoConverter.py.

◆ getSpecialDirectories()

List[str] lsst.obs.base.gen2to3.rootRepoConverter.RootRepoConverter.getSpecialDirectories (   self)
Return a list of directory paths that should not be searched for
files.

These may be directories that simply do not contain datasets (or
contain datasets in another repository), or directories whose datasets
are handled specially by a subclass.

Returns
-------
directories : `list` [`str`]
    The full paths of directories to skip, relative to the repository
    root.

Reimplemented from lsst.obs.base.gen2to3.repoConverter.RepoConverter.

Definition at line 74 of file rootRepoConverter.py.

◆ handleDataIdExpansionFailure()

def lsst.obs.base.gen2to3.repoConverter.RepoConverter.handleDataIdExpansionFailure (   self,
FileDataset  dataset,
LookupError  err 
)
inherited

Definition at line 424 of file repoConverter.py.

◆ ingest()

def lsst.obs.base.gen2to3.rootRepoConverter.RootRepoConverter.ingest (   self)
Insert converted datasets into the Gen3 repository.

Subclasses may override this method, but must delegate to the base
class implementation at some point in their own logic.

This method is guaranteed to be called after `expandDataIds`.

Reimplemented from lsst.obs.base.gen2to3.repoConverter.RepoConverter.

Definition at line 162 of file rootRepoConverter.py.

◆ insertDimensionData()

def lsst.obs.base.gen2to3.rootRepoConverter.RootRepoConverter.insertDimensionData (   self)
Insert any dimension records uniquely derived from this repository
into the registry.

Subclasses may override this method, but may not need to; the default
implementation does nothing.

SkyMap and SkyPix dimensions should instead be handled by calling
`ConvertRepoTask.useSkyMap` or `ConvertRepoTask.useSkyPix`, because
these dimensions are in general shared by multiple Gen2 repositories.

This method is guaranteed to be called between `prep` and
`expandDataIds`.

Reimplemented from lsst.obs.base.gen2to3.repoConverter.RepoConverter.

Definition at line 130 of file rootRepoConverter.py.

◆ isDatasetTypeSpecial()

bool lsst.obs.base.gen2to3.rootRepoConverter.RootRepoConverter.isDatasetTypeSpecial (   self,
str  datasetTypeName 
)
Test whether the given dataset is handled specially by this
converter and hence should be ignored by generic base-class logic that
searches for dataset types to convert.

Parameters
----------
datasetTypeName : `str`
    Name of the dataset type to test.

Returns
-------
special : `bool`
    `True` if the dataset type is special.

Reimplemented from lsst.obs.base.gen2to3.standardRepoConverter.StandardRepoConverter.

Definition at line 65 of file rootRepoConverter.py.

◆ iterDatasets()

Iterator[FileDataset] lsst.obs.base.gen2to3.rootRepoConverter.RootRepoConverter.iterDatasets (   self)
Iterate over datasets in the repository that should be ingested into
the Gen3 repository.

The base class implementation yields nothing; the datasets handled by
the `RepoConverter` base class itself are read directly in
`findDatasets`.

Subclasses should override this method if they support additional
datasets that are handled some other way.

Yields
------
dataset : `FileDataset`
    Structures representing datasets to be ingested.  Paths should be
    absolute.

Reimplemented from lsst.obs.base.gen2to3.standardRepoConverter.StandardRepoConverter.

Definition at line 139 of file rootRepoConverter.py.

◆ iterMappings()

Iterator[Tuple[str, CameraMapperMapping]] lsst.obs.base.gen2to3.standardRepoConverter.StandardRepoConverter.iterMappings (   self)
inherited
Iterate over all `CameraMapper` `Mapping` objects that should be
considered for conversion by this repository.

This this should include any datasets that may appear in the
repository, including those that are special (see
`isDatasetTypeSpecial`) and those that are being ignored (see
`ConvertRepoTask.isDatasetTypeIncluded`); this allows the converter
to identify and hence skip these datasets quietly instead of warning
about them as unrecognized.

Yields
------
datasetTypeName: `str`
    Name of the dataset type.
mapping : `lsst.obs.base.mapping.Mapping`
    Mapping object used by the Gen2 `CameraMapper` to describe the
    dataset type.

Reimplemented from lsst.obs.base.gen2to3.repoConverter.RepoConverter.

Definition at line 123 of file standardRepoConverter.py.

◆ makeRepoWalkerTarget()

RepoWalker.Target lsst.obs.base.gen2to3.standardRepoConverter.StandardRepoConverter.makeRepoWalkerTarget (   self,
str  datasetTypeName,
str  template,
Dict[str, type]  keys,
StorageClass  storageClass 
)
inherited
Make a struct that identifies a dataset type to be extracted by
walking the repo directory structure.

Parameters
----------
datasetTypeName : `str`
    Name of the dataset type (the same in both Gen2 and Gen3).
template : `str`
    The full Gen2 filename template.
keys : `dict` [`str`, `type`]
    A dictionary mapping Gen2 data ID key to the type of its value.
storageClass : `lsst.daf.butler.StorageClass`
    Gen3 storage class for this dataset type.

Returns
-------
target : `RepoWalker.Target`
    A struct containing information about the target dataset (much of
    it simplify forwarded from the arguments).

Reimplemented from lsst.obs.base.gen2to3.repoConverter.RepoConverter.

Definition at line 166 of file standardRepoConverter.py.

◆ prep()

def lsst.obs.base.gen2to3.rootRepoConverter.RootRepoConverter.prep (   self)
Perform preparatory work associated with the dataset types to be
converted from this repository (but not the datasets themselves).

Notes
-----
This should be a relatively fast operation that should not depend on
the size of the repository.

Subclasses may override this method, but must delegate to the base
class implementation at some point in their own logic.
More often, subclasses will specialize the behavior of `prep` by
overriding other methods to which the base class implementation
delegates.  These include:
 - `iterMappings`
 - `isDatasetTypeSpecial`
 - `getSpecialDirectories`
 - `makeRepoWalkerTarget`

This should not perform any write operations to the Gen3 repository.
It is guaranteed to be called before `insertDimensionData`.

Reimplemented from lsst.obs.base.gen2to3.standardRepoConverter.StandardRepoConverter.

Definition at line 90 of file rootRepoConverter.py.

Member Data Documentation

◆ butler2

lsst.obs.base.gen2to3.standardRepoConverter.StandardRepoConverter.butler2
inherited

Definition at line 90 of file standardRepoConverter.py.

◆ mapper

lsst.obs.base.gen2to3.standardRepoConverter.StandardRepoConverter.mapper
inherited

Definition at line 91 of file standardRepoConverter.py.

◆ root

lsst.obs.base.gen2to3.repoConverter.RepoConverter.root
inherited

Definition at line 205 of file repoConverter.py.

◆ subset

lsst.obs.base.gen2to3.repoConverter.RepoConverter.subset
inherited

Definition at line 206 of file repoConverter.py.

◆ task

lsst.obs.base.gen2to3.repoConverter.RepoConverter.task
inherited

Definition at line 204 of file repoConverter.py.


The documentation for this class was generated from the following file: