lsst.obs.base  19.0.0-18-g955d782+1
Public Member Functions | Public Attributes | List of all members
lsst.obs.base.gen2to3.repoConverter.RepoConverter Class Reference
Inheritance diagram for lsst.obs.base.gen2to3.repoConverter.RepoConverter:
lsst.obs.base.gen2to3.calibRepoConverter.CalibRepoConverter lsst.obs.base.gen2to3.standardRepoConverter.StandardRepoConverter lsst.obs.base.gen2to3.rootRepoConverter.RootRepoConverter

Public Member Functions

def __init__
 
def isDatasetTypeSpecial
 
def iterMappings (self)
 
def makeRepoWalkerTarget
 
def getSpecialDirectories (self)
 
def prep (self)
 
def iterDatasets (self)
 
def findDatasets (self)
 
def insertDimensionData (self)
 
def handleDataIdExpansionFailure
 
def expandDataIds (self)
 
def ingest (self)
 
def getButler
 

Public Attributes

 task
 
 root
 
 subset
 

Detailed Description

An abstract base class for objects that help `ConvertRepoTask` convert
datasets from a single Gen2 repository.

Parameters
----------
task : `ConvertRepoTask`
    Task instance that is using this helper object.
root : `str`
    Root of the Gen2 repo being converted.
collections : `list` of `str`
    Gen3 collections with which all converted datasets should be
    associated.
subset : `ConversionSubset, optional
    Helper object that implements a filter that restricts the data IDs that
    are converted.

Notes
-----
`RepoConverter` defines the only public API users of its subclasses should
use (`prep`, `insertDimensionRecords`, and `ingest`).  These delegate to
several abstract methods that subclasses must implement.  In some cases,
subclasses may reimplement the public methods as well, but are expected to
delegate to ``super()`` either at the beginning or end of their own
implementation.

Definition at line 176 of file repoConverter.py.

Constructor & Destructor Documentation

◆ __init__()

def lsst.obs.base.gen2to3.repoConverter.RepoConverter.__init__ (   self,
  task 
)

Definition at line 203 of file repoConverter.py.

Member Function Documentation

◆ expandDataIds()

def lsst.obs.base.gen2to3.repoConverter.RepoConverter.expandDataIds (   self)
Expand the data IDs for all datasets to be inserted.

Subclasses may override this method, but must delegate to the base
class implementation if they do.  If they wish to handle expected
failures in data ID expansion, they should override
`handleDataIdExpansionFailure` instead.

This involves queries to the registry, but not writes.  It is
guaranteed to be called between `insertDimensionData` and `ingest`.

Definition at line 428 of file repoConverter.py.

◆ findDatasets()

def lsst.obs.base.gen2to3.repoConverter.RepoConverter.findDatasets (   self)

Definition at line 393 of file repoConverter.py.

◆ getButler()

def lsst.obs.base.gen2to3.repoConverter.RepoConverter.getButler (   self,
  datasetTypeName 
)

Definition at line 478 of file repoConverter.py.

◆ getSpecialDirectories()

def lsst.obs.base.gen2to3.repoConverter.RepoConverter.getSpecialDirectories (   self,
  List,
  str 
)
Return a list of directory paths that should not be searched for
files.

These may be directories that simply do not contain datasets (or
contain datasets in another repository), or directories whose datasets
are handled specially by a subclass.

Returns
-------
directories : `list` [`str`]
    The full paths of directories to skip, relative to the repository
    root.

Definition at line 277 of file repoConverter.py.

◆ handleDataIdExpansionFailure()

def lsst.obs.base.gen2to3.repoConverter.RepoConverter.handleDataIdExpansionFailure (   self,
  dataset 
)

Definition at line 424 of file repoConverter.py.

◆ ingest()

def lsst.obs.base.gen2to3.repoConverter.RepoConverter.ingest (   self)
Insert converted datasets into the Gen3 repository.

Subclasses may override this method, but must delegate to the base
class implementation at some point in their own logic.

This method is guaranteed to be called after `expandDataIds`.

Definition at line 454 of file repoConverter.py.

◆ insertDimensionData()

def lsst.obs.base.gen2to3.repoConverter.RepoConverter.insertDimensionData (   self)
Insert any dimension records uniquely derived from this repository
into the registry.

Subclasses may override this method, but may not need to; the default
implementation does nothing.

SkyMap and SkyPix dimensions should instead be handled by calling
`ConvertRepoTask.useSkyMap` or `ConvertRepoTask.useSkyPix`, because
these dimensions are in general shared by multiple Gen2 repositories.

This method is guaranteed to be called between `prep` and
`expandDataIds`.

Definition at line 408 of file repoConverter.py.

◆ isDatasetTypeSpecial()

def lsst.obs.base.gen2to3.repoConverter.RepoConverter.isDatasetTypeSpecial (   self,
  datasetTypeName 
)

Definition at line 213 of file repoConverter.py.

◆ iterDatasets()

def lsst.obs.base.gen2to3.repoConverter.RepoConverter.iterDatasets (   self,
  Iterator,
  FileDataset 
)
Iterate over datasets in the repository that should be ingested into
the Gen3 repository.

The base class implementation yields nothing; the datasets handled by
the `RepoConverter` base class itself are read directly in
`findDatasets`.

Subclasses should override this method if they support additional
datasets that are handled some other way.

Yields
------
dataset : `FileDataset`
    Structures representing datasets to be ingested.  Paths should be
    absolute.

Definition at line 374 of file repoConverter.py.

◆ iterMappings()

def lsst.obs.base.gen2to3.repoConverter.RepoConverter.iterMappings (   self,
  Iterator,
  Tuple,
  str,
  CameraMapperMapping 
)
Iterate over all `CameraMapper` `Mapping` objects that should be
considered for conversion by this repository.

This this should include any datasets that may appear in the
repository, including those that are special (see
`isDatasetTypeSpecial`) and those that are being ignored (see
`ConvertRepoTask.isDatasetTypeIncluded`); this allows the converter
to identify and hence skip these datasets quietly instead of warning
about them as unrecognized.

Yields
------
datasetTypeName: `str`
    Name of the dataset type.
mapping : `lsst.obs.base.mapping.Mapping`
    Mapping object used by the Gen2 `CameraMapper` to describe the
    dataset type.

Definition at line 231 of file repoConverter.py.

◆ makeRepoWalkerTarget()

def lsst.obs.base.gen2to3.repoConverter.RepoConverter.makeRepoWalkerTarget (   self,
  datasetTypeName 
)

Definition at line 253 of file repoConverter.py.

◆ prep()

def lsst.obs.base.gen2to3.repoConverter.RepoConverter.prep (   self)
Perform preparatory work associated with the dataset types to be
converted from this repository (but not the datasets themselves).

Notes
-----
This should be a relatively fast operation that should not depend on
the size of the repository.

Subclasses may override this method, but must delegate to the base
class implementation at some point in their own logic.
More often, subclasses will specialize the behavior of `prep` by
overriding other methods to which the base class implementation
delegates.  These include:
 - `iterMappings`
 - `isDatasetTypeSpecial`
 - `getSpecialDirectories`
 - `makeRepoWalkerTarget`

This should not perform any write operations to the Gen3 repository.
It is guaranteed to be called before `insertDimensionData`.

Definition at line 293 of file repoConverter.py.

Member Data Documentation

◆ root

lsst.obs.base.gen2to3.repoConverter.RepoConverter.root

Definition at line 206 of file repoConverter.py.

◆ subset

lsst.obs.base.gen2to3.repoConverter.RepoConverter.subset

Definition at line 207 of file repoConverter.py.

◆ task

lsst.obs.base.gen2to3.repoConverter.RepoConverter.task

Definition at line 205 of file repoConverter.py.


The documentation for this class was generated from the following file: