lsst.pipe.tasks g8b9e2231ea+fa07cb600e
Loading...
Searching...
No Matches
Public Member Functions | Public Attributes | Protected Member Functions | Protected Attributes | List of all members
lsst.pipe.tasks.repositoryIterator.SourceData Class Reference

Public Member Functions

def __init__ (self, datasetType, sourceKeyTuple)
 
def addSourceMetrics (self, repoInfo, idKeyTuple, idValList, sourceTableList)
 
def finalize (self)
 

Public Attributes

 datasetType
 
 repoInfoList
 
 sourceArr
 
 sourceIdDict
 
 repoArr
 

Protected Member Functions

def _getSourceMetrics (self, idKeyTuple, idValList, sourceTableList)
 

Protected Attributes

 _sourceKeyTuple
 
 _idKeyTuple
 
 _idKeyDTypeList
 
 _sourceDTypeList
 
 _repoKeyTuple
 
 _repoDTypeList
 
 _tempDataList
 

Detailed Description

Accumulate a set of measurements from a set of source tables

To use:
- specify the desired source measurements when constructing this object
- call addSourceMetrics for each repository you harvest data from
- call finalize to produce the final data

Data available after calling finalize:
- self.sourceArr: a numpy structured array of shape (num repositories, num sources)
    containing named columns for:
    - source ID
    - each data ID key
    - each item of data extracted from the source table
- self.sourceIdDict: a dict of (source ID: index of axis 1 of self.sourceArr)
- self.repoArr: a numpy structured array of shape (num repositories,)
    containing a named column for each repository key (see RepositoryIterator)

@note: sources that had non-finite data (e.g. NaN) for every value extracted are silently omitted

Definition at line 54 of file repositoryIterator.py.

Constructor & Destructor Documentation

◆ __init__()

def lsst.pipe.tasks.repositoryIterator.SourceData.__init__ (   self,
  datasetType,
  sourceKeyTuple 
)
@param[in] datasetType: dataset type for source
@param[in] sourceKeyTuple: list of keys of data items to extract from the source tables

@raise RuntimeError if sourceKeyTuple is empty

Definition at line 75 of file repositoryIterator.py.

Member Function Documentation

◆ _getSourceMetrics()

def lsst.pipe.tasks.repositoryIterator.SourceData._getSourceMetrics (   self,
  idKeyTuple,
  idValList,
  sourceTableList 
)
protected
Obtain the desired source measurements from a list of source tables

Extracts a set of source measurements (specified by sourceKeyTuple) from a list of source tables
(one per data ID) and saves them as a dict of source ID: list of data

@param[in] idKeyTuple: a tuple of data ID keys; must be the same for each call
@param[in] idValList: a list of data ID value tuples;
    each tuple contains values in the order in idKeyTuple
@param[in] sourceTableList: a list of source tables, one per entry in idValList

@return a dict of source id: data id tuple + source data tuple
    where source data tuple order matches sourceKeyTuple
    and data id tuple matches self._idKeyTuple (which is set from the first idKeyTuple)

@raise RuntimeError if idKeyTuple is different than it was for the first call.

GetRepositoryDataTask.run returns idKeyTuple and idValList; you can easily make
a subclass of GetRepositoryDataTask that also returns sourceTableList.

Updates instance variables:
- self._idKeyTuple if not already set.

Definition at line 102 of file repositoryIterator.py.

◆ addSourceMetrics()

def lsst.pipe.tasks.repositoryIterator.SourceData.addSourceMetrics (   self,
  repoInfo,
  idKeyTuple,
  idValList,
  sourceTableList 
)
Accumulate source measurements from a list of source tables.

Once you have accumulated all source measurements, call finalize to process the data.

@param[in] repoInfo: a RepositoryInfo instance
@param[in] idKeyTuple: a tuple of data ID keys; must be the same for each call
@param[in] idValList: a list of data ID value tuples;
    each tuple contains values in the order in idKeyTuple
@param[in] sourceTableList: a list of source tables, one per entry in idValList

@raise RuntimeError if idKeyTuple is different than it was for the first call.

Accumulates the data in temporary cache self._tempDataList.

@return number of sources

Definition at line 153 of file repositoryIterator.py.

◆ finalize()

def lsst.pipe.tasks.repositoryIterator.SourceData.finalize (   self)
Process the accumulated source measurements to create the final data products.

Only call this after you have added all source metrics using addSourceMetrics.

Reads temporary cache self._tempDataList and then deletes it.

Definition at line 180 of file repositoryIterator.py.

Member Data Documentation

◆ _idKeyDTypeList

lsst.pipe.tasks.repositoryIterator.SourceData._idKeyDTypeList
protected

Definition at line 88 of file repositoryIterator.py.

◆ _idKeyTuple

lsst.pipe.tasks.repositoryIterator.SourceData._idKeyTuple
protected

Definition at line 87 of file repositoryIterator.py.

◆ _repoDTypeList

lsst.pipe.tasks.repositoryIterator.SourceData._repoDTypeList
protected

Definition at line 93 of file repositoryIterator.py.

◆ _repoKeyTuple

lsst.pipe.tasks.repositoryIterator.SourceData._repoKeyTuple
protected

Definition at line 92 of file repositoryIterator.py.

◆ _sourceDTypeList

lsst.pipe.tasks.repositoryIterator.SourceData._sourceDTypeList
protected

Definition at line 90 of file repositoryIterator.py.

◆ _sourceKeyTuple

lsst.pipe.tasks.repositoryIterator.SourceData._sourceKeyTuple
protected

Definition at line 85 of file repositoryIterator.py.

◆ _tempDataList

lsst.pipe.tasks.repositoryIterator.SourceData._tempDataList
protected

Definition at line 96 of file repositoryIterator.py.

◆ datasetType

lsst.pipe.tasks.repositoryIterator.SourceData.datasetType

Definition at line 84 of file repositoryIterator.py.

◆ repoArr

lsst.pipe.tasks.repositoryIterator.SourceData.repoArr

Definition at line 209 of file repositoryIterator.py.

◆ repoInfoList

lsst.pipe.tasks.repositoryIterator.SourceData.repoInfoList

Definition at line 100 of file repositoryIterator.py.

◆ sourceArr

lsst.pipe.tasks.repositoryIterator.SourceData.sourceArr

Definition at line 202 of file repositoryIterator.py.

◆ sourceIdDict

lsst.pipe.tasks.repositoryIterator.SourceData.sourceIdDict

Definition at line 205 of file repositoryIterator.py.


The documentation for this class was generated from the following file: