lsst.pipe.tasks g59f6dce098+83b7f37e51
Loading...
Searching...
No Matches
Public Member Functions | Public Attributes | Protected Member Functions | Protected Attributes | Static Protected Attributes | List of all members
lsst.pipe.tasks.functors.Functor Class Reference
Inheritance diagram for lsst.pipe.tasks.functors.Functor:
lsst.pipe.tasks.functors.Color lsst.pipe.tasks.functors.Column lsst.pipe.tasks.functors.CompositeFunctor lsst.pipe.tasks.functors.CustomFunctor lsst.pipe.tasks.functors.DeconvolvedMoments lsst.pipe.tasks.functors.E1 lsst.pipe.tasks.functors.E2 lsst.pipe.tasks.functors.Ebv lsst.pipe.tasks.functors.HsmFwhm lsst.pipe.tasks.functors.HsmTraceSize lsst.pipe.tasks.functors.HtmIndex20 lsst.pipe.tasks.functors.Index lsst.pipe.tasks.functors.Labeller lsst.pipe.tasks.functors.LocalPhotometry lsst.pipe.tasks.functors.LocalWcs lsst.pipe.tasks.functors.Mag lsst.pipe.tasks.functors.MagDiff lsst.pipe.tasks.functors.Photometry lsst.pipe.tasks.functors.PsfHsmTraceSizeDiff lsst.pipe.tasks.functors.PsfSdssTraceSizeDiff lsst.pipe.tasks.functors.RadiusFromQuadrupole lsst.pipe.tasks.functors.Ratio lsst.pipe.tasks.functors.ReferenceBand lsst.pipe.tasks.functors.SdssTraceSize

Public Member Functions

def __init__ (self, filt=None, dataset=None, noDup=None)
 
def noDup (self)
 
def columns (self)
 
def multilevelColumns (self, data, columnIndex=None, returnTuple=False)
 
def __call__ (self, data, dropna=False)
 
def difference (self, data1, data2, **kwargs)
 
def fail (self, df)
 
def name (self)
 
def shortname (self)
 

Public Attributes

 filt
 
 dataset
 
 log
 

Protected Member Functions

def _get_data_columnLevels (self, data, columnIndex=None)
 
def _get_data_columnLevelNames (self, data, columnIndex=None)
 
def _colsFromDict (self, colDict, columnIndex=None)
 
def _func (self, df, dropna=True)
 
def _get_columnIndex (self, data)
 
def _get_data (self, data)
 
def _setLevels (self, df)
 
def _dropna (self, vals)
 

Protected Attributes

 _noDup
 

Static Protected Attributes

str _defaultDataset = 'ref'
 
tuple _dfLevels = ('column',)
 
bool _defaultNoDup = False
 

Detailed Description

Define and execute a calculation on a ParquetTable

The `__call__` method accepts either a `ParquetTable` object or a
`DeferredDatasetHandle`, and returns the
result of the calculation as a single column.  Each functor defines what
columns are needed for the calculation, and only these columns are read
from the `ParquetTable`.

The action of  `__call__` consists of two steps: first, loading the
necessary columns from disk into memory as a `pandas.DataFrame` object;
and second, performing the computation on this dataframe and returning the
result.


To define a new `Functor`, a subclass must define a `_func` method,
that takes a `pandas.DataFrame` and returns result in a `pandas.Series`.
In addition, it must define the following attributes

* `_columns`: The columns necessary to perform the calculation
* `name`: A name appropriate for a figure axis label
* `shortname`: A name appropriate for use as a dictionary key

On initialization, a `Functor` should declare what band (`filt` kwarg)
and dataset (e.g. `'ref'`, `'meas'`, `'forced_src'`) it is intended to be
applied to. This enables the `_get_data` method to extract the proper
columns from the parquet file. If not specified, the dataset will fall back
on the `_defaultDataset`attribute. If band is not specified and `dataset`
is anything other than `'ref'`, then an error will be raised when trying to
perform the calculation.

Originally, `Functor` was set up to expect
datasets formatted like the `deepCoadd_obj` dataset; that is, a
dataframe with a multi-level column index, with the levels of the
column index being `band`, `dataset`, and `column`.
It has since been generalized to apply to dataframes without mutli-level
indices and multi-level indices with just `dataset` and `column` levels.
In addition, the `_get_data` method that reads
the dataframe from the `ParquetTable` will return a dataframe with column
index levels defined by the `_dfLevels` attribute; by default, this is
`column`.

The `_dfLevels` attributes should generally not need to
be changed, unless `_func` needs columns from multiple filters or datasets
to do the calculation.
An example of this is the `lsst.pipe.tasks.functors.Color` functor, for
which `_dfLevels = ('band', 'column')`, and `_func` expects the dataframe
it gets to have those levels in the column index.

Parameters
----------
filt : str
    Filter upon which to do the calculation

dataset : str
    Dataset upon which to do the calculation
    (e.g., 'ref', 'meas', 'forced_src').

Definition at line 93 of file functors.py.

Constructor & Destructor Documentation

◆ __init__()

def lsst.pipe.tasks.functors.Functor.__init__ (   self,
  filt = None,
  dataset = None,
  noDup = None 
)

Member Function Documentation

◆ __call__()

def lsst.pipe.tasks.functors.Functor.__call__ (   self,
  data,
  dropna = False 
)

◆ _colsFromDict()

def lsst.pipe.tasks.functors.Functor._colsFromDict (   self,
  colDict,
  columnIndex = None 
)
protected
Converts dictionary column specficiation to a list of columns

This mirrors the original gen2 implementation within `pipe.tasks.parquetTable.MultilevelParquetTable`

Definition at line 224 of file functors.py.

◆ _dropna()

def lsst.pipe.tasks.functors.Functor._dropna (   self,
  vals 
)
protected

Definition at line 354 of file functors.py.

◆ _func()

def lsst.pipe.tasks.functors.Functor._func (   self,
  df,
  dropna = True 
)
protected

Reimplemented in lsst.pipe.tasks.functors.CustomFunctor, lsst.pipe.tasks.functors.Column, lsst.pipe.tasks.functors.Index, lsst.pipe.tasks.functors.IDColumn, lsst.pipe.tasks.functors.CoordColumn, lsst.pipe.tasks.functors.HtmIndex20, lsst.pipe.tasks.functors.Mag, lsst.pipe.tasks.functors.MagErr, lsst.pipe.tasks.functors.NanoMaggie, lsst.pipe.tasks.functors.MagDiff, lsst.pipe.tasks.functors.Color, lsst.pipe.tasks.functors.StarGalaxyLabeller, lsst.pipe.tasks.functors.NumStarLabeller, lsst.pipe.tasks.functors.DeconvolvedMoments, lsst.pipe.tasks.functors.SdssTraceSize, lsst.pipe.tasks.functors.PsfSdssTraceSizeDiff, lsst.pipe.tasks.functors.HsmTraceSize, lsst.pipe.tasks.functors.PsfHsmTraceSizeDiff, lsst.pipe.tasks.functors.HsmFwhm, lsst.pipe.tasks.functors.E1, lsst.pipe.tasks.functors.E2, lsst.pipe.tasks.functors.RadiusFromQuadrupole, lsst.pipe.tasks.functors.ComputePixelScale, lsst.pipe.tasks.functors.ConvertPixelToArcseconds, lsst.pipe.tasks.functors.ConvertPixelSqToArcsecondsSq, lsst.pipe.tasks.functors.NanoJansky, lsst.pipe.tasks.functors.NanoJanskyErr, lsst.pipe.tasks.functors.Magnitude, lsst.pipe.tasks.functors.MagnitudeErr, lsst.pipe.tasks.functors.LocalNanojansky, lsst.pipe.tasks.functors.LocalNanojanskyErr, lsst.pipe.tasks.functors.LocalMagnitude, lsst.pipe.tasks.functors.LocalMagnitudeErr, lsst.pipe.tasks.functors.LocalDipoleMeanFlux, lsst.pipe.tasks.functors.LocalDipoleMeanFluxErr, lsst.pipe.tasks.functors.LocalDipoleDiffFlux, lsst.pipe.tasks.functors.LocalDipoleDiffFluxErr, lsst.pipe.tasks.functors.Ratio, lsst.pipe.tasks.functors.Ebv, and lsst.pipe.tasks.functors.ReferenceBand.

Definition at line 296 of file functors.py.

◆ _get_columnIndex()

def lsst.pipe.tasks.functors.Functor._get_columnIndex (   self,
  data 
)
protected
Return columnIndex

Definition at line 299 of file functors.py.

◆ _get_data()

def lsst.pipe.tasks.functors.Functor._get_data (   self,
  data 
)
protected
Retrieve dataframe necessary for calculation.

The data argument can be a DataFrame, a ParquetTable instance, or a gen3 DeferredDatasetHandle

Returns dataframe upon which `self._func` can act.

N.B. while passing a raw pandas `DataFrame` *should* work here, it has not been tested.

Definition at line 308 of file functors.py.

◆ _get_data_columnLevelNames()

def lsst.pipe.tasks.functors.Functor._get_data_columnLevelNames (   self,
  data,
  columnIndex = None 
)
protected
Gets the content of each of the column levels for a multilevel table

Similar to `_get_data_columnLevels`, this enables backward compatibility with gen2.

Mirrors original gen2 implementation within `pipe.tasks.parquetTable.MultilevelParquetTable`

Definition at line 202 of file functors.py.

◆ _get_data_columnLevels()

def lsst.pipe.tasks.functors.Functor._get_data_columnLevels (   self,
  data,
  columnIndex = None 
)
protected
Gets the names of the column index levels

This should only be called in the context of a multilevel table.
The logic here is to enable this to work both with the gen2 `MultilevelParquetTable`
and with the gen3 `DeferredDatasetHandle`.

Parameters
----------
data : `MultilevelParquetTable` or `DeferredDatasetHandle`

columnnIndex (optional): pandas `Index` object
    if not passed, then it is read from the `DeferredDatasetHandle`

Definition at line 178 of file functors.py.

◆ _setLevels()

def lsst.pipe.tasks.functors.Functor._setLevels (   self,
  df 
)
protected

Definition at line 349 of file functors.py.

◆ columns()

def lsst.pipe.tasks.functors.Functor.columns (   self)

◆ difference()

def lsst.pipe.tasks.functors.Functor.difference (   self,
  data1,
  data2,
**  kwargs 
)
Computes difference between functor called on two different ParquetTable objects

Definition at line 369 of file functors.py.

◆ fail()

def lsst.pipe.tasks.functors.Functor.fail (   self,
  df 
)

Definition at line 374 of file functors.py.

◆ multilevelColumns()

def lsst.pipe.tasks.functors.Functor.multilevelColumns (   self,
  data,
  columnIndex = None,
  returnTuple = False 
)
Returns columns needed by functor from multilevel dataset

To access tables with multilevel column structure, the `MultilevelParquetTable`
or `DeferredDatasetHandle` need to be passed either a list of tuples or a
dictionary.

Parameters
----------
data : `MultilevelParquetTable` or `DeferredDatasetHandle`

columnIndex (optional): pandas `Index` object
    either passed or read in from `DeferredDatasetHandle`.

`returnTuple` : bool
    If true, then return a list of tuples rather than the column dictionary
    specification.  This is set to `True` by `CompositeFunctor` in order to be able to
    combine columns from the various component functors.

Reimplemented in lsst.pipe.tasks.functors.CompositeFunctor, and lsst.pipe.tasks.functors.Color.

Definition at line 246 of file functors.py.

◆ name()

def lsst.pipe.tasks.functors.Functor.name (   self)

◆ noDup()

def lsst.pipe.tasks.functors.Functor.noDup (   self)

Definition at line 164 of file functors.py.

◆ shortname()

def lsst.pipe.tasks.functors.Functor.shortname (   self)
Short name of functor (suitable for column name/dict key)

Reimplemented in lsst.pipe.tasks.functors.MagDiff, and lsst.pipe.tasks.functors.Color.

Definition at line 384 of file functors.py.

Member Data Documentation

◆ _defaultDataset

str lsst.pipe.tasks.functors.Functor._defaultDataset = 'ref'
staticprotected

Definition at line 153 of file functors.py.

◆ _defaultNoDup

bool lsst.pipe.tasks.functors.Functor._defaultNoDup = False
staticprotected

Definition at line 155 of file functors.py.

◆ _dfLevels

tuple lsst.pipe.tasks.functors.Functor._dfLevels = ('column',)
staticprotected

Definition at line 154 of file functors.py.

◆ _noDup

lsst.pipe.tasks.functors.Functor._noDup
protected

Definition at line 160 of file functors.py.

◆ dataset

lsst.pipe.tasks.functors.Functor.dataset

Definition at line 159 of file functors.py.

◆ filt

lsst.pipe.tasks.functors.Functor.filt

◆ log

lsst.pipe.tasks.functors.Functor.log

Definition at line 161 of file functors.py.


The documentation for this class was generated from the following file: