lsst.pipe.tasks g8b9e2231ea+fa07cb600e
|
Public Member Functions | |
def | __init__ (self, filt=None, dataset=None, noDup=None) |
def | noDup (self) |
def | columns (self) |
def | multilevelColumns (self, data, columnIndex=None, returnTuple=False) |
def | __call__ (self, data, dropna=False) |
def | difference (self, data1, data2, **kwargs) |
def | fail (self, df) |
def | name (self) |
def | shortname (self) |
Public Attributes | |
filt | |
dataset | |
Protected Member Functions | |
def | _get_data_columnLevels (self, data, columnIndex=None) |
def | _get_data_columnLevelNames (self, data, columnIndex=None) |
def | _colsFromDict (self, colDict, columnIndex=None) |
def | _func (self, df, dropna=True) |
def | _get_columnIndex (self, data) |
def | _get_data (self, data) |
def | _setLevels (self, df) |
def | _dropna (self, vals) |
Protected Attributes | |
_noDup | |
Static Protected Attributes | |
str | _defaultDataset = 'ref' |
tuple | _dfLevels = ('column',) |
bool | _defaultNoDup = False |
Define and execute a calculation on a ParquetTable The `__call__` method accepts either a `ParquetTable` object or a `DeferredDatasetHandle`, and returns the result of the calculation as a single column. Each functor defines what columns are needed for the calculation, and only these columns are read from the `ParquetTable`. The action of `__call__` consists of two steps: first, loading the necessary columns from disk into memory as a `pandas.DataFrame` object; and second, performing the computation on this dataframe and returning the result. To define a new `Functor`, a subclass must define a `_func` method, that takes a `pandas.DataFrame` and returns result in a `pandas.Series`. In addition, it must define the following attributes * `_columns`: The columns necessary to perform the calculation * `name`: A name appropriate for a figure axis label * `shortname`: A name appropriate for use as a dictionary key On initialization, a `Functor` should declare what band (`filt` kwarg) and dataset (e.g. `'ref'`, `'meas'`, `'forced_src'`) it is intended to be applied to. This enables the `_get_data` method to extract the proper columns from the parquet file. If not specified, the dataset will fall back on the `_defaultDataset`attribute. If band is not specified and `dataset` is anything other than `'ref'`, then an error will be raised when trying to perform the calculation. Originally, `Functor` was set up to expect datasets formatted like the `deepCoadd_obj` dataset; that is, a dataframe with a multi-level column index, with the levels of the column index being `band`, `dataset`, and `column`. It has since been generalized to apply to dataframes without mutli-level indices and multi-level indices with just `dataset` and `column` levels. In addition, the `_get_data` method that reads the dataframe from the `ParquetTable` will return a dataframe with column index levels defined by the `_dfLevels` attribute; by default, this is `column`. The `_dfLevels` attributes should generally not need to be changed, unless `_func` needs columns from multiple filters or datasets to do the calculation. An example of this is the `lsst.pipe.tasks.functors.Color` functor, for which `_dfLevels = ('band', 'column')`, and `_func` expects the dataframe it gets to have those levels in the column index. Parameters ---------- filt : str Filter upon which to do the calculation dataset : str Dataset upon which to do the calculation (e.g., 'ref', 'meas', 'forced_src').
Definition at line 80 of file functors.py.
def lsst.pipe.tasks.functors.Functor.__init__ | ( | self, | |
filt = None , |
|||
dataset = None , |
|||
noDup = None |
|||
) |
Reimplemented in lsst.pipe.tasks.functors.RAColumn, lsst.pipe.tasks.functors.DecColumn, lsst.pipe.tasks.functors.Ebv, lsst.pipe.tasks.functors.MagErr, lsst.pipe.tasks.functors.Column, lsst.pipe.tasks.functors.CoordColumn, lsst.pipe.tasks.functors.Mag, lsst.pipe.tasks.functors.ConvertPixelToArcseconds, lsst.pipe.tasks.functors.ConvertPixelSqToArcsecondsSq, lsst.pipe.tasks.functors.Color, lsst.pipe.tasks.functors.MagDiff, lsst.pipe.tasks.functors.LocalWcs, lsst.pipe.tasks.functors.Photometry, lsst.pipe.tasks.functors.E1, lsst.pipe.tasks.functors.E2, lsst.pipe.tasks.functors.RadiusFromQuadrupole, lsst.pipe.tasks.functors.CustomFunctor, lsst.pipe.tasks.functors.CompositeFunctor, lsst.pipe.tasks.functors.LocalPhotometry, lsst.pipe.tasks.functors.LocalDipoleMeanFlux, lsst.pipe.tasks.functors.Ratio, and lsst.pipe.tasks.functors.HtmIndex20.
Definition at line 144 of file functors.py.
def lsst.pipe.tasks.functors.Functor.__call__ | ( | self, | |
data, | |||
dropna = False |
|||
) |
Reimplemented in lsst.pipe.tasks.functors.RAColumn, lsst.pipe.tasks.functors.DecColumn, lsst.pipe.tasks.functors.CompositeFunctor, and lsst.pipe.tasks.functors.Labeller.
Definition at line 343 of file functors.py.
|
protected |
Converts dictionary column specficiation to a list of columns This mirrors the original gen2 implementation within `pipe.tasks.parquetTable.MultilevelParquetTable`
Definition at line 210 of file functors.py.
|
protected |
Definition at line 340 of file functors.py.
|
protected |
Reimplemented in lsst.pipe.tasks.functors.CustomFunctor, lsst.pipe.tasks.functors.Column, lsst.pipe.tasks.functors.Index, lsst.pipe.tasks.functors.IDColumn, lsst.pipe.tasks.functors.CoordColumn, lsst.pipe.tasks.functors.HtmIndex20, lsst.pipe.tasks.functors.Mag, lsst.pipe.tasks.functors.MagErr, lsst.pipe.tasks.functors.NanoMaggie, lsst.pipe.tasks.functors.MagDiff, lsst.pipe.tasks.functors.Color, lsst.pipe.tasks.functors.StarGalaxyLabeller, lsst.pipe.tasks.functors.NumStarLabeller, lsst.pipe.tasks.functors.DeconvolvedMoments, lsst.pipe.tasks.functors.SdssTraceSize, lsst.pipe.tasks.functors.PsfSdssTraceSizeDiff, lsst.pipe.tasks.functors.HsmTraceSize, lsst.pipe.tasks.functors.PsfHsmTraceSizeDiff, lsst.pipe.tasks.functors.HsmFwhm, lsst.pipe.tasks.functors.E1, lsst.pipe.tasks.functors.E2, lsst.pipe.tasks.functors.RadiusFromQuadrupole, lsst.pipe.tasks.functors.ComputePixelScale, lsst.pipe.tasks.functors.ConvertPixelToArcseconds, lsst.pipe.tasks.functors.ConvertPixelSqToArcsecondsSq, lsst.pipe.tasks.functors.NanoJansky, lsst.pipe.tasks.functors.NanoJanskyErr, lsst.pipe.tasks.functors.Magnitude, lsst.pipe.tasks.functors.MagnitudeErr, lsst.pipe.tasks.functors.LocalNanojansky, lsst.pipe.tasks.functors.LocalNanojanskyErr, lsst.pipe.tasks.functors.LocalMagnitude, lsst.pipe.tasks.functors.LocalMagnitudeErr, lsst.pipe.tasks.functors.LocalDipoleMeanFlux, lsst.pipe.tasks.functors.LocalDipoleMeanFluxErr, lsst.pipe.tasks.functors.LocalDipoleDiffFlux, lsst.pipe.tasks.functors.LocalDipoleDiffFluxErr, lsst.pipe.tasks.functors.Ratio, lsst.pipe.tasks.functors.Ebv, and lsst.pipe.tasks.functors.ReferenceBand.
Definition at line 282 of file functors.py.
|
protected |
Return columnIndex
Definition at line 285 of file functors.py.
|
protected |
Retrieve dataframe necessary for calculation. The data argument can be a DataFrame, a ParquetTable instance, or a gen3 DeferredDatasetHandle Returns dataframe upon which `self._func` can act. N.B. while passing a raw pandas `DataFrame` *should* work here, it has not been tested.
Definition at line 294 of file functors.py.
|
protected |
Gets the content of each of the column levels for a multilevel table Similar to `_get_data_columnLevels`, this enables backward compatibility with gen2. Mirrors original gen2 implementation within `pipe.tasks.parquetTable.MultilevelParquetTable`
Definition at line 188 of file functors.py.
|
protected |
Gets the names of the column index levels This should only be called in the context of a multilevel table. The logic here is to enable this to work both with the gen2 `MultilevelParquetTable` and with the gen3 `DeferredDatasetHandle`. Parameters ---------- data : `MultilevelParquetTable` or `DeferredDatasetHandle` columnnIndex (optional): pandas `Index` object if not passed, then it is read from the `DeferredDatasetHandle`
Definition at line 164 of file functors.py.
|
protected |
Definition at line 335 of file functors.py.
def lsst.pipe.tasks.functors.Functor.columns | ( | self | ) |
Columns required to perform calculation
Reimplemented in lsst.pipe.tasks.functors.CompositeFunctor, lsst.pipe.tasks.functors.CustomFunctor, lsst.pipe.tasks.functors.Column, lsst.pipe.tasks.functors.Mag, lsst.pipe.tasks.functors.MagErr, lsst.pipe.tasks.functors.MagDiff, lsst.pipe.tasks.functors.Color, lsst.pipe.tasks.functors.E1, lsst.pipe.tasks.functors.E2, lsst.pipe.tasks.functors.RadiusFromQuadrupole, lsst.pipe.tasks.functors.ComputePixelScale, lsst.pipe.tasks.functors.ConvertPixelToArcseconds, lsst.pipe.tasks.functors.ConvertPixelSqToArcsecondsSq, lsst.pipe.tasks.functors.ReferenceBand, lsst.pipe.tasks.functors.Photometry, lsst.pipe.tasks.functors.NanoJanskyErr, lsst.pipe.tasks.functors.MagnitudeErr, lsst.pipe.tasks.functors.LocalNanojansky, lsst.pipe.tasks.functors.LocalNanojanskyErr, lsst.pipe.tasks.functors.LocalMagnitude, lsst.pipe.tasks.functors.LocalMagnitudeErr, lsst.pipe.tasks.functors.LocalDipoleMeanFlux, lsst.pipe.tasks.functors.LocalDipoleMeanFluxErr, lsst.pipe.tasks.functors.LocalDipoleDiffFlux, lsst.pipe.tasks.functors.LocalDipoleDiffFluxErr, and lsst.pipe.tasks.functors.Ratio.
Definition at line 157 of file functors.py.
def lsst.pipe.tasks.functors.Functor.difference | ( | self, | |
data1, | |||
data2, | |||
** | kwargs | ||
) |
Computes difference between functor called on two different ParquetTable objects
Definition at line 354 of file functors.py.
def lsst.pipe.tasks.functors.Functor.fail | ( | self, | |
df | |||
) |
Definition at line 359 of file functors.py.
def lsst.pipe.tasks.functors.Functor.multilevelColumns | ( | self, | |
data, | |||
columnIndex = None , |
|||
returnTuple = False |
|||
) |
Returns columns needed by functor from multilevel dataset To access tables with multilevel column structure, the `MultilevelParquetTable` or `DeferredDatasetHandle` need to be passed either a list of tuples or a dictionary. Parameters ---------- data : `MultilevelParquetTable` or `DeferredDatasetHandle` columnIndex (optional): pandas `Index` object either passed or read in from `DeferredDatasetHandle`. `returnTuple` : bool If true, then return a list of tuples rather than the column dictionary specification. This is set to `True` by `CompositeFunctor` in order to be able to combine columns from the various component functors.
Reimplemented in lsst.pipe.tasks.functors.CompositeFunctor, and lsst.pipe.tasks.functors.Color.
Definition at line 232 of file functors.py.
def lsst.pipe.tasks.functors.Functor.name | ( | self | ) |
Full name of functor (suitable for figure labels)
Reimplemented in lsst.pipe.tasks.functors.CustomFunctor, lsst.pipe.tasks.functors.Column, lsst.pipe.tasks.functors.Mag, lsst.pipe.tasks.functors.MagErr, lsst.pipe.tasks.functors.MagDiff, lsst.pipe.tasks.functors.Color, lsst.pipe.tasks.functors.ConvertPixelToArcseconds, lsst.pipe.tasks.functors.ConvertPixelSqToArcsecondsSq, lsst.pipe.tasks.functors.Photometry, lsst.pipe.tasks.functors.LocalNanojansky, lsst.pipe.tasks.functors.LocalNanojanskyErr, lsst.pipe.tasks.functors.LocalMagnitude, lsst.pipe.tasks.functors.LocalMagnitudeErr, lsst.pipe.tasks.functors.LocalDipoleMeanFlux, lsst.pipe.tasks.functors.LocalDipoleMeanFluxErr, lsst.pipe.tasks.functors.LocalDipoleDiffFlux, lsst.pipe.tasks.functors.LocalDipoleDiffFluxErr, and lsst.pipe.tasks.functors.Ratio.
Definition at line 363 of file functors.py.
def lsst.pipe.tasks.functors.Functor.noDup | ( | self | ) |
Definition at line 150 of file functors.py.
def lsst.pipe.tasks.functors.Functor.shortname | ( | self | ) |
Short name of functor (suitable for column name/dict key)
Reimplemented in lsst.pipe.tasks.functors.MagDiff, and lsst.pipe.tasks.functors.Color.
Definition at line 369 of file functors.py.
|
staticprotected |
Definition at line 140 of file functors.py.
|
staticprotected |
Definition at line 142 of file functors.py.
|
staticprotected |
Definition at line 141 of file functors.py.
|
protected |
Definition at line 147 of file functors.py.
lsst.pipe.tasks.functors.Functor.dataset |
Definition at line 146 of file functors.py.
lsst.pipe.tasks.functors.Functor.filt |
Reimplemented in lsst.pipe.tasks.functors.CompositeFunctor, lsst.pipe.tasks.functors.Color, lsst.pipe.tasks.functors.CompositeFunctor, and lsst.pipe.tasks.functors.Color.
Definition at line 145 of file functors.py.