lsst.dax.apdb gfdc9b3dea0+60507bba20
Public Member Functions | Public Attributes | Static Public Attributes | List of all members
lsst.dax.apdb.apdbCassandra.ApdbCassandra Class Reference
Inheritance diagram for lsst.dax.apdb.apdbCassandra.ApdbCassandra:
lsst.dax.apdb.apdb.Apdb

Public Member Functions

def __init__ (self, ApdbCassandraConfig config)
 
Optional[TableDeftableDef (self, ApdbTables table)
 
None makeSchema (self, bool drop=False)
 
pandas.DataFrame getDiaObjects (self, sphgeom.Region region)
 
Optional[pandas.DataFrame] getDiaSources (self, sphgeom.Region region, Optional[Iterable[int]] object_ids, dafBase.DateTime visit_time)
 
Optional[pandas.DataFrame] getDiaForcedSources (self, sphgeom.Region region, Optional[Iterable[int]] object_ids, dafBase.DateTime visit_time)
 
pandas.DataFrame getDiaObjectsHistory (self, dafBase.DateTime start_time, Optional[dafBase.DateTime] end_time=None, Optional[sphgeom.Region] region=None)
 
pandas.DataFrame getDiaSourcesHistory (self, dafBase.DateTime start_time, Optional[dafBase.DateTime] end_time=None, Optional[sphgeom.Region] region=None)
 
pandas.DataFrame getDiaForcedSourcesHistory (self, dafBase.DateTime start_time, Optional[dafBase.DateTime] end_time=None, Optional[sphgeom.Region] region=None)
 
pandas.DataFrame getSSObjects (self)
 
None store (self, dafBase.DateTime visit_time, pandas.DataFrame objects, Optional[pandas.DataFrame] sources=None, Optional[pandas.DataFrame] forced_sources=None)
 
None storeSSObjects (self, pandas.DataFrame objects)
 
None reassignDiaSources (self, Mapping[int, int] idMap)
 
None dailyJob (self)
 
int countUnassociatedObjects (self)
 
- Public Member Functions inherited from lsst.dax.apdb.apdb.Apdb
ConfigurableField makeField (cls, str doc)
 

Public Attributes

 config
 

Static Public Attributes

 partition_zero_epoch = dafBase.DateTime(1970, 1, 1, 0, 0, 0, dafBase.DateTime.TAI)
 
- Static Public Attributes inherited from lsst.dax.apdb.apdb.Apdb
 ConfigClass = ApdbConfig
 

Detailed Description

Implementation of APDB database on to of Apache Cassandra.

The implementation is configured via standard ``pex_config`` mechanism
using `ApdbCassandraConfig` configuration class. For an example of
different configurations check config/ folder.

Parameters
----------
config : `ApdbCassandraConfig`
    Configuration object.

Member Function Documentation

◆ countUnassociatedObjects()

int lsst.dax.apdb.apdbCassandra.ApdbCassandra.countUnassociatedObjects (   self)
Return the number of DiaObjects that have only one DiaSource
associated with them.

Used as part of ap_verify metrics.

Returns
-------
count : `int`
    Number of DiaObjects with exactly one associated DiaSource.

Notes
-----
This method can be very inefficient or slow in some implementations.

Reimplemented from lsst.dax.apdb.apdb.Apdb.

◆ dailyJob()

None lsst.dax.apdb.apdbCassandra.ApdbCassandra.dailyJob (   self)
Implement daily activities like cleanup/vacuum.

What should be done during daily activities is determined by
specific implementation.

Reimplemented from lsst.dax.apdb.apdb.Apdb.

◆ getDiaForcedSources()

Optional[pandas.DataFrame] lsst.dax.apdb.apdbCassandra.ApdbCassandra.getDiaForcedSources (   self,
sphgeom.Region  region,
Optional[Iterable[int]]  object_ids,
dafBase.DateTime  visit_time 
)
Return catalog of DiaForcedSource instances from a given region.

Parameters
----------
region : `lsst.sphgeom.Region`
    Region to search for DIASources.
object_ids : iterable [ `int` ], optional
    List of DiaObject IDs to further constrain the set of returned
    sources. If list is empty then empty catalog is returned with a
    correct schema. If `None` then returned sources are not
    constrained. Some implementations may not support latter case.
visit_time : `lsst.daf.base.DateTime`
    Time of the current visit.

Returns
-------
catalog : `pandas.DataFrame`, or `None`
    Catalog containing DiaSource records. `None` is returned if
    ``read_forced_sources_months`` configuration parameter is set to 0.

Raises
------
NotImplementedError
    May be raised by some implementations if ``object_ids`` is `None`.

Notes
-----
This method returns DiaForcedSource catalog for a region with additional
filtering based on DiaObject IDs. Only a subset of DiaSource history
is returned limited by ``read_forced_sources_months`` config parameter,
w.r.t. ``visit_time``. If ``object_ids`` is empty then an empty catalog
is always returned with the correct schema (columns/types). If
``object_ids`` is `None` then no filtering is performed and some of the
returned records may be outside the specified region.

Reimplemented from lsst.dax.apdb.apdb.Apdb.

◆ getDiaForcedSourcesHistory()

pandas.DataFrame lsst.dax.apdb.apdbCassandra.ApdbCassandra.getDiaForcedSourcesHistory (   self,
dafBase.DateTime  start_time,
Optional[dafBase.DateTime]   end_time = None,
Optional[sphgeom.Region]   region = None 
)
Returns catalog of DiaForcedSource instances from a given time
period.

Parameters
----------
start_time : `dafBase.DateTime`
    Starting time for DiaForcedSource history search. DiaForcedSource
    record is selected when its ``midPointTai`` falls into an interval
    between ``start__time`` (inclusive) and ``end_time`` (exclusive).
end_time : `dafBase.DateTime`
    Upper limit on time for DiaForcedSource history search, if not
    specified then there is no restriction on upper limit.
region : `lsst.sphgeom.Region`, optional
    Region to search for DiaForcedSources, if not specified then whole
    sky is searched. If region is specified then some returned records
    may fall outside of this region.

Returns
-------
catalog : `pandas.DataFrame`
    Catalog containing DiaObject records.

Notes
-----
This part of API may not be very stable and can change before the
implementation finalizes.

Reimplemented from lsst.dax.apdb.apdb.Apdb.

◆ getDiaObjects()

pandas.DataFrame lsst.dax.apdb.apdbCassandra.ApdbCassandra.getDiaObjects (   self,
sphgeom.Region  region 
)
Returns catalog of DiaObject instances from a given region.

This method returns only the last version of each DiaObject. Some
records in a returned catalog may be outside the specified region, it
is up to a client to ignore those records or cleanup the catalog before
futher use.

Parameters
----------
region : `lsst.sphgeom.Region`
    Region to search for DIAObjects.

Returns
-------
catalog : `pandas.DataFrame`
    Catalog containing DiaObject records for a region that may be a
    superset of the specified region.

Reimplemented from lsst.dax.apdb.apdb.Apdb.

◆ getDiaObjectsHistory()

pandas.DataFrame lsst.dax.apdb.apdbCassandra.ApdbCassandra.getDiaObjectsHistory (   self,
dafBase.DateTime  start_time,
Optional[dafBase.DateTime]   end_time = None,
Optional[sphgeom.Region]   region = None 
)
Returns catalog of DiaObject instances from a given time period
including the history of each DiaObject.

Parameters
----------
start_time : `dafBase.DateTime`
    Starting time for DiaObject history search. DiaObject record is
    selected when its ``validityStart`` falls into an interval
    between ``start__time`` (inclusive) and ``end_time`` (exclusive).
end_time : `dafBase.DateTime`, optional
    Upper limit on time for DiaObject history search, if not specified
    then there is no restriction on upper limit.
region : `lsst.sphgeom.Region`, optional
    Region to search for DiaObjects, if not specified then whole sky
    is searched. If region is specified then some returned records may
    fall outside of this region.

Returns
-------
catalog : `pandas.DataFrame`
    Catalog containing DiaObject records.

Notes
-----
This part of API may not be very stable and can change before the
implementation finalizes.

Reimplemented from lsst.dax.apdb.apdb.Apdb.

◆ getDiaSources()

Optional[pandas.DataFrame] lsst.dax.apdb.apdbCassandra.ApdbCassandra.getDiaSources (   self,
sphgeom.Region  region,
Optional[Iterable[int]]  object_ids,
dafBase.DateTime  visit_time 
)
Return catalog of DiaSource instances from a given region.

Parameters
----------
region : `lsst.sphgeom.Region`
    Region to search for DIASources.
object_ids : iterable [ `int` ], optional
    List of DiaObject IDs to further constrain the set of returned
    sources. If `None` then returned sources are not constrained. If
    list is empty then empty catalog is returned with a correct
    schema.
visit_time : `lsst.daf.base.DateTime`
    Time of the current visit.

Returns
-------
catalog : `pandas.DataFrame`, or `None`
    Catalog containing DiaSource records. `None` is returned if
    ``read_sources_months`` configuration parameter is set to 0.

Notes
-----
This method returns DiaSource catalog for a region with additional
filtering based on DiaObject IDs. Only a subset of DiaSource history
is returned limited by ``read_sources_months`` config parameter, w.r.t.
``visit_time``. If ``object_ids`` is empty then an empty catalog is
always returned with the correct schema (columns/types). If
``object_ids`` is `None` then no filtering is performed and some of the
returned records may be outside the specified region.

Reimplemented from lsst.dax.apdb.apdb.Apdb.

◆ getDiaSourcesHistory()

pandas.DataFrame lsst.dax.apdb.apdbCassandra.ApdbCassandra.getDiaSourcesHistory (   self,
dafBase.DateTime  start_time,
Optional[dafBase.DateTime]   end_time = None,
Optional[sphgeom.Region]   region = None 
)
Returns catalog of DiaSource instances from a given time period.

Parameters
----------
start_time : `dafBase.DateTime`
    Starting time for DiaSource history search. DiaSource record is
    selected when its ``midPointTai`` falls into an interval between
    ``start__time`` (inclusive) and ``end_time`` (exclusive).
end_time : `dafBase.DateTime`
    Upper limit on time for DiaSource history search, if not specified
    then there is no restriction on upper limit.
region : `lsst.sphgeom.Region`, optional
    Region to search for DiaSources, if not specified then whole sky
    is searched. If region is specified then some returned records may
    fall outside of this region.

Returns
-------
catalog : `pandas.DataFrame`
    Catalog containing DiaObject records.

Notes
-----
This part of API may not be very stable and can change before the
implementation finalizes.

Reimplemented from lsst.dax.apdb.apdb.Apdb.

◆ getSSObjects()

pandas.DataFrame lsst.dax.apdb.apdbCassandra.ApdbCassandra.getSSObjects (   self)
Returns catalog of SSObject instances.

Returns
-------
catalog : `pandas.DataFrame`
    Catalog containing SSObject records, all existing records are
    returned.

Reimplemented from lsst.dax.apdb.apdb.Apdb.

◆ makeSchema()

None lsst.dax.apdb.apdbCassandra.ApdbCassandra.makeSchema (   self,
bool   drop = False 
)
Create or re-create whole database schema.

Parameters
----------
drop : `bool`
    If True then drop all tables before creating new ones.

Reimplemented from lsst.dax.apdb.apdb.Apdb.

◆ reassignDiaSources()

None lsst.dax.apdb.apdbCassandra.ApdbCassandra.reassignDiaSources (   self,
Mapping[int, int]  idMap 
)
Associate DiaSources with SSObjects, dis-associating them
from DiaObjects.

Parameters
----------
idMap : `Mapping`
    Maps DiaSource IDs to their new SSObject IDs.

Reimplemented from lsst.dax.apdb.apdb.Apdb.

◆ store()

None lsst.dax.apdb.apdbCassandra.ApdbCassandra.store (   self,
dafBase.DateTime  visit_time,
pandas.DataFrame  objects,
Optional[pandas.DataFrame]   sources = None,
Optional[pandas.DataFrame]   forced_sources = None 
)
Store all three types of catalogs in the database.

Parameters
----------
visit_time : `lsst.daf.base.DateTime`
    Time of the visit.
objects : `pandas.DataFrame`
    Catalog with DiaObject records.
sources : `pandas.DataFrame`, optional
    Catalog with DiaSource records.
forced_sources : `pandas.DataFrame`, optional
    Catalog with DiaForcedSource records.

Notes
-----
This methods takes DataFrame catalogs, their schema must be
compatible with the schema of APDB table:

  - column names must correspond to database table columns
  - types and units of the columns must match database definitions,
    no unit conversion is performed presently
  - columns that have default values in database schema can be
    omitted from catalog
  - this method knows how to fill interval-related columns of DiaObject
    (validityStart, validityEnd) they do not need to appear in a
    catalog
  - source catalogs have ``diaObjectId`` column associating sources
    with objects

Reimplemented from lsst.dax.apdb.apdb.Apdb.

◆ storeSSObjects()

None lsst.dax.apdb.apdbCassandra.ApdbCassandra.storeSSObjects (   self,
pandas.DataFrame  objects 
)
Store or update SSObject catalog.

Parameters
----------
objects : `pandas.DataFrame`
    Catalog with SSObject records.

Notes
-----
If SSObjects with matching IDs already exist in the database, their
records will be updated with the information from provided records.

Reimplemented from lsst.dax.apdb.apdb.Apdb.

◆ tableDef()

Optional[TableDef] lsst.dax.apdb.apdbCassandra.ApdbCassandra.tableDef (   self,
ApdbTables  table 
)
Return table schema definition for a given table.

Parameters
----------
table : `ApdbTables`
    One of the known APDB tables.

Returns
-------
tableSchema : `TableDef` or `None`
    Table schema description, `None` is returned if table is not
    defined by this implementation.

Reimplemented from lsst.dax.apdb.apdb.Apdb.


The documentation for this class was generated from the following file: