lsst.pipe.tasks  20.0.0-10-g1b4d8e16+2
Public Member Functions | Public Attributes | List of all members
lsst.pipe.tasks.parquetTable.ParquetTable Class Reference
Inheritance diagram for lsst.pipe.tasks.parquetTable.ParquetTable:
lsst.pipe.tasks.parquetTable.MultilevelParquetTable

Public Member Functions

def __init__ (self, filename=None, dataFrame=None)
 
def write (self, filename)
 
def pandasMd (self)
 
def columnIndex (self)
 
def columns (self)
 
def toDataFrame (self, columns=None)
 

Public Attributes

 filename
 

Detailed Description

Thin wrapper to pyarrow's ParquetFile object

Call `toDataFrame` method to get a `pandas.DataFrame` object,
optionally passing specific columns.

The main purpose of having this wrapper rather than directly
using `pyarrow.ParquetFile` is to make it nicer to load
selected subsets of columns, especially from dataframes with multi-level
column indices.

Instantiated with either a path to a parquet file or a dataFrame

Parameters
----------
filename : str, optional
    Path to Parquet file.
dataFrame : dataFrame, optional

Definition at line 34 of file parquetTable.py.

Constructor & Destructor Documentation

◆ __init__()

def lsst.pipe.tasks.parquetTable.ParquetTable.__init__ (   self,
  filename = None,
  dataFrame = None 
)

Definition at line 54 of file parquetTable.py.

Member Function Documentation

◆ columnIndex()

def lsst.pipe.tasks.parquetTable.ParquetTable.columnIndex (   self)
Columns as a pandas Index

Definition at line 91 of file parquetTable.py.

◆ columns()

def lsst.pipe.tasks.parquetTable.ParquetTable.columns (   self)
List of column names (or column index if df is set)

This may either be a list of column names, or a
pandas.Index object describing the column index, depending
on whether the ParquetTable object is wrapping a ParquetFile
or a DataFrame.

Definition at line 105 of file parquetTable.py.

◆ pandasMd()

def lsst.pipe.tasks.parquetTable.ParquetTable.pandasMd (   self)

Definition at line 83 of file parquetTable.py.

◆ toDataFrame()

def lsst.pipe.tasks.parquetTable.ParquetTable.toDataFrame (   self,
  columns = None 
)
Get table (or specified columns) as a pandas DataFrame

Parameters
----------
columns : list, optional
    Desired columns.  If `None`, then all columns will be
    returned.

Definition at line 126 of file parquetTable.py.

◆ write()

def lsst.pipe.tasks.parquetTable.ParquetTable.write (   self,
  filename 
)
Write pandas dataframe to parquet

Parameters
----------
filename : str
    Path to which to write.

Definition at line 69 of file parquetTable.py.

Member Data Documentation

◆ filename

lsst.pipe.tasks.parquetTable.ParquetTable.filename

Definition at line 55 of file parquetTable.py.


The documentation for this class was generated from the following file: