lsst.pipe.drivers  6.0b0-hsc-14-gc6fa718
Public Member Functions | Public Attributes | Static Public Attributes | List of all members
lsst.pipe.drivers.constructCalibs.SkyTask Class Reference
Inheritance diagram for lsst.pipe.drivers.constructCalibs.SkyTask:
lsst.pipe.drivers.constructCalibs.CalibTask lsst::ctrl::pool::parallel::BatchPoolTask lsst::ctrl::pool::parallel::BatchCmdLineTask

Public Member Functions

def __init__ (self, args, kwargs)
 
def scatterProcess (self, pool, ccdIdLists)
 Scatter the processing among the nodes. More...
 
def measureBackground (self, cache, dataId)
 Measure background model for CCD. More...
 
def processSingleBackground (self, dataRef)
 Process a single CCD for the background. More...
 
def processSingle (self, dataRef, backgrounds, scales)
 
def combine (self, cache, struct, outputId)
 Combine multiple background models of a particular CCD and write the output. More...
 
def batchWallTime (cls, time, parsedCmd, numCores)
 
def batchWallTime (cls, time, parsedCmd, numCores)
 
def run (self, expRefList, butler, calibId)
 Construct a calib from a list of exposure references. More...
 
def getOutputId (self, expRefList, calibId)
 Generate the data identifier for the output calib. More...
 
def getMjd (self, butler, dataId, timescale=dafBase.DateTime.UTC)
 
def getFilter (self, butler, dataId)
 
def addMissingKeys (self, dataId, butler, missingKeys=None, calibName=None)
 
def updateMetadata (self, calibImage, exposureTime, darkTime=None, kwargs)
 Update the metadata from the VisitInfo. More...
 
def process (self, cache, ccdId, outputName="postISRCCD", kwargs)
 Process a CCD, specified by a data identifier. More...
 
def processSingle (self, dataRef)
 
def processWrite (self, dataRef, exposure, outputName="postISRCCD")
 Write the processed CCD. More...
 
def processResult (self, exposure)
 
def scale (self, ccdIdLists, data)
 Determine scaling across CCDs and exposures. More...
 
def scatterCombine (self, pool, outputId, ccdIdLists, scales)
 Scatter the combination of exposures across multiple nodes. More...
 
def getFullyQualifiedOutputId (self, ccdName, butler, outputId)
 
def recordCalibInputs (self, butler, calib, dataIdList, outputId)
 Record metadata including the inputs and creation details. More...
 
def interpolateNans (self, image)
 
def write (self, butler, exposure, dataId)
 Write the final combined calib. More...
 
def makeCameraImage (self, camera, dataId, calibs)
 Create and write an image of the entire camera. More...
 
def checkCcdIdLists (self, ccdIdLists)
 
def parseAndRun (cls, args, kwargs)
 
def parseAndSubmit (cls, args=None, kwargs)
 
def batchCommand (cls, args)
 
def logOperation (self, operation, catch=False, trace=True)
 

Public Attributes

 isTrimmed
 
 images
 
 background
 

Static Public Attributes

 ConfigClass = SkyConfig
 
string calibName = "sky"
 
 RunnerClass = CalibTaskRunner
 
 filterName = None
 
float exposureTime = 1.0
 

Detailed Description

Task for sky frame construction

The sky frame is a (relatively) small-scale background
model, the response of the camera to the sky.

To construct, we first remove a large-scale background (e.g., caused
by moonlight) which may vary from image to image. Then we construct a
model of the sky, which is essentially a binned version of the image
(important configuration parameters: sky.background.[xy]BinSize).
It is these models which are coadded to yield the sky frame.

Definition at line 1145 of file constructCalibs.py.

Constructor & Destructor Documentation

◆ __init__()

def lsst.pipe.drivers.constructCalibs.SkyTask.__init__ (   self,
  args,
  kwargs 
)

Definition at line 1161 of file constructCalibs.py.

Member Function Documentation

◆ addMissingKeys()

def lsst.pipe.drivers.constructCalibs.CalibTask.addMissingKeys (   self,
  dataId,
  butler,
  missingKeys = None,
  calibName = None 
)
inherited

Definition at line 538 of file constructCalibs.py.

◆ batchWallTime()

def lsst.pipe.drivers.constructCalibs.CalibTask.batchWallTime (   cls,
  time,
  parsedCmd,
  numCores 
)
inherited

Definition at line 406 of file constructCalibs.py.

◆ checkCcdIdLists()

def lsst.pipe.drivers.constructCalibs.CalibTask.checkCcdIdLists (   self,
  ccdIdLists 
)
inherited
Check that the list of CCD dataIds is consistent

@param ccdIdLists  Dict of data identifier lists for each CCD name
@return Number of exposures, number of CCDs

Definition at line 846 of file constructCalibs.py.

◆ combine()

def lsst.pipe.drivers.constructCalibs.SkyTask.combine (   self,
  cache,
  struct,
  outputId 
)

Combine multiple background models of a particular CCD and write the output.

Only the slave nodes execute this method.

Parameters
cacheProcess pool cache
structParameters for the combination, which has the following components:
  • ccdName Name tuple for CCD
  • ccdIdList List of data identifiers for combination
outputIdData identifier for combined image (exposure part only)
Returns
binned calib image

Definition at line 1300 of file constructCalibs.py.

◆ getFilter()

def lsst.pipe.drivers.constructCalibs.CalibTask.getFilter (   self,
  butler,
  dataId 
)
inherited
Determine the filter from a data identifier

Definition at line 533 of file constructCalibs.py.

◆ getFullyQualifiedOutputId()

def lsst.pipe.drivers.constructCalibs.CalibTask.getFullyQualifiedOutputId (   self,
  ccdName,
  butler,
  outputId 
)
inherited
Get fully-qualified output data identifier

We may need to look up keys that aren't in the output dataId.

@param ccdName  Name tuple for CCD
@param butler  Data butler
@param outputId  Data identifier for combined image (exposure part only)
@return fully-qualified output dataId

Definition at line 699 of file constructCalibs.py.

◆ getMjd()

def lsst.pipe.drivers.constructCalibs.CalibTask.getMjd (   self,
  butler,
  dataId,
  timescale = dafBase.DateTime.UTC 
)
inherited
Determine the Modified Julian Date (MJD; in TAI) from a data identifier

Definition at line 520 of file constructCalibs.py.

◆ getOutputId()

def lsst.pipe.drivers.constructCalibs.CalibTask.getOutputId (   self,
  expRefList,
  calibId 
)
inherited

Generate the data identifier for the output calib.

The mean date and the common filter are included, using keywords from the configuration. The CCD-specific part is not included in the data identifier.

Parameters
expRefListList of data references at exposure level
calibIdData identifier elements for the calib provided by the user
Returns
data identifier

Definition at line 485 of file constructCalibs.py.

◆ interpolateNans()

def lsst.pipe.drivers.constructCalibs.CalibTask.interpolateNans (   self,
  image 
)
inherited
Interpolate over NANs in the combined image

NANs can result from masked areas on the CCD.  We don't want them getting
into our science images, so we replace them with the median of the image.

Definition at line 783 of file constructCalibs.py.

◆ makeCameraImage()

def lsst.pipe.drivers.constructCalibs.CalibTask.makeCameraImage (   self,
  camera,
  dataId,
  calibs 
)
inherited

Create and write an image of the entire camera.

This is useful for judging the quality or getting an overview of the features of the calib.

This requires that the 'ccd name' is a tuple containing only the detector ID. If that is not the case, change CalibConfig.ccdKeys or set CalibConfig.doCameraImage=False to disable this.

Parameters
cameraCamera object
dataIdData identifier for output
calibsDict mapping 'ccd name' to calib image

Definition at line 810 of file constructCalibs.py.

◆ measureBackground()

def lsst.pipe.drivers.constructCalibs.SkyTask.measureBackground (   self,
  cache,
  dataId 
)

Measure background model for CCD.

This method is executed by the slaves.

The background models for all CCDs in an exposure will be combined to form a full focal-plane background model.

Parameters
cacheProcess pool cache
dataIdData identifier
Returns
Bcakground model

Definition at line 1210 of file constructCalibs.py.

◆ process()

def lsst.pipe.drivers.constructCalibs.CalibTask.process (   self,
  cache,
  ccdId,
  outputName = "postISRCCD",
  kwargs 
)
inherited

Process a CCD, specified by a data identifier.

After processing, optionally returns a result (produced by the 'processResult' method) calculated from the processed exposure. These results will be gathered by the master node, and is a means for coordinated scaling of all CCDs for flats, etc.

Only slave nodes execute this method.

Parameters
cacheProcess pool cache
ccdIdData identifier for CCD
outputNameOutput dataset name for butler
Returns
result from 'processResult'

Definition at line 591 of file constructCalibs.py.

◆ processResult()

def lsst.pipe.drivers.constructCalibs.CalibTask.processResult (   self,
  exposure 
)
inherited
Extract processing results from a processed exposure

This method generates what is gathered by the master node.
This can be a background measurement or similar for scaling
flat-fields.  It must be picklable!

Only slave nodes execute this method.

Definition at line 649 of file constructCalibs.py.

◆ processSingle() [1/2]

def lsst.pipe.drivers.constructCalibs.CalibTask.processSingle (   self,
  dataRef 
)
inherited
Process a single CCD, specified by a data reference

Generally, this simply means doing ISR.

Only slave nodes execute this method.

Definition at line 626 of file constructCalibs.py.

◆ processSingle() [2/2]

def lsst.pipe.drivers.constructCalibs.SkyTask.processSingle (   self,
  dataRef,
  backgrounds,
  scales 
)
Process a single CCD, specified by a data reference

We subtract the appropriate focal plane background model,
divide by the appropriate scale and measure the background.

Only slave nodes execute this method.

@param dataRef  Data reference for single CCD
@param backgrounds  Background model for each visit
@param scales  Scales for each visit
@return Processed exposure

Definition at line 1272 of file constructCalibs.py.

◆ processSingleBackground()

def lsst.pipe.drivers.constructCalibs.SkyTask.processSingleBackground (   self,
  dataRef 
)

Process a single CCD for the background.

This method is executed by the slaves.

Because we're interested in the background, we detect and mask astrophysical sources, and pixels above the noise level.

Parameters
dataRefData reference for CCD.
Returns
processed exposure

Definition at line 1233 of file constructCalibs.py.

◆ processWrite()

def lsst.pipe.drivers.constructCalibs.CalibTask.processWrite (   self,
  dataRef,
  exposure,
  outputName = "postISRCCD" 
)
inherited

Write the processed CCD.

We need to write these out because we can't hold them all in memory at once.

Only slave nodes execute this method.

Parameters
dataRefData reference
exposureCCD exposure to write
outputNameOutput dataset name for butler.

Definition at line 635 of file constructCalibs.py.

◆ recordCalibInputs()

def lsst.pipe.drivers.constructCalibs.CalibTask.recordCalibInputs (   self,
  butler,
  calib,
  dataIdList,
  outputId 
)
inherited

Record metadata including the inputs and creation details.

This metadata will go into the FITS header.

Parameters
butlerData butler
calibCombined calib exposure.
dataIdListList of data identifiers for calibration inputs
outputIdData identifier for output

Definition at line 753 of file constructCalibs.py.

◆ run()

def lsst.pipe.drivers.constructCalibs.CalibTask.run (   self,
  expRefList,
  butler,
  calibId 
)
inherited

Construct a calib from a list of exposure references.

This is the entry point, called by the TaskRunner.__call__

Only the master node executes this method.

Parameters
expRefListList of data references at the exposure level
butlerData butler
calibIdIdentifier dict for calib

Definition at line 418 of file constructCalibs.py.

◆ scale()

def lsst.pipe.drivers.constructCalibs.CalibTask.scale (   self,
  ccdIdLists,
  data 
)
inherited

Determine scaling across CCDs and exposures.

This is necessary mainly for flats, so as to determine a consistent scaling across the entire focal plane. This implementation is simply a placeholder.

Only the master node executes this method.

Parameters
ccdIdListsDict of data identifier lists for each CCD tuple
dataDict of lists of returned data for each CCD tuple
Returns
dict of Struct(ccdScale: scaling for CCD, expScales: scaling for each exposure ) for each CCD tuple

Definition at line 660 of file constructCalibs.py.

◆ scatterCombine()

def lsst.pipe.drivers.constructCalibs.CalibTask.scatterCombine (   self,
  pool,
  outputId,
  ccdIdLists,
  scales 
)
inherited

Scatter the combination of exposures across multiple nodes.

In this case, we can only scatter across as many nodes as there are CCDs.

Only the master node executes this method.

Parameters
poolProcess pool
outputIdOutput identifier (exposure part only)
ccdIdListsDict of data identifier lists for each CCD name
scalesDict of structs with scales, for each CCD name
dictof binned images

Definition at line 679 of file constructCalibs.py.

◆ scatterProcess()

def lsst.pipe.drivers.constructCalibs.SkyTask.scatterProcess (   self,
  pool,
  ccdIdLists 
)

Scatter the processing among the nodes.

Only the master node executes this method, assigning work to the slaves.

We measure and subtract off a large-scale background model across all CCDs, which requires a scatter/gather. Then we process the individual CCDs, subtracting the large-scale background model and the residual background model measured. These residuals will be combined for the sky frame.

Parameters
poolProcess pool
ccdIdListsDict of data identifier lists for each CCD name
Returns
Dict of lists of returned data for each CCD name

Definition at line 1167 of file constructCalibs.py.

◆ updateMetadata()

def lsst.pipe.drivers.constructCalibs.CalibTask.updateMetadata (   self,
  calibImage,
  exposureTime,
  darkTime = None,
  kwargs 
)
inherited

Update the metadata from the VisitInfo.

Parameters
calibImageThe image whose metadata is to be set
exposureTimeThe exposure time for the image
darkTimeThe time since the last read (default: exposureTime)

Definition at line 559 of file constructCalibs.py.

◆ write()

def lsst.pipe.drivers.constructCalibs.CalibTask.write (   self,
  butler,
  exposure,
  dataId 
)
inherited

Write the final combined calib.

Only the slave nodes execute this method

Parameters
butlerData butler
exposureCCD exposure to write
dataIdData identifier for output

Definition at line 798 of file constructCalibs.py.

Member Data Documentation

◆ background

lsst.pipe.drivers.constructCalibs.CalibTask.background
inherited

Definition at line 834 of file constructCalibs.py.

◆ calibName

string lsst.pipe.drivers.constructCalibs.SkyTask.calibName = "sky"
static

Definition at line 1159 of file constructCalibs.py.

◆ ConfigClass

lsst.pipe.drivers.constructCalibs.SkyTask.ConfigClass = SkyConfig
static

Definition at line 1157 of file constructCalibs.py.

◆ exposureTime

float lsst.pipe.drivers.constructCalibs.CalibTask.exposureTime = 1.0
staticinherited

Definition at line 397 of file constructCalibs.py.

◆ filterName

lsst.pipe.drivers.constructCalibs.CalibTask.filterName = None
staticinherited

Definition at line 395 of file constructCalibs.py.

◆ images

lsst.pipe.drivers.constructCalibs.CalibTask.images
inherited

Definition at line 833 of file constructCalibs.py.

◆ isTrimmed

lsst.pipe.drivers.constructCalibs.CalibTask.isTrimmed
inherited

Definition at line 832 of file constructCalibs.py.

◆ RunnerClass

lsst.pipe.drivers.constructCalibs.CalibTask.RunnerClass = CalibTaskRunner
staticinherited

Definition at line 394 of file constructCalibs.py.


The documentation for this class was generated from the following file: