23 """Base classes for forced measurement plugins and the driver task for these. 25 In forced measurement, a reference catalog is used to define restricted measurements (usually just fluxes) 26 on an image. As the reference catalog may be deeper than the detection limit of the measurement image, we 27 do not assume that we can use detection and deblend information from the measurement image. Instead, we 28 assume this information is present in the reference catalog and can be "transformed" in some sense to 29 the measurement frame. At the very least, this means that Footprints from the reference catalog should 30 be transformed and installed as Footprints in the output measurement catalog. If we have a procedure that 31 can transform HeavyFootprints, we can then proceed with measurement as usual, but using the reference 32 catalog's id and parent fields to define deblend families. If this transformation does not preserve 33 HeavyFootprints (this is currently the case, at least for CCD forced photometry), then we will only 34 be able to replace objects with noise one deblend family at a time, and hence measurements run in 35 single-object mode may be contaminated by neighbors when run on objects with parent != 0. 37 Measurements are generally recorded in the coordinate system of the image being measured (and all 38 slot-eligible fields must be), but non-slot fields may be recorded in other coordinate systems if necessary 39 to avoid information loss (this should, of course, be indicated in the field documentation). Note that 40 the reference catalog may be in a different coordinate system; it is the responsibility of plugins 41 to transform the data they need themselves, using the reference WCS provided. However, for plugins 42 that only require a position or shape, they may simply use output SourceCatalog's centroid or shape slots, 43 which will generally be set to the transformed position of the reference object before any other plugins are 44 run, and hence avoid using the reference catalog at all. 46 Command-line driver tasks for forced measurement can be found in forcedPhotImage.py, including 47 ForcedPhotImageTask, ForcedPhotCcdTask, and ForcedPhotCoaddTask. 49 from builtins
import zip
54 from .pluginRegistry
import PluginRegistry
55 from .baseMeasurement
import (BaseMeasurementPluginConfig, BaseMeasurementPlugin,
56 BaseMeasurementConfig, BaseMeasurementTask)
57 from .noiseReplacer
import NoiseReplacer, DummyNoiseReplacer
59 __all__ = (
"ForcedPluginConfig",
"ForcedPlugin",
60 "ForcedMeasurementConfig",
"ForcedMeasurementTask")
64 """Base class for configs of forced measurement plugins.""" 73 ConfigClass = ForcedPluginConfig
75 def __init__(self, config, name, schemaMapper, metadata, logName=None):
76 """Initialize the measurement object. 78 @param[in] config An instance of this class's ConfigClass. 79 @param[in] name The string the plugin was registered with. 80 @param[in,out] schemaMapper A SchemaMapper that maps reference catalog fields to output 81 catalog fields. Output fields should be added to the 82 output schema. While most plugins will not need to map 83 fields from the reference schema, if they do so, those fields 84 will be transferred before any plugins are run. 85 @param[in] metadata Plugin metadata that will be attached to the output catalog 87 BaseMeasurementPlugin.__init__(self, config, name, logName=logName)
89 def measure(self, measRecord, exposure, refRecord, refWcs):
90 """Measure the properties of a source on a single image, given data from a 93 @param[in] exposure lsst.afw.image.ExposureF, containing the pixel data to 94 be measured and the associated Psf, Wcs, etc. All 95 other sources in the image will have been replaced by 96 noise according to deblender outputs. 97 @param[in,out] measRecord lsst.afw.table.SourceRecord to be filled with outputs, 98 and from which previously-measured quantities can be 100 @param[in] refRecord lsst.afw.table.SimpleRecord that contains additional 101 parameters to define the fit, as measured elsewhere. 102 @param[in] refWcs The coordinate system for the reference catalog values. 103 An afw.geom.Angle may be passed, indicating that a 104 local tangent Wcs should be created for each object 105 using afw.image.makeLocalWcs and the given angle as 108 In the normal mode of operation, the source centroid will be set to the 109 WCS-transformed position of the reference object, so plugins that only 110 require a reference position should not have to access the reference object 113 raise NotImplementedError()
115 def measureN(self, measCat, exposure, refCat, refWcs):
116 """Measure the properties of a group of blended sources on a single image, 117 given data from a reference record. 119 @param[in] exposure lsst.afw.image.ExposureF, containing the pixel data to 120 be measured and the associated Psf, Wcs, etc. Sources 121 not in the blended hierarchy to be measured will have 122 been replaced with noise using deblender outputs. 123 @param[in,out] measCat lsst.afw.table.SourceCatalog to be filled with outputs, 124 and from which previously-measured quantities can be 125 retrieved, containing only the sources that should be 126 measured together in this call. 127 @param[in] refCat lsst.afw.table.SimpleCatalog that contains additional 128 parameters to define the fit, as measured elsewhere. 129 Ordered such that zip(sources, references) may be used. 130 @param[in] refWcs The coordinate system for the reference catalog values. 131 An afw.geom.Angle may be passed, indicating that a 132 local tangent Wcs should be created for each object 133 using afw.image.makeLocalWcs and the given Angle as 136 In the normal mode of operation, the source centroids will be set to the 137 WCS-transformed position of the reference object, so plugins that only 138 require a reference position should not have to access the reference object 141 raise NotImplementedError()
145 """Config class for forced measurement driver task.""" 147 plugins = ForcedPlugin.registry.makeField(
149 default=[
"base_PixelFlags",
150 "base_TransformedCentroid",
151 "base_TransformedShape",
153 "base_CircularApertureFlux",
156 doc=
"Plugins to be run and their configuration" 158 algorithms = property(
lambda self: self.
plugins, doc=
"backwards-compatibility alias for plugins")
159 undeblended = ForcedPlugin.registry.makeField(
162 doc=
"Plugins to run on undeblended image" 165 copyColumns = lsst.pex.config.DictField(
166 keytype=str, itemtype=str, doc=
"Mapping of reference columns to source columns",
167 default={
"id":
"objectId",
"parent":
"parentObjectId",
"deblend_nChild":
"deblend_nChild",
168 "coord_ra":
"coord_ra",
"coord_dec":
"coord_dec"}
171 checkUnitsParseStrict = lsst.pex.config.Field(
172 doc=
"Strictness of Astropy unit compatibility check, can be 'raise', 'warn' or 'silent'",
178 self.
slots.centroid =
"base_TransformedCentroid" 179 self.
slots.shape =
"base_TransformedShape" 180 self.
slots.apFlux =
None 181 self.
slots.modelFlux =
None 182 self.
slots.psfFlux =
None 183 self.
slots.instFlux =
None 184 self.
slots.calibFlux =
None 196 \anchor ForcedMeasurementTask_ 198 \brief A subtask for measuring the properties of sources on a single 199 exposure, using an existing "reference" catalog to constrain some aspects 202 The task is configured with a list of "plugins": each plugin defines the values it 203 measures (i.e. the columns in a table it will fill) and conducts that measurement 204 on each detected source (see ForcedPlugin). The job of the 205 measurement task is to initialize the set of plugins (which includes setting up the 206 catalog schema) from their configuration, and then invoke each plugin on each 209 Most of the time, ForcedMeasurementTask will be used via one of the subclasses of 210 ForcedPhotImageTask, ForcedPhotCcdTask and ForcedPhotCoaddTask. These combine 211 this measurement subtask with a "references" subtask (see BaseReferencesTask and 212 CoaddSrcReferencesTask) to perform forced measurement using measurements performed on 213 another image as the references. There is generally little reason to use 214 ForcedMeasurementTask outside of one of these drivers, unless it is necessary to avoid 215 using the Butler for I/O. 217 ForcedMeasurementTask has only three methods: __init__(), run(), and generateMeasCat(). 218 For configuration options, see SingleFrameMeasurementConfig. 221 ConfigClass = ForcedMeasurementConfig
223 def __init__(self, refSchema, algMetadata=None, **kwds):
225 Initialize the task. Set up the execution order of the plugins and initialize 226 the plugins, giving each plugin an opportunity to add its measurement fields to 227 the output schema and to record information in the task metadata. 229 Note that while SingleFrameMeasurementTask is passed an initial Schema that is 230 appended to in order to create the output Schema, ForcedMeasurementTask is 231 initialized with the Schema of the reference catalog, from which a new Schema 232 for the output catalog is created. Fields to be copied directly from the 233 reference Schema are added before Plugin fields are added. 235 @param[in] refSchema Schema of the reference catalog. Must match the catalog 236 later passed to generateMeasCat() and/or run(). 237 @param[in,out] algMetadata lsst.daf.base.PropertyList used to record information about 238 each algorithm. An empty PropertyList will be created if None. 239 @param[in] **kwds Keyword arguments passed from lsst.pipe.base.Task.__init__ 241 super(ForcedMeasurementTask, self).
__init__(algMetadata=algMetadata, **kwds)
244 self.config.slots.setupSchema(self.
mapper.editOutputSchema())
245 for refName, targetName
in self.config.copyColumns.items():
246 refItem = refSchema.find(refName)
247 self.
mapper.addMapping(refItem.key, targetName)
248 self.config.slots.setupSchema(self.
mapper.editOutputSchema())
251 self.
schema.checkUnits(parse_strict=self.config.checkUnitsParseStrict)
253 def run(self, measCat, exposure, refCat, refWcs, exposureId=None, beginOrder=None, endOrder=None):
255 Perform forced measurement. 257 @param[in] exposure lsst.afw.image.ExposureF to be measured; must have at least a Wcs attached. 258 @param[in] measCat Source catalog for measurement results; must be initialized with empty 259 records already corresponding to those in refCat (via e.g. generateMeasCat). 260 @param[in] refCat A sequence of SourceRecord objects that provide reference information 261 for the measurement. These will be passed to each Plugin in addition 262 to the output SourceRecord. 263 @param[in] refWcs Wcs that defines the X,Y coordinate system of refCat 264 @param[in] exposureId optional unique exposureId used to calculate random number 265 generator seed in the NoiseReplacer. 266 @param[in] beginOrder beginning execution order (inclusive): measurements with 267 executionOrder < beginOrder are not executed. None for no limit. 268 @param[in] endOrder ending execution order (exclusive): measurements with 269 executionOrder >= endOrder are not executed. None for no limit. 271 Fills the initial empty SourceCatalog with forced measurement results. Two steps must occur 272 before run() can be called: 273 - generateMeasCat() must be called to create the output measCat argument. 274 - Footprints appropriate for the forced sources must be attached to the measCat records. The 275 attachTransformedFootprints() method can be used to do this, but this degrades HeavyFootprints 276 to regular Footprints, leading to non-deblended measurement, so most callers should provide 277 Footprints some other way. Typically, calling code will have access to information that will 278 allow them to provide HeavyFootprints - for instance, ForcedPhotCoaddTask uses the HeavyFootprints 279 from deblending run in the same band just before non-forced is run measurement in that band. 290 refCatIdDict = {ref.getId(): ref.getParent()
for ref
in refCat}
295 if topId
not in refCatIdDict:
296 raise RuntimeError(
"Reference catalog contains a child for which at least " 297 "one parent in its parent chain is not in the catalog.")
298 topId = refCatIdDict[topId]
303 footprints = {ref.getId(): (ref.getParent(), measRecord.getFootprint())
304 for (ref, measRecord)
in zip(refCat, measCat)}
306 self.log.info(
"Performing forced measurement on %d source%s", len(refCat),
307 "" if len(refCat) == 1
else "s")
309 if self.config.doReplaceWithNoise:
310 noiseReplacer =
NoiseReplacer(self.config.noiseReplacer, exposure,
311 footprints, log=self.log, exposureId=exposureId)
312 algMetadata = measCat.getTable().getMetadata()
313 if algMetadata
is not None:
314 algMetadata.addInt(
"NOISE_SEED_MULTIPLIER", self.config.noiseReplacer.noiseSeedMultiplier)
315 algMetadata.addString(
"NOISE_SOURCE", self.config.noiseReplacer.noiseSource)
316 algMetadata.addDouble(
"NOISE_OFFSET", self.config.noiseReplacer.noiseOffset)
317 if exposureId
is not None:
318 algMetadata.addLong(
"NOISE_EXPOSURE_ID", exposureId)
324 refParentCat, measParentCat = refCat.getChildren(0, measCat)
325 for parentIdx, (refParentRecord, measParentRecord)
in enumerate(zip(refParentCat, measParentCat)):
328 refChildCat, measChildCat = refCat.getChildren(refParentRecord.getId(), measCat)
330 for refChildRecord, measChildRecord
in zip(refChildCat, measChildCat):
331 noiseReplacer.insertSource(refChildRecord.getId())
332 self.
callMeasure(measChildRecord, exposure, refChildRecord, refWcs,
333 beginOrder=beginOrder, endOrder=endOrder)
334 noiseReplacer.removeSource(refChildRecord.getId())
337 noiseReplacer.insertSource(refParentRecord.getId())
338 self.
callMeasure(measParentRecord, exposure, refParentRecord, refWcs,
339 beginOrder=beginOrder, endOrder=endOrder)
340 self.
callMeasureN(measParentCat[parentIdx:parentIdx+1], exposure,
341 refParentCat[parentIdx:parentIdx+1],
342 beginOrder=beginOrder, endOrder=endOrder)
345 beginOrder=beginOrder, endOrder=endOrder)
346 noiseReplacer.removeSource(refParentRecord.getId())
351 for measRecord, refRecord
in zip(measCat, refCat):
353 self.
doMeasurement(plugin, measRecord, exposure, refRecord, refWcs)
356 """!Initialize an output SourceCatalog using information from the reference catalog. 358 This generates a new blank SourceRecord for each record in refCat. Note that this 359 method does not attach any Footprints. Doing so is up to the caller (who may 360 call attachedTransformedFootprints or define their own method - see run() for more 363 @param[in] exposure Exposure to be measured 364 @param[in] refCat Sequence (not necessarily a SourceCatalog) of reference SourceRecords. 365 @param[in] refWcs Wcs that defines the X,Y coordinate system of refCat 366 @param[in] idFactory factory for creating IDs for sources 368 @return Source catalog ready for measurement 370 if idFactory
is None:
374 table = measCat.table
376 table.preallocate(len(refCat))
378 newSource = measCat.addNew()
379 newSource.assign(ref, self.
mapper)
383 """!Default implementation for attaching Footprints to blank sources prior to measurement 385 Footprints for forced photometry must be in the pixel coordinate system of the image being 386 measured, while the actual detections may start out in a different coordinate system. 387 This default implementation transforms the Footprints from the reference catalog from the 388 refWcs to the exposure's Wcs, which downgrades HeavyFootprints into regular Footprints, 389 destroying deblend information. 391 Note that ForcedPhotImageTask delegates to this method in its own attachFootprints method. 392 attachFootprints can then be overridden by its subclasses to define how their Footprints 395 See the documentation for run() for information about the relationships between run(), 396 generateMeasCat(), and attachTransformedFootprints(). 398 exposureWcs = exposure.getWcs()
399 region = exposure.getBBox(lsst.afw.image.PARENT)
400 for srcRecord, refRecord
in zip(sources, refCat):
401 srcRecord.setFootprint(refRecord.getFootprint().
transform(refWcs, exposureWcs, region))
Base config class for all measurement plugins.
def callMeasure(self, measRecord, args, kwds)
Call the measure() method on all plugins, handling exceptions in a consistent way.
def __init__(self, refSchema, algMetadata=None, kwds)
Initialize the task.
A subtask for measuring the properties of sources on a single exposure, using an existing "reference"...
def callMeasureN(self, measCat, args, kwds)
Call the measureN() method on all plugins, handling exceptions in a consistent way.
def run(self, measCat, exposure, refCat, refWcs, exposureId=None, beginOrder=None, endOrder=None)
Perform forced measurement.
def attachTransformedFootprints(self, sources, refCat, exposure, refWcs)
Default implementation for attaching Footprints to blank sources prior to measurement.
static std::shared_ptr< IdFactory > makeSimple()
def __init__(self, config, name, schemaMapper, metadata, logName=None)
def generateMeasCat(self, exposure, refCat, refWcs, idFactory=None)
Initialize an output SourceCatalog using information from the reference catalog.
static Schema makeMinimalSchema()
def measure(self, measRecord, exposure, refRecord, refWcs)
Base class for plugin registries.
A do-nothing standin for NoiseReplacer, used when we want to disable NoiseReplacer.
def measureN(self, measCat, exposure, refCat, refWcs)
static std::shared_ptr< SourceTable > make(Schema const &schema, std::shared_ptr< IdFactory > const &idFactory)
Class that handles replacing sources with noise during measurement.
Ultimate base class for all measurement tasks.
Base config class for all measurement driver tasks.
def initializePlugins(self, kwds)
def doMeasurement(self, plugin, measRecord, args, kwds)
Call the measure() method on the nominated plugin, handling exceptions in a consistent way...