23 """Base classes for forced measurement plugins and the driver task for these. 25 In forced measurement, a reference catalog is used to define restricted measurements (usually just fluxes) 26 on an image. As the reference catalog may be deeper than the detection limit of the measurement image, we 27 do not assume that we can use detection and deblend information from the measurement image. Instead, we 28 assume this information is present in the reference catalog and can be "transformed" in some sense to 29 the measurement frame. At the very least, this means that Footprints from the reference catalog should 30 be transformed and installed as Footprints in the output measurement catalog. If we have a procedure that 31 can transform HeavyFootprints, we can then proceed with measurement as usual, but using the reference 32 catalog's id and parent fields to define deblend families. If this transformation does not preserve 33 HeavyFootprints (this is currently the case, at least for CCD forced photometry), then we will only 34 be able to replace objects with noise one deblend family at a time, and hence measurements run in 35 single-object mode may be contaminated by neighbors when run on objects with parent != 0. 37 Measurements are generally recorded in the coordinate system of the image being measured (and all 38 slot-eligible fields must be), but non-slot fields may be recorded in other coordinate systems if necessary 39 to avoid information loss (this should, of course, be indicated in the field documentation). Note that 40 the reference catalog may be in a different coordinate system; it is the responsibility of plugins 41 to transform the data they need themselves, using the reference WCS provided. However, for plugins 42 that only require a position or shape, they may simply use output SourceCatalog's centroid or shape slots, 43 which will generally be set to the transformed position of the reference object before any other plugins are 44 run, and hence avoid using the reference catalog at all. 46 Command-line driver tasks for forced measurement can be found in forcedPhotImage.py, including 47 ForcedPhotImageTask, ForcedPhotCcdTask, and ForcedPhotCoaddTask. 49 from builtins
import zip
54 from .pluginRegistry
import PluginRegistry
55 from .baseMeasurement
import (BaseMeasurementPluginConfig, BaseMeasurementPlugin,
56 BaseMeasurementConfig, BaseMeasurementTask)
57 from .noiseReplacer
import NoiseReplacer, DummyNoiseReplacer
59 __all__ = (
"ForcedPluginConfig",
"ForcedPlugin",
60 "ForcedMeasurementConfig",
"ForcedMeasurementTask")
64 """Base class for configs of forced measurement plugins.""" 73 ConfigClass = ForcedPluginConfig
75 def __init__(self, config, name, schemaMapper, metadata, logName=None):
76 """Initialize the measurement object. 78 @param[in] config An instance of this class's ConfigClass. 79 @param[in] name The string the plugin was registered with. 80 @param[in,out] schemaMapper A SchemaMapper that maps reference catalog fields to output 81 catalog fields. Output fields should be added to the 82 output schema. While most plugins will not need to map 83 fields from the reference schema, if they do so, those fields 84 will be transferred before any plugins are run. 85 @param[in] metadata Plugin metadata that will be attached to the output catalog 87 BaseMeasurementPlugin.__init__(self, config, name, logName=logName)
89 def measure(self, measRecord, exposure, refRecord, refWcs):
90 """Measure the properties of a source on a single image, given data from a 93 @param[in] exposure lsst.afw.image.ExposureF, containing the pixel data to 94 be measured and the associated Psf, Wcs, etc. All 95 other sources in the image will have been replaced by 96 noise according to deblender outputs. 97 @param[in,out] measRecord lsst.afw.table.SourceRecord to be filled with outputs, 98 and from which previously-measured quantities can be 100 @param[in] refRecord lsst.afw.table.SimpleRecord that contains additional 101 parameters to define the fit, as measured elsewhere. 102 @param[in] refWcs The coordinate system for the reference catalog values. 103 An afw.geom.Angle may be passed, indicating that a 104 local tangent Wcs should be created for each object 105 using afw.image.makeLocalWcs and the given angle as 108 In the normal mode of operation, the source centroid will be set to the 109 WCS-transformed position of the reference object, so plugins that only 110 require a reference position should not have to access the reference object 113 raise NotImplementedError()
115 def measureN(self, measCat, exposure, refCat, refWcs):
116 """Measure the properties of a group of blended sources on a single image, 117 given data from a reference record. 119 @param[in] exposure lsst.afw.image.ExposureF, containing the pixel data to 120 be measured and the associated Psf, Wcs, etc. Sources 121 not in the blended hierarchy to be measured will have 122 been replaced with noise using deblender outputs. 123 @param[in,out] measCat lsst.afw.table.SourceCatalog to be filled with outputs, 124 and from which previously-measured quantities can be 125 retrieved, containing only the sources that should be 126 measured together in this call. 127 @param[in] refCat lsst.afw.table.SimpleCatalog that contains additional 128 parameters to define the fit, as measured elsewhere. 129 Ordered such that zip(sources, references) may be used. 130 @param[in] refWcs The coordinate system for the reference catalog values. 131 An afw.geom.Angle may be passed, indicating that a 132 local tangent Wcs should be created for each object 133 using afw.image.makeLocalWcs and the given Angle as 136 In the normal mode of operation, the source centroids will be set to the 137 WCS-transformed position of the reference object, so plugins that only 138 require a reference position should not have to access the reference object 141 raise NotImplementedError()
145 """Config class for forced measurement driver task.""" 147 plugins = ForcedPlugin.registry.makeField(
149 default=[
"base_PixelFlags",
150 "base_TransformedCentroid",
152 "base_TransformedShape",
155 "base_CircularApertureFlux",
157 "base_LocalBackground",
159 doc=
"Plugins to be run and their configuration" 161 algorithms = property(
lambda self: self.
plugins, doc=
"backwards-compatibility alias for plugins")
162 undeblended = ForcedPlugin.registry.makeField(
165 doc=
"Plugins to run on undeblended image" 168 copyColumns = lsst.pex.config.DictField(
169 keytype=str, itemtype=str, doc=
"Mapping of reference columns to source columns",
170 default={
"id":
"objectId",
"parent":
"parentObjectId",
"deblend_nChild":
"deblend_nChild",
171 "coord_ra":
"coord_ra",
"coord_dec":
"coord_dec"}
174 checkUnitsParseStrict = lsst.pex.config.Field(
175 doc=
"Strictness of Astropy unit compatibility check, can be 'raise', 'warn' or 'silent'",
181 self.
slots.centroid =
"base_TransformedCentroid" 182 self.
slots.shape =
"base_TransformedShape" 183 self.
slots.apFlux =
None 184 self.
slots.modelFlux =
None 185 self.
slots.psfFlux =
None 186 self.
slots.instFlux =
None 187 self.
slots.calibFlux =
None 199 \anchor ForcedMeasurementTask_ 201 \brief A subtask for measuring the properties of sources on a single 202 exposure, using an existing "reference" catalog to constrain some aspects 205 The task is configured with a list of "plugins": each plugin defines the values it 206 measures (i.e. the columns in a table it will fill) and conducts that measurement 207 on each detected source (see ForcedPlugin). The job of the 208 measurement task is to initialize the set of plugins (which includes setting up the 209 catalog schema) from their configuration, and then invoke each plugin on each 212 Most of the time, ForcedMeasurementTask will be used via one of the subclasses of 213 ForcedPhotImageTask, ForcedPhotCcdTask and ForcedPhotCoaddTask. These combine 214 this measurement subtask with a "references" subtask (see BaseReferencesTask and 215 CoaddSrcReferencesTask) to perform forced measurement using measurements performed on 216 another image as the references. There is generally little reason to use 217 ForcedMeasurementTask outside of one of these drivers, unless it is necessary to avoid 218 using the Butler for I/O. 220 ForcedMeasurementTask has only three methods: __init__(), run(), and generateMeasCat(). 221 For configuration options, see SingleFrameMeasurementConfig. 226 *Forced* measurement means that the plugins are provided with a reference 227 source containing centroid and/or shape measurements that they may use 228 however they see fit. Some plugins can use these to set the location and 229 size of apertures, but others may choose to ignore this information, 230 essentially performing an unforced measurement starting at the position 231 of the reference source (which may nevertheless be useful for certain 232 investigations). Knowing how the plugin uses the reference information is 233 essential to interpreting its resulting measurements. Typically, centroid 234 and shape measurement plugins (e.g., ``SdssCentroid`` and ``SdssShape``) 235 are performing unforced measurements. 238 ConfigClass = ForcedMeasurementConfig
240 def __init__(self, refSchema, algMetadata=None, **kwds):
242 Initialize the task. Set up the execution order of the plugins and initialize 243 the plugins, giving each plugin an opportunity to add its measurement fields to 244 the output schema and to record information in the task metadata. 246 Note that while SingleFrameMeasurementTask is passed an initial Schema that is 247 appended to in order to create the output Schema, ForcedMeasurementTask is 248 initialized with the Schema of the reference catalog, from which a new Schema 249 for the output catalog is created. Fields to be copied directly from the 250 reference Schema are added before Plugin fields are added. 252 @param[in] refSchema Schema of the reference catalog. Must match the catalog 253 later passed to generateMeasCat() and/or run(). 254 @param[in,out] algMetadata lsst.daf.base.PropertyList used to record information about 255 each algorithm. An empty PropertyList will be created if None. 256 @param[in] **kwds Keyword arguments passed from lsst.pipe.base.Task.__init__ 258 super(ForcedMeasurementTask, self).
__init__(algMetadata=algMetadata, **kwds)
261 self.config.slots.setupSchema(self.
mapper.editOutputSchema())
262 for refName, targetName
in self.config.copyColumns.items():
263 refItem = refSchema.find(refName)
264 self.
mapper.addMapping(refItem.key, targetName)
265 self.config.slots.setupSchema(self.
mapper.editOutputSchema())
268 self.
schema.checkUnits(parse_strict=self.config.checkUnitsParseStrict)
270 def run(self, measCat, exposure, refCat, refWcs, exposureId=None, beginOrder=None, endOrder=None):
272 Perform forced measurement. 274 @param[in] exposure lsst.afw.image.ExposureF to be measured; must have at least a Wcs attached. 275 @param[in] measCat Source catalog for measurement results; must be initialized with empty 276 records already corresponding to those in refCat (via e.g. generateMeasCat). 277 @param[in] refCat A sequence of SourceRecord objects that provide reference information 278 for the measurement. These will be passed to each Plugin in addition 279 to the output SourceRecord. 280 @param[in] refWcs Wcs that defines the X,Y coordinate system of refCat 281 @param[in] exposureId optional unique exposureId used to calculate random number 282 generator seed in the NoiseReplacer. 283 @param[in] beginOrder beginning execution order (inclusive): measurements with 284 executionOrder < beginOrder are not executed. None for no limit. 285 @param[in] endOrder ending execution order (exclusive): measurements with 286 executionOrder >= endOrder are not executed. None for no limit. 288 Fills the initial empty SourceCatalog with forced measurement results. Two steps must occur 289 before run() can be called: 290 - generateMeasCat() must be called to create the output measCat argument. 291 - Footprints appropriate for the forced sources must be attached to the measCat records. The 292 attachTransformedFootprints() method can be used to do this, but this degrades HeavyFootprints 293 to regular Footprints, leading to non-deblended measurement, so most callers should provide 294 Footprints some other way. Typically, calling code will have access to information that will 295 allow them to provide HeavyFootprints - for instance, ForcedPhotCoaddTask uses the HeavyFootprints 296 from deblending run in the same band just before non-forced is run measurement in that band. 307 refCatIdDict = {ref.getId(): ref.getParent()
for ref
in refCat}
312 if topId
not in refCatIdDict:
313 raise RuntimeError(
"Reference catalog contains a child for which at least " 314 "one parent in its parent chain is not in the catalog.")
315 topId = refCatIdDict[topId]
320 footprints = {ref.getId(): (ref.getParent(), measRecord.getFootprint())
321 for (ref, measRecord)
in zip(refCat, measCat)}
323 self.log.info(
"Performing forced measurement on %d source%s", len(refCat),
324 "" if len(refCat) == 1
else "s")
326 if self.config.doReplaceWithNoise:
327 noiseReplacer =
NoiseReplacer(self.config.noiseReplacer, exposure,
328 footprints, log=self.log, exposureId=exposureId)
329 algMetadata = measCat.getTable().getMetadata()
330 if algMetadata
is not None:
331 algMetadata.addInt(
"NOISE_SEED_MULTIPLIER", self.config.noiseReplacer.noiseSeedMultiplier)
332 algMetadata.addString(
"NOISE_SOURCE", self.config.noiseReplacer.noiseSource)
333 algMetadata.addDouble(
"NOISE_OFFSET", self.config.noiseReplacer.noiseOffset)
334 if exposureId
is not None:
335 algMetadata.addLong(
"NOISE_EXPOSURE_ID", exposureId)
341 refParentCat, measParentCat = refCat.getChildren(0, measCat)
342 for parentIdx, (refParentRecord, measParentRecord)
in enumerate(zip(refParentCat, measParentCat)):
345 refChildCat, measChildCat = refCat.getChildren(refParentRecord.getId(), measCat)
347 for refChildRecord, measChildRecord
in zip(refChildCat, measChildCat):
348 noiseReplacer.insertSource(refChildRecord.getId())
349 self.
callMeasure(measChildRecord, exposure, refChildRecord, refWcs,
350 beginOrder=beginOrder, endOrder=endOrder)
351 noiseReplacer.removeSource(refChildRecord.getId())
354 noiseReplacer.insertSource(refParentRecord.getId())
355 self.
callMeasure(measParentRecord, exposure, refParentRecord, refWcs,
356 beginOrder=beginOrder, endOrder=endOrder)
357 self.
callMeasureN(measParentCat[parentIdx:parentIdx+1], exposure,
358 refParentCat[parentIdx:parentIdx+1],
359 beginOrder=beginOrder, endOrder=endOrder)
362 beginOrder=beginOrder, endOrder=endOrder)
363 noiseReplacer.removeSource(refParentRecord.getId())
368 for measRecord, refRecord
in zip(measCat, refCat):
370 self.
doMeasurement(plugin, measRecord, exposure, refRecord, refWcs)
373 """!Initialize an output SourceCatalog using information from the reference catalog. 375 This generates a new blank SourceRecord for each record in refCat. Note that this 376 method does not attach any Footprints. Doing so is up to the caller (who may 377 call attachedTransformedFootprints or define their own method - see run() for more 380 @param[in] exposure Exposure to be measured 381 @param[in] refCat Sequence (not necessarily a SourceCatalog) of reference SourceRecords. 382 @param[in] refWcs Wcs that defines the X,Y coordinate system of refCat 383 @param[in] idFactory factory for creating IDs for sources 385 @return Source catalog ready for measurement 387 if idFactory
is None:
391 table = measCat.table
393 table.preallocate(len(refCat))
395 newSource = measCat.addNew()
396 newSource.assign(ref, self.
mapper)
400 """!Default implementation for attaching Footprints to blank sources prior to measurement 402 Footprints for forced photometry must be in the pixel coordinate system of the image being 403 measured, while the actual detections may start out in a different coordinate system. 404 This default implementation transforms the Footprints from the reference catalog from the 405 refWcs to the exposure's Wcs, which downgrades HeavyFootprints into regular Footprints, 406 destroying deblend information. 408 Note that ForcedPhotImageTask delegates to this method in its own attachFootprints method. 409 attachFootprints can then be overridden by its subclasses to define how their Footprints 412 See the documentation for run() for information about the relationships between run(), 413 generateMeasCat(), and attachTransformedFootprints(). 415 exposureWcs = exposure.getWcs()
416 region = exposure.getBBox(lsst.afw.image.PARENT)
417 for srcRecord, refRecord
in zip(sources, refCat):
418 srcRecord.setFootprint(refRecord.getFootprint().
transform(refWcs, exposureWcs, region))
Base config class for all measurement plugins.
def callMeasure(self, measRecord, args, kwds)
Call the measure() method on all plugins, handling exceptions in a consistent way.
def __init__(self, refSchema, algMetadata=None, kwds)
Initialize the task.
A subtask for measuring the properties of sources on a single exposure, using an existing "reference"...
def callMeasureN(self, measCat, args, kwds)
Call the measureN() method on all plugins, handling exceptions in a consistent way.
def run(self, measCat, exposure, refCat, refWcs, exposureId=None, beginOrder=None, endOrder=None)
Perform forced measurement.
def attachTransformedFootprints(self, sources, refCat, exposure, refWcs)
Default implementation for attaching Footprints to blank sources prior to measurement.
def __init__(self, config, name, schemaMapper, metadata, logName=None)
def generateMeasCat(self, exposure, refCat, refWcs, idFactory=None)
Initialize an output SourceCatalog using information from the reference catalog.
static Schema makeMinimalSchema()
def measure(self, measRecord, exposure, refRecord, refWcs)
Base class for plugin registries.
static std::shared_ptr< SourceTable > make(Schema const &schema, std::shared_ptr< IdFactory > const &idFactory)
A do-nothing standin for NoiseReplacer, used when we want to disable NoiseReplacer.
def measureN(self, measCat, exposure, refCat, refWcs)
Class that handles replacing sources with noise during measurement.
static std::shared_ptr< IdFactory > makeSimple()
Ultimate base class for all measurement tasks.
Base config class for all measurement driver tasks.
def initializePlugins(self, kwds)
def doMeasurement(self, plugin, measRecord, args, kwds)
Call the measure() method on the nominated plugin, handling exceptions in a consistent way...