23 """Base classes for forced measurement plugins and the driver task for these. 25 In forced measurement, a reference catalog is used to define restricted measurements (usually just fluxes) 26 on an image. As the reference catalog may be deeper than the detection limit of the measurement image, we 27 do not assume that we can use detection and deblend information from the measurement image. Instead, we 28 assume this information is present in the reference catalog and can be "transformed" in some sense to 29 the measurement frame. At the very least, this means that Footprints from the reference catalog should 30 be transformed and installed as Footprints in the output measurement catalog. If we have a procedure that 31 can transform HeavyFootprints, we can then proceed with measurement as usual, but using the reference 32 catalog's id and parent fields to define deblend families. If this transformation does not preserve 33 HeavyFootprints (this is currently the case, at least for CCD forced photometry), then we will only 34 be able to replace objects with noise one deblend family at a time, and hence measurements run in 35 single-object mode may be contaminated by neighbors when run on objects with parent != 0. 37 Measurements are generally recorded in the coordinate system of the image being measured (and all 38 slot-eligible fields must be), but non-slot fields may be recorded in other coordinate systems if necessary 39 to avoid information loss (this should, of course, be indicated in the field documentation). Note that 40 the reference catalog may be in a different coordinate system; it is the responsibility of plugins 41 to transform the data they need themselves, using the reference WCS provided. However, for plugins 42 that only require a position or shape, they may simply use output SourceCatalog's centroid or shape slots, 43 which will generally be set to the transformed position of the reference object before any other plugins are 44 run, and hence avoid using the reference catalog at all. 46 Command-line driver tasks for forced measurement can be found in forcedPhotImage.py, including 47 ForcedPhotImageTask, ForcedPhotCcdTask, and ForcedPhotCoaddTask. 53 from .pluginRegistry
import PluginRegistry
54 from .baseMeasurement
import (BaseMeasurementPluginConfig, BaseMeasurementPlugin,
55 BaseMeasurementConfig, BaseMeasurementTask)
56 from .noiseReplacer
import NoiseReplacer, DummyNoiseReplacer
58 __all__ = (
"ForcedPluginConfig",
"ForcedPlugin",
59 "ForcedMeasurementConfig",
"ForcedMeasurementTask")
63 """Base class for configs of forced measurement plugins.""" 72 ConfigClass = ForcedPluginConfig
74 def __init__(self, config, name, schemaMapper, metadata, logName=None):
75 """Initialize the measurement object. 77 @param[in] config An instance of this class's ConfigClass. 78 @param[in] name The string the plugin was registered with. 79 @param[in,out] schemaMapper A SchemaMapper that maps reference catalog fields to output 80 catalog fields. Output fields should be added to the 81 output schema. While most plugins will not need to map 82 fields from the reference schema, if they do so, those fields 83 will be transferred before any plugins are run. 84 @param[in] metadata Plugin metadata that will be attached to the output catalog 86 BaseMeasurementPlugin.__init__(self, config, name, logName=logName)
88 def measure(self, measRecord, exposure, refRecord, refWcs):
89 """Measure the properties of a source on a single image, given data from a 92 @param[in] exposure lsst.afw.image.ExposureF, containing the pixel data to 93 be measured and the associated Psf, Wcs, etc. All 94 other sources in the image will have been replaced by 95 noise according to deblender outputs. 96 @param[in,out] measRecord lsst.afw.table.SourceRecord to be filled with outputs, 97 and from which previously-measured quantities can be 99 @param[in] refRecord lsst.afw.table.SimpleRecord that contains additional 100 parameters to define the fit, as measured elsewhere. 101 @param[in] refWcs The coordinate system for the reference catalog values. 102 An lsst.geom.Angle may be passed, indicating that a 103 local tangent Wcs should be created for each object 104 using afw.image.makeLocalWcs and the given angle as 107 In the normal mode of operation, the source centroid will be set to the 108 WCS-transformed position of the reference object, so plugins that only 109 require a reference position should not have to access the reference object 112 raise NotImplementedError()
114 def measureN(self, measCat, exposure, refCat, refWcs):
115 """Measure the properties of a group of blended sources on a single image, 116 given data from a reference record. 118 @param[in] exposure lsst.afw.image.ExposureF, containing the pixel data to 119 be measured and the associated Psf, Wcs, etc. Sources 120 not in the blended hierarchy to be measured will have 121 been replaced with noise using deblender outputs. 122 @param[in,out] measCat lsst.afw.table.SourceCatalog to be filled with outputs, 123 and from which previously-measured quantities can be 124 retrieved, containing only the sources that should be 125 measured together in this call. 126 @param[in] refCat lsst.afw.table.SimpleCatalog that contains additional 127 parameters to define the fit, as measured elsewhere. 128 Ordered such that zip(sources, references) may be used. 129 @param[in] refWcs The coordinate system for the reference catalog values. 130 An lsst.geom.Angle may be passed, indicating that a 131 local tangent Wcs should be created for each object 132 using afw.image.makeLocalWcs and the given Angle as 135 In the normal mode of operation, the source centroids will be set to the 136 WCS-transformed position of the reference object, so plugins that only 137 require a reference position should not have to access the reference object 140 raise NotImplementedError()
144 """Config class for forced measurement driver task.""" 146 plugins = ForcedPlugin.registry.makeField(
148 default=[
"base_PixelFlags",
149 "base_TransformedCentroid",
151 "base_TransformedShape",
154 "base_CircularApertureFlux",
156 "base_LocalBackground",
158 doc=
"Plugins to be run and their configuration" 160 algorithms = property(
lambda self: self.
plugins, doc=
"backwards-compatibility alias for plugins")
161 undeblended = ForcedPlugin.registry.makeField(
164 doc=
"Plugins to run on undeblended image" 167 copyColumns = lsst.pex.config.DictField(
168 keytype=str, itemtype=str, doc=
"Mapping of reference columns to source columns",
169 default={
"id":
"objectId",
"parent":
"parentObjectId",
"deblend_nChild":
"deblend_nChild",
170 "coord_ra":
"coord_ra",
"coord_dec":
"coord_dec"}
173 checkUnitsParseStrict = lsst.pex.config.Field(
174 doc=
"Strictness of Astropy unit compatibility check, can be 'raise', 'warn' or 'silent'",
180 self.
slots.centroid =
"base_TransformedCentroid" 181 self.
slots.shape =
"base_TransformedShape" 182 self.
slots.apFlux =
None 183 self.
slots.modelFlux =
None 184 self.
slots.psfFlux =
None 185 self.
slots.gaussianFlux =
None 186 self.
slots.calibFlux =
None 198 @anchor ForcedMeasurementTask_ 200 @brief A subtask for measuring the properties of sources on a single 201 exposure, using an existing "reference" catalog to constrain some aspects 204 The task is configured with a list of "plugins": each plugin defines the values it 205 measures (i.e. the columns in a table it will fill) and conducts that measurement 206 on each detected source (see ForcedPlugin). The job of the 207 measurement task is to initialize the set of plugins (which includes setting up the 208 catalog schema) from their configuration, and then invoke each plugin on each 211 Most of the time, ForcedMeasurementTask will be used via one of the subclasses of 212 ForcedPhotImageTask, ForcedPhotCcdTask and ForcedPhotCoaddTask. These combine 213 this measurement subtask with a "references" subtask (see BaseReferencesTask and 214 CoaddSrcReferencesTask) to perform forced measurement using measurements performed on 215 another image as the references. There is generally little reason to use 216 ForcedMeasurementTask outside of one of these drivers, unless it is necessary to avoid 217 using the Butler for I/O. 219 ForcedMeasurementTask has only three methods: __init__(), run(), and generateMeasCat(). 220 For configuration options, see SingleFrameMeasurementConfig. 225 *Forced* measurement means that the plugins are provided with a reference 226 source containing centroid and/or shape measurements that they may use 227 however they see fit. Some plugins can use these to set the location and 228 size of apertures, but others may choose to ignore this information, 229 essentially performing an unforced measurement starting at the position 230 of the reference source (which may nevertheless be useful for certain 231 investigations). Knowing how the plugin uses the reference information is 232 essential to interpreting its resulting measurements. Typically, centroid 233 and shape measurement plugins (e.g., ``SdssCentroid`` and ``SdssShape``) 234 are performing unforced measurements. 237 ConfigClass = ForcedMeasurementConfig
239 def __init__(self, refSchema, algMetadata=None, **kwds):
241 Initialize the task. Set up the execution order of the plugins and initialize 242 the plugins, giving each plugin an opportunity to add its measurement fields to 243 the output schema and to record information in the task metadata. 245 Note that while SingleFrameMeasurementTask is passed an initial Schema that is 246 appended to in order to create the output Schema, ForcedMeasurementTask is 247 initialized with the Schema of the reference catalog, from which a new Schema 248 for the output catalog is created. Fields to be copied directly from the 249 reference Schema are added before Plugin fields are added. 251 @param[in] refSchema Schema of the reference catalog. Must match the catalog 252 later passed to generateMeasCat() and/or run(). 253 @param[in,out] algMetadata lsst.daf.base.PropertyList used to record information about 254 each algorithm. An empty PropertyList will be created if None. 255 @param[in] **kwds Keyword arguments passed from lsst.pipe.base.Task.__init__ 257 super(ForcedMeasurementTask, self).
__init__(algMetadata=algMetadata, **kwds)
260 self.config.slots.setupSchema(self.
mapper.editOutputSchema())
261 for refName, targetName
in self.config.copyColumns.items():
262 refItem = refSchema.find(refName)
263 self.
mapper.addMapping(refItem.key, targetName)
264 self.config.slots.setupSchema(self.
mapper.editOutputSchema())
267 self.
schema.checkUnits(parse_strict=self.config.checkUnitsParseStrict)
269 def run(self, measCat, exposure, refCat, refWcs, exposureId=None, beginOrder=None, endOrder=None):
271 Perform forced measurement. 273 @param[in] exposure lsst.afw.image.ExposureF to be measured; must have at least a Wcs attached. 274 @param[in] measCat Source catalog for measurement results; must be initialized with empty 275 records already corresponding to those in refCat (via e.g. generateMeasCat). 276 @param[in] refCat A sequence of SourceRecord objects that provide reference information 277 for the measurement. These will be passed to each Plugin in addition 278 to the output SourceRecord. 279 @param[in] refWcs Wcs that defines the X,Y coordinate system of refCat 280 @param[in] exposureId optional unique exposureId used to calculate random number 281 generator seed in the NoiseReplacer. 282 @param[in] beginOrder beginning execution order (inclusive): measurements with 283 executionOrder < beginOrder are not executed. None for no limit. 284 @param[in] endOrder ending execution order (exclusive): measurements with 285 executionOrder >= endOrder are not executed. None for no limit. 287 Fills the initial empty SourceCatalog with forced measurement results. Two steps must occur 288 before run() can be called: 289 - generateMeasCat() must be called to create the output measCat argument. 290 - Footprints appropriate for the forced sources must be attached to the measCat records. The 291 attachTransformedFootprints() method can be used to do this, but this degrades HeavyFootprints 292 to regular Footprints, leading to non-deblended measurement, so most callers should provide 293 Footprints some other way. Typically, calling code will have access to information that will 294 allow them to provide HeavyFootprints - for instance, ForcedPhotCoaddTask uses the HeavyFootprints 295 from deblending run in the same band just before non-forced is run measurement in that band. 306 refCatIdDict = {ref.getId(): ref.getParent()
for ref
in refCat}
311 if topId
not in refCatIdDict:
312 raise RuntimeError(
"Reference catalog contains a child for which at least " 313 "one parent in its parent chain is not in the catalog.")
314 topId = refCatIdDict[topId]
319 footprints = {ref.getId(): (ref.getParent(), measRecord.getFootprint())
320 for (ref, measRecord)
in zip(refCat, measCat)}
322 self.log.info(
"Performing forced measurement on %d source%s", len(refCat),
323 "" if len(refCat) == 1
else "s")
325 if self.config.doReplaceWithNoise:
326 noiseReplacer =
NoiseReplacer(self.config.noiseReplacer, exposure,
327 footprints, log=self.log, exposureId=exposureId)
328 algMetadata = measCat.getTable().getMetadata()
329 if algMetadata
is not None:
330 algMetadata.addInt(
"NOISE_SEED_MULTIPLIER", self.config.noiseReplacer.noiseSeedMultiplier)
331 algMetadata.addString(
"NOISE_SOURCE", self.config.noiseReplacer.noiseSource)
332 algMetadata.addDouble(
"NOISE_OFFSET", self.config.noiseReplacer.noiseOffset)
333 if exposureId
is not None:
334 algMetadata.addLong(
"NOISE_EXPOSURE_ID", exposureId)
340 refParentCat, measParentCat = refCat.getChildren(0, measCat)
341 for parentIdx, (refParentRecord, measParentRecord)
in enumerate(zip(refParentCat, measParentCat)):
344 refChildCat, measChildCat = refCat.getChildren(refParentRecord.getId(), measCat)
346 for refChildRecord, measChildRecord
in zip(refChildCat, measChildCat):
347 noiseReplacer.insertSource(refChildRecord.getId())
348 self.
callMeasure(measChildRecord, exposure, refChildRecord, refWcs,
349 beginOrder=beginOrder, endOrder=endOrder)
350 noiseReplacer.removeSource(refChildRecord.getId())
353 noiseReplacer.insertSource(refParentRecord.getId())
354 self.
callMeasure(measParentRecord, exposure, refParentRecord, refWcs,
355 beginOrder=beginOrder, endOrder=endOrder)
356 self.
callMeasureN(measParentCat[parentIdx:parentIdx+1], exposure,
357 refParentCat[parentIdx:parentIdx+1],
358 beginOrder=beginOrder, endOrder=endOrder)
361 beginOrder=beginOrder, endOrder=endOrder)
362 noiseReplacer.removeSource(refParentRecord.getId())
367 for measRecord, refRecord
in zip(measCat, refCat):
369 self.
doMeasurement(plugin, measRecord, exposure, refRecord, refWcs)
372 """!Initialize an output SourceCatalog using information from the reference catalog. 374 This generates a new blank SourceRecord for each record in refCat. Note that this 375 method does not attach any Footprints. Doing so is up to the caller (who may 376 call attachedTransformedFootprints or define their own method - see run() for more 379 @param[in] exposure Exposure to be measured 380 @param[in] refCat Sequence (not necessarily a SourceCatalog) of reference SourceRecords. 381 @param[in] refWcs Wcs that defines the X,Y coordinate system of refCat 382 @param[in] idFactory factory for creating IDs for sources 384 @return Source catalog ready for measurement 386 if idFactory
is None:
390 table = measCat.table
392 table.preallocate(len(refCat))
394 newSource = measCat.addNew()
395 newSource.assign(ref, self.
mapper)
399 """!Default implementation for attaching Footprints to blank sources prior to measurement 401 Footprints for forced photometry must be in the pixel coordinate system of the image being 402 measured, while the actual detections may start out in a different coordinate system. 403 This default implementation transforms the Footprints from the reference catalog from the 404 refWcs to the exposure's Wcs, which downgrades HeavyFootprints into regular Footprints, 405 destroying deblend information. 407 Note that ForcedPhotImageTask delegates to this method in its own attachFootprints method. 408 attachFootprints can then be overridden by its subclasses to define how their Footprints 411 See the documentation for run() for information about the relationships between run(), 412 generateMeasCat(), and attachTransformedFootprints(). 414 exposureWcs = exposure.getWcs()
415 region = exposure.getBBox(lsst.afw.image.PARENT)
416 for srcRecord, refRecord
in zip(sources, refCat):
417 srcRecord.setFootprint(refRecord.getFootprint().
transform(refWcs, exposureWcs, region))
Base config class for all measurement plugins.
def callMeasure(self, measRecord, args, kwds)
Call the measure() method on all plugins, handling exceptions in a consistent way.
def __init__(self, refSchema, algMetadata=None, kwds)
Initialize the task.
A subtask for measuring the properties of sources on a single exposure, using an existing "reference"...
def callMeasureN(self, measCat, args, kwds)
Call the measureN() method on all plugins, handling exceptions in a consistent way.
def run(self, measCat, exposure, refCat, refWcs, exposureId=None, beginOrder=None, endOrder=None)
Perform forced measurement.
def attachTransformedFootprints(self, sources, refCat, exposure, refWcs)
Default implementation for attaching Footprints to blank sources prior to measurement.
def __init__(self, config, name, schemaMapper, metadata, logName=None)
def generateMeasCat(self, exposure, refCat, refWcs, idFactory=None)
Initialize an output SourceCatalog using information from the reference catalog.
static Schema makeMinimalSchema()
def measure(self, measRecord, exposure, refRecord, refWcs)
Base class for plugin registries.
static std::shared_ptr< SourceTable > make(Schema const &schema, std::shared_ptr< IdFactory > const &idFactory)
A do-nothing standin for NoiseReplacer, used when we want to disable NoiseReplacer.
def measureN(self, measCat, exposure, refCat, refWcs)
Class that handles replacing sources with noise during measurement.
static std::shared_ptr< IdFactory > makeSimple()
Ultimate base class for all measurement tasks.
Base config class for all measurement driver tasks.
def initializePlugins(self, kwds)
def doMeasurement(self, plugin, measRecord, args, kwds)
Call the measure() method on the nominated plugin, handling exceptions in a consistent way...