23 """Base classes for forced measurement plugins and the driver task for these.
25 In forced measurement, a reference catalog is used to define restricted measurements (usually just fluxes)
26 on an image. As the reference catalog may be deeper than the detection limit of the measurement image, we
27 do not assume that we can use detection and deblend information from the measurement image. Instead, we
28 assume this information is present in the reference catalog and can be "transformed" in some sense to
29 the measurement frame. At the very least, this means that Footprints from the reference catalog should
30 be transformed and installed as Footprints in the output measurement catalog. If we have a procedure that
31 can transform HeavyFootprints, we can then proceed with measurement as usual, but using the reference
32 catalog's id and parent fields to define deblend families. If this transformation does not preserve
33 HeavyFootprints (this is currently the case, at least for CCD forced photometry), then we will only
34 be able to replace objects with noise one deblend family at a time, and hence measurements run in
35 single-object mode may be contaminated by neighbors when run on objects with parent != 0.
37 Measurements are generally recorded in the coordinate system of the image being measured (and all
38 slot-eligible fields must be), but non-slot fields may be recorded in other coordinate systems if necessary
39 to avoid information loss (this should, of course, be indicated in the field documentation). Note that
40 the reference catalog may be in a different coordinate system; it is the responsibility of plugins
41 to transform the data they need themselves, using the reference WCS provided. However, for plugins
42 that only require a position or shape, they may simply use output SourceCatalog's centroid or shape slots,
43 which will generally be set to the transformed position of the reference object before any other plugins are
44 run, and hence avoid using the reference catalog at all.
46 Command-line driver tasks for forced measurement can be found in forcedPhotImage.py, including
47 ForcedPhotImageTask, ForcedPhotCcdTask, and ForcedPhotCoaddTask.
49 from builtins
import zip
51 import lsst.pex.config
54 from .pluginRegistry
import PluginRegistry
55 from .baseMeasurement
import (BaseMeasurementPluginConfig, BaseMeasurementPlugin,
56 BaseMeasurementConfig, BaseMeasurementTask)
57 from .noiseReplacer
import NoiseReplacer, DummyNoiseReplacer
59 __all__ = (
"ForcedPluginConfig",
"ForcedPlugin",
60 "ForcedMeasurementConfig",
"ForcedMeasurementTask")
64 """Base class for configs of forced measurement plugins."""
71 registry = PluginRegistry(ForcedPluginConfig)
73 ConfigClass = ForcedPluginConfig
75 def __init__(self, config, name, schemaMapper, metadata, logName=None):
76 """Initialize the measurement object.
78 @param[in] config An instance of this class's ConfigClass.
79 @param[in] name The string the plugin was registered with.
80 @param[in,out] schemaMapper A SchemaMapper that maps reference catalog fields to output
81 catalog fields. Output fields should be added to the
82 output schema. While most plugins will not need to map
83 fields from the reference schema, if they do so, those fields
84 will be transferred before any plugins are run.
85 @param[in] metadata Plugin metadata that will be attached to the output catalog
87 BaseMeasurementPlugin.__init__(self, config, name, logName=logName)
89 def measure(self, measRecord, exposure, refRecord, refWcs):
90 """Measure the properties of a source on a single image, given data from a
93 @param[in] exposure lsst.afw.image.ExposureF, containing the pixel data to
94 be measured and the associated Psf, Wcs, etc. All
95 other sources in the image will have been replaced by
96 noise according to deblender outputs.
97 @param[in,out] measRecord lsst.afw.table.SourceRecord to be filled with outputs,
98 and from which previously-measured quantities can be
100 @param[in] refRecord lsst.afw.table.SimpleRecord that contains additional
101 parameters to define the fit, as measured elsewhere.
102 @param[in] refWcs The coordinate system for the reference catalog values.
103 An afw.geom.Angle may be passed, indicating that a
104 local tangent Wcs should be created for each object
105 using afw.image.makeLocalWcs and the given angle as
108 In the normal mode of operation, the source centroid will be set to the
109 WCS-transformed position of the reference object, so plugins that only
110 require a reference position should not have to access the reference object
113 raise NotImplementedError()
115 def measureN(self, measCat, exposure, refCat, refWcs):
116 """Measure the properties of a group of blended sources on a single image,
117 given data from a reference record.
119 @param[in] exposure lsst.afw.image.ExposureF, containing the pixel data to
120 be measured and the associated Psf, Wcs, etc. Sources
121 not in the blended hierarchy to be measured will have
122 been replaced with noise using deblender outputs.
123 @param[in,out] measCat lsst.afw.table.SourceCatalog to be filled with outputs,
124 and from which previously-measured quantities can be
125 retrieved, containing only the sources that should be
126 measured together in this call.
127 @param[in] refCat lsst.afw.table.SimpleCatalog that contains additional
128 parameters to define the fit, as measured elsewhere.
129 Ordered such that zip(sources, references) may be used.
130 @param[in] refWcs The coordinate system for the reference catalog values.
131 An afw.geom.Angle may be passed, indicating that a
132 local tangent Wcs should be created for each object
133 using afw.image.makeLocalWcs and the given Angle as
136 In the normal mode of operation, the source centroids will be set to the
137 WCS-transformed position of the reference object, so plugins that only
138 require a reference position should not have to access the reference object
141 raise NotImplementedError()
145 """Config class for forced measurement driver task."""
147 plugins = ForcedPlugin.registry.makeField(
149 default=[
"base_PixelFlags",
150 "base_TransformedCentroid",
151 "base_TransformedShape",
153 "base_CircularApertureFlux",
156 doc=
"Plugins to be run and their configuration"
158 algorithms = property(
lambda self: self.
plugins, doc=
"backwards-compatibility alias for plugins")
159 undeblended = ForcedPlugin.registry.makeField(
162 doc=
"Plugins to run on undeblended image"
165 copyColumns = lsst.pex.config.DictField(
166 keytype=str, itemtype=str, doc=
"Mapping of reference columns to source columns",
167 default={
"id":
"objectId",
"parent":
"parentObjectId",
"deblend_nChild":
"deblend_nChild",
168 "coord_ra":
"coord_ra",
"coord_dec":
"coord_dec"}
171 checkUnitsParseStrict = lsst.pex.config.Field(
172 doc=
"Strictness of Astropy unit compatibility check, can be 'raise', 'warn' or 'silent'",
178 self.slots.centroid =
"base_TransformedCentroid"
179 self.slots.shape =
"base_TransformedShape"
180 self.slots.apFlux =
None
181 self.slots.modelFlux =
None
182 self.slots.psfFlux =
None
183 self.slots.instFlux =
None
184 self.slots.calibFlux =
None
196 \anchor ForcedMeasurementTask_
198 \brief A subtask for measuring the properties of sources on a single
199 exposure, using an existing "reference" catalog to constrain some aspects
202 The task is configured with a list of "plugins": each plugin defines the values it
203 measures (i.e. the columns in a table it will fill) and conducts that measurement
204 on each detected source (see ForcedPlugin). The job of the
205 measurement task is to initialize the set of plugins (which includes setting up the
206 catalog schema) from their configuration, and then invoke each plugin on each
209 Most of the time, ForcedMeasurementTask will be used via one of the subclasses of
210 ForcedPhotImageTask, ForcedPhotCcdTask and ForcedPhotCoaddTask. These combine
211 this measurement subtask with a "references" subtask (see BaseReferencesTask and
212 CoaddSrcReferencesTask) to perform forced measurement using measurements performed on
213 another image as the references. There is generally little reason to use
214 ForcedMeasurementTask outside of one of these drivers, unless it is necessary to avoid
215 using the Butler for I/O.
217 ForcedMeasurementTask has only three methods: __init__(), run(), and generateMeasCat().
218 For configuration options, see SingleFrameMeasurementConfig.
221 ConfigClass = ForcedMeasurementConfig
223 def __init__(self, refSchema, algMetadata=None, **kwds):
225 Initialize the task. Set up the execution order of the plugins and initialize
226 the plugins, giving each plugin an opportunity to add its measurement fields to
227 the output schema and to record information in the task metadata.
229 Note that while SingleFrameMeasurementTask is passed an initial Schema that is
230 appended to in order to create the output Schema, ForcedMeasurementTask is
231 initialized with the Schema of the reference catalog, from which a new Schema
232 for the output catalog is created. Fields to be copied directly from the
233 reference Schema are added before Plugin fields are added.
235 @param[in] refSchema Schema of the reference catalog. Must match the catalog
236 later passed to generateMeasCat() and/or run().
237 @param[in,out] algMetadata lsst.daf.base.PropertyList used to record information about
238 each algorithm. An empty PropertyList will be created if None.
239 @param[in] **kwds Keyword arguments passed from lsst.pipe.base.Task.__init__
241 super(ForcedMeasurementTask, self).
__init__(algMetadata=algMetadata, **kwds)
242 self.
mapper = lsst.afw.table.SchemaMapper(refSchema)
243 self.mapper.addMinimalSchema(lsst.afw.table.SourceTable.makeMinimalSchema(),
False)
244 self.config.slots.setupSchema(self.mapper.editOutputSchema())
245 for refName, targetName
in self.config.copyColumns.items():
246 refItem = refSchema.find(refName)
247 self.mapper.addMapping(refItem.key, targetName)
248 self.config.slots.setupSchema(self.mapper.editOutputSchema())
249 self.initializePlugins(schemaMapper=self.
mapper)
250 self.
schema = self.mapper.getOutputSchema()
251 self.schema.checkUnits(parse_strict=self.config.checkUnitsParseStrict)
253 def run(self, measCat, exposure, refCat, refWcs, exposureId=None, beginOrder=None, endOrder=None):
255 Perform forced measurement.
257 @param[in] exposure lsst.afw.image.ExposureF to be measured; must have at least a Wcs attached.
258 @param[in] measCat Source catalog for measurement results; must be initialized with empty
259 records already corresponding to those in refCat (via e.g. generateMeasCat).
260 @param[in] refCat A sequence of SourceRecord objects that provide reference information
261 for the measurement. These will be passed to each Plugin in addition
262 to the output SourceRecord.
263 @param[in] refWcs Wcs that defines the X,Y coordinate system of refCat
264 @param[in] exposureId optional unique exposureId used to calculate random number
265 generator seed in the NoiseReplacer.
266 @param[in] beginOrder beginning execution order (inclusive): measurements with
267 executionOrder < beginOrder are not executed. None for no limit.
268 @param[in] endOrder ending execution order (exclusive): measurements with
269 executionOrder >= endOrder are not executed. None for no limit.
271 Fills the initial empty SourceCatalog with forced measurement results. Two steps must occur
272 before run() can be called:
273 - generateMeasCat() must be called to create the output measCat argument.
274 - Footprints appropriate for the forced sources must be attached to the measCat records. The
275 attachTransformedFootprints() method can be used to do this, but this degrades HeavyFootprints
276 to regular Footprints, leading to non-deblended measurement, so most callers should provide
277 Footprints some other way. Typically, calling code will have access to information that will
278 allow them to provide HeavyFootprints - for instance, ForcedPhotCoaddTask uses the HeavyFootprints
279 from deblending run in the same band just before non-forced is run measurement in that band.
290 refCatIdDict = {ref.getId(): ref.getParent()
for ref
in refCat}
295 if topId
not in refCatIdDict:
296 raise RuntimeError(
"Reference catalog contains a child for which at least "
297 "one parent in its parent chain is not in the catalog.")
298 topId = refCatIdDict[topId]
303 footprints = {ref.getId(): (ref.getParent(), measRecord.getFootprint())
304 for (ref, measRecord)
in zip(refCat, measCat)}
306 self.log.info(
"Performing forced measurement on %d sources" % (len(refCat),))
308 if self.config.doReplaceWithNoise:
309 noiseReplacer = NoiseReplacer(self.config.noiseReplacer, exposure,
310 footprints, log=self.log, exposureId=exposureId)
311 algMetadata = measCat.getTable().getMetadata()
312 if algMetadata
is not None:
313 algMetadata.addInt(
"NOISE_SEED_MULTIPLIER", self.config.noiseReplacer.noiseSeedMultiplier)
314 algMetadata.addString(
"NOISE_SOURCE", self.config.noiseReplacer.noiseSource)
315 algMetadata.addDouble(
"NOISE_OFFSET", self.config.noiseReplacer.noiseOffset)
316 if exposureId
is not None:
317 algMetadata.addLong(
"NOISE_EXPOSURE_ID", exposureId)
319 noiseReplacer = DummyNoiseReplacer()
323 refParentCat, measParentCat = refCat.getChildren(0, measCat)
324 for parentIdx, (refParentRecord, measParentRecord)
in enumerate(zip(refParentCat, measParentCat)):
327 refChildCat, measChildCat = refCat.getChildren(refParentRecord.getId(), measCat)
329 for refChildRecord, measChildRecord
in zip(refChildCat, measChildCat):
330 noiseReplacer.insertSource(refChildRecord.getId())
331 self.callMeasure(measChildRecord, exposure, refChildRecord, refWcs,
332 beginOrder=beginOrder, endOrder=endOrder)
333 noiseReplacer.removeSource(refChildRecord.getId())
336 noiseReplacer.insertSource(refParentRecord.getId())
337 self.callMeasure(measParentRecord, exposure, refParentRecord, refWcs,
338 beginOrder=beginOrder, endOrder=endOrder)
339 self.callMeasureN(measParentCat[parentIdx:parentIdx+1], exposure,
340 refParentCat[parentIdx:parentIdx+1],
341 beginOrder=beginOrder, endOrder=endOrder)
343 self.callMeasureN(measChildCat, exposure, refChildCat,
344 beginOrder=beginOrder, endOrder=endOrder)
345 noiseReplacer.removeSource(refParentRecord.getId())
350 for measRecord, refRecord
in zip(measCat, refCat):
351 for plugin
in self.undeblendedPlugins.iter():
352 self.doMeasurement(plugin, measRecord, exposure, refRecord, refWcs)
355 """!Initialize an output SourceCatalog using information from the reference catalog.
357 This generates a new blank SourceRecord for each record in refCat. Note that this
358 method does not attach any Footprints. Doing so is up to the caller (who may
359 call attachedTransformedFootprints or define their own method - see run() for more
362 @param[in] exposure Exposure to be measured
363 @param[in] refCat Sequence (not necessarily a SourceCatalog) of reference SourceRecords.
364 @param[in] refWcs Wcs that defines the X,Y coordinate system of refCat
365 @param[in] idFactory factory for creating IDs for sources
367 @return Source catalog ready for measurement
369 if idFactory
is None:
370 idFactory = lsst.afw.table.IdFactory.makeSimple()
371 table = lsst.afw.table.SourceTable.make(self.
schema, idFactory)
372 measCat = lsst.afw.table.SourceCatalog(table)
373 table = measCat.table
374 table.setMetadata(self.algMetadata)
375 table.preallocate(len(refCat))
377 newSource = measCat.addNew()
378 newSource.assign(ref, self.
mapper)
382 """!Default implementation for attaching Footprints to blank sources prior to measurement
384 Footprints for forced photometry must be in the pixel coordinate system of the image being
385 measured, while the actual detections may start out in a different coordinate system.
386 This default implementation transforms the Footprints from the reference catalog from the
387 refWcs to the exposure's Wcs, which downgrades HeavyFootprints into regular Footprints,
388 destroying deblend information.
390 Note that ForcedPhotImageTask delegates to this method in its own attachFootprints method.
391 attachFootprints can then be overridden by its subclasses to define how their Footprints
394 See the documentation for run() for information about the relationships between run(),
395 generateMeasCat(), and attachTransformedFootprints().
397 exposureWcs = exposure.getWcs()
398 region = exposure.getBBox(lsst.afw.image.PARENT)
399 for srcRecord, refRecord
in zip(sources, refCat):
400 srcRecord.setFootprint(refRecord.getFootprint().transform(refWcs, exposureWcs, region))
A subtask for measuring the properties of sources on a single exposure, using an existing "reference"...
def __init__
Initialize the task.
def attachTransformedFootprints
Default implementation for attaching Footprints to blank sources prior to measurement.
def generateMeasCat
Initialize an output SourceCatalog using information from the reference catalog.
def run
Perform forced measurement.