Coverage for python/lsst/pipe/tasks/measurePsf.py : 40%

Hot-keys on this page
r m x p toggle line displays
j k next/prev highlighted chunk
0 (zero) top of page
1 (one) first highlighted chunk
1# This file is part of pipe_tasks.
2#
3# Developed for the LSST Data Management System.
4# This product includes software developed by the LSST Project
5# (https://www.lsst.org).
6# See the COPYRIGHT file at the top-level directory of this distribution
7# for details of code ownership.
8#
9# This program is free software: you can redistribute it and/or modify
10# it under the terms of the GNU General Public License as published by
11# the Free Software Foundation, either version 3 of the License, or
12# (at your option) any later version.
13#
14# This program is distributed in the hope that it will be useful,
15# but WITHOUT ANY WARRANTY; without even the implied warranty of
16# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
17# GNU General Public License for more details.
18#
19# You should have received a copy of the GNU General Public License
20# along with this program. If not, see <https://www.gnu.org/licenses/>.
22import lsst.afw.display as afwDisplay
23import lsst.afw.math as afwMath
24import lsst.meas.algorithms as measAlg
25import lsst.meas.algorithms.utils as maUtils
26import lsst.pex.config as pexConfig
27import lsst.pipe.base as pipeBase
28import lsst.meas.extensions.psfex.psfexPsfDeterminer # noqa: F401
31class MeasurePsfConfig(pexConfig.Config):
32 starSelector = measAlg.sourceSelectorRegistry.makeField(
33 "Star selection algorithm",
34 default="objectSize"
35 )
36 makePsfCandidates = pexConfig.ConfigurableField(
37 target=measAlg.MakePsfCandidatesTask,
38 doc="Task to make psf candidates from selected stars.",
39 )
40 psfDeterminer = measAlg.psfDeterminerRegistry.makeField(
41 "PSF Determination algorithm",
42 default="psfex"
43 )
44 reserve = pexConfig.ConfigurableField(
45 target=measAlg.ReserveSourcesTask,
46 doc="Reserve sources from fitting"
47 )
49## @addtogroup LSST_task_documentation
50## @{
51## @page MeasurePsfTask
52## @ref MeasurePsfTask_ "MeasurePsfTask"
53## @copybrief MeasurePsfTask
54## @}
57class MeasurePsfTask(pipeBase.Task):
58 r"""!
59@anchor MeasurePsfTask_
61@brief Measure the PSF
63@section pipe_tasks_measurePsf_Contents Contents
65 - @ref pipe_tasks_measurePsf_Purpose
66 - @ref pipe_tasks_measurePsf_Initialize
67 - @ref pipe_tasks_measurePsf_IO
68 - @ref pipe_tasks_measurePsf_Config
69 - @ref pipe_tasks_measurePsf_Debug
70 - @ref pipe_tasks_measurePsf_Example
72@section pipe_tasks_measurePsf_Purpose Description
74A task that selects stars from a catalog of sources and uses those to measure the PSF.
76The star selector is a subclass of
77@ref lsst.meas.algorithms.starSelector.BaseStarSelectorTask "lsst.meas.algorithms.BaseStarSelectorTask"
78and the PSF determiner is a sublcass of
79@ref lsst.meas.algorithms.psfDeterminer.BasePsfDeterminerTask "lsst.meas.algorithms.BasePsfDeterminerTask"
81@warning
82There is no establised set of configuration parameters for these algorithms, so once you start modifying
83parameters (as we do in @ref pipe_tasks_measurePsf_Example) your code is no longer portable.
85@section pipe_tasks_measurePsf_Initialize Task initialisation
87@copydoc \_\_init\_\_
89@section pipe_tasks_measurePsf_IO Invoking the Task
91@copydoc run
93@section pipe_tasks_measurePsf_Config Configuration parameters
95See @ref MeasurePsfConfig.
97@section pipe_tasks_measurePsf_Debug Debug variables
99The @link lsst.pipe.base.cmdLineTask.CmdLineTask command line task@endlink interface supports a
100flag @c -d to import @b debug.py from your @c PYTHONPATH; see @ref baseDebug for more about @b debug.py files.
102<DL>
103 <DT> @c display
104 <DD> If True, display debugging plots
105 <DT> displayExposure
106 <DD> display the Exposure + spatialCells
107 <DT> displayPsfCandidates
108 <DD> show mosaic of candidates
109 <DT> showBadCandidates
110 <DD> Include bad candidates
111 <DT> displayPsfMosaic
112 <DD> show mosaic of reconstructed PSF(xy)
113 <DT> displayResiduals
114 <DD> show residuals
115 <DT> normalizeResiduals
116 <DD> Normalise residuals by object amplitude
117</DL>
119Additionally you can enable any debug outputs that your chosen star selector and psf determiner support.
121@section pipe_tasks_measurePsf_Example A complete example of using MeasurePsfTask
123This code is in @link measurePsfTask.py@endlink in the examples directory, and can be run as @em e.g.
124@code
125examples/measurePsfTask.py --doDisplay
126@endcode
127@dontinclude measurePsfTask.py
129The example also runs SourceDetectionTask and SingleFrameMeasurementTask;
130see @ref meas_algorithms_measurement_Example for more explanation.
132Import the tasks (there are some other standard imports; read the file to see them all):
134@skip SourceDetectionTask
135@until MeasurePsfTask
137We need to create the tasks before processing any data as the task constructor
138can add an extra column to the schema, but first we need an almost-empty
139Schema:
141@skipline makeMinimalSchema
143We can now call the constructors for the tasks we need to find and characterize candidate
144PSF stars:
146@skip SourceDetectionTask.ConfigClass
147@until measureTask
149Note that we've chosen a minimal set of measurement plugins: we need the
150outputs of @c base_SdssCentroid, @c base_SdssShape and @c base_CircularApertureFlux as
151inputs to the PSF measurement algorithm, while @c base_PixelFlags identifies
152and flags bad sources (e.g. with pixels too close to the edge) so they can be
153removed later.
155Now we can create and configure the task that we're interested in:
157@skip MeasurePsfTask
158@until measurePsfTask
160We're now ready to process the data (we could loop over multiple exposures/catalogues using the same
161task objects). First create the output table:
163@skipline afwTable
165And process the image:
167@skip sources =
168@until result
170We can then unpack and use the results:
172@skip psf
173@until cellSet
175If you specified @c --doDisplay you can see the PSF candidates:
177@skip display
178@until RED
180<HR>
182To investigate the @ref pipe_tasks_measurePsf_Debug, put something like
183@code{.py}
184 import lsstDebug
185 def DebugInfo(name):
186 di = lsstDebug.getInfo(name) # N.b. lsstDebug.Info(name) would call us recursively
188 if name == "lsst.pipe.tasks.measurePsf" :
189 di.display = True
190 di.displayExposure = False # display the Exposure + spatialCells
191 di.displayPsfCandidates = True # show mosaic of candidates
192 di.displayPsfMosaic = True # show mosaic of reconstructed PSF(xy)
193 di.displayResiduals = True # show residuals
194 di.showBadCandidates = True # Include bad candidates
195 di.normalizeResiduals = False # Normalise residuals by object amplitude
197 return di
199 lsstDebug.Info = DebugInfo
200@endcode
201into your debug.py file and run measurePsfTask.py with the @c --debug flag.
202 """
203 ConfigClass = MeasurePsfConfig
204 _DefaultName = "measurePsf"
206 def __init__(self, schema=None, **kwargs):
207 """!Create the detection task. Most arguments are simply passed onto pipe.base.Task.
209 @param schema An lsst::afw::table::Schema used to create the output lsst.afw.table.SourceCatalog
210 @param **kwargs Keyword arguments passed to lsst.pipe.base.task.Task.__init__.
212 If schema is not None, 'calib_psf_candidate' and 'calib_psf_used' fields will be added to
213 identify which stars were employed in the PSF estimation.
215 @note This task can add fields to the schema, so any code calling this task must ensure that
216 these fields are indeed present in the input table.
217 """
219 pipeBase.Task.__init__(self, **kwargs)
220 if schema is not None: 220 ↛ 232line 220 didn't jump to line 232, because the condition on line 220 was never false
221 self.candidateKey = schema.addField(
222 "calib_psf_candidate", type="Flag",
223 doc=("Flag set if the source was a candidate for PSF determination, "
224 "as determined by the star selector.")
225 )
226 self.usedKey = schema.addField(
227 "calib_psf_used", type="Flag",
228 doc=("Flag set if the source was actually used for PSF determination, "
229 "as determined by the '%s' PSF determiner.") % self.config.psfDeterminer.name
230 )
231 else:
232 self.candidateKey = None
233 self.usedKey = None
234 self.makeSubtask("starSelector")
235 self.makeSubtask("makePsfCandidates")
236 self.makeSubtask("psfDeterminer", schema=schema)
237 self.makeSubtask("reserve", columnName="calib_psf", schema=schema,
238 doc="set if source was reserved from PSF determination")
240 @pipeBase.timeMethod
241 def run(self, exposure, sources, expId=0, matches=None):
242 """!Measure the PSF
244 @param[in,out] exposure Exposure to process; measured PSF will be added.
245 @param[in,out] sources Measured sources on exposure; flag fields will be set marking
246 stars chosen by the star selector and the PSF determiner if a schema
247 was passed to the task constructor.
248 @param[in] expId Exposure id used for generating random seed.
249 @param[in] matches A list of lsst.afw.table.ReferenceMatch objects
250 (@em i.e. of lsst.afw.table.Match
251 with @c first being of type lsst.afw.table.SimpleRecord and @c second
252 type lsst.afw.table.SourceRecord --- the reference object and detected
253 object respectively) as returned by @em e.g. the AstrometryTask.
254 Used by star selectors that choose to refer to an external catalog.
256 @return a pipe.base.Struct with fields:
257 - psf: The measured PSF (also set in the input exposure)
258 - cellSet: an lsst.afw.math.SpatialCellSet containing the PSF candidates
259 as returned by the psf determiner.
260 """
261 self.log.info("Measuring PSF")
263 import lsstDebug
264 display = lsstDebug.Info(__name__).display
265 displayExposure = lsstDebug.Info(__name__).displayExposure # display the Exposure + spatialCells
266 displayPsfMosaic = lsstDebug.Info(__name__).displayPsfMosaic # show mosaic of reconstructed PSF(x,y)
267 displayPsfCandidates = lsstDebug.Info(__name__).displayPsfCandidates # show mosaic of candidates
268 displayResiduals = lsstDebug.Info(__name__).displayResiduals # show residuals
269 showBadCandidates = lsstDebug.Info(__name__).showBadCandidates # include bad candidates
270 normalizeResiduals = lsstDebug.Info(__name__).normalizeResiduals # normalise residuals by object peak
272 #
273 # Run star selector
274 #
275 stars = self.starSelector.run(sourceCat=sources, matches=matches, exposure=exposure)
276 selectionResult = self.makePsfCandidates.run(stars.sourceCat, exposure=exposure)
277 self.log.info("PSF star selector found %d candidates" % len(selectionResult.psfCandidates))
278 reserveResult = self.reserve.run(selectionResult.goodStarCat, expId=expId)
279 # Make list of psf candidates to send to the determiner (omitting those marked as reserved)
280 psfDeterminerList = [cand for cand, use
281 in zip(selectionResult.psfCandidates, reserveResult.use) if use]
283 if selectionResult.psfCandidates and self.candidateKey is not None: 283 ↛ 288line 283 didn't jump to line 288, because the condition on line 283 was never false
284 for cand in selectionResult.psfCandidates:
285 source = cand.getSource()
286 source.set(self.candidateKey, True)
288 self.log.info("Sending %d candidates to PSF determiner" % len(psfDeterminerList))
290 if display: 290 ↛ 291line 290 didn't jump to line 291, because the condition on line 290 was never true
291 frame = 1
292 if displayExposure:
293 disp = afwDisplay.Display(frame=frame)
294 disp.mtv(exposure, title="psf determination")
295 frame += 1
296 #
297 # Determine PSF
298 #
299 psf, cellSet = self.psfDeterminer.determinePsf(exposure, psfDeterminerList, self.metadata,
300 flagKey=self.usedKey)
301 self.log.info("PSF determination using %d/%d stars." %
302 (self.metadata.getScalar("numGoodStars"), self.metadata.getScalar("numAvailStars")))
304 exposure.setPsf(psf)
306 if display: 306 ↛ 307line 306 didn't jump to line 307, because the condition on line 306 was never true
307 frame = display
308 if displayExposure:
309 disp = afwDisplay.Display(frame=frame)
310 showPsfSpatialCells(exposure, cellSet, showBadCandidates, frame=frame)
311 frame += 1
313 if displayPsfCandidates: # Show a mosaic of PSF candidates
314 plotPsfCandidates(cellSet, showBadCandidates=showBadCandidates, frame=frame)
315 frame += 1
317 if displayResiduals:
318 frame = plotResiduals(exposure, cellSet,
319 showBadCandidates=showBadCandidates,
320 normalizeResiduals=normalizeResiduals,
321 frame=frame)
322 if displayPsfMosaic:
323 disp = afwDisplay.Display(frame=frame)
324 maUtils.showPsfMosaic(exposure, psf, display=disp, showFwhm=True)
325 disp.scale("linear", 0, 1)
326 frame += 1
328 return pipeBase.Struct(
329 psf=psf,
330 cellSet=cellSet,
331 )
333 @property
334 def usesMatches(self):
335 """Return True if this task makes use of the "matches" argument to the run method"""
336 return self.starSelector.usesMatches
338#
339# Debug code
340#
343def showPsfSpatialCells(exposure, cellSet, showBadCandidates, frame=1):
344 disp = afwDisplay.Display(frame=frame)
345 maUtils.showPsfSpatialCells(exposure, cellSet,
346 symb="o", ctype=afwDisplay.CYAN, ctypeUnused=afwDisplay.YELLOW,
347 size=4, display=disp)
348 for cell in cellSet.getCellList():
349 for cand in cell.begin(not showBadCandidates): # maybe include bad candidates
350 status = cand.getStatus()
351 disp.dot('+', *cand.getSource().getCentroid(),
352 ctype=afwDisplay.GREEN if status == afwMath.SpatialCellCandidate.GOOD else
353 afwDisplay.YELLOW if status == afwMath.SpatialCellCandidate.UNKNOWN else afwDisplay.RED)
356def plotPsfCandidates(cellSet, showBadCandidates=False, frame=1):
357 stamps = []
358 for cell in cellSet.getCellList():
359 for cand in cell.begin(not showBadCandidates): # maybe include bad candidates
360 try:
361 im = cand.getMaskedImage()
363 chi2 = cand.getChi2()
364 if chi2 < 1e100:
365 chi2 = "%.1f" % chi2
366 else:
367 chi2 = float("nan")
369 stamps.append((im, "%d%s" %
370 (maUtils.splitId(cand.getSource().getId(), True)["objId"], chi2),
371 cand.getStatus()))
372 except Exception:
373 continue
375 mos = afwDisplay.utils.Mosaic()
376 disp = afwDisplay.Display(frame=frame)
377 for im, label, status in stamps:
378 im = type(im)(im, True)
379 try:
380 im /= afwMath.makeStatistics(im, afwMath.MAX).getValue()
381 except NotImplementedError:
382 pass
384 mos.append(im, label,
385 afwDisplay.GREEN if status == afwMath.SpatialCellCandidate.GOOD else
386 afwDisplay.YELLOW if status == afwMath.SpatialCellCandidate.UNKNOWN else afwDisplay.RED)
388 if mos.images:
389 disp.mtv(mos.makeMosaic(), title="Psf Candidates")
392def plotResiduals(exposure, cellSet, showBadCandidates=False, normalizeResiduals=True, frame=2):
393 psf = exposure.getPsf()
394 disp = afwDisplay.Display(frame=frame)
395 while True:
396 try:
397 maUtils.showPsfCandidates(exposure, cellSet, psf=psf, display=disp,
398 normalize=normalizeResiduals,
399 showBadCandidates=showBadCandidates)
400 frame += 1
401 maUtils.showPsfCandidates(exposure, cellSet, psf=psf, display=disp,
402 normalize=normalizeResiduals,
403 showBadCandidates=showBadCandidates,
404 variance=True)
405 frame += 1
406 except Exception:
407 if not showBadCandidates:
408 showBadCandidates = True
409 continue
410 break
412 return frame