Coverage for python/lsst/pipe/tasks/measurePsf.py: 17%
Shortcuts on this page
r m x p toggle line displays
j k next/prev highlighted chunk
0 (zero) top of page
1 (one) first highlighted chunk
Shortcuts on this page
r m x p toggle line displays
j k next/prev highlighted chunk
0 (zero) top of page
1 (one) first highlighted chunk
1# This file is part of pipe_tasks.
2#
3# Developed for the LSST Data Management System.
4# This product includes software developed by the LSST Project
5# (https://www.lsst.org).
6# See the COPYRIGHT file at the top-level directory of this distribution
7# for details of code ownership.
8#
9# This program is free software: you can redistribute it and/or modify
10# it under the terms of the GNU General Public License as published by
11# the Free Software Foundation, either version 3 of the License, or
12# (at your option) any later version.
13#
14# This program is distributed in the hope that it will be useful,
15# but WITHOUT ANY WARRANTY; without even the implied warranty of
16# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
17# GNU General Public License for more details.
18#
19# You should have received a copy of the GNU General Public License
20# along with this program. If not, see <https://www.gnu.org/licenses/>.
22import lsst.afw.display as afwDisplay
23import lsst.afw.math as afwMath
24import lsst.meas.algorithms as measAlg
25import lsst.meas.algorithms.utils as maUtils
26import lsst.pex.config as pexConfig
27import lsst.pipe.base as pipeBase
28import lsst.meas.extensions.piff.piffPsfDeterminer # noqa: F401
29from lsst.utils.timer import timeMethod
32class MeasurePsfConfig(pexConfig.Config):
33 starSelector = measAlg.sourceSelectorRegistry.makeField(
34 "Star selection algorithm",
35 default="objectSize"
36 )
37 makePsfCandidates = pexConfig.ConfigurableField(
38 target=measAlg.MakePsfCandidatesTask,
39 doc="Task to make psf candidates from selected stars.",
40 )
41 psfDeterminer = measAlg.psfDeterminerRegistry.makeField(
42 "PSF Determination algorithm",
43 default="piff"
44 )
45 reserve = pexConfig.ConfigurableField(
46 target=measAlg.ReserveSourcesTask,
47 doc="Reserve sources from fitting"
48 )
50## @addtogroup LSST_task_documentation
51## @{
52## @page page_MeasurePsfTask MeasurePsfTask
53## @ref MeasurePsfTask_ "MeasurePsfTask"
54## @copybrief MeasurePsfTask
55## @}
58class MeasurePsfTask(pipeBase.Task):
59 r"""!
60@anchor MeasurePsfTask_
62@brief Measure the PSF
64@section pipe_tasks_measurePsf_Contents Contents
66 - @ref pipe_tasks_measurePsf_Purpose
67 - @ref pipe_tasks_measurePsf_Initialize
68 - @ref pipe_tasks_measurePsf_IO
69 - @ref pipe_tasks_measurePsf_Config
70 - @ref pipe_tasks_measurePsf_Debug
71 - @ref pipe_tasks_measurePsf_Example
73@section pipe_tasks_measurePsf_Purpose Description
75A task that selects stars from a catalog of sources and uses those to measure the PSF.
77The star selector is a subclass of
78@ref lsst.meas.algorithms.starSelector.BaseStarSelectorTask "lsst.meas.algorithms.BaseStarSelectorTask"
79and the PSF determiner is a sublcass of
80@ref lsst.meas.algorithms.psfDeterminer.BasePsfDeterminerTask "lsst.meas.algorithms.BasePsfDeterminerTask"
82@warning
83There is no establised set of configuration parameters for these algorithms, so once you start modifying
84parameters (as we do in @ref pipe_tasks_measurePsf_Example) your code is no longer portable.
86@section pipe_tasks_measurePsf_Initialize Task initialisation
88@copydoc \_\_init\_\_
90@section pipe_tasks_measurePsf_IO Invoking the Task
92@copydoc run
94@section pipe_tasks_measurePsf_Config Configuration parameters
96See @ref MeasurePsfConfig.
98@section pipe_tasks_measurePsf_Debug Debug variables
100The command line task interface supports a
101flag @c -d to import @b debug.py from your @c PYTHONPATH; see
102<a href="https://pipelines.lsst.io/modules/lsstDebug/">the lsstDebug documentation</a>
103for more about @b debug.py files.
105<DL>
106 <DT> @c display
107 <DD> If True, display debugging plots
108 <DT> displayExposure
109 <DD> display the Exposure + spatialCells
110 <DT> displayPsfCandidates
111 <DD> show mosaic of candidates
112 <DT> showBadCandidates
113 <DD> Include bad candidates
114 <DT> displayPsfMosaic
115 <DD> show mosaic of reconstructed PSF(xy)
116 <DT> displayResiduals
117 <DD> show residuals
118 <DT> normalizeResiduals
119 <DD> Normalise residuals by object amplitude
120</DL>
122Additionally you can enable any debug outputs that your chosen star selector and psf determiner support.
124@section pipe_tasks_measurePsf_Example A complete example of using MeasurePsfTask
126This code is in `measurePsfTask.py` in the examples directory, and can be run as @em e.g.
127@code
128examples/measurePsfTask.py --doDisplay
129@endcode
130@dontinclude measurePsfTask.py
132The example also runs SourceDetectionTask and SingleFrameMeasurementTask.
134Import the tasks (there are some other standard imports; read the file to see them all):
136@skip SourceDetectionTask
137@until MeasurePsfTask
139We need to create the tasks before processing any data as the task constructor
140can add an extra column to the schema, but first we need an almost-empty
141Schema:
143@skipline makeMinimalSchema
145We can now call the constructors for the tasks we need to find and characterize candidate
146PSF stars:
148@skip SourceDetectionTask.ConfigClass
149@until measureTask
151Note that we've chosen a minimal set of measurement plugins: we need the
152outputs of @c base_SdssCentroid, @c base_SdssShape and @c base_CircularApertureFlux as
153inputs to the PSF measurement algorithm, while @c base_PixelFlags identifies
154and flags bad sources (e.g. with pixels too close to the edge) so they can be
155removed later.
157Now we can create and configure the task that we're interested in:
159@skip MeasurePsfTask
160@until measurePsfTask
162We're now ready to process the data (we could loop over multiple exposures/catalogues using the same
163task objects). First create the output table:
165@skipline afwTable
167And process the image:
169@skip sources =
170@until result
172We can then unpack and use the results:
174@skip psf
175@until cellSet
177If you specified @c --doDisplay you can see the PSF candidates:
179@skip display
180@until RED
182<HR>
184To investigate the @ref pipe_tasks_measurePsf_Debug, put something like
185@code{.py}
186 import lsstDebug
187 def DebugInfo(name):
188 di = lsstDebug.getInfo(name) # N.b. lsstDebug.Info(name) would call us recursively
190 if name == "lsst.pipe.tasks.measurePsf" :
191 di.display = True
192 di.displayExposure = False # display the Exposure + spatialCells
193 di.displayPsfCandidates = True # show mosaic of candidates
194 di.displayPsfMosaic = True # show mosaic of reconstructed PSF(xy)
195 di.displayResiduals = True # show residuals
196 di.showBadCandidates = True # Include bad candidates
197 di.normalizeResiduals = False # Normalise residuals by object amplitude
199 return di
201 lsstDebug.Info = DebugInfo
202@endcode
203into your debug.py file and run measurePsfTask.py with the @c --debug flag.
204 """
205 ConfigClass = MeasurePsfConfig
206 _DefaultName = "measurePsf"
208 def __init__(self, schema=None, **kwargs):
209 """!Create the detection task. Most arguments are simply passed onto pipe.base.Task.
211 @param schema An lsst::afw::table::Schema used to create the output lsst.afw.table.SourceCatalog
212 @param **kwargs Keyword arguments passed to lsst.pipe.base.task.Task.__init__.
214 If schema is not None, 'calib_psf_candidate' and 'calib_psf_used' fields will be added to
215 identify which stars were employed in the PSF estimation.
217 @note This task can add fields to the schema, so any code calling this task must ensure that
218 these fields are indeed present in the input table.
219 """
221 pipeBase.Task.__init__(self, **kwargs)
222 if schema is not None:
223 self.candidateKey = schema.addField(
224 "calib_psf_candidate", type="Flag",
225 doc=("Flag set if the source was a candidate for PSF determination, "
226 "as determined by the star selector.")
227 )
228 self.usedKey = schema.addField(
229 "calib_psf_used", type="Flag",
230 doc=("Flag set if the source was actually used for PSF determination, "
231 "as determined by the '%s' PSF determiner.") % self.config.psfDeterminer.name
232 )
233 else:
234 self.candidateKey = None
235 self.usedKey = None
236 self.makeSubtask("starSelector")
237 self.makeSubtask("makePsfCandidates")
238 self.makeSubtask("psfDeterminer", schema=schema)
239 self.makeSubtask("reserve", columnName="calib_psf", schema=schema,
240 doc="set if source was reserved from PSF determination")
242 @timeMethod
243 def run(self, exposure, sources, expId=0, matches=None):
244 """!Measure the PSF
246 @param[in,out] exposure Exposure to process; measured PSF will be added.
247 @param[in,out] sources Measured sources on exposure; flag fields will be set marking
248 stars chosen by the star selector and the PSF determiner if a schema
249 was passed to the task constructor.
250 @param[in] expId Exposure id used for generating random seed.
251 @param[in] matches A list of lsst.afw.table.ReferenceMatch objects
252 (@em i.e. of lsst.afw.table.Match
253 with @c first being of type lsst.afw.table.SimpleRecord and @c second
254 type lsst.afw.table.SourceRecord --- the reference object and detected
255 object respectively) as returned by @em e.g. the AstrometryTask.
256 Used by star selectors that choose to refer to an external catalog.
258 @return a pipe.base.Struct with fields:
259 - psf: The measured PSF (also set in the input exposure)
260 - cellSet: an lsst.afw.math.SpatialCellSet containing the PSF candidates
261 as returned by the psf determiner.
262 """
263 self.log.info("Measuring PSF")
265 import lsstDebug
266 display = lsstDebug.Info(__name__).display
267 displayExposure = lsstDebug.Info(__name__).displayExposure # display the Exposure + spatialCells
268 displayPsfMosaic = lsstDebug.Info(__name__).displayPsfMosaic # show mosaic of reconstructed PSF(x,y)
269 displayPsfCandidates = lsstDebug.Info(__name__).displayPsfCandidates # show mosaic of candidates
270 displayResiduals = lsstDebug.Info(__name__).displayResiduals # show residuals
271 showBadCandidates = lsstDebug.Info(__name__).showBadCandidates # include bad candidates
272 normalizeResiduals = lsstDebug.Info(__name__).normalizeResiduals # normalise residuals by object peak
274 #
275 # Run star selector
276 #
277 stars = self.starSelector.run(sourceCat=sources, matches=matches, exposure=exposure)
278 selectionResult = self.makePsfCandidates.run(stars.sourceCat, exposure=exposure)
279 self.log.info("PSF star selector found %d candidates", len(selectionResult.psfCandidates))
280 reserveResult = self.reserve.run(selectionResult.goodStarCat, expId=expId)
281 # Make list of psf candidates to send to the determiner (omitting those marked as reserved)
282 psfDeterminerList = [cand for cand, use
283 in zip(selectionResult.psfCandidates, reserveResult.use) if use]
285 if selectionResult.psfCandidates and self.candidateKey is not None:
286 for cand in selectionResult.psfCandidates:
287 source = cand.getSource()
288 source.set(self.candidateKey, True)
290 self.log.info("Sending %d candidates to PSF determiner", len(psfDeterminerList))
292 if display:
293 frame = 1
294 if displayExposure:
295 disp = afwDisplay.Display(frame=frame)
296 disp.mtv(exposure, title="psf determination")
297 frame += 1
298 #
299 # Determine PSF
300 #
301 psf, cellSet = self.psfDeterminer.determinePsf(exposure, psfDeterminerList, self.metadata,
302 flagKey=self.usedKey)
303 self.log.info("PSF determination using %d/%d stars.",
304 self.metadata.getScalar("numGoodStars"), self.metadata.getScalar("numAvailStars"))
306 exposure.setPsf(psf)
308 if display:
309 frame = display
310 if displayExposure:
311 disp = afwDisplay.Display(frame=frame)
312 showPsfSpatialCells(exposure, cellSet, showBadCandidates, frame=frame)
313 frame += 1
315 if displayPsfCandidates: # Show a mosaic of PSF candidates
316 plotPsfCandidates(cellSet, showBadCandidates=showBadCandidates, frame=frame)
317 frame += 1
319 if displayResiduals:
320 frame = plotResiduals(exposure, cellSet,
321 showBadCandidates=showBadCandidates,
322 normalizeResiduals=normalizeResiduals,
323 frame=frame)
324 if displayPsfMosaic:
325 disp = afwDisplay.Display(frame=frame)
326 maUtils.showPsfMosaic(exposure, psf, display=disp, showFwhm=True)
327 disp.scale("linear", 0, 1)
328 frame += 1
330 return pipeBase.Struct(
331 psf=psf,
332 cellSet=cellSet,
333 )
335 @property
336 def usesMatches(self):
337 """Return True if this task makes use of the "matches" argument to the run method"""
338 return self.starSelector.usesMatches
340#
341# Debug code
342#
345def showPsfSpatialCells(exposure, cellSet, showBadCandidates, frame=1):
346 disp = afwDisplay.Display(frame=frame)
347 maUtils.showPsfSpatialCells(exposure, cellSet,
348 symb="o", ctype=afwDisplay.CYAN, ctypeUnused=afwDisplay.YELLOW,
349 size=4, display=disp)
350 for cell in cellSet.getCellList():
351 for cand in cell.begin(not showBadCandidates): # maybe include bad candidates
352 status = cand.getStatus()
353 disp.dot('+', *cand.getSource().getCentroid(),
354 ctype=afwDisplay.GREEN if status == afwMath.SpatialCellCandidate.GOOD else
355 afwDisplay.YELLOW if status == afwMath.SpatialCellCandidate.UNKNOWN else afwDisplay.RED)
358def plotPsfCandidates(cellSet, showBadCandidates=False, frame=1):
359 stamps = []
360 for cell in cellSet.getCellList():
361 for cand in cell.begin(not showBadCandidates): # maybe include bad candidates
362 try:
363 im = cand.getMaskedImage()
365 chi2 = cand.getChi2()
366 if chi2 < 1e100:
367 chi2 = "%.1f" % chi2
368 else:
369 chi2 = float("nan")
371 stamps.append((im, "%d%s" %
372 (maUtils.splitId(cand.getSource().getId(), True)["objId"], chi2),
373 cand.getStatus()))
374 except Exception:
375 continue
377 mos = afwDisplay.utils.Mosaic()
378 disp = afwDisplay.Display(frame=frame)
379 for im, label, status in stamps:
380 im = type(im)(im, True)
381 try:
382 im /= afwMath.makeStatistics(im, afwMath.MAX).getValue()
383 except NotImplementedError:
384 pass
386 mos.append(im, label,
387 afwDisplay.GREEN if status == afwMath.SpatialCellCandidate.GOOD else
388 afwDisplay.YELLOW if status == afwMath.SpatialCellCandidate.UNKNOWN else afwDisplay.RED)
390 if mos.images:
391 disp.mtv(mos.makeMosaic(), title="Psf Candidates")
394def plotResiduals(exposure, cellSet, showBadCandidates=False, normalizeResiduals=True, frame=2):
395 psf = exposure.getPsf()
396 disp = afwDisplay.Display(frame=frame)
397 while True:
398 try:
399 maUtils.showPsfCandidates(exposure, cellSet, psf=psf, display=disp,
400 normalize=normalizeResiduals,
401 showBadCandidates=showBadCandidates)
402 frame += 1
403 maUtils.showPsfCandidates(exposure, cellSet, psf=psf, display=disp,
404 normalize=normalizeResiduals,
405 showBadCandidates=showBadCandidates,
406 variance=True)
407 frame += 1
408 except Exception:
409 if not showBadCandidates:
410 showBadCandidates = True
411 continue
412 break
414 return frame