Coverage for python/lsst/pipe/tasks/measurePsf.py : 13%

Hot-keys on this page
r m x p toggle line displays
j k next/prev highlighted chunk
0 (zero) top of page
1 (one) first highlighted chunk
1# This file is part of pipe_tasks.
2#
3# Developed for the LSST Data Management System.
4# This product includes software developed by the LSST Project
5# (https://www.lsst.org).
6# See the COPYRIGHT file at the top-level directory of this distribution
7# for details of code ownership.
8#
9# This program is free software: you can redistribute it and/or modify
10# it under the terms of the GNU General Public License as published by
11# the Free Software Foundation, either version 3 of the License, or
12# (at your option) any later version.
13#
14# This program is distributed in the hope that it will be useful,
15# but WITHOUT ANY WARRANTY; without even the implied warranty of
16# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
17# GNU General Public License for more details.
18#
19# You should have received a copy of the GNU General Public License
20# along with this program. If not, see <https://www.gnu.org/licenses/>.
22import lsst.afw.display as afwDisplay
23import lsst.afw.math as afwMath
24import lsst.meas.algorithms as measAlg
25import lsst.meas.algorithms.utils as maUtils
26import lsst.pex.config as pexConfig
27import lsst.pipe.base as pipeBase
30class MeasurePsfConfig(pexConfig.Config):
31 starSelector = measAlg.sourceSelectorRegistry.makeField(
32 "Star selection algorithm",
33 default="objectSize"
34 )
35 makePsfCandidates = pexConfig.ConfigurableField(
36 target=measAlg.MakePsfCandidatesTask,
37 doc="Task to make psf candidates from selected stars.",
38 )
39 psfDeterminer = measAlg.psfDeterminerRegistry.makeField(
40 "PSF Determination algorithm",
41 default="pca"
42 )
43 reserve = pexConfig.ConfigurableField(
44 target=measAlg.ReserveSourcesTask,
45 doc="Reserve sources from fitting"
46 )
48## @addtogroup LSST_task_documentation
49## @{
50## @page MeasurePsfTask
51## @ref MeasurePsfTask_ "MeasurePsfTask"
52## @copybrief MeasurePsfTask
53## @}
56class MeasurePsfTask(pipeBase.Task):
57 r"""!
58@anchor MeasurePsfTask_
60@brief Measure the PSF
62@section pipe_tasks_measurePsf_Contents Contents
64 - @ref pipe_tasks_measurePsf_Purpose
65 - @ref pipe_tasks_measurePsf_Initialize
66 - @ref pipe_tasks_measurePsf_IO
67 - @ref pipe_tasks_measurePsf_Config
68 - @ref pipe_tasks_measurePsf_Debug
69 - @ref pipe_tasks_measurePsf_Example
71@section pipe_tasks_measurePsf_Purpose Description
73A task that selects stars from a catalog of sources and uses those to measure the PSF.
75The star selector is a subclass of
76@ref lsst.meas.algorithms.starSelector.BaseStarSelectorTask "lsst.meas.algorithms.BaseStarSelectorTask"
77and the PSF determiner is a sublcass of
78@ref lsst.meas.algorithms.psfDeterminer.BasePsfDeterminerTask "lsst.meas.algorithms.BasePsfDeterminerTask"
80@warning
81There is no establised set of configuration parameters for these algorithms, so once you start modifying
82parameters (as we do in @ref pipe_tasks_measurePsf_Example) your code is no longer portable.
84@section pipe_tasks_measurePsf_Initialize Task initialisation
86@copydoc \_\_init\_\_
88@section pipe_tasks_measurePsf_IO Invoking the Task
90@copydoc run
92@section pipe_tasks_measurePsf_Config Configuration parameters
94See @ref MeasurePsfConfig.
96@section pipe_tasks_measurePsf_Debug Debug variables
98The @link lsst.pipe.base.cmdLineTask.CmdLineTask command line task@endlink interface supports a
99flag @c -d to import @b debug.py from your @c PYTHONPATH; see @ref baseDebug for more about @b debug.py files.
101<DL>
102 <DT> @c display
103 <DD> If True, display debugging plots
104 <DT> displayExposure
105 <DD> display the Exposure + spatialCells
106 <DT> displayPsfCandidates
107 <DD> show mosaic of candidates
108 <DT> showBadCandidates
109 <DD> Include bad candidates
110 <DT> displayPsfMosaic
111 <DD> show mosaic of reconstructed PSF(xy)
112 <DT> displayResiduals
113 <DD> show residuals
114 <DT> normalizeResiduals
115 <DD> Normalise residuals by object amplitude
116</DL>
118Additionally you can enable any debug outputs that your chosen star selector and psf determiner support.
120@section pipe_tasks_measurePsf_Example A complete example of using MeasurePsfTask
122This code is in @link measurePsfTask.py@endlink in the examples directory, and can be run as @em e.g.
123@code
124examples/measurePsfTask.py --doDisplay
125@endcode
126@dontinclude measurePsfTask.py
128The example also runs SourceDetectionTask and SingleFrameMeasurementTask;
129see @ref meas_algorithms_measurement_Example for more explanation.
131Import the tasks (there are some other standard imports; read the file to see them all):
133@skip SourceDetectionTask
134@until MeasurePsfTask
136We need to create the tasks before processing any data as the task constructor
137can add an extra column to the schema, but first we need an almost-empty
138Schema:
140@skipline makeMinimalSchema
142We can now call the constructors for the tasks we need to find and characterize candidate
143PSF stars:
145@skip SourceDetectionTask.ConfigClass
146@until measureTask
148Note that we've chosen a minimal set of measurement plugins: we need the
149outputs of @c base_SdssCentroid, @c base_SdssShape and @c base_CircularApertureFlux as
150inputs to the PSF measurement algorithm, while @c base_PixelFlags identifies
151and flags bad sources (e.g. with pixels too close to the edge) so they can be
152removed later.
154Now we can create and configure the task that we're interested in:
156@skip MeasurePsfTask
157@until measurePsfTask
159We're now ready to process the data (we could loop over multiple exposures/catalogues using the same
160task objects). First create the output table:
162@skipline afwTable
164And process the image:
166@skip sources =
167@until result
169We can then unpack and use the results:
171@skip psf
172@until cellSet
174If you specified @c --doDisplay you can see the PSF candidates:
176@skip display
177@until RED
179<HR>
181To investigate the @ref pipe_tasks_measurePsf_Debug, put something like
182@code{.py}
183 import lsstDebug
184 def DebugInfo(name):
185 di = lsstDebug.getInfo(name) # N.b. lsstDebug.Info(name) would call us recursively
187 if name == "lsst.pipe.tasks.measurePsf" :
188 di.display = True
189 di.displayExposure = False # display the Exposure + spatialCells
190 di.displayPsfCandidates = True # show mosaic of candidates
191 di.displayPsfMosaic = True # show mosaic of reconstructed PSF(xy)
192 di.displayResiduals = True # show residuals
193 di.showBadCandidates = True # Include bad candidates
194 di.normalizeResiduals = False # Normalise residuals by object amplitude
196 return di
198 lsstDebug.Info = DebugInfo
199@endcode
200into your debug.py file and run measurePsfTask.py with the @c --debug flag.
201 """
202 ConfigClass = MeasurePsfConfig
203 _DefaultName = "measurePsf"
205 def __init__(self, schema=None, **kwargs):
206 """!Create the detection task. Most arguments are simply passed onto pipe.base.Task.
208 @param schema An lsst::afw::table::Schema used to create the output lsst.afw.table.SourceCatalog
209 @param **kwargs Keyword arguments passed to lsst.pipe.base.task.Task.__init__.
211 If schema is not None, 'calib_psf_candidate' and 'calib_psf_used' fields will be added to
212 identify which stars were employed in the PSF estimation.
214 @note This task can add fields to the schema, so any code calling this task must ensure that
215 these fields are indeed present in the input table.
216 """
218 pipeBase.Task.__init__(self, **kwargs)
219 if schema is not None:
220 self.candidateKey = schema.addField(
221 "calib_psf_candidate", type="Flag",
222 doc=("Flag set if the source was a candidate for PSF determination, "
223 "as determined by the star selector.")
224 )
225 self.usedKey = schema.addField(
226 "calib_psf_used", type="Flag",
227 doc=("Flag set if the source was actually used for PSF determination, "
228 "as determined by the '%s' PSF determiner.") % self.config.psfDeterminer.name
229 )
230 else:
231 self.candidateKey = None
232 self.usedKey = None
233 self.makeSubtask("starSelector")
234 self.makeSubtask("makePsfCandidates")
235 self.makeSubtask("psfDeterminer", schema=schema)
236 self.makeSubtask("reserve", columnName="calib_psf", schema=schema,
237 doc="set if source was reserved from PSF determination")
239 @pipeBase.timeMethod
240 def run(self, exposure, sources, expId=0, matches=None):
241 """!Measure the PSF
243 @param[in,out] exposure Exposure to process; measured PSF will be added.
244 @param[in,out] sources Measured sources on exposure; flag fields will be set marking
245 stars chosen by the star selector and the PSF determiner if a schema
246 was passed to the task constructor.
247 @param[in] expId Exposure id used for generating random seed.
248 @param[in] matches A list of lsst.afw.table.ReferenceMatch objects
249 (@em i.e. of lsst.afw.table.Match
250 with @c first being of type lsst.afw.table.SimpleRecord and @c second
251 type lsst.afw.table.SourceRecord --- the reference object and detected
252 object respectively) as returned by @em e.g. the AstrometryTask.
253 Used by star selectors that choose to refer to an external catalog.
255 @return a pipe.base.Struct with fields:
256 - psf: The measured PSF (also set in the input exposure)
257 - cellSet: an lsst.afw.math.SpatialCellSet containing the PSF candidates
258 as returned by the psf determiner.
259 """
260 self.log.info("Measuring PSF")
262 import lsstDebug
263 display = lsstDebug.Info(__name__).display
264 displayExposure = lsstDebug.Info(__name__).displayExposure # display the Exposure + spatialCells
265 displayPsfMosaic = lsstDebug.Info(__name__).displayPsfMosaic # show mosaic of reconstructed PSF(x,y)
266 displayPsfCandidates = lsstDebug.Info(__name__).displayPsfCandidates # show mosaic of candidates
267 displayResiduals = lsstDebug.Info(__name__).displayResiduals # show residuals
268 showBadCandidates = lsstDebug.Info(__name__).showBadCandidates # include bad candidates
269 normalizeResiduals = lsstDebug.Info(__name__).normalizeResiduals # normalise residuals by object peak
271 #
272 # Run star selector
273 #
274 stars = self.starSelector.run(sourceCat=sources, matches=matches, exposure=exposure)
275 selectionResult = self.makePsfCandidates.run(stars.sourceCat, exposure=exposure)
276 self.log.info("PSF star selector found %d candidates" % len(selectionResult.psfCandidates))
277 reserveResult = self.reserve.run(selectionResult.goodStarCat, expId=expId)
278 # Make list of psf candidates to send to the determiner (omitting those marked as reserved)
279 psfDeterminerList = [cand for cand, use
280 in zip(selectionResult.psfCandidates, reserveResult.use) if use]
282 if selectionResult.psfCandidates and self.candidateKey is not None:
283 for cand in selectionResult.psfCandidates:
284 source = cand.getSource()
285 source.set(self.candidateKey, True)
287 self.log.info("Sending %d candidates to PSF determiner" % len(psfDeterminerList))
289 if display:
290 frame = 1
291 if displayExposure:
292 disp = afwDisplay.Display(frame=frame)
293 disp.mtv(exposure, title="psf determination")
294 frame += 1
295 #
296 # Determine PSF
297 #
298 psf, cellSet = self.psfDeterminer.determinePsf(exposure, psfDeterminerList, self.metadata,
299 flagKey=self.usedKey)
300 self.log.info("PSF determination using %d/%d stars." %
301 (self.metadata.getScalar("numGoodStars"), self.metadata.getScalar("numAvailStars")))
303 exposure.setPsf(psf)
305 if display:
306 frame = display
307 if displayExposure:
308 disp = afwDisplay.Display(frame=frame)
309 showPsfSpatialCells(exposure, cellSet, showBadCandidates, frame=frame)
310 frame += 1
312 if displayPsfCandidates: # Show a mosaic of PSF candidates
313 plotPsfCandidates(cellSet, showBadCandidates=showBadCandidates, frame=frame)
314 frame += 1
316 if displayResiduals:
317 frame = plotResiduals(exposure, cellSet,
318 showBadCandidates=showBadCandidates,
319 normalizeResiduals=normalizeResiduals,
320 frame=frame)
321 if displayPsfMosaic:
322 disp = afwDisplay.Display(frame=frame)
323 maUtils.showPsfMosaic(exposure, psf, display=disp, showFwhm=True)
324 disp.scale("linear", 0, 1)
325 frame += 1
327 return pipeBase.Struct(
328 psf=psf,
329 cellSet=cellSet,
330 )
332 @property
333 def usesMatches(self):
334 """Return True if this task makes use of the "matches" argument to the run method"""
335 return self.starSelector.usesMatches
337#
338# Debug code
339#
342def showPsfSpatialCells(exposure, cellSet, showBadCandidates, frame=1):
343 disp = afwDisplay.Display(frame=frame)
344 maUtils.showPsfSpatialCells(exposure, cellSet,
345 symb="o", ctype=afwDisplay.CYAN, ctypeUnused=afwDisplay.YELLOW,
346 size=4, display=disp)
347 for cell in cellSet.getCellList():
348 for cand in cell.begin(not showBadCandidates): # maybe include bad candidates
349 status = cand.getStatus()
350 disp.dot('+', *cand.getSource().getCentroid(),
351 ctype=afwDisplay.GREEN if status == afwMath.SpatialCellCandidate.GOOD else
352 afwDisplay.YELLOW if status == afwMath.SpatialCellCandidate.UNKNOWN else afwDisplay.RED)
355def plotPsfCandidates(cellSet, showBadCandidates=False, frame=1):
356 stamps = []
357 for cell in cellSet.getCellList():
358 for cand in cell.begin(not showBadCandidates): # maybe include bad candidates
359 try:
360 im = cand.getMaskedImage()
362 chi2 = cand.getChi2()
363 if chi2 < 1e100:
364 chi2 = "%.1f" % chi2
365 else:
366 chi2 = float("nan")
368 stamps.append((im, "%d%s" %
369 (maUtils.splitId(cand.getSource().getId(), True)["objId"], chi2),
370 cand.getStatus()))
371 except Exception:
372 continue
374 mos = afwDisplay.utils.Mosaic()
375 disp = afwDisplay.Display(frame=frame)
376 for im, label, status in stamps:
377 im = type(im)(im, True)
378 try:
379 im /= afwMath.makeStatistics(im, afwMath.MAX).getValue()
380 except NotImplementedError:
381 pass
383 mos.append(im, label,
384 afwDisplay.GREEN if status == afwMath.SpatialCellCandidate.GOOD else
385 afwDisplay.YELLOW if status == afwMath.SpatialCellCandidate.UNKNOWN else afwDisplay.RED)
387 if mos.images:
388 disp.mtv(mos.makeMosaic(), title="Psf Candidates")
391def plotResiduals(exposure, cellSet, showBadCandidates=False, normalizeResiduals=True, frame=2):
392 psf = exposure.getPsf()
393 disp = afwDisplay.Display(frame=frame)
394 while True:
395 try:
396 maUtils.showPsfCandidates(exposure, cellSet, psf=psf, display=disp,
397 normalize=normalizeResiduals,
398 showBadCandidates=showBadCandidates)
399 frame += 1
400 maUtils.showPsfCandidates(exposure, cellSet, psf=psf, display=disp,
401 normalize=normalizeResiduals,
402 showBadCandidates=showBadCandidates,
403 variance=True)
404 frame += 1
405 except Exception:
406 if not showBadCandidates:
407 showBadCandidates = True
408 continue
409 break
411 return frame