Coverage for python/lsst/pipe/tasks/measurePsf.py: 17%

Shortcuts on this page

r m x p   toggle line displays

j k   next/prev highlighted chunk

0   (zero) top of page

1   (one) first highlighted chunk

122 statements  

1# This file is part of pipe_tasks. 

2# 

3# Developed for the LSST Data Management System. 

4# This product includes software developed by the LSST Project 

5# (https://www.lsst.org). 

6# See the COPYRIGHT file at the top-level directory of this distribution 

7# for details of code ownership. 

8# 

9# This program is free software: you can redistribute it and/or modify 

10# it under the terms of the GNU General Public License as published by 

11# the Free Software Foundation, either version 3 of the License, or 

12# (at your option) any later version. 

13# 

14# This program is distributed in the hope that it will be useful, 

15# but WITHOUT ANY WARRANTY; without even the implied warranty of 

16# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 

17# GNU General Public License for more details. 

18# 

19# You should have received a copy of the GNU General Public License 

20# along with this program. If not, see <https://www.gnu.org/licenses/>. 

21 

22import lsst.afw.display as afwDisplay 

23import lsst.afw.math as afwMath 

24import lsst.meas.algorithms as measAlg 

25import lsst.meas.algorithms.utils as maUtils 

26import lsst.pex.config as pexConfig 

27import lsst.pipe.base as pipeBase 

28import lsst.meas.extensions.piff.piffPsfDeterminer # noqa: F401 

29from lsst.utils.timer import timeMethod 

30 

31 

32class MeasurePsfConfig(pexConfig.Config): 

33 starSelector = measAlg.sourceSelectorRegistry.makeField( 

34 "Star selection algorithm", 

35 default="objectSize" 

36 ) 

37 makePsfCandidates = pexConfig.ConfigurableField( 

38 target=measAlg.MakePsfCandidatesTask, 

39 doc="Task to make psf candidates from selected stars.", 

40 ) 

41 psfDeterminer = measAlg.psfDeterminerRegistry.makeField( 

42 "PSF Determination algorithm", 

43 default="piff" 

44 ) 

45 reserve = pexConfig.ConfigurableField( 

46 target=measAlg.ReserveSourcesTask, 

47 doc="Reserve sources from fitting" 

48 ) 

49 

50## @addtogroup LSST_task_documentation 

51## @{ 

52## @page page_MeasurePsfTask MeasurePsfTask 

53## @ref MeasurePsfTask_ "MeasurePsfTask" 

54## @copybrief MeasurePsfTask 

55## @} 

56 

57 

58class MeasurePsfTask(pipeBase.Task): 

59 r"""! 

60@anchor MeasurePsfTask_ 

61 

62@brief Measure the PSF 

63 

64@section pipe_tasks_measurePsf_Contents Contents 

65 

66 - @ref pipe_tasks_measurePsf_Purpose 

67 - @ref pipe_tasks_measurePsf_Initialize 

68 - @ref pipe_tasks_measurePsf_IO 

69 - @ref pipe_tasks_measurePsf_Config 

70 - @ref pipe_tasks_measurePsf_Debug 

71 - @ref pipe_tasks_measurePsf_Example 

72 

73@section pipe_tasks_measurePsf_Purpose Description 

74 

75A task that selects stars from a catalog of sources and uses those to measure the PSF. 

76 

77The star selector is a subclass of 

78@ref lsst.meas.algorithms.starSelector.BaseStarSelectorTask "lsst.meas.algorithms.BaseStarSelectorTask" 

79and the PSF determiner is a sublcass of 

80@ref lsst.meas.algorithms.psfDeterminer.BasePsfDeterminerTask "lsst.meas.algorithms.BasePsfDeterminerTask" 

81 

82@warning 

83There is no establised set of configuration parameters for these algorithms, so once you start modifying 

84parameters (as we do in @ref pipe_tasks_measurePsf_Example) your code is no longer portable. 

85 

86@section pipe_tasks_measurePsf_Initialize Task initialisation 

87 

88@copydoc \_\_init\_\_ 

89 

90@section pipe_tasks_measurePsf_IO Invoking the Task 

91 

92@copydoc run 

93 

94@section pipe_tasks_measurePsf_Config Configuration parameters 

95 

96See @ref MeasurePsfConfig. 

97 

98@section pipe_tasks_measurePsf_Debug Debug variables 

99 

100The command line task interface supports a 

101flag @c -d to import @b debug.py from your @c PYTHONPATH; see 

102<a href="https://pipelines.lsst.io/modules/lsstDebug/">the lsstDebug documentation</a> 

103for more about @b debug.py files. 

104 

105<DL> 

106 <DT> @c display 

107 <DD> If True, display debugging plots 

108 <DT> displayExposure 

109 <DD> display the Exposure + spatialCells 

110 <DT> displayPsfCandidates 

111 <DD> show mosaic of candidates 

112 <DT> showBadCandidates 

113 <DD> Include bad candidates 

114 <DT> displayPsfMosaic 

115 <DD> show mosaic of reconstructed PSF(xy) 

116 <DT> displayResiduals 

117 <DD> show residuals 

118 <DT> normalizeResiduals 

119 <DD> Normalise residuals by object amplitude 

120</DL> 

121 

122Additionally you can enable any debug outputs that your chosen star selector and psf determiner support. 

123 

124@section pipe_tasks_measurePsf_Example A complete example of using MeasurePsfTask 

125 

126This code is in `measurePsfTask.py` in the examples directory, and can be run as @em e.g. 

127@code 

128examples/measurePsfTask.py --doDisplay 

129@endcode 

130@dontinclude measurePsfTask.py 

131 

132The example also runs SourceDetectionTask and SingleFrameMeasurementTask. 

133 

134Import the tasks (there are some other standard imports; read the file to see them all): 

135 

136@skip SourceDetectionTask 

137@until MeasurePsfTask 

138 

139We need to create the tasks before processing any data as the task constructor 

140can add an extra column to the schema, but first we need an almost-empty 

141Schema: 

142 

143@skipline makeMinimalSchema 

144 

145We can now call the constructors for the tasks we need to find and characterize candidate 

146PSF stars: 

147 

148@skip SourceDetectionTask.ConfigClass 

149@until measureTask 

150 

151Note that we've chosen a minimal set of measurement plugins: we need the 

152outputs of @c base_SdssCentroid, @c base_SdssShape and @c base_CircularApertureFlux as 

153inputs to the PSF measurement algorithm, while @c base_PixelFlags identifies 

154and flags bad sources (e.g. with pixels too close to the edge) so they can be 

155removed later. 

156 

157Now we can create and configure the task that we're interested in: 

158 

159@skip MeasurePsfTask 

160@until measurePsfTask 

161 

162We're now ready to process the data (we could loop over multiple exposures/catalogues using the same 

163task objects). First create the output table: 

164 

165@skipline afwTable 

166 

167And process the image: 

168 

169@skip sources = 

170@until result 

171 

172We can then unpack and use the results: 

173 

174@skip psf 

175@until cellSet 

176 

177If you specified @c --doDisplay you can see the PSF candidates: 

178 

179@skip display 

180@until RED 

181 

182<HR> 

183 

184To investigate the @ref pipe_tasks_measurePsf_Debug, put something like 

185@code{.py} 

186 import lsstDebug 

187 def DebugInfo(name): 

188 di = lsstDebug.getInfo(name) # N.b. lsstDebug.Info(name) would call us recursively 

189 

190 if name == "lsst.pipe.tasks.measurePsf" : 

191 di.display = True 

192 di.displayExposure = False # display the Exposure + spatialCells 

193 di.displayPsfCandidates = True # show mosaic of candidates 

194 di.displayPsfMosaic = True # show mosaic of reconstructed PSF(xy) 

195 di.displayResiduals = True # show residuals 

196 di.showBadCandidates = True # Include bad candidates 

197 di.normalizeResiduals = False # Normalise residuals by object amplitude 

198 

199 return di 

200 

201 lsstDebug.Info = DebugInfo 

202@endcode 

203into your debug.py file and run measurePsfTask.py with the @c --debug flag. 

204 """ 

205 ConfigClass = MeasurePsfConfig 

206 _DefaultName = "measurePsf" 

207 

208 def __init__(self, schema=None, **kwargs): 

209 """!Create the detection task. Most arguments are simply passed onto pipe.base.Task. 

210 

211 @param schema An lsst::afw::table::Schema used to create the output lsst.afw.table.SourceCatalog 

212 @param **kwargs Keyword arguments passed to lsst.pipe.base.task.Task.__init__. 

213 

214 If schema is not None, 'calib_psf_candidate' and 'calib_psf_used' fields will be added to 

215 identify which stars were employed in the PSF estimation. 

216 

217 @note This task can add fields to the schema, so any code calling this task must ensure that 

218 these fields are indeed present in the input table. 

219 """ 

220 

221 pipeBase.Task.__init__(self, **kwargs) 

222 if schema is not None: 

223 self.candidateKey = schema.addField( 

224 "calib_psf_candidate", type="Flag", 

225 doc=("Flag set if the source was a candidate for PSF determination, " 

226 "as determined by the star selector.") 

227 ) 

228 self.usedKey = schema.addField( 

229 "calib_psf_used", type="Flag", 

230 doc=("Flag set if the source was actually used for PSF determination, " 

231 "as determined by the '%s' PSF determiner.") % self.config.psfDeterminer.name 

232 ) 

233 else: 

234 self.candidateKey = None 

235 self.usedKey = None 

236 self.makeSubtask("starSelector") 

237 self.makeSubtask("makePsfCandidates") 

238 self.makeSubtask("psfDeterminer", schema=schema) 

239 self.makeSubtask("reserve", columnName="calib_psf", schema=schema, 

240 doc="set if source was reserved from PSF determination") 

241 

242 @timeMethod 

243 def run(self, exposure, sources, expId=0, matches=None): 

244 """!Measure the PSF 

245 

246 @param[in,out] exposure Exposure to process; measured PSF will be added. 

247 @param[in,out] sources Measured sources on exposure; flag fields will be set marking 

248 stars chosen by the star selector and the PSF determiner if a schema 

249 was passed to the task constructor. 

250 @param[in] expId Exposure id used for generating random seed. 

251 @param[in] matches A list of lsst.afw.table.ReferenceMatch objects 

252 (@em i.e. of lsst.afw.table.Match 

253 with @c first being of type lsst.afw.table.SimpleRecord and @c second 

254 type lsst.afw.table.SourceRecord --- the reference object and detected 

255 object respectively) as returned by @em e.g. the AstrometryTask. 

256 Used by star selectors that choose to refer to an external catalog. 

257 

258 @return a pipe.base.Struct with fields: 

259 - psf: The measured PSF (also set in the input exposure) 

260 - cellSet: an lsst.afw.math.SpatialCellSet containing the PSF candidates 

261 as returned by the psf determiner. 

262 """ 

263 self.log.info("Measuring PSF") 

264 

265 import lsstDebug 

266 display = lsstDebug.Info(__name__).display 

267 displayExposure = lsstDebug.Info(__name__).displayExposure # display the Exposure + spatialCells 

268 displayPsfMosaic = lsstDebug.Info(__name__).displayPsfMosaic # show mosaic of reconstructed PSF(x,y) 

269 displayPsfCandidates = lsstDebug.Info(__name__).displayPsfCandidates # show mosaic of candidates 

270 displayResiduals = lsstDebug.Info(__name__).displayResiduals # show residuals 

271 showBadCandidates = lsstDebug.Info(__name__).showBadCandidates # include bad candidates 

272 normalizeResiduals = lsstDebug.Info(__name__).normalizeResiduals # normalise residuals by object peak 

273 

274 # 

275 # Run star selector 

276 # 

277 stars = self.starSelector.run(sourceCat=sources, matches=matches, exposure=exposure) 

278 selectionResult = self.makePsfCandidates.run(stars.sourceCat, exposure=exposure) 

279 self.log.info("PSF star selector found %d candidates", len(selectionResult.psfCandidates)) 

280 reserveResult = self.reserve.run(selectionResult.goodStarCat, expId=expId) 

281 # Make list of psf candidates to send to the determiner (omitting those marked as reserved) 

282 psfDeterminerList = [cand for cand, use 

283 in zip(selectionResult.psfCandidates, reserveResult.use) if use] 

284 

285 if selectionResult.psfCandidates and self.candidateKey is not None: 

286 for cand in selectionResult.psfCandidates: 

287 source = cand.getSource() 

288 source.set(self.candidateKey, True) 

289 

290 self.log.info("Sending %d candidates to PSF determiner", len(psfDeterminerList)) 

291 

292 if display: 

293 frame = 1 

294 if displayExposure: 

295 disp = afwDisplay.Display(frame=frame) 

296 disp.mtv(exposure, title="psf determination") 

297 frame += 1 

298 # 

299 # Determine PSF 

300 # 

301 psf, cellSet = self.psfDeterminer.determinePsf(exposure, psfDeterminerList, self.metadata, 

302 flagKey=self.usedKey) 

303 self.log.info("PSF determination using %d/%d stars.", 

304 self.metadata.getScalar("numGoodStars"), self.metadata.getScalar("numAvailStars")) 

305 

306 exposure.setPsf(psf) 

307 

308 if display: 

309 frame = display 

310 if displayExposure: 

311 disp = afwDisplay.Display(frame=frame) 

312 showPsfSpatialCells(exposure, cellSet, showBadCandidates, frame=frame) 

313 frame += 1 

314 

315 if displayPsfCandidates: # Show a mosaic of PSF candidates 

316 plotPsfCandidates(cellSet, showBadCandidates=showBadCandidates, frame=frame) 

317 frame += 1 

318 

319 if displayResiduals: 

320 frame = plotResiduals(exposure, cellSet, 

321 showBadCandidates=showBadCandidates, 

322 normalizeResiduals=normalizeResiduals, 

323 frame=frame) 

324 if displayPsfMosaic: 

325 disp = afwDisplay.Display(frame=frame) 

326 maUtils.showPsfMosaic(exposure, psf, display=disp, showFwhm=True) 

327 disp.scale("linear", 0, 1) 

328 frame += 1 

329 

330 return pipeBase.Struct( 

331 psf=psf, 

332 cellSet=cellSet, 

333 ) 

334 

335 @property 

336 def usesMatches(self): 

337 """Return True if this task makes use of the "matches" argument to the run method""" 

338 return self.starSelector.usesMatches 

339 

340# 

341# Debug code 

342# 

343 

344 

345def showPsfSpatialCells(exposure, cellSet, showBadCandidates, frame=1): 

346 disp = afwDisplay.Display(frame=frame) 

347 maUtils.showPsfSpatialCells(exposure, cellSet, 

348 symb="o", ctype=afwDisplay.CYAN, ctypeUnused=afwDisplay.YELLOW, 

349 size=4, display=disp) 

350 for cell in cellSet.getCellList(): 

351 for cand in cell.begin(not showBadCandidates): # maybe include bad candidates 

352 status = cand.getStatus() 

353 disp.dot('+', *cand.getSource().getCentroid(), 

354 ctype=afwDisplay.GREEN if status == afwMath.SpatialCellCandidate.GOOD else 

355 afwDisplay.YELLOW if status == afwMath.SpatialCellCandidate.UNKNOWN else afwDisplay.RED) 

356 

357 

358def plotPsfCandidates(cellSet, showBadCandidates=False, frame=1): 

359 stamps = [] 

360 for cell in cellSet.getCellList(): 

361 for cand in cell.begin(not showBadCandidates): # maybe include bad candidates 

362 try: 

363 im = cand.getMaskedImage() 

364 

365 chi2 = cand.getChi2() 

366 if chi2 < 1e100: 

367 chi2 = "%.1f" % chi2 

368 else: 

369 chi2 = float("nan") 

370 

371 stamps.append((im, "%d%s" % 

372 (maUtils.splitId(cand.getSource().getId(), True)["objId"], chi2), 

373 cand.getStatus())) 

374 except Exception: 

375 continue 

376 

377 mos = afwDisplay.utils.Mosaic() 

378 disp = afwDisplay.Display(frame=frame) 

379 for im, label, status in stamps: 

380 im = type(im)(im, True) 

381 try: 

382 im /= afwMath.makeStatistics(im, afwMath.MAX).getValue() 

383 except NotImplementedError: 

384 pass 

385 

386 mos.append(im, label, 

387 afwDisplay.GREEN if status == afwMath.SpatialCellCandidate.GOOD else 

388 afwDisplay.YELLOW if status == afwMath.SpatialCellCandidate.UNKNOWN else afwDisplay.RED) 

389 

390 if mos.images: 

391 disp.mtv(mos.makeMosaic(), title="Psf Candidates") 

392 

393 

394def plotResiduals(exposure, cellSet, showBadCandidates=False, normalizeResiduals=True, frame=2): 

395 psf = exposure.getPsf() 

396 disp = afwDisplay.Display(frame=frame) 

397 while True: 

398 try: 

399 maUtils.showPsfCandidates(exposure, cellSet, psf=psf, display=disp, 

400 normalize=normalizeResiduals, 

401 showBadCandidates=showBadCandidates) 

402 frame += 1 

403 maUtils.showPsfCandidates(exposure, cellSet, psf=psf, display=disp, 

404 normalize=normalizeResiduals, 

405 showBadCandidates=showBadCandidates, 

406 variance=True) 

407 frame += 1 

408 except Exception: 

409 if not showBadCandidates: 

410 showBadCandidates = True 

411 continue 

412 break 

413 

414 return frame