Hide keyboard shortcuts

Hot-keys on this page

r m x p   toggle line displays

j k   next/prev highlighted chunk

0   (zero) top of page

1   (one) first highlighted chunk

1# This file is part of pipe_tasks. 

2# 

3# Developed for the LSST Data Management System. 

4# This product includes software developed by the LSST Project 

5# (https://www.lsst.org). 

6# See the COPYRIGHT file at the top-level directory of this distribution 

7# for details of code ownership. 

8# 

9# This program is free software: you can redistribute it and/or modify 

10# it under the terms of the GNU General Public License as published by 

11# the Free Software Foundation, either version 3 of the License, or 

12# (at your option) any later version. 

13# 

14# This program is distributed in the hope that it will be useful, 

15# but WITHOUT ANY WARRANTY; without even the implied warranty of 

16# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 

17# GNU General Public License for more details. 

18# 

19# You should have received a copy of the GNU General Public License 

20# along with this program. If not, see <https://www.gnu.org/licenses/>. 

21 

22import lsst.afw.display as afwDisplay 

23import lsst.afw.math as afwMath 

24import lsst.meas.algorithms as measAlg 

25import lsst.meas.algorithms.utils as maUtils 

26import lsst.pex.config as pexConfig 

27import lsst.pipe.base as pipeBase 

28import lsst.meas.extensions.psfex.psfexPsfDeterminer # noqa: F401 

29 

30 

31class MeasurePsfConfig(pexConfig.Config): 

32 starSelector = measAlg.sourceSelectorRegistry.makeField( 

33 "Star selection algorithm", 

34 default="objectSize" 

35 ) 

36 makePsfCandidates = pexConfig.ConfigurableField( 

37 target=measAlg.MakePsfCandidatesTask, 

38 doc="Task to make psf candidates from selected stars.", 

39 ) 

40 psfDeterminer = measAlg.psfDeterminerRegistry.makeField( 

41 "PSF Determination algorithm", 

42 default="psfex" 

43 ) 

44 reserve = pexConfig.ConfigurableField( 

45 target=measAlg.ReserveSourcesTask, 

46 doc="Reserve sources from fitting" 

47 ) 

48 

49## @addtogroup LSST_task_documentation 

50## @{ 

51## @page MeasurePsfTask 

52## @ref MeasurePsfTask_ "MeasurePsfTask" 

53## @copybrief MeasurePsfTask 

54## @} 

55 

56 

57class MeasurePsfTask(pipeBase.Task): 

58 r"""! 

59@anchor MeasurePsfTask_ 

60 

61@brief Measure the PSF 

62 

63@section pipe_tasks_measurePsf_Contents Contents 

64 

65 - @ref pipe_tasks_measurePsf_Purpose 

66 - @ref pipe_tasks_measurePsf_Initialize 

67 - @ref pipe_tasks_measurePsf_IO 

68 - @ref pipe_tasks_measurePsf_Config 

69 - @ref pipe_tasks_measurePsf_Debug 

70 - @ref pipe_tasks_measurePsf_Example 

71 

72@section pipe_tasks_measurePsf_Purpose Description 

73 

74A task that selects stars from a catalog of sources and uses those to measure the PSF. 

75 

76The star selector is a subclass of 

77@ref lsst.meas.algorithms.starSelector.BaseStarSelectorTask "lsst.meas.algorithms.BaseStarSelectorTask" 

78and the PSF determiner is a sublcass of 

79@ref lsst.meas.algorithms.psfDeterminer.BasePsfDeterminerTask "lsst.meas.algorithms.BasePsfDeterminerTask" 

80 

81@warning 

82There is no establised set of configuration parameters for these algorithms, so once you start modifying 

83parameters (as we do in @ref pipe_tasks_measurePsf_Example) your code is no longer portable. 

84 

85@section pipe_tasks_measurePsf_Initialize Task initialisation 

86 

87@copydoc \_\_init\_\_ 

88 

89@section pipe_tasks_measurePsf_IO Invoking the Task 

90 

91@copydoc run 

92 

93@section pipe_tasks_measurePsf_Config Configuration parameters 

94 

95See @ref MeasurePsfConfig. 

96 

97@section pipe_tasks_measurePsf_Debug Debug variables 

98 

99The @link lsst.pipe.base.cmdLineTask.CmdLineTask command line task@endlink interface supports a 

100flag @c -d to import @b debug.py from your @c PYTHONPATH; see @ref baseDebug for more about @b debug.py files. 

101 

102<DL> 

103 <DT> @c display 

104 <DD> If True, display debugging plots 

105 <DT> displayExposure 

106 <DD> display the Exposure + spatialCells 

107 <DT> displayPsfCandidates 

108 <DD> show mosaic of candidates 

109 <DT> showBadCandidates 

110 <DD> Include bad candidates 

111 <DT> displayPsfMosaic 

112 <DD> show mosaic of reconstructed PSF(xy) 

113 <DT> displayResiduals 

114 <DD> show residuals 

115 <DT> normalizeResiduals 

116 <DD> Normalise residuals by object amplitude 

117</DL> 

118 

119Additionally you can enable any debug outputs that your chosen star selector and psf determiner support. 

120 

121@section pipe_tasks_measurePsf_Example A complete example of using MeasurePsfTask 

122 

123This code is in @link measurePsfTask.py@endlink in the examples directory, and can be run as @em e.g. 

124@code 

125examples/measurePsfTask.py --doDisplay 

126@endcode 

127@dontinclude measurePsfTask.py 

128 

129The example also runs SourceDetectionTask and SingleFrameMeasurementTask; 

130see @ref meas_algorithms_measurement_Example for more explanation. 

131 

132Import the tasks (there are some other standard imports; read the file to see them all): 

133 

134@skip SourceDetectionTask 

135@until MeasurePsfTask 

136 

137We need to create the tasks before processing any data as the task constructor 

138can add an extra column to the schema, but first we need an almost-empty 

139Schema: 

140 

141@skipline makeMinimalSchema 

142 

143We can now call the constructors for the tasks we need to find and characterize candidate 

144PSF stars: 

145 

146@skip SourceDetectionTask.ConfigClass 

147@until measureTask 

148 

149Note that we've chosen a minimal set of measurement plugins: we need the 

150outputs of @c base_SdssCentroid, @c base_SdssShape and @c base_CircularApertureFlux as 

151inputs to the PSF measurement algorithm, while @c base_PixelFlags identifies 

152and flags bad sources (e.g. with pixels too close to the edge) so they can be 

153removed later. 

154 

155Now we can create and configure the task that we're interested in: 

156 

157@skip MeasurePsfTask 

158@until measurePsfTask 

159 

160We're now ready to process the data (we could loop over multiple exposures/catalogues using the same 

161task objects). First create the output table: 

162 

163@skipline afwTable 

164 

165And process the image: 

166 

167@skip sources = 

168@until result 

169 

170We can then unpack and use the results: 

171 

172@skip psf 

173@until cellSet 

174 

175If you specified @c --doDisplay you can see the PSF candidates: 

176 

177@skip display 

178@until RED 

179 

180<HR> 

181 

182To investigate the @ref pipe_tasks_measurePsf_Debug, put something like 

183@code{.py} 

184 import lsstDebug 

185 def DebugInfo(name): 

186 di = lsstDebug.getInfo(name) # N.b. lsstDebug.Info(name) would call us recursively 

187 

188 if name == "lsst.pipe.tasks.measurePsf" : 

189 di.display = True 

190 di.displayExposure = False # display the Exposure + spatialCells 

191 di.displayPsfCandidates = True # show mosaic of candidates 

192 di.displayPsfMosaic = True # show mosaic of reconstructed PSF(xy) 

193 di.displayResiduals = True # show residuals 

194 di.showBadCandidates = True # Include bad candidates 

195 di.normalizeResiduals = False # Normalise residuals by object amplitude 

196 

197 return di 

198 

199 lsstDebug.Info = DebugInfo 

200@endcode 

201into your debug.py file and run measurePsfTask.py with the @c --debug flag. 

202 """ 

203 ConfigClass = MeasurePsfConfig 

204 _DefaultName = "measurePsf" 

205 

206 def __init__(self, schema=None, **kwargs): 

207 """!Create the detection task. Most arguments are simply passed onto pipe.base.Task. 

208 

209 @param schema An lsst::afw::table::Schema used to create the output lsst.afw.table.SourceCatalog 

210 @param **kwargs Keyword arguments passed to lsst.pipe.base.task.Task.__init__. 

211 

212 If schema is not None, 'calib_psf_candidate' and 'calib_psf_used' fields will be added to 

213 identify which stars were employed in the PSF estimation. 

214 

215 @note This task can add fields to the schema, so any code calling this task must ensure that 

216 these fields are indeed present in the input table. 

217 """ 

218 

219 pipeBase.Task.__init__(self, **kwargs) 

220 if schema is not None: 

221 self.candidateKey = schema.addField( 

222 "calib_psf_candidate", type="Flag", 

223 doc=("Flag set if the source was a candidate for PSF determination, " 

224 "as determined by the star selector.") 

225 ) 

226 self.usedKey = schema.addField( 

227 "calib_psf_used", type="Flag", 

228 doc=("Flag set if the source was actually used for PSF determination, " 

229 "as determined by the '%s' PSF determiner.") % self.config.psfDeterminer.name 

230 ) 

231 else: 

232 self.candidateKey = None 

233 self.usedKey = None 

234 self.makeSubtask("starSelector") 

235 self.makeSubtask("makePsfCandidates") 

236 self.makeSubtask("psfDeterminer", schema=schema) 

237 self.makeSubtask("reserve", columnName="calib_psf", schema=schema, 

238 doc="set if source was reserved from PSF determination") 

239 

240 @pipeBase.timeMethod 

241 def run(self, exposure, sources, expId=0, matches=None): 

242 """!Measure the PSF 

243 

244 @param[in,out] exposure Exposure to process; measured PSF will be added. 

245 @param[in,out] sources Measured sources on exposure; flag fields will be set marking 

246 stars chosen by the star selector and the PSF determiner if a schema 

247 was passed to the task constructor. 

248 @param[in] expId Exposure id used for generating random seed. 

249 @param[in] matches A list of lsst.afw.table.ReferenceMatch objects 

250 (@em i.e. of lsst.afw.table.Match 

251 with @c first being of type lsst.afw.table.SimpleRecord and @c second 

252 type lsst.afw.table.SourceRecord --- the reference object and detected 

253 object respectively) as returned by @em e.g. the AstrometryTask. 

254 Used by star selectors that choose to refer to an external catalog. 

255 

256 @return a pipe.base.Struct with fields: 

257 - psf: The measured PSF (also set in the input exposure) 

258 - cellSet: an lsst.afw.math.SpatialCellSet containing the PSF candidates 

259 as returned by the psf determiner. 

260 """ 

261 self.log.info("Measuring PSF") 

262 

263 import lsstDebug 

264 display = lsstDebug.Info(__name__).display 

265 displayExposure = lsstDebug.Info(__name__).displayExposure # display the Exposure + spatialCells 

266 displayPsfMosaic = lsstDebug.Info(__name__).displayPsfMosaic # show mosaic of reconstructed PSF(x,y) 

267 displayPsfCandidates = lsstDebug.Info(__name__).displayPsfCandidates # show mosaic of candidates 

268 displayResiduals = lsstDebug.Info(__name__).displayResiduals # show residuals 

269 showBadCandidates = lsstDebug.Info(__name__).showBadCandidates # include bad candidates 

270 normalizeResiduals = lsstDebug.Info(__name__).normalizeResiduals # normalise residuals by object peak 

271 

272 # 

273 # Run star selector 

274 # 

275 stars = self.starSelector.run(sourceCat=sources, matches=matches, exposure=exposure) 

276 selectionResult = self.makePsfCandidates.run(stars.sourceCat, exposure=exposure) 

277 self.log.info("PSF star selector found %d candidates" % len(selectionResult.psfCandidates)) 

278 reserveResult = self.reserve.run(selectionResult.goodStarCat, expId=expId) 

279 # Make list of psf candidates to send to the determiner (omitting those marked as reserved) 

280 psfDeterminerList = [cand for cand, use 

281 in zip(selectionResult.psfCandidates, reserveResult.use) if use] 

282 

283 if selectionResult.psfCandidates and self.candidateKey is not None: 

284 for cand in selectionResult.psfCandidates: 

285 source = cand.getSource() 

286 source.set(self.candidateKey, True) 

287 

288 self.log.info("Sending %d candidates to PSF determiner" % len(psfDeterminerList)) 

289 

290 if display: 

291 frame = 1 

292 if displayExposure: 

293 disp = afwDisplay.Display(frame=frame) 

294 disp.mtv(exposure, title="psf determination") 

295 frame += 1 

296 # 

297 # Determine PSF 

298 # 

299 psf, cellSet = self.psfDeterminer.determinePsf(exposure, psfDeterminerList, self.metadata, 

300 flagKey=self.usedKey) 

301 self.log.info("PSF determination using %d/%d stars." % 

302 (self.metadata.getScalar("numGoodStars"), self.metadata.getScalar("numAvailStars"))) 

303 

304 exposure.setPsf(psf) 

305 

306 if display: 

307 frame = display 

308 if displayExposure: 

309 disp = afwDisplay.Display(frame=frame) 

310 showPsfSpatialCells(exposure, cellSet, showBadCandidates, frame=frame) 

311 frame += 1 

312 

313 if displayPsfCandidates: # Show a mosaic of PSF candidates 

314 plotPsfCandidates(cellSet, showBadCandidates=showBadCandidates, frame=frame) 

315 frame += 1 

316 

317 if displayResiduals: 

318 frame = plotResiduals(exposure, cellSet, 

319 showBadCandidates=showBadCandidates, 

320 normalizeResiduals=normalizeResiduals, 

321 frame=frame) 

322 if displayPsfMosaic: 

323 disp = afwDisplay.Display(frame=frame) 

324 maUtils.showPsfMosaic(exposure, psf, display=disp, showFwhm=True) 

325 disp.scale("linear", 0, 1) 

326 frame += 1 

327 

328 return pipeBase.Struct( 

329 psf=psf, 

330 cellSet=cellSet, 

331 ) 

332 

333 @property 

334 def usesMatches(self): 

335 """Return True if this task makes use of the "matches" argument to the run method""" 

336 return self.starSelector.usesMatches 

337 

338# 

339# Debug code 

340# 

341 

342 

343def showPsfSpatialCells(exposure, cellSet, showBadCandidates, frame=1): 

344 disp = afwDisplay.Display(frame=frame) 

345 maUtils.showPsfSpatialCells(exposure, cellSet, 

346 symb="o", ctype=afwDisplay.CYAN, ctypeUnused=afwDisplay.YELLOW, 

347 size=4, display=disp) 

348 for cell in cellSet.getCellList(): 

349 for cand in cell.begin(not showBadCandidates): # maybe include bad candidates 

350 status = cand.getStatus() 

351 disp.dot('+', *cand.getSource().getCentroid(), 

352 ctype=afwDisplay.GREEN if status == afwMath.SpatialCellCandidate.GOOD else 

353 afwDisplay.YELLOW if status == afwMath.SpatialCellCandidate.UNKNOWN else afwDisplay.RED) 

354 

355 

356def plotPsfCandidates(cellSet, showBadCandidates=False, frame=1): 

357 stamps = [] 

358 for cell in cellSet.getCellList(): 

359 for cand in cell.begin(not showBadCandidates): # maybe include bad candidates 

360 try: 

361 im = cand.getMaskedImage() 

362 

363 chi2 = cand.getChi2() 

364 if chi2 < 1e100: 

365 chi2 = "%.1f" % chi2 

366 else: 

367 chi2 = float("nan") 

368 

369 stamps.append((im, "%d%s" % 

370 (maUtils.splitId(cand.getSource().getId(), True)["objId"], chi2), 

371 cand.getStatus())) 

372 except Exception: 

373 continue 

374 

375 mos = afwDisplay.utils.Mosaic() 

376 disp = afwDisplay.Display(frame=frame) 

377 for im, label, status in stamps: 

378 im = type(im)(im, True) 

379 try: 

380 im /= afwMath.makeStatistics(im, afwMath.MAX).getValue() 

381 except NotImplementedError: 

382 pass 

383 

384 mos.append(im, label, 

385 afwDisplay.GREEN if status == afwMath.SpatialCellCandidate.GOOD else 

386 afwDisplay.YELLOW if status == afwMath.SpatialCellCandidate.UNKNOWN else afwDisplay.RED) 

387 

388 if mos.images: 

389 disp.mtv(mos.makeMosaic(), title="Psf Candidates") 

390 

391 

392def plotResiduals(exposure, cellSet, showBadCandidates=False, normalizeResiduals=True, frame=2): 

393 psf = exposure.getPsf() 

394 disp = afwDisplay.Display(frame=frame) 

395 while True: 

396 try: 

397 maUtils.showPsfCandidates(exposure, cellSet, psf=psf, display=disp, 

398 normalize=normalizeResiduals, 

399 showBadCandidates=showBadCandidates) 

400 frame += 1 

401 maUtils.showPsfCandidates(exposure, cellSet, psf=psf, display=disp, 

402 normalize=normalizeResiduals, 

403 showBadCandidates=showBadCandidates, 

404 variance=True) 

405 frame += 1 

406 except Exception: 

407 if not showBadCandidates: 

408 showBadCandidates = True 

409 continue 

410 break 

411 

412 return frame