Hide keyboard shortcuts

Hot-keys on this page

r m x p   toggle line displays

j k   next/prev highlighted chunk

0   (zero) top of page

1   (one) first highlighted chunk

1# This file is part of pipe_tasks. 

2# 

3# Developed for the LSST Data Management System. 

4# This product includes software developed by the LSST Project 

5# (https://www.lsst.org). 

6# See the COPYRIGHT file at the top-level directory of this distribution 

7# for details of code ownership. 

8# 

9# This program is free software: you can redistribute it and/or modify 

10# it under the terms of the GNU General Public License as published by 

11# the Free Software Foundation, either version 3 of the License, or 

12# (at your option) any later version. 

13# 

14# This program is distributed in the hope that it will be useful, 

15# but WITHOUT ANY WARRANTY; without even the implied warranty of 

16# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 

17# GNU General Public License for more details. 

18# 

19# You should have received a copy of the GNU General Public License 

20# along with this program. If not, see <https://www.gnu.org/licenses/>. 

21 

22import lsst.afw.display as afwDisplay 

23import lsst.afw.math as afwMath 

24import lsst.meas.algorithms as measAlg 

25import lsst.meas.algorithms.utils as maUtils 

26import lsst.pex.config as pexConfig 

27import lsst.pipe.base as pipeBase 

28 

29 

30class MeasurePsfConfig(pexConfig.Config): 

31 starSelector = measAlg.sourceSelectorRegistry.makeField( 

32 "Star selection algorithm", 

33 default="objectSize" 

34 ) 

35 makePsfCandidates = pexConfig.ConfigurableField( 

36 target=measAlg.MakePsfCandidatesTask, 

37 doc="Task to make psf candidates from selected stars.", 

38 ) 

39 psfDeterminer = measAlg.psfDeterminerRegistry.makeField( 

40 "PSF Determination algorithm", 

41 default="pca" 

42 ) 

43 reserve = pexConfig.ConfigurableField( 

44 target=measAlg.ReserveSourcesTask, 

45 doc="Reserve sources from fitting" 

46 ) 

47 

48## @addtogroup LSST_task_documentation 

49## @{ 

50## @page MeasurePsfTask 

51## @ref MeasurePsfTask_ "MeasurePsfTask" 

52## @copybrief MeasurePsfTask 

53## @} 

54 

55 

56class MeasurePsfTask(pipeBase.Task): 

57 r"""! 

58@anchor MeasurePsfTask_ 

59 

60@brief Measure the PSF 

61 

62@section pipe_tasks_measurePsf_Contents Contents 

63 

64 - @ref pipe_tasks_measurePsf_Purpose 

65 - @ref pipe_tasks_measurePsf_Initialize 

66 - @ref pipe_tasks_measurePsf_IO 

67 - @ref pipe_tasks_measurePsf_Config 

68 - @ref pipe_tasks_measurePsf_Debug 

69 - @ref pipe_tasks_measurePsf_Example 

70 

71@section pipe_tasks_measurePsf_Purpose Description 

72 

73A task that selects stars from a catalog of sources and uses those to measure the PSF. 

74 

75The star selector is a subclass of 

76@ref lsst.meas.algorithms.starSelector.BaseStarSelectorTask "lsst.meas.algorithms.BaseStarSelectorTask" 

77and the PSF determiner is a sublcass of 

78@ref lsst.meas.algorithms.psfDeterminer.BasePsfDeterminerTask "lsst.meas.algorithms.BasePsfDeterminerTask" 

79 

80@warning 

81There is no establised set of configuration parameters for these algorithms, so once you start modifying 

82parameters (as we do in @ref pipe_tasks_measurePsf_Example) your code is no longer portable. 

83 

84@section pipe_tasks_measurePsf_Initialize Task initialisation 

85 

86@copydoc \_\_init\_\_ 

87 

88@section pipe_tasks_measurePsf_IO Invoking the Task 

89 

90@copydoc run 

91 

92@section pipe_tasks_measurePsf_Config Configuration parameters 

93 

94See @ref MeasurePsfConfig. 

95 

96@section pipe_tasks_measurePsf_Debug Debug variables 

97 

98The @link lsst.pipe.base.cmdLineTask.CmdLineTask command line task@endlink interface supports a 

99flag @c -d to import @b debug.py from your @c PYTHONPATH; see @ref baseDebug for more about @b debug.py files. 

100 

101<DL> 

102 <DT> @c display 

103 <DD> If True, display debugging plots 

104 <DT> displayExposure 

105 <DD> display the Exposure + spatialCells 

106 <DT> displayPsfCandidates 

107 <DD> show mosaic of candidates 

108 <DT> showBadCandidates 

109 <DD> Include bad candidates 

110 <DT> displayPsfMosaic 

111 <DD> show mosaic of reconstructed PSF(xy) 

112 <DT> displayResiduals 

113 <DD> show residuals 

114 <DT> normalizeResiduals 

115 <DD> Normalise residuals by object amplitude 

116</DL> 

117 

118Additionally you can enable any debug outputs that your chosen star selector and psf determiner support. 

119 

120@section pipe_tasks_measurePsf_Example A complete example of using MeasurePsfTask 

121 

122This code is in @link measurePsfTask.py@endlink in the examples directory, and can be run as @em e.g. 

123@code 

124examples/measurePsfTask.py --doDisplay 

125@endcode 

126@dontinclude measurePsfTask.py 

127 

128The example also runs SourceDetectionTask and SingleFrameMeasurementTask; 

129see @ref meas_algorithms_measurement_Example for more explanation. 

130 

131Import the tasks (there are some other standard imports; read the file to see them all): 

132 

133@skip SourceDetectionTask 

134@until MeasurePsfTask 

135 

136We need to create the tasks before processing any data as the task constructor 

137can add an extra column to the schema, but first we need an almost-empty 

138Schema: 

139 

140@skipline makeMinimalSchema 

141 

142We can now call the constructors for the tasks we need to find and characterize candidate 

143PSF stars: 

144 

145@skip SourceDetectionTask.ConfigClass 

146@until measureTask 

147 

148Note that we've chosen a minimal set of measurement plugins: we need the 

149outputs of @c base_SdssCentroid, @c base_SdssShape and @c base_CircularApertureFlux as 

150inputs to the PSF measurement algorithm, while @c base_PixelFlags identifies 

151and flags bad sources (e.g. with pixels too close to the edge) so they can be 

152removed later. 

153 

154Now we can create and configure the task that we're interested in: 

155 

156@skip MeasurePsfTask 

157@until measurePsfTask 

158 

159We're now ready to process the data (we could loop over multiple exposures/catalogues using the same 

160task objects). First create the output table: 

161 

162@skipline afwTable 

163 

164And process the image: 

165 

166@skip sources = 

167@until result 

168 

169We can then unpack and use the results: 

170 

171@skip psf 

172@until cellSet 

173 

174If you specified @c --doDisplay you can see the PSF candidates: 

175 

176@skip display 

177@until RED 

178 

179<HR> 

180 

181To investigate the @ref pipe_tasks_measurePsf_Debug, put something like 

182@code{.py} 

183 import lsstDebug 

184 def DebugInfo(name): 

185 di = lsstDebug.getInfo(name) # N.b. lsstDebug.Info(name) would call us recursively 

186 

187 if name == "lsst.pipe.tasks.measurePsf" : 

188 di.display = True 

189 di.displayExposure = False # display the Exposure + spatialCells 

190 di.displayPsfCandidates = True # show mosaic of candidates 

191 di.displayPsfMosaic = True # show mosaic of reconstructed PSF(xy) 

192 di.displayResiduals = True # show residuals 

193 di.showBadCandidates = True # Include bad candidates 

194 di.normalizeResiduals = False # Normalise residuals by object amplitude 

195 

196 return di 

197 

198 lsstDebug.Info = DebugInfo 

199@endcode 

200into your debug.py file and run measurePsfTask.py with the @c --debug flag. 

201 """ 

202 ConfigClass = MeasurePsfConfig 

203 _DefaultName = "measurePsf" 

204 

205 def __init__(self, schema=None, **kwargs): 

206 """!Create the detection task. Most arguments are simply passed onto pipe.base.Task. 

207 

208 @param schema An lsst::afw::table::Schema used to create the output lsst.afw.table.SourceCatalog 

209 @param **kwargs Keyword arguments passed to lsst.pipe.base.task.Task.__init__. 

210 

211 If schema is not None, 'calib_psf_candidate' and 'calib_psf_used' fields will be added to 

212 identify which stars were employed in the PSF estimation. 

213 

214 @note This task can add fields to the schema, so any code calling this task must ensure that 

215 these fields are indeed present in the input table. 

216 """ 

217 

218 pipeBase.Task.__init__(self, **kwargs) 

219 if schema is not None: 219 ↛ 231line 219 didn't jump to line 231, because the condition on line 219 was never false

220 self.candidateKey = schema.addField( 

221 "calib_psf_candidate", type="Flag", 

222 doc=("Flag set if the source was a candidate for PSF determination, " 

223 "as determined by the star selector.") 

224 ) 

225 self.usedKey = schema.addField( 

226 "calib_psf_used", type="Flag", 

227 doc=("Flag set if the source was actually used for PSF determination, " 

228 "as determined by the '%s' PSF determiner.") % self.config.psfDeterminer.name 

229 ) 

230 else: 

231 self.candidateKey = None 

232 self.usedKey = None 

233 self.makeSubtask("starSelector") 

234 self.makeSubtask("makePsfCandidates") 

235 self.makeSubtask("psfDeterminer", schema=schema) 

236 self.makeSubtask("reserve", columnName="calib_psf", schema=schema, 

237 doc="set if source was reserved from PSF determination") 

238 

239 @pipeBase.timeMethod 

240 def run(self, exposure, sources, expId=0, matches=None): 

241 """!Measure the PSF 

242 

243 @param[in,out] exposure Exposure to process; measured PSF will be added. 

244 @param[in,out] sources Measured sources on exposure; flag fields will be set marking 

245 stars chosen by the star selector and the PSF determiner if a schema 

246 was passed to the task constructor. 

247 @param[in] expId Exposure id used for generating random seed. 

248 @param[in] matches A list of lsst.afw.table.ReferenceMatch objects 

249 (@em i.e. of lsst.afw.table.Match 

250 with @c first being of type lsst.afw.table.SimpleRecord and @c second 

251 type lsst.afw.table.SourceRecord --- the reference object and detected 

252 object respectively) as returned by @em e.g. the AstrometryTask. 

253 Used by star selectors that choose to refer to an external catalog. 

254 

255 @return a pipe.base.Struct with fields: 

256 - psf: The measured PSF (also set in the input exposure) 

257 - cellSet: an lsst.afw.math.SpatialCellSet containing the PSF candidates 

258 as returned by the psf determiner. 

259 """ 

260 self.log.info("Measuring PSF") 

261 

262 import lsstDebug 

263 display = lsstDebug.Info(__name__).display 

264 displayExposure = lsstDebug.Info(__name__).displayExposure # display the Exposure + spatialCells 

265 displayPsfMosaic = lsstDebug.Info(__name__).displayPsfMosaic # show mosaic of reconstructed PSF(x,y) 

266 displayPsfCandidates = lsstDebug.Info(__name__).displayPsfCandidates # show mosaic of candidates 

267 displayResiduals = lsstDebug.Info(__name__).displayResiduals # show residuals 

268 showBadCandidates = lsstDebug.Info(__name__).showBadCandidates # include bad candidates 

269 normalizeResiduals = lsstDebug.Info(__name__).normalizeResiduals # normalise residuals by object peak 

270 

271 # 

272 # Run star selector 

273 # 

274 stars = self.starSelector.run(sourceCat=sources, matches=matches, exposure=exposure) 

275 selectionResult = self.makePsfCandidates.run(stars.sourceCat, exposure=exposure) 

276 self.log.info("PSF star selector found %d candidates" % len(selectionResult.psfCandidates)) 

277 reserveResult = self.reserve.run(selectionResult.goodStarCat, expId=expId) 

278 # Make list of psf candidates to send to the determiner (omitting those marked as reserved) 

279 psfDeterminerList = [cand for cand, use 

280 in zip(selectionResult.psfCandidates, reserveResult.use) if use] 

281 

282 if selectionResult.psfCandidates and self.candidateKey is not None: 282 ↛ 287line 282 didn't jump to line 287, because the condition on line 282 was never false

283 for cand in selectionResult.psfCandidates: 

284 source = cand.getSource() 

285 source.set(self.candidateKey, True) 

286 

287 self.log.info("Sending %d candidates to PSF determiner" % len(psfDeterminerList)) 

288 

289 if display: 289 ↛ 290line 289 didn't jump to line 290, because the condition on line 289 was never true

290 frame = 1 

291 if displayExposure: 

292 disp = afwDisplay.Display(frame=frame) 

293 disp.mtv(exposure, title="psf determination") 

294 frame += 1 

295 # 

296 # Determine PSF 

297 # 

298 psf, cellSet = self.psfDeterminer.determinePsf(exposure, psfDeterminerList, self.metadata, 

299 flagKey=self.usedKey) 

300 self.log.info("PSF determination using %d/%d stars." % 

301 (self.metadata.getScalar("numGoodStars"), self.metadata.getScalar("numAvailStars"))) 

302 

303 exposure.setPsf(psf) 

304 

305 if display: 305 ↛ 306line 305 didn't jump to line 306, because the condition on line 305 was never true

306 frame = display 

307 if displayExposure: 

308 disp = afwDisplay.Display(frame=frame) 

309 showPsfSpatialCells(exposure, cellSet, showBadCandidates, frame=frame) 

310 frame += 1 

311 

312 if displayPsfCandidates: # Show a mosaic of PSF candidates 

313 plotPsfCandidates(cellSet, showBadCandidates=showBadCandidates, frame=frame) 

314 frame += 1 

315 

316 if displayResiduals: 

317 frame = plotResiduals(exposure, cellSet, 

318 showBadCandidates=showBadCandidates, 

319 normalizeResiduals=normalizeResiduals, 

320 frame=frame) 

321 if displayPsfMosaic: 

322 disp = afwDisplay.Display(frame=frame) 

323 maUtils.showPsfMosaic(exposure, psf, display=disp, showFwhm=True) 

324 disp.scale("linear", 0, 1) 

325 frame += 1 

326 

327 return pipeBase.Struct( 

328 psf=psf, 

329 cellSet=cellSet, 

330 ) 

331 

332 @property 

333 def usesMatches(self): 

334 """Return True if this task makes use of the "matches" argument to the run method""" 

335 return self.starSelector.usesMatches 

336 

337# 

338# Debug code 

339# 

340 

341 

342def showPsfSpatialCells(exposure, cellSet, showBadCandidates, frame=1): 

343 disp = afwDisplay.Display(frame=frame) 

344 maUtils.showPsfSpatialCells(exposure, cellSet, 

345 symb="o", ctype=afwDisplay.CYAN, ctypeUnused=afwDisplay.YELLOW, 

346 size=4, display=disp) 

347 for cell in cellSet.getCellList(): 

348 for cand in cell.begin(not showBadCandidates): # maybe include bad candidates 

349 status = cand.getStatus() 

350 disp.dot('+', *cand.getSource().getCentroid(), 

351 ctype=afwDisplay.GREEN if status == afwMath.SpatialCellCandidate.GOOD else 

352 afwDisplay.YELLOW if status == afwMath.SpatialCellCandidate.UNKNOWN else afwDisplay.RED) 

353 

354 

355def plotPsfCandidates(cellSet, showBadCandidates=False, frame=1): 

356 stamps = [] 

357 for cell in cellSet.getCellList(): 

358 for cand in cell.begin(not showBadCandidates): # maybe include bad candidates 

359 try: 

360 im = cand.getMaskedImage() 

361 

362 chi2 = cand.getChi2() 

363 if chi2 < 1e100: 

364 chi2 = "%.1f" % chi2 

365 else: 

366 chi2 = float("nan") 

367 

368 stamps.append((im, "%d%s" % 

369 (maUtils.splitId(cand.getSource().getId(), True)["objId"], chi2), 

370 cand.getStatus())) 

371 except Exception: 

372 continue 

373 

374 mos = afwDisplay.utils.Mosaic() 

375 disp = afwDisplay.Display(frame=frame) 

376 for im, label, status in stamps: 

377 im = type(im)(im, True) 

378 try: 

379 im /= afwMath.makeStatistics(im, afwMath.MAX).getValue() 

380 except NotImplementedError: 

381 pass 

382 

383 mos.append(im, label, 

384 afwDisplay.GREEN if status == afwMath.SpatialCellCandidate.GOOD else 

385 afwDisplay.YELLOW if status == afwMath.SpatialCellCandidate.UNKNOWN else afwDisplay.RED) 

386 

387 if mos.images: 

388 disp.mtv(mos.makeMosaic(), title="Psf Candidates") 

389 

390 

391def plotResiduals(exposure, cellSet, showBadCandidates=False, normalizeResiduals=True, frame=2): 

392 psf = exposure.getPsf() 

393 disp = afwDisplay.Display(frame=frame) 

394 while True: 

395 try: 

396 maUtils.showPsfCandidates(exposure, cellSet, psf=psf, display=disp, 

397 normalize=normalizeResiduals, 

398 showBadCandidates=showBadCandidates) 

399 frame += 1 

400 maUtils.showPsfCandidates(exposure, cellSet, psf=psf, display=disp, 

401 normalize=normalizeResiduals, 

402 showBadCandidates=showBadCandidates, 

403 variance=True) 

404 frame += 1 

405 except Exception: 

406 if not showBadCandidates: 

407 showBadCandidates = True 

408 continue 

409 break 

410 

411 return frame