Hide keyboard shortcuts

Hot-keys on this page

r m x p   toggle line displays

j k   next/prev highlighted chunk

0   (zero) top of page

1   (one) first highlighted chunk

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

49

50

51

52

53

54

55

56

57

58

59

60

61

62

63

64

65

66

67

68

69

70

71

72

73

74

75

76

77

78

79

80

81

82

83

84

85

86

87

88

89

90

91

92

93

94

95

96

97

98

99

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

157

158

159

160

161

162

163

164

165

166

167

168

169

170

171

172

173

174

175

176

177

178

179

180

181

182

183

184

185

186

187

188

189

190

191

192

193

194

195

196

197

198

199

200

201

202

203

204

205

206

207

208

209

210

211

212

213

214

215

216

217

218

219

220

221

222

223

224

225

226

227

228

229

230

231

232

233

234

235

236

237

238

239

240

241

242

243

244

245

246

247

248

249

250

251

252

253

254

255

256

257

258

259

260

261

262

263

264

265

266

267

268

269

270

271

272

273

274

275

276

277

278

279

280

281

282

283

284

285

286

287

288

289

290

291

292

293

294

# 

# LSST Data Management System 

# Copyright 2008-2016 AURA/LSST. 

# 

# This product includes software developed by the 

# LSST Project (http://www.lsst.org/). 

# 

# This program is free software: you can redistribute it and/or modify 

# it under the terms of the GNU General Public License as published by 

# the Free Software Foundation, either version 3 of the License, or 

# (at your option) any later version. 

# 

# This program is distributed in the hope that it will be useful, 

# but WITHOUT ANY WARRANTY; without even the implied warranty of 

# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 

# GNU General Public License for more details. 

# 

# You should have received a copy of the LSST License Statement and 

# the GNU General Public License along with this program. If not, 

# see <https://www.lsstcorp.org/LegalNotices/>. 

# 

import lsst.pex.config as pexConfig 

import lsst.afw.math as afwMath 

import lsst.afw.detection as afwDet 

import lsst.meas.algorithms as measAlg 

import lsst.pipe.base as pipeBase 

from lsstDebug import getDebugFrame 

from lsst.afw.display import getDisplay 

from lsst.pipe.tasks.interpImage import InterpImageTask 

 

 

class RepairConfig(pexConfig.Config): 

doInterpolate = pexConfig.Field( 

dtype=bool, 

doc="Interpolate over defects? (ignored unless you provide a list of defects)", 

default=True, 

) 

doCosmicRay = pexConfig.Field( 

dtype=bool, 

doc="Find and mask out cosmic rays?", 

default=True, 

) 

cosmicray = pexConfig.ConfigField( 

dtype=measAlg.FindCosmicRaysConfig, 

doc="Options for finding and masking cosmic rays", 

) 

interp = pexConfig.ConfigurableField( 

target=InterpImageTask, 

doc="Interpolate over bad image pixels", 

) 

 

def setDefaults(self): 

self.interp.useFallbackValueAtEdge = True 

self.interp.fallbackValueType = "MEANCLIP" 

self.interp.negativeFallbackAllowed = True 

 

## @addtogroup LSST_task_documentation 

## @{ 

## @page RepairTask 

## @ref RepairTask_ "RepairTask" 

## @copybrief RepairTask 

## @} 

 

 

class RepairTask(pipeBase.Task): 

r"""! 

@anchor RepairTask_ 

 

@brief Interpolate over defects in an exposure and handle cosmic rays 

 

@section pipe_tasks_repair_Contents Contents 

 

- @ref pipe_tasks_repair_Purpose 

- @ref pipe_tasks_repair_Initialize 

- @ref pipe_tasks_repair_IO 

- @ref pipe_tasks_repair_Config 

- @ref pipe_tasks_repair_Debug 

- @ref pipe_tasks_repair_Example 

 

@section pipe_tasks_repair_Purpose Description 

 

@copybrief RepairTask 

 

This task operates on an lsst.afw.image.Exposure in place to interpolate over a set of 

lsst.meas.algorithms.Defect objects. 

It will also, optionally, find and interpolate any cosmic rays in the lsst.afw.image.Exposure. 

 

@section pipe_tasks_repair_Initialize Task initialization 

 

See: lsst.pipe.base.task.Task.__init__ 

 

@section pipe_tasks_repair_IO Inputs/Outputs to the run method 

 

@copydoc run 

 

@section pipe_tasks_repair_Config Configuration parameters 

 

See @ref RepairConfig 

 

@section pipe_tasks_repair_Debug Debug variables 

 

The @link lsst.pipe.base.cmdLineTask.CmdLineTask command line task@endlink interface supports a 

flag @c -d to import @b debug.py from your @c PYTHONPATH; see <a 

href="http://lsst-web.ncsa.illinois.edu/~buildbot/doxygen/x_masterDoxyDoc/base_debug.html"> 

Using lsstDebug to control debugging output</a> for more about @b debug.py files. 

 

The available variables in RepairTask are: 

<DL> 

<DT> @c display 

<DD> A dictionary containing debug point names as keys with frame number as value. Valid keys are: 

<DL> 

<DT> repair.before 

<DD> display image before any repair is done 

<DT> repair.after 

<DD> display image after cosmic ray and defect correction 

</DL> 

<DT> @c displayCR 

<DD> If True, display the exposure on ds9's frame 1 and overlay bounding boxes around detects CRs. 

</DL> 

@section pipe_tasks_repair_Example A complete example of using RepairTask 

 

This code is in runRepair.py in the examples directory, and can be run as @em e.g. 

@code 

examples/runRepair.py 

@endcode 

@dontinclude runRepair.py 

Import the task. There are other imports. Read the source file for more info. 

@skipline RepairTask 

 

For this example, we manufacture a test image to run on. 

 

First, create a pure Poisson noise image and a Psf to go with it. The mask plane 

and variance can be constructed at the same time. 

@skip poisson 

@until mask 

 

Inject some cosmic rays and generate the Exposure. Exposures are MaskedImages (image + mask + variance) 

with other metadata (e.g. Psf and Wcs objects). 

@skip some CRs 

@until setPsf 

 

Defects are represented as bad columns of random lengths. A defect list must be constructed to pass 

on to the RepairTask. 

@bug This is addressed in <a href="https://jira.lsstcorp.org/browse/DM-963"> DM-963</a> 

 

@skip addDefects 

@until push_back 

 

Finally, the exposure can be repaired. Create an instance of the task and run it. The exposure 

is modified in place. 

@skip RepairTask 

@until repair.run 

 

<HR> 

To investigate the @ref pipe_tasks_repair_Debug, put something like 

@code{.py} 

import lsstDebug 

def DebugInfo(name): 

di = lsstDebug.getInfo(name) # N.b. lsstDebug.Info(name) would call us recursively 

if name == "lsst.pipe.tasks.repair": 

di.display = {'repair.before':2, 'repair.after':3} 

di.displayCR = True 

return di 

 

lsstDebug.Info = DebugInfo 

@endcode 

into your debug.py file and run runRepair.py with the @c --debug flag. 

 

 

Conversion notes: 

Display code should be updated once we settle on a standard way of controlling what is displayed. 

""" 

ConfigClass = RepairConfig 

_DefaultName = "repair" 

 

def __init__(self, **kwargs): 

pipeBase.Task.__init__(self, **kwargs) 

178 ↛ exitline 178 didn't return from function '__init__', because the condition on line 178 was never false if self.config.doInterpolate: 

self.makeSubtask("interp") 

 

@pipeBase.timeMethod 

def run(self, exposure, defects=None, keepCRs=None): 

"""!Repair an Exposure's defects and cosmic rays 

 

@param[in, out] exposure lsst.afw.image.Exposure to process. Exposure must have a valid Psf. 

Modified in place. 

@param[in] defects an lsst.meas.algorithms.DefectListT object. If None, do no 

defect correction. 

@param[in] keepCRs don't interpolate over the CR pixels (defer to RepairConfig if None) 

 

@throws AssertionError with the following strings: 

 

<DL> 

<DT> No exposure provided 

<DD> The object provided as exposure evaluates to False 

<DT> No PSF provided 

<DD> The Exposure has no associated Psf 

</DL> 

""" 

assert exposure, "No exposure provided" 

psf = exposure.getPsf() 

assert psf, "No PSF provided" 

 

frame = getDebugFrame(self._display, "repair.before") 

205 ↛ 206line 205 didn't jump to line 206, because the condition on line 205 was never true if frame: 

getDisplay(frame).mtv(exposure) 

 

208 ↛ 209line 208 didn't jump to line 209, because the condition on line 208 was never true if defects is not None and self.config.doInterpolate: 

self.interp.run(exposure, defects=defects) 

 

211 ↛ 214line 211 didn't jump to line 214, because the condition on line 211 was never false if self.config.doCosmicRay: 

self.cosmicRay(exposure, keepCRs=keepCRs) 

 

frame = getDebugFrame(self._display, "repair.after") 

215 ↛ 216line 215 didn't jump to line 216, because the condition on line 215 was never true if frame: 

getDisplay(frame).mtv(exposure) 

 

def cosmicRay(self, exposure, keepCRs=None): 

"""Mask cosmic rays 

 

@param[in,out] exposure Exposure to process 

@param[in] keepCRs Don't interpolate over the CR pixels (defer to pex_config if None) 

""" 

import lsstDebug 

display = lsstDebug.Info(__name__).display 

displayCR = lsstDebug.Info(__name__).displayCR 

 

assert exposure, "No exposure provided" 

psf = exposure.getPsf() 

assert psf, "No psf provided" 

 

# Blow away old mask 

try: 

mask = exposure.getMaskedImage().getMask() 

crBit = mask.getMaskPlane("CR") 

mask.clearMaskPlane(crBit) 

except Exception: 

pass 

 

exposure0 = exposure # initial value of exposure 

binSize = self.config.cosmicray.background.binSize 

nx, ny = exposure.getWidth()/binSize, exposure.getHeight()/binSize 

# Treat constant background as a special case to avoid the extra complexity in calling 

# measAlg.SubtractBackgroundTask(). 

245 ↛ 252line 245 didn't jump to line 252, because the condition on line 245 was never false if nx*ny <= 1: 

medianBg = afwMath.makeStatistics(exposure.getMaskedImage(), afwMath.MEDIAN).getValue() 

modelBg = None 

else: 

# make a deep copy of the exposure before subtracting its background, 

# because this routine is only allowed to modify the exposure by setting mask planes 

# and interpolating over defects, not changing the background level 

exposure = exposure.Factory(exposure, True) 

subtractBackgroundTask = measAlg.SubtractBackgroundTask(config=self.config.cosmicray.background) 

modelBg = subtractBackgroundTask.run(exposure).background 

medianBg = 0.0 

 

if keepCRs is None: 

keepCRs = self.config.cosmicray.keepCRs 

try: 

crs = measAlg.findCosmicRays(exposure.getMaskedImage(), psf, medianBg, 

pexConfig.makePolicy(self.config.cosmicray), keepCRs) 

262 ↛ 264line 262 didn't jump to line 264, because the condition on line 262 was never true if modelBg: 

# Add back background image 

img = exposure.getMaskedImage() 

img += modelBg.getImageF() 

del img 

# Replace original image with CR subtracted image 

exposure0.setMaskedImage(exposure.getMaskedImage()) 

 

except Exception: 

if display: 

import lsst.afw.display.ds9 as ds9 

ds9.mtv(exposure0, title="Failed CR") 

raise 

 

num = 0 

277 ↛ 294line 277 didn't jump to line 294, because the condition on line 277 was never false if crs is not None: 

mask = exposure0.getMaskedImage().getMask() 

crBit = mask.getPlaneBitMask("CR") 

afwDet.setMaskFromFootprintList(mask, crs, crBit) 

num = len(crs) 

 

283 ↛ 284line 283 didn't jump to line 284, because the condition on line 283 was never true if display and displayCR: 

import lsst.afw.display.ds9 as ds9 

import lsst.afw.display.utils as displayUtils 

 

ds9.incrDefaultFrame() 

ds9.mtv(exposure0, title="Post-CR") 

 

with ds9.Buffering(): 

for cr in crs: 

displayUtils.drawBBox(cr.getBBox(), borderWidth=0.55) 

 

self.log.info("Identified %s cosmic rays." % (num,))