Coverage for python/lsst/validate/drp/repeatability.py: 29%

Shortcuts on this page

r m x p   toggle line displays

j k   next/prev highlighted chunk

0   (zero) top of page

1   (one) first highlighted chunk

39 statements  

1# LSST Data Management System 

2# Copyright 2008-2019 AURA/LSST. 

3# 

4# This product includes software developed by the 

5# LSST Project (http://www.lsst.org/). 

6# 

7# This program is free software: you can redistribute it and/or modify 

8# it under the terms of the GNU General Public License as published by 

9# the Free Software Foundation, either version 3 of the License, or 

10# (at your option) any later version. 

11# 

12# This program is distributed in the hope that it will be useful, 

13# but WITHOUT ANY WARRANTY; without even the implied warranty of 

14# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 

15# GNU General Public License for more details. 

16# 

17# You should have received a copy of the LSST License Statement and 

18# the GNU General Public License along with this program. If not, 

19# see <https://www.lsstcorp.org/LegalNotices/>. 

20 

21import astropy.units as u 

22import math 

23import numpy as np 

24import random 

25from scipy.stats import norm 

26 

27import lsst.pipe.base as pipeBase 

28from lsst.verify import Measurement, Datum 

29 

30thousandDivSqrtTwo = 1000/math.sqrt(2) 

31 

32 

33def measurePhotRepeat(metric, filterName, *args, **kwargs): 

34 """Measurement of a photometric repeatability metric across a set of 

35 observations. 

36 

37 Parameters 

38 ---------- 

39 metric : `lsst.verify.Metric` 

40 A Metric to construct a Measurement for. 

41 filterName : `str` 

42 Name of filter used for all observations. 

43 *args 

44 Additional arguments to pass to `calcPhotRepeat`. 

45 **kwargs 

46 Additional keyword arguments to pass to `calcPhotRepeat`. 

47 

48 Returns 

49 ------- 

50 measurement : `lsst.verify.Measurement` 

51 Measurement of the repeatability and its associated metadata. 

52 

53 See also 

54 -------- 

55 calcPhotRepeat: Computes statistics of magnitudes differences of sources across 

56 multiple visits. This is the main computation function behind 

57 repeatability measurement. 

58 """ 

59 results = calcPhotRepeat(*args, **kwargs) 

60 datums = {} 

61 datums['filter_name'] = Datum(filterName, label='filter', 

62 description='Name of filter for this measurement') 

63 datums['rms'] = Datum(results['rms'], label='RMS', 

64 description='Photometric repeatability RMS of stellar pairs for ' 

65 'each random sampling') 

66 datums['iqr'] = Datum(results['iqr'], label='IQR', 

67 description='Photometric repeatability IQR of stellar pairs for ' 

68 'each random sample') 

69 datums['magDiff'] = Datum(results['magDiff'], label='Delta mag', 

70 description='Photometric repeatability differences magnitudes for ' 

71 'stellar pairs for each random sample') 

72 datums['magMean'] = Datum(results['magMean'], label='mag', 

73 description='Mean magnitude of pairs of stellar sources matched ' 

74 'across visits, for each random sample.') 

75 return Measurement(metric, results['repeatability'], extras=datums) 

76 

77 

78def calcPhotRepeat(matches, magKey, numRandomShuffles=50): 

79 """Calculate the photometric repeatability of measurements across a set 

80 of randomly selected pairs of visits. 

81 

82 Parameters 

83 ---------- 

84 matches : `lsst.afw.table.GroupView` 

85 `~lsst.afw.table.GroupView` of sources matched between visits, 

86 from MultiMatch, provided by 

87 `lsst.validate.drp.matchreduce.build_matched_dataset`. 

88 magKey : `lsst.afw.table` schema key 

89 Magnitude column key in the ``groupView``. 

90 E.g., ``magKey = allMatches.schema.find("slot_ModelFlux_mag").key`` 

91 where ``allMatches`` is the result of 

92 `lsst.afw.table.MultiMatch.finish()`. 

93 numRandomShuffles : int 

94 Number of times to draw random pairs from the different observations. 

95 

96 Returns 

97 ------- 

98 statistics : `dict` 

99 Statistics to compute model_phot_rep. Fields are: 

100 

101 - ``model_phot_rep``: scalar `~astropy.unit.Quantity` of mean ``iqr``. 

102 This is formally the model_phot_rep metric measurement. 

103 - ``rms``: `~astropy.unit.Quantity` array in mmag of photometric 

104 repeatability RMS across ``numRandomShuffles``. 

105 Shape: ``(nRandomSamples,)``. 

106 - ``iqr``: `~astropy.unit.Quantity` array in mmag of inter-quartile 

107 range of photometric repeatability distribution. 

108 Shape: ``(nRandomSamples,)``. 

109 - ``magDiff``: `~astropy.unit.Quantity` array of magnitude differences 

110 between pairs of sources. Shape: ``(nRandomSamples, nMatches)``. 

111 - ``magMean``: `~astropy.unit.Quantity` array of mean magnitudes of 

112 each pair of sources. Shape: ``(nRandomSamples, nMatches)``. 

113 

114 Notes 

115 ----- 

116 We calculate differences for ``numRandomShuffles`` different random 

117 realizations of the measurement pairs, to provide some estimate of the 

118 uncertainty on our RMS estimates due to the random shuffling. This 

119 estimate could be stated and calculated from a more formally derived 

120 motivation but in practice 50 should be sufficient. 

121 

122 The LSST Science Requirements Document (LPM-17), or SRD, characterizes the 

123 photometric repeatability by putting a requirement on the median RMS of 

124 measurements of non-variable bright stars. This quantity is PA1, with a 

125 design, minimum, and stretch goals of (5, 8, 3) millimag following LPM-17 

126 as of 2011-07-06, available at http://ls.st/LPM-17. model_phot_rep is a 

127 similar quantity measured for extended sources (almost entirely galaxies), 

128 for which no requirement currently exists in the SRD. 

129 

130 This present routine calculates this quantity in two different ways: 

131 

132 1. RMS 

133 2. interquartile range (IQR) 

134 

135 **The repeatability scalar measurement is the median of the IQR.** 

136 

137 This function also returns additional quantities of interest: 

138 

139 - the pair differences of observations of sources, 

140 - the mean magnitude of each source 

141 

142 Examples 

143 -------- 

144 Normally ``calcPhotRepeat`` is called by `measurePhotRepeat`, using 

145 data from `lsst.validate.drp.matchreduce.build_matched_dataset`. Here's an 

146 example of how to call ``calcPhotRepeat`` directly given the Butler output 

147 repository generated by examples/runHscQuickTest.sh: 

148 

149 >>> import lsst.daf.persistence as dafPersist 

150 >>> from lsst.afw.table import SourceCatalog, SchemaMapper, Field 

151 >>> from lsst.afw.table import MultiMatch, SourceRecord, GroupView 

152 >>> from lsst.validate.drp.repeatability import calcPhotRepeat 

153 >>> from lsst.validate.drp.util import discoverDataIds 

154 >>> import numpy as np 

155 >>> repo = 'HscQuick/output' 

156 >>> butler = dafPersist.Butler(repo) 

157 >>> dataset = 'src' 

158 >>> schema = butler.get(dataset + '_schema', immediate=True).schema 

159 >>> visitDataIds = discoverDataIds(repo) 

160 >>> mmatch = None 

161 >>> for vId in visitDataIds: 

162 ... cat = butler.get('src', vId) 

163 ... calib = butler.get('calexp_photoCalib', vId) 

164 ... cat = calib.calibrateCatalog(cat, ['modelfit_CModel']) 

165 ... if mmatch is None: 

166 ... mmatch = MultiMatch(cat.schema, 

167 ... dataIdFormat={'visit': np.int32, 'ccd': np.int32}, 

168 ... RecordClass=SourceRecord) 

169 ... mmatch.add(catalog=cat, dataId=vId) 

170 ... 

171 >>> matchCat = mmatch.finish() 

172 >>> allMatches = GroupView.build(matchCat) 

173 >>> magKey = allMatches.schema.find('slot_ModelFlux_mag').key 

174 >>> def matchFilter(cat): 

175 >>> if len(cat) < 2: 

176 >>> return False 

177 >>> return np.isfinite(cat.get(magKey)).all() 

178 >>> repeat = calcPhotRepeat(allMatches.where(matchFilter), magKey) 

179 

180 """ 

181 mprSamples = [calcPhotRepeatSample(matches, magKey) 

182 for _ in range(numRandomShuffles)] 

183 

184 rms = np.array([mpr.rms for mpr in mprSamples]) * u.mmag 

185 iqr = np.array([mpr.iqr for mpr in mprSamples]) * u.mmag 

186 magDiff = np.array([mpr.magDiffs for mpr in mprSamples]) * u.mmag 

187 magMean = np.array([mpr.magMean for mpr in mprSamples]) * u.mag 

188 repeat = np.mean(iqr) 

189 return {'rms': rms, 'iqr': iqr, 'magDiff': magDiff, 'magMean': magMean, 'repeatability': repeat} 

190 

191 

192def calcPhotRepeatSample(matches, magKey): 

193 """Compute one realization of repeatability by randomly sampling pairs of 

194 visits. 

195 

196 Parameters 

197 ---------- 

198 matches : `lsst.afw.table.GroupView` 

199 `~lsst.afw.table.GroupView` of sources matched between visits, 

200 from MultiMatch, provided by 

201 `lsst.validate.drp.matchreduce.build_matched_dataset`. 

202 magKey : `lsst.afw.table` schema key 

203 Magnitude column key in the ``groupView``. 

204 E.g., ``magKey = allMatches.schema.find("base_PsfFlux_mag").key`` 

205 where ``allMatches`` is the result of 

206 `lsst.afw.table.MultiMatch.finish()`. 

207 

208 Returns 

209 ------- 

210 metrics : `lsst.pipe.base.Struct` 

211 Metrics of pairs of sources matched between two visits. Fields are: 

212 

213 - ``rms``: scalar RMS of differences of sources observed in this 

214 randomly sampled pair of visits. 

215 - ``iqr``: scalar inter-quartile range (IQR) of differences of sources 

216 observed in a randomly sampled pair of visits. 

217 - ``magDiffs`: array, shape ``(nMatches,)``, of magnitude differences 

218 (mmag) for observed sources across a randomly sampled pair of visits. 

219 - ``magMean``: array, shape ``(nMatches,)``, of mean magnitudes 

220 of sources observed across a randomly sampled pair of visits. 

221 

222 See also 

223 -------- 

224 calcPhotRepeat : A wrapper that repeatedly calls this function to build 

225 the repeatability measurement. 

226 """ 

227 magDiffs = matches.aggregate(getRandomDiffRmsInMmags, field=magKey) 

228 magMean = matches.aggregate(np.mean, field=magKey) 

229 rms, iqr = computeWidths(magDiffs) 

230 return pipeBase.Struct(rms=rms, iqr=iqr, magDiffs=magDiffs, magMean=magMean,) 

231 

232 

233def getRandomDiffRmsInMmags(array): 

234 """Calculate the RMS difference in mmag between a random pairing of 

235 visits of a source. 

236 

237 Parameters 

238 ---------- 

239 array : `list` or `numpy.ndarray` 

240 Magnitudes from which to select the pair [mag]. 

241 

242 Returns 

243 ------- 

244 rmsMmags : `float` 

245 RMS difference in mmag from a random pair of visits. 

246 

247 Notes 

248 ----- 

249 The LSST SRD recommends computing repeatability from a histogram of 

250 magnitude differences for the same source measured on two visits 

251 (using a median over the magDiffs to reject outliers). 

252 Because we have N>=2 measurements for each source, we select a random 

253 pair of visits for each source. We divide each difference by sqrt(2) 

254 to obtain the RMS about the (unknown) mean magnitude, 

255 instead of obtaining just the RMS difference. 

256 

257 See Also 

258 -------- 

259 getRandomDiff : Get the difference between two randomly selected elements of an array. 

260 

261 Examples 

262 -------- 

263 >>> mag = [24.2, 25.5] 

264 >>> rms = getRandomDiffRmsInMmags(mag) 

265 >>> print(rms) 

266 212.132034 

267 """ 

268 return thousandDivSqrtTwo * getRandomDiff(array) 

269 

270 

271def getRandomDiff(array): 

272 """Get the difference between two randomly selected elements of an array. 

273 

274 Parameters 

275 ---------- 

276 array : `list` or `numpy.ndarray` 

277 Input array. 

278 

279 Returns 

280 ------- 

281 float or int 

282 Difference between two random elements of the array. 

283 """ 

284 a, b = random.sample(range(len(array)), 2) 

285 return array[a] - array[b] 

286 

287 

288def computeWidths(array): 

289 """Compute the RMS and the scaled inter-quartile range of an array. 

290 

291 Parameters 

292 ---------- 

293 array : `list` or `numpy.ndarray` 

294 Array. 

295 

296 Returns 

297 ------- 

298 rms : `float` 

299 RMS 

300 iqr : `float` 

301 Scaled inter-quartile range (IQR, see *Notes*). 

302 

303 Notes 

304 ----- 

305 We estimate the width of the histogram in two ways: 

306 

307 - using a simple RMS, 

308 - using the interquartile range (IQR) 

309 

310 The IQR is scaled by the IQR/RMS ratio for a Gaussian such that it 

311 if the array is Gaussian distributed, then the scaled IQR = RMS. 

312 """ 

313 # For scalars, math.sqrt is several times faster than numpy.sqrt. 

314 rmsSigma = math.sqrt(np.mean(array**2)) 

315 iqrSigma = np.subtract.reduce(np.percentile(array, [75, 25])) / (norm.ppf(0.75)*2) 

316 return rmsSigma, iqrSigma