Hide keyboard shortcuts

Hot-keys on this page

r m x p   toggle line displays

j k   next/prev highlighted chunk

0   (zero) top of page

1   (one) first highlighted chunk

1# 

2# LSST Data Management System 

3# Copyright 2008-2017 AURA/LSST. 

4# 

5# This product includes software developed by the 

6# LSST Project (http://www.lsst.org/). 

7# 

8# This program is free software: you can redistribute it and/or modify 

9# it under the terms of the GNU General Public License as published by 

10# the Free Software Foundation, either version 3 of the License, or 

11# (at your option) any later version. 

12# 

13# This program is distributed in the hope that it will be useful, 

14# but WITHOUT ANY WARRANTY; without even the implied warranty of 

15# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 

16# GNU General Public License for more details. 

17# 

18# You should have received a copy of the LSST License Statement and 

19# the GNU General Public License along with this program. If not, 

20# see <https://www.lsstcorp.org/LegalNotices/>. 

21# 

22""" 

23Define dataset class for MeasurePhotonTransferCurve task 

24""" 

25import numpy as np 

26from astropy.table import Table 

27 

28from lsst.ip.isr import IsrCalib 

29 

30__all__ = ['PhotonTransferCurveDataset'] 

31 

32 

33class PhotonTransferCurveDataset(IsrCalib): 

34 """A simple class to hold the output data from the PTC task. 

35 The dataset is made up of a dictionary for each item, keyed by the 

36 amplifiers' names, which much be supplied at construction time. 

37 New items cannot be added to the class to save accidentally saving to the 

38 wrong property, and the class can be frozen if desired. 

39 inputExpIdPairs records the exposures used to produce the data. 

40 When fitPtc() or fitCovariancesAstier() is run, a mask is built up, which 

41 is by definition always the same length as inputExpIdPairs, rawExpTimes, 

42 rawMeans and rawVars, and is a list of bools, which are incrementally set 

43 to False as points are discarded from the fits. 

44 PTC fit parameters for polynomials are stored in a list in ascending order 

45 of polynomial term, i.e. par[0]*x^0 + par[1]*x + par[2]*x^2 etc 

46 with the length of the list corresponding to the order of the polynomial 

47 plus one. 

48 

49 Parameters 

50 ---------- 

51 ampNames : `list` 

52 List with the names of the amplifiers of the detector at hand. 

53 

54 ptcFitType : `str` 

55 Type of model fitted to the PTC: "POLYNOMIAL", "EXPAPPROXIMATION", 

56 or "FULLCOVARIANCE". 

57 

58 covMatrixSide : `int` 

59 Maximum lag of covariances (size of square covariance matrices). 

60 

61 kwargs : `dict`, optional 

62 Other keyword arguments to pass to the parent init. 

63 

64 Notes 

65 ----- 

66 The stored attributes are: 

67 badAmps : `list` 

68 List with bad amplifiers names. 

69 inputExpIdPairs : `dict`, [`str`, `list`] 

70 Dictionary keyed by amp names containing the input exposures IDs. 

71 expIdMask : `dict`, [`str`, `list`] 

72 Dictionary keyed by amp names containing the mask produced after 

73 outlier rejection. The mask produced by the "FULLCOVARIANCE" 

74 option may differ from the one produced in the other two PTC 

75 fit types. 

76 rawExpTimes : `dict`, [`str`, `list`] 

77 Dictionary keyed by amp names containing the unmasked exposure times. 

78 rawMeans : `dict`, [`str`, `list`] 

79 Dictionary keyed by amp namescontaining the unmasked average of the 

80 means of the exposures in each flat pair. 

81 rawVars : `dict`, [`str`, `list`] 

82 Dictionary keyed by amp names containing the variance of the 

83 difference image of the exposures in each flat pair. 

84 gain : `dict`, [`str`, `list`] 

85 Dictionary keyed by amp names containing the fitted gains. 

86 gainErr : `dict`, [`str`, `list`] 

87 Dictionary keyed by amp names containing the errors on the 

88 fitted gains. 

89 noise : `dict`, [`str`, `list`] 

90 Dictionary keyed by amp names containing the fitted noise. 

91 noiseErr : `dict`, [`str`, `list`] 

92 Dictionary keyed by amp names containing the errors on the fitted noise. 

93 ptcFitPars : `dict`, [`str`, `list`] 

94 Dictionary keyed by amp names containing the fitted parameters of the 

95 PTC model for ptcFitTye in ["POLYNOMIAL", "EXPAPPROXIMATION"]. 

96 ptcFitParsError : `dict`, [`str`, `list`] 

97 Dictionary keyed by amp names containing the errors on the fitted 

98 parameters of the PTC model for ptcFitTye in 

99 ["POLYNOMIAL", "EXPAPPROXIMATION"]. 

100 ptcFitChiSq : `dict`, [`str`, `list`] 

101 Dictionary keyed by amp names containing the reduced chi squared 

102 of the fit for ptcFitTye in ["POLYNOMIAL", "EXPAPPROXIMATION"]. 

103 covariances : `dict`, [`str`, `list`] 

104 Dictionary keyed by amp names containing a list of measured 

105 covariances per mean flux. 

106 covariancesModel : `dict`, [`str`, `list`] 

107 Dictionary keyed by amp names containinging covariances model 

108 (Eq. 20 of Astier+19) per mean flux. 

109 covariancesSqrtWeights : `dict`, [`str`, `list`] 

110 Dictionary keyed by amp names containinging sqrt. of covariances 

111 weights. 

112 aMatrix : `dict`, [`str`, `list`] 

113 Dictionary keyed by amp names containing the "a" parameters from 

114 the model in Eq. 20 of Astier+19. 

115 bMatrix : `dict`, [`str`, `list`] 

116 Dictionary keyed by amp names containing the "b" parameters from 

117 the model in Eq. 20 of Astier+19. 

118 covariancesModelNoB : `dict`, [`str`, `list`] 

119 Dictionary keyed by amp names containing covariances model 

120 (with 'b'=0 in Eq. 20 of Astier+19) 

121 per mean flux. 

122 aMatrixNoB : `dict`, [`str`, `list`] 

123 Dictionary keyed by amp names containing the "a" parameters from the 

124 model in Eq. 20 of Astier+19 

125 (and 'b' = 0). 

126 finalVars : `dict`, [`str`, `list`] 

127 Dictionary keyed by amp names containing the masked variance of the 

128 difference image of each flat 

129 pair. If needed, each array will be right-padded with 

130 np.nan to match the length of rawExpTimes. 

131 finalModelVars : `dict`, [`str`, `list`] 

132 Dictionary keyed by amp names containing the masked modeled 

133 variance of the difference image of each flat pair. If needed, each 

134 array will be right-padded with np.nan to match the length of 

135 rawExpTimes. 

136 finalMeans : `dict`, [`str`, `list`] 

137 Dictionary keyed by amp names containing the masked average of the 

138 means of the exposures in each flat pair. If needed, each array 

139 will be right-padded with np.nan to match the length of 

140 rawExpTimes. 

141 photoCharge : `dict`, [`str`, `list`] 

142 Dictionary keyed by amp names containing the integrated photocharge 

143 for linearity calibration. 

144 

145 Returns 

146 ------- 

147 `lsst.cp.pipe.ptc.PhotonTransferCurveDataset` 

148 Output dataset from MeasurePhotonTransferCurveTask. 

149 """ 

150 

151 _OBSTYPE = 'PTC' 

152 _SCHEMA = 'Gen3 Photon Transfer Curve' 

153 _VERSION = 1.0 

154 

155 def __init__(self, ampNames=[], ptcFitType=None, covMatrixSide=1, **kwargs): 

156 

157 self.ptcFitType = ptcFitType 

158 self.ampNames = ampNames 

159 self.covMatrixSide = covMatrixSide 

160 

161 self.badAmps = [np.nan] 

162 

163 self.inputExpIdPairs = {ampName: [] for ampName in ampNames} 

164 self.expIdMask = {ampName: [] for ampName in ampNames} 

165 self.rawExpTimes = {ampName: [] for ampName in ampNames} 

166 self.rawMeans = {ampName: [] for ampName in ampNames} 

167 self.rawVars = {ampName: [] for ampName in ampNames} 

168 self.photoCharge = {ampName: [] for ampName in ampNames} 

169 

170 self.gain = {ampName: np.nan for ampName in ampNames} 

171 self.gainErr = {ampName: np.nan for ampName in ampNames} 

172 self.noise = {ampName: np.nan for ampName in ampNames} 

173 self.noiseErr = {ampName: np.nan for ampName in ampNames} 

174 

175 self.ptcFitPars = {ampName: [] for ampName in ampNames} 

176 self.ptcFitParsError = {ampName: [] for ampName in ampNames} 

177 self.ptcFitChiSq = {ampName: np.nan for ampName in ampNames} 

178 

179 self.covariances = {ampName: [] for ampName in ampNames} 

180 self.covariancesModel = {ampName: [] for ampName in ampNames} 

181 self.covariancesSqrtWeights = {ampName: [] for ampName in ampNames} 

182 self.aMatrix = {ampName: np.nan for ampName in ampNames} 

183 self.bMatrix = {ampName: np.nan for ampName in ampNames} 

184 self.covariancesModelNoB = {ampName: [] for ampName in ampNames} 

185 self.aMatrixNoB = {ampName: np.nan for ampName in ampNames} 

186 

187 self.finalVars = {ampName: [] for ampName in ampNames} 

188 self.finalModelVars = {ampName: [] for ampName in ampNames} 

189 self.finalMeans = {ampName: [] for ampName in ampNames} 

190 

191 super().__init__(**kwargs) 

192 self.requiredAttributes.update(['badAmps', 'inputExpIdPairs', 'expIdMask', 'rawExpTimes', 

193 'rawMeans', 'rawVars', 'gain', 'gainErr', 'noise', 'noiseErr', 

194 'ptcFitPars', 'ptcFitParsError', 'ptcFitChiSq', 'aMatrixNoB', 

195 'covariances', 'covariancesModel', 'covariancesSqrtWeights', 

196 'covariancesModelNoB', 

197 'aMatrix', 'bMatrix', 'finalVars', 'finalModelVars', 'finalMeans', 

198 'photoCharge']) 

199 

200 def __eq__(self, other): 

201 """Calibration equivalence 

202 """ 

203 if not isinstance(other, self.__class__): 

204 return False 

205 

206 for attr in self._requiredAttributes: 

207 attrSelf = getattr(self, attr) 

208 attrOther = getattr(other, attr) 

209 if isinstance(attrSelf, dict) and isinstance(attrOther, dict): 

210 for ampName in attrSelf: 

211 if not np.allclose(attrSelf[ampName], attrOther[ampName], equal_nan=True): 

212 return False 

213 else: 

214 if attrSelf != attrOther: 

215 return False 

216 return True 

217 

218 def setAmpValues(self, ampName, inputExpIdPair=[(np.nan, np.nan)], expIdMask=[np.nan], 

219 rawExpTime=[np.nan], rawMean=[np.nan], rawVar=[np.nan], photoCharge=[np.nan], 

220 gain=np.nan, gainErr=np.nan, noise=np.nan, noiseErr=np.nan, ptcFitPars=[np.nan], 

221 ptcFitParsError=[np.nan], ptcFitChiSq=np.nan, covArray=[], covArrayModel=[], 

222 covSqrtWeights=[], aMatrix=[], bMatrix=[], covArrayModelNoB=[], aMatrixNoB=[], 

223 finalVar=[np.nan], finalModelVar=[np.nan], finalMean=[np.nan]): 

224 """Function to initialize an amp of a PhotonTransferCurveDataset. 

225 

226 Notes 

227 ----- 

228 The parameters are all documented in `init`. 

229 """ 

230 nanMatrix = np.full((self.covMatrixSide, self.covMatrixSide), np.nan) 

231 if len(covArray) == 0: 

232 covArray = [nanMatrix] 

233 if len(covArrayModel) == 0: 

234 covArrayModel = [nanMatrix] 

235 if len(covSqrtWeights) == 0: 

236 covSqrtWeights = [nanMatrix] 

237 if len(covArrayModelNoB) == 0: 

238 covArrayModelNoB = [nanMatrix] 

239 if len(aMatrix) == 0: 

240 aMatrix = nanMatrix 

241 if len(bMatrix) == 0: 

242 bMatrix = nanMatrix 

243 if len(aMatrixNoB) == 0: 

244 aMatrixNoB = nanMatrix 

245 

246 self.inputExpIdPairs[ampName] = inputExpIdPair 

247 self.expIdMask[ampName] = expIdMask 

248 self.rawExpTimes[ampName] = rawExpTime 

249 self.rawMeans[ampName] = rawMean 

250 self.rawVars[ampName] = rawVar 

251 self.photoCharge[ampName] = photoCharge 

252 self.gain[ampName] = gain 

253 self.gainErr[ampName] = gainErr 

254 self.noise[ampName] = noise 

255 self.noiseErr[ampName] = noiseErr 

256 self.ptcFitPars[ampName] = ptcFitPars 

257 self.ptcFitParsError[ampName] = ptcFitParsError 

258 self.ptcFitChiSq[ampName] 

259 self.covariances[ampName] = covArray 

260 self.covariancesSqrtWeights[ampName] = covSqrtWeights 

261 self.covariancesModel[ampName] = covArrayModel 

262 self.covariancesModelNoB[ampName] = covArrayModelNoB 

263 self.aMatrix[ampName] = aMatrix 

264 self.bMatrix[ampName] = bMatrix 

265 self.aMatrixNoB[ampName] = aMatrixNoB 

266 self.ptcFitPars[ampName] = ptcFitPars 

267 self.ptcFitParsError[ampName] = ptcFitParsError 

268 self.ptcFitChiSq[ampName] = ptcFitChiSq 

269 self.finalVars[ampName] = finalVar 

270 self.finalModelVars[ampName] = finalModelVar 

271 self.finalMeans[ampName] = finalMean 

272 

273 def updateMetadata(self, setDate=False, **kwargs): 

274 """Update calibration metadata. 

275 This calls the base class's method after ensuring the required 

276 calibration keywords will be saved. 

277 Parameters 

278 ---------- 

279 setDate : `bool`, optional 

280 Update the CALIBDATE fields in the metadata to the current 

281 time. Defaults to False. 

282 kwargs : 

283 Other keyword parameters to set in the metadata. 

284 """ 

285 kwargs['PTC_FIT_TYPE'] = self.ptcFitType 

286 

287 super().updateMetadata(setDate=setDate, **kwargs) 

288 

289 @classmethod 

290 def fromDict(cls, dictionary): 

291 """Construct a calibration from a dictionary of properties. 

292 Must be implemented by the specific calibration subclasses. 

293 Parameters 

294 ---------- 

295 dictionary : `dict` 

296 Dictionary of properties. 

297 Returns 

298 ------- 

299 calib : `lsst.ip.isr.CalibType` 

300 Constructed calibration. 

301 Raises 

302 ------ 

303 RuntimeError : 

304 Raised if the supplied dictionary is for a different 

305 calibration. 

306 """ 

307 calib = cls() 

308 if calib._OBSTYPE != dictionary['metadata']['OBSTYPE']: 

309 raise RuntimeError(f"Incorrect Photon Transfer Curve dataset supplied. " 

310 f"Expected {calib._OBSTYPE}, found {dictionary['metadata']['OBSTYPE']}") 

311 calib.setMetadata(dictionary['metadata']) 

312 calib.ptcFitType = dictionary['ptcFitType'] 

313 calib.covMatrixSide = dictionary['covMatrixSide'] 

314 calib.badAmps = np.array(dictionary['badAmps'], 'str').tolist() 

315 # The cov matrices are square 

316 covMatrixSide = calib.covMatrixSide 

317 # Number of final signal levels 

318 covDimensionsProduct = len(np.array(list(dictionary['covariances'].values())[0]).ravel()) 

319 nSignalPoints = int(covDimensionsProduct/(covMatrixSide*covMatrixSide)) 

320 

321 for ampName in dictionary['ampNames']: 

322 calib.ampNames.append(ampName) 

323 calib.inputExpIdPairs[ampName] = np.array(dictionary['inputExpIdPairs'][ampName]).tolist() 

324 calib.expIdMask[ampName] = np.array(dictionary['expIdMask'][ampName]).tolist() 

325 calib.rawExpTimes[ampName] = np.array(dictionary['rawExpTimes'][ampName]).tolist() 

326 calib.rawMeans[ampName] = np.array(dictionary['rawMeans'][ampName]).tolist() 

327 calib.rawVars[ampName] = np.array(dictionary['rawVars'][ampName]).tolist() 

328 calib.gain[ampName] = np.array(dictionary['gain'][ampName]).tolist() 

329 calib.gainErr[ampName] = np.array(dictionary['gainErr'][ampName]).tolist() 

330 calib.noise[ampName] = np.array(dictionary['noise'][ampName]).tolist() 

331 calib.noiseErr[ampName] = np.array(dictionary['noiseErr'][ampName]).tolist() 

332 calib.ptcFitPars[ampName] = np.array(dictionary['ptcFitPars'][ampName]).tolist() 

333 calib.ptcFitParsError[ampName] = np.array(dictionary['ptcFitParsError'][ampName]).tolist() 

334 calib.ptcFitChiSq[ampName] = np.array(dictionary['ptcFitChiSq'][ampName]).tolist() 

335 calib.covariances[ampName] = np.array(dictionary['covariances'][ampName]).reshape( 

336 (nSignalPoints, covMatrixSide, covMatrixSide)).tolist() 

337 calib.covariancesModel[ampName] = np.array( 

338 dictionary['covariancesModel'][ampName]).reshape( 

339 (nSignalPoints, covMatrixSide, covMatrixSide)).tolist() 

340 calib.covariancesSqrtWeights[ampName] = np.array( 

341 dictionary['covariancesSqrtWeights'][ampName]).reshape( 

342 (nSignalPoints, covMatrixSide, covMatrixSide)).tolist() 

343 calib.aMatrix[ampName] = np.array(dictionary['aMatrix'][ampName]).reshape( 

344 (covMatrixSide, covMatrixSide)).tolist() 

345 calib.bMatrix[ampName] = np.array(dictionary['bMatrix'][ampName]).reshape( 

346 (covMatrixSide, covMatrixSide)).tolist() 

347 calib.covariancesModelNoB[ampName] = np.array( 

348 dictionary['covariancesModelNoB'][ampName]).reshape( 

349 (nSignalPoints, covMatrixSide, covMatrixSide)).tolist() 

350 calib.aMatrixNoB[ampName] = np.array( 

351 dictionary['aMatrixNoB'][ampName]).reshape((covMatrixSide, covMatrixSide)).tolist() 

352 calib.finalVars[ampName] = np.array(dictionary['finalVars'][ampName]).tolist() 

353 calib.finalModelVars[ampName] = np.array(dictionary['finalModelVars'][ampName]).tolist() 

354 calib.finalMeans[ampName] = np.array(dictionary['finalMeans'][ampName]).tolist() 

355 calib.photoCharge[ampName] = np.array(dictionary['photoCharge'][ampName]).tolist() 

356 calib.updateMetadata() 

357 return calib 

358 

359 def toDict(self): 

360 """Return a dictionary containing the calibration properties. 

361 The dictionary should be able to be round-tripped through 

362 `fromDict`. 

363 Returns 

364 ------- 

365 dictionary : `dict` 

366 Dictionary of properties. 

367 """ 

368 self.updateMetadata() 

369 

370 outDict = dict() 

371 metadata = self.getMetadata() 

372 outDict['metadata'] = metadata 

373 

374 outDict['ptcFitType'] = self.ptcFitType 

375 outDict['covMatrixSide'] = self.covMatrixSide 

376 outDict['ampNames'] = self.ampNames 

377 outDict['badAmps'] = self.badAmps 

378 outDict['inputExpIdPairs'] = self.inputExpIdPairs 

379 outDict['expIdMask'] = self.expIdMask 

380 outDict['rawExpTimes'] = self.rawExpTimes 

381 outDict['rawMeans'] = self.rawMeans 

382 outDict['rawVars'] = self.rawVars 

383 outDict['gain'] = self.gain 

384 outDict['gainErr'] = self.gainErr 

385 outDict['noise'] = self.noise 

386 outDict['noiseErr'] = self.noiseErr 

387 outDict['ptcFitPars'] = self.ptcFitPars 

388 outDict['ptcFitParsError'] = self.ptcFitParsError 

389 outDict['ptcFitChiSq'] = self.ptcFitChiSq 

390 outDict['covariances'] = self.covariances 

391 outDict['covariancesModel'] = self.covariancesModel 

392 outDict['covariancesSqrtWeights'] = self.covariancesSqrtWeights 

393 outDict['aMatrix'] = self.aMatrix 

394 outDict['bMatrix'] = self.bMatrix 

395 outDict['covariancesModelNoB'] = self.covariancesModelNoB 

396 outDict['aMatrixNoB'] = self.aMatrixNoB 

397 outDict['finalVars'] = self.finalVars 

398 outDict['finalModelVars'] = self.finalModelVars 

399 outDict['finalMeans'] = self.finalMeans 

400 outDict['photoCharge'] = self.photoCharge 

401 

402 return outDict 

403 

404 @classmethod 

405 def fromTable(cls, tableList): 

406 """Construct calibration from a list of tables. 

407 This method uses the `fromDict` method to create the 

408 calibration, after constructing an appropriate dictionary from 

409 the input tables. 

410 Parameters 

411 ---------- 

412 tableList : `list` [`lsst.afw.table.Table`] 

413 List of tables to use to construct the datasetPtc. 

414 Returns 

415 ------- 

416 calib : `lsst.cp.pipe.` 

417 The calibration defined in the tables. 

418 """ 

419 ptcTable = tableList[0] 

420 

421 metadata = ptcTable.meta 

422 inDict = dict() 

423 inDict['metadata'] = metadata 

424 inDict['ampNames'] = [] 

425 inDict['ptcFitType'] = [] 

426 inDict['covMatrixSide'] = [] 

427 inDict['inputExpIdPairs'] = dict() 

428 inDict['expIdMask'] = dict() 

429 inDict['rawExpTimes'] = dict() 

430 inDict['rawMeans'] = dict() 

431 inDict['rawVars'] = dict() 

432 inDict['gain'] = dict() 

433 inDict['gainErr'] = dict() 

434 inDict['noise'] = dict() 

435 inDict['noiseErr'] = dict() 

436 inDict['ptcFitPars'] = dict() 

437 inDict['ptcFitParsError'] = dict() 

438 inDict['ptcFitChiSq'] = dict() 

439 inDict['covariances'] = dict() 

440 inDict['covariancesModel'] = dict() 

441 inDict['covariancesSqrtWeights'] = dict() 

442 inDict['aMatrix'] = dict() 

443 inDict['bMatrix'] = dict() 

444 inDict['covariancesModelNoB'] = dict() 

445 inDict['aMatrixNoB'] = dict() 

446 inDict['finalVars'] = dict() 

447 inDict['finalModelVars'] = dict() 

448 inDict['finalMeans'] = dict() 

449 inDict['badAmps'] = [] 

450 inDict['photoCharge'] = dict() 

451 

452 for record in ptcTable: 

453 ampName = record['AMPLIFIER_NAME'] 

454 

455 inDict['ptcFitType'] = record['PTC_FIT_TYPE'] 

456 inDict['covMatrixSide'] = record['COV_MATRIX_SIDE'] 

457 inDict['ampNames'].append(ampName) 

458 inDict['inputExpIdPairs'][ampName] = record['INPUT_EXP_ID_PAIRS'] 

459 inDict['expIdMask'][ampName] = record['EXP_ID_MASK'] 

460 inDict['rawExpTimes'][ampName] = record['RAW_EXP_TIMES'] 

461 inDict['rawMeans'][ampName] = record['RAW_MEANS'] 

462 inDict['rawVars'][ampName] = record['RAW_VARS'] 

463 inDict['gain'][ampName] = record['GAIN'] 

464 inDict['gainErr'][ampName] = record['GAIN_ERR'] 

465 inDict['noise'][ampName] = record['NOISE'] 

466 inDict['noiseErr'][ampName] = record['NOISE_ERR'] 

467 inDict['ptcFitPars'][ampName] = record['PTC_FIT_PARS'] 

468 inDict['ptcFitParsError'][ampName] = record['PTC_FIT_PARS_ERROR'] 

469 inDict['ptcFitChiSq'][ampName] = record['PTC_FIT_CHI_SQ'] 

470 inDict['covariances'][ampName] = record['COVARIANCES'] 

471 inDict['covariancesModel'][ampName] = record['COVARIANCES_MODEL'] 

472 inDict['covariancesSqrtWeights'][ampName] = record['COVARIANCES_SQRT_WEIGHTS'] 

473 inDict['aMatrix'][ampName] = record['A_MATRIX'] 

474 inDict['bMatrix'][ampName] = record['B_MATRIX'] 

475 inDict['covariancesModelNoB'][ampName] = record['COVARIANCES_MODEL_NO_B'] 

476 inDict['aMatrixNoB'][ampName] = record['A_MATRIX_NO_B'] 

477 inDict['finalVars'][ampName] = record['FINAL_VARS'] 

478 inDict['finalModelVars'][ampName] = record['FINAL_MODEL_VARS'] 

479 inDict['finalMeans'][ampName] = record['FINAL_MEANS'] 

480 inDict['badAmps'] = record['BAD_AMPS'] 

481 inDict['photoCharge'][ampName] = record['PHOTO_CHARGE'] 

482 return cls().fromDict(inDict) 

483 

484 def toTable(self): 

485 """Construct a list of tables containing the information in this 

486 calibration. 

487 

488 The list of tables should create an identical calibration 

489 after being passed to this class's fromTable method. 

490 Returns 

491 ------- 

492 tableList : `list` [`astropy.table.Table`] 

493 List of tables containing the linearity calibration 

494 information. 

495 """ 

496 tableList = [] 

497 self.updateMetadata() 

498 nPoints = [] 

499 for i, ampName in enumerate(self.ampNames): 

500 nPoints.append(len(list(self.covariances.values())[i])) 

501 nSignalPoints = max(nPoints) 

502 nPadPoints = {} 

503 for i, ampName in enumerate(self.ampNames): 

504 nPadPoints[ampName] = nSignalPoints - len(list(self.covariances.values())[i]) 

505 covMatrixSide = self.covMatrixSide 

506 

507 catalog = Table([{'AMPLIFIER_NAME': ampName, 

508 'PTC_FIT_TYPE': self.ptcFitType, 

509 'COV_MATRIX_SIDE': self.covMatrixSide, 

510 'INPUT_EXP_ID_PAIRS': self.inputExpIdPairs[ampName] 

511 if len(self.expIdMask[ampName]) else np.nan, 

512 'EXP_ID_MASK': self.expIdMask[ampName] 

513 if len(self.expIdMask[ampName]) else np.nan, 

514 'RAW_EXP_TIMES': np.array(self.rawExpTimes[ampName]).tolist() 

515 if len(self.rawExpTimes[ampName]) else np.nan, 

516 'RAW_MEANS': np.array(self.rawMeans[ampName]).tolist() 

517 if len(self.rawMeans[ampName]) else np.nan, 

518 'RAW_VARS': np.array(self.rawVars[ampName]).tolist() 

519 if len(self.rawVars[ampName]) else np.nan, 

520 'GAIN': self.gain[ampName], 

521 'GAIN_ERR': self.gainErr[ampName], 

522 'NOISE': self.noise[ampName], 

523 'NOISE_ERR': self.noiseErr[ampName], 

524 'PTC_FIT_PARS': np.array(self.ptcFitPars[ampName]).tolist(), 

525 'PTC_FIT_PARS_ERROR': np.array(self.ptcFitParsError[ampName]).tolist(), 

526 'PTC_FIT_CHI_SQ': self.ptcFitChiSq[ampName], 

527 'COVARIANCES': np.pad(np.array(self.covariances[ampName]), 

528 ((0, nPadPoints[ampName]), (0, 0), (0, 0)), 

529 'constant', constant_values=np.nan).reshape( 

530 nSignalPoints*covMatrixSide**2).tolist(), 

531 'COVARIANCES_MODEL': np.pad(np.array(self.covariancesModel[ampName]), 

532 ((0, nPadPoints[ampName]), (0, 0), (0, 0)), 

533 'constant', constant_values=np.nan).reshape( 

534 nSignalPoints*covMatrixSide**2).tolist(), 

535 'COVARIANCES_SQRT_WEIGHTS': np.pad(np.array(self.covariancesSqrtWeights[ampName]), 

536 ((0, nPadPoints[ampName]), (0, 0), (0, 0)), 

537 'constant', constant_values=0.0).reshape( 

538 nSignalPoints*covMatrixSide**2).tolist(), 

539 'A_MATRIX': np.array(self.aMatrix[ampName]).reshape(covMatrixSide**2).tolist(), 

540 'B_MATRIX': np.array(self.bMatrix[ampName]).reshape(covMatrixSide**2).tolist(), 

541 'COVARIANCES_MODEL_NO_B': 

542 np.pad(np.array(self.covariancesModelNoB[ampName]), 

543 ((0, nPadPoints[ampName]), (0, 0), (0, 0)), 

544 'constant', constant_values=np.nan).reshape( 

545 nSignalPoints*covMatrixSide**2).tolist(), 

546 'A_MATRIX_NO_B': np.array(self.aMatrixNoB[ampName]).reshape( 

547 covMatrixSide**2).tolist(), 

548 'FINAL_VARS': np.pad(np.array(self.finalVars[ampName]), (0, nPadPoints[ampName]), 

549 'constant', constant_values=np.nan).tolist(), 

550 'FINAL_MODEL_VARS': np.pad(np.array(self.finalModelVars[ampName]), 

551 (0, nPadPoints[ampName]), 

552 'constant', constant_values=np.nan).tolist(), 

553 'FINAL_MEANS': np.pad(np.array(self.finalMeans[ampName]), 

554 (0, nPadPoints[ampName]), 

555 'constant', constant_values=np.nan).tolist(), 

556 'BAD_AMPS': np.array(self.badAmps).tolist() if len(self.badAmps) else np.nan, 

557 'PHOTO_CHARGE': np.array(self.photoCharge[ampName]).tolist(), 

558 } for ampName in self.ampNames]) 

559 inMeta = self.getMetadata().toDict() 

560 outMeta = {k: v for k, v in inMeta.items() if v is not None} 

561 outMeta.update({k: "" for k, v in inMeta.items() if v is None}) 

562 catalog.meta = outMeta 

563 tableList.append(catalog) 

564 

565 return(tableList) 

566 

567 def getExpIdsUsed(self, ampName): 

568 """Get the exposures used, i.e. not discarded, for a given amp. 

569 If no mask has been created yet, all exposures are returned. 

570 """ 

571 if len(self.expIdMask[ampName]) == 0: 

572 return self.inputExpIdPairs[ampName] 

573 

574 # if the mask exists it had better be the same length as the expIdPairs 

575 assert len(self.expIdMask[ampName]) == len(self.inputExpIdPairs[ampName]) 

576 

577 pairs = self.inputExpIdPairs[ampName] 

578 mask = self.expIdMask[ampName] 

579 # cast to bool required because numpy 

580 return [(exp1, exp2) for ((exp1, exp2), m) in zip(pairs, mask) if bool(m) is True] 

581 

582 def getGoodAmps(self): 

583 return [amp for amp in self.ampNames if amp not in self.badAmps]