Hide keyboard shortcuts

Hot-keys on this page

r m x p   toggle line displays

j k   next/prev highlighted chunk

0   (zero) top of page

1   (one) first highlighted chunk

1# 

2# LSST Data Management System 

3# Copyright 2008-2017 AURA/LSST. 

4# 

5# This product includes software developed by the 

6# LSST Project (http://www.lsst.org/). 

7# 

8# This program is free software: you can redistribute it and/or modify 

9# it under the terms of the GNU General Public License as published by 

10# the Free Software Foundation, either version 3 of the License, or 

11# (at your option) any later version. 

12# 

13# This program is distributed in the hope that it will be useful, 

14# but WITHOUT ANY WARRANTY; without even the implied warranty of 

15# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 

16# GNU General Public License for more details. 

17# 

18# You should have received a copy of the LSST License Statement and 

19# the GNU General Public License along with this program. If not, 

20# see <https://www.lsstcorp.org/LegalNotices/>. 

21# 

22""" 

23Define dataset class for MeasurePhotonTransferCurve task 

24""" 

25import numpy as np 

26from astropy.table import Table 

27 

28from lsst.ip.isr import IsrCalib 

29 

30__all__ = ['PhotonTransferCurveDataset'] 

31 

32 

33class PhotonTransferCurveDataset(IsrCalib): 

34 """A simple class to hold the output data from the PTC task. 

35 The dataset is made up of a dictionary for each item, keyed by the 

36 amplifiers' names, which much be supplied at construction time. 

37 New items cannot be added to the class to save accidentally saving to the 

38 wrong property, and the class can be frozen if desired. 

39 inputExpIdPairs records the exposures used to produce the data. 

40 When fitPtc() or fitCovariancesAstier() is run, a mask is built up, which 

41 is by definition always the same length as inputExpIdPairs, rawExpTimes, 

42 rawMeans and rawVars, and is a list of bools, which are incrementally set 

43 to False as points are discarded from the fits. 

44 PTC fit parameters for polynomials are stored in a list in ascending order 

45 of polynomial term, i.e. par[0]*x^0 + par[1]*x + par[2]*x^2 etc 

46 with the length of the list corresponding to the order of the polynomial 

47 plus one. 

48 Parameters 

49 ---------- 

50 ampNames : `list` 

51 List with the names of the amplifiers of the detector at hand. 

52 ptcFitType : `str` 

53 Type of model fitted to the PTC: "POLYNOMIAL", "EXPAPPROXIMATION", 

54 or "FULLCOVARIANCE". 

55 kwargs : `dict`, optional 

56 Other keyword arguments to pass to the parent init. 

57 Notes 

58 ----- 

59 The stored attributes are: 

60 badAmps : `list` 

61 List with bad amplifiers names. 

62 inputExpIdPairs : `dict`, [`str`, `list`] 

63 Dictionary keyed by amp names containing the input exposures IDs. 

64 expIdMask : `dict`, [`str`, `list`] 

65 Dictionary keyed by amp names containing the mask produced after 

66 outlier rejection. The mask produced by the "FULLCOVARIANCE" 

67 option may differ from the one produced in the other two PTC 

68 fit types. 

69 rawExpTimes : `dict`, [`str`, `list`] 

70 Dictionary keyed by amp names containing the unmasked exposure times. 

71 rawMeans : `dict`, [`str`, `list`] 

72 Dictionary keyed by amp namescontaining the unmasked average of the 

73 means of the exposures in each flat pair. 

74 rawVars : `dict`, [`str`, `list`] 

75 Dictionary keyed by amp names containing the variance of the 

76 difference image of the exposures in each flat pair. 

77 gain : `dict`, [`str`, `list`] 

78 Dictionary keyed by amp names containing the fitted gains. 

79 gainErr : `dict`, [`str`, `list`] 

80 Dictionary keyed by amp names containing the errors on the 

81 fitted gains. 

82 noise : `dict`, [`str`, `list`] 

83 Dictionary keyed by amp names containing the fitted noise. 

84 noiseErr : `dict`, [`str`, `list`] 

85 Dictionary keyed by amp names containing the errors on the fitted noise. 

86 ptcFitPars : `dict`, [`str`, `list`] 

87 Dictionary keyed by amp names containing the fitted parameters of the 

88 PTC model for ptcFitTye in ["POLYNOMIAL", "EXPAPPROXIMATION"]. 

89 ptcFitParsError : `dict`, [`str`, `list`] 

90 Dictionary keyed by amp names containing the errors on the fitted 

91 parameters of the PTC model for ptcFitTye in 

92 ["POLYNOMIAL", "EXPAPPROXIMATION"]. 

93 ptcFitChiSq : `dict`, [`str`, `list`] 

94 Dictionary keyed by amp names containing the reduced chi squared 

95 of the fit for ptcFitTye in ["POLYNOMIAL", "EXPAPPROXIMATION"]. 

96 covariances : `dict`, [`str`, `list`] 

97 Dictionary keyed by amp names containing a list of measured 

98 covariances per mean flux. 

99 covariancesModel : `dict`, [`str`, `list`] 

100 Dictionary keyed by amp names containinging covariances model 

101 (Eq. 20 of Astier+19) per mean flux. 

102 covariancesSqrtWeights : `dict`, [`str`, `list`] 

103 Dictionary keyed by amp names containinging sqrt. of covariances 

104 weights. 

105 aMatrix : `dict`, [`str`, `list`] 

106 Dictionary keyed by amp names containing the "a" parameters from 

107 the model in Eq. 20 of Astier+19. 

108 bMatrix : `dict`, [`str`, `list`] 

109 Dictionary keyed by amp names containing the "b" parameters from 

110 the model in Eq. 20 of Astier+19. 

111 covariancesNoB : `dict`, [`str`, `list`] 

112 Dictionary keyed by amp names containing a list of measured 

113 covariances per mean flux ('b'=0 in 

114 Astier+19). 

115 covariancesModelNoB : `dict`, [`str`, `list`] 

116 Dictionary keyed by amp names containing covariances model 

117 (with 'b'=0 in Eq. 20 of Astier+19) 

118 per mean flux. 

119 covariancesSqrtWeightsNoB : `dict`, [`str`, `list`] 

120 Dictionary keyed by amp names containing sqrt. of covariances weights 

121 ('b' = 0 in Eq. 20 of 

122 Astier+19). 

123 aMatrixNoB : `dict`, [`str`, `list`] 

124 Dictionary keyed by amp names containing the "a" parameters from the 

125 model in Eq. 20 of Astier+19 

126 (and 'b' = 0). 

127 finalVars : `dict`, [`str`, `list`] 

128 Dictionary keyed by amp names containing the masked variance of the 

129 difference image of each flat 

130 pair. If needed, each array will be right-padded with 

131 np.nan to match the length of rawExpTimes. 

132 finalModelVars : `dict`, [`str`, `list`] 

133 Dictionary keyed by amp names containing the masked modeled 

134 variance of the difference image of each flat pair. If needed, each 

135 array will be right-padded with np.nan to match the length of 

136 rawExpTimes. 

137 finalMeans : `dict`, [`str`, `list`] 

138 Dictionary keyed by amp names containing the masked average of the 

139 means of the exposures in each flat pair. If needed, each array 

140 will be right-padded with np.nan to match the length of 

141 rawExpTimes. 

142 photoCharge : `dict`, [`str`, `list`] 

143 Dictionary keyed by amp names containing the integrated photocharge 

144 for linearity calibration. 

145 

146 Returns 

147 ------- 

148 `lsst.cp.pipe.ptc.PhotonTransferCurveDataset` 

149 Output dataset from MeasurePhotonTransferCurveTask. 

150 """ 

151 

152 _OBSTYPE = 'PTC' 

153 _SCHEMA = 'Gen3 Photon Transfer Curve' 

154 _VERSION = 1.0 

155 

156 def __init__(self, ampNames=[], ptcFitType=None, **kwargs): 

157 

158 self.ptcFitType = ptcFitType 

159 self.ampNames = ampNames 

160 self.badAmps = [] 

161 

162 self.inputExpIdPairs = {ampName: [] for ampName in ampNames} 

163 self.expIdMask = {ampName: [] for ampName in ampNames} 

164 self.rawExpTimes = {ampName: [] for ampName in ampNames} 

165 self.rawMeans = {ampName: [] for ampName in ampNames} 

166 self.rawVars = {ampName: [] for ampName in ampNames} 

167 self.photoCharge = {ampName: [] for ampName in ampNames} 

168 

169 self.gain = {ampName: -1. for ampName in ampNames} 

170 self.gainErr = {ampName: -1. for ampName in ampNames} 

171 self.noise = {ampName: -1. for ampName in ampNames} 

172 self.noiseErr = {ampName: -1. for ampName in ampNames} 

173 

174 self.ptcFitPars = {ampName: [] for ampName in ampNames} 

175 self.ptcFitParsError = {ampName: [] for ampName in ampNames} 

176 self.ptcFitChiSq = {ampName: [] for ampName in ampNames} 

177 

178 self.covariances = {ampName: [] for ampName in ampNames} 

179 self.covariancesModel = {ampName: [] for ampName in ampNames} 

180 self.covariancesSqrtWeights = {ampName: [] for ampName in ampNames} 

181 self.aMatrix = {ampName: [] for ampName in ampNames} 

182 self.bMatrix = {ampName: [] for ampName in ampNames} 

183 self.covariancesNoB = {ampName: [] for ampName in ampNames} 

184 self.covariancesModelNoB = {ampName: [] for ampName in ampNames} 

185 self.covariancesSqrtWeightsNoB = {ampName: [] for ampName in ampNames} 

186 self.aMatrixNoB = {ampName: [] for ampName in ampNames} 

187 

188 self.finalVars = {ampName: [] for ampName in ampNames} 

189 self.finalModelVars = {ampName: [] for ampName in ampNames} 

190 self.finalMeans = {ampName: [] for ampName in ampNames} 

191 

192 super().__init__(**kwargs) 

193 self.requiredAttributes.update(['badAmps', 'inputExpIdPairs', 'expIdMask', 'rawExpTimes', 

194 'rawMeans', 'rawVars', 'gain', 'gainErr', 'noise', 'noiseErr', 

195 'ptcFitPars', 'ptcFitParsError', 'ptcFitChiSq', 'aMatrixNoB', 

196 'covariances', 'covariancesModel', 'covariancesSqrtWeights', 

197 'covariancesNoB', 'covariancesModelNoB', 'covariancesSqrtWeightsNoB', 

198 'aMatrix', 'bMatrix', 'finalVars', 'finalModelVars', 'finalMeans', 

199 'photoCharge']) 

200 

201 def __eq__(self, other): 

202 """Calibration equivalence 

203 """ 

204 if not isinstance(other, self.__class__): 

205 return False 

206 

207 for attr in self._requiredAttributes: 

208 attrSelf = getattr(self, attr) 

209 attrOther = getattr(other, attr) 

210 if isinstance(attrSelf, dict) and isinstance(attrOther, dict): 

211 for ampName in attrSelf: 

212 if not np.allclose(attrSelf[ampName], attrOther[ampName], equal_nan=True): 

213 return False 

214 else: 

215 if attrSelf != attrOther: 

216 return False 

217 return True 

218 

219 def updateMetadata(self, setDate=False, **kwargs): 

220 """Update calibration metadata. 

221 This calls the base class's method after ensuring the required 

222 calibration keywords will be saved. 

223 Parameters 

224 ---------- 

225 setDate : `bool`, optional 

226 Update the CALIBDATE fields in the metadata to the current 

227 time. Defaults to False. 

228 kwargs : 

229 Other keyword parameters to set in the metadata. 

230 """ 

231 kwargs['PTC_FIT_TYPE'] = self.ptcFitType 

232 

233 super().updateMetadata(setDate=setDate, **kwargs) 

234 

235 @classmethod 

236 def fromDict(cls, dictionary): 

237 """Construct a calibration from a dictionary of properties. 

238 Must be implemented by the specific calibration subclasses. 

239 Parameters 

240 ---------- 

241 dictionary : `dict` 

242 Dictionary of properties. 

243 Returns 

244 ------- 

245 calib : `lsst.ip.isr.CalibType` 

246 Constructed calibration. 

247 Raises 

248 ------ 

249 RuntimeError : 

250 Raised if the supplied dictionary is for a different 

251 calibration. 

252 """ 

253 calib = cls() 

254 if calib._OBSTYPE != dictionary['metadata']['OBSTYPE']: 

255 raise RuntimeError(f"Incorrect Photon Transfer Curve dataset supplied. " 

256 f"Expected {calib._OBSTYPE}, found {dictionary['metadata']['OBSTYPE']}") 

257 calib.setMetadata(dictionary['metadata']) 

258 calib.ptcFitType = dictionary['ptcFitType'] 

259 calib.badAmps = np.array(dictionary['badAmps'], 'str').tolist() 

260 # The cov matrices are square 

261 covMatrixSide = int(np.sqrt(len(np.array(list(dictionary['aMatrix'].values())[0]).ravel()))) 

262 # Number of final signal levels 

263 covDimensionsProduct = len(np.array(list(dictionary['covariances'].values())[0]).ravel()) 

264 nSignalPoints = int(covDimensionsProduct/(covMatrixSide*covMatrixSide)) 

265 

266 for ampName in dictionary['ampNames']: 

267 calib.ampNames.append(ampName) 

268 calib.inputExpIdPairs[ampName] = np.array(dictionary['inputExpIdPairs'][ampName]).tolist() 

269 calib.expIdMask[ampName] = np.array(dictionary['expIdMask'][ampName]).tolist() 

270 calib.rawExpTimes[ampName] = np.array(dictionary['rawExpTimes'][ampName]).tolist() 

271 calib.rawMeans[ampName] = np.array(dictionary['rawMeans'][ampName]).tolist() 

272 calib.rawVars[ampName] = np.array(dictionary['rawVars'][ampName]).tolist() 

273 calib.gain[ampName] = np.array(dictionary['gain'][ampName]).tolist() 

274 calib.gainErr[ampName] = np.array(dictionary['gainErr'][ampName]).tolist() 

275 calib.noise[ampName] = np.array(dictionary['noise'][ampName]).tolist() 

276 calib.noiseErr[ampName] = np.array(dictionary['noiseErr'][ampName]).tolist() 

277 calib.ptcFitPars[ampName] = np.array(dictionary['ptcFitPars'][ampName]).tolist() 

278 calib.ptcFitParsError[ampName] = np.array(dictionary['ptcFitParsError'][ampName]).tolist() 

279 calib.ptcFitChiSq[ampName] = np.array(dictionary['ptcFitChiSq'][ampName]).tolist() 

280 calib.covariances[ampName] = np.array(dictionary['covariances'][ampName]).reshape( 

281 (nSignalPoints, covMatrixSide, covMatrixSide)).tolist() 

282 calib.covariancesModel[ampName] = np.array( 

283 dictionary['covariancesModel'][ampName]).reshape( 

284 (nSignalPoints, covMatrixSide, covMatrixSide)).tolist() 

285 calib.covariancesSqrtWeights[ampName] = np.array( 

286 dictionary['covariancesSqrtWeights'][ampName]).reshape( 

287 (nSignalPoints, covMatrixSide, covMatrixSide)).tolist() 

288 calib.aMatrix[ampName] = np.array(dictionary['aMatrix'][ampName]).reshape( 

289 (covMatrixSide, covMatrixSide)).tolist() 

290 calib.bMatrix[ampName] = np.array(dictionary['bMatrix'][ampName]).reshape( 

291 (covMatrixSide, covMatrixSide)).tolist() 

292 calib.covariancesNoB[ampName] = np.array( 

293 dictionary['covariancesNoB'][ampName]).reshape( 

294 (nSignalPoints, covMatrixSide, covMatrixSide)).tolist() 

295 calib.covariancesModelNoB[ampName] = np.array( 

296 dictionary['covariancesModelNoB'][ampName]).reshape( 

297 (nSignalPoints, covMatrixSide, covMatrixSide)).tolist() 

298 calib.covariancesSqrtWeightsNoB[ampName] = np.array( 

299 dictionary['covariancesSqrtWeightsNoB'][ampName]).reshape( 

300 (nSignalPoints, covMatrixSide, covMatrixSide)).tolist() 

301 calib.aMatrixNoB[ampName] = np.array( 

302 dictionary['aMatrixNoB'][ampName]).reshape((covMatrixSide, covMatrixSide)).tolist() 

303 calib.finalVars[ampName] = np.array(dictionary['finalVars'][ampName]).tolist() 

304 calib.finalModelVars[ampName] = np.array(dictionary['finalModelVars'][ampName]).tolist() 

305 calib.finalMeans[ampName] = np.array(dictionary['finalMeans'][ampName]).tolist() 

306 calib.photoCharge[ampName] = np.array(dictionary['photoCharge'][ampName]).tolist() 

307 calib.updateMetadata() 

308 return calib 

309 

310 def toDict(self): 

311 """Return a dictionary containing the calibration properties. 

312 The dictionary should be able to be round-tripped through 

313 `fromDict`. 

314 Returns 

315 ------- 

316 dictionary : `dict` 

317 Dictionary of properties. 

318 """ 

319 self.updateMetadata() 

320 

321 outDict = dict() 

322 metadata = self.getMetadata() 

323 outDict['metadata'] = metadata 

324 

325 outDict['ptcFitType'] = self.ptcFitType 

326 outDict['ampNames'] = self.ampNames 

327 outDict['badAmps'] = self.badAmps 

328 outDict['inputExpIdPairs'] = self.inputExpIdPairs 

329 outDict['expIdMask'] = self.expIdMask 

330 outDict['rawExpTimes'] = self.rawExpTimes 

331 outDict['rawMeans'] = self.rawMeans 

332 outDict['rawVars'] = self.rawVars 

333 outDict['gain'] = self.gain 

334 outDict['gainErr'] = self.gainErr 

335 outDict['noise'] = self.noise 

336 outDict['noiseErr'] = self.noiseErr 

337 outDict['ptcFitPars'] = self.ptcFitPars 

338 outDict['ptcFitParsError'] = self.ptcFitParsError 

339 outDict['ptcFitChiSq'] = self.ptcFitChiSq 

340 outDict['covariances'] = self.covariances 

341 outDict['covariancesModel'] = self.covariancesModel 

342 outDict['covariancesSqrtWeights'] = self.covariancesSqrtWeights 

343 outDict['aMatrix'] = self.aMatrix 

344 outDict['bMatrix'] = self.bMatrix 

345 outDict['covariancesNoB'] = self.covariancesNoB 

346 outDict['covariancesModelNoB'] = self.covariancesModelNoB 

347 outDict['covariancesSqrtWeightsNoB'] = self.covariancesSqrtWeightsNoB 

348 outDict['aMatrixNoB'] = self.aMatrixNoB 

349 outDict['finalVars'] = self.finalVars 

350 outDict['finalModelVars'] = self.finalModelVars 

351 outDict['finalMeans'] = self.finalMeans 

352 outDict['photoCharge'] = self.photoCharge 

353 

354 return outDict 

355 

356 @classmethod 

357 def fromTable(cls, tableList): 

358 """Construct calibration from a list of tables. 

359 This method uses the `fromDict` method to create the 

360 calibration, after constructing an appropriate dictionary from 

361 the input tables. 

362 Parameters 

363 ---------- 

364 tableList : `list` [`lsst.afw.table.Table`] 

365 List of tables to use to construct the datasetPtc. 

366 Returns 

367 ------- 

368 calib : `lsst.cp.pipe.` 

369 The calibration defined in the tables. 

370 """ 

371 ptcTable = tableList[0] 

372 

373 metadata = ptcTable.meta 

374 inDict = dict() 

375 inDict['metadata'] = metadata 

376 inDict['ampNames'] = [] 

377 inDict['ptcFitType'] = [] 

378 inDict['inputExpIdPairs'] = dict() 

379 inDict['expIdMask'] = dict() 

380 inDict['rawExpTimes'] = dict() 

381 inDict['rawMeans'] = dict() 

382 inDict['rawVars'] = dict() 

383 inDict['gain'] = dict() 

384 inDict['gainErr'] = dict() 

385 inDict['noise'] = dict() 

386 inDict['noiseErr'] = dict() 

387 inDict['ptcFitPars'] = dict() 

388 inDict['ptcFitParsError'] = dict() 

389 inDict['ptcFitChiSq'] = dict() 

390 inDict['covariances'] = dict() 

391 inDict['covariancesModel'] = dict() 

392 inDict['covariancesSqrtWeights'] = dict() 

393 inDict['aMatrix'] = dict() 

394 inDict['bMatrix'] = dict() 

395 inDict['covariancesNoB'] = dict() 

396 inDict['covariancesModelNoB'] = dict() 

397 inDict['covariancesSqrtWeightsNoB'] = dict() 

398 inDict['aMatrixNoB'] = dict() 

399 inDict['finalVars'] = dict() 

400 inDict['finalModelVars'] = dict() 

401 inDict['finalMeans'] = dict() 

402 inDict['badAmps'] = [] 

403 inDict['photoCharge'] = dict() 

404 

405 for record in ptcTable: 

406 ampName = record['AMPLIFIER_NAME'] 

407 

408 inDict['ptcFitType'] = record['PTC_FIT_TYPE'] 

409 inDict['ampNames'].append(ampName) 

410 inDict['inputExpIdPairs'][ampName] = record['INPUT_EXP_ID_PAIRS'] 

411 inDict['expIdMask'][ampName] = record['EXP_ID_MASK'] 

412 inDict['rawExpTimes'][ampName] = record['RAW_EXP_TIMES'] 

413 inDict['rawMeans'][ampName] = record['RAW_MEANS'] 

414 inDict['rawVars'][ampName] = record['RAW_VARS'] 

415 inDict['gain'][ampName] = record['GAIN'] 

416 inDict['gainErr'][ampName] = record['GAIN_ERR'] 

417 inDict['noise'][ampName] = record['NOISE'] 

418 inDict['noiseErr'][ampName] = record['NOISE_ERR'] 

419 inDict['ptcFitPars'][ampName] = record['PTC_FIT_PARS'] 

420 inDict['ptcFitParsError'][ampName] = record['PTC_FIT_PARS_ERROR'] 

421 inDict['ptcFitChiSq'][ampName] = record['PTC_FIT_CHI_SQ'] 

422 inDict['covariances'][ampName] = record['COVARIANCES'] 

423 inDict['covariancesModel'][ampName] = record['COVARIANCES_MODEL'] 

424 inDict['covariancesSqrtWeights'][ampName] = record['COVARIANCES_SQRT_WEIGHTS'] 

425 inDict['aMatrix'][ampName] = record['A_MATRIX'] 

426 inDict['bMatrix'][ampName] = record['B_MATRIX'] 

427 inDict['covariancesNoB'][ampName] = record['COVARIANCES_NO_B'] 

428 inDict['covariancesModelNoB'][ampName] = record['COVARIANCES_MODEL_NO_B'] 

429 inDict['covariancesSqrtWeightsNoB'][ampName] = record['COVARIANCES_SQRT_WEIGHTS_NO_B'] 

430 inDict['aMatrixNoB'][ampName] = record['A_MATRIX_NO_B'] 

431 inDict['finalVars'][ampName] = record['FINAL_VARS'] 

432 inDict['finalModelVars'][ampName] = record['FINAL_MODEL_VARS'] 

433 inDict['finalMeans'][ampName] = record['FINAL_MEANS'] 

434 inDict['badAmps'] = record['BAD_AMPS'] 

435 inDict['photoCharge'][ampName] = record['PHOTO_CHARGE'] 

436 return cls().fromDict(inDict) 

437 

438 def toTable(self): 

439 """Construct a list of tables containing the information in this 

440 calibration. 

441 

442 The list of tables should create an identical calibration 

443 after being passed to this class's fromTable method. 

444 Returns 

445 ------- 

446 tableList : `list` [`astropy.table.Table`] 

447 List of tables containing the linearity calibration 

448 information. 

449 """ 

450 tableList = [] 

451 self.updateMetadata() 

452 nPoints = [] 

453 for i, ampName in enumerate(self.ampNames): 

454 nPoints.append(len(list(self.covariances.values())[i])) 

455 nSignalPoints = max(nPoints) 

456 nPadPoints = {} 

457 for i, ampName in enumerate(self.ampNames): 

458 nPadPoints[ampName] = nSignalPoints - len(list(self.covariances.values())[i]) 

459 covMatrixSide = np.array(list(self.aMatrix.values())[0]).shape[0] 

460 catalog = Table([{'AMPLIFIER_NAME': ampName, 

461 'PTC_FIT_TYPE': self.ptcFitType, 

462 'INPUT_EXP_ID_PAIRS': self.inputExpIdPairs[ampName], 

463 'EXP_ID_MASK': np.pad(np.array(self.expIdMask[ampName], dtype=bool), 

464 (0, nPadPoints[ampName]), 'constant', 

465 constant_values=False).tolist(), 

466 'RAW_EXP_TIMES': np.pad(np.array(self.rawExpTimes[ampName]), 

467 (0, nPadPoints[ampName]), 'constant', 

468 constant_values=np.nan).tolist(), 

469 'RAW_MEANS': np.pad(np.array(self.rawMeans[ampName]), 

470 (0, nPadPoints[ampName]), 

471 'constant', constant_values=np.nan).tolist(), 

472 'RAW_VARS': np.pad(np.array(self.rawVars[ampName]), 

473 (0, nPadPoints[ampName]), 

474 'constant', constant_values=np.nan).tolist(), 

475 'GAIN': self.gain[ampName], 

476 'GAIN_ERR': self.gainErr[ampName], 

477 'NOISE': self.noise[ampName], 

478 'NOISE_ERR': self.noiseErr[ampName], 

479 'PTC_FIT_PARS': np.array(self.ptcFitPars[ampName]).tolist(), 

480 'PTC_FIT_PARS_ERROR': np.array(self.ptcFitParsError[ampName]).tolist(), 

481 'PTC_FIT_CHI_SQ': self.ptcFitChiSq[ampName], 

482 'COVARIANCES': np.pad(np.array(self.covariances[ampName]), 

483 ((0, nPadPoints[ampName]), (0, 0), (0, 0)), 

484 'constant', constant_values=np.nan).reshape( 

485 nSignalPoints*covMatrixSide**2).tolist(), 

486 'COVARIANCES_MODEL': np.pad(np.array(self.covariancesModel[ampName]), 

487 ((0, nPadPoints[ampName]), (0, 0), (0, 0)), 

488 'constant', constant_values=np.nan).reshape( 

489 nSignalPoints*covMatrixSide**2).tolist(), 

490 'COVARIANCES_SQRT_WEIGHTS': np.pad(np.array(self.covariancesSqrtWeights[ampName]), 

491 ((0, nPadPoints[ampName]), (0, 0), (0, 0)), 

492 'constant', constant_values=0.0).reshape( 

493 nSignalPoints*covMatrixSide**2).tolist(), 

494 'A_MATRIX': np.array(self.aMatrix[ampName]).reshape(covMatrixSide**2).tolist(), 

495 'B_MATRIX': np.array(self.bMatrix[ampName]).reshape(covMatrixSide**2).tolist(), 

496 'COVARIANCES_NO_B': np.pad(np.array(self.covariancesNoB[ampName]), 

497 ((0, nPadPoints[ampName]), (0, 0), (0, 0)), 

498 'constant', constant_values=np.nan).reshape( 

499 nSignalPoints*covMatrixSide**2).tolist(), 

500 'COVARIANCES_MODEL_NO_B': 

501 np.pad(np.array(self.covariancesModelNoB[ampName]), 

502 ((0, nPadPoints[ampName]), (0, 0), (0, 0)), 

503 'constant', constant_values=np.nan).reshape( 

504 nSignalPoints*covMatrixSide**2).tolist(), 

505 'COVARIANCES_SQRT_WEIGHTS_NO_B': 

506 np.pad(np.array(self.covariancesSqrtWeightsNoB[ampName]), 

507 ((0, nPadPoints[ampName]), (0, 0), (0, 0)), 

508 'constant', constant_values=0.0).reshape( 

509 nSignalPoints*covMatrixSide**2).tolist(), 

510 'A_MATRIX_NO_B': np.array(self.aMatrixNoB[ampName]).reshape( 

511 covMatrixSide**2).tolist(), 

512 'FINAL_VARS': np.pad(np.array(self.finalVars[ampName]), (0, nPadPoints[ampName]), 

513 'constant', constant_values=np.nan).tolist(), 

514 'FINAL_MODEL_VARS': np.pad(np.array(self.finalModelVars[ampName]), 

515 (0, nPadPoints[ampName]), 

516 'constant', constant_values=np.nan).tolist(), 

517 'FINAL_MEANS': np.pad(np.array(self.finalMeans[ampName]), 

518 (0, nPadPoints[ampName]), 

519 'constant', constant_values=np.nan).tolist(), 

520 'BAD_AMPS': np.array(self.badAmps).tolist() if len(self.badAmps) else np.nan, 

521 'PHOTO_CHARGE': np.array(self.photoCharge[ampName]).tolist(), 

522 } for ampName in self.ampNames]) 

523 inMeta = self.getMetadata().toDict() 

524 outMeta = {k: v for k, v in inMeta.items() if v is not None} 

525 outMeta.update({k: "" for k, v in inMeta.items() if v is None}) 

526 catalog.meta = outMeta 

527 tableList.append(catalog) 

528 

529 return(tableList) 

530 

531 def getExpIdsUsed(self, ampName): 

532 """Get the exposures used, i.e. not discarded, for a given amp. 

533 If no mask has been created yet, all exposures are returned. 

534 """ 

535 if len(self.expIdMask[ampName]) == 0: 

536 return self.inputExpIdPairs[ampName] 

537 

538 # if the mask exists it had better be the same length as the expIdPairs 

539 assert len(self.expIdMask[ampName]) == len(self.inputExpIdPairs[ampName]) 

540 

541 pairs = self.inputExpIdPairs[ampName] 

542 mask = self.expIdMask[ampName] 

543 # cast to bool required because numpy 

544 return [(exp1, exp2) for ((exp1, exp2), m) in zip(pairs, mask) if bool(m) is True] 

545 

546 def getGoodAmps(self): 

547 return [amp for amp in self.ampNames if amp not in self.badAmps]