Hide keyboard shortcuts

Hot-keys on this page

r m x p   toggle line displays

j k   next/prev highlighted chunk

0   (zero) top of page

1   (one) first highlighted chunk

1# This file is part of ctrl_mpexec. 

2# 

3# Developed for the LSST Data Management System. 

4# This product includes software developed by the LSST Project 

5# (http://www.lsst.org). 

6# See the COPYRIGHT file at the top-level directory of this distribution 

7# for details of code ownership. 

8# 

9# This program is free software: you can redistribute it and/or modify 

10# it under the terms of the GNU General Public License as published by 

11# the Free Software Foundation, either version 3 of the License, or 

12# (at your option) any later version. 

13# 

14# This program is distributed in the hope that it will be useful, 

15# but WITHOUT ANY WARRANTY; without even the implied warranty of 

16# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 

17# GNU General Public License for more details. 

18# 

19# You should have received a copy of the GNU General Public License 

20# along with this program. If not, see <http://www.gnu.org/licenses/>. 

21 

22__all__ = ['ExecutionGraphFixup'] 

23 

24from abc import ABC, abstractmethod 

25from typing import Iterable 

26 

27from lsst.pipe.base import QuantumIterData 

28 

29 

30class ExecutionGraphFixup(ABC): 

31 """Interface for classes which update quantum graphs before execution. 

32 

33 Primary goal of this class is to modify quanta dependencies which may 

34 not be possible to reflect in a quantum graph using standard tools. 

35 One known use case for that is to guarantee particular execution order 

36 of visits in CI jobs for cases when outcome depends on the processing 

37 order of visits (e.g. AP association pipeline). 

38 

39 Instances of this class receive pre-ordered sequence of quanta 

40 (`~lsst.pipe.base.QuantumIterData` instances) and they are allowed to 

41 modify quanta data in place, for example update ``dependencies`` field to 

42 add additional dependencies. Returned list of quanta will be re-ordered 

43 once again by the graph executor to reflect new dependencies. 

44 """ 

45 

46 @abstractmethod 

47 def fixupQuanta(self, quanta: Iterable[QuantumIterData]) -> Iterable[QuantumIterData]: 

48 """Update quanta in a graph. 

49 

50 Potentially anything in the graph could be changed if it does not 

51 break executor assumptions. Returned quanta will be re-ordered by 

52 executor, if modifications result in a dependency cycle the executor 

53 will raise an exception. 

54 

55 Parameters 

56 ---------- 

57 quanta : iterable [`~lsst.pipe.base.QuantumIterData`] 

58 Iterable of topologically ordered quanta as returned from 

59 `lsst.pipe.base.QuantumGraph.traverse` method. 

60 

61 Yieds 

62 ----- 

63 quantum : `~lsst.pipe.base.QuantumIterData` 

64 """ 

65 raise NotImplementedError