An object that recursively walks a Gen2 data repository tree, extracting
Gen3 `FileDataset` objects and warning about unrecognized or unconvertable
Gen2 datasets.
Parameters
----------
inputs : `~collections.abc.Iterable` of `Target` or `Skip`
Structs that indicate dataset types to be extracted (`Target`) or
explicitly skipped (`Skip`). Skips may include a warning message to
log when matching entries are encountered.
fileIgnoreRegEx : `re.Pattern`, optional
A regular expression pattern that identifies non-dataset files that
can be ignored, to be applied at all levels of the directory tree.
dirIgnoreRegEx : `re.Pattern`, optional
A regular expression pattern that identifies non-dataset subdirectories
that can be ignored, to be applied at all levels of the directory tree.
Definition at line 51 of file walker.py.
Mapping[DatasetType, List[FileDataset]] lsst.obs.base.gen2to3.repoWalker.walker.RepoWalker.walk |
( |
|
self, |
|
|
str |
root, |
|
|
*Log |
log, |
|
|
Optional[Callable[[DataCoordinate], bool]]
|
predicate |
|
) |
| |
Walk a Gen2 repository root to extract Gen3 `FileDataset` instances
from it.
Parameters
----------
root : `str`
Absolute path to the repository root.
log : `Log`
Logger for warnings and diagnostic information.
predicate : `~collections.abc.Callable`, optional
If not `None`, a callable that returns `True` if a `DataCoordinate`
is consistent with what we want to extract. If ``predicate``
returns `False`, the file or directory that data ID was extracted
from will not be processed, even if it includes target dataset
types.
Returns
-------
datasets : `defaultdict` [`DatasetType`, `list`[`FileDataset`]]
Extracted datasets, grouped by Gen3 `DatasetType`.
Definition at line 100 of file walker.py.