|
def | __init__ (self, *args, **kwargs) |
|
def | __del__ (self) |
|
def | log (self, msg, *args) |
|
def | command (self, cmd) |
|
def | map (self, context, func, dataList, *args, **kwargs) |
| Scatter work to slaves and gather the results. More...
|
|
def | reduce (self, context, reducer, func, dataList, *args, **kwargs) |
| Scatter work to slaves and reduce the results. More...
|
|
def | mapNoBalance (self, context, func, dataList, *args, **kwargs) |
| Scatter work to slaves and gather the results. More...
|
|
def | reduceNoBalance (self, context, reducer, func, dataList, *args, **kwargs) |
| Scatter work to slaves and reduce the results. More...
|
|
def | mapToPrevious (self, context, func, dataList, *args, **kwargs) |
| Scatter work to the same target as before. More...
|
|
def | reduceToPrevious (self, context, reducer, func, dataList, *args, **kwargs) |
| Reduction where work goes to the same target as before. More...
|
|
def | storeSet (self, context, **kwargs) |
| Store data on slave for a particular context. More...
|
|
def | storeDel (self, context, *nameList) |
|
def | storeClear (self, context) |
|
def | cacheClear (self, context) |
|
def | cacheList (self, context) |
|
def | storeList (self, context) |
|
def | exit (self) |
|
Master node instance of MPI process pool
Only the master node should instantiate this.
WARNING: You should not let a pool instance hang around at program
termination, as the garbage collection behaves differently, and may
cause a segmentation fault (signal 11).
Definition at line 626 of file pool.py.
def lsst.ctrl.pool.pool.PoolMaster.map |
( |
|
self, |
|
|
|
context, |
|
|
|
func, |
|
|
|
dataList, |
|
|
* |
args, |
|
|
** |
kwargs |
|
) |
| |
Scatter work to slaves and gather the results.
Work is distributed dynamically, so that slaves that finish
quickly will receive more work.
Each slave applies the function to the data they're provided.
The slaves may optionally be passed a cache instance, which
they can use to store data for subsequent executions (to ensure
subsequent data is distributed in the same pattern as before,
use the 'mapToPrevious' method). The cache also contains
data that has been stored on the slaves.
The 'func' signature should be func(cache, data, *args, **kwargs)
if 'context' is non-None; otherwise func(data, *args, **kwargs).
@param context: Namespace for cache
@param func: function for slaves to run; must be picklable
@param dataList: List of data to distribute to slaves; must be picklable
@param args: List of constant arguments
@param kwargs: Dict of constant arguments
@return list of results from applying 'func' to dataList
Definition at line 656 of file pool.py.
def lsst.ctrl.pool.pool.PoolMaster.mapNoBalance |
( |
|
self, |
|
|
|
context, |
|
|
|
func, |
|
|
|
dataList, |
|
|
* |
args, |
|
|
** |
kwargs |
|
) |
| |
Scatter work to slaves and gather the results.
Work is distributed statically, so there is no load balancing.
Each slave applies the function to the data they're provided.
The slaves may optionally be passed a cache instance, which
they can store data in for subsequent executions (to ensure
subsequent data is distributed in the same pattern as before,
use the 'mapToPrevious' method). The cache also contains
data that has been stored on the slaves.
The 'func' signature should be func(cache, data, *args, **kwargs)
if 'context' is true; otherwise func(data, *args, **kwargs).
@param context: Namespace for cache
@param func: function for slaves to run; must be picklable
@param dataList: List of data to distribute to slaves; must be picklable
@param args: List of constant arguments
@param kwargs: Dict of constant arguments
@return list of results from applying 'func' to dataList
Definition at line 763 of file pool.py.
def lsst.ctrl.pool.pool.PoolMaster.mapToPrevious |
( |
|
self, |
|
|
|
context, |
|
|
|
func, |
|
|
|
dataList, |
|
|
* |
args, |
|
|
** |
kwargs |
|
) |
| |
Scatter work to the same target as before.
Work is distributed so that each slave handles the same
indices in the dataList as when 'map' was called.
This allows the right data to go to the right cache.
It is assumed that the dataList is the same length as when it was
passed to 'map'.
The 'func' signature should be func(cache, data, *args, **kwargs).
@param context: Namespace for cache
@param func: function for slaves to run; must be picklable
@param dataList: List of data to distribute to slaves; must be picklable
@param args: List of constant arguments
@param kwargs: Dict of constant arguments
@return list of results from applying 'func' to dataList
Definition at line 885 of file pool.py.
def lsst.ctrl.pool.pool.PoolMaster.reduce |
( |
|
self, |
|
|
|
context, |
|
|
|
reducer, |
|
|
|
func, |
|
|
|
dataList, |
|
|
* |
args, |
|
|
** |
kwargs |
|
) |
| |
Scatter work to slaves and reduce the results.
Work is distributed dynamically, so that slaves that finish
quickly will receive more work.
Each slave applies the function to the data they're provided.
The slaves may optionally be passed a cache instance, which
they can use to store data for subsequent executions (to ensure
subsequent data is distributed in the same pattern as before,
use the 'mapToPrevious' method). The cache also contains
data that has been stored on the slaves.
The 'func' signature should be func(cache, data, *args, **kwargs)
if 'context' is non-None; otherwise func(data, *args, **kwargs).
The 'reducer' signature should be reducer(old, new). If the 'reducer'
is None, then we will return the full list of results
@param context: Namespace for cache
@param reducer: function for master to run to reduce slave results; or None
@param func: function for slaves to run; must be picklable
@param dataList: List of data to distribute to slaves; must be picklable
@param args: List of constant arguments
@param kwargs: Dict of constant arguments
@return reduced result (if reducer is non-None) or list of results
from applying 'func' to dataList
Definition at line 683 of file pool.py.
def lsst.ctrl.pool.pool.PoolMaster.reduceNoBalance |
( |
|
self, |
|
|
|
context, |
|
|
|
reducer, |
|
|
|
func, |
|
|
|
dataList, |
|
|
* |
args, |
|
|
** |
kwargs |
|
) |
| |
Scatter work to slaves and reduce the results.
Work is distributed statically, so there is no load balancing.
Each slave applies the function to the data they're provided.
The slaves may optionally be passed a cache instance, which
they can store data in for subsequent executions (to ensure
subsequent data is distributed in the same pattern as before,
use the 'mapToPrevious' method). The cache also contains
data that has been stored on the slaves.
The 'func' signature should be func(cache, data, *args, **kwargs)
if 'context' is true; otherwise func(data, *args, **kwargs).
The 'reducer' signature should be reducer(old, new). If the 'reducer'
is None, then we will return the full list of results
@param context: Namespace for cache
@param reducer: function for master to run to reduce slave results; or None
@param func: function for slaves to run; must be picklable
@param dataList: List of data to distribute to slaves; must be picklable
@param args: List of constant arguments
@param kwargs: Dict of constant arguments
@return reduced result (if reducer is non-None) or list of results
from applying 'func' to dataList
Definition at line 789 of file pool.py.
def lsst.ctrl.pool.pool.PoolMaster.reduceToPrevious |
( |
|
self, |
|
|
|
context, |
|
|
|
reducer, |
|
|
|
func, |
|
|
|
dataList, |
|
|
* |
args, |
|
|
** |
kwargs |
|
) |
| |
Reduction where work goes to the same target as before.
Work is distributed so that each slave handles the same
indices in the dataList as when 'map' was called.
This allows the right data to go to the right cache.
It is assumed that the dataList is the same length as when it was
passed to 'map'.
The 'func' signature should be func(cache, data, *args, **kwargs).
The 'reducer' signature should be reducer(old, new). If the 'reducer'
is None, then we will return the full list of results
@param context: Namespace for cache
@param reducer: function for master to run to reduce slave results; or None
@param func: function for slaves to run; must be picklable
@param dataList: List of data to distribute to slaves; must be picklable
@param args: List of constant arguments
@param kwargs: Dict of constant arguments
@return reduced result (if reducer is non-None) or list of results
from applying 'func' to dataList
Definition at line 908 of file pool.py.
def lsst.ctrl.pool.pool.PoolMaster.storeSet |
( |
|
self, |
|
|
|
context, |
|
|
** |
kwargs |
|
) |
| |
Store data on slave for a particular context.
The data is made available to functions through the cache. The
stored data differs from the cache in that it is identical for
all operations, whereas the cache is specific to the data being
operated upon.
@param context: namespace for store
@param kwargs: dict of name=value pairs
Reimplemented from lsst.ctrl.pool.pool.PoolNode.
Definition at line 989 of file pool.py.