bempp.api.utils.pool

Routines to administrate a process pool.

Module Contents

Classes

Pool	A simple pool.

Functions

worker(in_queue, out_queue, worker_id, nworkers, buf, ...)	Definition of a worker.
as_array(dtype, offset, shape)	Return part of the buffer as array.
to_buffer(*args)	Save a number of numpy arrays to a buffer.
from_buffer(arrays)	Retrieve arrays from buffer.
_raise_if_not_worker(name)	Raise exception if not in worker.
number_of_workers()	Return number of workers.
insert_data(key, data)	Insert data.
is_initialised()	Return true if pool is initialised.
clear_data()	Clear the data in all workers.
get_data(key)	Return data.
get_id()	Return my id.
remove_key(key)	Remove data from pool.
has_key(key)	Return if key exists in pool data.
is_worker()	Return true if called from worker process.
create_device_pool(identifier[, buffer_size, log, ...])	Create a pool based on a given platform identifer.
create_pool(nworkers[, buffer_size, log, log_level])	Create a pool.
nworkers()	Get number of workers.
map(fun, args)	Corresponds to multiprocessing map.
starmap(fun, args)	Corresponds to multiprocessing map.
execute(fun, *args)	Execute function with the same arguments on all workers.
_assign_ids(nworkers)	Assign pool ids.
shutdown()	Shutdown the pool.
_execute_function_without_arguments(fun)	Execute function without arguments.
_remove_key_worker(key)
_init_device_worker(identifier, precision)	Worker to initialise device.
_clear_data_worker()	Clear worker.

Attributes

_DATA
_MY_ID
_USE_THREADS
_POOL
_NWORKERS
_IN_WORKER
_BUFFER

bempp.api.utils.pool._DATA

bempp.api.utils.pool._MY_ID

bempp.api.utils.pool._USE_THREADS

bempp.api.utils.pool._POOL

bempp.api.utils.pool._NWORKERS

bempp.api.utils.pool._IN_WORKER = False

bempp.api.utils.pool._BUFFER

bempp.api.utils.pool.worker(in_queue, out_queue, worker_id, nworkers, buf, log, log_level)

Definition of a worker.

bempp.api.utils.pool.as_array(dtype, offset, shape)

Return part of the buffer as array.

Parameters

dtypeNumpy dtype object

The type of the array

offsetint

Start index in buffer

nitemsint

Number of items of type dtype

bempp.api.utils.pool.to_buffer(*args)

Save a number of numpy arrays to a buffer.

bempp.api.utils.pool.from_buffer(arrays)

Retrieve arrays from buffer.

arrays is a list of tuples [(dtype1, shape1), (dtype2, shape2), …], where dtype is the type of the array and shape is a shape tuple.

class bempp.api.utils.pool.Pool(nworkers, buffer_size=100, log=False, log_level='info')

Bases: object

A simple pool.

This pool is different from the multiprocessing pool in that map operations are guaranteed to execute on all processes, and we can directly target individual processes.

property number_of_workers

Return number of workers.

_map_impl(fun, args, options)

Map implementation.

map(fun, args=None)

Map function onto workers.

starmap(fun, args)

Map function onto workers.

shutdown()

Shutdown all workers.

bempp.api.utils.pool._raise_if_not_worker(name)

Raise exception if not in worker.

bempp.api.utils.pool.number_of_workers()

Return number of workers.

bempp.api.utils.pool.insert_data(key, data)

Insert data.

bempp.api.utils.pool.is_initialised()

Return true if pool is initialised.

bempp.api.utils.pool.clear_data()

Clear the data in all workers.

bempp.api.utils.pool.get_data(key)

Return data.

bempp.api.utils.pool.get_id()

Return my id.

bempp.api.utils.pool.remove_key(key)

Remove data from pool.

bempp.api.utils.pool.has_key(key)

Return if key exists in pool data.

bempp.api.utils.pool.is_worker()

Return true if called from worker process.

bempp.api.utils.pool.create_device_pool(identifier, buffer_size=100, log=False, log_level='info', max_workers=-1, precision=None)

Create a pool based on a given platform identifer.

identifierstring

A unique identifier that is part of the platform name. Used to find the correct platform.

buffer_sizeint

Shared memory buffer size in MB

logBoolean

Set to True to log workers.

log_levelString

Logging level. One of ‘debug’, ‘info’, ‘warning’, ‘error’, ‘critical’

max_workersint

Maximum number of workers. If max_workers=-1 (default) the maximum number of workers is identical to the number of devices in the pool.

precisionstring or None

Precision for the devices in the pool. If precision is None use single precision for GPU devices and double precision for CPU devices. If precision is ‘single’ or ‘double’ use the corresponding mode for the pool.

bempp.api.utils.pool.create_pool(nworkers, buffer_size=100, log=False, log_level='info')

Create a pool.

bempp.api.utils.pool.nworkers()

Get number of workers.

bempp.api.utils.pool.map(fun, args)

Corresponds to multiprocessing map.

bempp.api.utils.pool.starmap(fun, args)

Corresponds to multiprocessing map.

bempp.api.utils.pool.execute(fun, *args)

Execute function with the same arguments on all workers.

bempp.api.utils.pool._assign_ids(nworkers)

Assign pool ids.

bempp.api.utils.pool.shutdown()

Shutdown the pool.

bempp.api.utils.pool._execute_function_without_arguments(fun)

Execute function without arguments.

bempp.api.utils.pool._remove_key_worker(key)

bempp.api.utils.pool._init_device_worker(identifier, precision)

Worker to initialise device.

bempp.api.utils.pool._clear_data_worker()

Clear worker.