NAME
DESCRIPTION
split_data
Splits an NDArray into `num_slice` slices along `batch_axis`.
Usually used for data parallelism where each slices is sent
to one device (i.e. GPU).
Parameters
----------
$data : NDArray
A batch of data.
$num_slice : int
Number of desired slices.
$batch_axis=0 : int, default 0
The axis along which to slice.
:$even_split=1 : bool, default True
Whether to force all slices to have the same number of elements.
If `True`, an error will be raised when `num_slice` does not evenly
divide `data.shape[batch_axis]`.
Returns
-------
array ref of NDArray
Return value is a array ref even if `num_slice` is 1.
|
split_and_load
Splits an NDArray into `len(ctx_list)` slices along `batch_axis` and loads
each slice to one context in `ctx_list`.
Parameters
----------
$data : AcceptableInput
A batch of data.
:$ctx_list : list of Context
A list of Contexts.
:$batch_axis : int, default 0
The axis along which to slice.
:$even_split : bool, default True
Whether to force all slices to have the same number of elements.
Returns
-------
list of NDArray
Each corresponds to a context in `ctx_list`.
|
clip_global_norm
Rescales NDArrays so that the sum of their 2-norm is smaller than `max_norm`.
|
check_sha1
Check whether the sha1 hash of the file content matches the expected hash.
Parameters
----------
filename : str
Path to the file.
sha1_hash : str
Expected sha1 hash in hexadecimal digits.
Returns
-------
bool
Whether the file content matches the expected hash.
|
download
Download an given URL
Parameters
----------
$url : str
URL to download
:$path : str, optional
Destination path to store downloaded file. By default stores to the
current directory with same name as in url.
:$overwrite : bool, optional
Whether to overwrite destination file if already exists.
:$sha1_hash : str, optional
Expected sha1 hash in hexadecimal digits. Will ignore existing file when hash is specified
but doesn't match.
Returns
-------
str
The file path of the downloaded file.
|