videoflow.utils package¶
Submodules¶
videoflow.utils.downloader module¶
-
videoflow.utils.downloader.get_file(fname, origin, untar=False, md5_hash=None, file_hash=None, cache_subdir='models', hash_algorithm='auto', extract=False, archive_format='auto', cache_dir=None)¶ Copied from Keras repository.
Downloads a file from a URL if it not already in the cache. By default the file at the url origin is downloaded to the cache_dir ~/.videoflow, placed in the cache_subdir models, and given the filename fname. The final location of a file example.txt would therefore be ~/.videoflow/models/example.txt. Files in tar, tar.gz, tar.bz, and zip formats can also be extracted. Passing a hash will verify the file after download. The command line programs shasum and sha256sum can compute the hash.
- Arguments:
- fname: Name of the file. If an absolute path /path/to/file.txt is
specified the file will be saved at that location.
origin: Original URL of the file.
- untar: Deprecated in favor of ‘extract’.
boolean, whether the file should be decompressed
- md5_hash: Deprecated in favor of ‘file_hash’.
md5 hash of the file for verification
- file_hash: The expected hash string of the file after download.
The sha256 and md5 hash algorithms are both supported.
- cache_subdir: Subdirectory under the Keras cache dir where the file is
saved. If an absolute path /path/to/folder is specified the file will be saved at that location.
- hash_algorithm: Select the hash algorithm to verify the file.
options are ‘md5’, ‘sha256’, and ‘auto’. The default ‘auto’ detects the hash algorithm in use.
extract: True tries extracting the file as an Archive, like tar or zip.
- archive_format: Archive format to try for extracting the file.
Options are ‘auto’, ‘tar’, ‘zip’, and None. ‘tar’ includes tar, tar.gz, and tar.bz files. The default ‘auto’ is [‘tar’, ‘zip’]. None or an empty list will return no matches found.
- cache_dir: Location to store cached files, when None it
defaults to the [Keras Directory](/faq/#where-is-the-keras-configuration-filed-stored).
- Returns
Path to the downloaded file
-
videoflow.utils.downloader.validate_file(fpath, file_hash, algorithm='auto', chunk_size=65535)¶ Validates a file against a sha256 or md5 hash.
- Arguments:
fpath: path to the file being validated
file_hash: The expected hash string of the file. The sha256 and md5 hash algorithms are both supported.
algorithm: Hash algorithm, one of ‘auto’, ‘sha256’, or ‘md5’. The default ‘auto’ detects the hash algorithm in use.
chunk_size: Bytes to read at a time, important for large files.
- Returns:
Whether the file is valid
videoflow.utils.generic_utils module¶
-
class
videoflow.utils.generic_utils.DelayedInterrupt(signals)¶ Bases:
objectclass based on: http://stackoverflow.com/a/21919644/487556
It delays interrupts until the code exits the entered block
-
class
videoflow.utils.generic_utils.DelayedKeyboardInterrupt¶
-
class
videoflow.utils.generic_utils.Progbar(target, width=30, verbose=1, interval=0.05, stateful_metrics=None)¶ Bases:
objectDisplays a progress bar.
- Arguments
target: Total number of steps expected, None if unknown.
width: Progress bar width on screen.
verbose: Verbosity mode, 0 (silent), 1 (verbose), 2 (semi-verbose)
stateful_metrics: Iterable of string names of metrics that should not be averaged over time. Metrics in this list will be displayed as-is. All others will be averaged by the progbar before display.
interval: Minimum visual progress update interval (in seconds).
-
add(n, values=None)¶
-
update(current, values=None)¶ Updates the progress bar.
- Arguments
current: Index of current step.
values: List of tuples: (name, value_for_last_step). If name is in stateful_metrics, value_for_last_step will be displayed as-is. Else, an average of the metric over time will be displayed.
videoflow.utils.graph module¶
-
videoflow.utils.graph.flatten(items)¶ Returns flattened iterable from any nested iterable
-
videoflow.utils.graph.has_cycle(producers)¶ Used to detect if the graph is not acyclical. Returns true if it finds a cycle in the graph. It begins exploring the graph from producers down all the way to consumers.
-
videoflow.utils.graph.topological_sort(producers)¶ Creates a topological sort of the computation graph.
- Arguments:
producers: a list of producer nodes, that is, nodes with no parents.
- Returns:
stack: a list of nodes in topological order. If a node A appears before a node B on the list, it means that node A does not depend on node B output
videoflow.utils.parsers module¶
-
videoflow.utils.parsers.parse_label_map(path_to_labels: str)¶ - Arguments:
path_to_labels (str): path to pbtx file
- Returns:
dict of form { id(int) : label(str)}
videoflow.utils.system module¶
-
videoflow.utils.system.get_gpus_available_to_process() → [<class 'int'>]¶ Returns the list of ids of the gpus available to the process calling the function. It first gets the set of ids of the gpus in the system. Then it gets the set of ids marked as available by
CUDA_VISIBLE_DEVICES. It returns the intersection of those two sets as a list.
-
videoflow.utils.system.get_number_of_gpus() → int¶ Returns the number of gpus in the system
-
videoflow.utils.system.get_system_gpus() → set¶ Returns the ids of gpus in the machine as a set of integers