util package

Submodules

util.cpu module

CPU-related utility functions

util.cpu.compute_cpu_shares(shares_list, num_cpus)[source]

Computes lists of non-overlapping CPUs lists based on a list of CPU shares. A CPU share is a value between 1 and 99 and corresponds to the percentage of total system CPUs that should be allocated for a process. When a share of 0 or 100 is specified, a list with all system CPUs is returned.

Parameters:shares_list (list<int>) – values with the CPU percentages for each process
Returns:a tuple with the CPU lists for each process
Raises:ValueError – When input leads to an invalid CPU sharing.
Return type:tuple<list<int>>

util.customsubprocess module

Extended subprocess functionality

util.customsubprocess.check_output_streaming(cmd, prefix='', queue=None, block_flag=True)[source]

Redirect output to stderr, printing it whenever a new line is detected (bufsize=1 denotes line buffered output). This can be considered as “realtime” printing. Additionally, prints output by prepending it with a user-defined prefix. Furthermore, it optionally forwards output to a queue in order e.g. to be processed by a separate thread in real-time.

Parameters:
  • cmd (list<str>) – the command line list.
  • prefix (str) – the user defined prefix for output.
  • queue (multiprocessing.Queue) – queue to forward console output to.
  • block_flag (bool) – defines if the execution will be performed in block mode
Returns:

return exit status of command.

Raises:

subprocess.CalledProcessError – If the exit status of the executed command is not 0.

Return type:

int

util.file_ops module

file handling utilities

util.file_ops.check_filelist(file_list)[source]

Takes a list of files and checks if those files exist and if those files are executables (have execution right in their privileges). If something is wrong then an exception is raised

Parameters:file_list (list<str>) – A list of files
util.file_ops.check_files_executables(file_list)[source]

Checks if all files in a list have exe permissions.

Parameters:file_list (list<str>) – list of filenames to check
Raises:Exception – When file in file_list does not exist or is not executable.
Returns:list<str>
Return type:list<str>
util.file_ops.check_files_exist(file_list)[source]

Checks if all files in a list exist.

Parameters:file_list (list<str>) – list of filenames to check
Returns:list<str>
Return type:list<str>
util.file_ops.file_exists(fpath)[source]

Checks if file exists in filesystem.

Parameters:fpath (str) – file path to check
Returns:bool
Return type:bool
util.file_ops.is_file_exe(fpath)[source]

Checks if a file is executable.

Parameters:fpath (str) – file path to check
Returns:bool
Return type:bool
util.file_ops.make_file_exe(fpath)[source]

Gives executable rights to a file.

Parameters:fpath (str) – file path to handle
util.file_ops.merge_dict_and_avg(results_add, results_delete)[source]

Takes two dictionaries results_add, results_delete and returns a merged dictionary. Special purpose method for the NSTAT NorthBound stress test.

Parameters:
  • results_add (dict) – Includes the add functionality results
  • results_delete (dict) – Includes the delete functionality results
Returns:

the exit code of the command to be executed remotely and the combined stdout - stderr of the executed command

Return type:

dict

util.html module

HTML generation utilities

util.html.generate_html_foot(foot_content='')[source]

Returns the ending and the contents at the end of html page.

Returns:HTML code of the page ending.
Return type:str
util.html.generate_html_head(css_style='', javascript_functions='', head_content='')[source]

Creates the head of an html file gets as input the styling of the page as a css string parameter and returns a string value, which is the html code of the header.

Parameters:
  • css_style (str) – the css styling text of the html page we produce. this parameter must be given as a multiline string.
  • parameters_dictionary (str) – Parameters of the specific testing scenario
  • javascript_functions (str) – JavaScript functions for report. This parameter must be given as a multiline string.
  • head_content (str) – this is the html code that will appear at the top of the page, just after the <body> tag. We can place here the Title of the page and everything else we want to appear at the top.
Returns:

str (the html code of the Header of the html page.)

Return type:

str

util.html.get_ordered_index(field_value, map_dictionary)[source]

Returns the index of a key in map_dictionary, which is defined by the field_value. If no match is found None is returned

Parameters:
  • field_value (str) – the value of the key we want to search for, inside the map_dictionary. This is a helper function called inside the single_dict_to_html() and multi_dict_to_html() to avoid repetitions of the same code blocks in these functions
  • map_dictionary (collections.OrderedDict) – An instance of OrderedDict which has mappings between keys and their values as they appear inside an html generated document.
Returns:

The index where the field_value should be placed, depending on the ordering inside the map_dictionary

Return type:

int

util.html.isalistofdictionaries(lst)[source]

Takes a list as argument and checks if all the elements of this list are dictionaries. If they are then it returns true, else it returns false. If the input argument type is not list then it also returns false.

Parameters:lst (list<dict>) – The list that we want to check its elements
Returns:True if all the elements of the list are dictionaries. False otherwise
Return type:bool
util.html.multi_dict_to_html(data, table_title='', map_dictionary=None, row_ordering_key=None)[source]

Generates html tables according to the input it gets from the input arguments.

Parameters:
  • data (list<dict>) – This datastructure is a list that contains dictionaries of strings that contains the pair of field_name:value of all the data we want to convert to HTML table.
  • table_title (str) – An optional argument that contains the title of the table we want to generate.
  • map_dictionary (controller.OrderedDict) – data structure that contains the pair of field_name:value. It defines which fields will be kept from the data input dictionary and how their value will be translated in the HTML table.
Returns:

str

Return type:

str

util.html.single_dict_table_data(data_values, td_style=None, td_class=None)[source]

Returns a <td> ... </td> html element for the single_dict_to_html() type of tables, data columns. In case that the input data_values is a list of dictionaries, this function calls recursively single_dict_to_html() and generates sub tables inside the <td> ... </td> html element.

Parameters:
  • data_values (str or list<dict>) – the data we want to place into a <td> ... </td> html element.
  • td_style – the styling code for the content of <td>...</td> html element.
  • cls (str) – the class name attribute of the <td>...</td> html element
Returns:

html code of a <td> </td> element for single_dict_to_html() tables

Return type:

str

util.html.single_dict_to_html(data, key_title, data_title, table_title='', map_dictionary=None)[source]

Generates html tables according to the input it gets from the input arguments. The table generated is by row, which means that on each row the first element is the key value of the dictionary, the second element is the data value of the dictionary for the correspondent key. The columns are always 2 and their title are defined in the input arguments

Parameters:
  • data (dict) – this data structure is a dictionary of strings that contains the pair of field_name:value for all the data we want to convert to HTML table.
  • key_title (str) – the title of the column, under which the dictionary key values will be placed.
  • data_title (str) – the title of the column, under which the dictionary data values will be placed.
  • table_title (str) – Optional argument. Contains the title of the table we want to generate
  • map_dictionary (collections.OrderedDict) – This data structure contains the pair of field_name:value. It defines which fields should be kept from the data input dictionary and how their value will be translated in the HTML table.
Returns:

str

Return type:

str

util.netutil module

General network utilities

util.netutil.copy_dir_local_to_remote(ip, ssh_port, username, password, local_path, remote_path)[source]

Copy a local directory on a remote machine.

Parameters:
  • connection (namedtuple<>) – A named tuple with all the connection information. It must have the following elements: [‘name’, ‘ip’, ‘ssh_port’, ‘username’, ‘password’] :param local_path: directory path from local machine to be copied, full location required
  • remote_path (str) – directory path on the remote node, full location required
util.netutil.copy_dir_remote_to_local(ip, ssh_port, username, password, remote_path, local_path)[source]

Copy recursively remote directories (Copies all files and other sub-directories).

Parameters:
  • connection (namedtuple<>) – named tuple with connection information: [‘name’, ‘ip’, ‘ssh_port’, ‘username’, ‘password’]
  • remote_path (str) – full remote path we want to copy
  • local_path (str) – full local path we want to copy
util.netutil.create_dir_remote(ip, ssh_port, username, password, remote_path)[source]

Opens an ssh connection to a remote machine and creates a new directory.

Parameters:
  • connection (namedtuple<>) – named tuple with connection information: [‘name’, ‘ip’, ‘ssh_port’, ‘username’, ‘password’]
  • remote_path (str) –
util.netutil.isdir(path, sftp)[source]

Checks if a given remote path is a directory

Parameters:
  • path (str) – A string with the full path we want to check
  • sftp (paramiko.SFTPClient) – An sftp connection object (paramiko)
Returns:

True if the given path is a directory false otherwise.

Return type:

bool

util.netutil.isfile(ip, port, username, password, file_list)[source]

Checks if all files in a given list exist. All files are located remotely

Parameters:
  • ip (str) – ip address of the remote host
  • port (int) – port number of the remote host
  • username (str) – username of the remote host
  • password (str) – password of the remote host
  • file_list (list<str>) – list of filenames to be checked
Returns:

True if the given path is a directory false otherwise.

Return type:

bool

util.netutil.make_remote_file_executable(ip, port, username, password, remote_file)[source]

Makes the remote file executable.

Parameters:
  • connection (namedtuple<>) – named tuple with connection information: [‘name’, ‘ip’, ‘ssh_port’, ‘username’, ‘password’]
  • remote_file (str) – remote file to make executable
util.netutil.remove_remote_directory(ip, ssh_port, username, password, path)[source]

Removes recursively remote directories (removes all files and other sub-directories).

Parameters:
  • connection (namedtuple<>) – named tuple with connection information: [‘name’, ‘ip’, ‘ssh_port’, ‘username’, ‘password’]
  • path (str) – A string with the full path we want to remove
util.netutil.ssh_connect_or_return(ip, ssh_port, username, password, maxretries)[source]

Opens a connection and returns a connection object. If it fails to open a connection after a specified number of tries, it returns -1.

Parameters:
  • ip (str) – controller IP address
  • ssh_port (int) – controller port
  • username (str) – username of the remote user
  • password (str) – password of the remote user
  • maxretries (int) – maximum number of times to connect
Returns:

an ssh connection handle or -1 on failure

Return type:

paramiko.SSHClient (or -1 when failure)

util.netutil.ssh_connection_close(sftp, transport_layer)[source]

Closes an ssh connection with a remote node

Parameters:
  • sftp (paramiko.SFTPClient) –
  • transport_layer (paramiko.Transport) –
util.netutil.ssh_connection_open(ip, ssh_port, username, password)[source]

Opens an ssh connection on a remote node

Parameters:
  • ip (str) – ip address of the remote host
  • ssh_port (int) – port number of the remote host
  • username (str) – username of the remote host
  • password (str) – password of the remote host
Returns:

sftp, transport_layer

Return type:

tuple<paramiko.SFTPClient, paramiko.Transport>

util.netutil.ssh_copy_file_to_target(ip, ssh_port, username, password, local_file, remote_file)[source]

Copies local file on a remote machine target.

Parameters:
  • ip (str) – ip address of the remote host
  • ssh_port (int) – port number of the remote host
  • username (str) – username of the remote host
  • password (str) – password of the remote host
  • local_file (str) – file from local machine to copy,full location required
  • remote_file (str) – remote destination, full location required i.e /tmp/foo.txt
util.netutil.ssh_delete_file_if_exists(ip, ssh_port, username, password, remote_file)[source]

Deletes the file on a remote machine, if exists

Parameters:
  • connection (collections.namedtuple) – named tuple with connection information: [‘name’, ‘ip’, ‘ssh_port’, ‘username’, ‘password’]
  • remote_file (str) – remote file to remove, full path must be used.
util.netutil.ssh_run_command(ssh_client, command_to_run, prefix='', lines_queue=None, print_flag=True, block_flag=True, getpty_flag=False)[source]

Runs the specified command on a remote machine

Parameters:
  • ssh_client (paramiko.SSHClient) – SSH client provided by paramiko to run the command
  • command_to_run (str) – Command to execute
  • prefix (str) – prefix of log message
  • lines_queue (queue<str>) – Queue datastructure to buffer the result of execution
  • print_flag (bool) – Flag that defines if the output of the command will be printed on screen
  • block_flag (bool) – Defines if we block execution waiting for the running command to return its exit status
  • getpty_flag (bool) – add a pseudo-terminal console (pty console) to the channel
Returns:

the exit code of the command to be executed remotely and the combined stdout - stderr of the executed command

Return type:

tuple

util.numeric module

Numeric utilities

util.numeric.tonum(str2cast)[source]

Casts a string to the appropriate numeric type, i.e. int or float.

Parameters:str2cast – string to cast
Returns:the numeric value in the appropriate type or “Impossible cast” string :rtype: float :type str

util.plot_json module

Methods that implement plotting functionality in NSTAT

util.plot_json.multiplot_errorbar_json(results_file, x_axis_key, y_axis_key, z_axis_key, plot_subtitle_keys, plot_options)[source]

Draw multiple collection of errorbars over a set of samples from a JSON file. For each different z value do the following: for each different x value, the function finds one or more corresponding y values and plots an errorbar over them. The x and y values are determined by the x_axis_key and y_axis_key arguments. The z value is determined by the z_axis_key argument. Prerequisites: 1. the result JSON file must have the following format: [ {“k1”: v1, “k2”: v2, ... }, # 1st line (sample) {“k1”: v3, “k2”: v4, ... }, # 2nd line (sample) {“k1”: v5, “k2”: v6, ... }, # ... ... ] 2. the values for x_axis_key, y_axis_key and z_axis_key must be numeric

Parameters:
  • results_file (str) – results file to plot samples from
  • x_axis_key (str) – some key from the results file with numeric type value, which is intended to serve as the x-axis key
  • y_axis_key (str) – some key from the result file with numeric type value, which is intended to serve as the y-axis key
  • z_axis_key (str) – some key from the result file with numeric type value, which is intended to serve as the z-axis key
  • plot_subtitle_keys (list<str>) – list of keys from the result file which we would like to print as key-value pairs in the plot subtitle
  • plot_options (PlotOptions) – object containing configuration parameters of the produced plot.
util.plot_json.multiplot_scatter_json(results_file, x_axis_key, y_axis_key, z_axis_key, plot_subtitle_keys, plot_options)[source]

Draw multiple scatter-plots over a set of samples from a JSON file. Each scatter-plot is determined by a specific value of the z_axis_key For each different z value do the following: for each different x value, plot a point for every corresponding y value found. The x and y values are determined by the x_axis_key and y_axis_key arguments. The z value is determined by the z_axis_key argument. Prerequisites: 1. the result JSON file must have the following format: [ {“k1”: v1, “k2”: v2, ... }, # 1st line (sample) {“k1”: v3, “k2”: v4, ... }, # 2nd line (sample) {“k1”: v5, “k2”: v6, ... }, # ... ... ] 2. the values for x_axis_key, y_axis_key and z_axis_key must be numeric

Parameters:
  • results_file (str) – results file to plot samples from
  • x_axis_key (str) – some key from the results file with numeric type value, which is intended to serve as the x-axis key
  • y_axis_key (str) – some key from the result file with numeric type value, which is intended to serve as the y-axis key
  • z_axis_key (str) – some key from the result file with numeric type value, which is intended to serve as the z-axis key
  • plot_subtitle_keys (list<str>) – list of keys from the result file which we would like to print as key-value pairs in the plot subtitle
  • plot_options (PlotOptions) – object containing configuration parameters of the produced plot.
util.plot_json.plot_errorbar_json(results_file, x_axis_key, y_axis_key, plot_subtitle_keys, plot_options)[source]

Draw a single collection of errorbars over a set of samples from a JSON file. For each different x value, the function finds one or more corresponding y values and plots an errorbar over them. The x and y values are determined by the x_axis_key and y_axis_key arguments. Prerequisites: 1. the result JSON file must have the following format: [ {“k1”: v1, “k2”: v2, ... }, # 1st line (sample) {“k1”: v3, “k2”: v4, ... }, # 2nd line (sample) {“k1”: v5, “k2”: v6, ... }, # ... ... ] 2. the values for both the x_axis_key and y_axis_key must be numeric

Parameters:
  • results_file (str) – results file to plot samples from
  • x_axis_key (str) – some key from the results file with numeric type value, which is intended to serve as the x-axis key
  • y_axis_key (str) – some key from the result file with numeric type value, which is intended to serve as the y-axis key
  • plot_subtitle_keys (list<str>) – list of keys from the result file which we would like to print as key-value pairs in the plot subtitle
  • plot_options (PlotOptions) – object containing configuration parameters of the produced plot.
util.plot_json.plot_json(results_file, x_axis_key, y_axis_key, z_axis_key, plot_type, plot_subtitle_keys, plot_options)[source]

Acts as a wrapper method for plotting a set of samples from a JSON file. The method ends up calling specific methods for one of the following plot types: errorbar plots, scatter plots Prerequisites: 1. the result JSON file must have the following format: [ {“k1”: v1, “k2”: v2, ... }, # 1st line (sample) {“k1”: v3, “k2”: v4, ... }, # 2nd line (sample) {“k1”: v5, “k2”: v6, ... }, # ... ... ] 2. the values for both the x_axis_key and y_axis_key must be numeric

Parameters:
  • results_file (str) – results file to plot samples from
  • x_axis_key (str) – some key from the results file with numeric type value, which is intended to serve as the x-axis key
  • y_axis_key (str) – some key from the result file with numeric type value, which is intended to serve as the y-axis key
  • z_axis_key (str) – some key from the result file with numeric type value, which is intended to serve as the z-axis key
  • plot_type (str) – plot type, one of: - ‘errorbar - ‘errorbar_connected’ - ‘scatter’ - ‘multi_errorbar - ‘multi_errorbar_connected’ - ‘multi_scatter’ :param plot_subtitle_keys: list of keys from the result file which we would like to print as key-value pairs in the plot subtitle :param plot_options: object containing configuration parameters of the produced plot.
Raises:

ValueError – When we give an invalid plot_type.

util.plot_json.plot_scatter_json(results_file, x_axis_key, y_axis_key, plot_subtitle_keys, plot_options)[source]

Draw a single scatter-plot over a set of samples from a JSON file. For each different x value, the function plots a point for every corresponding y values it finds. The x and y values are determined by the x_axis_key and y_axis_key arguments. Prerequisites: 1. the result JSON file must have the following format: [ {“k1”: v1, “k2”: v2, ... }, # 1st line (sample) {“k1”: v3, “k2”: v4, ... }, # 2nd line (sample) {“k1”: v5, “k2”: v6, ... }, # ... ... ] 2. the values for both the x_axis_key and y_axis_key must be numeric

Parameters:
  • results_file (str) – results file to plot samples from
  • x_axis_key (str) – some key from the results file with numeric type value, which is intended to serve as the x-axis key
  • y_axis_key (str) – some key from the result file with numeric type value, which is intended to serve as the y-axis key
  • plot_title – description for the plot title
  • plot_subtitle_keys (list<str>) – list of keys from the result file which we would like to print as key-value pairs in the plot subtitle
  • plot_options (PlotOptions) – object containing configuration parameters of the produced plot.
util.plot_json.self_test()[source]

Function used for self testing purposes

util.plot_utils module

Module to facilitate plotting and promote code re-use

class util.plot_utils.PlotOptions[source]

Bases: object

Contains the various plot options attributes.

util.plot_utils.create_xy_dict_from_file(results_file, x_axis_key, y_axis_key)[source]

Reads a json file and returns the contents of the file as a dictionary as well as a dictionary that maps y_axis_keys to x_axis_keys

Parameters:
  • results_file (str) – filepath of json file that contains the results
  • x_axis_key (str) – field name from the result json that has the data for x axis
  • y_axis_key (str) – field name from the result json that has the data for y axis
Returns:

contents of results json files and x,y coordinations of the values defined by x_axis_key and y_axis_key.

Return type:

tuple<dictionary>

util.plot_utils.create_xyz_dict_from_file(results_file, x_axis_key, y_axis_key, z_axis_key)[source]

Reads a json file and returns the contents of the file as a dictionary as well as a dictionary that maps y_axis_keys to x_axis_keys that in turn map to z_axis_keys

Parameters:
  • results_file (str) – filepath of json file that contains the results
  • x_axis_key (str) – field name from the result json that has the data for x axis
  • y_axis_key (str) – field name from the result json that has the data for y axis
  • z_axis_key (str) – field name from the result json that has the data for z axis
Returns:

contents of results json files and x,y coordinations of the values defined by x_axis_key and y_axis_key

Return type:

tuple<dict>

util.plot_utils.finish_plotting(plot_options)[source]

Configures the plots axis and saves the figure to file

Parameters:plot_options (PlotOptions) – object containing configuration parameters of the produced plot.
util.plot_utils.plot_errorbar(x_keys_sorted, y_mean, y_diff_minus, y_diff_plus, plot_options)[source]

Creates a single errorbar figure.

Parameters:
  • x_keys_sorted (list<int>) – values of x axis.
  • y_mean (list<float>) – values of y mean (one value for each x_key)
  • y_diff_minus (list<float>) – values of y_diff_minus (one value for each x_key, y_diff_minus[i] = y_mean[i] - diff_minus). See calculation of diff_minus at plot_json, plot_errorbar_json() :param y_diff_plus: values of y_diff_mplus (one value for each x_key, y_diff_plus[i] = y_mean[i] + diff_plus). See calculation of diff_plus at plot_json, plot_errorbar_json() :param plot_options: object containing configuration parameters of the produced plot.
util.plot_utils.plot_errorbar_helper(x_keys_sorted, y_mean, y_diff_minus, y_diff_plus, plot_options)[source]

Draws a single errorbar.

Parameters:
  • x_keys_sorted (list<int>) – values of x axis
  • y_mean (list<float>) – values of y mean (one value for each x_key)
  • y_diff_minus (list<float>) – values of y_diff_minus (one value for each x_key, y_diff_minus[i] = y_mean[i] - diff_minus). See calculation of diff_minus at plot_json, plot_errorbar_json()
  • y_diff_plus (list<float>) – values of y_diff_mplus (one value for each x_key, y_diff_plus[i] = y_mean[i] + diff_plus). See calculation of diff_plus at plot_json, plot_errorbar_json()
  • plot_options (PlotOptions) – object containing configuration parameters of the produced plot.
Returns:

An errorbar plot.

Return type:

depends on the input of [z_values]

util.plot_utils.plot_multi_errorbar(y_values, z_axis_key, plot_options)[source]

Creates a multiple errorbars figure.

Parameters:
  • y_values (list<float>) – values of y axis.
  • z_axis_key (str) – field names from results to be used for z axis.
  • plot_options (PlotOptions) – object containing configuration parameters of the produced plot.
util.plot_utils.plot_multi_scatter(y_values, z_axis_key, plot_options)[source]

Creates a multiple scatter plots figure

Parameters:
  • y_values (list<float>) – list<float>
  • z_axis_key (list<str>) – field names from results to be used for z axis.
  • plot_options (PlotOptions) – object containing configuration parameters of the produced plot.
util.plot_utils.plot_scatter(x_coords, y_coords, plot_options)[source]

Creates a single scatter plot figure.

Parameters:
  • x_coords – values of x axis.
  • y_coords – Values of y axis.
  • plot_options (PlotOptions) – object containing configuration parameters of the produced plot.
util.plot_utils.plot_scatter_helper(x_coords, y_coords, plot_options, marker_arg='o', color='b')[source]

Produces a single scatter plot with a specific color.

Parameters:
  • x_coords – values of x axis.
  • y_coords – Values of y axis.
  • marker_arg='o' – Marker type of a point on the graph.
  • color (str) – The color of the markers.
Returns:

A scatter plot.

Return type:

matplotlib.pyplot.

util.plot_utils.setup_plot(plot_options)[source]

Sets axis labels, title and subtitle of a plot.

Parameters:plot_options (PlotOptions) – object containing configuration parameters of the produced plot.

util.process module

Process-related utilities

util.process.getpid_listeningonport(port, ssh_client=None)[source]

Finds if there is a running process listening for network connections on a specific port.

Parameters:
  • port (int) – the port number that we investigate.
  • ssh_client (paramiko.SSHClient) – SSH client provided by paramiko to run the command
Returns:

-1, if no process is listening on port 0, if some process is listening on port but we are not owner of it <pid> of the process listening on port and we are owner of it

Return type:

int

util.process.is_process_running(pid, ssh_client=None)[source]

Finds if a process is running, using its process ID.

Parameters:
  • pid (int) – The process ID of the target process
  • ssh_client (paramiko.SSHClient) – SSH client provided by paramiko to run the command
Returns:

True, if the process is running False, otherwise

Return type:

bool

util.process.wait_until_process_finishes(pid, ssh_client=None)[source]

Waits until the process with the specified ID finishes

Parameters:
  • pid (int) – process id
  • ssh_client (paramiko.SSHClient) – SSH client provided by paramiko to run the command

util.stats module

Module that computes statistic properties given a list of samples

util.stats.coefvariance(samples)[source]

Computes the co-efficient of variation of a number list.

Parameters:samples (list<float>) – a list of float numbers, to calculate their coefficient.
Returns:the co-efficient of variation of the float numbers in the list
Return type:float
util.stats.mean(samples)[source]

Computes the mean of a number list

Parameters:s – a list of float number, to calculate their mean.
Returns:the mean of the float numbers in the list
Return type:float
util.stats.stddev(samples)[source]

Computes the standard deviation of a number list.

Parameters:s – a list of float numbers, to calculate their standard deviation.
Returns:the standard deviation of the float numbers in the list.
Return type:float
util.stats.variance(samples)[source]

Computes the variance of a number list

Parameters:s – a list of float numbers, to calculate their variance.
Returns:the variance of the float numbers in the list
Return type:float

util.sysstats module

Module with functions for getting system and process statistics

util.sysstats.command_exec_wrapper(cmd, ssh_client=None, return_type='str')[source]

Executes a command either locally or remotely and returns the result

Parameters:
  • cmd (str) – the command to be executed
  • ssh_client (paramiko.SSHClient) – SSH client provided by paramiko to run the command
  • return_type (str) – Defines the return type of the command output
Returns:

The commands execution result as string

Return type:

str, int, float

util.sysstats.get_java_options(pid, ssh_client=None)[source]

Returns a list with all java options of a process defined by its process ID

Parameters:
  • pid (int) – process id of the process we want to get the javaopts
  • ssh_client (paramiko.SSHClient) – SSH client provided by paramiko to run the command
Returns:

a list with all java options

Return type:

list<str>

util.sysstats.get_units_base(unit_key)[source]

Gets the units in string format and returns the base in order to make appropriate conversions :param unit_key: the type of unit we want to convert :type unit_key: str :returns: the base for the conversion of the units :rtype: int

util.sysstats.proc_cmdline(pid, ssh_client=None)[source]

Returns the command line of a process as a string.

Parameters:
  • pid (int) – the process ID of the target process
  • ssh_client (paramiko.SSHClient) – SSH client provided by paramiko to run the command
Returns:

The command execution output

Return type:

str

util.sysstats.proc_cpu_system_time(pid, ssh_client=None)[source]

Method that returns the CPU system time of a process.

Parameters:
  • pid (int) – the process ID of the target process
  • ssh_client (paramiko.SSHClient) – SSH client provided by paramiko to run the command
Returns:

the CPU system time of a process

Return type:

float

util.sysstats.proc_cpu_user_time(pid, ssh_client=None)[source]

Method that returns the CPU user time of a process.

Parameters:
  • pid (int) – the process ID of the target process
  • ssh_client (paramiko.SSHClient) – SSH client provided by paramiko to run the command
Returns:

the CPU user time of a process

Return type:

float

util.sysstats.proc_cwd(pid, ssh_client=None)[source]

Method that returns the process current working directory.

Parameters:
  • pid (int) – the process ID of the target process
  • ssh_client (paramiko.SSHClient) – SSH client provided by paramiko to run the command
Returns:

the full path of working directory

Return type:

str

util.sysstats.proc_num_fds(pid, ssh_client=None)[source]

Returns the number of file descriptors opened by this process.

Parameters:
  • pid (int) – the process ID of the target process
  • ssh_client (paramiko.SSHClient) – SSH client provided by paramiko to run the command
Returns:

total amount of open files for the specific process ID

Return type:

int

util.sysstats.proc_num_threads(pid, ssh_client=None)[source]

Returns the number of threads used by this process.

Parameters:
  • pid (int) – the process ID of the target process
  • ssh_client (paramiko.SSHClient) – SSH client provided by paramiko to run the command
Returns:

the number of threads for the specific pid

Return type:

int

util.sysstats.proc_vm_size(pid, ssh_client=None)[source]

Method that returns the virtual memory size of a process.

Parameters:
  • pid (int) – the process ID of the target process
  • ssh_client (paramiko.SSHClient) – SSH client provided by paramiko to run the command
Returns:

the virtual memory size of a process

Return type:

int

util.sysstats.sys_free_memory_bytes(ssh_client=None)[source]

Returns system free memory in bytes

Parameters:ssh_client (paramiko.SSHClient) – SSH client provided by paramiko to run the command
Returns:the amount of free RAM memory in the system in bytes
Return type:int
util.sysstats.sys_free_ram_mb(ssh_client=None)[source]

Returns system free memory in MB.

Parameters:ssh_client (paramiko.SSHClient) – SSH client provided by paramiko to run the command
Returns:the amount of free RAM memory in the system in MB.
Return type:int
util.sysstats.sys_iowait_time(ssh_client=None)[source]

For a given CPU, the I/O wait time is the time during which that CPU was idle (i.e. didn’t execute any tasks) and there was at least one outstanding disk I/O operation requested by a task scheduled on that CPU (at the time it generated that I/O request).

Parameters:ssh_client (paramiko.SSHClient) – SSH client provided by paramiko to run the command
Returns:the I/O wait time
Return type:float
util.sysstats.sys_load_average(ssh_client=None)[source]

Returns the system load average.

Parameters:ssh_client (paramiko.SSHClient) – SSH client provided by paramiko to run the command
Returns:tuple of floats with the 1-,5- and 15-min load average
Return type:tuple<float>
util.sysstats.sys_nprocs(ssh_client=None)[source]

Returns the number of CPUs in the system.

Parameters:ssh_client (paramiko.SSHClient) – SSH client provided by paramiko to run the command
Returns:the number of CPUs in the system
Return type:int
util.sysstats.sys_total_memory_bytes(ssh_client=None)[source]

Returns system total memory in bytes

Parameters:ssh_client (paramiko.SSHClient) – SSH client provided by paramiko to run the command
Returns:total system memory in bytes
Return type:int
util.sysstats.sys_used_memory_bytes(ssh_client=None)[source]

Returns system used memory in bytes.

Parameters:ssh_client (paramiko.SSHClient) – SSH client provided by paramiko to run the command
Returns:The amount of used RAM memory in the system in bytes.
Return type:int
util.sysstats.sys_used_ram_mb(ssh_client=None)[source]

Returns system used memory in MB.

Parameters:ssh_client (paramiko.SSHClient) – SSH client provided by paramiko to run the command
Returns:the amount of used RAM memory in the system in MB.
Return type:int

Module contents