Core functionality

Suite

class BenchmarkSuite(log_dir=None, log_id=None, use_timestamp=True, parallel=None, slurm=None)

Bases: object

Class to orchestrate the execution of multiple experiments.

__init__(log_dir=None, log_id=None, use_timestamp=True, parallel=None, slurm=None)

Constructor.

Parameters:

• log_dir (str) – path to the log directory (Default: ./logs or /work/scratch/$USER)
• log_id (str) – log id (Default: benchmark[_YYYY-mm-dd-HH-MM-SS])
• use_timestamp (bool) – select if a timestamp should be appended to the log id
• parallel (dict, None) – parameters that are passed to the run_parallel method of the experiment
• slurm (dict, None) – parameters that are passed to the run_slurm method of the experiment

add_experiments(environment_name, environment_builder_params, agent_names_list, agent_builders_params, **run_params)

Add a set of experiments for the same environment to the suite.

Parameters:

• environment_name (str) – name of the environment for the experiment (E.g. Gym.Pendulum-v0);
• environment_builder_params (dict) – parameters for the environment builder;
• agent_names_list (list) – list of names of the agents for the experiments;
• agent_builders_params (list) – list of dictionaries containing the parameters for the agent builder;
• run_params – Parameters that are passed to the run method of the experiment.

add_experiments_sweeps(environment_name, environment_builder_params, agent_names_list, agent_builders_params, sweeps_list, **run_params)

Add a set of experiments sweeps for the same environment to the suite.

Parameters:

• environment_name (str) – name of the environment for the experiment (E.g. Gym.Pendulum-v0);
• environment_builder_params (dict) – parameters for the environment builder;
• agent_names_list (list) – list of names of the agents for the experiments;
• agent_builders_params (list) – list of dictionaries containing the parameters for the agent builder;
• sweeps_list (list) – list of dictionaries containing the parameter sweep to be executed;
• run_params – Parameters that are passed to the run method of the experiment.

add_environment(environment_name, environment_builder_params, **run_params)

Add an environment to the benchmarking suite.

Parameters:

• environment_name (str) – name of the environment for the experiment (E.g. Gym.Pendulum-v0);
• environment_builder_params (dict) – parameters for the environment builder;
• run_params – Parameters that are passed to the run method of the experiment.

add_agent(environment_name, agent_name, agent_params)

Add an agent to the benchmarking suite.

Parameters:

• environment_name (str) – name of the environment for the experiment (E.g. Gym.Pendulum-v0);
• agent_name (str) – name of the agent for the experiments;
• agent_params (list) – dictionary containing the parameters for the agent builder.

add_sweep(environment_name, agent_name, agent_params, sweep_dict)

Add an agent sweep to the benchmarking suite.

Parameters:

• environment_name (str) – name of the environment for the experiment (E.g. Gym.Pendulum-v0);
• agent_name (str) – name of the agent for the experiments;
• agent_params (list) – dictionary containing the parameters for the agent builder;
• sweep_dict (dict) – dictionary with the sweep configurations.

run(exec_type='sequential')

Run all experiments in the suite.

print_experiments()

Print the experiments in the suite.

save_parameters()

Save the experiment parameters in yaml files inside the parameters folder

save_plots(**plot_params)

Save the result plots to the log directory.

Parameters:**plot_params – parameters to be passed to the suite visualizer.

show_plots(**plot_params)

Display the result plots.

Parameters:**plot_params – parameters to be passed to the suite visualizer.

Experiment

class BenchmarkExperiment(agent_builder, env_builder, logger)

Bases: object

Class to create and run an experiment using MushroomRL

__init__(agent_builder, env_builder, logger)

Constructor.

Parameters:

• agent_builder (AgentBuilder) – instance of a specific agent builder;
• env_builder (EnvironmentBuilder) – instance of an environment builder;
• logger (BenchmarkLogger) – instance of a benchmark logger.

run(exec_type='sequential', **run_params)

Execute the experiment.

Parameters:

• exec_type (str, 'sequential') – type of executing the experiment [sequential|parallel|slurm];
• **run_params – parameters for the selected execution type.

run_sequential(n_runs, n_runs_completed=0, save_plot=True, **run_params)

Execute the experiment sequential.

Parameters:

• n_runs (int) – number of total runs of the experiment;
• n_runs_completed (int, 0) – number of completed runs of the experiment;
• save_plot (bool, True) – select if a plot of the experiment should be saved to the log directory;
• **run_params – parameters for executing a benchmark run.

run_parallel(n_runs, n_runs_completed=0, threading=False, save_plot=True, max_concurrent_runs=None, **run_params)

Execute the experiment in parallel threads.

Parameters:

• n_runs (int) – number of total runs of the experiment;
• n_runs_completed (int, 0) – number of completed runs of the experiment;
• threading (bool, False) – select to use threads instead of processes;
• save_plot (bool, True) – select if a plot of the experiment should be saved to the log directory;
• max_concurrent_runs (int, -1) – maximum number of concurrent runs. By default it uses the number of cores;
• **run_params – parameters for executing a benchmark run.

run_slurm(n_runs, n_runs_completed=0, aggregation_job=True, aggregate_hours=3, aggregate_minutes=0, aggregate_seconds=0, only_print=False, **run_params)

Execute the experiment with SLURM.

Parameters:

• n_runs (int) – number of total runs of the experiment;
• n_runs_completed (int, 0) – number of completed runs of the experiment;
• aggregation_job (bool, True) – select if an aggregation job should be scheduled;
• aggregate_hours (int, 3) – maximum number of hours for the aggregation job;
• aggregate_minutes (int, 0) – maximum number of minutes for the aggregation job;
• aggregate_seconds (int, 0) – maximum number of seconds for the aggregation job;
• only_print (bool, False) – if True, don’t launch the benchmarks, only print the submitted commands to the terminal;
• **run_params – parameters for executing a benchmark run.

reset()

Reset the internal state of the experiment.

resume(logger)

Resume an experiment from disk

start_timer()

Start the timer.

stop_timer()

Stop the timer.

save_builders()

Save agent and environment builder to the log directory.

extend_and_save_J(J)

Extend J with another datapoint and save the current state to the log directory.

extend_and_save_R(R)

Extend R with another datapoint and save the current state to the log directory.

extend_and_save_V(V)

Extend V with another datapoint and save the current state to the log directory.

extend_and_save_entropy(entropy)

Extend entropy with another datapoint and save the current state to the log directory.

set_and_save_config(**settings)

Save the experiment configuration to the log directory.

set_and_save_stats(**info)

Save the run statistics to the log directory.

save_plot()

Save the result plot to the log directory.

show_plot()

Display the result plot.

Logger

class BenchmarkLogger(log_dir=None, log_id=None, use_timestamp=True)

Bases: mushroom_rl.core.logger.console_logger.ConsoleLogger

Class to handle all interactions with the log directory.

__init__(log_dir=None, log_id=None, use_timestamp=True)

Constructor.

Parameters:

• log_dir (str, None) – path to the log directory, if not specified defaults to ./logs or to /work/scratch/$USER if the second directory exists;
• log_id (str, None) – log id, if not specified defaults to: benchmark[_YY-mm-ddTHH:MM:SS.zzz]);
• use_timestamp (bool, True) – select if a timestamp should be appended to the log id.

set_log_dir(log_dir)

Set the directory for logging.

Parameters:log_dir (str) – path of the directory.

get_log_dir()

Returns:The path of the logging directory.

set_log_id(log_id, use_timestamp=True)

Set the id of the logged folder.

Parameters:

• log_id (str) – id of the logged folder;
• use_timestamp (bool, True) – whether to use the timestamp or not.

get_log_id()

Returns:The id of the logged folder.

get_path(filename='')

Get the path of the given file. If no filename is given, it returns the path of the logging folder.

Parameters:filename (str, '') – the name of the file. Returns:The complete path of the logged file.

get_params_path(filename='')

Get the path of the parameters of the given file. If no filename is given, it returns the path of the parameters folder.

Parameters:filename (str, '') – the name of the file. Returns:The complete path of the logged file.

get_figure_path(filename='', subfolder=None)

Get the path of the figures of the given file. If no filename is given, it returns the path of the figures folder.

Parameters:

• filename (str, '') – the name of the file;
• subfolder (None) – the name of a subfolder to add.

Returns:

The complete path of the logged file.

save_J(J)

Save the log of the cumulative discounted reward.

load_J()

Returns:The log of the cumulative discounted reward.

save_R(R)

Save the log of the cumulative reward.

load_R()

Returns:The log of the cumulative reward.

save_V(V)

Save the log of the value function.

load_V()

Returns:The log of the value function.

save_entropy(entropy)

Save the log of the entropy function.

load_entropy()

Returns:The log of the entropy function.

exists_policy_entropy()

Returns:True if the log of the entropy exists, False otherwise.

exists_value_function()

Returns:True if the log of the value function exists, False otherwise.

save_best_agent(agent)

Save the best agent in the respective path.

Parameters:agent (object) – the agent to save.

save_last_agent(agent)

Save the last agent in the respective path.

Parameters:agent (object) – the agent to save.

exists_best_agent()

Returns:True if the entropy file exists, False otherwise.

load_best_agent()

Returns:The best agent.

load_last_agent()

Returns:The last agent.

save_environment_builder(env_builder)

Save the environment builder using the respective path.

Parameters:env_builder (str) – the environment builder to save.

load_environment_builder()

Returns:The environment builder.

save_agent_builder(agent_builder)

Save the agent builder using the respective path.

Parameters:agent_builder (str) – the agent builder to save.

load_agent_builder()

Returns:The agent builder.

save_config(config)

Save the config file using the respective path.

Parameters:config (str) – the config file to save.

load_config()

Returns:The config file.

exists_stats()

Returns:True if the entropy file exists, False otherwise.

save_stats(stats)

Save the statistic file using the respective path.

Parameters:stats (str) – the statistics file to save.

load_stats()

Returns:The statistics file.

save_params(env, params)

Save the parameters file.

Parameters:

• env (str) – the environment used;
• params (str) – the parameters file to save.

save_figure(figure, figname, subfolder=None, as_pdf=False, transparent=True)

Save the figure file using the respective path.

Parameters:

• figure (object) – the figure to save;
• figname (str) – the name of the figure;
• subfolder (str, None) – optional subfolder where to save the figure;
• as_pdf (bool, False) – whether to save the figure in PDF or not;
• transparent (bool, True) – whether the figure should be transparent or not.

classmethod from_path(path)

Method to create a BenchmarkLogger from a path.

Visualizer

class BenchmarkVisualizer(logger, data=None, has_entropy=None, has_value=None, id=1)

Bases: object

Class to handle all visualizations of the experiment.

plot_counter = 0

__init__(logger, data=None, has_entropy=None, has_value=None, id=1)

Constructor.

Parameters:

• logger (BenchmarkLogger) – logger to be used;
• data (dict, None) – dictionary with data points for visualization;
• has_entropy (bool, None) – select if entropy is available for the algorithm.

is_data_persisted

Check if data was passed as dictionary or should be read from log directory.

get_J()

Get J from dictionary or log directory.

get_R()

Get R from dictionary or log directory.

get_V()

Get V from dictionary or log directory.

get_entropy()

Get entropy from dictionary or log directory.

get_report()

Create report plot with matplotlib.

save_report(file_name='report_plot')

Method to save an image of a report of the training metrics from a performend experiment.

show_report()

Method to show a report of the training metrics from a performend experiment.

show_agent(episodes=5, mdp_render=False)

Method to run and visualize the best builders in the environment.

classmethod from_path(path)

Method to create a BenchmarkVisualizer from a path.

class BenchmarkSuiteVisualizer(logger, is_sweep, color_cycle=None, y_limit=None, legend=None)

Bases: object

Class to handle visualization of a benchmark suite.

plot_counter = 0

__init__(logger, is_sweep, color_cycle=None, y_limit=None, legend=None)

Constructor.

Parameters:

• logger (BenchmarkLogger) – logger to be used;
• is_sweep (bool) – whether the benchmark is a parameter sweep.
• color_cycle (dict, None) – dictionary with colors to be used for each algorithm;
• y_limit (dict, None) – dictionary with environment specific plot limits.
• legend (dict, None) – dictionary with environment specific legend parameters.

get_report(env, data_type, selected_alg=None)

Create report plot with matplotlib.

get_boxplot(env, metric_type, data_type, selected_alg=None)

Create boxplot with matplotlib for a given metric.

Parameters:

• env (str) – The environment name;
• metric_type (str) – The metric to compute.

Returns:

A figure with the desired boxplot of the given metric.

save_reports(as_pdf=True, transparent=True, alg_sweep=False)

Method to save an image of a report of the training metrics from a performed experiment.

Parameters:

• as_pdf (bool, True) – whether to save the reports as pdf files or png;
• transparent (bool, True) – If true, the figure background is transparent and not white;
• alg_sweep (bool, False) – If true, thw method will generate a separate figure for each algorithm sweep.

save_boxplots(as_pdf=True, transparent=True, alg_sweep=False)

Method to save an image of a report of the training metrics from a performed experiment.

Parameters:

• as_pdf (bool, True) – whether to save the reports as pdf files or png;
• transparent (bool, True) – If true, the figure background is transparent and not white;
• alg_sweep (bool, False) – If true, thw method will generate a separate figure for each algorithm sweep.

show_reports(boxplots=True, alg_sweep=False)

Method to show a report of the training metrics from a performend experiment.

Parameters:alg_sweep (bool, False) – If true, thw method will generate a separate figure for each algorithm sweep.