Core functionality

Suite

class mushroom_rl_benchmark.core.suite.BenchmarkSuite(log_dir=None, log_id=None, use_timestamp=True, parallel=None, slurm=None)

Bases: object

Class to orchestrate the execution of multiple experiments.

__init__(log_dir=None, log_id=None, use_timestamp=True, parallel=None, slurm=None)

Constructor.

Parameters:

• log_dir (str) – path to the log directory (Default: ./logs or /work/scratch/$USER)
• log_id (str) – log id (Default: benchmark[_YYYY-mm-dd-HH-MM-SS])
• use_timestamp (bool) – select if a timestamp should be appended to the log id
• parallel (dict, None) – parameters that are passed to the run_parallel method of the experiment
• slurm (dict, None) – parameters that are passed to the run_slurm method of the experiment

add_experiments(environment_name, environment_builder_params, agent_names_list, agent_builders_params, **run_params)

Add a set of experiments for the same environment to the suite.

Parameters:

• environment_name (str) – name of the environment for the experiment (E.g. Gym.Pendulum-v0);
• environment_builder_params (dict) – parameters for the environment builder;
• agent_names_list (list) – list of names of the agents for the experiments;
• agent_builders_params (list) – list of dictionaries containing the parameters for the agent builder;
• run_params – Parameters that are passed to the run method of the experiment.

add_environment(environment_name, environment_builder_params, **run_params)

Add an environment to the benchmarking suite.

Parameters:

• environment_name (str) – name of the environment for the experiment (E.g. Gym.Pendulum-v0);
• environment_builder_params (dict) – parameters for the environment builder;
• run_params – Parameters that are passed to the run method of the experiment.

add_agent(environment_name, agent_name, agent_params)

Add an agent to the benchmarking suite.

Parameters:

• environment_name (str) – name of the environment for the experiment (E.g. Gym.Pendulum-v0);
• agent_name (str) – name of the agent for the experiments;
• agent_params (list) – dictionary containing the parameters for the agent builder.

run(exec_type='sequential')

Run all experiments in the suite.

print_experiments()

Print the experiments in the suite.

save_parameters()

Save the experiment parameters in yaml files inside the parameters folder

save_plots(**plot_params)

Save the result plots to the log directory.

Parameters:**plot_params – parameters to be passed to the suite visualizer.

show_plots(**plot_params)

Display the result plots.

Parameters:**plot_params – parameters to be passed to the suite visualizer.

Experiment

class mushroom_rl_benchmark.core.experiment.BenchmarkExperiment(agent_builder, env_builder, logger)

Bases: object

Class to create and run an experiment using MushroomRL

__init__(agent_builder, env_builder, logger)

Constructor.

Parameters:

• agent_builder (AgentBuilder) – instance of a specific agent builder;
• env_builder (EnvironmentBuilder) – instance of an environment builder;
• logger (BenchmarkLogger) – instance of a benchmark logger.

run(exec_type='sequential', **run_params)

Execute the experiment.

Parameters:

• exec_type (str, 'sequential') – type of executing the experiment [sequential|parallel|slurm];
• **run_params – parameters for the selected execution type.

run_sequential(n_runs, n_runs_completed=0, save_plot=True, **run_params)

Execute the experiment sequential.

Parameters:

• n_runs (int) – number of total runs of the experiment;
• n_runs_completed (int, 0) – number of completed runs of the experiment;
• save_plot (bool, True) – select if a plot of the experiment should be saved to the log directory;
• **run_params – parameters for executing a benchmark run.

run_parallel(n_runs, n_runs_completed=0, threading=False, save_plot=True, max_concurrent_runs=None, **run_params)

Execute the experiment in parallel threads.

Parameters:

• n_runs (int) – number of total runs of the experiment;
• n_runs_completed (int, 0) – number of completed runs of the experiment;
• threading (bool, False) – select to use threads instead of processes;
• save_plot (bool, True) – select if a plot of the experiment should be saved to the log directory;
• max_concurrent_runs (int, -1) – maximum number of concurrent runs. By default it uses the number of cores;
• **run_params – parameters for executing a benchmark run.

run_slurm(n_runs, n_runs_completed=0, aggregation_job=True, aggregate_hours=3, aggregate_minutes=0, aggregate_seconds=0, only_print=False, **run_params)

Execute the experiment with SLURM.

Parameters:

• n_runs (int) – number of total runs of the experiment;
• n_runs_completed (int, 0) – number of completed runs of the experiment;
• aggregation_job (bool, True) – select if an aggregation job should be scheduled;
• aggregate_hours (int, 3) – maximum number of hours for the aggregation job;
• aggregate_minutes (int, 0) – maximum number of minutes for the aggregation job;
• aggregate_seconds (int, 0) – maximum number of seconds for the aggregation job;
• only_print (bool, False) – if True, don’t launch the benchmarks, only print the submitted commands to the terminal;
• **run_params – parameters for executing a benchmark run.

reset()

Reset the internal state of the experiment.

resume(logger)

Resume an experiment from disk

start_timer()

Start the timer.

stop_timer()

Stop the timer.

save_builders()

Save agent and environment builder to the log directory.

extend_and_save_J(J)

Extend J with another datapoint and save the current state to the log directory.

extend_and_save_R(R)

Extend R with another datapoint and save the current state to the log directory.

extend_and_save_V(V)

Extend V with another datapoint and save the current state to the log directory.

extend_and_save_entropy(entropy)

Extend entropy with another datapoint and save the current state to the log directory.

set_and_save_config(**settings)

Save the experiment configuration to the log directory.

set_and_save_stats(**info)

Save the run statistics to the log directory.

save_plot()

Save the result plot to the log directory.

show_plot()

Display the result plot.

Logger

class mushroom_rl_benchmark.core.logger.BenchmarkLogger(log_dir=None, log_id=None, use_timestamp=True)

Bases: mushroom_rl.core.logger.console_logger.ConsoleLogger

Class to handle all interactions with the log directory.

__init__(log_dir=None, log_id=None, use_timestamp=True)

Constructor.

Parameters:

• log_dir (str, None) – path to the log directory, if not specified defaults to ./logs or to /work/scratch/$USER if the second directory exists;
• log_id (str, None) – log id, if not specified defaults to: benchmark[_YY-mm-ddTHH:MM:SS.zzz]);
• use_timestamp (bool, True) – select if a timestamp should be appended to the log id.

set_log_dir(log_dir)

get_log_dir()

set_log_id(log_id, use_timestamp=True)

get_log_id()

get_path(filename='')

get_params_path(filename='')

get_figure_path(filename='', subfolder=None)

save_J(J)

load_J()

save_R(R)

load_R()

save_V(V)

load_V()

save_entropy(entropy)

load_entropy()

exists_policy_entropy()

save_best_agent(agent)

save_last_agent(agent)

exists_best_agent()

load_best_agent()

load_last_agent()

save_environment_builder(env_builder)

load_environment_builder()

save_agent_builder(agent_builder)

load_agent_builder()

save_config(config)

load_config()

exists_stats()

save_stats(stats)

load_stats()

save_params(env, params)

save_figure(figure, figname, subfolder=None, as_pdf=False, transparent=True)

classmethod from_path(path)

Method to create a BenchmarkLogger from a path.

Visualizer

class mushroom_rl_benchmark.core.visualizer.BenchmarkVisualizer(logger, data=None, has_entropy=None, id=1)

Bases: object

Class to handle all visualizations of the experiment.

plot_counter = 0

__init__(logger, data=None, has_entropy=None, id=1)

Constructor.

Parameters:

• logger (BenchmarkLogger) – logger to be used;
• data (dict, None) – dictionary with data points for visualization;
• has_entropy (bool, None) – select if entropy is available for the algorithm.

is_data_persisted

Check if data was passed as dictionary or should be read from log directory.

get_J()

Get J from dictionary or log directory.

get_R()

Get R from dictionary or log directory.

get_V()

Get V from dictionary or log directory.

get_entropy()

Get entropy from dictionary or log directory.

get_report()

Create report plot with matplotlib.

save_report(file_name='report_plot')

Method to save an image of a report of the training metrics from a performend experiment.

show_report()

Method to show a report of the training metrics from a performend experiment.

show_agent(episodes=5, mdp_render=False)

Method to run and visualize the best builders in the environment.

classmethod from_path(path)

Method to create a BenchmarkVisualizer from a path.