Calls irace.main() using command-line options, maybe parsed from the command line used to invoke R.

## Usage

irace.cmdline(argv = commandArgs(trailingOnly = TRUE))

## Arguments

argv

(character())
The arguments provided on the R command line as a character vector, e.g., c("--scenario", "scenario.txt", "-p", "parameters.txt"). Using the default value (not providing the parameter) is the easiest way to call irace.cmdline.

## Value

(invisible(data.frame)) A data frame with the set of best algorithm configurations found by irace. The data frame has the following columns:

• .ID. : Internal id of the candidate configuration.

• Parameter names : One column per parameter name in parameters.

• .PARENT. : Internal id of the parent candidate configuration.

Additionally, this function saves an R data file containing an object called iraceResults. The path of the file is indicated in scenario$logFile. The iraceResults object is a list with the following structure: scenario The scenario R object containing the irace options used for the execution. See defaultScenario for more information. parameters The parameters R object containing the description of the target algorithm parameters. See readParameters. allConfigurations The target algorithm configurations generated by irace. This object is a data frame, each row is a candidate configuration, the first column (.ID.) indicates the internal identifier of the configuration, the following columns correspond to the parameter values, each column named as the parameter name specified in the parameter object. The final column (.PARENT.) is the identifier of the configuration from which model the actual configuration was sampled. allElites A list that contains one element per iteration, each element contains the internal identifier of the elite candidate configurations of the corresponding iteration (identifiers correspond to allConfigurations$.ID.).

iterationElites

A vector containing the best candidate configuration internal identifier of each iteration. The best configuration found corresponds to the last one of this vector.

experiments

A matrix with configurations as columns and instances as rows. Column names correspond to the internal identifier of the configuration (allConfigurations$.ID.). experimentLog A matrix with columns iteration, instance, configuration, time. This matrix contains the log of all the experiments that irace performs during its execution. The instance column refers to the index of the scenario$instancesList data frame. Time is saved ONLY when reported by the targetRunner.

softRestart

A logical vector that indicates if a soft restart was performed on each iteration. If FALSE, then no soft restart was performed.

state

A list that contains the state of irace, the recovery is done using the information contained in this object.

testing

A list that contains the testing results. The elements of this list are: experiments a matrix with the testing experiments of the selected configurations in the same format as the explained above and seeds a vector with the seeds used to execute each experiment.

## Details

The function reads the parameters given on the command line used to invoke R, finds the name of the scenario file, initializes the scenario from the file (with the function readScenario) and possibly from parameters passed in the command line. It finally starts irace by calling irace.main.

List of command-line options:

-h,--help                Show this help.
-v,--version             Show irace package version.
-c,--check               Check scenario.
-i,--init                Initialize the working directory with template config
files.
--only-test           Only test the configurations given in the file passed
as argument.
-s,--scenario            File that describes the configuration scenario setup
and other irace settings.
-p,--parameter-file      File that contains the description of the parameters
of the target algorithm.
--exec-dir            Directory where the programs will be run.
-l,--log-file            File to save tuning results as an R dataset, either
absolute path or relative to execDir.
--recovery-file       Previously saved log file to recover the execution of
irace, either absolute path or relative to the current
directory.  If empty or NULL, recovery is not
performed.
--train-instances-dir  Directory where training instances are located;
either absolute path or relative to current directory.
If no trainInstancesFiles is provided, all the files
in trainInstancesDir will be listed as instances.
--train-instances-file  File that contains a list of training instances and
optionally additional parameters for them. If
trainInstancesDir is provided, irace will search for
the files in this folder.
--configurations-file  File that contains a table of initial configurations.
If empty or NULL, all initial configurations are
randomly generated.
--forbidden-file      File that contains a list of logical expressions that
cannot be TRUE for any evaluated configuration. If
empty or NULL, do not use forbidden expressions.
--target-runner       Script called for each configuration that executes the
target algorithm to be tuned. See the templates and
examples provided.
--target-runner-launcher  An executable that will be used to launch the
target runner, when targetRunner cannot be executed
directly (.e.g, a Python script in Windows).
--target-runner-args  Command-line arguments provided to
targetRunnerLauncher. The substrings \{targetRunner\}
and \{targetRunnerArgs\} will be replaced by the value
of the option targetRunner and by the arguments
usually passed when calling targetRunner,
respectively. Example: "-m \{targetRunner\ --args
\{targetRunnerArgs\}"}.
--target-runner-retries  Number of times to retry a call to targetRunner if
the call failed.
--target-evaluator    Optional script or R function that provides a numeric
value for each configuration. See
templates/target-evaluator.tmpl
--max-experiments     Maximum number of runs (invocations of targetRunner)
that will be performed. It determines the maximum
budget of experiments for the tuning.
--max-time            Maximum total execution time in seconds for the
executions of targetRunner. targetRunner must return
two values: cost and time.
--budget-estimation   Fraction (smaller than 1) of the budget used to
estimate the mean computation time of a configuration.
Only used when maxTime > 0
--min-measurable-time  Minimum time unit that is still (significantly)
measureable.
--digits              Maximum number of decimal places that are significant
for numerical (real) parameters.
--debug-level         Debug level of the output of irace. Set this to 0 to
silence all debug messages. Higher values provide more
verbose debug messages.
--iterations          Maximum number of iterations.
--experiments-per-iteration  Number of runs of the target algorithm per
iteration.
--sample-instances    Randomly sample the training instances or use them in
the order given.
--test-type           Statistical test used for elimination. The default
value selects t-test if capping is enabled or F-test,
otherwise. Valid values are: F-test (Friedman test),
t-test (pairwise t-tests with no correction),
t-test-bonferroni (t-test with Bonferroni's correction
for multiple comparisons), t-test-holm (t-test with
Holm's correction for multiple comparisons).
--first-test          Number of instances evaluated before the first
elimination test. It must be a multiple of eachTest.
--each-test           Number of instances evaluated between elimination
tests.
--min-survival        Minimum number of configurations needed to continue
the execution of each race (iteration).
--num-configurations  Number of configurations to be sampled and evaluated
at each iteration.
--mu                  Parameter used to define the number of configurations
sampled and evaluated at each iteration.
--confidence          Confidence level for the elimination test.
--deterministic       If the target algorithm is deterministic,
configurations will be evaluated only once per
instance.
--seed                Seed of the random number generator (by default,
generate a random seed).
--parallel            Number of calls to targetRunner to execute in
parallel. Values 0 or 1 mean no parallelization.
experiments in parallel. Load-balancing makes better
use of computing resources, but increases
--mpi                 Enable/disable MPI. Use Rmpi to execute targetRunner
in parallel (parameter parallel is the number of
slaves).
--batchmode           Specify how irace waits for jobs to finish when
targetRunner submits jobs to a batch cluster: sge,
pbs, torque, slurm or htcondor. targetRunner must
submit jobs to the cluster using, for example, qsub.
--soft-restart        Enable/disable the soft restart strategy that avoids
premature convergence of the probabilistic model.
--soft-restart-threshold  Soft restart threshold value for numerical
parameters. If NA, NULL or "", it is computed as
10^-digits.
--test-instances-dir  Directory where testing instances are located, either
absolute or relative to current directory.
--test-instances-file  File containing a list of test instances and
--test-num-elites     Number of elite configurations returned by irace that
will be tested if test instances are provided.
--test-iteration-elites  Enable/disable testing the elite configurations
found at each iteration.
-e,--elitist             Enable/disable elitist irace.
--elitist-new-instances  Number of instances added to the execution list
before previous instances in elitist irace.
--elitist-limit       In elitist irace, maximum number per race of
elimination tests that do not eliminate a
configuration. Use 0 for no limit.
--capping             Enable the use of adaptive capping, a technique
designed for minimizing the computation time of
configurations. This is only available when elitist is
active.
--capping-type        Measure used to obtain the execution bound from the
performance of the elite configurations: median, mean,
worst, best.
--bound-type          Method to calculate the mean performance of elite
configurations: candidate or instance.
--bound-max           Maximum execution bound for targetRunner. It must be
specified when capping is enabled.
--bound-digits        Precision used for calculating the execution time. It
must be specified when capping is enabled.
--bound-par           Penalization constant for timed out executions
(executions that reach boundMax execution time).
--bound-as-timeout    Replace the configuration cost of bounded executions
with boundMax.
--postselection       Percentage of the configuration budget used to perform
a postselection race of the best configurations of
each iteration after the execution of irace.
--aclib               Enable/disable AClib mode. This option enables
compatibility with GenericWrapper4AC as targetRunner
script.


irace.main to start irace with a given scenario.

## Author

Manuel López-Ibáñez and Jérémie Dubois-Lacoste

## Examples

irace.cmdline("--version")
#> #------------------------------------------------------------------------------
#> # irace: An implementation in R of (Elitist) Iterated Racing
#> # Version: 3.5.ec8110c-dirty
#> # Manuel Lopez-Ibanez     <manuel.lopez-ibanez@manchester.ac.uk>
#> # Jeremie Dubois-Lacoste
#> # Leslie Perez Caceres    <leslie.perez.caceres@ulb.ac.be>
#> #
#> # This is free software, and you are welcome to redistribute it under certain
#> # conditions.  See the GNU General Public License for details. There is NO
#> # WARRANTY; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
#> #
#> # irace builds upon previous code from the race package:
#> #     race: Racing methods for the selection of the best
#> #     Copyright (C) 2003 Mauro Birattari
#> #------------------------------------------------------------------------------
#> # installed at: /home/runner/work/_temp/Library/irace
#> # called with: --version
#>
#> To cite package 'irace' in publications use:
#>
#>   Manuel López-Ibáñez, Jérémie Dubois-Lacoste, Leslie Pérez Cáceres,
#>   Thomas Stützle, and Mauro Birattari. The irace package: Iterated
#>   Racing for Automatic Algorithm Configuration. Operations Research
#>   Perspectives, 3:43-58, 2016. doi:10.1016/j.orp.2016.09.002
#>
#> A BibTeX entry for LaTeX users is
#>
#>   @Article{,
#>     title = {The irace package: Iterated Racing for Automatic Algorithm Configuration},
#>     author = {Manuel López-Ibáñez and Jérémie Dubois-Lacoste and Leslie {Pérez Cáceres} and Thomas Stützle and Mauro Birattari},
#>     year = {2016},
#>     journal = {Operations Research Perspectives},
#>     doi = {10.1016/j.orp.2016.09.002},
#>     volume = {3},
#>     pages = {43--58},
#>   }
#>