\documentclass[british]{article}
\usepackage[T1]{fontenc}
\usepackage[latin9]{inputenc}
\usepackage{geometry}
\geometry{verbose,tmargin=3.5cm,bmargin=3.5cm,lmargin=3cm,rmargin=3cm}
\usepackage{array}
\usepackage{amstext}
\usepackage{enumitem}
\usepackage{graphicx}
\usepackage{color}
\usepackage{caption}

\newcommand{\tabincell}[2]{\begin{tabular}{@{}#1@{}}#2\end{tabular}}

\newcommand{\mytablefontsize}{7pt}
\newcommand{\mytablebaselineskip}{0.7}
\newcommand{\mytabcolsep}{3pt}

\newcommand{\medianInterval}[1]{}

\makeatletter

\newif\ifdecision
@@decisionBool@@

\title{Parallel portfolio report}
\author{ \emph{Sparkle} }

\makeatother

\usepackage{babel}

\begin{document}
\maketitle %

\section{Introduction}
\label{sec:Introduction}

\emph{Sparkle} \cite{Hoos15} is a multi-agent problem-solving platform based on Programming by Optimisation (PbO) \cite{Hoos12}, and provides a number of effective algorithm optimisation techniques (such as automated algorithm configuration, portfolio-based algorithm selection, etc.) to accelerate existing solvers.

This experimental report is automatically generated by \emph{Sparkle}. This report presents experimental results of a \emph{Sparkle} parallel algorithm portfolio containing @@numSolvers@@ solver(s).

\section{Experimental Preliminaries}
\label{sec:Experimental_Preliminaries}

This section presents the experimental preliminaries, including the list of solvers in the portfolio, the list of instance sets and information about the experimental setup.

\subsection{Solvers}
\label{sec:Solvers}
There are @@numSolvers@@ solver(s) included in \emph{Sparkle}, as listed below.

\begin{enumerate}[nolistsep] 
@@solverList@@
\end{enumerate}

\subsection{Instance Set(s)}
\label{sec:Instance_Sets}
There are @@numInstanceClasses@@ instance set(s) included in \emph{Sparkle}, as listed below.

\begin{enumerate}
@@instanceClassList@@
\end{enumerate}

\subsection{Experimental Setup}
\label{sec:Experimental_Setup}

The experimental setup is described below.

\ifdecision \textbf{Performance computation:}
\else \textbf{Optimisation results:}
\fi
\emph{Sparkle} runs the portfolio one time on each instance. The cutoff time for the computation run is set to @@cutoffTime@@ seconds. The outcome of the computation is listed below.
\ifdecision
The scores of the outcomes are calculated according to @@performanceMetric@@, this means that for each instance the solver which solved the instance is scored its runtime and the remaining solvers are scored the runtime times ten. If however the portfolio reaches the cutofftime, which means that no solvers solved the instance, all solvers are scored the cutofftime times ten.
\fi

\begin{enumerate}
@@solversWithSolution@@
\end{enumerate}
\ifdecision
In the table below the computed @@performanceMetric@@ scores of all solvers have been listed aswell as for the parallel algorithm portfolio itself.
\begin{table}[ht]
@@resultsTable@@
\end{table}
\fi
\subsection{Scatter Plot Analysis}
\label{sec:Scatter_Plot_Analysis}

Figure~\ref{fig:sparkle_vs_sbs} shows the empirical comparison between the actual parallel algorithm portfolio in \emph{Sparkle} and the single best solver (\emph{SBS}).

\begin{figure}[htbp]
\noindent \begin{centering}
    @@figure-parallel-portfolio-sparkle-vs-sbs@@
\par\end{centering}

\caption{Empirical comparison between the actual parallel algorithm portfolio in \emph{Sparkle} and the \emph{SBS}.}\label{fig:sparkle_vs_sbs}
\end{figure}

\bibliographystyle{plain}
\bibliography{@@bibliographypath@@}

\end{document}
