\part{Introduction}


\chapter{Overview}

\noindent
This document is both a user's guide and a reference manual for
\Stan's probabilistic modeling language.  This introductory chapter
provides a high-level overview of \Stan.  The next chapter provides a
hands-on quick-start guide showing how \Stan works in practice.
Installation instructions are in \refappendix{install}. The remaining
parts of this document include a practically-oriented user's guide for
programming models and a detailed reference manual for \Stan's
modeling language and associated programs and data formats.

\section{\Stan Programs}

A \Stan program defines a statistical model through a conditional
probability function $p(\theta|y;x)$, where $\theta$ is a sequence of
modeled unknown values (e.g., model parameters, latent variables, missing
data, future predictions), $y$ is a sequence of modeled known 
values, and $x$ is a sequence of unmodeled predictors and constants
(e.g., sizes, hyperparameters).

\Stan programs consist of variable type declarations and statements.
Variable types include constrained and unconstrained integer, scalar,
vector, and matrix types, as well as (multidimensional) arrays of
other types.  Variables are declared in blocks corresponding to the
variable's use: data, transformed data, parameter, transformed
parameter, or generated quantity.  Unconstrained local variables may
be declared within statement blocks.

Statements in \Stan are interpreted imperatively, so their order
matters.  Atomic statements involve the assignment of a value to a
variable.  Sequences of statements (and optionally local variable
declarations) may be organized into a block.  \Stan also provides bounded
for-each loops of the sort used in \R and \BUGS.

The transformed data, transformed parameter, and generated quantities
blocks contain statements defining the variables declared in their
blocks.  A special model block consists of statements defining the log
probability for the model.

Within the model block, \BUGS-style sampling notation may be used as
shorthand for incrementing an underlying log probability variable, the
value of which defines the log probability function.  The log
probability variable may also be accessed directly, allowing
user-defined probability functions and Jacobians of transforms.


\section{Compiling and Running \Stan Programs}

A \Stan program is first compiled to a \Cpp program by the \Stan
compiler \stanc, then the \Cpp program compiled to a self-contained
platform-specific executable.  \Stan can generate executables for
various flavors of Windows, Mac OS X, and Linux.%
%
\footnote{A \Stan program may also be compiled to a dynamically
  linkable object file for use in a higher-level scripting language
  such as \R or Python.}
%
Running the \Stan executable for a model first reads in and validates
the known values $y$ and $x$, then generates a sequence of
(non-independent) identically distributed samples $\theta^{(1)},
\theta^{(2)}, \ldots$, each of which has the marginal distribution
$p(\theta|y;x)$.


\section{Sampling}

For continuous parameters, \Stan uses Hamiltonian Monte Carlo (\HMC)
sampling \citep{Duane:1987, Neal:1994, Neal:2011}, a form of Markov chain Monte
Carlo (\MCMC) sampling \citep{Metropolis:1953}.  \Stan 1.0 does not do
discrete sampling.%
%
\footnote{Plans are in place to add full discrete sampling in \Stan
  2.0.  An intermediate step will be to allow forward sampling of
  discrete variables in the generated quantities block for predictive
  modeling and model checking.}
%
\refchapter{mixture-modeling} discusses how finite discrete parameters
can be summed out of models.

\HMC accelerates both convergence to the stationary distribution and
subsequent parameter exploration by using the gradient of the log
probability function.  The unknown quantity vector $\theta$ is
interpreted as the position of a fictional particle.  Each iteration
generates a random momentum and simulates the path of the particle
with potential energy determined the (negative) log probability
function.  Hamilton's decomposition shows that the gradient of this
potential determines change in momentum and the momentum determines
the change in position.  These continuous changes over time are
approximated using the leapfrog algorithm, which breaks the time into
discrete steps which are easily simulated.  A Metropolis reject step
is then applied to correct for any simulation error and ensure
detailed balance of the resulting Markov chain transitions
\citep{Metropolis:1953, Hastings:1970}.

Standard \HMC involves three ``tuning'' parameters to which its
behavior is quite sensitive.  \Stan's samplers allow these parameters
to be set by hand or set automatically without user intervention.

The first two tuning parameters set the temporal step size of the
discretization of the Hamiltonian and the total number of steps taken
per iteration (with their product determining total simulation time).
\Stan can be configured with a user-specified step size or it can
estimate an optimal step size during warmup using dual averaging
\citep{Nesterov:2009, Hoffman-Gelman:2011, Hoffman-Gelman:2013}.  
In either case, additional randomization may be applied to draw the 
step size from an interval of possible step sizes \citep{Neal:2011}.

\Stan can be set to use a specified number of steps, or it can
automatically adapt the number of steps during sampling using the
No-U-Turn (\NUTS) sampler 
\citep{Hoffman-Gelman:2011, Hoffman-Gelman:2013}.  

The third tuning parameter is a mass matrix for the fictional
particle.  \Stan can be configured to estimate a diagonal mass matrix
or a full mass matrix during warmup; Stan will support user-specified
mass matrices in the future.  Estimating a diagonal mass matrix
normalizes the scale of each element $\theta_k$ of the unknown
variable sequence $\theta$, whereas estimating a full mass matrix
accounts for both scaling and rotation,%
%
\footnote{These estimated mass matrices are global, meaning they are
  applied to every point in the parameter space being sampled.
  Riemann-manifold HMC generalizes this to allow the curvature implied
  by the mass matrix to vary by position.}
%
but is more memory and computation intensive per leapfrog step due to
the underlying matrix operations.

\subsection{Convergence Monitoring and Effective Sample Size}

Samples in a Markov chain are only drawn with the marginal
distribution $p(\theta|y;x)$ after the chain has converged to its
equilibrium distribution.  There are several methods to test whether
an \MCMC method has failed to converge; unfortunately, passing the
tests does not guarantee convergence.  The recommended method for
\Stan is to run multiple Markov chains each with different diffuse
initial parameter values, discard the warmup/adaptation samples, then
split the remainder of each chain in half and compute the potential
scale reduction statistic, $\hat{R}$ \citep{GelmanRubin:1992}.

When estimating a mean based on $M$ independent samples, the
estimation error is proportional to $1/\sqrt{M}$.  If the samples are
positively correlated, as they typically are when drawn using \MCMC
methods, the error is proportional to $1/\sqrt{\mbox{\sc ess}}$, where
{\sc ess} is the effective sample size.  Thus it is standard practice
to also monitor (an estimate of) the effective sample size of
parameters of interest in order to estimate the additional estimation
error due to correlated samples.

\subsection{Bayesian Inference and Monte Carlo Methods}

\Stan was developed to support full Bayesian inference.  Bayesian
inference is based in part on Bayes's rule,
\[
p(\theta|y;x) \propto p(y|\theta;x) \, p(\theta;x),
\]
which, in this unnormalized form, states that the posterior
probability $p(\theta|y;x)$ of parameters $\theta$ given data $y$ (and
constants $x$) is proportional (for fixed $y$ and $x$) to the
product of the likelihood function $p(y|\theta;x)$ and prior
$p(\theta;x)$.

For \Stan, Bayesian modeling involves coding the posterior probability
function up to a proportion, which Bayes's rule shows is equivalent to
modeling the product of the likelihood function and prior up to a
proportion.

Full Bayesian inference involves propagating the uncertainty in the
value of parameters $\theta$ modeled by the posterior $p(\theta|y;x)$.
This can be accomplished by basing inference on a sequence of samples
from the posterior using plug-in estimates for quantities of interest
such as posterior means, posterior intervals, predictions based on the
posterior such as event outcomes or the values of as yet unobserved
data.


\section{Optimization}

\Stan also supports optimization-based inference for models.  Given a
posterior $p(\theta|y)$, Stan can find the posterior mode $\theta^*$,
which is defined by
%
\[
\theta^{*} = \mbox{argmax}_{\theta} \ p(\theta|y).
\]
%
Here the notation $\mbox{argmax}_u \ f(v)$ is used to pick out the value
of $v$ at which $f(v)$ is maximized.  

If the prior is uniform, the posterior mode corresponds to the maximum
likelihood estimate (MLE) of the parameters.  If the prior is not
uniform, the posterior mode is sometimes called the maximum a
posterior (MAP) estimate.  If parameters (typically hierarchical) have
been marginalized out, it's sometimes called a maximum marginal
likelihood (MML) estimate. 


\subsection{Inference with Point Estimates}

The estimate $\theta^{*}$ is a so-called ``point estimate,'' meaning
that it summarizes the posterior distribution by a single point,
rather than with a distribution.  Of course, a point estimate does
not, in and of itself, take into account estimation variance.
Posterior predictive inferences $p(\tilde{y} | y)$ can be made using
the posterior mode given data $y$ as $p(\tilde{y}|\theta^*)$, but they
are not Bayesian inferences, even if the model involves a prior,
because they do not take posterior uncertainty into account.  If the
posterior variance is low and the posterior mean is near the posterior
mode, inference with point estimates can be very similar to full
Bayesian inference.

\subsubsection{``Empirical Bayes''}

Fitting point estimates of priors and then using them for subsequent
inference is sometimes called ``empirical Bayes'' (see, e.g.,
\citep{Efron:2012}).%
%
\footnote{The scare quotes on ``empirical Bayes'' are because the
  approach is no more empirical than full Bayes.  Empirical Bayes
  approaches merely ignore some posterior uncertainty to make
  inference more efficient computationally.}
%
Typically these optimizations will be done using maximum marignal
likelihood rather than posterior modes of a full model.  Sometimes
Empirical Bayes point estimates will be obtained using moment matching
(see, e.g., the rat-tumor example in Chapter 5 of
\citep{GelmanCarlinSternRubin:2003}).


\subsection{Experimental Feature}

Stan's optimizers have not been as well tested as its samplers, so
they are still considered an ``experimental'' feature.  We would love
to hear back about successess or failures users have with
optimization.  



\chapter{Getting Started}

\noindent
This chapter is designed to help users get acquainted with the overall
design of the \Stan language and calling \Stan from the command line.
Later chapters are devoted to expanding on the material in this
chapter with full reference documentation.  The content is identical
to that found on the getting-started with the command-line
documentation on the Stan home page, \url{http://mc-stan.org/}.

\section{For BUGS Users}

\refappendix{stan-for-bugs} describes some similarities
and important differences between Stan and BUGS (including WinBUGS,
OpenBUGs, and JAGS).


\section{Installation}

For information about supported versions of Windows, Mac and Linux
platforms along with step-by-step installation instructions, see
\refappendix{install}.

\section{Building Stan}

Building Stan itself works the same way across platforms.
To build Stan, first open a command-line terminal application.  Then change
directories to the directory in which Stan is installed (i.e., the
directory containing the file named \code{makefile}).
%
\begin{quote}
\begin{Verbatim}[fontshape=sl]
> cd <stan-home>
\end{Verbatim}
\end{quote}
%
Then make the library with the following make command
%
\begin{quote}
\begin{Verbatim}[fontshape=sl]
> make bin/libstan.a
\end{Verbatim}
\end{quote}
%
then make the model parser and code generator with the following call
%
\begin{quote}
\begin{Verbatim}[fontshape=sl]
> make bin/stanc
\end{Verbatim}
\end{quote}
%
\emph{Warning}: The \code{make} program may take 10+ minutes and
consume 2+ GB of memory to build \code{libstan.a} and \code{stanc}.
Compiler warnings, including \code{uname: not found}, may be safely ignored.
 
Finally, make the Stan output summary program with the following
make command
%
\begin{quote}
\begin{Verbatim}[fontshape=sl]
> make bin/print
\end{Verbatim}
\end{quote}
%

Building \code{libstan.a}, \code{bin/stanc}, and \code{bin/print} need only be done once.

\section{Compiling and Executing a Model}

The rest of this quick-start guide explains how to code
and run a very simple Bayesian model.

\subsection{A Simple Bernoulli Model}

The following simple model is available in the source
distribution located at \code{<stan-home>} as
%
\begin{quote}
\nolinkurl{src/models/basic_estimators/bernoulli.stan}
\end{quote}
%
The file contains the following model.
%
\begin{quote}
\begin{Verbatim}
data { 
  int<lower=0> N; 
  int<lower=0,upper=1> y[N];
} 
parameters {
  real<lower=0,upper=1> theta;
} 
model {
  theta ~ beta(1,1);
  for (n in 1:N) 
    y[n] ~ bernoulli(theta);
}
\end{Verbatim}
\end{quote}
%
The model assumes the binary observed data \code{y[1],...,y[N]}
are i.i.d.\ with Bernoulli chance-of-success \code{theta}.  The
prior on \code{theta} is \code{beta(1,1)} (i.e., uniform).

\subsubsection{Implicit Uniform Priors}

If no prior is specified for a parameter, it is implicitly given a
uniform prior on its support.  For parameters such as \code{theta} in
the example, which are constrained to fall between 0 and 1, this
produces a proper uniform distribution on the support of \code{theta}.
Because $\distro{Beta}(1,1)$ is the uniform distribution, the
following sampling statement can be eliminated from the model without
changing the log probability calculation.
%
\begin{quote}
\begin{Verbatim}
  theta ~ beta(1,1);
\end{Verbatim}
\end{quote}

For parameters with unbounded support, the implicit uniform prior is
improper.  Stan allows improper priors to be specified in models, but
posteriors must be proper in order for sampling to succeed.


\subsubsection{Constraints on Parameters}

The variable \code{theta} is defined with lower and upper bounds,
which constrain its value.  Parameters with constrained support should
always specify appropriate constraints in the parameter declaration;
if the constraints are absent, sampling will either slow down or stop
altogether based on whether the initial values satisfy the constraints.

\subsubsection{Vectorizing Sampling Statements}

Iterations of the model will be faster if the loop over sampling
statements is \textit{vectorized} by replacing
%
\begin{quote}
\begin{Verbatim}
  for (n in 1:N) 
    y[n] ~ bernoulli(theta);
\end{Verbatim}
\end{quote}
%
with the equivalent vectorized form,
%
\begin{quote}
\begin{Verbatim}
  y ~ bernoulli(theta);
\end{Verbatim}
\end{quote}
%
Performance gains from vectorization are not because loops are slow in
Stan, but because calls to sampling statements are slow.
Vectorization allows multiple calls to a sampling statement to be
replaced with a single call that can share common calculations for the
log probability function, its gradients, and error checking.  For more
tips on optimizing the performance of Stan models, see
\refchapter{optimization}.


\subsection{Data Set}

A data set of $\mbox{\code{N}}=10$ observations is available in the file
%
\begin{quote}
\nolinkurl{src/models/basic_estimators/bernoulli.data.R}
\end{quote}
%
The content of the file is as follows.
%
\begin{quote}
\begin{Verbatim}
N <- 10
y <- c(0,1,0,0,0,0,0,0,0,1)
\end{Verbatim}
\end{quote}
%
This defines the contents of two variables, \code{N} and \code{y},
using an R-like syntax (see \refchapter{dump} for more information).



\subsection{Generating and Compiling the Model}

A single call to \code{make} will generate the \Cpp code for a
model with a name ending in \code{.stan} and compile it for
execution.  This call will also compile the library \code{libstan.a}
and the parser/code generator \code{stanc} if they have not already
been compiled.

First, change directories to \code{<stan-home>}, the directory where
Stan was unpacked that contains the file named \code{makefile} and
a subdirectory called \code{src/}.
%
\begin{quote}
\begin{Verbatim}[fontshape=sl]
> cd <stan-home>
\end{Verbatim}
\end{quote}
%
Then issue the following command:
%
\begin{quote}
\begin{Verbatim}[fontshape=sl]
> make src/models/basic_estimators/bernoulli 
\end{Verbatim}
\end{quote}
And yes, those are forward slashes in the make target for Windows.

The \code{make} command may be applied to files in locations
that are not subdirectories issued from another directory as follows.
Just replace the relative path \code{src/models/...} with the
actual path.

The C++ generated for the model and its compiled executable
form will be placed in the same directory as the model.


\subsection{Sampling from the Model}

The model can be executed from the directory in which it resides.
%
\begin{quote}
\begin{Verbatim}[fontshape=sl]
> cd src/models/basic_estimators 
\end{Verbatim}
\end{quote}
%
To execute sampling of the model under Linux or Mac, use
%
\begin{quote}
\begin{Verbatim}[fontshape=sl]
> ./bernoulli sample data file=bernoulli.data.R
\end{Verbatim}
\end{quote}
%
The \code{./} prefix before the executable is only required under
Linux and the Mac when executing a model from the directory in which
it resides.

For the Windows DOS terminal, the \code{./} prefix is not needed,
resulting in the following command.
%
\begin{quote}
\begin{Verbatim}[fontshape=sl]
> bernoulli sample data file=bernoulli.data.R
\end{Verbatim}
\end{quote}
%
Whether the command is run in Windows, Linux, or on the Mac, the
output is the same.  First, the parameters are echoed to the standard output,
which shows up on the terminal as follows.
%
\begin{quote}
\begin{Verbatim}[fontsize=\small]
 method = sample (Default)
   sample
     num_samples = 1000 (Default)
     num_warmup = 1000 (Default)
     save_warmup = 0 (Default)
     thin = 1 (Default)
     adapt
       engaged = 1 (Default)
       gamma = 0.050000000000000003 (Default)
       delta = 0.65000000000000002 (Default)
       kappa = 0.75 (Default)
       t0 = 10 (Default)
     algorithm = hmc (Default)
       hmc
         engine = nuts (Default)
           nuts
             max_depth = 10 (Default)
         metric = diag_e (Default)
         stepsize = 1 (Default)
         stepsize_jitter = 0 (Default)
 id = 0 (Default)
 data = src/models/basic_estimators/bernoulli.data.R
 init = 2 (Default)
 random
   seed = 1170969085
 output
   file = samples.csv (Default)
   append_sample = 0 (Default)
   diagnostic_file =  (Default)
   append_diagnostic = 0 (Default)
   refresh = 100 (Default)
...
\end{Verbatim}
\end{quote}
%
The ellipses (\code{...}) indicate that the output continues (as
described below).

Next, the sampler counts up the iterations in place, reporting
percentage completed, ending as follows.
%
\begin{quote}
\begin{Verbatim}[fontsize=\small]
...
Iteration: 2000 / 2000 [100%]  (Sampling)
...
\end{Verbatim}
\end{quote}

\subsubsection{Sampler Output}

Each execution of the model results in the samples from a single
Markov chain being written to a file in comma-separated value (CSV) format.
The default name of the output file is \nolinkurl{samples.csv}.

The first part of the output file just repeats the parameters
as comments (i.e., lines beginning with the pound sign (\Verb|#|)).
%
\begin{quote}
\begin{Verbatim}[fontsize=\small]
# stan_version_major = 1
# stan_version_minor = 3
# stan_version_patch = 0
# model = bernoulli_model
# method = sample (Default)
#   sample
#     num_samples = 1000 (Default)
#     num_warmup = 1000 (Default)
#     save_warmup = 0 (Default)
#     thin = 1 (Default)
#     adapt
#       engaged = 1 (Default)
#       gamma = 0.050000000000000003 (Default)
#       delta = 0.65000000000000002 (Default)
#       kappa = 0.75 (Default)
#       t0 = 10 (Default)
#     algorithm = hmc (Default)
#       hmc
#         engine = nuts (Default)
#           nuts
#             max_depth = 10 (Default)
#         metric = diag_e (Default)
#         stepsize = 1 (Default)
#         stepsize_jitter = 0 (Default)
# id = 0 (Default)
# data = src/models/basic_estimators/bernoulli.data.R
# init = 2 (Default)
# random
#   seed = 1170969085
# output
#   file = samples.csv (Default)
#   append_sample = 0 (Default)
#   diagnostic_file =  (Default)
#   append_diagnostic = 0 (Default)
#   refresh = 100 (Default)
...
\end{Verbatim}
\end{quote}
%
This is then followed by a header indicating the
names of the values sampled.
%
\begin{quote}
\begin{Verbatim}[fontsize=\small]
...
lp__,accept_stat__,stepsize__,treedepth__,theta
...
\end{Verbatim}
\end{quote}
%
The first column gives the log probability while the second, third,
and forth columns provide sampler-dependent information.
The single model parameter \code{theta} is stored in the fourth column.

\begin{center}
\begin{tabular}{l|l|l}
{\it Sampler} & {\it Parameter} & {\it Description} 
\\ \hline \hline
\HMC & \Verb| accept_stat__ | &  Metropolis acceptance probability
\\
\HMC & \Verb| stepsize__  | & Integrator step size
\\
\HMC & \Verb| int_time__  | & Total integration time
\\
\NUTS & \Verb| accept_stat__  | & Metropolis acceptance probability
\\
& & averaged over samples in the slice 
\\
\NUTS & \Verb| stepsize__ | & Integrator step size
\\
\NUTS & \Verb| treedepth__  | & Tree depth
\\
\end{tabular}
\end{center}

Next up is the result of adaptation, reported as comments.
%
\begin{quote}
\begin{Verbatim}[fontsize=\small]
...
# Adaptation terminated
# Step size = 1.81311
# Diagonal elements of inverse mass matrix:
# 0.415719
...
\end{Verbatim}
\end{quote}
%
This report says that NUTS step-size adaptation during warmup
settled on a step size of 1.80648.  The next two lines
indicate the multipliers for scaling individual parameters, here
just a single multiplier, 0.446024, corresponding to the single
parameter \code{theta}.

Samples from each iteration follow,%
%
\footnote{There are repeated entries due to the Metropolis accept step
in the No-U-Turn sampling algorithm.}
%
%
\begin{quote}
\begin{Verbatim}[fontsize=\small]
...
-6.85935,1,1.81311,0,0.194323
-6.85935,0.0001295,1.81311,0,0.194323
-6.79366,1,1.81311,0,0.288971
-6.77936,1,1.81311,0,0.282131
-6.77936,0.886654,1.81311,0,0.282131
-6.74812,1,1.81311,0,0.248232
-6.74812,0.0793306,1.81311,0,0.248232
-7.04438,0.777304,1.81311,0,0.353339
-7.04438,0.480879,1.81311,0,0.353339
-7.07619,0.849599,1.81311,0,0.158857
...
-6.74838,0.540129,1.81311,0,0.253368
-6.74838,0.638693,1.81311,0,0.253368
-6.74838,0.546,1.81311,0,0.253368
-7.01801,0.794591,1.81311,0,0.348376
-7.01801,0.0564743,1.81311,0,0.348376
-7.17267,0.863864,1.81311,0,0.375023
-6.86632,1,1.81311,0,0.192719
-6.86632,0.0077144,1.81311,0,0.192719
-8.21853,0.512368,1.81311,0,0.084075
-8.7342,0.862726,1.81311,0,0.0669754
...
\end{Verbatim}
\end{quote}
%

The output ends with timing details,%
\begin{quote}
\begin{Verbatim}[fontsize=\small]
...
#  Elapsed Time: 0.006811 seconds (Warm-up)
#                0.011645 seconds (Sampling)
#                0.018456 seconds (Total)
\end{Verbatim}
\end{quote}

\subsubsection{Summarizing Sampler Output}

The command-line program \code{bin/print} will display summary
information about the run (for more information, see
\refchapter{print-command}). To run \code{print} on the output file
generated for \code{bernoulli} on Linux or Mac use
%
\begin{quote}
\begin{Verbatim}[fontshape=sl]
> <stan-home>/bin/print samples.csv
\end{Verbatim}
\end{quote}
%
For Windows use backslashes for the executable,
%
\begin{quote}
\begin{Verbatim}[fontshape=sl]
> <stan-home>\bin\print samples.csv
\end{Verbatim}
\end{quote}
%
The output of the command will display information about the run
followed by information for each parameter and generated quantity. For
\code{bernoulli}, we ran 1 chain and saved 1000 iterations. The information is
echoed to the standard output stream.
%
\begin{Verbatim}[fontshape=sl,fontsize=\footnotesize]
Inference for Stan model: bernoulli_model
1 chains: each with iter=(1000); warmup=(0); thin=(1); 1000 iterations saved.

Warmup took (0.0066) seconds, 0.0066 seconds total
Sampling took (0.011) seconds, 0.011 seconds total

                 Mean     MCSE   StdDev        5%   50%   95%  N_Eff  N_Eff/s  R_hat
lp__             -7.3  3.5e-02  6.9e-01  -8.7e+00  -7.0  -6.7    390    34020   1.00
accept_stat__    0.64  1.2e-02  3.6e-01   5.1e-03  0.74   1.0    882    76898   1.00
stepsize__        1.8  7.8e-15  5.6e-15   1.8e+00   1.8   1.8   0.50       44   1.00
treedepth__     0.076  8.6e-03  2.7e-01   0.0e+00  0.00   1.0    942    82167   1.00
theta            0.25  4.2e-03  1.2e-01   9.0e-02  0.23  0.47    827    72146   1.00

Samples were drawn using hmc with nuts.
For each parameter, N_Eff is a crude measure of effective sample size,
and R_hat is the potential scale reduction factor on split chains (at 
convergence, R_hat=1).
\end{Verbatim}
%
In addition to the general information about the runs, \code{print}
displays summary statistics for each parameter and generated
quantity.

In the \code{bernoulli} model, there is a single parameter,
\code{theta}. The mean, standard error of the mean, standard
deviation, the 5\%, 50\%, and 95\% quantiles,
number of effective samples, and $\hat{R}$ value are displayed.
These quantities and their uses are described in detail in 
\refchapter{mcmc}.

The command \code{bin/print} can be called with more than one csv file
by separating filenames with spaces. It will also take wildcards in
specifying filenames. A typical usage of Stan from the command line
would first create one or more Markov chains by calling the model
executable, typically in parallel, writing the output CSV file for
each into its own directory.  Next, after all of the processes are
finished, the results would be analyzed using \code{print} to assess
convergence and inspect the means and quantiles of the fitted
variables.  Additionally, downstream inferences may be performed using
the samples (e.g., to make decisions or predictions for unseen data).

\subsection{Optimization}

Stan can be used for finding posterior modes as well as sampling from
the posterior.   The model does not need to be recompiled in order to
switch from optimization to sampling, and the data input format is the
same.  Although many command-line arguments may be provided to
configure the optimizer, the following minimal command suffices, using
defaults for everything but where to find the data file.
%
\begin{quote}
\begin{Verbatim}[fontshape=sl]
./bernoulli optimize data file=bernoulli.data.R
\end{Verbatim}
\end{quote}
%
which prints out the following.
%
\begin{quote}
\begin{Verbatim}[fontsize=\footnotesize]
 method = optimize
   optimize
     algorithm = bfgs (Default)
       bfgs
         init_alpha = 0.001 (Default)
         tol_obj = 1e-08 (Default)
         tol_grad = 1e-08 (Default)
         tol_param = 1e-08 (Default)
     iter = 2000 (Default)
     save_iterations = 0 (Default)
 id = 0 (Default)
 data
   file = bernoulli.data.R
 init = 2 (Default)
 random
   seed = 2907588507
 output
   file = samples.csv (Default)
   append_sample = 0 (Default)
   diagnostic_file =  (Default)
   append_diagnostic = 0 (Default)
   refresh = 100 (Default)

initial log joint probability = -10.9308
    Iter      log prob        ||dx||      ||grad||    alpha  # evals  Notes 
       7      -5.00402   3.67055e-07   3.06339e-11        1       10   
Optimization terminated normally: 
  Convergence detected: change in objective function was below
  tolerance
\end{Verbatim}
\end{quote}
%
The first part of the output reports on the configuration used, here
indicating the default BFGS optimizer, with default initial stepsize
and tolerances for monitoring convergence.  The second part of the
output indicates how well the algorithm fared, here converging and
terminating normally.  The numbers reported indicate that it took 7
iterations and 10 gradient evaluations, resulting in a final state
state where the change in parameters was roughly 3e-7 and the length
of the gradient roughly 3e-11.  The \code{alpha} value is for step
size used.  This is, not surprisingly, far fewer than required for
sampling; even fewer would be used with larger convergence tolerances.


\subsubsection{Output from Optimization}

The output from optimization is somewhat confusingly written into the
file \code{samples.csv} by default, because the default is global.
It follows the same pattern as the output for sampling, first dumping
the entire set of parameters used.
%
\begin{quote}
\begin{Verbatim}[fontsize=\small]
# stan_version_major = 2
# stan_version_minor = 0
# stan_version_patch = 0
# model = bernoulli_model
# method = optimize
#   optimize
#     algorithm = bfgs (Default)
#       bfgs
#         init_alpha = 0.001 (Default)
#         tol_obj = 1e-08 (Default)
#         tol_grad = 1e-08 (Default)
#         tol_param = 1e-08 (Default)
#     iter = 2000 (Default)
#     save_iterations = 0 (Default)
# id = 0 (Default)
# data
#   file = bernoulli.data.R
# init = 2 (Default)
# random
#   seed = 2907588507
# output
#   file = samples.csv (Default)
#   append_sample = 0 (Default)
#   diagnostic_file =  (Default)
#   append_diagnostic = 0 (Default)
#   refresh = 100 (Default)
lp__,theta
-5.00402,0.2000000000030634
\end{Verbatim}
\end{quote}
%
Note that everything is a comment other than a line for the header,
and a line for the values.  Here, the header indicates the
unnormalized log probability with \code{lp\_\_} and the model
parameter \code{theta}.  The maximum log probability is -5.0 and the
posterior mode for \code{theta} is 0.20.  The mode exactly matches
what we would expect from the data.  
%
\footnote{The Jacobian adjustment included for the sampler's log
  probability function is not applied during optimization, because it can
  change the shape of the posterior and hence the solution.}
%
Because the prior was uniform, the result 0.20 represents the maximum
likelihood estimate (MLE) for the very simple Bernoulli model.  Note
that no uncertainty is reported.  


\subsection{Configuring Command-Line Options}

The command-line options for running a model are detailed in
\refchapter{stan-cmd}. They can also be printed on the command line
using Linux or Mac OS with
%
\begin{quote}
\begin{Verbatim}[fontshape=sl]
> ./bernoulli help-all
\end{Verbatim}
\end{quote}
%
and on Windows with
%
\begin{quote}
\begin{Verbatim}[fontshape=sl]
> bernoulli help-all
\end{Verbatim}
\end{quote}
%
It may help to glance at the command-line skeletons in
\reffigure{nuts-command} through \reffigure{diagnostic-command} to get
a handle on the options then read the detailed descriptions earlier in
\refchapter{stan-cmd}.

\subsection{Testing Stan}

To run the Stan unit tests of basic functionality, run the
following commands from a shell (where \code{<stan-home>} is replaced
top-level directory into which Stan was unpacked; it should contain a
file named \code{makefile}).
%
\begin{quote}
\begin{Verbatim}[fontshape=sl]
> cd <stan-home>
> make O=0 test-unit
> make O=0 test-distributions
> make O=3 test-models
\end{Verbatim}
\end{quote}
%
Warnings can be safely ignored if the tests complete without a
\code{FAIL} error.

Code optimization is specified by the letter `O' followed by an equal
sign followed by the digit `0' for no optimization and `3' for more
optimization; optimization slows down compilation of the executable
but reduces its execution time.

\emph{Warning}: The unit test can take 30+ minutes and consume 3+ GB
of memory with the default compiler, \code{g++}.  It is faster
to run the Clang compiler (option \code{CC=clang++}), and to run in
multiple threads (option \code{-j4} for four threads).


