\documentclass[modern]{aastex62}

\input{xostyle}
\input{xosymbols}

\begin{document}\raggedbottom\sloppy\sloppypar\frenchspacing

\title{%
{\bf NOTE: this is a very rough draft of this manuscript --- read at your own risk!}
\\
\project{exoplanet}:
A toolkit for scalable inference for exoplanetary systems using transits,
radial velocities, \& astrometry
%An efficient and robust toolkit for gradient-based
%probabilistic characterization of exoplanets using transits and radial
%velocities
}

\author[0000-0002-9328-5652]{Daniel Foreman-Mackey}
\affil{Center for Computational Astrophysics, Flatiron Institute, 162 5th Ave, New York, NY 10010}

\author[0000-0002-1483-8811]{Ian Czekala}
\altaffiliation{NASA Hubble Fellowship Program Sagan Fellow}
\affiliation{Department of Astronomy, 501 Campbell Hall, University of California, Berkeley, CA 94720}

\author[0000-0002-0802-9145]{Eric Agol}\altaffiliation{Guggenheim Fellow}
\affil{Department~of~Astronomy, University~of~Washington, Seattle, WA 98195}
\affil{Virtual~Planetary~Laboratory, University~of~Washington, Seattle, WA 98195}

\author[0000-0002-0296-3826]{Rodrigo Luger}
\affil{Center for Computational Astrophysics, Flatiron Institute, 162 5th Ave, New York, NY 10010}

\begin{abstract}

As larger and more precise datasets continue to be collected for the discovery and characterization of exoplanets, we must also develop computationally efficient and rigorous methods for making inferences based on these data.
The efficiency and robustness of numerical methods for probabilistic inference
can be improved by using derivatives of the model with respect to the
parameters.
In this paper, I derive methods for exploiting these gradient-based methods
when fitting exoplanet data based on radial velocity, astrometry, and
transits.
Alongside the paper, I release an open source, well-tested, and
computationally efficient implementation of these methods.
These methods can substantially reduce the computational cost of fitting large
datasets and systems with more than one exoplanet.

\end{abstract}

\keywords{%
methods: data analysis ---
methods: statistical
}

\section{Outline}

\begin{enumerate}

{\item Introduction
\begin{itemize}
{\item the datasets available and forthcoming}
{\item existing tools for exoplanet fitting}
{\item motivation for gradients}
\end{itemize}}

\item Automatic differentiation and inference frameworks

\item Hamiltonian Monte Carlo and variants

\item Custom gradients required for exoplanet datasets
\begin{itemize}
\item Radial velocities and Kepler's equation
\item Astrometry
\item Transits
\end{itemize}

\item Implementation details and benchmarks

\item Examples

\item Discussion

\end{enumerate}

\section{Introduction}

The astronomical community has been investing in time domain survey astronomy and---thanks to this investment---large, homogeneous, public time-domain datasets are becoming common, and the questions that the community is asking are becoming more ambitious.
One application of these surveys is the discovery and characterization of stellar systems including multiple exoplanets and stars.
Sophisticated statistical methods have been developed for these applications and deployed at large scale, but the computational cost and scaling of these methods limit their application to restricted datasets or models with fewer data points or parameters.
This paper describes a new class of methods that have not previously been applied to time series analysis in astrophysics.
The key benefit of these methods is that they improve the computational scaling of robust probabilistic inference as a function of the number of parameters in the model; this improved efficiency can reduce runtime for inference in multi-planet systems by orders of magnitude.

The crucial difference between the methods discussed in this paper compared to previous work is that we derive computationally efficient first derivatives of the model with respect to its parameters.
Some work has been done to derive some of the relevant derivatives for specific sub problems (CITE) often with the goal of Fisher Information analysis to predict the performance of future surveys, but there is not a general framework that can propagate derivatives for (the majority of) the applications that exoplanet astronomers require.
To achieve this goal, we build on methods from applied mathematics and computer science, and work within a framework developed for machine learning.
The key concept is \emph{automatic differentiation} that allows efficient and (as the name suggests) automatic propagation of derivatives through complicated computer code.
Automatic differentiation has been widely applied across the sciences and it is the fundamental development that enabled the application of machine learning methods like deep learning.
There have been some applications within astrophysics, but its general application has been somewhat limited for both historical and practical reasons.
One reason for this is that models in astrophysics are often physically motivated and their calculation often involves special functions or numerical methods that have been developed and implemented in special purpose frameworks that can't be easily be differentiated or otherwise combined with existing automatic differentiation interfaces.
There has been progress made towards general purpose differentiation of physical systems (CITE examples of ODE solvers with variation), but there is still a significant amount of overhead when applying these methods for astrophysics applications.

This paper doesn't solve this problem, but it does describe the derivation and implementation of a large number of the physical models used in exoplanet characterization.
While the examples in this paper are (mostly) focused on the characterization of exoplanetary systems, it is also applicable to other time domain applications out of the box.
Futhermore, a pedagogical introduction to the relevant ideas is included to help enable other applications in astrophysics.

This paper is divided into three overarching ``chapters''.
The first presents an introduction to the ideas and algorithms that enable automatic differentiation and gradient-based inference.
Then the second chapter presents the derivation of the core models (and their derivatives) that are used by the \exoplanet\ project.
Finally, in the third chapter, we demonstrate the application of these methods on several problems in exoplanet astrophysics.

% This paper presents an example of how these tools and methods can be
% exploited in the specific astronomical application of fitting exoplanet
% datasets.
% We go through the derivation and implementation of the custom functions and
% their gradients that are required for gradient-based characterization of
% exoplanets.

% These methods and their implementation are not an all-in-one package designed
% to do the fitting.
% Instead, it provided the framework needed to use these tools within pipelines.

\begin{figure}[htbp]
\begin{centering}
\includegraphics[width=0.8\linewidth]{figures/gaussians.pdf}
\oscaption{gaussians}{\label{fig:gaussians}
This is a figure.}
\end{centering}
\end{figure}

\section{Automatic differentiation}

The main limitation of gradient-based inference methods is that, in order to
use them, you must \emph{compute the gradients}!
The fundamental quantity of interest is the first derivative (gradient) of the
log-likelihood function (or some other goodness-of-fit function) with respect
to the parameters of the model.
In all but the simplest cases, symbolically deriving this gradient function is
tedious and error-prone.
Instead, it is generally preferable to use a method (called automatic
differentiation) that can automatically compute the exact gradients of the
model \emph{to machine precision} at compile time.
We would like to emphasize that automatic differentiation does not compute numerical derivatives like finite difference methods, but it does not compute symbolic derivatives either.
Instead, it efficiently evaluates the numerical value of the derivative of a piece of code for some given inputs by implicitly applying the chain rule to each operation in the code.

The basic idea behind automatic differentiation is that code is always written
as the composition of basic functions and, if we know how to differentiate the
subcomponents, then we can apply the chain rule to differentiate the larger
function.
The realization that this method could be applied to neural network models was the key discoveries that launched the field of ``deep learning'', where a specific type of automatic differentiation is used and called ``backpropagation'' (CITE).
In many ways, motivated by this application, there has been substantial research and investment into makeing these methods general and computationally efficient.
Many open source libraries have been released that ease the use of these methods (e.g.\ TensorFlow, PyTorch, Stan, ceres, Eigen, etc.\ CITATIONS).
Despite the existence of these projects, automatic differentiation has not
been widely adopted in the astronomical literature because there is some
learning curve associated with porting existing code bases to a new language
or framework.
Furthermore, many projects in astronomy involve fitting realistic, physically
motivated models that involve numerical solutions to differential equations
or special functions that are not natively supported by the popular automatic
differentiation frameworks.

In this paper, we demonstrate how to incorporate exoplanet-specific functions
into these frameworks and derive the functions needed to characterize
exoplanets using gradient-based methods applied to radial velocity,
astrometry, and transit datasets.
Similar derivations would certainly be tractable for microlensing, direct
imaging, and other exoplanet characterization methods, but these are left to a
future paper.

\subsection{An introduction to automatic differentiation}

% https://alexey.radul.name/ideas/2013/introduction-to-automatic-differentiation/

First of all, we want to emphasize again that automatic differentiation is distinct from both numerical and symbolic differentiation.
It is a method for computing the \emph{value} of the derivative of a piece of code, without symbolically differentiating the operations.

\begin{deluxetable}{lll}
  \tablehead{\colhead{Forward pass} & \colhead{Forward tangent pass} & \colhead{Reverse pass}}
  \startdata
  $u_1 = x = \mathbf{2}$ & $\dot{u_1} = \mathbf{1}$ & \\
  $u_2 = y = \mathbf{-1}$ & $\dot{u_2} = \mathbf{0}$ & \\
  $u_3 = {u_1}^2 = \mathbf{4}$ &  $\dot{u_3} = 2\,u_1\,\dot{u_1} = \mathbf{4}$ & \\
  $u_4 = 2\,u_2 = \mathbf{-2}$ & $\dot{u_4} = 2\,\dot{u_2} = \mathbf{0}$ & \\
  $u_5 = u_3 + u_4 = \mathbf{2}$ & $\dot{u_5} = \dot{u_3} + \dot{u_4} = \mathbf{4}$ & \\
  $u_6 = \sqrt{u_5} = \mathbf{1.414}$ & $\dot{u_6} = \dot{u_5} / 2\,\sqrt{u_5} = \mathbf{1.414}$ & \\
  $u_7 = \sin{u_1} = \mathbf{0.909}$ & $\dot{u_7} = \dot{u_1}\,\cos{u_1} = \mathbf{-0.416}$ & \\
  $u_8 = u_1\,u_7 = \mathbf{1.819}$ & $\dot{u_8} = u_1\,\dot{u_7} + \dot{u_1}\,u_7 = \mathbf{0.077}$ & $\overline{u_1} = \overline{u_8}\,u_7 = \mathbf{-0.909}$, $\overline{u_7} = \overline{u_8}\,u_1 = \mathbf{-2}$ \\
  $u_9 = u_6 - u_8 = \mathbf{-0.404}$ & $\dot{u_9} = \dot{u_6} - \dot{u_8} = \mathbf{1.337}$ & $\overline{u_6} = \overline{u_9} = \mathbf{1}$, $\overline{u_8} = -\overline{u_9} = \mathbf{-1}$ \\
  & & $\overline{u_9} = \mathbf{1}$ \\
  \enddata
\end{deluxetable}


Example:
\begin{eqnarray}
f(x) = u(x)\,v(x)
\end{eqnarray}
\begin{eqnarray}
  \frac{\dd f(x)}{\dd x} = \frac{\dd u(x)}{\dd x}\,v(x) + u(x)\,\frac{\dd v(x)}{\dd x}
\end{eqnarray}
Instead, define $x \to x + \epsilon\,\delta x$ where $\epsilon^2 = 0$.
Therefore,
\begin{eqnarray}
  u(x + \epsilon\,\delta x) = u(x) + \frac{\dd u(x)}{\dd x}\,\epsilon\,\delta x
\end{eqnarray}
This equality is exact because of our definition that $\epsilon^2 = 0$.
Therefore,
\begin{eqnarray}
f(x) + \epsilon\,\delta f &=& u(x + \epsilon\,\delta x)\,v(x + \epsilon\,\delta x) \\
&=& \left[u(x) + \frac{\dd u(x)}{\dd x}\,\epsilon\,\delta x\right]\,
    \left[v(x) + \frac{\dd v(x)}{\dd x}\,\epsilon\,\delta x\right] \\
&=& u(x)\,v(x) + \epsilon\,\left[\frac{\dd u(x)}{\dd x}\,v(x) + u(x)\,\frac{\dd v(x)}{\dd x}\right]\,\delta x
\end{eqnarray}

The basic idea is that, at its roots, a computer program is the composition of simpler operations.
If the derivatives of the simplest operations are known, the automatic differentiation library can automatically apply the chain rule to each step of the calculation to accumulate the derivative of the full program.

\subsection{Choice of modeling framework}

At present, there are many model-building libraries available in every programming language, and it is not clear which should be preferred.
Since it is a popular language for code development in astrophysics, we focus on libraries with interfaces implemented in the \python\ programming language.
The two most popular libraries for model building in \python\ are the well-supported and actively-developed packages \tensorflow\ \citep{Abadi:2016} and \pytorch\ \citep{Paszke:2017}.
Each of these libraries come with extensions that enable probabilistic modeling and gradient-based inference: \project{tensorflow-probability} \citep{Dillon:2017} and \project{Pyro} \citep{Bingham:2018}.
Both of these libraries (\tensorflow\ and \pytorch) and their inference engines are built for high performance machine learning and, while many of those features are also useful for our purposes, some of the inference goals in astrophysics are somewhat different.
For example, the primary posterior inference method implemented by these packages is variational inference \dfmtodo{(CITE)}.
\dfmtodo{We discuss the application of variational inference later in this paper}, but the primary inference method used in astrophysics is Markov chain Monte Carlo (MCMC) and the implementations of gradient-based MCMC (HMC, NUTS etc) within these packages is not as fully featured as the state-of-the-art MCMC inference packages like \pymc\ \citep{Salvatier:2016} and \stan\ \citep{Carpenter:2015, Carpenter:2017}.
So, for the purposes of this paper, we restrict our focus to \pymc\ and its modeling engine \theano\ \citep{Theano-Development-Team:2016}.

\theano\ is a model building framework for \python\ that, like \tensorflow, provides an interface for defining a computational graph that can be used to efficiently compute a model and its derivatives.
It was originally developed for deep learning, but it was adopted by the \pymc\ project and extended to support probabilistic modeling and gradient-based inference methods like Hamiltonian Monte Carlo, no U-turn sampling, and variational inference.
In our experiments, the combination of \theano\ and \pymc\ provided the best balance of modern inference features, ease of use, and computational efficiency of all the existing \python-first modeling frameworks.
The \stan\ project provides a more mature and feature-rich set of inference methods, but it is more difficult to build and debug the complicated physical models that are required by exoplanet applications using the \stan\ modeling language.
Furthermore, as discussed below, we must provide custom astronomy-specific operations as part of this library and, while it is possible to extend the \stan\ math library, we found it to be much easier to develop, test, and release the necessary features using \pymc\ and \theano.


\section{Inference methods}

Gradient-based methods can be applied to both optimization and inference tasks.
For example, the task of finding the maximum likelihood or maximum a posteriori parameters can be significantly accelerated using the gradient of the objective with respect to the model parameters.
While computing the gradient will be somewhat more expensive than computing only the value of the function, it is generally significantly more efficient than using finite difference methods.
Furthermore, since the gradients computed using automatic differentiation is exact, unlike finite difference which will suffer from numerical issues unless the step size is carefully tuned, the lack of numerical noise generally reduces the number of steps the optimizer must take before reaching the optimum.

\subsection{Optimization}

One standard inference procedure is maximum likelihood or maximum a posteriori inference by optimization.
In this case, the objective (either the likelihood or posterior function) is maximized (numerically) with respect to the input parameters.
Operations like this are commonly performed within the astronomical literature (CITE) and gradient-free methods are commonly used for this purpose (LM, NM).
These methods can work reasonably well in low dimensions when the objective is well behaved, but this performance generally degrades rapidly with the number of parameters.
In other scientific fields, gradient-based optimization is the industry standard (CITE), and these methods have been demonstrated to scale to large parameter spaces.
In astronomy, it is now common to use the \project{BFGS} optimization routine (CITE) combined with finite difference estimates of the derivatives.
These methods can be applied to larger problems, but the computational cost of evaluating the model scales with the number of dimensions and the results are generally sensitive to the choice of step size in the finite difference.
Furthermore, the numerical noise introduced by the finite difference calculations reduces the performance of the optimization routine and increases the number of model evaluations needed to reach the optimum.

In this paper, we describe the computations needed to compute the required gradients using backpropagation and these derivatives can be used to substantially improve the performance of optimization applied to exoplanet datasets.

\dfmtodo{Make an example.}

For example, we computed the number of function evaluations and total run time needed to find an estimate of the maximum likelihood parameters for a simulated radial velocity data set and compared the precision of this estimate to the \dfmtodo{true} maximum.
In this case, finite difference required \dfmtodo{how many?} as many more function evaluations.

Similarly, we compare the performance of finite difference and exoplanet for optimizing the parameters for a transiting exoplanet.
In this case... \dfmtodo{finish this}.

\subsection{Sampling}

A popular method for evaluating posterior integrals is to us Markov chain Monte Carlo (MCMC).
These methods can be substantial accelerated using gradients of the log probability with respect to the input parameters using Langevin or Hamiltonian methods.

\dfmtodo{Talk about Langevin methods? CITE things}



\subsection{Variational inference}

\section{Orbital conventions}
A general introduction to Keplerian orbits is given by \citet{Murray:2010}; here we describe the implementation choices specific to the \emph{exoplanet} codebase.
We follow a set of internally consistent orbital conventions that also attempts to respect as many of the established conventions of the stellar and exoplanetary fields as possible. The location of a body on a Keplerian orbit with respect to the center of mass is given in the perifocal plane by
\begin{equation}
  \boldsymbol{r} =
  \left [
    \begin{array}{c}
      x\\
      y\\
      z\\
    \end{array}
  \right ].
\end{equation}
By construction, the orbit of the body in the perifocal frame is constrained entirely to the $x -y$ plane (see Figure~\ref{fig:orbit3D}). The range of orbital convention choices mainly stems from how the location of the body in the perifocal frame is converted to the observer frame,
\begin{equation}
  \boldsymbol{R} =
  \left [
    \begin{array}{c}
      X\\
      Y\\
      Z\\
    \end{array}
  \right ].
\end{equation}
We choose to orient the $\hat{\boldsymbol{X}}$ unit vector in the north direction, $\hat{\boldsymbol{Y}}$ in the east direction, and $\hat{\boldsymbol{Z}}$ towards the observer. Under this convention, the inclination of an orbit is defined by the dot-product of the orbital angular momentum vector with $\hat{\boldsymbol{Z}}$, conforming to the common astrometric convention that a zero inclination ($i = 0\degr$) orbit is face-on, with the test body orbiting in a counter-clockwise manner around the primary body.

\begin{figure}[htbp]
  \begin{centering}
  \includegraphics{figures/orbit3D.pdf}
  \oscaption{orbit3D}{\figurelabel{orbit3D}
  The conventions used to orient the perifocal frame ($x, y, z$) relative to the observer frame ($X,Y,Z$).}
  \end{centering}
\end{figure}

In the stellar and exoplanetary fields, there is less agreement on which  segment of the line of nodes is defined as the \emph{ascending} node. We choose to define the ascending node as the point where the orbiting body crosses the plane of the sky moving \emph{away} from the observer (i.e., crossing from $Z > 0$ to $Z< 0$). This convention has historically been the choice of the visual binary field; the opposite convention occasionally appears in exoplanet and planetary studies.

To implement the transformation from perifocal frame to observer frame, we consider the rotation matrices
\begin{equation}
  \boldsymbol{P}_x(\phi) = \left [
  \begin{array}{ccc}
    1 & 0 & 0 \\
    0 & \cos \phi & - \sin \phi \\
    0 & \sin \phi & \cos \phi \\
    \end{array}\right]
\end{equation}
\begin{equation}
  \boldsymbol{P}_z (\phi) = \left [
  \begin{array}{ccc}
    \cos \phi & - \sin \phi & 0\\
    \sin \phi & \cos \phi & 0 \\
    0 & 0 & 1 \\
    \end{array}\right].
\end{equation}
which result in a \emph{clockwise} rotation of the axes, as defined using the right hand rule.\footnote{This means when we look down the $z$-axis, for a positive angle $\phi$, it would be as if the $x$ and $y$ axes rotated clockwise.
In order to find out what defines counter-clockwise when considering the other rotations, we look to the right hand rule and cross products of the axes unit vectors. Since $\hat{\bm x} \times \hat{\bm y} = \hat{\bm z}$, when looking down the $z$ axis the direction of the $x$-axis towards the $y$-axis defines counter clockwise.
Similarly, we have $\hat{{\bm y}} \times \hat{{\bm z}} = \hat{{\bm x}}$, and $\hat{{\bm z}} \times \hat{{\bm x}} = \hat{{\bm y}}$.}

To convert a position $\boldsymbol{r}$ in the perifocal frame to the observer frame $\boldsymbol{R}$ (referencing Figure~\ref{fig:orbit3D}), three rotations are required: (i) a rotation about the $z$-axis through an angle $\omega$ so that the $x$-axis coincides with the line of nodes at the ascending node (ii) a rotation about the $x$-axis through an angle ($-i$) so that the two planes are coincident and finally (iii) a rotation about the $z$-axis through an angle $\Omega$. Applying these rotation matrices yields
\begin{equation}
  \boldsymbol{R} =
  \boldsymbol{P}_z (\Omega) \boldsymbol{P}_x(-i) \boldsymbol{P}_z(\omega)
  \boldsymbol{r}.
\end{equation}

As a function of true anomaly $f$, the position in the observer frame is given by
\begin{equation}
  \begin{array}{lc}
    X =& r [ \cos \Omega (\cos \omega \cos f - \sin \omega \sin f)  - \sin \Omega  \cos i (\sin \omega \cos f + \cos \omega \sin f) ] \\
    Y =& r [ \sin \Omega (\cos \omega \cos f - \sin \omega \sin f) + \cos \Omega \cos i(\sin \omega \cos f + \cos \omega \sin f) ] \\
    Z =& - r \sin i (\sin \omega \cos f + \cos \omega \sin f).\\
\end{array}
\label{eqn:Z}
\end{equation}

Simplifying the equations using the sum angle identities, we find
\begin{equation}
  \begin{array}{lc}
    X =& r (\cos \Omega \cos(\omega + f) - \sin(\Omega) \sin(\omega + f) \cos(i)) \\
    Y =& r (\sin \Omega \cos(\omega + f) + \cos(\Omega) \sin(\omega + f) \cos(i)) \\
    Z =& - r \sin(\omega + f) \sin(i).\\
\end{array}
\label{eqn:Z}
\end{equation}

\section{Parameterizing an orbit}

At is heart, \exoplanet\ models Keplerian orbits.
There is some support for relaxing this assumption (REFERENCE N-BODY W REBOUND), but it is important that we clearly present how to represent a Keplerian orbit before generalizing.
In \exoplanet\ these orbits are specified specified with respect to a single central body (generally the most massive body) and then a system of non-interacting bodies orbiting the central.
This is a good parameterization for exoplanetary systems and binary stars, but it is sometimes not sufficient for systems with multiple massive bodies where the interactions will be important to the dynamics.

The central body is defined by any two of radius $R_\star$, mass $M_\star$, or density $\rho_\star$.
The third physical parameter can always be computed using the other two so \exoplanet\ will throw an error if three are given (even if they are numerically self-consistent).

The orbits are then defined by:
\begin{itemize}
  \item the period $P$ or semi-major axis $a$ of the orbit,
  \item the time of conjunction $t_0$ or time of periastron passage $t_\mathrm{p}$,
  \item the planet's radius $R_\mathrm{pl}$ and mass $M_\mathrm{pl}$,
  \item the eccentricity $e$, argument of periastron $\omega$ and the longitude of ascending node $\Omega$, and
  \item the inclination of the orbital plane $i$ or the impact parameter $b$.
\end{itemize}

A \texttt{KeplerianOrbit} object in \exoplanet\ will always be fully specified.
For example, if the orbit is defined using the period $P$, the semi-major axis $a$ will be computed using Kepler's third law.

\section{Parameterizing transiting planets}

It is common practice to parameterize a transiting planet using period $P$, time of conjunction $t_0$, and impact parameter $b$.
If parameters are well constrained then specific choices about priors won't be important, but the eccentricity and argument of periastron are rarely well constrained by the transit alone, so it can be important to consider the details of the parameterization.

From geometry, we can derive that expected prior distribution for $\cos i$ is uniform from $-1$ to $1$.
This doesn't directly translate to a uniform prior on $b$.
Instead, if we want to sample in $b$ we need to take into account the Jacobian in the change of variables
\begin{eqnarray}
p(\cos i) &=& \left| \frac{\dd b}{\dd \cos i} \right|\,p(b) \quad.
\end{eqnarray}
Since,
\begin{eqnarray}
b = \frac{a}{R_\star}\,\cos i\,\left[\frac{1 - e^2}{1 + e\,\sin\omega}\right]\quad,
\end{eqnarray}
the relevant Jacobian is
\begin{eqnarray}
\frac{\mathrm{d}b}{\mathrm{d}\cos i} = \frac{R_\star}{a}\,\left[\frac{1 + e\,\sin\omega}{1 - e^2}\right] \quad.
\end{eqnarray}

Similarly, in order for a transit to occur, the impact parameter must satisfy
\begin{eqnarray}
  \left|b\right| < 1 + R_\mathrm{pl}/R_\star \quad.
\end{eqnarray}
In other words, the conditional prior probability for the impact parameter (conditioned on the assumption that the planet transits and the other orbital parameters) is
\begin{eqnarray}
p(b\,|\,a,\,e,\,\omega,\,R_\mathrm{pl},\,R_\star) =
\frac{R_\star}{a}\,\left[\frac{1 + e\,\sin\omega}{1 - e^2}\right]\,
\left\{\begin{array}{ll}
  1 / [2\,(1 + R_\mathrm{pl}/R_\star)] & \mathrm{if}\,\left|b\right| < 1 + R_\mathrm{pl}/R_\star \\
  0 & \mathrm{otherwise}
\end{array}\right.
\end{eqnarray}

Similarly, the prior on the planet's radius should also depend on the detectability of the signal, but this will only be important for small planets with radius error bars that span the detection threshold which won't be common.

As discussed by \dfmtodo{CITE exofast}, there is a strong degeneracy between $e$, $b$ and $\omega$ that is introduced by the fact that the duration is the main observable that constrains these three quantities.
While it is possible to change variables to fit in duration instead of $e$ and analytically marginalize over the branches of the solution for $e$, both the bookkeeping and reparameterization is difficult to get right.
In particular, NUTS requires that all parameters have infinite support, but the allowed range of the duration parameter depends on the branch and the other parameters.
Furthermore, there is a funnel situation since at some values of $\omega$, all values of $e$ are identical.

\section{The custom operations provided by exoplanet}

\subsection{A solver for Kepler's equation}

A key assumption that is often made when fitting exoplanets is that the
planets are orbiting the central star on independent bound Keplerian orbits
(CITE).
To compute the coordinates of a planet on its orbit, we must solve Kepler's
equation
\begin{eqnarray}
M &=& E - e\,\sin E
\end{eqnarray}
for the eccentric anomaly $E$ at fixed eccentricity $e$ and mean anomaly $M$.
This equation must be solved numerically and there is a rich literature
discussing methods for this implementation.
We have found the method from CITE to provide the right balance of numerical
stability (with a relative accuracy of about $10^{-20}$ for all values of $M$
and $e$) and computational efficiency.

As well as solving Kepler's equation, we must also efficiently evaluate the
gradients (or, more precisely, \emph{back-propagate} the gradients) of $E$
with respect to $e$ and $M$.
After solving for $E$, this can be computed efficiently and robustly using
implicit differentiation.
For numerical methods like this, it is often possible to compute the gradients
without applying automatic differentiation to the implementation directly and
that is generally preferred for the purposes of numerical stability.
In this case, the relevant gradients are
\begin{eqnarray}
\frac{\dd E}{\dd e} &=& \frac{\sin E}{1 - e\,\cos E} \\
\frac{\dd E}{\dd M} &=& \frac{1}{1 - e\,\cos E} \quad.
\end{eqnarray}

In practice, we have found it more efficient to fuse the calculations of the eccentric anomaly $E$ and the true anomaly $f$ working only with the sine and cosines of these quantites.
To compute the true anomaly for a given eccentric anomaly, the relation is
\begin{equation}
\tan\frac{f}{2} = \sqrt{\frac{1 + e}{1-e}}\tan\frac{E}{2}
\end{equation}
and we can compute this efficiently using the following trigonometric identities
\begin{eqnarray}
\tan \frac{E}{2} &=& \frac{\sin E}{1 + \cos E} \\
\sin f &=& \frac{2\,\tan f/2}{1 + \tan^2 f/2} \\
\cos f &=& \frac{1 - \tan^2 f/2}{1 + \tan^2 f/2}
\end{eqnarray}
with the special case $\sin f = 0$ and $\cos f = -1$ when $\cos E = -1$.

Then, the relevant derivatives can be evaluated using the results ($\sin f$ and $\cos f$) of the forward solve:
\begin{eqnarray}
\frac{\dd f}{\dd M} &=&  \frac{\left(1 + e\,\cos f\right)^2}{\left(1-e^2\right)^{3/2}} \\
\frac{\dd f}{\dd e} &=& \frac{\left(2 + e\,\cos f\right)\,\sin f}{1 - e^2} \quad.
\end{eqnarray}
\dfmtodo{Add a plot showing the relative cost of the forward and reverse evaluation.}

\subsection{Transit light curves}

Another custom operation included as part of \exoplanet\ is for calculating model light curves for the transits (and occultations) including limb-darkening and exposure time integration.
This core functionality is provided by the code \starry\ \citep{Luger:2019} \dfmtodo{cite limbdark}.
Briefly, \starry\ represents the star using a spherical harmonic representation of its surface brightness and then uses this representation to analytically integrate the surface of the star to compute the model light curve and the derivatives with respect to the physical parameters.
This algorithm is a generalization of the popular quadratic limb darkening model \citep{Mandel:2002} to a much more flexible class of stellar surface maps \dfmtodo{cite other limb darkening models}.
For our purposes, \starry's key feature is the fact that it not only computes the model light curve (as has been done previously \dfmtodo{cite}), but also the derivatives of this model with respect to the parameters of the system (the orbital parameters and the spherical harmonic coefficients).
In the case of pure limb darkening, these derivatives are calculated analytically following \dfmtodo{cite limbdark} while, in the more general case, the derivatives are accumulated using automatic differentiation in the \cpp\ code (using Eigen \dfmtodo{cite Eigen}).
\dfmtodo{Make a plot of the gradient light curves.}
This means that \starry\ can be included within the backpropagation framework needed by \exoplanet.

From an interface perspective, \starry\ is included as a set of custom \theano\ operations that compute the matrices in the \starry\ equation as a function of the sky coordinates of the bodies in the system and the spherical harmonic coefficients.
The linear algebra is performed directly in \theano.
This means that the gradients are efficiently backpropagated using the linear algebra routines provided by \theano.

\paragraph{Exposure time integration}
In many cases, the finite exposure time integration of transit light curves must be included in the model light curve in order to get correct constraints on the physical properties of the system.
\exoplanet\ includes several routines for integrating the model.
First, it includes an oversampling routine as recommended by \dfmtodo{cite kipping}, including generalizations to higher order integration schemes.
This has the benefit that it is simple to use and can generally be applied to any model light curve.
However, this method also introduces significant memory overhead because each data point requires many (at least tens) of model evaluations and these must be stored in memory for the purposes of backpropagation.

Because of this shortcoming, \exoplanet\ also implements an adaptive exposure time integration routine directly in \cpp\ that can be used for Keplerian orbits.
In this case, the time integral is performed using an adaptive Simpson's rule \dfmtodo{cite} directly in the backend, reducing the computational cost and memory overhead.
This routine works by recursively subdividing the exposure interval until a parabolic approximation to the flux in the subexposure matches the computed flux to a specified tolerance.
The derivative of this integral is accumulated using during the forward pass.

The second aspect of exposure time integration incorporated into \exoplanet\
applies to the Gaussian Process kernel.  The standard \celerite\ kernel assumes that each
exposure time is infinitesimal in duration.  This is an adequate approximation
if the kernel varies slowly and smoothly across an exposure time;
however, the finite exposure time modifies the kernel significantly when it is
comparable to, or longer than, any of the timescales occurring in the components of
the kernel.  In appendix \ref{sec:time_integration} we present novel expressions for
the time-integrated \celerite\ kernel, which retains the necessary properties
for fast computation when the exposures times do not overlap.

\paragraph{Performance}
As demonstrated in the papers describing this algorithm \citep{Luger:2019} \dfmtodo{cite limbdark}, the performance of the algorithms implemented in \starry\ is competitive with the other standard light curve implementations \dfmtodo{cite}.
Figure \dfmtodo{whatever} demonstrates this in the context of \exoplanet.
\dfmtodo{Make a runtime figure and explain it.}


\subsection{Scalable Gaussian Processes for time series}

\exoplanet\ also includes an implementation of scalable Gaussian Processes (GPs) using the \celerite\ algorithm \citep{Foreman-Mackey:2017}.
To make this algorithm compatible with the \theano\ automatic differentiation framework, we also derived the backpropagation functions for \celerite\ \citep{Foreman-Mackey:2018a}.
We implemented these functions in \cpp\ as a custom \theano\ operation so that they can be seamlessly integrated into a \pymc\ probabilistic model.

\celerite\ provides an algorithm for performing exact inference with GP models that scales linearly with the number of data points, instead of the typical cubic scaling.
This scaling is achieved by exploiting structure in the covariance matrix when the covariance function is restricted to be a specific stationary form, with one-dimensional inputs.
The observations can, however, be un-evenly sampled and heteroschedastic without any loss of accuracy or computational performance.
As discussed by \citet{Foreman-Mackey:2017}, this covariance function can be interpreted as the covariance generated by a mixture of stochastically-driven, damped simple harmonic oscillators, which makes this a good model for the variability of stars in the time domain.
\exoplanet\ provides an interface for constructing covariance functions that satisfy the constraints of the \celerite\ algorithm and for solving the relevant linear algebra.

This operation is useful for many exoplanet applications where the datasets are large enough that traditional GP modeling is intractable.
For example, the time series for a typical short cadence target from the \tess\ mission has a light curve with about 20,000 data points.
Evaluating a GP model on these data using high-performance general linear algebra libraries would take \dfmtodo{how long?}.

\section{Distributions}

\exoplanet\ also comes equipped with several custom ``distributions'' that can be used to simplify modeling for exoplanet specific applications, but some also have broader applicability.

\subsection{General distributions}

\paragraph{Angles}

There are several angles that must be included in exoplanet applications.
For example, the argument of periastron and the phase of the orbit.
This can be difficult to sample using most MCMC algorithms because of the periodic boundary conditions of the parameter space.
Therefore, for angles the argument of periastron $\omega$ that have a related amplitude (eccentricity $e$ in this case), it is common practice to fit for $e$ and $\omega$ together using
\begin{eqnarray}
h = \sqrt{e}\,\sin \omega & \quad \mathrm{and}\quad & k = \sqrt{e}\,\cos \omega
\end{eqnarray}
as the parameters with the constraint $h^2 + k^2 < 1$.
It has been demonstrated (CITE) that this still imposes a uniform prior on both $e$ and $\omega$.
This is not a good parameterization, however, when using gradient based inference schemes because these do not do a good job of handling hard constraints $0 \le e < 1$.
Instead, for angular parameters, we sample in
\begin{eqnarray}
n_1 \sim \mathcal{N}(0,\,1) & \quad \mathrm{and}\quad & n_2 \sim \mathcal{N}(0,\,1)
\end{eqnarray}
where $\mathcal{N}(0,\,1)$ is the standard normal and compute the angle as $\omega = \mathrm{arctan2}(n_1,\,n_2)$.
This also produces a uniform prior on $\omega$, but there are no longer any hard boundaries in the problem, at the cost of one extra parameter.

One issue that arises with this parameterization is that the joint posterior in $n_1$ and $n_2$ can become ``funnel shaped'' when the angle is well constrained.
This dynamic range can cause serious performance issues for gradient-based samplers (CITE)
.
To deal with this, we regularize the density as
\begin{eqnarray}
\log p (n_1,\,n_2) = -\frac{{n_1}^2 + {n_2}^2}{2} + \alpha\,\log({n_1}^2 + {n_2}^2) + \mathrm{const}
\end{eqnarray}
where $\alpha$ is a parameter that controls the strength of the regularization.
We find that a default value of $\alpha = 10$ gives good performance in most cases.

\paragraph{Unit vector}

Another generally useful distribution is the distribution over unit vectors.
We follow (CITE Stan) and include a custom distribution for \pymc\ that provides support for unit vectors by reparameterizing as $\bvec{n} \sim \mathcal{N}(\bvec{0},\,\bvec{1})$ and computing the unit vector as $\bvec{u} = \bvec{n} / || \bvec{n} ||_{2}$.


\subsection{Quadratic limb darkening parameters}

\exoplanet\ has support for limb darkening profiles of arbitrary order as described in LIMB DARK, but a commonly used model is the quadratic limb darkening model so exoplanet includes some features to make this easier.
The algorithm for efficiently evaluating the light curve for a transit of a quadratically limb darkened star was first presented by \citet{Mandel:2002}, but the light curve model in \exoplanet\ is evaluated following Agol et al.\ including the performance and numerical stability improvements described in that paper.
The form of the limb darkening law is
\begin{equation}
I(\mu)/I(1) = 1 - u_1\,(1 - \mu) - u_2\,(1 - \mu)^2
\end{equation}
where $\mu = \sqrt{1 - b^2}$ and $b$ is the distance from the center of the star in units where the stellar radius is one.
For this limb darkening profile to be everywhere positive and monotonically decreasing as a function of $b$, the parameters $u_1$ and $u_2$ must satisfy the following relations
\begin{eqnarray}
u_1 + u_2 &<& 1 \\
u_1 &>& 0 \\
u_1 + 2\,u_2 &>& 0
\end{eqnarray}
It has been demonstrated \citep[by][]{Kipping:2013} that these constraints can be enforced by reparameterizing as
\begin{eqnarray}
q_1 &=& (u_1 + u_2)^2 \\
q_2 &=& \frac{u_1}{2\,(u_1 + u_2)}
\end{eqnarray}
with the simpler constraints
\begin{eqnarray}
0 < q_1 < 1 \\
0 < q_2 < 1
\end{eqnarray}
\citet{Kipping:2013} argues that this is an ``uninformative'' parameterization because the log determinant of the Jacobian of the transformation is zero, \dfmtodo{discuss details of this:} but it's worth noting that the marginal priors $p(u_1)$ and $p(u_2)$ are not uniform.

In \exoplanet, we make one further change of variables to
\begin{eqnarray}
\tilde{q} &=& \log(q) - \log(1-q)
\end{eqnarray}
where $\tilde{q}$ has support across the full real line.
This is not an uninformative change of variables so we must take this into account by adding the log determinant of the Jacobian to the log probability.
For this change of variables, the correction is
\begin{eqnarray}
\log\left(\frac{\dd q}{\dd\tilde{q}}\right) &=& \log q + \log(1 - q) \quad.
\end{eqnarray}

\subsection{Radius and impact parameter}

A parameterization has been proposed to parameterize the radius ratio $r$ and impact parameter $b$ to be sampled uniformly within the trapezoid $r_\mathrm{min} < r < r_\mathrm{max}$ and $0 < b < 1 + r$ (CITE).
This parameterization is included within \exoplanet, but implies a non-uniform marginal for radius ratio (\dfmtodo{make plot}).

The marginal density for the Espinoza distribution is
\begin{eqnarray}
p(r\,b) &=& \left\{\begin{array}{ll}
1/A & \mathrm{if}\,r_\mathrm{min} \le r < r_\mathrm{max}\,\mathrm{and}\,b\le 1+r\\
0 & \mathrm{otherwise}
\end{array} \right.
\end{eqnarray}
where
\begin{eqnarray}
A &=& \int_{r_\mathrm{min}}^{r_\mathrm{max}} \int_{0}^{1+r}\dd b\,\dd r \\
&=& \int_{r_\mathrm{min}}^{r_\mathrm{max}} (1 + r)\,\dd r\\
&=& (r_\mathrm{max} - r_\mathrm{min}) + \frac{1}{2}\,(r_\mathrm{max}^2 - r_\mathrm{min}^2)
\end{eqnarray}
therefore
\begin{eqnarray}
p(r) &=& \frac{1}{A}\,\int_{0}^{1+r}\dd b \\
&=& \left\{\begin{array}{ll}
  (1+r)/A & \mathrm{if}\,r_\mathrm{min} \le r < r_\mathrm{max} \\
  0 & \mathrm{otherwise}
\end{array}\right.
\end{eqnarray}
This means that the inference will be biased towards measuring large radius planets.
Instead, we argue that the less informative prior would be uniform in log radius or log radius ratio.
Therefore, we recommend reparameterizing as
\begin{eqnarray}
r &\sim& \mathrm{LogUniform}(r_\mathrm{min},\,r_\mathrm{max}) \\
b &\sim& \mathrm{Uniform}(0,\,1+r)
\end{eqnarray}
which will introduce less bias when inferring radius ratios.
This can be efficiently sampled using \exoplanet.

\section{Examples}



\appendix

\section{Solving Kepler's equation}

A crucial component for all of the models in this paper is an accurate and
high-performance solver for Kepler's equation
\begin{eqnarray}\eqlabel{kepler-equation}
M &=& E - e\,\sin E \quad.
\end{eqnarray}
We use the method presented by \citet{Nijenhuis:1991}.
Unlike most of the methods used in astrophysics, this algorithm \emph{is not
iterative}.
This can lead to much better compiler optimization.
Furthermore, this algorithm doesn't require the evaluation of trigonometric
functions directly.
As well as improving performance, this also increases the accuracy of the
results because it avoids catastrophic cancellations when evaluating $E - \sin
E$ and $1 - \cos E$ by evaluating these differences directly using a series
expansion.

The procedure is as follows:
\begin{enumerate}

\item Initial estimates are selected for the parameters by partitioning the
parameter space into 4 regions and applying a set of heuristics specific to
each part of parameter space.

\item This estimate is refined using either a single iteration of Halley's
second order root finding algorithm (with the sine function with its series
expansion) or a variant of the cubic approximation from \citet{Mikkola:1987}.

\item Finally, the result is updated by a single step of a high-order
generalized Newton method where the order of the method controls the target
accuracy of the method. In the \exoplanet\ implementation, we use a 4th order
update.

\end{enumerate}

The details of the implementation are exactly ported from the implementation
in \citet{Nijenhuis:1991} so we won't reproduce that here, but we did an
experiment to compare the performance and accuracy of this method to some
implementations of Kepler solvers commonly used for exoplanet fitting.
Specifically we compare to the implementations from the \batman\ transit
fitting framework \citep{Kreidberg:2015} and the \radvel\ radial velocity
fitting library \citep{Fulton:2017, Fulton:2018}.
In both cases, the solver is implemented in \project{C} and exposed to
\python\ using the \python\ \project{C}-API directly or \cython\
\citep{Behnel:2011} respectively.
Similarly, the implementation in \exoplanet\ is written in \cpp\ with \python\
bindings exposed by \theano\ \citep{Theano-Development-Team:2016}.

To test the implementation, we generated 100000 true eccentric anomalies
$E_\mathrm{true}$ uniformly distributed between $0$ and $2\,\pi$.
Then, for a range of eccentricities $e$, we computed the mean anomaly $M$ using
\eq{kepler-equation} and used each library to solve \eq{kepler-equation} for
$E_\mathrm{calc}$.
The top panel of \Figure{kepler_solver} shows the average computational cost per
solve of \eq{kepler-equation} for each library as a function of eccentricity.
The implementation of the \citet{Nijenhuis:1991} algorithm in \exoplanet\ is
more than an order of magnitude more efficient than the other methods and this
cost does not depend sensitively on the eccentricity of the orbit.
It is worth noting that both \batman\ and \exoplanet\ feature ``fused''
solvers that return both the eccentric anomaly and the true anomaly whereas
\radvel\ only computes the eccentric anomaly so there is a small amount of
computational overhead introduced into both \exoplanet\ and \batman\ when
compared to \radvel.

The bottom panel of \Figure{kepler_solver} shows the 90th percentile of the
distribution of absolute errors when comparing $E_\mathrm{true}$ and
$E_\mathrm{calc}$ for each method as a function of eccentricity.
Across all values of eccentricity, \exoplanet\ computes the eccentric anomaly
with an accuracy of better than 15 decimal places.
In some cases, the \radvel\ solver is slightly better, but at some
eccentricities, the error is more than an order of magnitude worse for
\radvel\ than for \exoplanet.
In all cases, the accuracy of the implementation from \batman\ is several
orders of magnitude worse than the other implementations.
This accuracy could be improved by decreasing the convergence tolerance in the
iterative solver at the cost of longer run times, but the current value of
$10^{-7}$ is currently hard coded\footnote{\url{%
https://github.com/lkreidberg/batman/blob/70f2c0c609124bdf4f17041bf09d5426f3c93334/c\_src/\_rsky.c\#L45}}.

\begin{figure}[htbp]
\begin{centering}
\includegraphics[width=0.8\linewidth]{figures/kepler_solver.pdf}
\oscaption{kepler_solver}{\figurelabel{kepler_solver}
The performance of the Kepler solver used by \exoplanet\ compared to
the solvers from the \radvel\ \citep{Fulton:2017, Fulton:2018} and \batman\
\citep{Kreidberg:2015} libraries.
\emph{top}: The average wall time required to solve Kepler's equation once for
the eccentric anomaly $E$ conditioned on eccentricity $e$ and mean anomaly $M$.
Smaller numbers correspond to faster solutions.
\emph{bottom}: The 90-th percentile of the absolute error of the computed
eccentric anomaly for 100000 true values of $E$ in the range $0 \le E < 2\,\pi$.
Smaller numbers correspond to more accurate solutions.}
\end{centering}
\end{figure}

\section{Contact points}

In the plane of the orbit, $S_0$, the coordinates of the orbit satisfy the
equation
\begin{eqnarray}\eqlabel{constraint0}
{({x_0} - a\,e)}^2 + \frac{{y_0}^2}{1-e^2} &=& a^2 \quad.
\end{eqnarray}
To rotate into the observing plane, we perform the following transformation
\begin{eqnarray}\eqlabel{rotation}
\bvec{x}_2 &=& R_i\,R_\omega\,\bvec{x}_0 \\
\left(\begin{array}{c}x_2\\y_2\\z_2\end{array}\right) &=&
\left(\begin{array}{ccc}
1 & 0 & 0\\
0 & \cos i & \sin i \\
0 & -\sin i & \cos i \\
\end{array}\right) \,
\left(\begin{array}{ccc}
\cos\omega & \sin\omega & 0\\
-\sin\omega & \cos\omega & 0 \\
0 & 0 & 1 \\
\end{array}\right) \,
\left(\begin{array}{c}x_0\\y_0\\0\end{array}\right) \quad.
\end{eqnarray}
In this space, the planet will transit whenever
\begin{eqnarray}
z_2 < 0 &\mathrm{and}& {x_2}^2 + {y_2}^2 < L^2
\end{eqnarray}
where $L = R_\star + R_P$.
The contact point therefore occurs when
\begin{eqnarray} \eqlabel{quad1}
\hat{x_2}^2 + \hat{y_2}^2 = L^2
\end{eqnarray}
where the hat indicates that

Using the inverse of \eq{rotation} to re-write \eq{constraint0} in terms of
$x_2$ and $y_2$, we find the following quadratic equation
\begin{eqnarray} \eqlabel{quad2}
A\,{x_2}^2 + B\,{x_2\,y_2} + C\,{y_2}^2 + D\,x_2 + E\,y_2 + F &=& 0
\end{eqnarray}
with
\begin{eqnarray}
A &=& \left(e^{2}\,\cos^{2}{\omega} - 1\right)\,\cos^{2}{i} \\
B &=& 2\,e^{2}\,\cos{i}\,\sin{\omega}\,\cos{\omega}\\
C &=& e^{2}\,\sin^{2}{\omega} - 1\\
D &=& 2\,a\,e\,\left(1-e^{2}\right)\,\cos^{2}{i}\,\cos{\omega}\\
E &=& 2\,a\,e\,\left(1-e^{2}\right)\,\sin{\omega} \cos{i}\\
F &=& a^{2}\,\left(e^{2} - 1\right)^{2}\,\cos^{2}{i}
\end{eqnarray}

The pair of quadratic equations defined by \eq{quad1} and \eq{quad2} can be
combined to give a quartic equation for $x_2$
\begin{eqnarray}
a_0 + a_1\,{x_2} + a_2\,{x_2}^2 + a_3\,{x_2}^3 + a_4\,{x_2}^4 = 0
\end{eqnarray}
where
\begin{eqnarray}
a_0 &=& \left(C L^{2} - E L + F\right) \left(C L^{2} + E L + F\right) \\
a_1 &=& - 2 \left(B E L^{2} - C D L^{2} - D F\right)\\
a_2 &=& 2 A C L^{2} + 2 A F - B^{2} L^{2} - 2 C^{2} L^{2} - 2 C F + D^{2} + E^{2}\\
a_3 &=& 2 \left(A D + B E - C D\right)\\
a_4 &=& A^{2} - 2 A C + B^{2} + C^{2}
\end{eqnarray}
which can be solved symbolically (CITE Kipping) or numerically.

\paragraph{Edge-on orbits}
For an edge-on orbit, $\cos{i} = 0$ and the contact points occur at
\begin{eqnarray}\eqlabel{edgeon1}
x_2 &=& \pm L \\
y_2 &=& 0 \quad,
\end{eqnarray}
but care must be taken when evaluating $z_2$.
To do this, we substitute \eq{edgeon1} into \eq{constraint0} to get the
following quadratic equation for $z_2$
\begin{eqnarray}
b_0 + b_1\,{z_2} + b_2\,{z_2}^2 &=& 0
\end{eqnarray}
where
\begin{eqnarray}
b_{0,\pm} &=& L^2\,(e^2\,\cos^2\omega - 1) \mp 2\,a\,e\,L\,\cos\omega\,(e^2-1) +
a^2\,{(e^2 -1)}^2 \\
b_{1,\pm} &=& -2\,a\,e\,\sin\omega\,(e^2-1) \pm 2\,e^2\,L\,\sin\omega\,\cos\omega\\
b_{2,\pm} &=& e^{2} \sin^{2}{\left (\omega \right )} - 1 \quad.
\end{eqnarray}
There are 4 solutions to this system of which we are only interested in the
ones where $z_2 < 0$ (the others are the contact points for the occultation).

\begin{figure}[htbp]
\begin{centering}
\includegraphics[width=0.8\linewidth]{figures/contact_points.pdf}
\oscaption{contact_points}{\label{fig:contact_points}
This is a figure.}
\end{centering}
\end{figure}

\section{Exposure time integration for celerite models} \label{sec:time_integration}

In this section we give expressions for the {\it exposure time}-integrated versions of the \celerite\ kernel and its power
spectrum.  It turns out that the time-integrated \celerite\ kernel
remains expressible as a semi-separable matrix when the exposures
do not overlap, but when exposures overlap for non-zero time lags,
the kernel no longer is semi-separable.

For the \celerite\ model, the general kernel function is expressed as
\begin{equation}
k(\tau) = a\,e^{-c\,\tau}\,\cos(d\,\tau) + b\,e^{-c\,\tau}\,\sin(d\,\tau) \quad,
\end{equation}
where $\tau = \vert t_n-t_m \vert$ is the lag between times $t_n$ and $t_m$, and the parameters of the model are $a$, $b$, $c$, and $d$.
To find the exposure time-integrated kernel, we must evaluate the following integral
\begin{eqnarray}
k_\Delta(\tau) = \frac{1}{\Delta^2}\int_{t_i-\Delta/2}^{t_i+\Delta/2} \mathrm{d}t \,\int_{t_j-\Delta/2}^{t_j+\Delta/2}\mathrm{d}t^\prime\,k(|t - t^\prime|)
\end{eqnarray}
where $\Delta$ is the length of the exposure.

When $\Delta \le \tau$ the exposures do not overlap and the exposure time integrated model still falls within the class of \celerite\ models:
\begin{proof}{celerite-integral}\eqlabel{delta-small}
k_{\Delta \le \tau}(\tau) = A\,e^{-c\,\tau}\,\cos{(d\,\tau)} + B\,e^{-c\,\tau}\,\sin{(d\,\tau)} \quad,
\end{proof}
where $c$ and $d$ have not changed, but the amplitudes are given by the updated expressions:
\begin{proof}{celerite-integral}
  A = \frac{2\,C_1\,\left[\cosh(c\,\Delta)\,\cos(d\,\Delta) - 1\right]
  - 2\,C_2\,\sinh(c\,\Delta)\,\sin(d\,\Delta)
}{\Delta^2\,(c^2 + d^2)^2},
\end{proof}
and
\begin{proof}{celerite-integral}
  B = \frac{2\,C_2\,\left[\cosh(c\,\Delta)\,\cos(d\,\Delta) - 1\right]
  + 2\,C_1\,\sinh(c\,\Delta)\,\sin(d\,\Delta)}{\Delta^2\,(c^2 + d^2)^2},
\end{proof}
with
\begin{align}
C_1 = a\,c^2 - a\,d^2 + 2\,b\,c\,d
\quad\mathrm{and}\quad
C_2 = b\,c^2 - b\,d^2 - 2\,a\,c\,d \quad.
\end{align}
For most real datasets, the constraint that $\Delta \le \tau$ will be satisfied because, for a light curve from a single telescope and a uniform exposure time, the times of each cadence will be separated by greater than the exposure time.
In this case, \celerite\ can still be used to efficiently evaluate the GP model.

For the case of overlapping exposures $0 \le \tau < \Delta$, the result is not as simple because of the absolute value in the definition of $\tau$.
In this case, we separate the integral into three integrals that can be easily evaluated
\begin{eqnarray}
  \Delta^2\,k_{0 \le \tau < \Delta}(\tau)
&=& \int_{t_j+\Delta/2}^{t_i+\Delta/2} \mathrm{d}t \,\int_{t_j-\Delta/2}^{t_j+\Delta/2}\mathrm{d}t^\prime\,k(t - t^\prime) \nonumber\\
&+& \int_{t_i-\Delta/2}^{t_j+\Delta/2} \mathrm{d}t \,\int_{t_j-\Delta/2}^{t}\mathrm{d}t^\prime\,k(t - t^\prime) \nonumber\\
&+& \int_{t_i-\Delta/2}^{t_j+\Delta/2} \mathrm{d}t \,\int_{t}^{t_j+\Delta/2}\mathrm{d}t^\prime\,k(t^\prime - t) \quad.
\end{eqnarray}
Evaluating and simplifying the result gives
\begin{proof}{celerite-integral}
k_{0 \le \tau < \Delta}(\tau) =& k_{\Delta\le\tau}(\tau) + 2\,\left[
  (a\,c + b\,d)\,(c^2 + d^2)\,(\Delta-\tau) \right. \nonumber\\
&  - C_1\,\sinh(c\,\Delta-c\,\tau)\,\cos(d\,\Delta-d\,\tau) \nonumber \\
&  \left. + C_2\,\cosh(c\,\Delta-c\,\tau)\,\sin(d\,\Delta-d\,\tau) \right] / [\Delta\,(c^2+d^2)]^2
\end{proof}
where $k_{\Delta\le\tau}(\tau)$ is the result from above where $\Delta \le \tau$ (\eqalt{delta-small}).

The power spectrum of the time-integrated term derived above can be computed by taking the Fourier transform and, after some simplifications, it reduces to the expected result
\begin{eqnarray}
S_\Delta(\omega) &=& S(\omega)\,\mathrm{sinc}^{2}(\omega\,\Delta/2)\nonumber\\
&=& \sqrt{\frac{2}{\pi}} \left(\frac{(ac+bd)(c^2+d^2)+(ac-bd)\omega^2}
  {\omega^4+2(c^2-d^2)\omega^2+(c^2+d^2)^2}\right)
  \left( \frac{\sin(\tfrac{1}{2}\omega \Delta)}{\tfrac{1}{2}\omega\Delta}\right)^2
\end{eqnarray}
where $S(\omega)$ is the power spectrum of the original term and $\mathrm{sinc}(x) = \sin(x)/x$ is the Fourier transform of a top hat representing the exposure.
An interesting consequence of this form is that, since the sinc function goes to zero when $\omega\,\Delta = 2\,\pi\,n$ for integers $n \ne 0$, there is no power at integer multiples of the sampling frequency $f = \frac{2\pi}{\omega} = \Delta^{-1}$.

An important implementation note is that, while a non-integrated \celerite\ term would generate a matrix with the amplitude $a$ on the diagonal, the diagonal for the integrated term is not simply $A$ because the lag on the diagonal is $\tau = 0 < \Delta$.
This means that the diagonal entries in the covariance matrix for an time-integrated \celerite\ model are
\begin{align}
k_\Delta(0) = A + \frac{2}{[\Delta\,(c^2+d^2)]^2}\,&\left[
  (a\,c + b\,d)\,(c^2 + d^2)\,\Delta \right. \nonumber\\
  & - C_1\,\sinh(c\,\Delta)\,\cos(d\,\Delta) \nonumber\\
  & \left. + C_2\,\cosh(c\,\Delta)\,\sin(d\,\Delta)\right]
\end{align}
which can be simplified to
\begin{proof}{celerite-integral}
k_\Delta(0) = \frac{2\,\left[ (a\,c + b\,d)\,(c^2+d^2)\,\Delta - C_1 + e^{-c\,\Delta}\,\left(C_1\,\cos(d\,\Delta)+C_2\,\sin(d\,\Delta)\right) \right]}{\Delta^2\,(c^2 + d^2)^2}
\quad.
\end{proof}
While, in general, this diagonal can be negative, for positive semi-definite kernels where $S(w) \ge 0$, the diagonal entry can also be computed as
\begin{eqnarray}
k_\Delta (0) = \sqrt{2\,\pi}\,\int_{-\infty} ^{\infty} S(\omega)\,\mathrm{sinc}^2(\omega\,\Delta/2)\,\dd\omega
\end{eqnarray}
which will be everywhere non-negative since $S(\omega) \ge 0$ and $\mathrm{sinc}^2(\omega\,\Delta/2) \ge 0$.

Examples of these kernels and power spectra are shown in Figure \ref{fig:celerite-integral}
for the simple-harmonic oscillator kernel which specifies the coefficients
$a = S_0\omega_0 Q, b=a/\gamma, c=\omega_0/(2Q)$ and $d=c\gamma$ in terms of a characteristic
frequency, $\omega_0$, a quality factor, $Q > \tfrac{1}{2}$, and a normalization
factor, $S_0$, where $\gamma = \sqrt{4Q^2-1}$;  see \citet{Foreman-Mackey:2017} for the
expressions when $Q \le \tfrac{1}{2}$.
The kernels become smoother and lower amplitude for longer integration times,
while the power spectra are suppressed at high frequency, and show a series of
zeros at integer multiples of the sampling frequency, which is a consequence
of aliasing at these frequencies.

For the case of $b=0$ and $c=0$ (called the ``real'' term by \citealt{Foreman-Mackey:2017}), the integrated kernel simplifies to
\begin{proof}{celerite-integral}
  k_\Delta(\tau) = \frac{2\,a\,e^{-c\,\tau}}{\left(c\,\Delta\right)^2}\,
    \begin{cases}
    e^{-c\,\Delta} - 1 + c\,\Delta & \tau = 0 \\
    \left[c\,(\Delta - \tau) - \sinh{(c\,\Delta-c\,\tau)}\right] e^{c\,\tau} + \cosh{(c\,\Delta)} -1 & 0 < \tau \le \Delta \\
    \cosh(c\,\Delta) - 1  & \Delta < \tau
    \end{cases} \quad.
\end{proof}

\begin{figure}[htbp]
\begin{centering}
\includegraphics[width=0.8\linewidth]{figures/celerite-integral.pdf}
\oscaption{celerite-integral}{\label{fig:celerite-integral}
(Top) Time-integrated \celerite\ kernels for $S_0 = 1, \omega_0 = 2\pi/5$, and
$Q=2$, with $\Delta =t_{exp} = (4, 2, \tfrac{1}{2})$ (blue, orange, green
solid lines), as well as no time integration
(dashed black line).  Vertical dotted lines correspond to the sampling
timescale. (Bottom) Corresponding power spectra.  The vertical dotted lines
indicate the sampling frequency in all three time-integrated cases.}
\end{centering}
\end{figure}

\newpage
\acknowledgements

This research was partially conducted during the Exostar19 program at the Kavli Institute for Theoretical Physics at UC Santa Barbara, which was supported in part by the National Science Foundation under Grant No. NSF PHY-1748958.

\newpage
\bibliography{exoplanet}


\end{document}
