<h1>Model Violation Analysis: Robust data scaling</h1>
<p>GST datasets are often <em>very</em> inconsistent with the Markovian gateset model.  This is relatively unsurprising, and means only that real qubits often drift or experience other forms of noise that aren't stationary and Markovian.  But this "voids the warranty" on GST's results, at least in principle.  The properties of the estimated gates usually appear to be meaningful anyway, but when the model is violated, normal methods for generating <em>error bars</em> become radically overoptimistic.  As a partial remedy for this, pyGSTi can be configured to generate "robust" analyses of model-violating data, by artificially deprecating data that are inconsistent with the fit (a variant of some robust statistics methods).</p>

<!-- Toggle over essentially the remainder of the tab, since it's all about scaling. -->
<!-- #iftoggle(ShowScaling) -->
{% if config['ShowScaling'] %}

<!-- Toggle descriptive text -->
<!-- #iftoggle(CombineRobust) -->
{% if config['CombineRobust'] %}

<p>If the estimate currently selected on the sidebar used this technique (often denoted by a <q>.robust</q> suffix), then this tab shows several important quantities.  Before describing these, however, it is important to note that <b>all of the other model violation tabs</b> (and relevant figures in the <q>Summary</q> tab) <b>show you the model violation <em>before</em> any data deprecation was perfomed.  <em>This tab</em> shows the model violation <em>after</em> the data deprecation</b>, and so, by construction, the fit metrics shown here should always look pretty good.  The first several figures replicate those of the other model violation tabs (except for the <em>post-scaled</em> data!), and the final plot shows how much each individual experiment (circuit) was deprecated (essentially, by throwing out many of the counts for that circuit while keeping the overall observed frequencies constant).  When a figure shows up as <q>N/A</q> then it means that the currently-selected estimate has not been deprecated at all, and so there's nothing to show.</p>

<!-- #elsetoggle -->
{% else %}

<p>If the estimate currently selected on the sidebar used this technique (often denoted by a <q>.robust</q> suffix), then this tab shows how much each individual experiment (circuit) was deprecated (essentially, by throwing out many of the counts for that circuit while keeping the overall observed frequencies constant).  When a figure shows up as <q>N/A</q> then it means that the currently-selected estimate has not been altered at all.</p>

{% endif %}
<!-- #endtoggle -->

<!-- Toggle showing of post-scaling plots -->
<!-- #iftoggle(CombineRobust) -->
{% if config['CombineRobust'] %}
<figure id="final_model_fit_progress_bar_plot_scl">
	<figcaption><span class="captiontitle">SCALED Model violation summary.</span> <span class="captiondetail">This plot summarizes how well GST was able to fit the data -- or subsets of it -- to a gateset.  Bars indicate the difference between the actual and expected log-likelihood values, and are given in units of standard deviations of the appropriate <span class="math">\chi^2</span> distribution. Each bar corresponds to a <em>subset</em> of the data including only circuits of length up to <span class="math">\sim L</span>; the rightmost bar corresponds to the full dataset.  Low values are better (less model violation), and bars are colored according to the <q>star</q> rating found in a later table detailing the overall model violation.</span></figcaption>
	{{ final_model_fit_progress_bar_plot_scl|render }}
</figure>

<figure id="final_model_fit_histogram_scl">
	<figcaption><span class="captiontitle">SCALED Histogram of per-circuit model violation.</span> <span class="captiondetail">This figure is about goodness-of-fit.  When the estimate doesn't fit the data perfectly, we can quantify how well it fails to predict each individual circuit in the dataset, using the excess loglikelihood (<span class="math">-2\log\mathrm{Pr}(\mathrm{data}|\mathrm{gateset})</span>) above and beyond the minimum value (<span class="math">-2 \log \mathrm{Pr}(\mathrm{data}|\mathrm{observed\ frequencies})</span>).  This plot shows a histogram of the those values for all the circuits in the dataset.  Ideally, they should have the <span class="math">\chi^2</span> distribution shown by the solid line.  Red indicates data that are inconsistent with the model at the 0.95 confidence level, as shown in more detail in the Model Violation tab.</span> </figcaption>
	{{ final_model_fit_histogram_scl|render }}
</figure>

<figure id="final_model_fit_progress_table_scl" class='tbl'>
	<figcaption><span class="captiontitle">SCALED Detailed overall model violation.</span> <span class="captiondetail"> This table provides a detailed look at how the observed model violation -- defined by how badly the GST model fits the data -- evolves as more and more of the data are incorporated into the fit.  PyGSTi fits the data iteratively, starting by just fitting data from the shortest circuits (<span class="math">L=1</span>), and then adding longer and longer sequences.  Each subset of the data, defined by its maximum sequence length <span class="math">L</span>, yields an independent fit that is analyzed here.  The key quantity is the difference between the observed and expected maximum loglikelihood (<span class="math">\log(\mathcal{L})</span>).  If the model fits, then <span class="math">2\Delta\log(\mathcal{L})</span> should be a <span class="math">\chi^2_k</span> random variable, where <span class="math">k</span> (the degrees of freedom) is the difference between <span class="math">N_S</span> (the number of independent data points) and <span class="math">N_p</span> (the number of model parameters).  So <span class="math">2\Delta\log(\mathcal{L})</span> should lie in <span class="math">[k-\sqrt{2k},k+\sqrt{2k}]</span>, and <span class="math">N_\sigma = (2\Delta\log(\mathcal{L})-k)/\sqrt{2k}</span> quantifies how many standard deviations it falls above the mean (a <span class="math">p</span>-value can be straightforwardly derived from <span class="math">N_\sigma</span>).  The rating from 1 to 5 stars gives a very crude indication of goodness of fit.  Heading tool tips provide descriptions of each column's value.</span></figcaption>
	<!--<span class="math">p</span> is the p-value derived from a <span class="math">\chi^2_k</span> distribution.(For example, if <span class="math">p=0.05</span>, then the probability of observing a <span class="math">\chi^{2}</span> value as large as, or larger than, the one indicated in the table is 5%%, assuming the GST model is valid.) -->
	{{ final_model_fit_progress_table_scl|render }}
</figure>

<figure id="final_model_fit_colorscatter_plot_scl">
	<figcaption><span class="captiontitle">SCALED Per-circuit model violation vs. circuit length</span> <span class="captiondetail">The fit's total <span class="math">2\Delta\log(\mathcal{L})</span> is a sum over all <span class="math">N_s</span> circuits used for GST.  This plot shows <span class="math">2\Delta\log(\mathcal{L})</span> for each individual circuit, plotted against that circuit's length (on the X axis).  Certain forms of non-Markovian noise, like slow drift, produce a characteristic linear relationship.  Note that the length plotted here is the <em>actual</em> length of the circuit, not its nominal <span class="math">L</span>.</span> </figcaption>
	{{ final_model_fit_colorscatter_plot_scl|render }}
</figure>


<figure id="final_model_fit_colorbox_plot_scl">
	{{ final_model_fit_colorbox_plot_scl|render }}
	<figcaption><span class="captiontitle">SCALED Per-sequence model violation box plot.</span><span class="captiondetail"> This plot shows the <span class="math">2\Delta\log(\mathcal{L})</span> contribution for each individual circuit in the dataset.  Each box represents a single gate sequence, and its color indicates whether GST was able to fit the corresponding frequency well.  Shades of white/gray indicate typical (within the expected) values. Red squares represent statistically significant evidence for model violation (non-Markovianity), and the probabilty that <i>any</i> red squares appear is {{ linlg_pcntle|render }}%% when the data really are Markovian. Each square block of pixels (<q>plaquette</q>) corresponds to a particular germ-power "base sequence", and each pixel within a block corresponds to a specific "fiducial pair" -- i.e., choice of pre- and post-fiducial sequences.  The base sequences are arranged by germ (varying from row to row), and by power/length (varying from column to column).  Hovering over a colored box will pop up the exact circuit to which it corresponds, the observed frequencies, and the corresponding probabilities predicted by the GST estimate of the gateset.  The slider below the figure permits switching between different estimates, labeled by <span class="math">L</span>, which were obtained from subsets of the data that included only base sequences of length up to <span class="math">L</span>. </span></figcaption>
</figure>

{% endif %}
<!-- #endtoggle -->

<figure id="data_scaling_colorbox_plot">
	{{ data_scaling_colorbox_plot|render }}
	<figcaption><span class="captiontitle">Data scaling factor for each circuit in the dataset.</span> <span class="captiondetail">Each colored box represents a single experiment (circuit), arranged in the same way as in other related tabs.  A circuit's color indicates the how much the original data counts were scaled down when they were used to compute the log-likelihood or <span class="math">\chi^2</span> for this estimate (and its error bars).  A white box (value 1.0) indicates that all of the original data was used, because that circuit was not originally seen to be inconsistent with the fit. On the other hand, gray or black boxes (numbers between 0 and 1) indicate that the total number of counts for that circuit was scaled down (multiplied by the given factor) to reduce its significance, and therefore that circuit's inconsistency with the fit.  Generally, the only circuits scaled down are those deemed significantly inconsistent in the original (unscaled) fit.</span></figcaption>
</figure>

<!-- #elsetoggle -->
{% else %}
<p>Note: Data-scaling figures are not shown because none of the estimates in this report have scaled data.</p>
{% endif %}
<!-- #endtoggle -->
