.. _whatsnew_100b4:

1.0.0b4 (February 7, 2020)
--------------------------

This is the fourth 1.0 beta release. Further changes beyond the pull
requests listed here may have been applied in (:pull:`321`).

The biggest change is a new data model for Reports so that metrics,
processed data, and generated Bokeh javascript and SVG figures are
stored in JSON in the API. This removes the requirement for Pandoc and
HTML reports can be generated using only Jinja. Other changes include
report figure generated from a common data source, better error
handling if computing a report fails, and actual data validation
results, including the number of missing and dropped values, in the
report.


API Changes
~~~~~~~~~~~
* :py:func:`solarforecastarbiter.metrics.prepocessing.apply_validation`
  now expects a single observation DataFrame with value and
  quality_flag columns instead of a dict. It also returns the
  processed Series along with a dict of the number of values removed
  for each quality flag. (:pull:`291`)
* :py:func:`solarforecastarbiter.metrics.prepocessing.resample_and_align`
  now expects the forecast and observation series explicitly. It also
  returns the resampled and aligned values instead of a
  `datamodel.ProcessedForecastObservation` (:pull:`291`)
* Move functionality of
  :py:func:`solarforecastarbiter.reports.main.validate_resample_align`
  to
  :py:func:`solarforecastarbiter.metrics.prepocessing.process_forecast_observations`
  (:pull:`291`)
* Rename ``io.utils.serialize_data`` to
  :py:func:`solarforecastarbiter.io.utils.serialize_timeseries`
  (:pull:`291`)
* :py:func:`solarforecastarbiter.metrics.calculator.calculate_metrics`
  returns a list of
  :py:class:`solarforecastarbiter.datamodel.MetricResult` and
  :py:func:`solarforecastarbiter.metrics.calculator.calculate_deterministic_metrics`
  returns a single
  :py:class:`solarforecastarbiter.datamodel.MetricResult`
  (:pull:`291`)
* :py:class:`solarforecastarbiter.datamodel.ProcessedForecastObservation`
  now requires a name parameter (:pull:`291`)
* :py:func:`solarforecastarbiter.reports.figures.construct_timeseries_cds`
  now takes a single :py:class:`solarforecastarbiter.datamodel.Report`
  object and returns separate Bokeh ColumnDataSources for timeseries
  and metadata. (:pull:`307`)
* :py:func:`solarforecastarbiter.reports.figures.construct_metrics_cds`
  updated to accept a list of
  :py:class:`solarforecastarbiter.datamodel.MetricResult` as its first
  argument. (:pull:`307`)
* :py:func:`solarforecastarbiter.reports.figures.timeseries` and
  :py:func:`solarforecastarbiter.figures.scatter` now take two Bokeh
  ColumnDataSources as their first two arguments in place of a list of
  (ProcesseForecastObservation, ColumnDataSource) tuples. See the
  return value of
  :py:func:`solarforecastarbiter.figures.construct_timeseries_cds` for
  the format of these ColumnDataSources. (:pull:`307`)
* Replaced ``solarforecastarbiter.reports.full_html`` with
  :py:func:`solarforecastarbiter.reports.template.render_html` which
  accepts a :py:class:`solarforecastarbiter.datamodel.Report` object,
  a dashboard url and optional flags to include timeseries plots or
  generate a standalone html report. (:pull:`311`)
* Add :py:func:`solarforecastarbiter.reports.figures.output_svg` to
  save Bokeh figures as SVG (:pull:`313`)
* Add optional status keyword to
  :py:func:`solarforecastarbiter.io.api.APISession.post_raw_report`
  (:pull:`318`)
* :py:class:`solarforecastarbiter.datamodel.PreprocessingResult` Added
  to track prepocessing summary of results (:pull:`320`)
* Add the :py:class:`solarforecastarbiter.datamodel.ReportParameters`
  class, add the ``report_parameters`` parameters to
  :py:class:`solarforecastarbiter.datamodel.Report`, and move most
  parameters (:pull:`324`)
* Remove the ``ReportMetadata`` class and move some parameters to
  :py:class:`solarforecastarbiter.datamodel.RawReport` (:pull:`324`)
* Remove ``solarforecastarbiter.reports.main.create_metadata`` (:pull:`324`)
* :py:func:`solarforecastarbiter.reports.main.get_versions` now
  returns a tuple of (package, version) tuples (:pull:`325`)

Enhancements
~~~~~~~~~~~~
* Automatically verify selected metrics are valid for deterministic
  forecasts. (:issue:`267`) (:pull:`301`)
* Automatically verify selected metrics are valid for probabilistic
  forecasts. (:issue:`302`) (:pull:`306`)
* Add schema for de/serializing timeseries in
  :py:mod:`solarforecastarbiter.io.utils` (:pull:`291`)
* Add decorator factory
  :py:func:`solarforecastarbiter.reports.main.capture_report_failure`
  to better log report failures to the end user. (:pull:`318`)
* Add missing tests for functionality in
  :py:mod:`solarforecastarbiter.reports.main` (:issue:`312`)
  (:pull:`318`)
* Add missing tests for :py:mod:`solarforecastarbiter.reports.figures`
  (:pull:`318`)
* Add data validation summary table in the html report (:issue:`299`)
  (:pull:`320`)
* Track data preprocessing results add section and summary table in
  the html report (:issue:`299`) (:pull:`320`)
* Generate error page templates for failed reports. (:pull:`325`)
* Replace the Bokeh tables with HTML tables in reports (:pull:`321`)
* Add :py:func:`solarforecastarbiter.io.utils.load_report_values`
  convenience function (:pull:`331`)
* Adjust observations and forecasts table and make all processed pairs
  in a report have a unique name (:pull:`333`)
* Update versions in requirements (:pull:`335`)
* Add an example notebook to create reference reports (:issue:`222`)
  (:pull:`338`)

Bug fixes
~~~~~~~~~
* Fix handling of empty observation timeseries in metrics
  preprocessing. (:issue:`295`) (:pull:`296`)
* Fix handling of `interval_label == ending` in the `groupby`
  categories in `metrics.calculator`. (:issue:`234`) (:pull:`297`)
* Document `resample_and_align()` behavior when the `interval_label` of the
  forecast and observation don't match. (:issue:`300`) (:pull:`303`)
* Fix scatter plot limits when some data is NaN (:issue:`282`) (:pull:`311`)
* Handle errors on SVG export (:issue:`310`) (:pull:`313`)
* Test and handle errors in report creation for empty data
  (:issue:`217`) (:pull:`318`)
* Fix :py:func:`solarforecastarbiter.datamodel.__check_units__` with
  no arguments (:pull:`318`)
* Fix field seperator conflict with the name when creating figures
  (:issue:`337`) (:pull:`338`)
* Fix dumping TimeOfDayFilter to dict (:pull:`338`)
* Allow the posting of filters in
  :py:func:`solarforecastarbiter.io.api.APISession.create_report`
  (:pull:`338`)

Contributors
~~~~~~~~~~~~

* Will Holmgren (:ghuser:`wholmgren`)
* Leland Boeman (:ghuser:`lboeman`)
* Cliff Hansen (:ghuser:`cwhanse`)
* Tony Lorenzo (:ghuser:`alorenzo175`)
* Justin Sharp (:ghuser:`MrWindAndSolar`)
* Aidan Tuohy
* Adam Wigington (:ghuser:`awig`)
* David Larson (:ghuser:`dplarson`)
