# SPDX-License-Identifier: AGPL-3.0-or-later
# SPDX-FileCopyrightText: University of Washington <https://www.washington.edu>
# SPDX-FileContributor: 2021-22 Bradley M. Bell
# ----------------------------------------------------------------------------
{xrst_begin_parent release_notes}

Release Notes for at_cascade
############################

{xrst_end release_notes}
-----------------------------------------------------------------------------
{xrst_begin 2022}
{xrst_spell
   avgint
   cov
   dd
   distutils
   mtall
   mtspecific
   mm
   numpy
   optimizer
   perturb
   pyspellchecker
   rst
   spline
   smoothings
   mtspecific
}

Release Notes for 2022
######################

all_omega Branch
****************

mm-dd
*****

10-20
=====
1. Remove the avgint Table item from the wish list
   (it was done previously).
2. Remove default None values for arguments to :ref:`check_cascade_node`
   and include argument type asserts in Syntax.
3. Change all_mtall to `all_omega` and remove all_mtspecific .
   In other words, do the calculation of mtall minus mtspecific
   (to get the omega constraint) outside of at_cascade.
   This completed and removed the mtall wish list entry.
4. In csv.fit, change all_predict.csv to
   :ref:`csv_fit@Output Files@sam_predict.csv`.
   Make the column names in this file specialized for
   csv fit predictions.
   In addition, add :ref:`csv_fit@Output Files@fit_predict.csv`
   to the csv fit predictions.

10-19
=====
1. Add :ref:`csv_fit@Input Files@option_in.csv@max_fit`
   to the csv option table.
2. Change the definition of a
   :ref:`csv_module@Notation@Rectangular Grid` so that it can be
   different for each value in the selection column.
3. Use the fact that the child rates (random effects) and the
   covariate multipliers are constant in age and time to simplify csv_fit.
   To be specific, use priors instead of smoothings for this functions
   (which enabled us to remove smoothing names from csv_fit); see
   :ref:`csv_fit@Input Files@parent_rate.csv`,
   :ref:`csv_fit@Input Files@child_rate.csv`,
   :ref:`csv_fit@Input Files@mulcov.csv`.
4. There was an error in the all_predict.csv file.
   To be specific, the samples were not properly limited
   to the sex value corresponding to the fit.
   This has been fixed.
5. The covariate multipliers for sex are now frozen after
   the both fit at the root node level.
   The other covariate multipliers are frozen after the female and male
   split at the root level.

10-18
=====
Change fit option.csv to
:ref:`csv_fit@Input Files@option_in.csv` ,
add default value for all the fit options,
and place resulting options in
:ref:`csv_fit@Output Files@option_out.csv` .

10-27
=====
First tested version of :ref:`csv_fit` .

10-16
=====
Add the :ref:`csv_empty_str` utility for converting
empty strings to, and from, None.

10-15
=====
Change license notice to SPDX_ format.

.. _SPDX: https://spdx.dev/ids/

10-14
=====
1. Improve the description of the location of the fit results; see
   :ref:`cascade_root_node@Output dismod.db` .
2. Add the mtall_data and avgint Table wish list entries.
3. Fix bug in :ref:`csv_simulate` . To be specific, one of the rate functions
   was being used for all the rates. A regression test for this bug was added
   (where is more than one rate in
   :ref:`csv_simulate@Input Files@no_effect_rate.csv` ).

10-10
=====
Fix the return value for :ref:`bilinear@spline_dict` function calls::

   z = spline_dict[z_name](x, y)

so that the result z is a float and not a numpy array of size one.

10-09
=====
1. Move the csv interface files to the csv sub-module; see
   :ref:`csv_module`.
2. Move

   - read_csv_table to :ref:`csv_read_table`
   - write_csv_table to :ref:`csv_write_table`

10-06
=====
Fix some documentation spelling errors detected by new version of pyspellchecker.

09-18
=====
1. Adapt to xrst-2022.9.18. This changes the location of documentation
   html files from the ``doc/rst`` directory to the ``doc`` directory.
2. Change the linking text for web pages from the page title to the
   page name. For example, use :ref:`all_node_db` instead of
   :ref:`all_node_db-0` to link to the all node database documentation.

08-27
=====
1. Remove the relative_tolerance option from :ref:`csv_simulate` and
   increase absolute_tolerance in its :ref:`example<csv_simulate_xam>`.
   (It was removed from ``dismod_at.average_integrand`` because
   testing it with :ref:`csv_simulate_xam` revealed problems.)
2. Remove ``\begin{align*}`` from math directives
   (sphinx automatically adds a split to math directives).

08-25
=====
Add the absolute_tolerance, relative_tolerance, and float_precision
columns to the :ref:`csv_simulate@Input Files@option.csv` file.

08-23
=====
1. Add tracing to :ref:`csv_simulate` ; i.e., it will report is progress.
2. Change number of spaces corresponding an indent from 4 to 3.
   (This interacts better with the sphinx documentation system.)

08-21
=====
1. Fix some error message and add some tracing to :ref:`csv_simulate` .
2. Update documentation to use xrst-2022.8.21.

08-16
=====
Add the section :ref:`ode_iota_omega` which derives the dismod ODE solution
when rates are constant and only iota, omega are non-zero.

08-15
=====
Remove use of the python distutils package because it has been deprecated.

08-12
=====
Add the :ref:`bilinear` spline routine.

08-09
=====
1. Advance to xrst-2022.8.8 (this improves the documentation).
2. Improve the :ref:`index<genindex>`
   using the keyword file to exclude some entries .
3. Add the :ref:`csv_read_table` and :ref:`csv_write_table` routines
   to the at_cascade module.

08-01
=====
Start construction of a cascade data simulation tool; see
:ref:`csv_simulate` .

04-18
=====
Remove shared memory and LD_LIBRARY_PATH items from the
:ref:`wish_list`
because they have been completed.

03-22
=====
Add :ref:`all_option_table@balance_fit` to the all_option table.

03-14
=====
1. Fix a bug in how
   :ref:`avgint_parent_grid` and
   :ref:`create_shift_db` pass
   priors down the cascade. To be more specific, the split reference
   covariate effects were not being handled properly.
2. A check of the prior mean for iota was added at the end of
   the :ref:`split_covariate` example
3. Add the :ref:`no_ode_fit@fit_type` argument to no_ode_fit.

03-12
=====
The shared memory names were shortened,
to be specific _at_cascade was removed from theses names.
In addition, the all node database was added as an argument
to :ref:`clear_shared` so it would know the shared memory prefix.

03-10
=====
Add the :ref:`cascade_root_node@fit_type_list` argument to
cascade_root_node and continue_cascade.

03-08
=====
The :ref:`remission` example was added.

03-07
=====
The :ref:`all_option_table@shared_memory_prefix` was added to the
all_option table.

03-05
=====
Change :ref:`no_ode_fit` to always use Gaussian data densities.

02-05
=====
Use :ref:`get_cov_reference` to compute references for relative covariates
and remove the all_cov_reference table from the :ref:`all_node_db`.

02-04
=====
In the examples, fix the following typos:
Change 'are checked for check_cascade_node'
to 'are checked by check_cascade_node'.
Change 'rood_node_db routine' to 'root_node_db routine'.

02-03
=====
Add new :ref:`wish_list` entries and remove old ones the have been completed.

01-28
=====
Add the :ref:`add_log_entry`, :ref:`move_table`, and :ref:`empty_avgint_table`
routines to the at_cascade module.

01-27
=====
1. The max_fit value is now doubled when doing
   :ref:`all_option_table@max_fit@mulcov_freeze` fit.

2. If :ref:`all_option_table@max_number_cpu` was one,
   the optimizer trace was not printed to the screen.
   This has been fixed.

01-26
=====
Change perturb_optimization_scaling to
:ref:`all_option_table@perturb_optimization_scale` and add the
:ref:`all_option_table@perturb_optimization_start` option.

01-25
=====
The :ref:`clear_shared` routine was added to the
at_cascade module.

01-08
=====
1. Fix a bug in :ref:`create_shift_db`.
   To be specific, the mean int a value prior child variable in the shifted
   database (child database) might not satisfy the prior limits.
   This violated the dismod_at assumptions about the value prior means.

2. Change :ref:`create_shift_db@shift_databases@dage and dtime Priors`
   to be difference predicted by parent fit.

01-02
=====
1. The cascade would hang at the end, instead of detecting completion,
   when :ref:`all_option_table@max_number_cpu` was greater the one.
   This has been fixed.

2. Use :ref:`create_job_table@job_table@job_name`
   as the key for each job in the :ref:`check_log@message_dict` return value.

{xrst_end 2022}
-----------------------------------------------------------------------------
{xrst_begin 2021}
{xrst_spell
   dd
   dir
   iter
   mm
   py
}

Release Notes for 2021
######################

mm-dd
*****

12-29
=====
1. Fix error recovery and add more job status reporting to
   :ref:`run_parallel`.

2. Document the :ref:`run_parallel@trace.out` file.

12-28
=====
1. Change results_dir to result_dir.

2. Remove the *trace_fit* parameter.

12-26
=====
1. Add exception handling to running in parallel so that
   program does not hang when a job fails.

2. Change the run_one_job trace_fit argument to
   :ref:`run_one_job@trace_file_obj` so that opening and closing the file
   is managed by the calling program.

12-24
=====
1. The base_directory was changed to the :ref:`all_option_table@result_dir`
   and now a required option the all_option table.

2. Change required precision in example/mulcov_freeze.py to reduce probability
   of random test failure.

3. Replace :ref:`no_ode_fit` max_fit and max_abs_effect arguments by
   :ref:`no_ode_fit@all_option_dict`.

4. In no_ode_fit, change in_database to :ref:`no_ode_fit@root_node_database`,
   and change out_database to :ref:`no_ode_fit@root_fit_database`.

12-21
=====
1. The output files are now relative to the base_directory
   (they used to be relative to the current working directory).

2. Starting from a :ref:`glossary@root_node` that was not at the top
   of the node tree did not work. The :ref:`one_at_function` was
   modified to test for this problem and the problem was fixed.

12-08
=====
Remove the convergence problem :ref:`wish_list`
entry because it was completed
when the :ref:`all_option_table@perturb_optimization_scale` option
was added.

12-06
=====
Add the :ref:`mulcov_freeze` table which enables one to freeze the
value of a covariate multiplier so it is the same for all fits
below a certain point.

12-05
=====
1. Add the :ref:`check_log` routine.

2. Add the :ref:`all_option_table@perturb_optimization_scale` option
   and list options in alphabetical order.

12-03
=====
1. Add the :ref:`run_parallel` routine which
   runs :ref:`create_job_table@job_table` jobs
   sequentially (in parallel) when :ref:`run_parallel@max_number_cpu` is
   one (greater than one).

2. Add a :ref:`wish_list` for at_cascade.

12-02
=====
1. The :ref:`no_ode_fit` routine was setting the dismod_at
   max_num_iter_fixed option to 100 for all fits.
   This has been fixed and the option table setting is now used for all fits.

2. Move cascade_fit_node to :ref:`cascade_root_node` (better name now that
   the routine is not recursive. In addition, change the fit_node_database
   argument name to :ref:`cascade_root_node@root_node_database`.

12-01
=====
1. Add :ref:`get_database_dir`.
   This routine is for use with parallel processing.

2. Change fit_node_database base create_job_table argument to
   :ref:`create_job_table@node_table`.

3. Add :ref:`create_job_table@job_table@start_child_job_id` and
   :ref:`create_job_table@job_table@end_child_job_id` to the job table.

11-30
=====
Change the name get_job_table to
:ref:`create_job_table`.

11-29
=====
1. Add get_job_table. This routine is under construction and planned
   and part of a re-factoring for parallel processing.

2. Add :ref:`all_option_table@root_split_reference_name`
   to the all_option table (so that get_job_table can use any
   fit_node_database).

11-28
=====
1. Change the :ref:`create_all_node_db@all_option` argument to
   create_all_node_db to be a complete representation of the
   :ref:`all_option_table`.
   This reduces the amount of necessary documentation.
   It also requires that root_node_name be in the all_option argument.

2. Change the :ref:`create_all_node_db@split_reference_table` argument to
   have a default value (corresponding to the empty table).

3. Change split_level in the all_option table to the
   :ref:`node_split_table`.


{xrst_end 2021}
