|Version and Date       | Description           |
|-----------------------|-----------------------|
| 0.0.0<br>February 2022 | Initial commit. |
| 0.0.1<br>February 2022 | Added plotting library from util.plot and updated <br> requirements. |
| 0.0.2<br>February 2022 | Fixing pip install errors. |
| 0.0.2<br>February 2022 | Fixing pip install errors. |
| 0.0.2<br>February 2022 | Fixing pip install errors. |
| 0.0.2<br>February 2022 | Fixing pip install errors. |
| 0.0.3<br>February 2022 | Added well spaced random designs over the box and <br> ball. |
| 0.0.4<br>February 2022 | Added 'approximate' and support for generic surface <br> points in 'Plot.add_function'. |
| 0.0.5<br>March 2022 | APOS model added to library. |
| 0.0.6<br>March 2022 | Added math library utilities, included update ratio <br> in APOS model training record. |
| 0.0.7<br>March 2022 | Added missing import to random box function. |
| 0.0.8<br>March 2022 | Updating setup script to attempt Fortran compilation <br> early. |
| 0.0.9<br>March 2022 | Updated memory layout and incorporated SVD into data <br> normalization. Modified adaptive parameter update to <br> use a sliding linear number of parameters instead of <br> the binary search protocol. |
| 0.0.9<br>March 2022 | Updated memory layout and incorporated SVD into data <br> normalization. Modified adaptive parameter update to <br> use a sliding linear number of parameters instead of <br> the binary search protocol. |
| 0.0.10<br>March 2022 | Removing APOS compiler overrides. |
| 0.0.11<br>March 2022 | Made initial apositional model output normalized <br> correctly to zero mean and unit variance. |
| 0.0.12<br>April 2022 | Updated work size to int64 type. |
| 0.0.13<br>April 2022 | Preventing negative array sizes and numbers of <br> threads. |
| 0.0.14<br>April 2022 | Cleaned APOS memory usage, disabled compiled fmath, <br> added testing codes for least squares fits. |
| 0.0.15<br>April 2022 | Added large format testing for APOS and parallelized <br> the evaluation of internal state rank. |
| 0.0.16<br>April 2022 | Modified APOS fit routines to allow weighting the <br> value of individual Y with the new parameter YW. |
| 0.0.17<br>April 2022 | Minor patch, prevent division by zero in <br> CONDITION_MODEL. |
| 0.0.18<br>April 2022 | Implemented numpy evaluation class without need for <br> compiled code. Added EQUALIZE_Y configuration to <br> determine whether Y principal components are <br> flattened. Made default <br> ORTHOGONALIZING_STEP_FREQUENCY=0 to disable the <br> orthogonalizing entirely. |
| 0.0.19<br>April 2022 | Fixed a few bugs introduced after version 15. Now <br> refactored code still converges correctly for <br> apositional inputs. |
| 0.0.20<br>May 2022 | Added fast ball tree nearest neighbor codes to the <br> 'approximate' subpackage. |
| 0.0.21<br>May 2022 | Cleaning repository and pushing some waiting APOS <br> safeguards. Next step is well founded basis <br> replacement. |
| 0.0.22<br>June 2022 | Renamed APOS to AXY. Fixed error in aggregator <br> output mean calculation. Refactored CONDITION_MODEL <br> subroutines in preparation for further development. |
| 0.0.23<br>June 2022 | Added Data module. |
| 0.0.24<br>June 2022 | Added edge case checks for empty data, updated <br> memory usage patterns to improve model throughput <br> for large amounts of data, temporarily disabled all <br> local allocations that relate to in-development <br> code. Began implementing random pair generation for <br> pairwise batched aggregation. |
| 0.0.25<br>July 2022 | Cleaning up build code for package, relocating to <br> setup file. Added Timer class from util.system into <br> this package. Removed redundant variables from AXY <br> model for handling categoricals. |
| 0.0.26<br>July 2022 | Patching AXY build and import bugs. |
| 0.0.27<br>July 2022 | Updated configuration to include OpenMP symbols for <br> Ball Tree code. Updated ball tree code with slightly <br> better naming convention and removed default <br> randomness for constrained budget approximate <br> searches. |
| 0.0.28<br>July 2022 | Fixed integer encoding in AXY for yi to have unique <br> integers assigned for different categorical outputs. <br> Enforced correct shape on x and ax provided to AXY <br> models at evaluation time. Set default for rescaling <br> outputs to True. Moved most AXY tests to separate <br> file with only one remaining in the main module. |
| 0.0.29<br>July 2022 | Implemented unbuilt ball tree query operation and <br> updated test cases to reflect its new abilities. |
| 0.0.30<br>July 2022 | Updated MANIFEST to exclude compiled files in <br> distribution since those are designed to be compiled <br> on active hosts executing the code. |
| 0.0.31<br>July 2022 | Updating MANIFEST again because packager invalidated <br> previous changes. |
| 0.0.32<br>August 2022 | Added compiled regular expression library to this <br> repository. Updating some AXY documentation and <br> making random vector creation more robust to prevent <br> the return of zero vectors in high dimension. |
| 0.0.33<br>September 2022 | Significant rewrite for AXY.minimize_mse that pushes <br> loop parallelism down into the called subroutines, <br> where only single top level calls and the primary <br> fit loop remains in that code. This will enable <br> future codes that withold data for error estimation <br> and use batching during the fit operation. Also <br> added timing code and summaries for displaying ratio <br> of compute and wall time spent fitting and <br> evaluating the model. |
| 0.0.33<br>September 2022 | Significant rewrite for AXY.minimize_mse that pushes <br> loop parallelism down into the called subroutines, <br> where only single top level calls and the primary <br> fit loop remains in that code. This will enable <br> future codes that withold data for error estimation <br> and use batching during the fit operation. Also <br> added timing code and summaries for displaying ratio <br> of compute and wall time spent fitting and <br> evaluating the model. |
| 0.0.33<br>February 2023 | Introduced batching and data fetching logic to AXY <br> model to allow for fitting a model iteratively over <br> subsets of data to reduce the overall memory <br> footprint. Significantly expanded testing suite, <br> working on more thorough evaluations of new <br> features. |
| 0.0.34<br>February 2023 | Introduced batching and data fetching logic to AXY <br> model to allow for fitting a model iteratively over <br> subsets of data to reduce the overall memory <br> footprint. Significantly expanded testing suite, <br> working on more thorough evaluations of new <br> features. |
| 0.0.35<br>February 2023 | Expanding AXY test cases with scenario generation. <br> Fixed RWORK_SIZE bug that made memory for <br> AXI_RESCALE overlap with other normalization factors <br> and resulted in failed attempts to fit a model. |
| 0.0.36<br>February 2023 | Removing NaN and Inf from inputs after batching <br> update. Added AXI scenario. Changed conditioning of <br> embeddings to push towards zero mean while also <br> enforcing a max norm bound on the embedding vectors. <br> Updated default for AXI and XI to be not normalized, <br> meaning they will get normalized before training. |
| 0.0.37<br>March 2023 | Added many tests and fixed commensurate errors <br> discovered within the AXY module. Training appears <br> to be stable again. Most notable change is that the <br> gradient for the aggregator and all embeddings was <br> not being calculated correctly, so performance in <br> aggregated or embedding scenarios might improve <br> markedly. |
| 0.0.37<br>March 2023 | More thoroughly tested and implemented the pairwise <br> aggregation feature. Works well on simple test <br> problem, still not performing well on next letter <br> prediction at scale of hundreds of thousands of <br> examples. |
| 0.0.38<br>March 2023 | More thoroughly tested and implemented the pairwise <br> aggregation feature. Works well on simple test <br> problem, still not performing well on next letter <br> prediction at scale of hundreds of thousands of <br> examples. |
| 0.0.39<br>March 2023 | Replaced IRAND with F2008 conformant code. <br> Reintroduced high-level JOINT parallelism into <br> FIT_MODEL instead of lower level parallelism inside <br> evaluation and gradient routines. Transitioned all <br> logical types to C_BOOL instead of INT8 or others. <br> Introduced random reshuffling of batches after a <br> cycle is completed with RESHUFFLE parameter and <br> enabled the configuration of initial curvature <br> estimates for the model gradient with <br> INITIAL_CURV_ESTIMATE. |
| 0.0.40<br>March 2023 | Updated tests to cover more iteration cases, moved <br> iterators to RANDOM module, removed parallelism from <br> ARGSORT, included plotted summary of internal states <br> in failure modes, removed the adaptive mean and <br> curvature estimates based on success and failure. |
| 0.0.41<br>April 2023 | Updated 'tlux.data.flatten' since it appears to fail <br> for strings in the newest version of python. Renamed <br> AXY dependency modules to be prefixed with 'axy_'. <br> Fixed 'AXY.predict' for constrained pairwise <br> aggregation by separately compiling RANDOM module <br> and including that in 'setup.py'. Separated out the <br> python data preprocessing code 'to_array' into its <br> own file, preparing for more thorough testing and <br> improved parallelism. Cleaned up tests for AXY <br> libraries. Improved 'AXY.matrix_ops.orthogonalize' <br> to further prevent emergent colinearities by <br> checking back against the first vectors after <br> scaling any vector up (in case scaling up magnified <br> rounding error). |
| 0.0.42<br>April 2023 | Removed some debugging print statements that were <br> creating noise. |
| 0.0.43<br>April 2023 | Created 'unique' C extension for python that is <br> designed to extract the sorted unique elements from <br> a given array with maximized parallelism. This <br> should greatly accelerate the process of mapping <br> large Python string arrays into integers for the AXY <br> model. It will be a temporary slowdown for large <br> integer inputs, but the path to supporting an <br> integer variant is relatively short. Also adding a <br> 'profiling' module that uses 'sys.settrace' to track <br> time spent and process memory shifts at the line <br> level. |
| 0.0.44<br>May 2023 | Reimplemented and updated tlux.unique ByteArray <br> class and attached C library. The library is pure <br> now, rather than being a Python extension. Updated <br> all test cases for AXY to make sure things are <br> working and it appears they are. Added maximum MSE <br> to the AXY model so that it can safely revert <br> whenever extrodinarily large MSE are encountered <br> during the fit procedure. This is just a patch for a <br> problem that needs considerably more research. |
| 0.0.44<br>May 2023 | Reimplemented and updated tlux.unique ByteArray <br> class and attached C library. The library is pure <br> now, rather than being a Python extension. Updated <br> all test cases for AXY to make sure things are <br> working and it appears they are. Added maximum MSE <br> to the AXY model so that it can safely revert <br> whenever extrodinarily large MSE are encountered <br> during the fit procedure. This is just a patch for a <br> problem that needs considerably more research. |
| 0.0.45<br>May 2023 | Updated build scripts in 'tlux.setup' that were not <br> working correctly. Now modules build on the first <br> try and the AXY code correctly uses flags for <br> librarys when available. |
| 0.0.46<br>May 2023 | Moved allocation of UPDATE_INDICES to LWORK (outside <br> of the Fortran library). Refactor and cleaning <br> without logical consequences. Preparing to make the <br> FIT_MODEL subroutine callable repeatedly so that the <br> Fortran library can intermittently return control <br> flow to python during the fit. Modified AXY.fit to <br> allow the provision of a 'callback' function that is <br> passed a 'tlux.approximate.axy.summary.Details' <br> object describing the current state of the model and <br> fit. It is called at most every 'print_delay_sec' <br> seconds, which currently defaults to two seconds. It <br> will always be called at least once, but that might <br> be after the normalization, fit, and encoding have <br> finished. |
| 0.0.47<br>June 2023 | AXY implementation of partial aggregation. New <br> Permuted Congruential Generator (PCG) as a source of <br> high quality random numbers. And associated updates <br> to tests for the AXY modules. |
