metpx-sr3 (3.00.47) unstable; urgency=medium

  * fixed #834 change filename default from "WHATFN" to None
  * fixed #833 directory option with multiple time codes, bad result.
  * fixed #827 --dangerWillRobinson=25 match # of configs for destructive stuff.
  * fixed #829 crash when :TIME: present in sundew pattern.
  * began unit tests for config (file name interpretation as per #829)
  * fixed #828 log.py/after_post printing wrong exchange 
  * fixed #825 sr3 disable crash.
  * fixed #817 metrics file rotation (so they do not always grow)
  * fixed #816 crash in API when metrics directory missing.
  * fixed #808 poll start hang because exchanges missing.

 -- peter <peter@bsqt.homeip.net>  Mon, 06 Nov 2023 15:19:02 -0500

metpx-sr3 (3.00.46) unstable; urgency=medium

  * closes #796, #791 messages published after transfer with mirror off.
  * closes #793 move metrics to sub-directory (for Greg's work on cloud)
  * fixes for windows executable generation. 
  * branch re-org: v03 now assumed, so *development* replaces v03_wip
  * github container repository (GHCR) updates... "latest" now versioned.
  * GHCR development snapshot image now available.
  * GHCR both images built from _base image that does all dependencies.
  * closes #811 - documentated default wrong for acceptUnmatch.
  * closes #809 - flow_api_consumer example crashing.
  * closes #804 - warning message on poll with exchange name mismatch.
  * closes #802 - adds POST_BROKER_USER variable.

 -- peter <peter.silva@ssc-spc.gc.ca>  Mon, 06 Nov 2023 11:08:57 -0400

metpx-sr3 (3.00.45) unstable; urgency=medium

  * use topic field in message as an override when available to 
    determine publishing topic of downloaded files.
  * log rotation setting parsing now uses normal parsing (same as
    all other options) CHANGE: logRotateInterval default units is seconds,
    like all other sr3 *duration* settings
  * work-around #785 windows bug... sr3_logRotateManually introduced for now.
  * improved messages when failing to connect to brokers.
  * log messages content reviewed for improved readability/usability.
  * fix #784 starting foreground with patterns.
  * fix #783 metrics calculations (large negative number on startup)
  * fix #780 disabled instances prevents finding missing ones.
    (sr3 sanity not working when disable configs present.)
  * Adding NASA API plugin support #769
  * interim fix #772 rename bug in mirroring.
  * fix #773 DESTFNSCRIPT arguments different from callback.
  * fix #774 DESTFNSCRIPT plugin finds msg['new_file'] undefined
  * fix #780 find_missing_instances doesn't when disabled configs exist.

 -- Peter <peter.silva@ssc-spc.gc.ca>  Tue, 10 Oct 2023 15:38:23 -0400

metpx-sr3 (3.00.44) unstable; urgency=medium

  * fix #760 'source' missing sometimes when needed.
  * fix #753 failed magic import improved detection.
  * fix #752 scheduled plugins not working.
  * improved #757 unit tests for all flowcb plugins.
  * fix #758 initial support for WISKI at Ontario Power
    Generation. (EXPERIMENTAL)
  * work on #732 improved publish timeouts #761
  * issue #762 re-enable reporting (EXPERIMENTAL) 
  * update MQTT code for PR https://github.com/eclipse/paho.mqtt.python/pull/753

 -- Peter <peter.silva@ssc-spc.gc.ca>  Sat, 19 Aug 2023 00:44:33 -0400

metpx-sr3 (3.00.43) unstable; urgency=medium

  * issue #634 sr3 dump is now pure JSON (Thanks Mohammed!)
  * issue #748 sr3 foreground on windows will not start.
  * windows self-extracting executable is now usable.
    (dependency fixes)

 -- Peter Silva <peter.silva@ssc-spc.gc.ca>  Tue, 15 Aug 2023 14:32:56 -0400

metpx-sr3 (3.00.42) unstable; urgency=medium

  * issue #747 mqtt passing more tests (but restart_server still fails.)
  * issue #745 bad poll cod key choice, refactored nodupe.
  * issue #738 new: *sr3 features* action to inventory dependencies obviously.
  * PR #735, issue #721 *filetype* becomes extra feature.
  * issue #720 bug about plugin option being specified twice causing parser
    crash.
  * PR #733, fixes #643 on_cleanup entry points in flowcb now functional.
  * issue #415 MQTT/websocket support added. other fixes for MQTT.
  * issue #724 sr3 cli crashes when state files in a bad state.
  * issue #730, #731 many additions to unit testing.
  * issue #727 bugfixes found during unit test development.
  * issue #719 complain about each undeclared options only once.
  * issue #712 crash when plugin declares a duration option without default
    value.
  * issue #709 fixes for cases when downloaded file size different from
    expected.
  * can now install all optional extras with pip install metpx-sr3[all]

 -- Peter Silva <peter.silva@ssc-spc.gc.ca>  Tue, 27 Jun 2023 14:26:23 -0400

metpx-sr3 (3.00.41) unstable; urgency=medium

  * issue #700 nodupe_redis driver (experimental for now)
  * issue #703 v03 message encoding change integrity to identity
  * issue #699 v03 message encoding adding contentType
  * issue #698 WIS format encoding support added (experimental)
  * issue #695 fix for broken discard option.
  * issue #704 improved how version is stored for installation dependency
    management.
  * issue #697 nodupe unit tests created.

 -- Peter <peter.silva@ssc-spc.gc.ca>  Sat, 03 Jun 2023 01:43:49 -0400

metpx-sr3 (3.00.40) unstable; urgency=medium

  * resolved #681, #661 python API broken 
  * Github CI/CD flow tests for python API added to maintenance tests.
  * progress on #617 changed moth python API so messages calculate own topic.
    makes it compatible with networks that have different topic schemes,
    such as WMO WIS. (change in postformat API, and moth calls to it.)
  * Unit testing framework introduced #676  (from @gcglinton )
  * CD flow tests with redis closer to working (two tests still fail). ( #690 )
  * improvements on #506 MQTT functionality (but still not passing, but
    failures related to retry/error recovery.)
  * documentation generation had broken (external changes) extensive review and updates.
  * See UPGRADING.rst for details of API changes.
  * documentation #677 improvement.
  * bugfix #660 sr3 crash with certain config errors.
  * another part of #617 post_format should override post_topicPrefix for formatting.
  * resolved #684 (documentation generation) 
  * resolved #678 messages for (other) certain bad config setting.
  * resolved #671 crash for certain bad config setting.
    
 -- Peter <peter.silva@ssc-spc.gc.ca>  Sun, 14 May 2023 11:01:45 -0400

metpx-sr3 (3.00.39) unstable; urgency=medium

  * fix #667 had not been completely merged. adding. 

 -- Peter <peter.silva@ssc-spc.gc.ca>  Sat, 13 May 2023 09:55:09 -0400

metpx-sr3 (3.00.38) unstable; urgency=medium

  * fix #669 regression that sr3 cleanup is broken in 3.00.37
  * fix #667 new_file name not honoured for republish with no transfer
  * fix #663 should not call expanduser on remote paths
  * fix #661 regression python api broken by missing 'dry_run' default setting.
  * fix #656 sr3 convert get verb misinterpreted in polls
  * fix #658 sr3 convert acceptUnmatch wrong on senders and polls
  * fix #653 invoking c components in the foreground
  * fix #665 polling default apache tabular file index 
  * fix foreground invocation of C components from sr3 did not work.

 -- Peter Silva <peter.silva@ssc-spc.gc.ca>  Tue, 25 Apr 2023 16:54:51 -0400

metpx-sr3 (3.00.37) unstable; urgency=medium

  * sr3 --dry_run --users declare works.
  * sr3 --dry_run stop works.
  * sr3 --dry_run cleanup works
  * sr3 --dry_run remove works. 
  * sr3 --dry_run sanity works.  Also sr3 sanity works. #647 
  * bugfix #626 re-implemented sanity_log_dead support in sr sanity
  * bugfix #647 sr3 sanity sometimes restarting things it should not.
  * bugfix: sr3 --users declare would declare users of remote brokers. as
    local ones
  * Requires network-online (stronger dependency in systemd unit file.)
  * improved behaviour of management cli with disabled configurations. #647
  * able to shutdown properly with when failing to connect to broker #648
  * fix for #651 crash in sr_watch.
  * replace // at root of sftp: and file: urls (sugar.)
  * removing some debug messages (too chatty even for debug.)
  * remove (useless) posts of directory modifications 
  * header option works on messages received, as well as originated ones.

 -- Peter <peter.silva@ssc-spc.gc.ca>  Fri, 14 Apr 2023 11:43:28 -0400

metpx-sr3 (3.00.36) unstable; urgency=medium

  * sr3 status now show wVip for processes waiting for a Vip. #614
  * add to_clusters header when encoding v02 messages for compatibility
  * get default post_baseDir from post_baseUrl  #641
  * redisqueue retry queue implementation based on redis thanks to Greg
    Linton! (EXPERIMENTAL)
  * made systemd unit file depend on network-online, rather than just up #645
  * when failing to rename a symlink, fallback should try to just create it
    with the new name, same for directories. before it just gave up. 
  * 

 -- Peter <peter.silva@ssc-spc.gc.ca>  Thu, 13 Apr 2023 09:48:52 -0400

metpx-sr3 (3.00.35) unstable; urgency=medium

  * bugfix #641 - baseDir it should remove baseDir from directory being
    written.

 -- Peter <peter.silva@ssc-spc.gc.ca>  Mon, 03 Apr 2023 17:17:08 -0400

metpx-sr3 (3.00.34) unstable; urgency=medium

  * slight patch (more symlink combined with strip stuff.) 
  * display changes.

 -- Peter <peter.silva@ssc-spc.gc.ca>  Mon, 03 Apr 2023 14:37:09 -0400

metpx-sr3 (3.00.33) unstable; urgency=medium

  * published to pypi (copy of 3.00.32) by mistake. deleted. will see if can
    re-publish later... perhaps will have to skip this version.
  * yup... cannot re-publish ... have to increment.

 -- Peter <peter.silva@ssc-spc.gc.ca>  Sun, 02 Apr 2023 10:03:59 -0400

metpx-sr3 (3.00.32) unstable; urgency=medium

  * slight patch to updateFieldsAccepted (more symlink stuff.) 

 -- Peter <peter.silva@ssc-spc.gc.ca>  Fri, 31 Mar 2023 08:50:00 -0400

metpx-sr3 (3.00.31) unstable; urgency=medium

  * another (final?) fix for ln -sf case (fileOp field transformation.)

 -- Peter <peter.silva@ssc-spc.gc.ca>  Wed, 29 Mar 2023 00:16:25 -0400

metpx-sr3 (3.00.30) unstable; urgency=medium

  * bugfix #638 discard uption not working in sr3
  * another fix for relative paths in certain case.

 -- Peter Silva <peter.silva@ssc-spc.gc.ca>  Tue, 28 Mar 2023 10:27:55 -0400

metpx-sr3 (3.00.29) unstable; urgency=medium

  * bugfix #632 create root of config and state directories when missing.
  * enhancement #614 adding verbosity & metrics to sr status.
  * fixes to processing when strip option is in use, particularly with
    symbolic links. 

 -- Peter <peter.silva@ssc-spc.gc.ca>  Thu, 09 Mar 2023 17:57:32 -0500

metpx-sr3 (3.00.28) unstable; urgency=medium

  * apply strip, pstrip, and flattern to paths in fileOp fields (rename & symlink) HPC mirroring.
  * fixed mqtt support (broken since 3.0.15) #506
  * bugfix for deleting exchanges only if unused #382
  * refactor class name "encoding" to postformat. #617
  * finished review of defaults from v2 #405
  * ported last plugins from v2 #433
  * ported last v2 (sundew porting) documentation to sr3
  * bugfix #621 removing double slashes in paths.
  * bugfix... queunames not being chosen properly. (multiple instances using different queues.) 
  * plugin api: sarracenia.Message.updatePaths added current values as defaults
  * plugin api: flowcb.accept.pathreplace uses improved sets 
  * plugin api: add_option( improved set support.) added all_values of set.

 -- Peter <peter.silva@ssc-spc.gc.ca>  Fri, 7 Mar 2023 10:48:17 -0500

metpx-sr3 (3.00.27) unstable; urgency=medium

  * adding directory creation/deletion events #558
  * re-introducing *recursive* option as a result of #558. to suppress when
  * directories are posted.
  * change set parsing to be less surprising #608

 -- Peter <peter.silva@ssc-spc.gc.ca>  Wed, 25 Jan 2023 18:07:29 -0500

metpx-sr3 (3.00.26) unstable; urgency=medium

  * #433 adding nodupe_fileAgeMin, and having nodupe honour inflight durations
  * #433 made nodupe.v2wrapper.sumstr
  * #433 made flowcb __init__ routine more robust to empty options for testing.
  * #605 attempt to fix delete option in file: transfer protocol
  * flowcb now accepts child class logger to propagate priority.

 -- Peter <peter.silva@ssc-spc.gc.ca>  Wed, 11 Jan 2023 13:41:21 -0500

metpx-sr3 (3.00.25) unstable; urgency=medium

  * change: default acceptUnmatched is now True everywhere ( #601)
  * change: translation of v2 poll *destination* & *get* to sr3 *path*
  * #433 add interpretation of offsets in strftime date patterns.
  * local file copy was broken.
  * documentation improvements
  * optimization: skip evaluating a regex if pattern is .*
  * #433 more advice/information about porting polls from v2 to sr3 
  * #433 added message downloader for mail servers flowcb/download/mail_ingest.py
  * #433 conversion of poll configurations improved to remove redundant accepts.
  * #433 much more explanations around v2->sr3 porting of polls
  * Sender setting change: remoteUrl -> sendTo (grandfathered with synonyms.)
  * bugfix: sr3 --users declare does nothing if no component configurations exist.
  * new default: when post_baseUrl use file:/... use /... to set post_baseDir.

 -- Peter Silva <peter.silva@ssc-spc.gc.ca>  Fri, 09 Dec 2022 11:02:27 -0500

metpx-sr3 (3.00.24) unstable; urgency=medium

  * critical: sr3 would crash when sum none present in a configuration.
  * documentation touchups to README.txt (shown on pypi.python.org)

 -- Peter Silva <peter.silva@ssc-spc.gc.ca>  Wed, 07 Dec 2022 15:42:47 -0500

metpx-sr3 (3.00.23) unstable; urgency=medium

  * new: date pattern expansion from datetime.strftime.
  * fixing #595 flows taking a long time to stop.
  * more robust decode of v2 messages.
  * added conversion of sum field to sr3 convert.
  * added flowcb.scheduled as built-in way to easily post periodically.
  * tilde expansion for pathnames for more settings.
  * configuration conversion now includes date patterns.
  * configuration conversion now includes sum/integrity option arguments.
  * variable expansion now added for path option.
  * more path expansion for settings likely to be paths.
  * added rename setting, that honours variable expansion.
  * vestiges of md5name (already gone from code) removed from documentation.
  * removed please_stop_immediately in favour of improved behaviour to always exit more quickly.
  * bugfix incorrectly generated v2 messages (sum: N instead of n)
  * bugfix, many date patterns not honoured correctly (different reasons.)
  * bugfix, sum z,x not parsed correctly.

 -- Peter Silva <peter.silva@ssc-spc.gc.ca>  Sat, 03 Dec 2022 19:53:21 +0000

metpx-sr3 (3.00.22) unstable; urgency=medium

  * Client seeing many bad messages. Force reading of entire file on transfers. warn when messages are "wrong"
  * AM sender and server plugins added. (Thanks André)
  * send only one message per rename operation (added v2compatRenameDoublePost to get old behaviour.)
  * bad logic in one case of rename, where dest file already exists but should be overwritten
  * added please_stop_immediately option. experimental means to accellerate shutdowns.
  * added varTimeOffset option. To modify date substitutions in variable expansion.
  * option change: destination -> pollUrl in polls.
  * option change: destination -> remoteUrl in senders.
  * added flowcb/poll/sr3bucket module.
  * added support for new_retPath for plugins (part of porting work.)
  * improved load_library module search to be easier/shorter to work with.
  * improved api Config.variableExpansion() entry point, for use in callbacks.
  * improved documentation of flowcallbacks and v2 porting advice.
  * flowcb/work/age.py did not report correct file ages. fixed.
  * callbackprepend was broken. fixed.
  * list option parsing was broken. fixed.
  * ftp server line parsing was broken. fixed.
  * fixes for duplicate suppression of cod integrity case.
  * removed unused vestiges of "index" files for ftp and sftp protocols.
  
 -- Peter Silva <peter.silva@ssc-spc.gc.ca>  Fri, 02 Dec 2022 10:32:41 -0500

metpx-sr3 (3.00.21) unstable; urgency=medium

  * added "flow" component, vanilla flow class, to increased flexibility. #575
  * more documentation on porting plugins from v2 to sr3 #589
  * documenting --dangerWillRobinson #582
  * sr3 stop no longer stops *foreground* flows without --dangerWillRobinson
  * worked through the "run" plugin, to run arbitrary code.
  * more bugfixes related to configuration file parsing.
  * changed some internal variables to apply camelCase
  * added expandvars and expanduser to credentials parsing.
  * better error messages when there are problems with credentials.
  * Many improvements to French documentation.
  * many corrected links in documents (mostly obsolete branch references.)
  * added plugins and configuration to reproduce #131
  * work-around for #563 to avoid declaring unnamed queues. 

 -- Peter Silva <peter.silva@ssc-spc.gc.ca>  Thu, 03 Nov 2022 12:47:41 -0400

metpx-sr3 (3.00.20) unstable; urgency=medium

  * dynamic flow CI/CD, for late propagation, use a retry_queue instead of blithely proceeding ( 6012c79105ee1f7c34f4bc81dc5e03271722a29b )
  * fixing #566, co-ordination among polls sharing a vip, so no missing files, no double posts.
  * fixing #566, better nodupe choices for performance with the cod integrity checksum case.
  * fixing #555 and #369 clean shutdown assured by having start/stop ignore foreground tasks.
  * fixes for postRateLimiting parsing, and invalid broker specifications.
  * more documentation/web site information in French. 

 -- Peter Silva <peter.silva@ssc-spc.gc.ca>  Mon, 17 Oct 2022 11:40:52 -0400

metpx-sr3 (3.00.19) unstable; urgency=medium

  * detection/fallbacks for python libraries to get partial functionality.

 -- Peter Silva <peter.silva@ssc-spc.gc.ca>  Wed, 05 Oct 2022 10:12:09 -0400

metpx-sr3 (3.00.18) unstable; urgency=medium

  * --dry_run now working for download and send.
  * -s now working.

 -- Peter Silva <peter@fractal>  Wed, 28 Sep 2022 08:39:06 -0400

metpx-sr3 (3.00.17) unstable; urgency=medium

  * #539 French documentation for sr3 now exists.
  * #547, #504 v03 post format revised to separate checksums from non-data file operations.
  * #466 retry queue for failed send, also post retry queue.
  * improved documentation #552 
  * #522 honor permLog 
  * #465 simulation mode for sender (not yet complete.)
  * #438 v2 to sr3 config converter added (still has some problems, but mostly working.)
  * #513 improved shutdown on signal receipt.  * 

 -- Peter Silva <peter.silva@ssc-spc.gc.ca>  Wed, 21 Sep 2022 11:59:03 -0400

metpx-sr3 (3.00.16) unstable; urgency=medium

  * wrapped all calls to callbacks with try/except and error messaging.
  * more obvious errors when extra dependencies are missing.
  * more documentation improvements.
  * change appauthor from 'science.gc.ca' to 'MetPX'

 -- Peter Silva <peter.silva@ssc-spc.gc.ca>  Sat, 02 Apr 2022 10:47:21 -0400

metpx-sr3 (3.00.15b2) unstable; urgency=medium

  * #490 implemented CI/CD matrix doing flow tests with multiple python versions.
  * #489 sourceFromExchange missing implementation.
  * #488 there was a problem with recovering from connection failures.
  * #487 fixed to ignore unjustified flow test failures.
  * #486 problem with inflight.
  * #483, #455, #479 debian packaging working again, based on use of extras (optional components.)
  * improved messaging of mdelaylatest flow callback.
  * large, numerous improvements in the documentation (approaching release quality.) 
  * tls_rigour becomes tlsRigour (in implementation, was only documented that way before.)
  * #480 sr3 status display problem not ignoring files that are not .conf ones.
  * #477 sr3 edit of default.conf, credentials.conf, admin.conf work again.
  * #447 sr3 disabled now with state file, not config.

 -- Peter Silva <peter.silva@ssc-spc.gc.ca>  Wed, 30 Mar 2022 22:04:37 -0400

metpx-sr3 (3.00.014b1) unstable; urgency=medium

  * forked off from v2.
  * has MQTT support.
  * just incrementing minor release as dev. 
  * non-stable releases for now.
  * beta: no more breaking changes expected.

 -- Peter Silva <peter.silva@ssc-spc.gc.ca>  Mon, 21 Mar 2022 09:18:01 -0400
