cback3-amazons3-sync(1)

NAME
cback3-amazons3-sync - Synchronize a local directory with an Amazon S3 bucket

SYNOPSIS
cback3-amazons3-sync [switches] sourceDir s3BucketUrl

DESCRIPTION

This is the Cedar Backup 3 Amazon S3 sync tool. It synchronizes a local directory to an Amazon S3 cloud storage bucket. After the sync is complete, a validation step is taken. An error is reported if the contents of the bucket do not match the source directory, or if the indicated size for any file differs.

Generally, one can run the cback3-amazons3-sync command with no special switches. This will start it using the default Cedar Backup log file, etc. You only need to use the switches if you need to change the default behavior.

MIGRATING FROM VERSION 2 TO VERSION 3

The main difference between Cedar Backup version 2 and Cedar Backup version 3 is the targeted Python interpreter. For most users, migration should be straightforward. See the discussion found at cback3(1) or reference the Cedar Backup user guide.

ARGUMENTS

sourceDir
    The source directory on a local disk.

s3BucketUrl
    The URL specifying the location of the Amazon S3 cloud storage bucket to synchronize with, like s3://example.com-backup/subdir.

SWITCHES

-h, --help
    Display usage/help listing.

-V, --version
    Display version information.

-b, --verbose
    Print verbose output to the screen as well writing to the logfile. When this option is enabled, most information that would normally be written to the logfile will also be written to the screen.

-l, --logfile
    Specify the path to an alternate logfile. The default logfile file is /var/log/cback3.log.

-o, --owner
    Specify the ownership of the logfile, in the form user:group. The default ownership is root:adm, to match the Debian standard for most logfiles. This value will only be used when creating a new logfile. If the logfile already exists when the cback3 script is executed, it will retain its existing ownership and mode. Only user and group names may be used, not numeric uid and gid values.

-m, --mode
    Specify the permissions for the logfile, using the numeric mode as in chmod(1). The default mode is 640 (-rw-r-----). This value will only be used when creating a new logfile. If the logfile already exists when the cback3 script is executed, it will retain its existing ownership and mode.

-O, --output
    Record some sub-command output to the logfile. When this option is enabled, all output from system commands will be logged. This might be useful for debugging or just for reference.

-d, --debug
    Write debugging information to the logfile. This option produces a high volume of output, and would generally only be needed when debugging a problem. This option implies the --output option, as well.

-s, --stack
    Dump a Python stack trace instead of swallowing exceptions. This forces Cedar Backup to dump the entire Python stack trace associated with an error, rather than just propagating last message it received back up to the user interface. Under some circumstances, this is useful information to include along with a bug report.

-D, --diagnostics
    Display runtime diagnostic information and then exit. This diagnostic information is often useful when filing a bug report.

-v, --verifyOnly
    Only verify the S3 bucket contents against the directory on disk. Do not make any changes to the S3 bucket or transfer any files. This is intended as a quick check to see whether the sync is up-to-date. Although no files are transferred, the tool will still execute the source filename encoding check.

-u, --uploadOnly
    Implement a partial or "upload only" sync, instead of a full synchronization. Normally, synchronization would remove files that exist in S3 but do not exist in the directory on disk. With this flag, new files are uploaded, but no files are removed in S3.

-w, --ignoreWarnings
    The AWS CLI S3 sync process is very picky about filename encoding. Files that the Linux filesystem handles with no problems can cause problems in S3 if the filename cannot be encoded properly in your configured locale. As of this writing, filenames like this will cause the sync process to abort without transferring all files as expected. To avoid confusion, the tool tries to guess which files in the source directory will cause problems, and refuses to execute the AWS CLI S3 sync if any problematic files exist. If you'd rather proceed anyway, use this flag.

RETURN VALUES

This command returns 0 (zero) upon normal completion, and several other error codes related to particular errors.

1   The Python interpreter version is < 3.7.

2   Error processing command-line arguments.

3   Error configuring logging.

5   Backup was interrupted with a CTRL-C or similar.

6   Other error during processing.

NOTES

This tool is a wrapper over the Amazon AWS CLI interface found in the aws(1) command. Specifically, cback3-amazons3-sync invokes "aws s3 sync" followed by "aws s3api list-objects".

Cedar Backup itself is designed to run as root. However, cback3-amazons3-sync can be run safely as any user that is configured to use the Amazon AWS CLI interface. The aws(1) command will be executed by the same user which is executing cback3-amazons3-sync.

You must configure the AWS CLI interface to have a valid connection to Amazon S3 infrastructure before using cback3-amazons3-sync. For more information about how to accomplish this, see the Cedar Backup user guide.

SEE ALSO
cback3(1)

FILES

/var/log/cback3.log - Default log file

URLS

The project homepage is: https://github.com/pronovic/cedar-backup3

BUGS

If you find a bug, please report it.

If possible, give me the output from --diagnostics, all of the error messages that the script printed into its log, and also any stack-traces (exceptions) that Python printed. It would be even better if you could tell me how to reproduce the problem, for instance by sending me your configuration file.

Report bugs to <support@cedar-solutions.com> or via GitHub issues tracker.

AUTHOR
Written and maintained by Kenneth J. Pronovici <pronovic@ieee.org> with contributions from others.

COPYRIGHT
Copyright (c) 2004-2020 Kenneth J. Pronovici.

This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.