Sourcecode / SVN

From duply (simple duplicity) - a duplicity shell frontend
Jump to navigation Jump to search


duply sourcecode

duply's svn repository is not the most recent source for the code. Usually the svn only gets updated on releases. But not always. Development snapshots are taken daily. These are not guaranteed to work, but might contain unreleased fixes. Check the snapshot's changelog header accordingly.

SVN access

Browse the repository

https://sourceforge.net/p/ftplicity/code/HEAD/tree/duply/trunk/

SVN access

svn co https://svn.code.sf.net/p/ftplicity/code/duply/trunk duply

SVN Log

https://sourceforge.net/p/ftplicity/code/commit_browser

Latest Development Snapshot

mod time 2021-09-17

plain/text -> duply.sh

   1 #!/usr/bin/env bash
   2 #
   3 ################################################################################
   4 #  duply (grown out of ftplicity), is a shell front end to duplicity that      #
   5 #  simplifies the usage by managing settings for backup jobs in profiles.      #
   6 #  It supports executing multiple commands in a batch mode to enable single    #
   7 #  line cron entries and executes pre/post backup scripts.                     #
   8 #  Since version 1.5.0 all duplicity backends are supported. Hence the name    #
   9 #  changed from ftplicity to duply.                                            #
  10 #  See http://duply.net or http://ftplicity.sourceforge.net/ for more info.    #
  11 #  (c) 2006 Christiane Ruetten, Heise Zeitschriften Verlag, Germany            #
  12 #  (c) 2008-2020 Edgar Soldin (changes since version 1.3)                      #
  13 ################################################################################
  14 #  LICENSE:                                                                    #
  15 #  This program is licensed under GPLv2.                                       #
  16 #  Please read the accompanying license information in gpl-2.0.txt.            #
  17 ################################################################################
  18 #  TODO/IDEAS/KNOWN PROBLEMS:
  19 #  - possibility to restore time frames (incl. deleted files)
  20 #    realizable by listing each backup and restore from 
  21 #    oldest to the newest, problem: not performant
  22 #  - search file in all backups function and show available
  23 #    versions with backups date (list old avail since 0.6.06)
  24 #  - edit profile opens conf file in vi 
  25 #  - implement log-fd interpretation
  26 #  - add a duplicity option check against the options pending 
  27 #    deprecation since 0.5.10 namely --time-separator
  28 #                               --short-filenames
  29 #                              --old-filenames
  30 #  - add 'exclude_<command>' list usage e.g. exclude_verify
  31 #  - featreq 25: a download/install duplicity option
  32 #  - hint on install software if a piece is missing
  33 #  - import/export profile from/to .tgz function !!!
  34 #
  35 #  CHANGELOG:
  36 #  2.3.2 (unreleased)
  37 #  - bugfix #127: date_from_nsecs ignores format string
  38 #
  39 #  2.3.1 (11.2.2021)
  40 #  - bugfix 123: symmetric encryption errs out, asks for '' private key
  41 #
  42 #  2.3 (30.12.2020)
  43 #  - don't import whole key pair anymore if only pub/sec is requested
  44 #  - gpg import routine informs on missing key files in profile now
  45 #  - add check/import needed secret key for decryption
  46 #  - featreq 50: Disable GPG key backups, implemented/added settings
  47 #      GPG_IMPORT/GPG_EXPORT='disabled' to conf template
  48 #
  49 #  2.2.2 (24.02.2020)
  50 #  - bugfix 120: Failures in "Autoset trust of key" during restore 
  51 #    because of gpg2.2 fingerprint output change
  52 #
  53 #  2.2.1 (22.01.2020)
  54 #  - featreq 46: Example systemd units & Howto, courtesy of Jozef Riha
  55 #  - featreq 47: Clarify message about keeping the profile, also by Jozef Riha
  56 #  - fix abbreviation spelling of 'e.g.'
  57 #
  58 #  2.2 (30.12.2018)
  59 #  - featreq 44: implement grouping for batch commands
  60 #      new separators are [] (square brackets) or groupIn/groupOut
  61 #      command 'backup' translates now to [pre_bkp_post] to be skipped as
  62 #      one block in case a condition was set in the batch instruction
  63 #
  64 #  2.1 (23.07.2018)
  65 #  - be more verbose when duplicity version detection fails
  66 #  - using info shows python binary's path for easier identification now
  67 #  - reworked python interpreter handling, it's either
  68 #      configured per PYTHON var
  69 #      unconfigured, parsed from duplicity shebang
  70 #      or set to current duplicity default 'python2' (was 'python' until now)
  71 #  - do not quotewrap strings because of slashes (e.g. paths) anymore
  72 #  - bugfix: improved in/exclude stripping from conf DUPL_PARAMS
  73 #
  74 #  2.0.4 (20.02.2018)
  75 #  - bugfix 114: "duply usage is not current" wrt. purgeFull/Incr
  76 #  - bugfix 115: typo in error message - "Not GPG_KEY entries" should be "No"
  77 #  - bugfix 117: no duply_ prefix when ARCH_DIR is set in conf
  78 #  - bugfix debian 882159: duply: occasionally shows negative runtimes
  79 #
  80 #  2.0.3 (29.08.2017)
  81 #  - bugfix: "line 2231: CMDS: bad array subscript"
  82 #  - bugfix 112: "env: illegal option -- u" on MacOSX
  83 #
  84 #  2.0.2 (23.05.2017)
  85 #  - bugfix: never insert creds into file:// targets
  86 #  - bugfix: avail profiles hint sometimes shortend the names by one char
  87 #  - bugfix 108: CMD_NEXT variable should ignore conditional commands (and, or)
  88 #  - export condition before/after next/prev command as CND_PREV,CND_NEXT now
  89 #  - bugfix 97: Unknown command should be ERROR, not WARNING 
  90 #
  91 #  2.0.1 (16.11.2016)
  92 #  - bugfix 104: Duply 2.0 sets wrong archive dir, --name was always 'duply_'
  93 #
  94 #  2.0 (27.10.2016)
  95 #  made this a major version change, as we broke backward compatibility anyway
  96 #  (see last change in v1.10). got complaints that rightfully pointed out
  97 #  that should only come w/ a major version change. so, here we go ;)
  98 #  if your backend stops working w/ this version create a new profile and
  99 #  export the env vars needed as described in the comments of the conf file
 100 #  directly above the SOURCE setting.
 101 #  Changes:
 102 #  - making sure multi spaces in TARGET survive awk processing
 103 #  - new env var PROFILE exported to scripts 
 104 #  - fix 102: expose a unique timestamp variable for pre/post scripts
 105 #    actually a featreq. exporting RUN_START nanosec unix timestamp
 106 #  - fix 101: GPG_AGENT_INFO is 'bogus' (thx Thomas Harning Jr.)
 107 #  - fix 96: duply cannot handle two consecutive spaces in paths
 108 #
 109 #  1.11.3 (29.5.2016)
 110 #  - fix wrong "WARNING: No running gpg-agent ..." when sign key was not set
 111 #
 112 #  1.11.2 (11.2.2016)
 113 #  - fix "gpg: unsafe" version print out 
 114 #  - bugfix 91: v1.11 [r47] broke asymmetric encryption when using GPG_KEYS_ENC
 115 #  - bugfix 90: S3: TARGET_USER/PASS have no effect, added additional 
 116 #    documentation about needed env vars to template conf file
 117 #
 118 #  1.11.1 (18.12.2015)
 119 #  - bugfix 89: "Duply has trouble with PYTHON-interpreter" on OSX homebrew
 120 #  - reverted duply's default PYTHON to 'python'
 121 #
 122 #  1.11 (24.11.2015)
 123 #  - remove obsolete --ssh-askpass routine
 124 #  - add PYTHON conf var to allow global override of used python interpreter
 125 #  - enforced usage of "python2" in PATH as default interpreter for internal
 126 #    use _and_ to run duplicity (setup.py changed the shebang to the fixed
 127 #    path /usr/bin/python until 0.7.05, which we circumvent this way)
 128 #  - featreq 36: support gpg-connect-agent as a means to detect if an agent is 
 129 #    running (thx Thomas Harning Jr.), used gpg-agent for detection though
 130 #  - quotewrapped run_cmd parameters to protect it from spaces e.g. in TMP path
 131 #  - key export routine respects gpg-agent usage now
 132 #
 133 #  1.10.1 (19.8.2015)
 134 #  - bugfix 86: Duply+Swift outputs warning
 135 #  - bugfix 87: Swift fails without BACKEND_URL
 136 #
 137 #  1.10 (31.7.2015)
 138 #  - featreq 37: busybox issues - fix awk, grep version detection,
 139 #    fix grep failure because --color=never switch is unsupported
 140 #    (thx Thomas Harning Jr. for reporting and helping to debug/fix it)
 141 #  - bugfix 81: --exclude-globbing-filelist is deprecated since 0.7.03
 142 #    (thx Joachim Wiedorn, also for maintaining the debian package)
 143 #  - implemented base-/dirname as bash functions
 144 #  - featreq 31 " Support for duplicity Azure backend " - ignored a 
 145 #    contributed patch by Scott McKenzie and instead opted for removing almost
 146 #    all code that deals with special env vars required by backends.
 147 #    adding and modifying these results in too much overhead so i dropped this
 148 #    feature. the future alternative for users is to consult the duplicity 
 149 #    manpage and add the needed export definitions to the conf file.
 150 #    appended a commented example to the template conf below the auth section.
 151 #
 152 #  1.9.2 (21.6.2015)
 153 #  - bugfix: exporting keys with gpg2.1 works now (thx Philip Jocks)
 154 #  - documented GPG_OPTS needed for gpg2.1 to conf template (thx Troy Engel)
 155 #  - bugfix 82: GREP_OPTIONS=--color=always disrupted time calculation
 156 #  - added GPG conf var (see conf template for details)
 157 #  - added grep version output as it is an integral needed binary
 158 #  - added PYTHONPATH printout in version output
 159 #
 160 #  1.9.1 (13.10.2014)
 161 #  - export CMD_ERR now for scripts to detect if CMD_PREV failed/succeeded
 162 #  - bugfix: CMD_PREV contained command even if it was skipped
 163 #
 164 #  1.9.0 (24.8.2014)
 165 #  - bugfix: env vars were not exported when external script was executable
 166 #  - rework GPG_KEY handling, allow virtually anything now (uid, keyid etc.) 
 167 #    see gpg manpage, section "How to specify a user ID"
 168 #    let gpg complain when the delivered values are invalid for whatever reason
 169 #  - started to rework tmp space checking, exposed folder & writable check
 170 #    TODO: reimplement enough file space available checking
 171 #
 172 #  1.8.0 (13.7.2014)
 173 #  - add command verifyPath to expose 'verify --file-to-restore' action
 174 #  - add time parameter support to verify command
 175 #  - add section time formats to usage output 
 176 #
 177 #  1.7.4 (24.6.2014)
 178 #  - remove ubuntu one support, service is discontinued
 179 #  - featreq 31: add authenticated swift (contributed by Justus Seifert)
 180 #
 181 #  1.7.3 (3.4.2014)
 182 #  - bugfix: test routines, gpg2 asked for passphrase although GPG_PW was set
 183 #
 184 #  1.7.2 (1.4.2014 "April,April")
 185 #  - bugfix: debian Bug#743190 "duply no longer allows restoration without 
 186 #     gpg passphrase in conf file"
 187 #     GPG_AGENT_INFO env var is now needed to trigger --use-agent
 188 #  - bugfix: gpg keyenc test routines didn't work if GPG_PW was not set
 189 #
 190 #  1.7.1 (30.3.2014)
 191 #  - bugfix: purge-* commands renamed to purgeFull, purgeIncr due to 
 192 #     incompatibility with new minus batch separator 
 193 #
 194 #  1.7.0 (20.3.2014)
 195 #  - disabled gpg key id plausibility check, too many valid possibilities
 196 #  - featreq 7 "Halt if precondition fails":
 197 #     added and(+), or(-) batch command(separator) support
 198 #  - featreq 26 "pre/post script with shebang line": 
 199 #     if a script is flagged executable it's executed in a subshell 
 200 #     now as opposed to sourced to bash, which is the default
 201 #  - bugfix: do not check if dpbx, swift credentials are set anymore 
 202 #  - bugfix: properly escape profile name, archdir if used as arguments
 203 #  - add DUPL_PRECMD conf setting for use with e.g. trickle
 204 #
 205 #  1.6.0 (1.1.2014)
 206 #  - support gs backend
 207 #  - support dropbox backend
 208 #  - add gpg-agent support to gpg test routines
 209 #  - autoenable --use-agent if passwords were not defined in config
 210 #  - GPG_OPTS are now honored everywhere, keyrings or complete gpg
 211 #    homedir can thus be configured to be located anywhere
 212 #  - always import both secret and public key if avail from config profile
 213 #  - new explanatory comments in initial exclude file
 214 #  - bugfix 7: Duply only imports one key at a time 
 215 #
 216 #  1.5.11 (19.07.2013)
 217 #  - purge-incr command for remove-all-inc-of-but-n-full feature added
 218 #    patch provided by Moritz Augsburger, thanks!
 219 #  - documented version command in man page
 220 #
 221 #  1.5.10 (26.03.2013)
 222 #  - minor indent and documentation fixes
 223 #  - bugfix: exclude filter failed on ubuntu, mawk w/o posix char class support
 224 #  - bugfix: fix url_decoding generally and for python3
 225 #  - bugfix 3609075: wrong script results in status line (thx David Epping)
 226 #
 227 #  1.5.9 (22.11.2012)
 228 #  - bugfix 3588926: filter --exclude* params for restore/fetch ate too much
 229 #  - restore/fetch now also ignores --include* or --exclude='foobar' 
 230 #
 231 #  1.5.8 (26.10.2012)
 232 #  - bugfix 3575487: implement proper cloud files support
 233 #
 234 #  1.5.7 (10.06.2012)
 235 #  - bugfix 3531450: Cannot use space in target URL (file:///) anymore
 236 #
 237 #  1.5.6 (24.5.2012)
 238 #  - commands purge, purge-full have no default value anymore for security 
 239 #    reasons; instead max value can be given via cmd line or must be set
 240 #    in profile; else an error is shown.
 241 #  - minor man page modifications
 242 #
 243 #  versioning scheme will be simplified to [major].[minor].[patch] version
 244 #  with the next version raise
 245 #
 246 #  1.5.5.5 (4.2.2012)
 247 #  - bugfix 3479605: SEL context confused profile folder's permission check
 248 #  - colon ':' in url passphrase got ignored, added python driven url_decoding
 249 #    for user & pass to better process special chars
 250 #
 251 #  1.5.5.4 (16.10.2011)
 252 #  - bugfix 3421268: SFTP passwords from conf ignored and always prompted for
 253 #  - add support for separate sign passphrase (needs duplicity 0.6.14+)
 254 #
 255 #  1.5.5.3 (1.10.2011)
 256 #  - bugfix 3416690: preview threw echo1 error
 257 #  - fix unknown cmds error usage & friends if more than 2 params were given
 258 #
 259 #  1.5.5.2 (23.9.2011)
 260 #  - bugfix 3409643: ssh key auth did ask for passphrase (--ssh-askpass ?)
 261 #  - bugfix: mawk does not support \W and did not split multikey definitions
 262 #  - all parameters should survive  single (') and double (") quotes now
 263 #
 264 #  1.5.5.1 (7.6.2011)
 265 #  - featreq 3311881: add ftps as supported by duplicity 0.6.13 (thx mape2k)
 266 #  - bugfix 3312208: signing detection broke symmetric gpg test routine
 267 #
 268 #  1.5.5 (2.5.2011)
 269 #  - bugfix: fetch problem with space char in path, escape all params 
 270 #    containing non word chars
 271 #  - list available profiles, if given profile cannot be found
 272 #  - added --use-agent configuration hint
 273 #  - bugfix 3174133: --exclude* params in conf DUPL_PARAMS broke 
 274 #    fetch/restore
 275 #  - version command now prints out 'using installed' info
 276 #  - featreq 3166169: autotrust imported keys, based on code submitted by 
 277 #    Martin Ellis - imported keys are now automagically trusted ultimately 
 278 #  - new txt2man feature to create manpages for package maintainers
 279 #
 280 #  1.5.4.2 (6.1.2011)
 281 #  - new command changelog
 282 #  - bugfix 3109884: freebsd awk segfaulted on printf '%*', use print again
 283 #  - bugfix: freebsd awk hangs on 'awk -W version' 
 284 #  - bugfix 3150244: mawk does not know '--version'
 285 #  - minor help text improvements
 286 #  - new env vars CMD_PREV,CMD_NEXT replacing CMD env var for scripts
 287 #
 288 #  1.5.4.1 (4.12.2010)
 289 #  - output awk, python, bash version now in prolog
 290 #  - shebang uses /usr/bin/env now for freebsd compatibility, 
 291 #    bash not in /bin/bash 
 292 #  - new --disable-encryption parameter, 
 293 #    to override profile encr settings for one run
 294 #  - added exclude-if-present setting to conf template
 295 #  - bug 3126972: GPG_PW only needed for signing/symmetric encryption 
 296 #    (even though duplicity still needs it)
 297 #
 298 #  1.5.4 (15.11.2010)
 299 #  - as of 1.5.3 already, new ARCH_DIR config option
 300 #  - multiple key support
 301 #  - ftplicity-Feature Requests-2994929: separate encryption and signing key
 302 #  - key signing of symmetric encryption possible (duplicity patch committed)
 303 #  - gpg tests disable switch
 304 #  - gpg tests now previewable and more intelligent
 305 #
 306 #  1.5.3 (1.11.2010)
 307 #  - bugfix 3056628: improve busybox compatibility, grep did not have -m param
 308 #  - bugfix 2995408: allow empty password for PGP key
 309 #  - bugfix 2996459: Duply erroneously escapes '-' symbol in username
 310 #  - url_encode function is now pythonized
 311 #  - rsync uses FTP_PASSWORD now if duplicity 0.6.10+ , else issue warning
 312 #  - feature 3059262: Make pre and post aware of parameters, 
 313 #                     internal parameters + CMD of pre or post 
 314 #
 315 #  1.5.2.3 (16.4.2010)
 316 #  - bugfix: date again, should now work virtually anywhere
 317 #
 318 #  1.5.2.2 (3.4.2010)
 319 #  - minor bugfix: duplicity 0.6.8b version string now parsable
 320 #  - added INSTALL.txt
 321 #
 322 #  1.5.2.1 (23.3.2010)
 323 #  - bugfix: date formatting is awked now and should work on all platforms
 324 #
 325 #  1.5.2 (2.3.2010)
 326 #  - bugfix: errors print to STD_ERR now, failed tasks print an error message
 327 #  - added --name=duply_<profile> for duplicity 0.6.01+ to name cache folder
 328 #  - simplified & cleaned profileless commands, removed second instance
 329 #  - generalized separator time routines
 330 #  - added support for --no-encryption (GPG_KEY='disabled'), see conf examples
 331 #  - minor fixes
 332 #
 333 #  1.5.1.5 (5.2.2010)
 334 #  - bugfix: added special handling of credentials for rsync, imap(s)
 335 #
 336 #  1.5.1.4 (7.1.2010)
 337 #  - bugfix: nsecs defaults now to zeroes if date does not deliver [0-9]{9}
 338 #  - check if ncftp binary is available if url protocol is ftp
 339 #  - bugfix: duplicity output is now printed to screen directly to resolve
 340 #            'mem alloc problem' bug report
 341 #  - bugfix: passwords will not be in the url anymore to solve the 'duply shows
 342 #            sensitive data in process listing' bug report
 343 #
 344 #  1.5.1.3 (24.12.2009) 'merry xmas'
 345 #  - bugfix: gpg pass now apostrophed to allow space and friends
 346 #  - bugfix: credentials are now url encoded to allow special chars in them
 347 #            a note about url encoding has been added to the conf template
 348 #
 349 #  1.5.1.2 (1.11.2009)
 350 #  - bugfix: open parenthesis in password broke duplicity execution
 351 #  - bugfix: ssh/scp backend does not always need credentials e.g. key auth
 352 #
 353 #  1.5.1.1 (21.09.2009)
 354 #  - bugfix: fixed s3[+http] TARGET_PASS not needed routine
 355 #  - bugfix: TYPO in duply 1.5.1 prohibited the use of /etc/duply
 356 #    see https://sourceforge.net/tracker/index.php?func=detail&
 357 #                aid=2864410&group_id=217745&atid=1041147
 358 #
 359 #  1.5.1 (21.09.2009) - duply (fka. ftplicity)
 360 #  - first things first: ftplicity (being able to support all backends since 
 361 #    some time) will be called duply (fka. ftplicity) from now on. The addendum
 362 #    is for the time being to circumvent confusion.
 363 #  - bugfix: exit code is 1 (error) not 0 (success), if at least on duplicity 
 364 #            command failed
 365 #  - s3[+http] now supported natively by translating user/pass to access_key/
 366 #    secret_key environment variables needed by duplicity s3 boto backend 
 367 #  - bugfix: additional output lines do not confuse version check anymore
 368 #  - list command supports now age parameter (patch by stefan on feature 
 369 #    request tracker)
 370 #  - bugfix: option/param pairs are now correctly passed on to duplicity
 371 #  - bugfix: s3[+http] needs no TARGET_PASS if command is read only
 372 #
 373 #  1.5.0.2 (31.07.1009)
 374 #  - bugfix: insert password in target url didn't work with debian mawk
 375 #            related to previous bug report
 376 #
 377 #  1.5.0.1 (23.07.2009)
 378 #  - bugfix: gawk gensub dependency raised an error on debian's default mawk
 379 #            replaced with match/substr command combination (bug report)
 380 #            https://sf.net/tracker/?func=detail&atid=1041147&aid=2825388&
 381 #            group_id=217745
 382 #
 383 #  1.5.0 (01.07.2009)
 384 #  - removed ftp limitation, all duplicity backends should work now
 385 #  - bugfix: date for separator failed on openwrt busybox date, added a 
 386 #    detecting workaround, milliseconds are not available w/ busybox date
 387 #
 388 #  1.4.2.1 (14.05.2009)
 389 #  - bugfix: free temp space detection failed with lvm, fixed awk parse routine
 390 #
 391 #  1.4.2 (22.04.2009)
 392 #  - gpg keys are now exported as gpgkey.[id].asc , the suffix reflects the
 393 #    armored ascii nature, the id helps if the key is switched for some reason
 394 #    im/export routines are updated accordingly (import is backward compatible 
 395 #    to the old profile/gpgkey files)         
 396 #  - profile argument is treated as path if it contains slashes 
 397 #    (for details see usage)
 398 #  - non-ftplicity options (all but --preview currently) are now passed 
 399 #    on to duplicity 
 400 #  - removed need for stat in secure_conf, it is ls based now
 401 #  - added profile folder readable check
 402 #  - added gpg version & home info output
 403 #  - awk utility availability is now checked, because it was mandatory already
 404 #  - tmp space is now checked on writability and space requirement
 405 #    test fails on less than 25MB or configured $VOLSIZE, 
 406 #    test warns if there is less than two times $VOLSIZE because 
 407 #    that's required for --asynchronous-upload option  
 408 #  - gpg functionality is tested now before executing duplicity 
 409 #    test drive contains encryption, decryption, comparison, cleanup
 410 #    this is meant to detect non trusted or other gpg errors early
 411 #  - added possibility of doing symmetric encryption with duplicity
 412 #    set GPG_KEY="" or simply comment it out
 413 #  - added hints in config template on the depreciation of 
 414 #    --short-filenames, --time-separator duplicity options
 415 #
 416 #  new versioning scheme 1.4.2b => 1.4.2, 
 417 #  beta b's are replaced by a patch count number e.g. 1.4.2.1 will be assigned
 418 #  to the first bug fixing version and 1.4.2.2 to the second and so on
 419 #  also the releases will now have a release date formatted (Day.Month.Year)
 420 #
 421 #  1.4.1b1 - bugfix: ftplicity changed filesystem permission of a folder
 422 #            named exactly as the profile if existing in executing dir
 423 #          - improved plausibility checking of config and profile folder
 424 #          - secure_conf only acts if needed and prints a warning now
 425 #
 426 #  1.4.1b  - introduce status (duplicity collection-status) command
 427 #          - pre/post script output printed always now, not only on errors
 428 #          - new config parameter GPG_OPTS to pass gpg options
 429 #            added examples & comments to profile template conf
 430 #          - reworked separator times, added duration display
 431 #          - added --preview switch, to preview generated command lines
 432 #          - disabled MAX_AGE, MAX_FULL_BACKUPS, VERBOSITY in generated
 433 #            profiles because they have reasonable defaults now if not set
 434 #
 435 #  1.4.0b1 - bugfix: incr forces incremental backups on duplicity,
 436 #            therefore backup translates to pre_bkp_post now
 437 #          - bugfix: new command bkp, which represents duplicity's 
 438 #            default action (incr or full if full_if_older matches
 439 #            or no earlier backup chain is found)
 440 #
 441 #  new versioning scheme 1.4 => 1.4.0, added new minor revision number
 442 #  this is meant to slow down the rapid version growing but still keep 
 443 #  versions cleanly separated.
 444 #  only additional features will raise the new minor revision number. 
 445 #  all releases start as beta, each bugfix release will raise the beta 
 446 #  count, usually new features arrive before a version 'ripes' to stable
 447 #    
 448 #  1.4.0b
 449 #    1.4b  - added startup info on version, time, selected profile
 450 #          - added time output to separation lines
 451 #          - introduced: command purge-full implements duplicity's 
 452 #            remove-all-but-n-full functionality (patch by unknown),
 453 #            uses config variable $MAX_FULL_BACKUPS (default = 1)
 454 #          - purge config var $MAX_AGE defaults to 1M (month) now 
 455 #          - command full does not execute pre/post anymore
 456 #            use batch command pre_full_post if needed 
 457 #          - introduced batch mode cmd1_cmd2_etc
 458 #            (in turn removed the bvp command)
 459 #          - unknown/undefined command issues a warning/error now
 460 #          - bugfix: version check works with 0.4.2 and older now
 461 #    1.3b3 - introduced pre/post commands to execute/debug scripts
 462 #          - introduced bvp (backup, verify, purge)
 463 #          - bugfix: removed need for awk gensub, now mawk compatible
 464 #    1.3b2 - removed pre/post need executable bit set 
 465 #          - profiles now under ~/.ftplicity as folders
 466 #          - root can keep profiles in /etc/ftplicity, folder must be
 467 #            created by hand, existing profiles must be moved there
 468 #          - removed ftplicity in path requirement
 469 #          - bugfix: bash < v.3 did not know '=~'
 470 #          - bugfix: purge works again 
 471 #    1.3   - introduces multiple profiles support
 472 #          - modified some script errors/docs
 473 #          - reordered gpg key check import routine
 474 #          - added 'gpg key id not set' check
 475 #          - added error_gpg (adds how to setup gpg key howto)
 476 #          - bugfix: duplicity 0.4.4RC4+ parameter syntax changed
 477 #          - duplicity_version_check routine introduced
 478 #          - added time separator, shortnames, volsize, full_if_older 
 479 #            duplicity options to config file (inspired by stevie 
 480 #            from http://weareroot.de) 
 481 #    1.1.1 - bugfix: encryption reactivated
 482 #    1.1   - introduced config directory
 483 #    1.0   - first release
 484 ################################################################################
 485 
 486 # utility functions overriding binaries
 487 
 488 # wrap grep to override possible env set GREP_OPTIONS=--color=always
 489 function grep {
 490   command env "GREP_OPTIONS=" grep "$@"
 491 }
 492 
 493 # implement basename in plain bash
 494 function basename {
 495   local stripped="${1%/}"
 496   echo "${stripped##*/}"
 497 }
 498 
 499 # implement dirname in plain bash
 500 function dirname {
 501   echo ${1%/*}
 502 }
 503 
 504 # implement basic which in plain bash
 505 function which {
 506   type -p "$@"
 507 }
 508 
 509 # check availability of executables via file name or file paths
 510 function lookup {
 511   local bin="$1"
 512   # look for file names in path via bash hash OR 
 513   # look for executables at given relative/absolute location
 514   ( [ "${bin##*/}" == "$bin" ] && hash "$bin" 2>/dev/null ) || [ -x "$bin" ]
 515 }
 516 
 517 # the python binary to use, exit code 0 when configured, else 1
 518 function python_binary {
 519   # if unset, parse from duplicity shebang
 520   if ! var_isset 'PYTHON'; then
 521     duplicity_python_binary_parse;
 522     echo $DUPL_PYTHON_BIN;
 523     return 1;
 524   else
 525     # tell if PYTHON was configured manually
 526     echo $PYTHON;
 527     return 0
 528   fi
 529 }
 530 
 531 # important definitions #######################################################
 532 
 533 ME_LONG="$0"
 534 ME="$(basename $0)"
 535 ME_NAME="${ME%%.*}"
 536 ME_VERSION="2.3.2dev"
 537 ME_WEBSITE="http://duply.net"
 538 
 539 # default config values
 540 DEFAULT_SOURCE='/path/of/source'
 541 DEFAULT_TARGET='scheme://user[:password]@host[:port]/[/]path'
 542 DEFAULT_TARGET_USER='_backend_username_'
 543 DEFAULT_TARGET_PASS='_backend_password_'
 544 DEFAULT_GPG='gpg'
 545 DEFAULT_GPG_KEY='_KEY_ID_'
 546 DEFAULT_GPG_PW='_GPG_PASSWORD_'
 547 DEFAULT_PYTHON='python2'
 548 
 549 # function definitions ##########################
 550 
 551 function set_config { # sets global config vars
 552   local CONFHOME_COMPAT="$HOME/.ftplicity"
 553   local CONFHOME="$HOME/.duply"
 554   local CONFHOME_ETC_COMPAT="/etc/ftplicity"
 555   local CONFHOME_ETC="/etc/duply"
 556 
 557   # confdir can be delivered as path (must contain /)
 558   if [ `echo $FTPLCFG | grep /` ] ; then 
 559     CONFDIR=$(readlink -f $FTPLCFG 2>/dev/null || \
 560               ( echo $FTPLCFG|grep -v '^/' 1>/dev/null 2>&1 \
 561                && echo $(pwd)/${FTPLCFG} ) || \
 562               echo ${FTPLCFG})          
 563   # or DEFAULT in home/.duply folder (NEW)
 564   elif [ -d "${CONFHOME}" ]; then
 565     CONFDIR="${CONFHOME}/${FTPLCFG}"
 566   # or in home/.ftplicity folder (OLD)
 567   elif [ -d "${CONFHOME_COMPAT}" ]; then
 568     CONFDIR="${CONFHOME_COMPAT}/${FTPLCFG}"
 569     warning_oldhome "${CONFHOME_COMPAT}" "${CONFHOME}"
 570   # root can put profiles under /etc/duply (NEW) if path exists
 571   elif [ -d "${CONFHOME_ETC}" ] && [ "$EUID" -eq 0 ]; then
 572     CONFDIR="${CONFHOME_ETC}/${FTPLCFG}"
 573   # root can keep profiles under /etc/ftplicity (OLD) if path exists
 574   elif [ -d "${CONFHOME_ETC_COMPAT}" ] && [ "$EUID" -eq 0 ]; then
 575     CONFDIR="${CONFHOME_ETC_COMPAT}/${FTPLCFG}"
 576     warning_oldhome "${CONFHOME_ETC_COMPAT}" "${CONFHOME_ETC}"
 577   # hmm no profile folder there, then use default for error later
 578   else
 579     CONFDIR="${CONFHOME}/${FTPLCFG}" # continue, will fail later in main
 580   fi
 581 
 582   # remove trailing slash, get profile name etc.
 583   CONFDIR="${CONFDIR%/}"
 584   PROFILE="${CONFDIR##*/}"
 585   CONF="$CONFDIR/conf"
 586   PRE="$CONFDIR/pre"
 587   POST="$CONFDIR/post"
 588   EXCLUDE="$CONFDIR/exclude"
 589   KEYFILE="$CONFDIR/gpgkey.asc"
 590 }
 591 
 592 function version_info { # print version information
 593   cat <<END
 594   $ME_NAME version $ME_VERSION
 595   ($ME_WEBSITE)
 596 END
 597 }
 598 
 599 function version_info_using { 
 600   cat <<END
 601 $(version_info)
 602 
 603   $(using_info)
 604 END
 605 }
 606 
 607 function using_info {
 608   # init needed vars into global name space
 609   lookup duplicity && { duplicity_python_binary_parse; duplicity_version_get; }
 610   local NOTFOUND="INVALID"
 611   local AWK_VERSION GREP_VERSION PYTHON_RUNNER
 612   # freebsd awk (--version only), debian mawk (-W version only), deliver '' so awk does not wait for input
 613   AWK_VERSION=$( lookup awk && (awk --version 2>/dev/null || awk -W version 2>&1) | awk 'NR<=2&&tolower($0)~/(busybox|awk)/{success=1;print;exit} END{if(success<1) print "unknown"}' || echo "$NOTFOUND" )
 614   GREP_VERSION=$( lookup grep && grep --version 2>&1 | awk 'NR<=2&&tolower($0)~/(busybox|grep.*[0-9]+\.[0-9]+)/{success=1;print;exit} END{if(success<1) print "unknown"}' || echo "$NOTFOUND" )
 615   PYTHON_RUNNER=$(python_binary)
 616   local PYTHON_VERSION=$(lookup "$PYTHON_RUNNER" && "$PYTHON_RUNNER" -V 2>&1| awk '{print tolower($0);exit}' || echo "'$PYTHON_RUNNER' $NOTFOUND" )
 617   local GPG_INFO=$(gpg_avail && gpg --version 2>&1| awk '/^gpg.*[0-9\.]+$/&&length(v)<1{v=$1" "$3}/^Home:/{h=" ("$0")"}END{print v""h}' || echo "gpg $NOTFOUND")
 618   local BASH_VERSION=$(bash --version | awk 'NR==1{IGNORECASE=1;sub(/GNU bash, version[ ]+/,"",$0);print $0}')
 619   # print out
 620   echo -e "Using installed duplicity version ${DUPL_VERSION:-$NOTFOUND}\
 621 ${PYTHON_VERSION+, $PYTHON_VERSION ${PYTHON_RUNNER:+($(which "$PYTHON_RUNNER"))}${PYTHONPATH:+ 'PYTHONPATH=$PYTHONPATH'}}\
 622 ${GPG_INFO:+, $GPG_INFO}${AWK_VERSION:+, awk '${AWK_VERSION}'}${GREP_VERSION:+, grep '${GREP_VERSION}'}\
 623 ${BASH_VERSION:+, bash '${BASH_VERSION}'}."
 624 }
 625 
 626 function usage_info { # print usage information
 627 
 628   cat <<USAGE_EOF
 629 VERSION:
 630 $(version_info)
 631   
 632 DESCRIPTION: 
 633   Duply deals as a wrapper for the mighty duplicity magic.
 634   It simplifies running duplicity with cron or on command line by:
 635 
 636     - keeping recurring settings in profiles per backup job
 637     - enabling batch operations e.g. backup_verify+purge
 638     - executing pre/post scripts (different actions possible 
 639       depending on previous or next command or it's exit status)
 640     - precondition checking for flawless duplicity operation
 641 
 642   For each backup job one configuration profile must be created.
 643   The profile folder will be stored under '~/.${ME_NAME}/<profile>'
 644   (where ~ is the current users home directory).
 645   Hint:  
 646    If the folder '/etc/${ME_NAME}' exists, the profiles for the super
 647    user root will be searched & created there.
 648 
 649 USAGE:
 650   first time usage (profile creation):  
 651     $ME <profile> create
 652 
 653   general usage in single or batch mode (see EXAMPLES):  
 654     $ME <profile> <command>[[_|+|-]<command>[_|+|-]...] [<options> ...]
 655 
 656   For batches the conditional separators can also be written as pseudo commands
 657   and(+), or(-). See SEPARATORS for details.
 658 
 659   Non $ME options are passed on to duplicity (see OPTIONS).
 660   All conf parameters can also be defined in the environment instead.
 661 
 662 PROFILE:
 663   Indicated by a path or a profile name (<profile>), which is resolved 
 664   to '~/.${ME_NAME}/<profile>' (~ expands to environment variable \$HOME).
 665 
 666   Superuser root can place profiles under '/etc/${ME_NAME}'. Simply create
 667   the folder manually before running $ME_NAME as superuser.
 668   Note:  
 669     Already existing profiles in root's home folder will cease to work
 670     unless they are moved to the new location manually.
 671 
 672   example 1:   $ME humbug backup
 673 
 674   Alternatively a _path_ might be used e.g. useful for quick testing, 
 675   restoring or exotic locations. Shell expansion should work as usual.
 676   Hint:  
 677     The path must contain at least one path separator '/', 
 678     e.g. './test' instead of only 'test'.
 679 
 680   example 2:   $ME ~/.${ME_NAME}/humbug backup
 681 
 682 SEPARATORS:
 683   _ (underscore)  
 684              neutral separator
 685   + (plus sign), _and_  
 686              conditional AND
 687              the next command will only be executed if the previous succeeded
 688   - (minus sign), _or_  
 689              conditional OR
 690              the next command will only be executed if the previous failed
 691   [] (square brackets), _groupIn_/_groupOut_  
 692              enables grouping of commands
 693 
 694    example:  
 695     'pre+[bkp-verify]_post' translates to
 696     'pre_and_groupIn_bkp_or_verify_groupOut_post'
 697 
 698 COMMANDS:
 699   usage      get usage help text
 700 
 701   and/or/groupIn/groupOut  
 702              pseudo commands used in batches (see SEPARATORS above)
 703 
 704   create     creates a configuration profile
 705   backup     backup with pre/post script execution (batch: [pre_bkp_post]),
 706               full (if full_if_older matches or no earlier backup is found)
 707               incremental (in all other cases)
 708   pre/post   execute '<profile>/$(basename "$PRE")', '<profile>/$(basename "$POST")' scripts
 709   bkp        as above but without executing pre/post scripts
 710   full       force full backup
 711   incr       force incremental backup
 712   list [<age>]  
 713              list all files in backup (as it was at <age>, default: now)
 714   status     prints backup sets and chains currently in repository
 715   verify [<age>] [--compare-data]  
 716              list files changed, since age if given
 717   verifyPath <rel_path_in_bkp> <local_path> [<age>] [--compare-data]  
 718              list changes of a file or folder path in backup compared to a
 719              local path, since age if given
 720   restore <target_path> [<age>]  
 721              restore the complete backup to <target_path> [as it was at <age>]
 722   fetch <src_path> <target_path> [<age>]  
 723              fetch single file/folder from backup [as it was at <age>]
 724   purge [<max_age>] [--force]  
 725              list outdated backup files (older than \$MAX_AGE)
 726               [use --force to actually delete these files]
 727   purgeFull [<max_full_backups>] [--force]  
 728              list outdated backup files (\$MAX_FULL_BACKUPS being the number of
 729              full backups and associated incrementals to keep, counting in 
 730              reverse chronological order)
 731               [use --force to actually delete these files]
 732   purgeIncr [<max_fulls_with_incrs>] [--force]  
 733              list outdated incremental backups (\$MAX_FULLS_WITH_INCRS being 
 734              the number of full backups which associated incrementals will be
 735              kept, counting in reverse chronological order) 
 736               [use --force to actually delete these files]
 737   cleanup [--force]  
 738              list broken backup chain files archives (e.g. after unfinished run)
 739               [use --force to actually delete these files]
 740 
 741   changelog  print changelog / todo list
 742   txt2man    feature for package maintainers - create a manpage based on the 
 743              usage output. download txt2man from http://mvertes.free.fr/, put 
 744              it in the PATH and run '$ME txt2man' to create a man page.
 745   version    show version information of $ME_NAME and needed programs
 746 
 747 OPTIONS:
 748   --force    passed to duplicity (see commands: 
 749              purge, purgeFull, purgeIncr, cleanup)
 750   --preview  do nothing but print out generated duplicity command lines
 751   --disable-encryption  
 752              disable encryption, overrides profile settings
 753 
 754 TIME FORMATS:
 755   For all time related parameters like age, max_age etc.
 756   Refer to the duplicity manpage for all available formats. Here some examples:
 757     2002-01-25T07:00:00+02:00 (full date time format string)
 758     2002/3/5 (date string YYYY/MM/DD)
 759     12D (interval, 12 days ago)
 760     1h78m (interval, 1 hour 78 minutes ago)
 761 
 762 PRE/POST SCRIPTS:
 763   Some useful internal duply variables are exported to the scripts.
 764 
 765     PROFILE, CONFDIR, SOURCE, TARGET_URL_<PROT|HOSTPATH|USER|PASS>, 
 766     GPG_<KEYS_ENC|KEY_SIGN|PW>, CMD_ERR, RUN_START,
 767     CMD_<PREV|NEXT> (previous/next command), 
 768     CND_<PREV|NEXT> (condition before/after)
 769 
 770   The CMD_* variables were introduced to allow different actions according to 
 771   the command the scripts were attached to e.g. 'pre_bkp_post_pre_verify_post' 
 772   will call the pre script two times, with CMD_NEXT variable set to 'bkp' 
 773   on the first and to 'verify' on the second run.
 774   CMD_ERR holds the exit code of the CMD_PREV .
 775 
 776 EXAMPLES:
 777   create profile 'humbug':  
 778     $ME humbug create (don't forget to edit this new conf file)
 779   backup 'humbug' now:  
 780     $ME humbug backup
 781   list available backup sets of profile 'humbug':  
 782     $ME humbug status
 783   list and delete outdated backups of 'humbug':  
 784     $ME humbug purge --force
 785   restore latest backup of 'humbug' to /mnt/restore:  
 786     $ME humbug restore /mnt/restore
 787   restore /etc/passwd of 'humbug' from 4 days ago to /root/pw:  
 788     $ME humbug fetch etc/passwd /root/pw 4D
 789     (see "duplicity manpage", section TIME FORMATS)
 790   a one line batch job on 'humbug' for cron execution:  
 791     $ME humbug backup_verify_purge --force
 792   batch job to run a full backup with pre/post scripts:  
 793     $ME humbug pre_full_post
 794 
 795 FILES:
 796   in profile folder '~/.${ME_NAME}/<profile>' or '/etc/${ME_NAME}'
 797   conf             profile configuration file
 798   pre,post         pre/post scripts (see above for details)
 799   gpgkey.*.asc     exported GPG key files
 800   exclude          a globbing list of included or excluded files/folders
 801                    (see "duplicity manpage", section FILE SELECTION)
 802 
 803 $(hint_profile)
 804 
 805 SEE ALSO:
 806   duplicity man page:
 807     duplicity(1) or http://duplicity.nongnu.org/duplicity.1.html
 808 USAGE_EOF
 809 }
 810 
 811 # to check call 'duply txt2man | man -l -'
 812 function usage_txt2man {
 813   usage_info | \
 814   awk '/^^[^[:lower:][:space:]][^[:lower:]]+$/{gsub(/[^[:upper:]]/," ",$0)}{print}' |\
 815   txt2man -t"$(toupper "${ME_NAME}")" -s1 -r"${ME_NAME}-${ME_VERSION}" -v'User Manuals'
 816 }
 817 
 818 function changelog {
 819   cat $ME_LONG | awk '/^#####/{on=on+1}(on==3){sub(/^#(  )?/,"",$0);print}'
 820 }
 821 
 822 function create_config {
 823   if [ ! -d "$CONFDIR" ] ; then
 824     mkdir -p "$CONFDIR" || error "Couldn't create config '$CONFDIR'."
 825   # create initial config file
 826     cat <<EOF >"$CONF"
 827 # gpg encryption settings, simple settings:
 828 #  GPG_KEY='disabled' - disables encryption alltogether
 829 #  GPG_KEY='<key1>[,<key2>]'; GPG_PW='pass' - encrypt with keys,
 830 #   sign if secret key of key1 is available use GPG_PW for sign & decrypt
 831 #  Note: you can specify keys via all methods described in gpg manpage,
 832 #        section "How to specify a user ID", escape commas (,) via backslash (\)
 833 #        e.g. 'Mueller, Horst', 'Bernd' -> 'Mueller\, Horst, Bernd'
 834 #        as they are used to separate the entries
 835 #  GPG_PW='passphrase' - symmetric encryption using passphrase only
 836 GPG_KEY='${DEFAULT_GPG_KEY}'
 837 GPG_PW='${DEFAULT_GPG_PW}'
 838 # gpg encryption settings in detail (extended settings)
 839 #  the above settings translate to the following more specific settings
 840 #  GPG_KEYS_ENC='<keyid1>[,<keyid2>,...]' - list of pubkeys to encrypt to
 841 #  GPG_KEY_SIGN='<keyid1>|disabled' - a secret key for signing
 842 #  GPG_PW='<passphrase>' - needed for signing, decryption and symmetric
 843 #   encryption. If you want to deliver different passphrases for e.g. 
 844 #   several keys or symmetric encryption plus key signing you can use
 845 #   gpg-agent. Simply make sure that GPG_AGENT_INFO is set in environment.
 846 #   also see "A NOTE ON SYMMETRIC ENCRYPTION AND SIGNING" in duplicity manpage 
 847 # notes on en/decryption
 848 #  private key and passphrase will only be needed for decryption or signing.
 849 #  decryption happens on restore and incrementals (compare archdir contents).
 850 #  for security reasons it makes sense to separate the signing key from the
 851 #  encryption keys. https://answers.launchpad.net/duplicity/+question/107216
 852 #GPG_KEYS_ENC='<pubkey1>,<pubkey2>,...'
 853 #GPG_KEY_SIGN='<prvkey>'
 854 # set if signing key passphrase differs from encryption (key) passphrase
 855 # NOTE: available since duplicity 0.6.14, translates to SIGN_PASSPHRASE
 856 #GPG_PW_SIGN='<signpass>'
 857 
 858 # uncomment and set a file path or name force duply to use this gpg executable
 859 # available in duplicity 0.7.04 and above (currently unreleased 06/2015)
 860 #GPG='/usr/local/gpg-2.1/bin/gpg'
 861 
 862 # gpg options passed from duplicity to gpg process (default='')
 863 # e.g. "--trust-model pgp|classic|direct|always" 
 864 #   or "--compress-algo=bzip2 --bzip2-compress-level=9"
 865 #   or "--personal-cipher-preferences AES256,AES192,AES..."
 866 #   or "--homedir ~/.duply" - keep keyring and gpg settings duply specific
 867 #   or "--pinentry-mode loopback" - needed for GPG 2.1+ _and_
 868 #      also enable allow-loopback-pinentry in your .gnupg/gpg-agent.conf
 869 #GPG_OPTS=''
 870 
 871 # disable preliminary tests with the following setting
 872 #GPG_TEST='disabled'
 873 # disable automatic gpg key importing altogether
 874 #GPG_IMPORT='disabled'
 875 # disable automatic gpg key exporting to profile folder
 876 #GPG_EXPORT='disabled'
 877 
 878 # backend, credentials & location of the backup target (URL-Format)
 879 # generic syntax is
 880 #   scheme://[user[:password]@]host[:port]/[/]path
 881 # e.g.
 882 #   sftp://bob:secret@backupserver.com//home/bob/dupbkp
 883 # for details and available backends see duplicity manpage, section URL Format
 884 #   http://duplicity.nongnu.org/duplicity.1.html#sect7
 885 # BE AWARE:
 886 #   some backends (cloudfiles, S3 etc.) need additional env vars to be set to
 887 #   work properly, read after the TARGET definition for more details.
 888 # ATTENTION:
 889 #   characters other than A-Za-z0-9.-_.~ in the URL have to be
 890 #   replaced by their url encoded pendants, see
 891 #     http://en.wikipedia.org/wiki/Url_encoding
 892 #   if you define the credentials as TARGET_USER, TARGET_PASS below $ME
 893 #   will try to url_encode them for you if the need arises.
 894 TARGET='${DEFAULT_TARGET}'
 895 # optionally the username/password can be defined as extra variables
 896 # setting them here _and_ in TARGET results in an error
 897 # ATTENTION:
 898 #   there are backends that do not support the user/pass auth scheme.
 899 #   prominent examples are S3, Azure, Cloudfiles. when in doubt consult the
 900 #   duplicity manpage. usually there is a NOTE section explaining if and which
 901 #   env vars should be set.
 902 #TARGET_USER='${DEFAULT_TARGET_USER}'
 903 #TARGET_PASS='${DEFAULT_TARGET_PASS}'
 904 # e.g. for cloud files backend it might look like this (uncomment for use!)
 905 #export CLOUDFILES_USERNAME='someuser'
 906 #export CLOUDFILES_APIKEY='somekey'
 907 #export CLOUDFILES_AUTHURL ='someurl'
 908 # the following is an incomplete list (<backend>: comma separated env vars list)
 909 # Azure: AZURE_ACCOUNT_NAME, AZURE_ACCOUNT_KEY
 910 # Cloudfiles: CLOUDFILES_USERNAME, CLOUDFILES_APIKEY, CLOUDFILES_AUTHURL
 911 # Google Cloud Storage: GS_ACCESS_KEY_ID, GS_SECRET_ACCESS_KEY
 912 # Pydrive: GOOGLE_DRIVE_ACCOUNT_KEY, GOOGLE_DRIVE_SETTINGS
 913 # S3: AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY
 914 # Swift: SWIFT_USERNAME, SWIFT_PASSWORD, SWIFT_AUTHURL,
 915 #        SWIFT_TENANTNAME OR SWIFT_PREAUTHURL, SWIFT_PREAUTHTOKEN
 916 
 917 # base directory to backup
 918 SOURCE='${DEFAULT_SOURCE}'
 919 
 920 # a command that runs duplicity e.g. 
 921 #  shape bandwidth use via trickle
 922 #  "trickle -s -u 640 -d 5120" # 5Mb up, 40Mb down"
 923 #DUPL_PRECMD=""
 924 
 925 # override the used python interpreter, defaults to
 926 #  - parsed result of duplicity's shebang or 'python2'
 927 #   e.g. "python2" or "/usr/bin/python2.7"
 928 #PYTHON="python"
 929 
 930 # exclude folders containing exclusion file (since duplicity 0.5.14)
 931 # Uncomment the following two lines to enable this setting.
 932 #FILENAME='.duplicity-ignore'
 933 #DUPL_PARAMS="\$DUPL_PARAMS --exclude-if-present '\$FILENAME'"
 934 
 935 # Time frame for old backups to keep, Used for the "purge" command.  
 936 # see duplicity man page, chapter TIME_FORMATS)
 937 #MAX_AGE=1M
 938 
 939 # Number of full backups to keep. Used for the "purgeFull" command. 
 940 # See duplicity man page, action "remove-all-but-n-full".
 941 #MAX_FULL_BACKUPS=1
 942 
 943 # Number of full backups for which incrementals will be kept for.
 944 # Used for the "purgeIncr" command.
 945 # See duplicity man page, action "remove-all-inc-of-but-n-full".
 946 #MAX_FULLS_WITH_INCRS=1
 947 
 948 # activates duplicity --full-if-older-than option (since duplicity v0.4.4.RC3) 
 949 # forces a full backup if last full backup reaches a specified age, for the 
 950 # format of MAX_FULLBKP_AGE see duplicity man page, chapter TIME_FORMATS
 951 # Uncomment the following two lines to enable this setting.
 952 #MAX_FULLBKP_AGE=1M
 953 #DUPL_PARAMS="\$DUPL_PARAMS --full-if-older-than \$MAX_FULLBKP_AGE " 
 954 
 955 # sets duplicity --volsize option (available since v0.4.3.RC7)
 956 # set the size of backup chunks to VOLSIZE MB instead of the default 25MB.
 957 # VOLSIZE must be number of MB's to set the volume size to.
 958 # Uncomment the following two lines to enable this setting. 
 959 #VOLSIZE=50
 960 #DUPL_PARAMS="\$DUPL_PARAMS --volsize \$VOLSIZE "
 961 
 962 # verbosity of output (error 0, warning 1-2, notice 3-4, info 5-8, debug 9)
 963 # default is 4, if not set
 964 #VERBOSITY=5
 965 
 966 # temporary file space. at least the size of the biggest file in backup
 967 # for a successful restoration process. (default is '/tmp', if not set)
 968 #TEMP_DIR=/tmp
 969 
 970 # Modifies archive-dir option (since 0.6.0) Defines a folder that holds 
 971 # unencrypted meta data of the backup, enabling new incrementals without the 
 972 # need to decrypt backend metadata first. If empty or deleted somehow, the 
 973 # private key and it's password are needed.
 974 # NOTE: This is confidential data. Put it somewhere safe. It can grow quite 
 975 #       big over time so you might want to put it not in the home dir.
 976 # default '~/.cache/duplicity/duply_<profile>/'
 977 # if set  '\${ARCH_DIR}/<profile>'
 978 #ARCH_DIR=/some/space/safe/.duply-cache
 979 
 980 # DEPRECATED setting
 981 # sets duplicity --time-separator option (since v0.4.4.RC2) to allow users 
 982 # to change the time separator from ':' to another character that will work 
 983 # on their system.  HINT: For Windows SMB shares, use --time-separator='_'.
 984 # NOTE: '-' is not valid as it conflicts with date separator.
 985 # ATTENTION: only use this with duplicity < 0.5.10, since then default file 
 986 #            naming is compatible and this option is pending depreciation 
 987 #DUPL_PARAMS="\$DUPL_PARAMS --time-separator _ "
 988 
 989 # DEPRECATED setting
 990 # activates duplicity --short-filenames option, when uploading to a file
 991 # system that can't have filenames longer than 30 characters (e.g. Mac OS 8)
 992 # or have problems with ':' as part of the filename (e.g. Microsoft Windows)
 993 # ATTENTION: only use this with duplicity < 0.5.10, later versions default file 
 994 #            naming is compatible and this option is pending depreciation
 995 #DUPL_PARAMS="\$DUPL_PARAMS --short-filenames "
 996 
 997 # more duplicity command line options can be added in the following way
 998 # don't forget to leave a separating space char at the end
 999 #DUPL_PARAMS="\$DUPL_PARAMS --put_your_options_here " 
1000 
1001 EOF
1002 
1003 # create initial exclude file
1004     cat <<EOF >"$EXCLUDE"
1005 # although called exclude, this file is actually a globbing file list
1006 # duplicity accepts some globbing patterns, even including ones here
1007 # here is an example, this incl. only 'dir/bar' except it's subfolder 'foo'
1008 # - dir/bar/foo
1009 # + dir/bar
1010 # - **
1011 # for more details see duplicity manpage, section File Selection
1012 # http://duplicity.nongnu.org/duplicity.1.html#sect9
1013 
1014 EOF
1015 
1016   # Hints on first usage
1017   cat <<EOF
1018 
1019 Congratulations. You just created the profile '$FTPLCFG'.
1020 The initial config file has been created as 
1021 '$CONF'.
1022 You should now adjust this config file to your needs.
1023 
1024 $(hint_profile)
1025 
1026 EOF
1027 fi
1028 
1029 }
1030 
1031 # used in usage AND create_config
1032 function hint_profile {
1033   cat <<EOF
1034 IMPORTANT:
1035   Copy the _whole_ profile folder after the first backup to a safe place.
1036   It contains everything (duply related) needed to restore your backups. 
1037 
1038   Pay attention to (possibly later added) external files such as credentials
1039   or auth files (e.g. netrc, .megarc, ssh keys) or environment variables
1040   (e.g. DPBX_ACCESS_TOKEN). 
1041   It is good policy to place those in the profile folder if possible at all.
1042     e.g. in case of 'multi://' target the config .json file
1043   Env vars should be added to duply profiles' conf file.
1044 
1045   Keep access to these files restricted as they contain information (gpg key,
1046   passphrases etc.) to access and modify your backups.
1047   
1048   Finally:
1049   You should attempt a restore from an unrelated host to be sure you really
1050   have everything needed for restoration.
1051 
1052   Repeat these steps after _all_ configuration changes. Some configuration 
1053   options are crucial for restoration.
1054 
1055 EOF
1056 }
1057 
1058 function separator {
1059   echo "--- $@ ---"
1060 }
1061 
1062 function inform {
1063   echo -e "\nINFO:\n\n$@\n"
1064 }
1065 
1066 function warning {
1067   echo -e "\nWARNING:\n\n$@\n"
1068 }
1069 
1070 function warning_oldhome {
1071   local old=$1 new=$2
1072   warning " ftplicity changed name to duply since you created your profiles.
1073   Please rename the old folder
1074   '$old'
1075   to
1076   '$new'
1077   and this warning will disappear.
1078   If you decide not to do so profiles will _only_ work from the old location."
1079 }
1080 
1081 function error_print {
1082   echo -e "$@" >&2
1083 }
1084 
1085 function error {
1086   error_print "\nSorry. A fatal ERROR occured:\n\n$@\n"
1087   exit -1
1088 }
1089 
1090 function error_gpg {
1091   [ -n "$2" ] && local hint="\n  $2\n\n  "
1092   
1093   error "$1
1094 
1095 Hint${hint:+s}:
1096   ${hint}Maybe you have not created a gpg key yet (e.g. gpg --gen-key)?
1097   Don't forget the used _password_ as you will need it.
1098   When done enter the 8 digit id & the password in the profile conf file.
1099 
1100   The key id can be found doing a 'gpg --list-keys'. In the example output 
1101   below the key id for the public key would be FFFFFFFF.
1102 
1103   pub   1024D/FFFFFFFF 2007-12-17
1104   uid                  duplicity
1105   sub   2048g/899FE27F 2007-12-17
1106 "
1107 }
1108 
1109 function error_gpg_test {
1110   [ -n "$2" ] && local hint="\n  $2\n\n  "
1111 
1112   error "$1
1113 
1114 Hint${hint:+s}:
1115   ${hint}This error means that gpg is probably misconfigured or not working 
1116   correctly. The error message above should help to solve the problem.
1117   However, if for some reason $ME_NAME should misinterpret the situation you 
1118   can define GPG_TEST='disabled' in the conf file to bypass the test.
1119   Please do not forget to report the bug in order to resolve the problem
1120   in future versions of $ME_NAME.
1121 "
1122 }
1123 
1124 function error_path {
1125   error "$@
1126 PATH='$PATH'
1127 "
1128 }
1129 
1130 function error_to_string {
1131 	[ -n "$1" ] && [ "$1" -eq 0 ] && echo "OK" || echo "FAILED 'code $1'"
1132 }
1133 
1134 function duplicity_version_get {
1135   # nothing to do, just print
1136   var_isset DUPL_VERSION && return
1137   
1138   local DUPL_VERSION_OUT DUPL_VERSION_AWK PYTHON_BIN CMD='duplicity'
1139   # only run with a user specific python if configured (running by default
1140   # breaks homebrew as they place a shell wrapper for duplicity in path)
1141   PYTHON_BIN="$(python_binary)" &&\
1142     CMD="$(qw "$PYTHON_BIN") $(which $CMD)"
1143   CMD="$CMD --version 2>&1"
1144   DUPL_VERSION_OUT=$(eval "$CMD")
1145   DUPL_VERSION=`echo $DUPL_VERSION_OUT | awk '/^duplicity /{print $2; exit;}'`
1146   #DUPL_VERSION='0.7.03' #'0.6.08b' #,0.4.4.RC4,0.6.08b
1147   DUPL_VERSION_VALUE=0
1148   DUPL_VERSION_AWK=$(awk -v v="$DUPL_VERSION" 'BEGIN{
1149   if (match(v,/[^\.0-9]+[0-9]*$/)){
1150     rest=substr(v,RSTART,RLENGTH);v=substr(v,0,RSTART-1);}
1151   if (pos=match(rest,/RC([0-9]+)$/)) rc=substr(rest,pos+2)
1152   split(v,f,"[. ]"); if(f[1]f[2]f[3]~/^[0-9]+$/) vvalue=f[1]*10000+f[2]*100+f[3]; else vvalue=0
1153   print "#"v"_"rest"("rc"):"f[1]"-"f[2]"-"f[3]
1154   print "DUPL_VERSION_VALUE=\047"vvalue"\047"
1155   print "DUPL_VERSION_RC=\047"rc"\047"
1156   print "DUPL_VERSION_SUFFIX=\047"rest"\047"
1157   }')
1158   eval "$DUPL_VERSION_AWK"
1159   #echo -e ",$DUPL_VERSION,$DUPL_VERSION_VALUE,$DUPL_VERSION_RC,$DUPL_VERSION_SUFFIX,"
1160 
1161   # doublecheck findings and report error
1162   if [ $DUPL_VERSION_VALUE -eq 0 ]; then
1163     inform "duplicity version check failed (please report, this is a bug)
1164 the command
1165   $CMD
1166 resulted in
1167   $DUPL_VERSION_OUT
1168 "
1169   elif [ $DUPL_VERSION_VALUE -le 404 ] && [ ${DUPL_VERSION_RC:-4} -lt 4 ]; then
1170     error "The installed version $DUPL_VERSION is incompatible with $ME_NAME v$ME_VERSION.
1171 You should upgrade your version of duplicity to at least v0.4.4RC4 or
1172 use the older ftplicity version 1.1.1 from $ME_WEBSITE."
1173   fi
1174 }
1175 
1176 function duplicity_version_ge {
1177   [ "$DUPL_VERSION_VALUE" -ge "$1" ]
1178 }
1179 
1180 function duplicity_version_lt {
1181   ! duplicity_version_ge "$1"
1182 }
1183 
1184 # parse interpreter from duplicity shebang
1185 function duplicity_python_binary_parse {
1186   # cached result
1187   ( var_isset 'PYTHON' || var_isset 'DUPL_PYTHON_BIN' ) && return
1188 
1189   # parse it or warn
1190   local DUPL_BIN=$(which duplicity)
1191   DUPL_PYTHON_BIN=$(awk 'NR==1&&/^#!/{sub(/^#!( *\/usr\/bin\/env *)?/,""); print}' < "$DUPL_BIN")
1192   if ! echo "$DUPL_PYTHON_BIN" | grep -q -i 'python'; then
1193     warning "Could not parse the python interpreter used from duplicity ($DUPL_BIN). Result was '$DUPL_PYTHON_BIN'.
1194 Will assume it is '$DEFAULT_PYTHON'."
1195     DUPL_PYTHON_BIN="$DEFAULT_PYTHON"
1196   fi
1197 }
1198 
1199 function run_script { # run pre/post scripts
1200   local ERR=0
1201   local SCRIPT="$1"
1202   if [ ! -z "$PREVIEW" ] ; then	
1203     echo "$([ ! -x "$SCRIPT" ] && echo ". ")$SCRIPT"
1204   elif [ -r "$SCRIPT" ] ; then 
1205     echo -n "Running '$SCRIPT' "
1206     if [ -x "$SCRIPT" ]; then
1207       OUT=$("$SCRIPT" 2>&1)
1208       ERR=$?
1209     else
1210       OUT=$(. "$SCRIPT" 2>&1)
1211       ERR=$?
1212     fi
1213     [ $ERR -eq "0" ] && echo "- OK" || echo "- FAILED (code $ERR)"
1214     echo -en ${OUT:+"Output: $OUT\n"} ;
1215   else
1216     echo "Skipping n/a script '$SCRIPT'."
1217   fi
1218   return $ERR
1219 }
1220 
1221 function run_cmd {
1222   # run or print escaped cmd string
1223   local CMD_ERR=0
1224   if [ -n "$PREVIEW" ]; then
1225     CMD_OUT=$( echo "$@ 2>&1" )
1226     CMD_MSG="-- Run cmd -- $CMD_MSG --\n$CMD_OUT"
1227   elif [ -n "$CMD_DISABLED" ]; then
1228     CMD_MSG="$CMD_MSG (DISABLED) - $CMD_DISABLED"
1229   else
1230     echo -n -e "$CMD_MSG"
1231     CMD_OUT=` eval "$@" 2>&1 `
1232     CMD_ERR=$?
1233     if [ "$CMD_ERR" = "0" ]; then
1234       CMD_MSG=" (OK)"
1235     else
1236       CMD_MSG=" (FAILED)"
1237     fi
1238   fi
1239   echo -e "$CMD_MSG"
1240   # reset
1241   unset CMD_DISABLED CMD_MSG
1242   return $CMD_ERR
1243 }
1244 
1245 function qw { quotewrap "$@"; }
1246 
1247 function quotewrap {
1248   local param="$@"
1249   # quote strings having non word chars (e.g. spaces)
1250   if echo "$param"  | awk '/[^A-Za-z0-9_\.\-\/]/{exit 0}{exit 1}'; then
1251     echo "$param" | awk '{\
1252       gsub(/[\047]/,"\047\\\047\047",$0);\
1253       gsub(/[\042]/,"\047\\\042\047",$0);\
1254       print "\047"$0"\047"}'
1255     return
1256   fi
1257   echo $param
1258 }
1259 
1260 function duplicity_params_global {
1261   # already done? return
1262   var_isset 'DUPL_PARAMS_GLOBAL' && return
1263   local DUPL_ARG_ENC
1264 
1265   # use key only if set in config, else leave it to symmetric encryption
1266   if gpg_disabled; then
1267     local DUPL_PARAM_ENC='--no-encryption'
1268   else
1269     local DUPL_PARAM_ENC=$(gpg_prefix_keyset '--encrypt-key' "${GPG_KEYS_ENC_ARRAY[@]}")
1270     gpg_signing && local DUPL_PARAM_SIGN=$(gpg_prefix_keyset '--sign-key' "$GPG_KEY_SIGN")
1271     # interpret password settings
1272     var_isset 'GPG_PW' && DUPL_ARG_ENC="PASSPHRASE=$(qw "${GPG_PW}")"
1273     var_isset 'GPG_PW_SIGN' && DUPL_ARG_ENC="${DUPL_ARG_ENC} SIGN_PASSPHRASE=$(qw "${GPG_PW_SIGN}")"
1274   fi
1275 
1276   local GPG_OPTS=${GPG_OPTS:+"--gpg-options $(qw "${GPG_OPTS}")"}
1277 
1278   # set name for dupl archive folder, since 0.6.0
1279   if duplicity_version_ge 601; then
1280     local DUPL_ARCHDIR=''
1281     if var_isset 'ARCH_DIR'; then
1282       DUPL_ARCHDIR="--archive-dir $(qw "${ARCH_DIR}")"
1283       # reuse erronously duply_ prefixed folders from bug #117
1284       if [ -d "$ARCH_DIR/duply_${PROFILE}" ]; then
1285         DUPL_ARCHDIR="${DUPL_ARCHDIR} --name $(qw "duply_${PROFILE}")"
1286       else
1287         DUPL_ARCHDIR="${DUPL_ARCHDIR} --name $(qw "${PROFILE}")"
1288       fi
1289     else
1290       DUPL_ARCHDIR="--name $(qw "duply_${PROFILE}")"
1291     fi
1292   fi
1293 
1294 DUPL_PARAMS_GLOBAL="${DUPL_ARCHDIR} ${DUPL_PARAM_ENC} \
1295 ${DUPL_PARAM_SIGN} --verbosity '${VERBOSITY:-4}' \
1296  ${GPG_OPTS}"
1297 
1298 DUPL_VARS_GLOBAL="TMPDIR='$TEMP_DIR' \
1299  ${DUPL_ARG_ENC}"
1300 }
1301 
1302 # function to filter the DUPL_PARAMS var from user conf
1303 function duplicity_params_conf {
1304   # reuse cmd var from main loop
1305   ## in/exclude parameters are currently not supported on restores
1306   if [ "$cmd" = "fetch" ] || [ "$cmd" = "restore" ] || [ "$cmd" = "status" ]; then
1307     # filter exclude params from fetch/restore/status
1308     eval "stripXcludes $DUPL_PARAMS"
1309     return
1310   fi
1311 
1312   # nothing done, print unchanged
1313   echo "$DUPL_PARAMS"
1314 }
1315 
1316 # strip in/exclude parameters from param string
1317 function stripXcludes {
1318   local STRIPNEXT OUT;
1319   for p in "$@"; do
1320     if [ -n "$STRIPNEXT" ]; then
1321       unset STRIPNEXT
1322       # strip the value of previous parameter
1323       continue
1324     elif echo "$p" | awk '/^\-\-(in|ex)clude(\-[a-zA-Z]+)?$/{exit 0;}{exit 1;}'; then
1325       # strips e.g. --include /foo/bar
1326       STRIPNEXT="yes"
1327       continue
1328     elif echo "$p" | awk '/^\-\-(in|ex)clude(\-[a-zA-Z]+)?=/{exit 0;}{exit 1;}'; then
1329       # strips e.g. --include=/foo/bar
1330       continue
1331     fi
1332     
1333     OUT="$OUT $(qw "$p")"
1334   done
1335   echo "$OUT"
1336 }
1337 
1338 function duplify { # the actual wrapper function
1339   local PARAMSNOW DUPL_CMD DUPL_CMD_PARAMS
1340 
1341   # put command (with params) first in duplicity parameters
1342   for param in "$@" ; do
1343   # split cmd from params (everything before splitchar --)
1344     if [ "$param" == "--" ] ; then
1345       PARAMSNOW=1
1346     else
1347       # wrap in quotes to protect from spaces
1348       [ ! $PARAMSNOW ] && \
1349         DUPL_CMD="$DUPL_CMD $(qw "$param")" \
1350       || \
1351         DUPL_CMD_PARAMS="$DUPL_CMD_PARAMS $(qw "$param")"
1352     fi
1353   done
1354 
1355   # init global duplicity parameters same for all tasks
1356   duplicity_params_global
1357 
1358   local RUN=eval BIN=duplicity DUPL_BIN PYTHON_BIN
1359   # run in cmd line preview mode if requested
1360   var_isset 'PREVIEW' && RUN=echo
1361   # try to resolve duplicity path for usage with python interpreter
1362   DUPL_BIN=$(which "$BIN") || DUPL_BIN="$BIN"
1363   # only run with a user specific python if configured (running by default
1364   # breaks homebrew as they place a shell wrapper for duplicity in path)
1365   PYTHON_BIN="$(python_binary)" &&\
1366     BIN="$(qw "$PYTHON_BIN") $(qw "$DUPL_BIN")"
1367 
1368 $RUN "${DUPL_VARS_GLOBAL} ${BACKEND_PARAMS}\
1369  ${DUPL_PRECMD} $BIN $DUPL_CMD $DUPL_PARAMS_GLOBAL $(duplicity_params_conf)\
1370  $GPG_USEAGENT $(gpg_custom_binary) $DUPL_CMD_PARAMS"
1371 
1372   local ERR=$?
1373   return $ERR
1374 }
1375 
1376 function secureconf { # secure the configuration dir
1377 	#PERMS=$(ls -la $(dirname $CONFDIR) | grep -e " $(basename $CONFDIR)\$" | awk '{print $1}')
1378 	local PERMS="$(ls -la "$CONFDIR/." | awk 'NR==2{print $1}')"
1379 	if [ "${PERMS/#drwx------*/OK}" != 'OK' ] ; then
1380 		chmod u+rwX,go= "$CONFDIR"; local ERR=$?
1381 		warning "The profile's folder 
1382 '$CONFDIR'
1383 permissions are not safe ($PERMS). Secure them now. - ($(error_to_string $ERR))"
1384 	fi
1385 }
1386 
1387 # params are $1=timeformatstring (default like date output), $2=epoch seconds since 1.1.1970 (default now)
1388 function date_fix {
1389 	local DEFAULTFORMAT='%a %b %d %H:%M:%S %Z %Y'
1390 	local date
1391 	#[ "$1" == "%N" ] && return #test the no nsec test below
1392 	# gnu date with -d @epoch
1393 	date=$(date ${2:+-d @$2} ${1:++"$1"} 2> /dev/null) && \
1394 		echo $date && return
1395 	# date bsd,osx with -r epoch
1396 	date=$(date ${2:+-r $2} ${1:++"$1"} 2> /dev/null) && \
1397 		echo $date && return	
1398 	# date busybox with -d epoch -D %s
1399 	date=$(date ${2:+-d $2 -D %s} ${1:++"$1"} 2> /dev/null) && \
1400 		echo $date && return
1401 	## some date commands do not support giving a time w/o setting it systemwide (irix,solaris,others?)
1402 	# python fallback
1403 	date=$("$(python_binary)" -c "import time;print time.strftime('${1:-$DEFAULTFORMAT}',time.localtime(${2}))" 2> /dev/null) && \
1404 		echo $date && return
1405 	# awk fallback
1406 	date=$(awk "BEGIN{print strftime(\"${1:-$DEFAULTFORMAT}\"${2:+,$2})}" 2> /dev/null) && \
1407 		echo $date && return
1408 	# perl fallback
1409 	date=$(perl  -e "use POSIX qw(strftime);\$date = strftime(\"${1:-$DEFAULTFORMAT}\",localtime(${2}));print \"\$date\n\";" 2> /dev/null) && \
1410 		echo $date && return
1411 	# error
1412 	echo "ERROR"
1413 	return 1
1414 }
1415 
1416 function nsecs {
1417 	local NSECS
1418 	# test if date supports nanosecond output
1419 	if ! var_isset NSECS_DISABLED; then
1420 		NSECS=$(date_fix %N 2> /dev/null | head -1 |grep -e "^[[:digit:]]\{9\}$")
1421 		[ -n "$NSECS" ] && NSECS_DISABLED=0 || NSECS_DISABLED=1
1422 	fi
1423 
1424 	# add 9 digits, not all date(s) deliver nsecs e.g. busybox date
1425 	if [ "$NSECS_DISABLED" == "1" ]; then
1426 		date_fix %s000000000
1427 	else
1428 		date_fix %s%N
1429 	fi
1430 }
1431 
1432 function nsecs_to_sec {
1433 	echo $(($1/1000000000)).$(printf "%03d" $(($1/1000000%1000)) )
1434 }
1435 
1436 function datefull_from_nsecs {
1437 	date_from_nsecs $1 '%F %T'
1438 }
1439 
1440 function date_from_nsecs {
1441 	local FORMAT=${2:-%T}
1442 	local TIME=$(nsecs_to_sec $1)
1443 	local SECS=${TIME%.*}
1444 	local DATE=$(date_fix "${FORMAT}" ${SECS:-0})
1445 	echo $DATE.${TIME#*.}
1446 }
1447 
1448 function var_isset {
1449   if [ -z "$1" ]; then
1450     echo "ERROR: function var_isset needs a string as parameter"
1451   elif eval "[ \"\${$1}\" == 'not_set' ]" || eval "[ \"\${$1-not_set}\" != 'not_set' ]"; then
1452     return 0
1453   fi
1454   return 1
1455 }
1456 
1457 function is_condition {
1458   local CMD=$(tolower "$1")
1459   [ "$CMD" == 'and' ] || [ "$CMD" == 'or' ]
1460 }
1461 
1462 function is_groupMarker {
1463   local CMD=$(tolower "$1")
1464   [ "$CMD" == 'groupin' ] || [ "$CMD" == 'groupout' ]
1465 }
1466 
1467 function is_command {
1468   local CMD=$(tolower "$1")
1469   ! is_condition "$CMD" && ! is_groupMarker "$CMD"
1470 }
1471 
1472 function url_encode {
1473   # utilize python, silently do nothing on error - because no python no duplicity
1474   OUT=$("$(python_binary)" -c "
1475 try: import urllib.request as urllib
1476 except ImportError: import urllib
1477 print(urllib.${2}quote('$1'));
1478 " 2>/dev/null ); ERR=$?
1479   [ "$ERR" -eq 0 ] && echo $OUT || echo $1
1480 }
1481 
1482 function url_decode {
1483   # reuse function above with a simple string param hack
1484   url_encode "$1" "un"
1485 }
1486 
1487 function toupper {
1488   echo "$@"|awk '$0=toupper($0)'
1489 }
1490 
1491 function tolower {
1492   echo "$@"|awk '$0=tolower($0)'
1493 }
1494 
1495 function isnumber {
1496   case "$*" in
1497     ''|*[!0-9]*) return 1;;
1498     *) return 0;;
1499   esac
1500 }
1501 
1502 #function tmp_space {
1503 #  
1504 #  if ! isnumber $VOLSIZE; then
1505 #    inform "failed to determine free space (please report, this is a bug)"
1506 #    return
1507 #  fi
1508 #  
1509 # get free temp space
1510 #  TEMP_FREE="$(df -P -k "$TEMP_DIR" 2>/dev/null | awk 'END{pos=(NF-2);if(pos>0) print $pos;}')"
1511 #  # check for free space or FAIL
1512 #  if [ $((${TEMP_FREE:-0}-${VOLSIZE:-0}*1024)) -lt 0-lt 0 ]; then
1513 #    error "Temporary file space '$TEMP_DIR' free space is smaller ($((TEMP_FREE/1024))MB)
1514 #than one duplicity volume (${VOLSIZE}MB).
1515 #    
1516 #  Hint: Free space or change TEMP_DIR setting."
1517 #fi
1518 #
1519 #}
1520 
1521 function gpg_disabled {
1522   echo "${GPG_KEY}" | grep -iq -e '^disabled$'
1523 }
1524 
1525 # usage: join SEPARATOR "entry1" "entry2"
1526 function join {
1527   local SEP="$1" ENTRY OUT; shift;
1528   for ENTRY in "$@"; do
1529     ENTRY=${ENTRY//$SEP/\\$SEP}
1530     [ -z "$OUT" ] && OUT=$ENTRY || OUT="$OUT$SEP$ENTRY"
1531   done
1532   echo $OUT
1533 }
1534 
1535 function gpg_testing {
1536   [ "$GPG_TEST" != "disabled" ]
1537 }
1538 
1539 function gpg_signing {
1540   echo ${GPG_KEY_SIGN} | grep -v -q -e '^disabled$'
1541 }
1542 
1543 function gpg_keytype {
1544   echo "$1" | awk '/^PUB$/{print "public"}/^SEC$/{print "secret"}'
1545 }
1546 
1547 # parameter key id, key_type
1548 function gpg_keyfile {
1549   local GPG_KEY=$(gpg_key_legalize $1) TYPE="$2"
1550   local KEYFILE="${KEYFILE//.asc/${GPG_KEY:+.$GPG_KEY}.asc}"
1551   echo "${KEYFILE//.asc/${TYPE:+.$(tolower $TYPE)}.asc}"
1552 }
1553 
1554 # parameter key id
1555 function gpg_import {
1556   local i FILE FOUND=0 KEY_ID="$1" KEY_TYPE="$2" KEY_FP="" ERR=0
1557   [ "$GPG_IMPORT" = "disabled" ] && {
1558     echo "Skipping import of needed $(gpg_keytype "$KEY_TYPE") key '$KEY_ID'. (GPG_IMPORT='disabled')"
1559     return
1560   }
1561 
1562   # create a list of legacy key file names and current naming scheme
1563   # we always import pub and sec if they are avail in conf folder
1564   local KEYFILES=( "$CONFDIR/gpgkey" $(gpg_keyfile "$KEY_ID") \
1565                    $(gpg_keyfile "$KEY_ID" "$KEY_TYPE") )
1566 
1567   # Try autoimport from existing old gpgkey files 
1568   # and new gpgkey.XXX.asc files (since v1.4.2)
1569   # and even newer gpgkey.XXX.[pub|sec].asc
1570   for (( i = 0 ; i < ${#KEYFILES[@]} ; i++ )); do
1571     FILE=${KEYFILES[$i]}
1572     if [ -f "$FILE" ]; then
1573       FOUND=1
1574       
1575       CMD_MSG="Import keyfile '$FILE' to keyring"
1576       run_cmd gpg $GPG_OPTS --batch --import $(qw "$FILE")
1577       if [ "$?" != "0" ]; then 
1578         warning "Import failed.${CMD_OUT:+\n$CMD_OUT}"
1579         ERR=1
1580         # continue with next
1581         continue
1582       fi
1583     fi
1584   done
1585 
1586   if [ "$FOUND" -eq 0 ]; then
1587     echo "Notice: No keyfile for '$KEY_ID' found in profile folder."
1588     return 1
1589   fi
1590 
1591   # try to set trust automagically
1592   CMD_MSG="Autoset trust of key '$KEY_ID' to ultimate"
1593   run_cmd echo $(gpg_fingerprint "$KEY_ID"):6: \| gpg $GPG_OPTS --import-ownertrust --batch --logger-fd 1
1594   if [ "$?" = "0" ] && [ -z "$PREVIEW" ]; then 
1595    # success on all levels, we're done
1596    return $ERR
1597   fi
1598 
1599   # failover: user has to set trust manually
1600   echo -e "For $ME_NAME to work you have to set the trust level 
1601 with the command \"trust\" to \"ultimate\" (5) now.
1602 Exit the edit mode of gpg with \"quit\"."
1603   CMD_MSG="Running gpg to manually edit key '$KEY_ID'"
1604   run_cmd sleep 5\; gpg $GPG_OPTS --edit-key $(qw "$KEY_ID")
1605 
1606   return $ERR
1607 }
1608 
1609 # see 'How to specify a user ID' on gpg manpage
1610 function gpg_fingerprint {
1611   gpg $GPG_OPTS --fingerprint "$1" 2>&1 | \
1612   awk 'NR==2{sub(/^.*=/,"");gsub(/[ \t]/,""); if ( $0 !~ /^[A-F0-9]+$/ || length($0) != 40 ) exit 1; print}'
1613 }
1614 
1615 function gpg_export_if_needed {
1616   [ "$GPG_EXPORT" = 'disabled' ] && { \
1617     echo "Skipping export of gpg keys. (GPG_EXPORT='disabled')"
1618     return
1619   }
1620 
1621   local SUCCESS FILE KEY_TYPE
1622   local TMPFILE="$TEMP_DIR/${ME_NAME}.$$.$(date_fix %s).gpgexp"
1623   for KEY_ID in "$@"; do
1624     # check if already exported, do it if not
1625     for KEY_TYPE in PUB SEC; do
1626       FILE="$(gpg_keyfile "$KEY_ID" $KEY_TYPE)"
1627       if [ ! -f "$FILE" ] && eval gpg_$(tolower $KEY_TYPE)_avail \"$KEY_ID\"; then
1628 
1629         # exporting
1630         CMD_MSG="Backup $(gpg_keytype "$KEY_TYPE") key '$KEY_ID' to profile."
1631         # gpg2.1 insists on passphrase here, gpg2.0- happily exports w/o it
1632         # we pipe an empty string when GPG_PW is not set to avoid gpg silently waiting for input
1633         run_cmd $(gpg_pass_pipein GPG_PW_SIGN GPG_PW) gpg $GPG_OPTS $GPG_USEAGENT $(gpg_param_passwd GPG_PW_SIGN GPG_PW) --armor --export"$(test "SEC" = "$KEY_TYPE" && echo -secret-keys)" $(qw "$KEY_ID") '>>' $(qw "$TMPFILE")
1634         CMD_ERR=$?
1635 
1636         if [ "$CMD_ERR" = "0" ]; then
1637           CMD_MSG="Write file '"$(basename "$FILE")"'"
1638           run_cmd mv $(qw "$TMPFILE") $(qw "$FILE")
1639         fi
1640 
1641         if [ "$CMD_ERR" != "0" ]; then
1642           warning "Backup failed.${CMD_OUT:+\n$CMD_OUT}"
1643         else
1644           SUCCESS=1
1645         fi
1646 
1647         # cleanup
1648         rm $(qw "$TMPFILE") 1>/dev/null 2>&1
1649       fi
1650     done
1651   done
1652   
1653   [ -n "$SUCCESS" ] && inform "$ME_NAME exported new keys to your profile.
1654 You should backup your changed profile folder now and store it in a safe place."
1655 }
1656 
1657 # replace all non-alnum chars with underscore (for file operations)
1658 function gpg_key_legalize {
1659   echo $* | awk '{gsub(/[^a-zA-Z0-9]/,"_",$0); print}'
1660 }
1661 
1662 function gpg_key_cache {
1663   local RES
1664   local MODE=$1
1665   shift
1666   local PREFIX="GPG_KEY"
1667   local SUFFIX=$(gpg_key_legalize "$@")
1668   local KEYID="$*"
1669   local CACHE="${PREFIX}_${MODE}_${SUFFIX}"
1670   if [ "$MODE" = "RESET" ]; then
1671     eval unset ${PREFIX}_PUB_$SUFFIX ${PREFIX}_SEC_$SUFFIX
1672     return 255
1673   elif ! var_isset "$CACHE"; then
1674     if [ "$MODE" = "PUB" ]; then
1675       RES=$(gpg $GPG_OPTS --list-key "$KEYID" > /dev/null 2>&1; echo -n $?)
1676     elif [ "$MODE" = "SEC" ]; then
1677       RES=$(gpg $GPG_OPTS --list-secret-key "$KEYID" > /dev/null 2>&1; echo -n $?)
1678     else
1679       return 255
1680     fi
1681     eval $CACHE=$RES
1682   fi
1683   eval return \$$CACHE
1684 }
1685 
1686 function gpg_pub_avail {
1687   gpg_key_cache PUB "$@"
1688 }
1689 
1690 function gpg_sec_avail {
1691   gpg_key_cache SEC "$@"
1692 }
1693 
1694 function gpg_key_format {
1695   echo $1 | grep -q '^[0-9a-fA-F]\{8\}$'
1696 }
1697 
1698 # splits a comma separated line into lines, respects escaped commas
1699 function gpg_split_keyset {
1700   local LIST
1701   LIST=$(echo "$@" | awk '{ gsub(/,/,"\n",$0); gsub(/\\\n/,",",$0); print $0 }')
1702   echo -e "$LIST"
1703 }
1704 
1705 function gpg_prefix_keyset {
1706   local PREFIX="$1" OUT=""
1707   shift
1708   for KEY_ID in "$@"; do
1709     OUT="${OUT} $PREFIX $(qw ${KEY_ID})"
1710   done
1711   echo $OUT
1712 }
1713 
1714 # grep a variable from conf text file (currently not used)
1715 function gpg_passwd {
1716   [ -r "$CONF" ] && \
1717   awk '/^[ \t]*GPG_PW[ \t=]/{\
1718         sub(/^[ \t]*GPG_PW[ \t]*=*/,"",$0);\
1719         gsub(/^[ \t]*[\047"]|[\047"][ \t]*$/,"",$0);\
1720         print $0; exit}' "$CONF"
1721 }
1722 
1723 # return success if at least one secret key is available
1724 function gpg_key_decryptable {
1725   # no keys, no problem
1726   gpg_symmetric && return 0
1727 
1728   local KEY_ID
1729   for KEY_ID in "${GPG_KEYS_ENC_ARRAY[@]}"; do
1730     gpg_sec_avail "$KEY_ID" && return 0
1731   done
1732   return 1
1733 }
1734 
1735 function gpg_symmetric {
1736   [ -z "${GPG_KEY}${GPG_KEYS_ENC_ARRAY}" ]
1737 }
1738 
1739 # checks for max two params if they are set, typically GPG_PW & GPG_PW_SIGN
1740 function gpg_param_passwd {
1741   var_isset GPG_USEAGENT && exit 1
1742   
1743   if ( [ -n "$1" ] && var_isset "$1" ) || ( [ -n "$2" ] && var_isset "$2" ); then
1744     echo "--passphrase-fd 0 --batch"
1745   fi
1746 }
1747 
1748 # select the earliest defined and create an "echo <value> |" string
1749 function gpg_pass_pipein {
1750   var_isset GPG_USEAGENT && exit 1
1751   
1752   for var in "$@"
1753   do
1754     if var_isset "$var"; then
1755       echo "echo $(qw $(eval echo \$$var)) |"
1756       return 0
1757     fi
1758   done
1759   
1760   return 1
1761 }
1762 
1763 # checks if gpg-agent is available, returns error code
1764 # 0 on success
1765 # 1 if GPG_AGENT_INFO is not set (unused, should probably be merged w/ 3)
1766 # 2 if GPG_AGENT_INFO is stale
1767 # 3 cannot connect to gpg-agent
1768 function gpg_agent_avail {
1769   # GPG_AGENT_INFO is deprecated in gpg2.1, 
1770   # first try to connect to a possibly running agent here
1771   local ERR=3
1772   gpg-agent > /dev/null 2>&1 && return 0
1773 
1774   # detect stale pid in legacy GPG_AGENT_INFO env var
1775   if var_isset GPG_AGENT_INFO; then
1776     # check if a pid matching process is running at all
1777     local GPG_AGENT_PID=$(echo $GPG_AGENT_INFO|awk -F: '{print $2}')
1778     if isnumber "$GPG_AGENT_PID"; then
1779       ps -p "$GPG_AGENT_PID" > /dev/null 2>&1 || ERR=2
1780     fi
1781   fi
1782 
1783   return $ERR
1784 }
1785 
1786 function gpg_custom_binary {
1787   var_isset GPG && [ "$GPG" != "$DEFAULT_GPG" ] &&\
1788     echo "--gpg-binary $(qw "$GPG")"
1789 }
1790 
1791 function gpg_binary {
1792   local BIN
1793   var_isset GPG && BIN="$GPG" || BIN="$DEFAULT_GPG"
1794   echo "$BIN"
1795 }
1796 
1797 function gpg_avail {
1798   lookup "$(gpg_binary)"
1799 }
1800 
1801 # enforce the use of our selected gpg binary
1802 function gpg {
1803   command "$(gpg_binary)" "$@"
1804 }
1805 export -f gpg
1806 
1807 # start of script #######################################################################
1808 
1809 # confidentiality first, all we create is only readable by us
1810 umask 077
1811 
1812 # check if ftplicity is there & executable
1813 [ -n "$ME_LONG" ] && [ -x "$ME_LONG" ] || error "$ME missing. Executable & available in path? ($ME_LONG)"
1814 
1815 if [ ${#@} -eq 1 ]; then
1816   cmd="${1}"
1817 else
1818   FTPLCFG="${1}" ; cmd="${2}"
1819 fi
1820 
1821 # deal with command before profile validation calls
1822 # show requested version
1823 # OR requested usage info
1824 # OR create a profile
1825 # OR fall through
1826 ##if [ ${#@} -le 2 ]; then
1827 case "$cmd" in
1828   changelog)
1829     changelog
1830     exit 0
1831     ;;
1832   create)
1833     set_config
1834     if [ -d "$CONFDIR" ]; then
1835       error "The profile '$FTPLCFG' already exists in
1836 '$CONFDIR'.
1837 
1838 Hint:
1839  If you _really_ want to create a new profile by this name you will 
1840  have to manually delete the existing profile folder first."
1841       exit 1
1842     else
1843       create_config
1844       exit 0
1845     fi
1846     ;;
1847   txt2man)
1848     set_config
1849     usage_txt2man
1850     exit 0
1851     ;;
1852   usage|-help|--help|-h|-H)
1853     set_config
1854     usage_info
1855     exit 0
1856     ;;
1857   version|-version|--version|-v|-V)
1858     # profile can override GPG/PYTHON, so import it if it was given
1859     var_isset FTPLCFG && {
1860       set_config
1861       [ -r "$CONF" ] && . "$CONF" || warning "Cannot import config '$CONF'."
1862     }
1863     version_info_using
1864     exit 0
1865     ;;
1866   # fallthrough.. we got a command that needs an existing profile
1867   *)
1868     # if we reach here, user either forgot profile or chose wrong profileless command
1869     if [ ${#@} -le 1 ]; then
1870       error "\
1871  Missing or wrong parameters. 
1872  Only the commands 
1873    changelog, create, usage, txt2man, version
1874  can be called without selecting an existing profile first.
1875  Your command was '$cmd'.
1876 
1877  Hint: Run '$ME usage' to get help."
1878     fi
1879 esac
1880 
1881 
1882 # Hello world
1883 echo "Start $ME v$ME_VERSION, time is $(date_fix '%F %T')."
1884 
1885 # check system environment
1886 
1887 # is duplicity avail
1888 lookup duplicity || error_path "duplicity missing. installed und available in path?"
1889 # init, exec duplicity version check info
1890 duplicity_python_binary_parse
1891 duplicity_version_get
1892 
1893 # check for certain important helper programs
1894 for f in awk grep "$(python_binary)"; do
1895   lookup "$f" || \
1896     error_path "$f missing. installed und available in path?"
1897 done
1898 
1899 ### read configuration
1900 set_config
1901 # check validity
1902 if [ ! -d "$CONFDIR" ]; then 
1903     error "Selected profile '$FTPLCFG' does not resolve to a profile folder in
1904 '$CONFDIR'.
1905 
1906 Hints:
1907 Select one of the available profiles: $(for d in "$(dirname "$CONFDIR")"/*/; do [ -e "$d" ] || [ -L "$d" ] || continue; printf "$sep'$(basename "$d")'"; sep=",";done)
1908  Use '$ME <name> create' to create a new profile.
1909  Use '$ME usage' to get usage help."
1910 elif [ ! -x "$CONFDIR" ]; then
1911     error "\
1912 Profile folder in '$CONFDIR' cannot be accessed.
1913 
1914 Hint: 
1915  Check the filesystem permissions and set directory accessible e.g. 'chmod 700'."
1916 elif [ ! -f "$CONF" ] ; then
1917   error "'$CONF' not found."
1918 elif [ ! -r "$CONF" ] ; then
1919   error "'$CONF' not readable."
1920 else
1921   . "$CONF"
1922   #KEYFILE="${KEYFILE//.asc/${GPG_KEY:+.$GPG_KEY}.asc}"
1923   TEMP_DIR=${TEMP_DIR:-'/tmp'}
1924   # backward compatibility: old TARGET_PW overrides silently new TARGET_PASS if set
1925   if var_isset 'TARGET_PW'; then
1926     TARGET_PASS="${TARGET_PW}"
1927   fi
1928 fi
1929 echo "Using profile '$CONFDIR'."
1930 
1931 # secure config dir, if needed w/ warning
1932 secureconf
1933 
1934 # split TARGET in handy variables
1935 TARGET_SPLIT_URL=$(echo "$TARGET" | awk '{ \
1936   target=$0; match(target,/^([^\/:]+):\/\//); \
1937   prot=substr(target,RSTART,RLENGTH);\
1938   rest=substr(target,RSTART+RLENGTH); \
1939   if (credsavail=match(rest,/^[^@]*@/)){\
1940     creds=substr(rest,RSTART,RLENGTH-1);\
1941     credcount=split(creds,cred,":");\
1942     rest=substr(rest,RLENGTH+1);\
1943     # split creds with regexp\
1944     match(creds,/^([^:]+)/);\
1945     user=substr(creds,RSTART,RLENGTH);\
1946     pass=substr(creds,RSTART+1+RLENGTH);\
1947   };\
1948   # filter quotes or escape them\
1949   gsub(/[\047\042]/,"",prot);\
1950   gsub(/[\047\042]/,"",rest);\
1951   gsub(/[\047]/,"\047\\\047\047",creds);\
1952   print "TARGET_URL_PROT=\047"prot"\047\n"\
1953          "TARGET_URL_HOSTPATH=\047"rest"\047\n"\
1954          "TARGET_URL_CREDS=\047"creds"\047\n";\
1955    if(user){\
1956      gsub(/[\047]/,"\047\\\047\047",user);\
1957      print "TARGET_URL_USER=\047"user"\047\n"}\
1958    if(pass){\
1959      gsub(/[\047]/,"\047\\\047\047",pass);\
1960      print "TARGET_URL_PASS=$(url_decode \047"pass"\047)\n"}\
1961   }')
1962 eval "${TARGET_SPLIT_URL}"
1963 
1964 # fetch commmand from parameters ########################################################
1965 # Hint: cmds is also used to check if authentification info sufficient in the next step 
1966 cmds="$2"; shift 2
1967 
1968 # complain if command(s) missing
1969 [ -z $cmds ] && error "  No command given.
1970 
1971   Hint: 
1972     Use '$ME usage' to get usage help."
1973 
1974 # process params
1975 for param in "$@"; do
1976   #echo !$param!
1977   case "$param" in
1978     # enable ftplicity preview mode
1979     '--preview')
1980       PREVIEW=1
1981       ;;
1982     # interpret duplicity disable encr switch
1983     '--disable-encryption')
1984       GPG_KEY='disabled'
1985       ;;
1986     *)
1987       if [ `echo "$param" | grep -e "^-"` ] || \
1988          [ `echo "$last_param" | grep -e "^-"` ] ; then
1989         # forward parameter[/option pairs] to duplicity
1990         dupl_opts["${#dupl_opts[@]}"]=${param}
1991       else
1992         # anything else must be a parameter (e.g. for fetch, ...)
1993         ftpl_pars["${#ftpl_pars[@]}"]=${param}
1994       fi
1995       last_param=${param}
1996       ;;
1997   esac
1998 done
1999 
2000 # plausibility check config - VARS & KEY ################################################
2001 # check if src, trg, trg pw
2002 # auth info sufficient 
2003 # gpg key, gpg pwd (might be empty) set in config
2004 # OR key in local gpg db
2005 # OR key can be imported from keyfile 
2006 # OR fail
2007 if [ -z "$SOURCE" ] || [ "$SOURCE" == "${DEFAULT_SOURCE}" ]; then
2008  error " Source Path (setting SOURCE) not set or still default value in conf file 
2009  '$CONF'."
2010 
2011 elif [ -z "$TARGET" ] || [ "$TARGET" == "${DEFAULT_TARGET}" ]; then
2012  error " Backup Target (setting TARGET) not set or still default value in conf file 
2013  '$CONF'."
2014 
2015 elif var_isset 'TARGET_USER' && var_isset 'TARGET_URL_USER' && \
2016      [ "${TARGET_USER}" != "${TARGET_URL_USER}" ]; then
2017  error " TARGET_USER ('${TARGET_USER}') _and_ user in TARGET url ('${TARGET_URL_USER}') 
2018  are configured with different values. There can be only one.
2019  
2020  Hint: Remove conflicting setting."
2021 
2022 elif var_isset 'TARGET_PASS' && var_isset 'TARGET_URL_PASS' && \
2023      [ "${TARGET_PASS}" != "${TARGET_URL_PASS}" ]; then
2024  error " TARGET_PASS ('${TARGET_PASS}') _and_ password in TARGET url ('${TARGET_URL_PASS}') 
2025  are configured with different values. There can be only one.
2026  
2027  Hint: Remove conflicting setting."
2028 fi
2029 
2030 # GPG config plausibility check1 (disabled check) #############################
2031 if gpg_disabled; then
2032   : # encryption disabled, all is well
2033 elif [ -z "${GPG_KEY}${GPG_KEYS_ENC}${GPG_KEY_SIGN}" ] && ! var_isset 'GPG_PW'; then
2034   warning "GPG_KEY, GPG_KEYS_ENC, GPG_KEY_SIGN and GPG_PW are empty/not set in conf file 
2035 '$CONF'.
2036 Will disable encryption for duplicity now.
2037 
2038 Hint: 
2039  If you really want to use _no_ encryption you can disable this warning by 
2040  setting GPG_KEY='disabled' in conf file."
2041   GPG_KEY='disabled'
2042 fi
2043 
2044 # GPG availability check (now we know if gpg is really needed)#################
2045 if ! gpg_disabled; then 
2046   gpg_avail || error_path "gpg '$(gpg_binary)' missing. installed und available in path?"
2047 fi
2048 
2049 # Output versions info ########################################################
2050 using_info
2051 
2052 # GPG create key settings, config check2 (needs gpg) ##########################
2053 if gpg_disabled; then
2054   : # the following tests are not necessary
2055 else
2056 
2057 # we test this early as any invocation gpg2.1+ starts gpg-agent automatically
2058 GPG_AGENT_ERR=$(gpg_agent_avail ; echo $?)
2059 
2060 # enc key still default?
2061 if [ "$GPG_KEY" == "${DEFAULT_GPG_KEY}" ]; then 
2062   error_gpg "Encryption Key GPG_KEY still default in conf file 
2063 '$CONF'."
2064 fi
2065 
2066 # create array of gpg encr keys, for further processing
2067 OIFS="$IFS" IFS=$'\n'
2068 GPG_KEYS_ENC_ARRAY=( $( gpg_split_keyset ${GPG_KEY},${GPG_KEYS_ENC} ) )
2069 IFS="$OIFS"
2070 
2071 # pw set? 
2072 # symmetric needs one, always
2073 if gpg_symmetric && ( [ -z "$GPG_PW" ] || [ "$GPG_PW" == "${DEFAULT_GPG_PW}" ] ) \
2074   ; then
2075   error_gpg "Encryption passphrase GPG_PW (needed for symmetric encryption) 
2076 is empty/not set or still default value in conf file 
2077 '$CONF'."
2078 fi
2079 # this is a technicality, we can only pump one pass via pipe into gpg
2080 # but symmetric already always needs one for encryption
2081 if gpg_symmetric && var_isset GPG_PW && var_isset GPG_PW_SIGN &&\
2082   [ -n "$GPG_PW_SIGN" ] && [ "$GPG_PW" != "$GPG_PW_SIGN" ]; then
2083   error_gpg "GPG_PW _and_ GPG_PW_SIGN are defined but not identical in config
2084 '$CONF'.
2085 This is unfortunately impossible. For details see duplicity manpage, 
2086 section 'A Note On Symmetric Encryption And Signing'.
2087 
2088 Tip: Separate signing keys may have empty passwords e.g. GPG_PW_SIGN=''.
2089 Tip2: Use gpg-agent."
2090 fi
2091 
2092 # test - GPG KEY AVAILABILITY ##################################################
2093 
2094 # check gpg public keys availability, try import if needed
2095 for (( i = 0 ; i < ${#GPG_KEYS_ENC_ARRAY[@]} ; i++ )); do
2096   KEY_ID="${GPG_KEYS_ENC_ARRAY[$i]}"
2097   # test availability, try to import, retest
2098   if ! gpg_pub_avail "${KEY_ID}"; then
2099     echo "Encryption public key '${KEY_ID}' not in keychain. Try to import from profile."
2100     gpg_import "${KEY_ID}" PUB
2101     gpg_key_cache RESET "${KEY_ID}"
2102     gpg_pub_avail "${KEY_ID}" || { \
2103       gpg_testing && error_gpg \
2104       "Needed public gpg key '${KEY_ID}' is not available in keychain." \
2105       "Doublecheck if the above key is listed by 'gpg --list-keys' or available
2106   as gpg key file '$(basename "$(gpg_keyfile "${KEY_ID}")")' in the profile folder.
2107   If not you can put it there and $ME_NAME will autoimport it on the next run.
2108   Alternatively import it manually as the user you plan to run $ME_NAME with."
2109     }
2110   else
2111     echo "Public key '${KEY_ID}' found in keychain."
2112   fi
2113 done
2114 
2115 # check gpg encr secret encryption keys availability and fail
2116 # if none is available after a round of importing trials
2117 gpg_key_decryptable || \
2118 {
2119   echo "Missing secret keys for decryption in keychain."
2120   for (( i = 0 ; i < ${#GPG_KEYS_ENC_ARRAY[@]} ; i++ )); do
2121     KEY_ID="${GPG_KEYS_ENC_ARRAY[$i]}"
2122     # test availability, try to import, retest
2123     if ! gpg_sec_avail "${KEY_ID}"; then
2124     echo "Try to import secret key '${KEY_ID}' from profile."
2125       gpg_import "${KEY_ID}" SEC
2126       gpg_key_cache RESET "${KEY_ID}"
2127     fi
2128   done
2129   gpg_key_decryptable || \
2130   {
2131     gpg_testing && error_gpg_test "None of the configured keys '$(join "','" "${GPG_KEYS_ENC_ARRAY[@]}")' \
2132 has a secret key in the keychain. Decryption will be impossible!"
2133   }
2134 }
2135 
2136 # gpg secret sign key availability
2137 # if none set, autoset first encryption key as sign key
2138 if ! gpg_signing; then
2139   echo "Signing disabled per configuration."
2140 # try first key, if one set
2141 elif ! var_isset 'GPG_KEY_SIGN'; then
2142   KEY_ID="${GPG_KEYS_ENC_ARRAY[0]}"
2143   if [ -z "${KEY_ID}" ]; then
2144     echo "Signing disabled. No GPG_KEY entries in config."
2145     GPG_KEY_SIGN='disabled'
2146   else  
2147     # use avail OR try import OR fail
2148     if gpg_sec_avail "${KEY_ID}"; then
2149       GPG_KEY_SIGN="${KEY_ID}"
2150     else
2151       echo "Signing secret key '${KEY_ID}' not found."
2152       gpg_import "${KEY_ID}" SEC
2153       gpg_key_cache RESET "${KEY_ID}"
2154       if gpg_sec_avail "${KEY_ID}"; then
2155         GPG_KEY_SIGN="${KEY_ID}"
2156       fi
2157     fi
2158 
2159     # interpret sign key setting
2160     if var_isset 'GPG_KEY_SIGN'; then
2161       echo "Autoset found secret key of first GPG_KEY entry '${KEY_ID}' for signing."
2162     else
2163       echo "Signing disabled. First GPG_KEY entry's '${KEY_ID}' private key is missing."
2164       GPG_KEY_SIGN='disabled'
2165     fi
2166   fi
2167 else
2168   KEY_ID="${GPG_KEY_SIGN}"
2169   if ! gpg_sec_avail "${KEY_ID}"; then
2170     inform "Secret signing key defined in setting GPG_KEY_SIGN='${KEY_ID}' not found.\nTry to import."
2171     gpg_import "${KEY_ID}" SEC
2172     gpg_key_cache RESET "${KEY_ID}"
2173     gpg_sec_avail "${KEY_ID}" || error_gpg_key "${KEY_ID}" "Private"
2174   fi
2175 fi
2176 
2177 # using GPG_AGENT_ERR set early above, try to autoenable gpg-agent or issue some warnings
2178 # key enc can deal without, but might profit from gpg-agent
2179 # if GPG_PW is not set alltogether
2180 # if signing key is different from first (main) enc key (we can only pipe one pass into gpg)
2181 if ! gpg_symmetric && \
2182    ( ! var_isset GPG_PW || \
2183      ( gpg_signing && ! var_isset GPG_PW_SIGN && [ "$GPG_KEY_SIGN" != "${GPG_KEYS_ENC_ARRAY[0]}" ] ) ); then
2184 
2185   if [ "$GPG_AGENT_ERR" -eq 1 ]; then
2186     warning "Cannot use gpg-agent. GPG_AGENT_INFO not set."
2187   elif [ "$GPG_AGENT_ERR" -eq 2 ]; then
2188     warning "Cannot use gpg-agent! GPG_AGENT_INFO contains stale pid."
2189   elif [ "$GPG_AGENT_ERR" -eq 3 ]; then
2190     warning "No running gpg-agent found although GPG_PW or GPG_PW_SIGN (enc != sign key) not set."
2191   else
2192     echo "Enable gpg-agent usage. Running gpg-agent instance found and GPG_PW or GPG_PW_SIGN (enc != sign key) not set."
2193     GPG_USEAGENT="--use-agent"
2194   fi
2195 fi
2196 
2197 # end GPG config plausibility check2 
2198 fi
2199 
2200 # config plausibility check - SPACE ###########################################
2201 
2202 # is tmp is a folder and writable
2203 CMD_MSG="Checking TEMP_DIR '${TEMP_DIR}' is a folder and writable"
2204 run_cmd test -d $(qw "$TEMP_DIR") '&&' test -w $(qw "$TEMP_DIR")
2205 if [ "$?" != "0" ]; then
2206     error "Temporary file space '$TEMP_DIR' is not a directory or writable."
2207 fi
2208 
2209 
2210 # get volsize, default duplicity volume size is 25MB since v0.5.07
2211 VOLSIZE=${VOLSIZE:-25}
2212 # double if asynch is on
2213 echo $@ $DUPL_PARAMS | grep -q -e '--asynchronous-upload' && FACTOR=2 || FACTOR=1
2214 
2215 # TODO: check for enough (async= upload space and WARN only
2216 #       use function tmp_space 
2217 #echo TODO: reimplent tmp space check
2218 
2219 
2220 # test - GPG SANITY #####################################################################
2221 # if encryption is disabled, skip this whole section
2222 if gpg_disabled; then
2223   echo -e "Test - En/Decryption skipped. (GPG='disabled')"
2224 elif ! gpg_testing; then 
2225   echo -e "Test - En/Decryption skipped. (GPG_TEST='disabled')"
2226 else
2227 
2228 GPG_TEST_PREFIX="$TEMP_DIR/${ME_NAME}.$$.$(date_fix %s)"
2229 function cleanup_gpgtest { 
2230   echo -en "Cleanup - Delete '${GPG_TEST_PREFIX}_*'"
2231   rm "${GPG_TEST_PREFIX}"_* 2>/dev/null && echo "(OK)" || echo "(FAILED)"
2232 }
2233 
2234 # signing enabled?
2235 if gpg_signing; then
2236   CMD_PARAM_SIGN="--sign --default-key $(qw ${GPG_KEY_SIGN})"
2237   CMD_MSG_SIGN="Sign with '${GPG_KEY_SIGN}'"
2238 fi
2239 
2240 # using keys
2241 if [ ${#GPG_KEYS_ENC_ARRAY[@]} -gt 0 ]; then
2242 
2243   for KEY_ID in "${GPG_KEYS_ENC_ARRAY[@]}"; do
2244     CMD_PARAMS="$CMD_PARAMS -r $(qw ${KEY_ID})"
2245   done
2246   # check encrypting
2247   CMD_MSG="Test - Encrypt to '$(join "','" "${GPG_KEYS_ENC_ARRAY[@]}")'${CMD_MSG_SIGN:+ & $CMD_MSG_SIGN}"
2248   run_cmd $(gpg_pass_pipein GPG_PW_SIGN GPG_PW) gpg $CMD_PARAM_SIGN $(gpg_param_passwd GPG_PW_SIGN GPG_PW) $CMD_PARAMS $GPG_USEAGENT --status-fd 1 $GPG_OPTS -o $(qw "${GPG_TEST_PREFIX}_ENC") -e $(qw "$ME_LONG")
2249   CMD_ERR=$?
2250 
2251   if [ "$CMD_ERR" != "0" ]; then 
2252     KEY_NOTRUST=$(echo "$CMD_OUT"|awk '/^\[GNUPG:\] INV_RECP 10/ { print $4 }')
2253     [ -n "$KEY_NOTRUST" ] && HINT="Key '${KEY_NOTRUST}' seems to be untrusted. If you really trust this key try to
2254   'gpg --edit-key "$KEY_NOTRUST"' and raise the trust level to ultimate. If you
2255   can trust all of your keys set GPG_OPTS='--trust-model always' in conf file."
2256     error_gpg_test "Encryption failed (Code $CMD_ERR).${CMD_OUT:+\n$CMD_OUT}" "$HINT"
2257   fi
2258 
2259   # check decrypting
2260   CMD_MSG="Test - Decrypt"
2261   gpg_key_decryptable || CMD_DISABLED="No matching secret key available."
2262   run_cmd $(gpg_pass_pipein GPG_PW) gpg $(gpg_param_passwd GPG_PW) $GPG_OPTS -o $(qw "${GPG_TEST_PREFIX}_DEC") $GPG_USEAGENT -d $(qw "${GPG_TEST_PREFIX}_ENC")
2263   CMD_ERR=$?
2264 
2265   if [ "$CMD_ERR" != "0" ]; then 
2266     error_gpg_test "Decryption failed.${CMD_OUT:+\n$CMD_OUT}"
2267   fi
2268 
2269 # symmetric only
2270 else
2271   # check encrypting
2272   CMD_MSG="Test - Encryption with passphrase${CMD_MSG_SIGN:+ & $CMD_MSG_SIGN}"
2273   run_cmd $(gpg_pass_pipein GPG_PW) gpg $GPG_OPTS $CMD_PARAM_SIGN --passphrase-fd 0 -o $(qw "${GPG_TEST_PREFIX}_ENC") --batch -c $(qw "$ME_LONG")
2274   CMD_ERR=$?
2275   if [ "$CMD_ERR" != "0" ]; then 
2276     error_gpg_test "Encryption failed.${CMD_OUT:+\n$CMD_OUT}"
2277   fi
2278 
2279   # check decrypting
2280   CMD_MSG="Test - Decryption with passphrase"
2281   run_cmd $(gpg_pass_pipein GPG_PW) gpg $GPG_OPTS --passphrase-fd 0 -o $(qw "${GPG_TEST_PREFIX}_DEC") --batch -d $(qw "${GPG_TEST_PREFIX}_ENC")
2282   CMD_ERR=$?
2283   if [ "$CMD_ERR" != "0" ]; then 
2284     error_gpg_test "Decryption failed.${CMD_OUT:+\n$CMD_OUT}"
2285   fi
2286 fi
2287 
2288 # compare original w/ decryptginal
2289 CMD_MSG="Test - Compare"
2290 [ -r "${GPG_TEST_PREFIX}_DEC" ] || CMD_DISABLED="File not found. Nothing to compare."
2291 run_cmd "test \"\$(cat '$ME_LONG')\" = \"\$(cat '${GPG_TEST_PREFIX}_DEC')\""
2292 CMD_ERR=$?
2293 if [ "$CMD_ERR" = "0" ]; then 
2294   cleanup_gpgtest
2295 else
2296   error_gpg_test "Comparision failed.${CMD_OUT:+\n$CMD_OUT}"
2297 fi
2298 
2299 fi # end disabled
2300 
2301 ## an empty line
2302 #echo
2303 
2304 # Exclude file is needed, create it if necessary
2305 [ -f "$EXCLUDE" ] || touch "$EXCLUDE"
2306 
2307 # export only used keys, if bkp not already exists ######################################
2308 gpg_export_if_needed "${GPG_KEYS_ENC_ARRAY[@]}" "$(gpg_signing && echo $GPG_KEY_SIGN)"
2309 
2310 
2311 # command execution #####################################################################
2312 
2313 # urldecode url vars into plain text
2314 var_isset 'TARGET_URL_USER' && TARGET_URL_USER="$(url_decode "$TARGET_URL_USER")"
2315 var_isset 'TARGET_URL_PASS' && TARGET_URL_PASS="$(url_decode "$TARGET_URL_PASS")"
2316 
2317 # defined TARGET_USER&PASS vars replace their URL pendants 
2318 # (double defs already dealt with)
2319 var_isset 'TARGET_USER' && TARGET_URL_USER="$TARGET_USER"
2320 var_isset 'TARGET_PASS' && TARGET_URL_PASS="$TARGET_PASS"
2321 
2322 TARGET_URL_PROT_lowercase="$(tolower "${TARGET_URL_PROT%%:*}")"
2323 
2324 # issue some warnings
2325 case "$TARGET_URL_PROT_lowercase" in
2326   'cf+http')
2327     # info on missing AUTH_URL
2328     if ! var_isset 'CLOUDFILES_AUTHURL'; then
2329       inform "No CLOUDFILES_AUTHURL exported (in conf).
2330 Will use default which is probably rackspace."
2331     fi
2332     ;;
2333    'swift')
2334     # info on possibly missing AUTH_URL
2335        var_isset 'SWIFT_AUTHURL' ||\
2336        warning "\
2337 Swift will probably fail because the conf var SWIFT_AUTHURL was not exported!"
2338     ;;
2339   'rsync')
2340     # everything in url (this backend does not support pass in env var)
2341     # this is obsolete from version 0.6.10 (buggy), hopefully fixed in 0.6.11
2342     # print warning older version is detected
2343     duplicity_version_lt 610 &&
2344       warning "\
2345 Duplicity version '$DUPL_VERSION' does not support providing the password as 
2346 env var for rsync backend. For security reasons you should consider to 
2347 update to a version greater than '0.6.10' of duplicity."
2348     ;;
2349 esac
2350 
2351 
2352 # for all protocols we put username in url and pass into env var 
2353 # for sec�rity reasons, we url_encode username to protect special chars
2354 # first sortout backends with special ways to handle password
2355 case "$TARGET_URL_PROT_lowercase" in
2356   'imap'|'imaps')
2357     var_isset 'TARGET_URL_PASS' && BACKEND_PARAMS="IMAP_PASSWORD=$(qw "${TARGET_URL_PASS}")"
2358     ;;
2359   *)
2360     # rest uses FTP_PASS var
2361     var_isset 'TARGET_URL_PASS' && \
2362       BACKEND_PARAMS="FTP_PASSWORD=$(qw "${TARGET_URL_PASS}")"
2363     ;;
2364 esac
2365 # insert url encoded username into target url if needed
2366 if var_isset 'TARGET_URL_USER' && [ "$TARGET_URL_PROT_lowercase" != "file" ]; then
2367   BACKEND_URL="${TARGET_URL_PROT}$(url_encode "${TARGET_URL_USER}")@${TARGET_URL_HOSTPATH}"
2368 else
2369   BACKEND_URL="$TARGET"
2370 fi
2371 
2372 
2373 # protect eval from special chars in url (e.g. open ')' in password, 
2374 # spaces in path, quotes) happens above in duplify() via quotewrap()
2375 SOURCE="$SOURCE"
2376 BACKEND_URL="$BACKEND_URL"
2377 EXCLUDE="$EXCLUDE"
2378 # since 0.7.03 --exclude-globbing-filelist is deprecated
2379 EXCLUDE_PARAM="--exclude$(duplicity_version_lt 703 && echo -globbing)-filelist" 
2380 
2381 # translate backup to batch command 
2382 cmds=${cmds//backup/groupIn_pre_bkp_post_groupOut}
2383 
2384 # replace magic separators to command equivalents (+=and,-=or,[=groupIn,]=groupOut)
2385 cmds=$(awk -v cmds="$cmds" "BEGIN{ \
2386   gsub(/\+/,\"_and_\",cmds);\
2387   gsub(/\-/,\"_or_\",cmds);\
2388   gsub(/\[/,\"_groupIn_\",cmds);\
2389   gsub(/\]/,\"_groupOut_\",cmds);\
2390   print cmds}")
2391 # convert cmds to array, lowercase for safety
2392 declare -a CMDS
2393 CMDS=( $(awk "BEGIN{ cmds=tolower(\"$cmds\"); gsub(/_/,\" \",cmds); print cmds }") )
2394 
2395 unset FTPL_ERR
2396 
2397 # run cmds
2398 for cmd in ${CMDS[*]};
2399 do
2400 
2401 ## init
2402 # raise index in cmd array for pre/post param
2403 var_isset 'CMD_NO' && CMD_NO=$((++CMD_NO)) || CMD_NO=0
2404 
2405 unset CMD_VALUE CMD_NEXT CMD_PREV CND_NEXT CND_PREV
2406 
2407 # get next cmd,cnd vars
2408 nextno=$(( $CMD_NO ))
2409 while ! var_isset 'CMD_NEXT'
2410 do
2411   nextno=$(($nextno+1))
2412   if [ "$nextno" -lt "${#CMDS[@]}" ]; then
2413     CMD_VALUE=${CMDS[$nextno]}
2414     is_condition "$CMD_VALUE" && CND_NEXT="$CMD_VALUE" && continue
2415     is_groupMarker "$CMD_VALUE" && continue
2416     CMD_NEXT="$CMD_VALUE"
2417   else
2418     CMD_NEXT='END'
2419   fi
2420 done
2421 
2422 # get prev cnd, cnds are skipped pseudocmds
2423 prevno=$(( $CMD_NO ));
2424 while ! var_isset 'CND_PREV'
2425 do
2426   prevno=$(($prevno-1))
2427   if [ "$prevno" -ge 0 ]; then
2428     CMD_VALUE=${CMDS[$prevno]}
2429     is_condition "$CMD_VALUE" && CND_PREV="$CMD_VALUE" && break
2430     is_command "$CMD_VALUE" && break
2431   else
2432     break
2433   fi
2434 done
2435 
2436 # get prev cmd command minus skipped commands, only executed
2437 prevno=$(( $CMD_NO - ${CMD_SKIPPED-0} ));
2438 while ! var_isset 'CMD_PREV'
2439 do
2440   prevno=$(($prevno-1))
2441   if [ "$prevno" -ge 0 ]; then
2442     CMD_VALUE=${CMDS[$prevno]}
2443     is_condition "$CMD_VALUE" && CND_PREV="$CMD_VALUE" && continue
2444     is_groupMarker "$CMD_VALUE" && continue
2445     CMD_PREV="$CMD_VALUE"
2446   else
2447     CMD_PREV='START'
2448   fi
2449 done
2450 
2451 function get_cmd_skip_count {
2452   # find closing bracket, get group skip count
2453   local nextno=$CMD_NO
2454   local GRP_OPEN=0
2455   local GRP_SKIP=0
2456   local CMD_VALUE
2457   while [ "$nextno" -lt "${#CMDS[@]}" ]
2458   do
2459     nextno=$(($nextno+1))
2460     CMD_VALUE=${CMDS[$nextno]}
2461     GRP_SKIP=$(( ${GRP_SKIP} + 1 ));
2462     if is_command "$CMD_VALUE" && [ "$GRP_OPEN" -lt 1 ]; then
2463         break;
2464     elif [ "$CMD_VALUE" == 'groupin' ]; then
2465       GRP_OPEN=$(( ${GRP_OPEN} + 1 ))
2466     elif [ "$CMD_VALUE" == 'groupout' ]; then
2467       GRP_OPEN=$(( ${GRP_OPEN} - 1 ))
2468       if [ "$GRP_OPEN" -lt 1 ]; then
2469         break;
2470       fi
2471     fi
2472   done
2473 
2474   echo $GRP_SKIP;
2475 }
2476 
2477 # decision time: are we skipping already or dealing with condition "commands" or other non-cmds?
2478 unset SKIP_NOW
2479 if var_isset 'CMD_SKIP' && [ $CMD_SKIP -gt 0 ]; then
2480   # skip cnd/grp cmds silently
2481   is_command "$cmd" && echo -e "\n--- Skipping command $(toupper $cmd) ! ---"
2482   CMD_SKIP=$(($CMD_SKIP - 1))
2483   SKIP_NOW="yes"
2484 elif ! var_isset 'PREVIEW' && [ "$cmd" == 'and' ] && [ "$CMD_ERR" -ne "0" ]; then
2485   CMD_SKIP=$(get_cmd_skip_count)
2486   # incl. this "cmd"
2487   CMD_SKIP=$(( $CMD_SKIP + 1 ))
2488   unset CMD_SKIPPED
2489   SKIP_NOW="yes"
2490 elif ! var_isset 'PREVIEW' && [ "$cmd" == 'or' ] && [ "$CMD_ERR" -eq "0" ]; then
2491   CMD_SKIP=$(get_cmd_skip_count)
2492   # incl. this "cmd"
2493   CMD_SKIP=$(( $CMD_SKIP + 1 ))
2494   unset CMD_SKIPPED
2495   SKIP_NOW="yes"
2496 elif is_condition "$cmd" || is_groupMarker "$cmd"; then
2497   unset 'CMD_SKIP';
2498   SKIP_NOW="yes"
2499 fi
2500 
2501 # let's do the skip now
2502 if [ -n "$SKIP_NOW" ]; then
2503   # sum up how many commands we actually skipped for the prev var routines
2504   CMD_SKIPPED=$((${CMD_SKIPPED-0} + 1))
2505   continue
2506 fi
2507 
2508 # save start time
2509 RUN_START=$(nsecs)
2510 
2511 # export some useful env vars for external scripts/programs to use
2512 export PROFILE CONFDIR SOURCE TARGET_URL_PROT TARGET_URL_HOSTPATH \
2513        TARGET_URL_USER TARGET_URL_PASS \
2514        GPG_KEYS_ENC=$(join "\n" "${GPG_KEYS_ENC_ARRAY[@]}") GPG_KEY_SIGN \
2515        GPG_PW CMD_PREV CMD_NEXT CMD_ERR CND_PREV CND_NEXT\
2516        RUN_START
2517 
2518 # user info
2519 echo; separator "Start running command $(toupper $cmd) at $(date_from_nsecs $RUN_START)"
2520 
2521 case "$(tolower $cmd)" in
2522   'pre'|'post')
2523     if [ "$cmd" == 'pre' ]; then
2524       script=$PRE
2525     else
2526       script=$POST
2527     fi
2528     # script execution in a subshell, protect us from failures/var overwrites
2529     ( run_script "$script" )
2530     ;;
2531   'bkp')
2532     duplify -- "${dupl_opts[@]}" $EXCLUDE_PARAM "$EXCLUDE" \
2533           "$SOURCE" "$BACKEND_URL"
2534     ;;
2535   'incr')
2536     duplify incr -- "${dupl_opts[@]}" $EXCLUDE_PARAM "$EXCLUDE" \
2537           "$SOURCE" "$BACKEND_URL"
2538     ;;
2539   'full')
2540     duplify full -- "${dupl_opts[@]}" $EXCLUDE_PARAM "$EXCLUDE" \
2541           "$SOURCE" "$BACKEND_URL"
2542     ;;
2543   'verify')
2544     TIME="${ftpl_pars[0]:+"-t ${ftpl_pars[0]}"}"
2545     duplify verify -- $TIME "${dupl_opts[@]}" $EXCLUDE_PARAM "$EXCLUDE" \
2546           "$BACKEND_URL" "$SOURCE"
2547     ;;
2548   'verifypath')
2549     TIME="${ftpl_pars[2]:+"-t ${ftpl_pars[2]}"}"
2550     IN_PATH="${ftpl_pars[0]}"; OUT_PATH="${ftpl_pars[1]}"; 
2551     ( [ -z "$IN_PATH" ] || [ -z "$OUT_PATH" ] ) && error "  Missing parameter <rel_bkp_path> or <local_path> for verifyPath.
2552   
2553   Hint: 
2554     Syntax is -> $ME <profile> verifyPath <rel_bkp_path> <local_path> [<age>]"
2555 
2556     duplify verify -- $TIME "${dupl_opts[@]}" $EXCLUDE_PARAM "$EXCLUDE" \
2557           --file-to-restore "$IN_PATH" "$BACKEND_URL" "$OUT_PATH"
2558     ;;
2559   'list')
2560     # time param exists since 0.5.10+
2561     TIME="${ftpl_pars[0]:+"-t ${ftpl_pars[0]}"}"
2562     duplify list-current-files -- $TIME "${dupl_opts[@]}" "$BACKEND_URL"
2563     ;;
2564   'cleanup')
2565     duplify cleanup -- "${dupl_opts[@]}" "$BACKEND_URL"
2566     ;;
2567   'purge')
2568     MAX_AGE=${ftpl_pars[0]:-$MAX_AGE}
2569     [ -z "$MAX_AGE" ] && error "  Missing parameter <max_age>. Can be set in profile or as command line parameter."
2570     
2571     duplify remove-older-than "${MAX_AGE}" \
2572           -- "${dupl_opts[@]}" "$BACKEND_URL"
2573     ;;
2574   'purgefull')
2575     MAX_FULL_BACKUPS=${ftpl_pars[0]:-$MAX_FULL_BACKUPS}
2576     [ -z "$MAX_FULL_BACKUPS" ] && error "  Missing parameter <max_full_backups>. Can be set in profile or as command line parameter."
2577   
2578     duplify remove-all-but-n-full "${MAX_FULL_BACKUPS}" \
2579           -- "${dupl_opts[@]}" "$BACKEND_URL"
2580     ;;
2581   'purgeincr')
2582     MAX_FULLS_WITH_INCRS=${ftpl_pars[0]:-$MAX_FULLS_WITH_INCRS}
2583     [ -z "$MAX_FULLS_WITH_INCRS" ] && error "  Missing parameter <max_fulls_with_incrs>. Can be set in profile or as command line parameter."
2584   
2585     duplify remove-all-inc-of-but-n-full "${MAX_FULLS_WITH_INCRS}" \
2586           -- "${dupl_opts[@]}" "$BACKEND_URL"
2587     ;;
2588   'restore')
2589     OUT_PATH="${ftpl_pars[0]}"; TIME="${ftpl_pars[1]:-now}";
2590     [ -z "$OUT_PATH" ] && error "  Missing parameter target_path for restore.
2591   
2592   Hint: 
2593     Syntax is -> $ME <profile> restore <target_path> [<age>]"
2594     
2595     duplify  -- -t "$TIME" "${dupl_opts[@]}" "$BACKEND_URL" "$OUT_PATH"
2596     ;;
2597   'fetch')
2598     IN_PATH="${ftpl_pars[0]}"; OUT_PATH="${ftpl_pars[1]}"; 
2599     TIME="${ftpl_pars[2]:-now}";
2600     ( [ -z "$IN_PATH" ] || [ -z "$OUT_PATH" ] ) && error "  Missing parameter <src_path> or <target_path> for fetch.
2601   
2602   Hint: 
2603     Syntax is -> $ME <profile> fetch <src_path> <target_path> [<age>]"
2604     
2605     # duplicity 0.4.7 doesnt like cmd restore in combination with --file-to-restore
2606     duplify -- --restore-time "$TIME" "${dupl_opts[@]}" \
2607               --file-to-restore "$IN_PATH" "$BACKEND_URL" "$OUT_PATH"
2608     ;;
2609   'status')
2610     duplify collection-status -- "${dupl_opts[@]}" "$BACKEND_URL"
2611     ;;    
2612   *)
2613     error "  Unknown command '$cmd'.
2614 
2615   Hint:
2616     Use '$ME usage' to get usage help."
2617     ;;
2618 esac
2619 
2620 CMD_ERR=$?
2621 RUN_END=$(nsecs)
2622 RUNTIME=$(( $RUN_END - $RUN_START ))
2623 
2624 # print message on error; set error code
2625 if [ "$CMD_ERR" -ne 0 ]; then
2626 	error_print "$(datefull_from_nsecs $RUN_END) Task '$(echo $cmd|awk '$0=toupper($0)')' failed with exit code '$CMD_ERR'."
2627 	FTPL_ERR=1
2628 fi
2629 
2630 separator "Finished state $(error_to_string $CMD_ERR) at $(date_from_nsecs $RUN_END) - \
2631 Runtime $(printf "%02d:%02d:%02d.%03d" $((RUNTIME/1000000000/60/60)) $((RUNTIME/1000000000/60%60)) $((RUNTIME/1000000000%60)) $((RUNTIME/1000000%1000)) )"
2632 
2633 done
2634 
2635 exit ${FTPL_ERR}