NAME
upfiles -- upload files to an FTP server, for push mirroring
SYNOPSIS
upfiles [--options] [filename...]DESCRIPTION
Upfiles uploads changed files from your local disk to an FTP server, for a simple kind of "push" mirroring.
Create files locally with the same directory structure as the target, and in a ~/.upfiles.conf file give the locations,
upfiles (local => '/my/directory',
remote => 'ftp://fred@some-server.org/pub/fred');This is actually Perl code, so you can put comment lines with #, write some conditionals, use $ENV{HOME}, etc. Then to upload run
upfilesOr to upload just some selected files (the local filenames),
upfiles /my/directory/foo.txt /my/directory/src/xyzzy.plYour username on the remote system is in the ftp:// remote URL. A password is taken from ~/.netrc the same as for the ftp program and other programs. See netrc(5) or Net::Netrc for the format of that file.
upfiles records what has been sent in an SQLite database file .upfiles.sqdb in each local toplevel directory, for example /my/directory/.upfiles.sqdb. Local changes are identified by comparing file mtimes and sizes against the database. This is faster than asking the remote server what it's got.
For convenience some local files are always excluded from the upload. Currently these are
.upfiles.sqdb from upfiles itself
foo~ Emacs backups
#foo# Emacs autosaves
.#foo Emacs lockfilesFiles are uploaded one by one. The upload goes first to a temporary file and is then renamed. This means an incomplete file isn't left if the connection is lost or upfiles is killed during transfer. Temporary files are noted in the database and leftovers are deleted on the next upfiles run if necessary.
File modification times are copied to the server if it has the draft standard MFMT or the common SITE UTIME extension (2-arg or 5-arg).
Plain RFC959 ftp doesn't have a notion of symlinks or hard links so upfiles follows any local links to actual content to upload. (Perhaps in the future the SITE SYMLINK extension could be used if available.)
CONFIGURATION
Each upfiles call in ~/.upfiles.conf takes the following parameters,
local(string)-
The local directory to upload from.
remote(string)-
The remote FTP server to upload to, as a URL. The path in the URL is the target directory, and if your username on the remote machine is not the same as your local username then include it with "@" syntax, like
remote => 'ftp://fred@some-server.org/pub/fred', exclude_regexps(arrayref of regexps)-
Additional filenames to exclude. For example to exclude a local Makefile
upfiles (local => '/my/directory', remote => 'ftp://some-server.org/pub/fred', exclude_regexps => [ qr{/(^|/)[Mm]akefile$} ]); copy_utime(0, 1, default"if_possible")-
Whether to copy file modification times to the server with the
MFMTorSITE UTIMEcommand. The default"if_possible"means do so if the server supports it. 0 means don't try. 1 means it must work.
COMMAND-LINE OPTIONS
The command line options are
- -n, --dry-run
-
Show what would be uploaded or deleted on the server, but don't actually do anything.
upfiles -n - --help
-
Print some brief help information.
- -V, --verbose, --verbose=N
-
Print some diagnostics about what's being done. With --verbose=2 or --verbose=3 print some technical details too.
upfiles --verbose - --version
-
Print the upfiles program version number. With
--verbose=2also print the version numbers of some modules used.
FILES
- ~/.upfiles.conf
-
Configuration file.
- ~/.netrc
-
FTP password file.
- /some/local/dir/.upfiles.sqdb
-
SQLite database of information about what has been sent.
Upfiles determines the home directory ~ using File::HomeDir. Net::Netrc has something similar for the .netrc file, and looks also for _netrc.
BUGS
Changing a local file from a file to a directory or vice versa probably doesn't work very well. Remove it and upload, then create the new and upload that.
FTP requires a couple of round trip command/responses to the server for every file. When uploading many small files something streaming or parallel would be faster. The temp file and rename adds a round trip too, but is desirable so anyone looking doesn't see a half file. Perhaps an option could turn this off if not needed (such as upfiles to a remote backup).
The temporary files are named using the local $$ PID added to the target filename. This is enough to protect against simultaneous uploads from the same source machine, but potentially unsafe if you're networked and are foolish enough to upfiles the same tree simultaneously from two different source machines. STOU would guarantee uniqueness, but does it leave a window while the name comes back which if interrupted could leave the file created but unknown? Net::FTP put_unique() doesn't return the name until after transfer too.
Filenames containing \r or \n cannot be sent by FTP protocol and Upfiles will refuse to operate on them.
There's a small chance of a 2-arg SITE UTIME attempt looking like a 5-arg to a pre-1.0.33 pure-ftpd server (of circa 2011), if the filename happens to be dates and "UTC". Perhaps more care could be taken to identify the SITE UTIME style the server expects. In practice such filenames should be unlikely, and there's no problem with recent pure-ftpd which has MDTM.
SEE ALSO
Net::FTP, netrc(5), Net::Netrc, DBD::SQLite
sitecopy(1), ftpmirror(1), ftp-upload(1), rsync(1)
HOME PAGE
http://user42.tuxfamily.org/upfiles/index.html
(Upfiles is good for uploading to tuxfamily.)
LICENSE
Copyright 2009, 2010, 2011, 2012, 2013, 2014, 2015 Kevin Ryde
Upfiles is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3, or (at your option) any later version.
Upfiles is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
You should have received a copy of the GNU General Public License along with Upfiles. If not, see http://www.gnu.org/licenses/.