-*- text -*-
Bugs:
- "sitecopy foo bar" always process foo and bar in the order they
are specified in the rcfile
- review w.r.t djb's FTP interop notes, http://cr.yp.to/ftp.html
- add support for MLST, latest draft:
http://www.ietf.org/internet-drafts/draft-ietf-ftpext-mlst-16.txt
- enable large file support by default when necessary.
- Synch mode won't transfer in ASCII mode, nor maintain symlinks, nor
correctly order file deletions.
- Filename conflicts are not handled by update or synch mode: case:
Directory exists remote called "foo". Delete locally, replace with
file called "foo". Try an update. Bang. Another nice one is:
mv foo tmp
mv bar foo
mv tmp bar
Which, in renames mode, will be correctly discovered, but when we
do an --update... bang.
- "sitecopy --rcfile=/" should say the right thing
- The man page does not make clear what "exclude" excludes *from*.
The point that exclude will translate into deleting existing files
from the remote site is important and confusing.
Known Standards Compliance Problems:
- RFC959: handle telnet control characters?
- RFC2518: accept collection URI's in DAV:href without trailing slash
- RFC2617: support 'domain'
* anything with (FEAPI) after requires frontend API change, not including
simple config options additions
- Move use_this out of struct site, use proper data structures in console
frontend to represent sites passed on cmd line.
- Anything in the below list.
- Do the hash table thang.
Required Features for one-point-oh:
- Verify mode, also a --force-overwrite to force updates... maybe
--prompt-overwrite too.
- Filename conflict resolution for update mode, as per bug... or at LEAST,
conflict detection: can be done in file_set, not too hard probably.
- Read HTTP proxy from HTTP_PROXY/http_proxy environment variable.
- Document files list order dependancies.
- Do default ports, netrc lookups for the proxy too
- Report corrupt info files back to the user (FEAPI)
- Check write return codes in site_writefiles, signal the error if
the info file doesn't get written properly.
- Get warning (i.e. non-fatal) messages to the user.
- Check remote directory exists on initial connection in FTP (chdir/ls)
- In davdriver.c:dir_remove, check whether the collection is empty before
deleting it.
- Because it's The Right Thing, add hash table indexing of files list.
Hash using DJBs *33 hash algorithm. Open hash so we don't have
worry about filling the table up. Size the hash table in the range
100kB-200kB to not give too much run-time bloat, but not to fill up
with reasonable-sized sites too quickly. Maybe do a user survey to determine
optimum hash-table size.
- "include" option to mirror "exclude".
"Would be Nice to Have But We Can Live Without" features:
- "sftp" support; FTP-over-SSL
- Console: don't read local site state in --fetch mode.
- Support FTP proxying (how?): there are several different ways this is done...
- Support for Content-MD5 header and allow server-side MD5'ing, rather
than having to download, checksum and discard. (remote-checksum)
- Symlink 'maintain' mode for FTP: can you create symlinks over FTP???
- Write complete documentation using GNU texinfo or DocBook, for a
printed manual and info pages.
Possible features, which need more consideration:
- Per-site lockfiles (FEAPI)
- Shortcircuit parm in file_set_* used for the FIRST site state read on a
site, to make the file search always fail - guarantee to be true, since if we
read local state before stored state, there is no stored state in the
list when we read the local state.
- Console: Some kind of 'first-time-use' option, `sitecopy --first-time'
runs a wizard a la the GNOME fe's one.
- Support for other better/faster checksumming algorithms: is SHA1
better/faster? (GPL implementation in GnuPG)
- Backup info file on write_stored... optionally? Only implement in frontend?
GNU-style $VERSION_CONTROL support?
- Have 'preconnect' and 'postconnect' options which run user-specified
programs before and after an update, synch etc.
- Some kind of user-feedback for slow startup in checksumming mode.
takes approx 1 sec to MD5 a 10mb file on a K5 166 -> okay for average-sized
sites. (FEAPI)...
- Abstract protocol drivers into a mc VFS-like 'open', 'read' etc.
Abstract sites code so that "local" and "remote" can be handled by any
of {file, http, ftp}. Then, update + synch could possibly merge, since
an synch is an update with the remote and local sides switched (kindof).
- Allow file->file sites (screem wants this)... as above, or simply by
implementing another protocol driver.
- consequently, read ls-laR.txt files and be more like 'mirror'
- Add quota management, specify a per-site quota and only do update
if the result means the site will stay under quota.
-> problem: a directory uses up k's, but how many?
- Abstract protocol drivers into a mc VFS-like 'open', 'read' etc.
Abstract sites code so that "local" and "remote" can be handled by any
of {file, http, ftp}. Allows file->file sites, which screem wants.
- Read ls-laR.txt files and be more like 'mirror'
Evaluation of sitecopy alternatives: weex
- weex beats sitecopy hands-down in new user ease-of-use: you just run it.
With sitecopy you have to do --fetch or --catchup first. This situation
is slightly improved in 0.9, where on the first 'sitecopy newsite'
invocation, you get told what to do next.
- sitecopy could improve by doing:
an interactive 'on first run for site', like the GNOME fe site
creation wizard. This could create the .sitecopy directory with the
correct permissions (but I am a bit dubious about this).
- Another alternative is the --first-time option. This could do:
mkdir .sitecopy with correct perms
create .sitecopyrc with correct perms
? enter a complete site definition, and run --init, or --fetch,
or catchup, as appropriate.
This could either be run automagically if no .sitecopyrc file is
found (don't let trigger by --rcfile= option), or, better, by a message
telling you to run --first-time.
--new-user might be better...
Definitely Not-till-after-1.0 thinkings:
The current "file list" is bad. It is pseudo-sorted by depth (probably).
The GNOME fe wants a proper directory tree representation.
To allow the possibility of doing the "spot moved directories" test,
we might want a proper dirtree; but, this is a complex task, and might
be achieved in another way.
To get decent "checkmoved" operation, need better than O(n) lookup.
O(1)-ish could be achieved using a a hash table.
- Find operation on the files list is O(n), making state reads O(n^2).
Hashing would be nice -> can use the MD5 csum.
- Intelligent file movement detector, to spot whole moved directories:
Possibly implement by checksumming relative filenames for EACH
directory (fairly nasty overhead); so each directory has a
children_checksum field. Need a clever checksumming algo; MD5 would
require identical ordering, which would be a heavy constraint.
- Make the protocol drivers and sites code thread-safe.
Things You Might Like to Do On A Rainy Day:
- Investigate any extra handling needed for servers which have case
insensitive filenames
- Native Windows port (e.g. reimplement socket.c using the Winsock API)
- Convince the maintainer that it's more productive to spend time
implementing features than carefully crafting a mile-long TODO list.
XSitecopy TODO
--------------
- Tooltip(s) for the site widgets; specifically safe mode.
- Integrating resynch into the app.
- View files using gnome mime types.
- Transition from time-size to checksum.
* Prefs
Future releases:
- Bring back optional 'slim' mode which will take up less desktop real estate.
- Popup menus for the site/file tree.
- Single file updates.
- Update all.
Longer term things:
- Panel applet for easily updating sites in a couple of clicks.
- Html based reports - integration into FE apps, and possible auto uploading
to the remote site. (useful as a "recent changes" page)
* Please send any suggestions you may have as to the format/design/type of
reports that you might find useful.