3. Checking out Ptolemy II

4. SVN vs CVS

Subversion was originally designed to be a better CVS, so it has most of CVS's features. Generally, Subversion's interface to a particular feature is similar to CVS's, except where there's a compelling reason to do otherwise.Subversion has since expanded beyond its original goal of replacing CVS, but its history influenced its feature and interface choices; Subversion today should still feel very familiar to CVS users.

CVS has CVS's features (CVS Wins)

Directories are versioned.

Subversion versions directories as first-class objects, just like files.(SVN Wins)

In CVS, directories are not first-class objects

Copying, deleting, and renaming are versioned

Copying and deleting are versioned operations. Renaming is also a versioned operation, albeit with some quirks.(SVN wins because of renaming)

Renaming in CVS is difficult

Free-form versioned metadata ("properties").

Subversion allows arbitrary metadata ("properties") to be attached to any file or directory. These properties are key/value pairs, and are versioned just like the objects they are attached to. Subversion also provides a way to attach arbitrary key/value properties to a revision (that is, to a committed changeset). These properties are not versioned, since they attach metadata to the version-space itself, but they can be changed at any time. (SVN Wins, though it is vendor lock-in)

CVS does not have this, but do we need it? Actually, it turns out that we can use these properties in Kepler to place a dependency on the ptII svn tree so that when a user checks out the kepler svn tree, they can automatically check out the ptII svn tree. This is a case of vendor lock-in, but does give SVN an advantage.

Atomic commits.

No part of a commit takes effect until the entire commit has succeeded. Revision numbers are per-commit, not per-file, and commit's log message is attached to its revision, not stored redundantly in all the files affected by that commit.

This is a UI issue, In Eclipse, it is possible to commit many changes at once. In 50k changes over 12 years, we have not had a problem. I could see in a very busy repository, this might help. but if the repository is that busy, then there are going to be plenty of conflict problems.

Branching and tagging are cheap (constant time) operations.

There is no reason for these operations to be expensive, so they aren't. Branches and tags are both implemented in terms of an underlying "copy" operation. A copy takes up a small, constant amount of space. Any copy is a tag; and if you start committing on a copy, then it's a branch as well. (This does away with CVS's "branch-point tagging", by removing the distinction that made branch-point tags necessary in the first place.)

Computers are fast, I've not noticed this as a problem.

Merge tracking.

Subversion 1.5 introduces merge tracking: automated assistance with managing the flow of changes between lines of development, and with the merging of branches back into their sources. The 1.5 release of merge tracking has basic support for common scenarios; we will be extending the feature in upcoming releases.

Again, Eclipse or diff help here.

File locking.

Subversion supports (but does not require) locking files so that users can be warned when multiple people try to edit the same file. A file can be marked as requiring a lock before being edited, in which case Subversion will present the file in read-only mode until a lock is acquired.

CVS can support file locking, but file locking is bad.

Symbolic links can be versioned.

Unix users can place symbolic links under version control. The links are recreated in Unix working copies, but not in win32 working copies.(SVN wins)

CVS does not create symbolic links. It would be nice if svn would create cygwin compatible links under win32.

Executable flag is preserved.

Subversion notices when a file is executable, and if that file is placed into version control, its executability will be preserved when it it checked out to other locations. (The mechanism Subversion uses to remember this is simply versioned properties, so executability can be manually edited when necessary, even from a client that does not acknowledge the file's executability, e.g., when having the wrong extension under Microsoft Windows).

CVS has something similar.

Apache network server option, with WebDAV/DeltaV protocol.

Subversion can use the HTTP-based WebDAV/DeltaV protocol for network communications, and the Apache web server to provide repository-side network service. This gives Subversion an advantage over CVS in interoperability, and allows certain features (such as authentication, wire compression) to be provided in a way that is already familiar to administrators (SVN Wins, if you are willing to have a significantly less secure and compartmentalized source server)

CVS does not have WebDAV access, so offsite developers behind firewalls have to jump through hoops. However, CVS is much more secure than SVN.

Standalone server option (svnserve).

Subversion offers a standalone server option using a custom protocol, since not everyone wants to run an Apache HTTPD server. The standalone server can run as an inetd service or in daemon mode, and offers the same level of authentication and authorization functionality as the HTTPD-based server. The standalone server can also be tunnelled over ssh.

CVS has pserver

Parseable output.

All output of the Subversion command-line client is carefully designed to be both human readable and automatically parseable; scriptability is a high priority.

CVS is similar

Localized messages.

Subversion uses gettext() to display translated error, informational, and help messages, based on current locale settings.(SVN Wins, but this is not that important)

CVS does not particularly have localization.

Interactive conflict resolution.

The Subversion command-line client (svn) offers various ways to resolve conflicting changes, include interactive resolution prompting. This mechanism is also made available via APIs, so that other clients (such as graphical clients) can offer interactive conflict resolution appropriate to their interfaces. (A slight win for SVN)

CVS indicates conflicts and expects the user to use their IDE.

Repository read-only mirroring.

Subversion supplies a utility, svnsync for synchronizing (via either push or pull) a read-only slave repository with a master repository. (A big win for SVN)

CVS does not have mirroring out of the box.

Write-through proxy over WebDAV.

Subversion 1.5 introduces a write-through proxy feature that allows slave repositories (see read-only mirroring) to handle all read operations themselves while passing write operations through to the master. This feature is only available with the Apache HTTPD (WebDAV) server option. (A win for SVN, if security is not a problem)

CVS does not have this feature.

Natively client/server, layered library design with clean APIs.

Subversion is designed to be client/server from the beginning; thus avoiding some of the maintenance problems which have plagued CVS. The code is structured as a set of modules with well-defined interfaces, designed to be called by other applications.

CVS is mature software with a idiosyncratic interface. This is neither an advantage nor a disadvantage.

Binary files handled efficiently.

Subversion is equally efficient on binary as on text files, because it uses a binary diffing algorithm to transmit and store successive revisions. (SVN wins, but size is not a huge problem these days and SVN repositories are almost 2x CVS repositories)

CVS handles binary files, but not efficiently.

Costs are proportional to change size, not data size.

In general, the time required for a Subversion operation is proportional to the size of the changes resulting from that operation, not to the absolute size of the project in which the changes are taking place.

CVS and SVN seem to have about the same speed over ssh. SVN over WebDAV will be slower than SVN over SSH.

Bindings to programming languages.

The Subversion APIs come with bindings for many programming languages, such as Python, Perl, Java, and Ruby. (Subversion itself is written in C.)

CVS is callable from the command line. I don't see this as important.

Changelists.

Subversion 1.5 introduces changelists, which allows a user to put modified files into named groups on the client side, and then commit by specifying a particular group. For those who work on logically separate changesets simultaneously in the same directory tree, changelists can help keep things organized.

Is this important?

5. SVN benefits to Kepler

WebDAV

Access from behind firewalls works.

LDAP access is easier than named Unix accounts

WebDAV has finer grained access control

Kepler found cvs2svn conversion to result in smaller modules, which is different than the ptII experience (see below).

7. Complaints against Subversion

Is svn that much better than cvs? http://subversion.tigris.org says "Subversion is meant to be a better CVS, so it has most of CVS's features". Typical arguments used for svn over cvs:

Merging in svn is better than in cvs. How many people actually use branches?

It is easier to move files in svn than in cvs. This has some merit, but is it worth the effort?

If svn is a better CVS, then why is the svn status command different than cvs status.

It turns out that svn info provides some of the info of cvs status, but how do I determine if keyword substitution is occurring? Under cvs, the cvs status command says -kb. What about svn?

Building the client requires way too many other packages. How can svn possibly stay secure if it depends on so many packages?

Subversion can optionally use Apache for access. Enabling a web server on a machine that does not already have one makes the machine less secure.

There is no decent svn Unix style man page. This is deliberate, see Bug 1508. This is not good. I want to know exactly what commands will work with a specific installation of SVN, not what the latest documentation for the latest version is.

On the server, svn is a disk hog.

A gzipped tar file of the Ptolemy II cvs tree is 372.9 MB.

A gzipped tar file of the same tree after running the conversion from cvs to svn is 570MB.

A user wrote:
"I found both Subversive and Subclipse painful; I had to perform significant repairs externally using TortoiseSVN. I even managed to get Subversive to delete a 10MB directory tree by mistake!"

"Both Subversive and Subclipse are very dangerous for CVS users. In CVS, folders do not have significant state so you can do what you like when you like. In SVN you MUST commit folder state immediately after committing file state. For instance if you add something to svn:ignore, you must commit the folder, otherwise you risk having an outgoing conflict between svn:ignore outgoing and child folder state incoming. Do not attempt to fix such a conflict with either of Subclipse or Subversive; you just dig a deeper hole. Use TortoiseSVN to get back to all green arrows."

"Unfortunately neither subclipse nor subversion have GUI state for folder state comparison so you get no clue as to why folders are in a mess. The GUI is generally inadequate in distinguishing between folder as content and folder as container of children."

Another drawback with svn is that the modtimes of the files are not

preserved. When I do a cvs checkout, the modtimes of the files
are preserved. This makes it much easier to see what has
recently changed.

Being able to quickly see which Java file was last updated in the cvs tree is a serious win.

Also, the ptII tree includes derived files that are generated by Autoconf, JavaCC and Antlr. If the commit times are not preserved, then the source file might have a mod time later than the derived file which means that either the tool must be run or else the build system must touch the derived file. This is gross. Why not just preserve the commit time?

Why doesn't svn preserve mod times?

Ian Brown mentioned that TortoiseSVN has an option that does this:

Set filedates to the "last commit time"

This option tells TortoiseSVN to set the filedates to the last commit time when doing a checkout or an update. Otherwise TortoiseSVN will use the current date. If you are developing software it is generally best to use the current date because build systems normally look at the datestamps to decide which files need compiling. If you use "last commit time" and revert to an older file revision, your project may not compile as you expect it to.

use-commit-times can be set in the Subversion config file. The tricky part is finding that file. For me, under Cygwin, the file was $HOME/.subversion/config. Other places to look are c:/Documents and Settings/cxh/Application Data/Subversion/config. Make this change:

### Set use-commit-times to make checkout/update/switch/revert### put last-committed timestamps on every file touched.use-commit-times = yes

svn2cl is slow. This is not a svn problem, but a problem with xsltproc. Running svn2cl on a the ptII tree takes 43 minutes. Running gnuify-changelog.pl takes 84 seconds. However, gnuify-changelog.pl does not include the names of the files that were committed.

Still, I miss cvs2cl, which had nice features like clumping non-atomic cvs commits into one block of commits.
It is a drawback in svn log that the output is not similar to cvs log.

The developer of svn2cl said:

I'm afraid that XSLT processing can be quite slow. Some improvements have been made in svn2cl in the past to improve performance but using XSLT there is not much more room for improvement that I can think off while keeping the thing maintainable (not that XSLT templates are maintainable to begin with ;-( ).

You could however try the --group-by-day option or generate the ChangeLog in batches and combine those as a post-processing step. Some projects have a yearly ChangeLog file. Another option is to keep the previous ChangeLog file around and only generate new entries (svn2cl can't do this automatically yet though).

Conversion from CVS lost a few files.

Conversion from CVS requires --default-eol=native. Instead, it should just convert files they are in the tree. Why do I have to specify a native default?

8.4 Problems

Why do is --default-eol=native not the default? My repository CVS repository is fine with regard to the binary mode, yet the tool ignores this? That is like having cp ignore null characters in files because they might cause a NullPointerException. Sheesh.

Problems with a file being in the Attic. This is a sign of corruption, see the cvs2svn faq. I just removed that attic file

Problems with encoding. Some of the files in ptII/ptolemy/domains/wireless/demo/CooperativeTerminals/Attic/ have the character \222. The fix is to use --fallback-encoding=utf_8. The command to restart is

We use shortened urls like svn co svn+ssh://source.eecs.berkeley.edu/chess/ptII/trunk ptII. The way we do this is by creating links in / for chess and other directories. svnserve has a -r option, which sets the virtual root. We don't use that, because if we did, then paths in svn co svn+ssh://source.eecs.berkeley.edu/home/svn/chess/ptII/trunk would not work. Instead, we create links for in /.

# svn (subversion)pass in quick on bge0 proto tcp from any to 128.32.48.234 port = 3690 flags S keep state group 100pass in quick on bge0 proto udp from any to 128.32.48.234 port = 3690 flags S keep state group 100

The second arg says to enable shared module support which is needed for a typical compile of mod_dav_svn (see below).

The third arg says to include debugging information. If you built Subversion with --enable-maintainer-mode, then you should do the same for Apache; there can be problems if one was compiled with debugging and the other without.

Note: if you have multiple db versions installed on your system, Apache might link to a different one than Subversion, causing failures when accessing the repository through Apache. To prevent this from happening, you have to tell Apache which db version to use and where to find db. Add --with-dbm=db4 and --with-berkeley-db=/usr/local/BerkeleyDB.4.2 to the configure line. Make sure this is the same db as the one Subversion uses. This note assumes you have installed Berkeley DB 4.2.52 at its default locations. For more info about the db requirement, see section I.5.

You may also want to include other modules in your build. Add --enable-ssl to turn on SSL support, and --enable-deflate to turn on compression support, for example. Consult the Apache documentation for more details.

<Location /svn/chess># # Note: The following must must be present to support # starting without SSL on platforms with no /dev/random equivalent # but a statically compiled-in mod_ssl. # <IfModule ssl_module>SSLRandomSeed startup builtinSSLRandomSeed connect builtin</IfModule>

However, the problem here is that we are downloading the CA file from the website we are trying to verify. Another workaround is to download the root CAs from Verisign. The file we are interested in is PCA3ss_v4.509.

14.2 Fixing OpenSSL

The OpenSSL software is shipped without any root CA certificate as the OpenSSL project does not have any policy on including or excluding any specific CA and does not intend to set up such a policy. Deciding about which CAs to support is up to application developers or administrators.

Other projects do have other policies so you can for example extract the CA bundle used by Mozilla and/or modssl as described in this article:

It would be nice if we could import all the Commercial CAs in one fell swoop.

15. Firewall and Proxy Problems

If you get:

RA layer request failedsvn: OPTIONS of 'https://source.eecs.berkeley.edu/svn/chess': Could not resolve hostname `source.eecs.berkeley.edu': The requested name is valid and was found in the database, but it does not have the correct associated data being resolved for.

Under TortoiseSVN, you could try setting the TortoiseSVN proxy to match the proxy of your web browser. If you have a proxy autoconfiguration URL, then try entering the URL into your browser and analyzing the code to determine the name of your proxy machine.

Another thing to try is to edit the \Documents and Settings\user\Application Data\Subversion\servers file and add

Why is it necessary to add have a pattern for every file? The answer is that Subversion decides that everything is a binary file and that it is safer to check things in and not modify them. However, there should be a repository wide way to set up config instead of requiring each user to do so.

Another limitation is that the enable-auto-props=yes line must be before the [auto-props] line. This is really lame. It is a natural mistake to have enable-auto-props=yes after [auto-props]. Instead, auto-props should be enabled if [auto-props]] is uncommented.

To test this out, if you have read/write permission to the source.eecs.berkeley.edu SVN repositories, try:

Note that testfile.txt had $Id$ properly substituted. If it had only $Id$ and not $Id: testfile.txt 8 2009-03-05 01:27:44Z cxh $
then keywords were not being substituted and that ~/.subversion/config had a problem.

then if the user creates a makefile, it will have keywords set, but if the user creates a Makefile it will not have the keywords set. If course, this only matters on real file systems such as Mac OS X or Unix. Windows users have the oddly antiquated case insensitive, case preserving file system.

It turns out one of our Eclipse users was using either Subclipse or Subversive to connect via ssh. They had the wrong password, so there were many bogus connections.

The account that was causing the problem was using Eclipse on the Mac with these versions
of Subversive:
(:source)
Subversive SVN Connectors 2.1.0
Subversive SVN Team Provider (Inclubation) 0.7.3
SVNKit 1.2.2 Implementation (Optional) 0.7.3
(:sourceend:)

I edited /etc/ssh/sshd_config:

# The maximum number of concurrent unauthenticated connections to sshd.# start:rate:full see sshd(1) for more information.# The default is 10 unauthenticated clients.#MaxStartups 10:30:60# cxh: Updated to 20 to avoid # "ssh_exchange_identification: Connection closed by remote host" messagesMaxStartups 20:30:60

21. Get repository folder children operation failed

After upgrading the svn server to 1.7.2 on source, Eclipse Indigo SR1 (M20110909-1335) with "SVNKit 1.3.5 Implementation (Optional) 2.2.2.I20110819-1700 org.polarion.eclipse.team.svn.connector.svnkit16.feature.group Polarion Software" was failing when the URL was svn+ssh://source.eecs.berkeley.edu/chess/ptII.

The problem is that clicking on trunk results in <An error occurred whil accessing the repository entry >. The stack trace is

org.eclipse.team.svn.core.connector.SVNConnectorException: svn:URL'svn+ssh://source.eecs.berkeley.edu/chess/ptII/trunk' non-existent in that revision
at org.polarion.team.svn.connector.svnkit.SVNKitConnector.handleClientException(SVNKitConnector.java:1415)
at org.polarion.team.svn.connector.svnkit.SVNKitConnector.list(SVNKitConnector.java:1264)
at org.eclipse.team.svn.core.extension.factory.ThreadNameModifier.list(ThreadNameModifier.java:278)
at org.eclipse.team.svn.core.utility.SVNUtility.list(SVNUtility.java:335)
at org.eclipse.team.svn.core.svnstorage.SVNRepositoryContainer.getChildren(SVNRepositoryContainer.java:79)
at org.eclipse.team.svn.core.operation.remote.GetRemoteFolderChildrenOperation.runImpl(GetRemoteFolderChildrenOperation.java:75)
at org.eclipse.team.svn.core.operation.AbstractActionOperation.run(AbstractActionOperation.java:81)
at org.eclipse.team.svn.core.utility.ProgressMonitorUtility.doTask(ProgressMonitorUtility.java:104)
at org.eclipse.team.svn.core.operation.CompositeOperation.runImpl(CompositeOperation.java:95)
at org.eclipse.team.svn.core.operation.AbstractActionOperation.run(AbstractActionOperation.java:81)
at org.eclipse.team.svn.core.operation.LoggedOperation.run(LoggedOperation.java:39)
at org.eclipse.team.svn.core.utility.ProgressMonitorUtility.doTask(ProgressMonitorUtility.java:104)
at org.eclipse.team.svn.core.utility.ProgressMonitorUtility.doTaskExternal(ProgressMonitorUtility.java:90)
at org.eclipse.team.svn.ui.utility.DefaultCancellableOperationWrapper.run(DefaultCancellableOperationWrapper.java:55)
at org.eclipse.team.svn.ui.utility.ScheduledOperationWrapper.run(ScheduledOperationWrapper.java:37)
at org.eclipse.core.internal.jobs.Worker.run(Worker.java:54)
Caused by: org.tigris.subversion.javahl.ClientException: svn:URL'svn+ssh://source.eecs.berkeley.edu/chess/ptII/trunk' non-existent in that revision
at org.tigris.subversion.javahl.JavaHLObjectFactory.throwException(JavaHLObjectFactory.java:779)
at org.tmatesoft.svn.core.javahl.SVNClientImpl.throwException(SVNClientImpl.java:1924)
at org.tmatesoft.svn.core.javahl.SVNClientImpl.list(SVNClientImpl.java:365)
at org.tmatesoft.svn.core.javahl.SVNClientImpl.list(SVNClientImpl.java:337)
at org.polarion.team.svn.connector.svnkit.SVNKitConnector.list(SVNKitConnector.java:1253)
... 14 more
Caused by: org.tmatesoft.svn.core.SVNException: svn:URL'svn+ssh://source.eecs.berkeley.edu/chess/ptII/trunk' non-existent in that revision
at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:64)
at org.tmatesoft.svn.core.internal.wc.SVNErrorManager.error(SVNErrorManager.java:51)
at org.tmatesoft.svn.core.wc.SVNLogClient.doList(SVNLogClient.java:1334)
at org.tmatesoft.svn.core.wc.SVNLogClient.doList(SVNLogClient.java:1217)
at org.tmatesoft.svn.core.javahl.SVNClientImpl.list(SVNClientImpl.java:350)
... 16 more

23. svn: This client is too old to work with working copy '.'; please get a newer Subversion client

If you use Subversion with Eclipse and install a Subversion plugin that is newer than the command line svn command, then when you run svn, you will get svn: This client is too old to work with working copy '.'; please get a newer Subversion client. The solution is to update the version of svn on your machine.

"Fortunately, Subversion provides support for externals definitions. An externals definition is a mapping of a local directory to the URL—and ideally a particular revision—of a versioned directory. In Subversion, you declare externals definitions in groups using the svn:externals property. You can create or modify this property using svn propset or svn propedit (see the section called “Manipulating Properties”). It can be set on any versioned directory, and its value describes both the external repository location and the client-side directory to which that location should be checked out."

So, in theory, we could add a svn:externals property to the ptII svn repo.

The above is partly an artifact of repo.eecs, but it is a general problem. I don't think repo.eecs can be modified to better handle this.

One idea was to do something complex with HTTPS and redirects, but that is not likely to work. I tried it before and reviewing the web shows that it would work with the browser, but svn will want the user to run svn relocate.