Apache Subversion 1.8 Release Notes

What's New in Apache Subversion 1.8

Apache Subversion 1.8 is a superset of all previous Subversion releases, and is as of the time of its release considered the current "best" release. Any feature or bugfix in 1.0.x through 1.7.x is also in 1.8, but 1.8 contains features and bugfixes not present in any earlier release. The new features will eventually be documented in a 1.8 version of the free Subversion book (svnbook.red-bean.com).

This page describes only major changes. For a complete list of changes, see the 1.8 section of the CHANGES file.

Compatibility Concerns

Older clients and servers interoperate transparently with 1.8 servers and clients. However, some of the new 1.8 features may not be available unless both client and server are the latest version. There are also cases where a new feature will work but will run less efficiently if the client is new and the server old.

There is no need to dump and reload your repositories. Subversion 1.8 servers can read and write to repositories created by earlier versions. To upgrade an existing server installation, just install the newest libraries and binaries on top of the older ones.

Subversion 1.8 maintains API/ABI compatibility with earlier releases, by only adding new functions, never removing old ones. A program written to any previous 1.x API can both compile and run using 1.8 libraries. However, a program written for 1.8 cannot necessarily compile or run against older libraries.

There may be limited cases where the behavior of old APIs has been slightly modified from previous releases. These are cases where edge cases of the functionality has been deemed buggy, and therefore improved or removed. Please consult the API errata for more detailed information on what these APIs are and what impact these changes may have.

New Feature Compatibility Table

New Feature Minimum Client1 Minimum Server Minimum Repository Notes
working copy records moves 1.8 any any
Automatic reintegration merge 1.8 1.5 1.5
neon support removed 1.8 any any Server-side configuration changes might be required for optimal performance with 1.8 clients which access the repository via HTTP.
Inheritable properties 1.8 any any A 1.8 server isn't required but will deliver superior performance when asked for inherited properties.
Repos dictated config props 1.8 any any
Password caching with gpg-agent 1.8 any any
FSFS enhancements any 1.8 1.8 Repository size reduction requires dump/load.
Authz file in repository any 1.8 any
1Reminder: when using the file:// repository access method, the Subversion program is both the client and the server.

Upgrading the Working Copy

Subversion 1.8 introduces changes to the working copy format. In previous releases of Subversion (1.6 and earlier), Subversion would automatically upgrade the working copy to the new format when a write operation was performed. Subversion 1.8, however, requires an upgrade for both read and write operations on the working copy, and makes the upgrade a manual step.

Before using Subversion 1.8 with existing working copies, users will be required to run the svn upgrade command to upgrade working copy metadata to the new format. This command may take a while, and for some users, it may be more practical to simply checkout a new working copy.

Subversion 1.8 can only upgrade working copies created with Subversion 1.6 and Subversion 1.7.

Note: Subversion 1.8 cannot upgrade working copies that a 1.6 client would have refused to operate upon before an svn cleanup was run (with a 1.6 client). In other words, before upgrading to 1.8, a 1.6 client must be used to run svn cleanup on all 1.6 working copies that require cleanup. Likewise, Subversion 1.8 cannot upgrade corrupt 1.6 working copies. Unfixable problems can arise from missing or corrupt meta-data inside .svn directories. Such damage to the 1.6 working copy is permanent, and cannot be fixed even if svn cleanup is run prior to the upgrade.

If your working copy does not upgrade cleanly, please check out a new one.

Command Line Output Changes

Although we try hard to keep output from the command line programs compatible between releases, new information sometimes has to be added. This can break scripts that rely on the exact format of the output. For this reason, we encourage programs which consume the output of the commandline client to consider using the --xml option, or accessing Subversion through the various bindings interfaces.

Improved output of svn mergeinfo

The default output of svn mergeinfo has been changed. Instead of being equivalent to svn mergeinfo --show-revs=merged, it now shows a diagrammatic summary of some information about merges between the two specified branches:

$ svn mergeinfo ^/subversion/branches/moves-scan-log
    youngest common ancestor
    |         last full merge
    |         |        tip of branch
    |         |        |         repository path

    1186287            1445648 
    |                  |       
       --| |------------         subversion/branches/moves-scan-log
      /         /              
     /         /               
  -------| |------------         subversion/trunk
              |        |       
              1241413  1445640 
$ 

Scripts using svn mergeinfo without a --show-revs= option should be updated to add --show-revs=merged.

New options for svn proplist and svn propget

The default output for these commands remains the same, but both support the new option --show-inherited-props which may produce output changes from 1.7.x. See inheritable properties for more information.

Improved output and new options for svnlook proplist and svnlook propget

The output of svnlook proplist and svnlook proplist --verbose has changed and now mimics the output of svn proplist and svn proplist --verbose respectively. svnlook propget now supports the --verbose option, which produces output similar to svn propget --verbose.

Both svnlook proplist and svnlook propget now support the new option --show-inherited-props which may produce output changes from 1.7.x. See inheritable properties for more information.

svn status and svn info now show local moves

svn status now shows moves in its output. See the section about local moves for more information.

svn status shows an extra line for each item involved in a move:

$ svn move epsilon/zeta zeta-moved
$ svn status
D       epsilon/zeta
        > moved to zeta-moved
A  +    zeta-moved
        > moved from epsilon/zeta
$ 

svn info shows extra lines for locally moved items, too. For example, if the file beta was moved to beta-new, svn info beta will show the following (some unrelated output has been omitted in this example):

$ svn info beta      
Path: beta
[...]
Schedule: delete
Moved To: beta-new
[...]

svn info now shows repository-relative URLs

svn info now includes repository-relative URLs of items in the working copy in its output. The command line client has been accepting this URL notation since Subversion 1.6.
$ svn info README                                       
Path: README
Name: README
Working Copy Root Path: /tmp/svn-trunk
URL: https://svn.apache.org/repos/asf/subversion/trunk/README
Relative URL: ^/subversion/trunk/README
Repository Root: https://svn.apache.org/repos/asf
Repository UUID: 13f79535-47bb-0310-9956-ffa450edef68
Revision: 1467597
Node Kind: file
Schedule: normal
Last Changed Author: danielsh
Last Changed Rev: 1242804
Last Changed Date: 2012-02-10 15:58:53 +0100 (Fri, 10 Feb 2012)
Text Last Updated: 2012-09-20 01:33:22 +0200 (Thu, 20 Sep 2012)
Checksum: a27c71319a591c4eebe2bb81129413947336a7c6

HTTP client support based on neon has been removed

HTTP client support based on neon has been removed in favor of HTTP client support based on serf.

serf is a high-performance HTTP client library which has formed the basis of an alternative HTTP repository access method for many years. In an effort to decrease the development burden of maintaining multiple HTTP client interfaces (and sets of bugs!) serf is now the only HTTP client provider for Subversion.

Note that server-side configuration changes might be required to avoid performance regressions for serf clients in some setups.

The MaxKeepAliveRequests option in httpd.conf needs to be increased from 100 (the default) to at least 1000 (there is no reason why it could not be 10000). This will improve performance by allowing serf clients to use fewer TCP connections to the server. Clients using neon will also work fine with this configuration.

Skelta style updates are now the default

The svn 1.8 client with serf defaults to skelta mode for update operations (checkout, update, merge and export) instead of the bulk update mode used by previous versions. Skelta mode was introduced in Subversion 1.6.0 and improved in 1.8.0. It uses one HTTP request and response per resource that needs to be fetched from the server, whereas bulk update mode fetches all resources in one massive reponse.

The main benefit of skelta mode is that it allows a correctly set up Apache server or intermediate proxy server to cache mod_dav_svn’s responses to handle later requests from the cache. This results in improved performance of svn client operations and reduced CPU usage on the server side. It also allows a more detailed audit of clients accessing resources in a Subversion repository.

Skelta mode has some disadvantages:

  • Apache server access log files will grow more quickly due to the larger number of requests and responses. As of 1.7.3, the httpd error logs may also grow more rapidly with serf clients than with neon clients; see r1239697.
  • Network traffic can increase drastically when Kerberos or NTLM authentication is used, as these add a 2 - 4 KB header per HTTP request and response.

This release introduces two options to control if the svn client will use skelta or bulk update mode.

  • For the server administrator: The SVNAllowBulkUpdates directive for mod_dav_svn now accepts Prefer. This will advise the svn client to always use bulk update mode. All svn client versions with a default configuration (see table) will respect this preference.
  • For the user: Set the new option http-bulk-updates in the servers runtime configuration file to yes to force the use of bulk updates, no to never use bulk updates. The default option auto makes svn use skelta mode with a 1.8 server (unless it has SVNAllowBulkUpdates set to Prefer), and bulk updates mode with older servers and 1.8 servers which prefer bulk updates.

Table explaining the mode used between each combination of svn client and server version and relevant configuration directives:

1.8 Server
with SVNAllowBulkUpdates:
1.7 and older Server
with SVNAllowBulkUpdates:
Subversion Client Off On* Prefer Off On*
1.8, http-bulk-updates: auto* Skelta mode Skelta mode Bulk mode Skelta mode Bulk mode
1.8, http-bulk-updates: yes Skelta mode Bulk mode Bulk mode Skelta mode Bulk mode
1.8, http-bulk-updates: no Skelta mode Skelta mode Skelta mode Skelta mode Skelta mode
1.7 and older with neon* Skelta mode Bulk mode Bulk mode Skelta mode Bulk mode
1.7 and older with serf Skelta mode Skelta mode Skelta mode Skelta mode Skelta mode

*Default configuration

The Berkeley DB-based repository back-end has been deprecated

The Subversion developers have decided to deprecate the repository back-end based on Berkeley DB. We simply do not have the necessary resources to continue developing two distinct repository back-ends. Instead, we will concentrate our efforts on improving FSFS with new features, robustness and performance and architectural enhancements.

What this means:

  • New repository features that may appear in future versions of Subversion will not be implemented for the BDB back-end, nor will we make any effort to improve its performance.
  • We will fix bugs and security issues that may be found.
  • At some point, support for the BDB back-end will be completely removed. We will announce such removal well in advance of its happening.

What this does not mean:

  • Users do not have to immediately migrate their repositories to FSFS. The BDB back-end will continue to work, and will receive as much test coverage as it has until now.

This being a volunteer project, we are of course always happy to accept contributions towards maintaining the Berkeley DB back-end.

User-visible changes

The only visible effect of the deprecation is that the svnadmin utility will print a warning when it creates a new Berkeley DB-based repository.

Miscellaneous Compatibility Notes

There are some additional specific areas where changes made in this release might necessitate further adjustment by administrators or users. We'll cover those in this section.

FS paths syntax in authz access rules

Prior to this release, the section headers in Subversion's authz access files—which contain repository names and repository filesystem paths—accepted section headers that would never be looked up, because the repository filesystem path (such as /trunk/secretdir) embedded in the section header is formatted differently from how Subversion formats those paths when it looks them up in the file. Subversion 1.7 and earlier would silently ignore those sections of the authz file; directives in those sections would never apply.

That's been fixed in this release: Subversion will now error out if a section header contains a repository filesystem path that either does not begin with a slash or contains two consecutive slashes. The practical fallout, though, is that your existing authz files might be depending (perhaps unintentionally) on the old behavior and the new behavior could result in all access to items in your repositories being unexpectedly denied as a result of upgrading to Subversion 1.8. The svnauthz tool, when linked to Subversion 1.8 libraries, can be used to test an authz file for validity using the validate subcommand. (The tool will error out on a file that the Subversion server will error out on.)

Repository listing now honors authz configuration

When Apache is configured with the SVNParentPath directive, the "Collection of Repositories" list will now be filtered based on read access to the root of each repository. Previously, all repositories were included in the list even if navigating to a repository would be forbidden. The "Collection of Repositories" will now be consistent with the directory lists within repositories.

NOTE: Access to "Collection of Repositories" is not restricted by mod_authz_svn, but is instead managed by mod_dav_svn itself. In order to require authentication on this location, the location should have "Satisfy All" (which is the default value of this directive). See examples in mod_authz_svn's INSTALL document for additional details.

FSFS 'svnadmin verify' finds mergeinfo-count and predecessor-count discrepancies

Up to 1.7.0, Subversion could create erroneous metadata on nodes in a FSFS-backed repository in certain situations involving concurrent commit attempts. (Only metadata was affected; file contents and properties was not.) As of 1.7.1 Subversion prevents new instances of the corruption, but only as of 1.8.0 does 'svnadmin verify' detect instances of that corruption in the history of a repository.

The fix to these issues is simple: perform a dump/load cycle. (As usual, svnsync can be used instead of dump/load.) The cycle can be done with any version of Subversion, and after the cycle the repository should be served exclusively by Subversion 1.7.5 or newer to prevent further instances of the bug from entering the repository.

See this summary of issue #4129 for more information about these problems.

Client prompting for SSL client certificates

Subversion has long supported the use of SSL client certificates for authentication against a server which accepts such. Users typically employ the ssl-client-cert-file option in their 'servers' runtime configuration file to inform the client regarding the location of the relevant certificate file. In interactive scenarios, Subversion can also prompt the user for the location of the certificate file when that runtime-configured value isn't set or otherwise suitable.

Prior to 1.8.0, this prompting was enabled by default. Unfortunately, not every server which accepts client certificates also requires them. Some users were being prompted for client certificate file locations in scenarios where no certificate was required (or perhaps even available). So in Subversion 1.8.0, the Subversion client defaults to not prompting for the location of an SSL client certificate file unless the user has set the new ssl-client-cert-file-prompt runtime configuration option (found in the [auth] section of the 'config' file) to "yes".

See issue #2410 for discussion and details.

Star-imports in the SWIG-Python bindings

The swig-py bindings have been changed to make *-imports of submodules to do the right thing: from svn.client import * will now import only symbols beginning with svn_ or SVN_. (Most of these symbols will be svn_client_* symbols, of course.) Star-imports of from svn have not changed (they import just the bare submodule names: 'fs', 'ra', etc), and star imports of from svn.core currently imports some select symbols (notably 'Pool' and 'SubversionException').

This change is incompatible: code that expected symbols such as 'commit_txn' (in 'fs') and 'apr_initialize' to be pulled by a star-import will have to change.

svnserve --config-file behavior with password and authz dbs

The behavior of the --config-file option to svnserve has changed. The password db and authz db files will be reloaded on each connection. In past versions these files were cached on startup when --config-file was used.

The svnserve.conf file directly passed to --config-file will still be cached. Provided that the locations you wish to use for the authz and password dbs have not changed, you will not need to restart svnserve in order to have the changes you make to these files applied. This makes the behavior of --config-file more consistent with configurations that do not use this option.

If you were depending on the configuration changes not being applied until you restarted svnserve you will need to adjust accordingly.

svnauthz-validate renamed to svnauthz

The svnauthz-validate command has been renamed to svnauthz and now has a 'validate' subcommand. Meaning the equivalent to svnauthz-validate file in 1.8 is svnauthz validate file. To maintain command line compatibility, if the svnauthz command is run with the command name of svnauthz-validate then it emulates the behavior of the svnauthz-validate command from 1.7. make install-tools installs a symlink svnauthz-validate to provide this compatibility functionality.

svnauthz also provides new functionality. See this section for details.

New Features

Working copy records moves as first-class operation

The effect of the svn move command is now different from running svn copy followed by svn delete. Moves are represented within the working copy as a copy and a delete which are linked to one another. These links are shown by svn status and svn info.

Some Subversion operations will now treat locally moved files and directories specially. Behavioural changes include:

  • svn move now refuses to move a mixed-revision working copy.

  • svn commit will only commit a moved file or directory if both sides of the move are part of the same commit. This ensures that moves are an atomic operation upon future updates and merges.

  • svn update, svn switch, and svn resolve can now resolve tree conflicts involving locally moved files or directories. By picking the 'mine-conflict' option at the conflict prompt, the update or switch operation can be applied to the new location of the moved file or directory, which resolves the tree conflict and allows the move to be committed. It is also possible to break a move instead of updating it, in which case the move becomes a copy which is no longer linked to a deletion.

Limitations:

  • Moves are recorded as such only within the working copy. The link between the copy and the delete is established only when a local move operation is performed, and is lost upon commit. The committed revision will show a copy and a delete, instead of a move.

Known issues:

  • Tree conflicts involving replacements are currently not detected when updating a moved file or directory (see issue #4318).

  • Tree conflicts flagged by svn merge cannot be automatically resolved yet. This will be addressed in a future release.

Automatic reintegration merge (--reintegrate option deprecated)

During merges which merge all eligible revisions from another branch, Subversion 1.8 will automatically decide whether or not the merge is reintegrating a branch. Therefore, reintegrating a branch does no longer require the --reintegrate option for correct operation.

The --reintegrate option of svn merge is now deprecated and its use is discouraged. To reintegrate a branch, have a clean working copy of trunk and run the following command in its top-level directory:

$ svn merge ^/branches/my-branch

This merge will still perform similar sanity checks which svn merge --reintegrate performed in earlier releases:

  • The working copy must not be a mixed-revision working copy.
  • The working copy must not have switched subtrees.
  • There must be no gaps in revision ranges merged from the reintegration target (e.g. the trunk) to the reintegration source (i.e. the branch to be reintegrated).

If any of these conditions are detected, the merge is aborted and the necessary steps must be taken to fix the problem before the branch can be reintegrated. In contrast to a --reintegrate merge, an automatic reintegration merge into a working copy with local modifications is allowed.

Merging to-and-fro between two branches in any order is possible using the automatic reintegration merge (the "keep-alive dance" is no longer necessary). For best results, it is recommended to always merge all eligible revisions, i.e. not using the -r or -c options of svn merge. Merging just a subset of eligible revisions increases the likelihood of problems during future merges.

Using --reintegrate in Subversion 1.8 will force a reintegration merge, whether or not that's the right merge to perform in the given situation.

Inherited Properties

Property inheritance provides a mechanism to find the versioned properties set on the parents of a given working copy or URL path. Conversely it can be viewed as a mechanism by which a versioned property set on a path also applies to that path's children.

It is important to note that this change does not introduce any such thing as an "inheritable" property. Any versioned property, explicitly set on a path, can be interpreted as inheritable by that path's children. No changes have been made to the ways in which properties are set or what valid property names and values are permitted. Nor does this change grant access to parent paths which a user doesn't have read access to. If a user has no read access to a parent path, he cannot inherit properties from that parent.

Other than the changes detailed below, the only user visible changes resulting from inheritable properties are to the svn proplist, svn propget, svnlook proplist, and svnlook propget subcommands when used with the new --show-inherited-props option.

This new option finds the explicit properties set on a given path's parent(s) lists them prior to the explicit properties on the target path itself. For example:

   > svn propget -vR --show-inherited-props tsvn:logwidthmarker ^/subversion/trunk/subversion/po
   Inherited properties on 'https://svn.apache.org/repos/asf/subversion/trunk/subversion/po',
   from 'https://svn.apache.org/repos/asf/subversion/trunk':
     tsvn:logwidthmarker
       78
   Properties on 'https://svn.apache.org/repos/asf/subversion/trunk/subversion/po':
     tsvn:logwidthmarker
       80

If a working copy is the target, some properties may be inherited from the working copy and some may be inherited from repository parent paths not present in the working copy. These are shown as inherited from a URL:

   > svn propget -vR --show-inherited-props tsvn:logwidthmarker po-wc\de.po
   Inherited properties on 'po-wc\de.po',
   from 'https://svn.apache.org/repos/asf/subversion/trunk':
     tsvn:logwidthmarker
       78
   Inherited properties on 'po-wc\de.po',
   from 'po-wc':
     tsvn:logwidthmarker
       80

The output with the --xml option also returns inherited properties, wrapping them in inherited_property rather than property tags:

   > svn propget --show-inherited-props --xml tsvn:logwidthmarker ^/subversion/trunk/subversion/po
   <?xml version="1.0" encoding="UTF-8"?>
   <properties>
   <target
      path="https://svn.apache.org/repos/asf/subversion/trunk">
   <inherited_property
      name="tsvn:logwidthmarker">78</inherited_property>
   </target>
   <target
      path="https://svn.apache.org/repos/asf/subversion/trunk/subversion/po">
   <property
      name="tsvn:logwidthmarker">80</property>
   </target>
   </properties>

The svnlook proplist and svnlook propget subcommands also support the new --show-inherited-props option. The output is similar to svn proplist --show-inherited-props and svn propget --show-inherited-props except that the paths are not working copy paths or URLs, but absolute repository paths:

   > svnlook propget asf-repos-mirror tsvn:logwidthmarker /subversion/trunk/subversion/po \
       --show-inherited-props -v
   Inherited properties on '/subversion/trunk/subversion/po',
   from '/subversion/trunk':
     tsvn:logwidthmarker
       78
   Properties on '/subversion/trunk/subversion/po':
     tsvn:logwidthmarker
       80

Subversion working copies maintain a cache of the properties that the root of the working copy inherits from its parent(s) in the repository. This cache is updated each time you checkout, update, or switch a working copy. This cache allows the working copy to access properties inherited from the repository without contacting the repository.

For the full details about inheritable properties see the design wiki. Some of this wiki is intended for Subversion developers and will be of little interest to most users. If you fall into the latter group you can focus on these particular sections:

Repository Dictated Configuration (For auto-props and ignores)

Two new Subversion reserved properties, svn:auto-props and svn:global-ignores make use of the new inherited properties feature to provide additional configuration information that overrides/extends some of the settings found in the user's runtime configuration. The svn:auto-props property overrides/extends the auto-props configuration setting. The svn:global-ignores property extends the global-ignores configuration setting as well as the svn:ignore property.

The format of svn:auto-props property values are the same as for the auto-props runtime configuration. The format of svn:global-ignores property values are the same as for the global-ignores runtime configuration.

Both properties work just like their analogs in the runtime configuration with two exceptions:

  • The new properties only effect the subtrees rooted at the path on which the property is set. Thus a given path may be affected by multiple svn:auto-props or svn:global-ignores properties set on different parents of that path.
  • The svn:auto-props property is not tied to the enable-auto-props runtime configuration option. So even if you have enable-auto-props=no set in your configuration or via the --config-option option, svn:auto-props are still applicable. Note however, that exactly like the runtime configuration and svn:ignore property, both the svn:auto-props and svn:global-ignores properties can be disregarded with the --no-ignore and --no-auto-props options respectively.

When multiple svn:global-ignores properties apply to a path, then the different values are appended to any runtime global-ignores in effect and the value of any svn:ignore property that applies to the path.

Patterns defined in svn:auto-props property override any identical patterns in the auto-props runtime config. When multiple svn:auto-props properties apply to a file, the pattern from the nearest inheritable property takes precedence. See this section of design wiki for a full explanation.

The design wiki for the repository dictated configuration feature was originally written for developers, but will prove useful to any repository administrator who wants to use the feature.

In-memory password caching via GnuPG Agent (Unix client)

Subversion 1.8 allows the use of the GnuPG Agent (gpg-agent) daemon to temporarily store Subversion passwords in memory.

This feature does not use PGP to encrypt passwords on disk! Rather, it caches passwords in memory (in plaintext) instead of saving them to disk.

To take advantage of this password cache, you'll need Subversion binaries built with gpg-agent support (which is the default on UNIX-like systems), the gpg-agent itself, and a suitable pinentry program which the gpg-agent will use to ask the user for the password. The gpg-agent must be running, and the Subversion client will need the GPG_AGENT_INFO and GPG_TTY environment variables set up correctly. See this page for more information about running the gpg-agent.

Cached passwords will persist in memory until the agent process is terminated, its configured time-to-live threshold is reached, or a HUP signal is sent to the daemon using the UNIX kill(1) utility.

Communication to the gpg-agent happens over a UNIX socket, which is located in a directory which only the user running Subversion can access. However, any program the user runs could access this socket and get the Subversion password if the program knows the "cache ID" Subversion uses for the password.

The cache ID is very easy to obtain for programs running as the same user. Subversion uses the MD5 of the realmstring as cache ID, and these checksums are also used as filenames within ~/.subversion/auth/svn.simple. Unlike with GNOME Keyring or KDE Wallet, the user is not prompted for permission if another program attempts to access the password.

WARNING: Therefore, while the gpg-agent is running and has the password cached, the password cache is no more secure than a file storing the password in plaintext.

FSFS size and performance enhancements

Subversion 1.8 introduces a number of improvements that can reduce the size of FSFS repositories, the total number of files on disk, the I/O overhead and gives the user fine-grained control over all of that. Some of these changes require a format bump, others are backward-compatible.

FSFS format bump

The default repository format created by svnadmin create is now FSFS version 6 which is not accessible by Subversion 1.7 or older. Older repository formats remain fully supported by Subversion 1.8 but will not support revprop packing. To create FSFS repositories compatible with Subversion 1.6 and 1.7, use the --compatible-version 1.6 parameter.

You may use svnadmin upgrade to upgrade existing repositories. However, to fully benefit from the latest repository size reductions, it is recommended to create a new repository, adjust its settings and then dump/load or svnsync the contents into it.

Packing revision properties

Format 6 repositories will not only pack the revision files but also the revision property files. Upgrading existing packed repositories will automatically pack existing revision properties. Using default settings, this will reduce the number of revprop files to less than 10 per 1000 revisions in typical usage scenarios.

The db/fsfs.conf file allows you to fine-tune the revprop packing strategy. See the comments in that file for a detailed description of the available options. Please note that no change in this configuration file will affect any existing data. Moreover, upgrading repositories will not extend the existing configuration file, i.e. the server will use the default settings unless the new options are added explicitly.

It is recommended to also activate revprop caching when you use revprop packing. Otherwise, access to timestamps etc. will suffer significant processing overhead.

Caching revision properties

Many operations will access revision properties to e.g. retrieve the commit time stamp. Therefore, thousands of revision property files may need to be read during a checkout. The UI to enable the revprop cache is similar to that for the other caches: svnserve now accepts the --cache-revprops yes parameter. For mod_dav_svn, the same looks like this in httpd.conf

<IfModule dav_svn_module>
    SVNCacheRevProps on
</IfModule>

With revision property packing enabled, revprop caching allows you to read hundreds of revprops at once from a single file. This can give a significant performance boost to e.g. svn log .

Directory and property storage reduction

For each changed node in a FSFS repository, new versions of all parent directories will be created. Larger repositories tend to have relatively deep directory structures with at least one level (branches, modules or projects) having tens of entries. The total size of that data may well exceed the size of the actual change. Subversion 1.8 now supports directory deltification which eliminates most of that overhead.

In db/fsfs.conf, you may now enable and disable directory deltification at any time and these settings will be applied in new revisions. For completeness, node properties may now be deltified as well although the reductions in repository size will usually be minimal.

By default, directory and property deltification are disabled. You must edit db/fsfs.conf to enable these features. The reason is that deltification may amplify I/O in certain situations. To minimize that effect, enable the txdelta cache.

Also, db/fsfs.conf now allows for fine-grained control over how deltification will be applied. See the comments in that file for a detailed description of the individual options.

Another optimization in 1.8 is that node properties with the same contents will only be stored once and then merely referenced. This not only slightly reduces the size of the repository but greatly improves cache effectiveness and may reduce I/O significantly.

It's backward compatible! Please note that the mechanism to read deltified data is the same for file contents as for directories and properties. Hence, Subversion 1.6 and 1.7 will be able to read them but will always write new revisions without that optimization. It is therefore perfectly legal to create a pre-1.8-compatible repository in 1.8, to enable various deltification options, to dump/load into that new repository using 1.8 tooling and to finally use the repository with a 1.6 or 1.7 server.

Rep-sharing within revisions

When representation sharing has been enabled, Subversion 1.8 will now be able to detect files and properties with identical contents within the same revision and only store them once. This is a common situation when you for instance import a non-incremental dump file or when users apply the same change to multiple branches in a single commit.

In repository authz

Subversion 1.8 allows authz files to be stored inside a Subversion repository. This allows you to employ the versioning features of Subversion for the configuration of the path-based authorization feature. You need not store the authz file in the same repository as the one to which its rules are being applied. However, the server which uses the authz file does require filesystem access to the repository in which that file is stored. Administrators should consider that one benefit of having the authz file stored in the same repository as the one to which its rules are being applied allows the authz file to be replicated by svnsync along with the rest of the data in the repository, simplifying administration of mirrored repositories.

When specifying the location of the authz file to Apache HTTP Server or svnserve, there are now four formats an administrator may use:

  1. Absolute path to a file (outside of a repository): /path/to/file or C:\path\to\file
  2. Relative path to a file (outside of a repository): path/to/file or path\to\file
  3. Absolute URL to file in repository: file:///path/to/repo/file
  4. Relative URL to file in a repository: ^/file

The first two formats are those that were already supported in versions prior to 1.8; the latter two are the new formats.

The new absolute URL format is similar to what you might use with svn cat to list the contents of a file versioned in a local repository. (Note that at this time, support exists for file:// URLs only, not for other Subversion URL flavors such as http://, svn://, and so on.)

The relative URL syntax should also look familiar, as it mimics the relative URL syntax that the command-line client recognizes. When parsing path specifications in this format, Subversion simply ignores the leading ^/ and looks for authz file at the remaining path in the repository which is being accessed.

Apache HTTP Server accepts all four formats for both the AuthzSVNAccessFile and AuthzSVNReposRelativeAccessFile configuration directives. The only difference between these two directives is the root path for the "relative path to a file outside a repository" format.

WARNING: Unlike authz files stored on the server's local disk, authz files stored in the repository are accessible via Subversion clients just like any other file in the repository. If you wish to protect the contents of the authz file you should configure appropriate access restrictions for it in the applicable authz file (which could potentially be the same file!).

WARNING: As with other versioned files, Subversion servers do not validate the internal syntax of a versioned authz file in any way. Administrators may wish to set up a pre-commit hook script to validate that the authz file is well-formed and/or the committing user has not removed all permissions to edit the file. If permissions have been removed to edit it via the network server(s) you can of course always edit it via a local (file://) checkout since ra_local does not observe path based permissions. See the validate-files.py hook script and its related configuration files in Subversion's tools/hook-scripts for examples of how to validate a versioned authz file's syntax and specific permissions.

New tools and utilities

svn-bench benchmarking client

Identifying bottlenecks in your Subversion infrastructure may be difficult because client-side influences (virus scanners, slow disks etc.) tend to mask other limitations. Therefore, Subversion 1.8 introduces a very lightweight client that skips most of that local processing.

svn-bench provides commands similar to the standard command line interface, adding a null- prefix to them. Currently implemented are null-export, null-list and null-log. They support most of the parameters of their standard command counterparts.

$ time svn-bench null-export svn://localhost/tsvn_repos/trunk
            179 directories
          3,661 files
     84,902,674 bytes in files
         20,560 properties
        315,031 bytes in properties

real  0m0.185s
user  0m0.128s
sys   0m0.040s

By running the client close to or directly on the server machine, you will be able to determine the raw server speed and compare that to what the client-side is seeing.

fsfs-stats repository statistics

This server-side tool will read a whole format 4 (Subversion 1.6) or newer repository and print out various statistics on the contents of the rev and pack files in it. It starts of with very general information and then breaks it down into various sub-demographics. The statistics include:

  • Total repository size bytes
  • Number of revisions and changes
  • Sum of original and deltified file sizes
  • Sum of original and deltified directory sizes
  • Structural overhead due to node revisions, change lists etc.
  • Effectiveness of representation sharing
  • Paths and revisions of the 64 largest single changes
  • By file extension: Number of changes, total file size and total internal representation size
  • Histograms on the above

Administrators may use this information to decide whether directory deltification should be enabled (ratio of directory representation vs. file representation sizes) or whether commit policies might to be revised (e.g. large .zip files accout for most of the repository). Some of the data, however, is useful for performance analysis by SVN developers only.

$ fsfs-stats /repositories/wesnoth_org 500 | gedit &

fsfs-stats takes a local path to the repository root and an optional cache size parameter. The latter is in MBytes and speeds up the analysis of large repositories. Because the tool's output may be up to a 1000 lines of text, you may want to redirect it to some file or application.

Note that the 32 bit version of this tool may fail on repositories with millions of changes or with revision / pack files larger than 4 GB. Use a 64 bit executable instead for those repositories.

Enhancements and Bugfixes

Improved conflict resolution

Conflicts are now always postponed during updates and merges

The svn update, svn switch, and svn merge commands line client now always postpone conflicts. Once the operation has completed, the interactive conflict resolver is automatically run on paths which received conflicts during the operation, which will either run in interactive mode or perform conflict resolution based on the value of the --accept option, if given.

This approach has the advantage that the connection to the server is not held open while interactive conflict resolution is performed. In Subversion 1.7, the connection could time out if the user spent too much time resolving a conflict (see issue #3846).

'svn resolve' can resolve conflicts interactively

The svn resolve command does not require the --accept option anymore. If invoked without any arguments, svn resolve will now perform interactive conflict resolution for any conflicted paths within the current working directory.

Built-in file merge tool for interactive conflict resolution

The command-line client now has a built-in file merge tool which merges any non-conflicting changes automatically and asks the user what to do about each individual merge conflict.

The built-in file merge tool can be invoked from the interactive conflict resolution prompt by choosing the (m) merge option. It displays conflicting file sections in a side-by-side view and offers various ways of resolving the conflict, as shown below:

$ svn update
Updating '.':
C    subversion/svn/file-merge.c
Updated to revision 1358165.
Summary of conflicts:
  Text conflicts: 1
Conflict discovered in file 'subversion/svn/file-merge.c'.
Select: (p) postpone, (df) diff-full, (e) edit, (m) merge,
        (mc) mine-conflict, (tc) theirs-conflict,
        (s) show all options: m
Merging 'subversion/svn/file-merge.c'.
Conflicting section found during merge.
(1) their version (at line 298)       |(2) your version (at line 391)        
--------------------------------------+--------------------------------------
  if (buf->len >= 2 &&                |  if (buf->len > 1)                   
      buf->data[buf->len - 2] == '\r' |    {                                 
      buf->data[buf->len - 1] == '\n')|      if (buf->data[buf->len - 2] == '
    svn_stringbuf_chop(buf, 2);       |        svn_stringbuf_chop(buf, 2);   
  else if (buf->len >= 1 &&           |    }                                 
           (buf->data[buf->len - 1] ==|  else if (buf->len > 0)              
            buf->data[buf->len - 1] ==|    {                                 
    svn_stringbuf_chop(buf, 1);       |      if (buf->data[buf->len - 1] == '
                                      |        svn_stringbuf_chop(buf, 1);   
                                      |    }                                 
--------------------------------------+--------------------------------------
Select: (1) use their version, (2) use your version,
        (e1) edit their version and use the result,
        (e2) edit your version and use the result,
        (eb) edit both versions and use the result,
        (p) postpone this conflicting section leaving conflict markers,
        (a) abort file merge and return to main menu: 

Checkout and update download pristine file data just once

Subversion keeps a cache of pristine copies of files in the working copy's meta data area inside the .svn directory. Before Subversion 1.8, pristine data was always downloaded from the server even if the same data already existed in the cache.

Subversion 1.8 avoids downloading pristine content that is already present in the cache, based on the content's SHA1 or MD5 checksum. This results in more efficient use of network throughput in cases where a lot of files in the working copy share the same content (for example, if a single working copy contains multiple branches).

Administration command-line tool