Skip to main content
Enterprise Agile Planning Image

This post is from the CollabNet VersionOne blog and has not been updated since the original publish date.

Last Updated Mar 10, 2009 — Enterprise Agile Planning expert

Packing FSFS Repositories

Enterprise Agile Planning

Subversion 1.5 introduced that idea of sharding for FSFS-backed repositories. For every commit to a FSFS repository, Subversion creates a single file which describes all the changes in that revision. Prior to 1.5, all of those files were stored in a single directory, which had several drawbacks: incremental backups took a long time, the repository could not be dymnically grown across different filesystems, and some filesystems have degraded performance when the number of directory entries grows too large. With sharding, these revision files were split into separate subdirectories, eliminating a large number of these problems.

Even with sharding, the filesystem still has some inefficiencies. For instance, due to the block size of the underlying filesystem, having many files can still lead to wasted space on disk, especially with many small commits. Subversion can open and read data from many revisions over the course of an operation, and using a large number of files means that Subversion can not exploit various operating system-level caches. Backing up and restoring a repository, although quicker, can still take a long time because of the large number of files spread across the repository.

One of the great ideas that came out of the 2008 Subversion Developers’ Summit was the notion that FSFS filesystems could be packed, that is, all the files in a completed shard could be glued together to create a single monster revision file.  This pack file would save space on disk, give the operating system a chance to do some caching, and generally improve the snappiness of the system.

In order to use FSFS packing, you simply need to ensure that target repository has been upgraded to the latest format, and then pack the repository using svnadmin.  Note that repositories do not automatically pack themselves, so for heavily used repositories, you may want to install a cron job or post commit hook to do the packing.  Users can continue to use the repository while it is being packed:

$ svnadmin upgrade repo
Repository lock acquired.
Please wait; upgrading the repository may take some time...
Upgrade completed.
$ svnadmin pack repo
Packing shard 0...done.
Packing shard 1...done.
Packing shard 2...done.
Packing shard 3...done.
Packing shard 4...done.
...
Packing shard 36...done.
$

To give an idea of the potential space savings, on my local 1.5-era copy of Subversion’s own repository I get the following results:

$ du -sh svnrepo-1.5/
659M	svnrepo-1.5/
$

While on a packed 1.6 copy of the same repository, with rep-sharing enabled, I see the following:

$ du -sh svnrepo-1.6/
593M	svnrepo-1.6/
$

That’s more than a 10% decrease in space, at no cost in performance. These space savings will vary depending upon your own repository and use habits, but we’re excited about the improvements in the FSFS backend in Subversion 1.6.

More from the Blog

View more
Apr 08, 2021

Making IT services more agile

Enterprise Agile Planning
The agile revolution completely transformed how we create digital prod ...
Read More
Feb 14, 2021

Reflecting on the 20th anniversary of the Agile Manifesto

Enterprise Agile Planning
Over the past 20 years, it’s been amazing to watch an idea from ...
Read More
Feb 08, 2021

How does agile apply to an entire organization?

Enterprise Agile Planning
Before we dive into the main subject of this blog post, it is importan ...
Read More
Feb 03, 2021

It took a pandemic to realize why digital transformation actually matters

Enterprise Agile Planning
Before anyone had ever heard of COVID-19, businesses across the globe ...
Read More
Contact Us