1. MERGE 2013THE PERFORCE CONFERENCESAN FRANCISCO APRIL 24 26
Perforce White Paper
To provide a solid foundation for software development
excellence in today’s demanding economy, it’s critical
to have a software version management solution that
can meet your demands.
Extracting depot paths into new
instances of their own
Mark Warren

2. 2 DEPOT SPLITTING
INTRODUCTION
As instances are used over time they naturally grow in file and metadata size. New files are
submitted and metadata increases in size and the instance become unwieldy. At some point
normal operations take long enough table locks that affect all users. To help mitigate this we
can increase hardware performance but there is a hard limit to what hardware you can replace.
A more practical method to release the building metadata pressure to move select datasets to
their own instance/depot.
Perfsplit 1
is a tooldeveloped by perforce that extracts a section of a depot from an existing
Perforce database. It will access a Perforce Server's database directly, and is platform and
release specific. Perfsplit does not remove data from the source Perforce server but does copy
archive files from it. Perfsplit is a good tool for this operation but does not resolve some
problems;
The need for zero downtime. Most instances that are in need of splitting have a very
large user-base. The need to keep instances up and running are compounded by the
number of users unable to access their instance.
Perfsplit does not rename the new instance depot. This is undesirable since it can be
confusing to users having the same depot name across multiple instances.
The need to use „p4 snap‟ to copy lazy integrated files to their physical lbr location.p4
snapcan considerably increase the size of the original depot depending on the size
area we are splitting off.
This document is intended to give guidelines on a method to resolve all these issues.
PREPARATION
To make sure we gather a complete dataset for migration from a live instance, it‟s necessary to
prevent users from making changes to the path(s) we are splitting. With super access rights,
this can be done by simply restricting ReadWrite access to this path and only allowing
ReadOnly. This is to ensure the metadata structure we are splitting off will be up-to-date. Once
this is done we need to create a checkpoint of the instanceto gather lbr records and we need
to have a running instance of this checkpoint for perfsplit use.
Despite the inadequacies perfsplit has, this process makes use of it;Perfsplit is necessary to
build the foundation of the new instance. The key function of perfsplit is using a map file to
direct it to the select path(s) to extract. Since we are splitting not only the initial path(s) but also
the integration history we will need to append this dataset to the splitmap. To get this, we need
to grep from the newly created original instances checkpoint the lbrFilerecord defined in
db.rev2
of all files associated with the depot path we are splitting.
For example
grep @db.rev@ /checkpoint.XXX | grep //targeted/path/to/split/
1
http://ftp.perforce.com/perforce/tools/perfsplit/perfsplit.html
2
http://www.perforce.com/perforce/doc.current/schema/#db.rev

3. 3 DEPOT SPLITTING
This will give you the db.rev entries for the path you want to split. From these entries you pull
the lbrfile column and remove all entries referring to the original path. This will give you the
location of all lazy integrated files.
We will add these paths to the splitmap(mapping) files already containing the path(s) we are
splitting from the original depot. This is a necessary step because we are not making use of
the p4 snap feature of perfsplit.
TRANSITION
Once we have thismapping we can begin our split using perfsplit using the minimum options,
source, output, splitmap file, but we also need an additional (undocumented) option „–a‟ to skip
the perfsplit archive file copy step. This will build a duplicate instance of the original metadata
for all depots associated with the originals split path in the output path. Since we don‟t want
two instances with depots of the same name, we need to take another checkpoint of this new
instance.
CONVERSION
With this new checkpoint, we can shape the metadata into a new data structures.To do this we
build another instance from the newly created checkpoint, but during creation (replay) we
make some substitutions to point the current data structure to what we want.
For example, to do this we would use the following commands:
p4d -r $p4root -f -jr – | sed –e s#//foo/path/#//bar/path/#
Now we have a new instance with the correct metadata.
CONNECTION
The conversion has now pointed the original metadata to a new depot area. We will need to
create this new depot „bar‟ to access this area. The files for this depot need to be copied from
the original location or if space is a factor it can be left in the original locationor moved to a new
one and the symlinked(depending on an installations particular circumstance).
Once this is done, you will have a new instance with a different name containing a complete
data structure of split files.
VERIFICATION
Verification of the new instance should be run to test the success of the transfer. There are
only two problems that can occur.
Verification returns a "BAD" error. This is reported when the MD5 digest of a file
revision stored in the Perforce database differs from the one calculated by the p4 verify
command. This error indicates that the file revision might be corrupted. This is most
likely due to changes to the physical files during transfer. Otherwise files should be
confirmed by someone familiar with them or by diffing them from the original.3
3
http://kb.perforce.com/article/961

4. 4 DEPOT SPLITTING
Verification returns a “MISSING” error. This indicates that Perforce cannot find the
specified revisions within the versioned file tree. Most likely this means the archive file
is missing from the versioned file tree. Check the lbrfile record of this file and make
sure this file is in it correct location, make sure the new instance can access this
location, and make sure this files location was part of the splitmap4
.
CLEANUP
You will notice perfsplit carried over all the depots from the original instance regardless if they
were part of the splitmap or not. These can be removed except for any depot containing data
from integrated files located on your original split path. These extra depots can be easily
hidden by restricting their view in the protection table making the new instance looks like it only
has the single depot.
COMPLETION
By implementing these steps using the perforce tool perfsplit, the issues regarding the zero
downtime, duplicate naming, and integration history are addressed. Resolution of these issues
makes Perfsplit a more desirable tool in a large installation environment ….
4
http://kb.perforce.com/article/693