The Old Joel on Software Forum: Part 3 (of 5) - Strategies for managing lib files in src control

Strategies for managing lib files in src control

At every software company I've worked at, there's been a standard directory structure in the source control system
Product
Version
src //code goes here
lib //jars go here
foo.jar
bin

This setup makes writing build scripts or configuring IDEs really easy: you just set the classpath to "everything in the lib directory". However, there's some problems with this approach.

1)The same jar is on likely in the source control system (and therefore lots of peoples hard drives) in multiple places: this is inefficient.

2)There's no easy way to get metadata about the jar: where product/product version it came from, what version it is, etc. The manifest file in each jar is supposed to have that information: in practice not all of them do, and some information that you need isn't ever there.

3)Controlling which jars are being used in development (or shipping with the product!) is difficult, since the jar location isn't centralized.

What alternative approachs or strategies have people tried to manage jars in a source control system? How did it work out? What I'm imagining is some kind of centralized "jar repository", that holds all the jars used by developers in a company.

j b
Saturday, November 1, 2003

This structure is usually used in SourceSafe repositories, and is mostly a result of Source Safe's data model.

In most version control systems, it is much easier to work with a _branch_ for each version, rather than a directory for each version. Also, external sources can be shared and tracked independently of the main tree.

CVS lets you do "vendor imports" and "vendor branches" for this purpose. I don't think it's intuitive to anyone, but it does work (Read the CVS book affectionally known as "the cederqvist" for detailed instructions).

If you guys are using VSS, I would suggest considering a different version control system. And if you're already using a version control system with proper support for external tracking and branching, use it.

Ori Berger
Saturday, November 1, 2003

1) Do you mean people should access it over a shared
file system?

2) This information can be in the submit comment so it
is accessible via the revision history?

3) From your diagram it looks like all your jars are under lib.

We centralize all our jars just because it is easier. We
have some 3rd party jars and our jars get built in the
lib directory.

We also have a classes directory for the classes that are
created.

son of parnas
Saturday, November 1, 2003

re:shared file system. No, I'm thinking of keeping it in source control, but centralizing libraries.

Something like this
all_jars
foo
foo_1.1
foo.jar
foo_1.2
foo.jar

but then you have to deal with 3rd party jars that came with products your software runs on top of. In that case the best thing I can come up with is doing something like this...

all_jars
base_platforms
jakarta-tomcat-4.1.24
servlet.jar

j b
Saturday, November 1, 2003

Something we did once:

Repository had a "thirdparty" project, under which were all our third-party libraries. Our build script would, as one of its first actions, copy the third party libraries into our bin directory.

The bin directory is NOT in source control; it's the location that all our projects built into.

Chris Tavares
Sunday, November 2, 2003

In my company we had three different kinds of project
trees:

1. The project tree held in the source control system. This
project tree held 3rd party files in zipped format (ready
to be unzipped if necessary).

2. The project tree that was used as a development tree
for each developer. In order to set this tree we used
to:
2.1 GET all project files from the source control system.
2.2 run a special 'mkdevtree' script that unzipped any
3rd party files etc. and extracted necessary JAR
files into their /lib directories in the development
tree. It also created development-only directories
and files (e.g., /classes, /lib). The script also set any
necessary environment variables and registry keys.

3. The deployment tree. This tree was created by the
installer software for the final product. It usually
contained a small subset of the directory hierarchy
that was used for the development tree. It also
contained some files that were missing from our
development tree (e.g., un-install information and help
files).

This separation of concerns contributed to a clear
project tree in the source control system that was
uncluttered by a myriad of 3rd party files (e.g., javadocs)
because it only contained the zipped files and the work
of preparing them for development work was delegated
to a specialized script.

Liron Levy
Sunday, November 2, 2003