| by Andrew Piskorski | Last Revised: 18 Aug 2001 |
This is intended to be a policy document and cheatsheet for doing version control and release management on a website, principally via use of CVS. While it contains extensive "howto" information, it is not intended to be either a reference manual or general tutorial on CVS - read the above articles first.
We will also try to make this document generally applicable, by placing any information highly specific to our own project in a separate companion document.
For web development, project version and configuration control falls roughtly into three main bins: CVS, Database Upgrades, and Other. This document originally dealt only with the CVS piece of the puzzle (hence the filename), but will now touch on all three.
If you've read Ron's article, above, you know there are three basic ways to use CVS for Web Development version control: One Branch, Two Branches, and Many Branches.
So the first question is which method of using CVS do we use? My past experience says to use either One Branch or Many Branches. For simple projects, One Branch will often be simplest, fastest, and best. For some large, complicated, or long-term projects - particularly after they already have a Production server launched and in regular use - the Many Branches method may be useful.
As for the the Two Branches method, I don't recomend it. While I have not run a project using it, it strikes me as just as much work as Many Branches, without all the benefits.
Also, if you do use the Many Branches you'll probably want to avoid mixing in the Two Branches method as well. I have mixed the two, and it can work fine, but the extra complexity usually isn't worth dealing with. Keep things simple if you can.
Note that nothing about CVS locks you into any of the above methods. We can and will use the "other methods" should circumstances warrant it.
Any and all changes not
handled via CVS go in the
upgrade.sql
file. This is usually database changes but may be anything at all.
In other words, if making the change happen involves anything
other than just doing a cvs update, it should probably be in the
upgrade.sql file.
Admitedly, shoving things like "Go to /acs-admin/apm/ and use the web
UI, install ACS package X, mount it on URL Y, and also do Z, because I
was to lazy to figure out how to have a script set that up
automatically" into upgrade.sql is something of a kludge.
But at least that way all needed non-cvs changes are noted in one
place.
You will probably see a natural change in your use of the
upgrade.sql file as your site moves from developing
something brand new, to launching and then maintaining a production
site. When you start a brand new project and do not have any
Production site launched, whenever you make changes to your data model
you will want to simply drop and re-create your package, so you won't
have much genuine data model upgrade stuff in there. Later on,
necessity may force you to make all data model changes, even
on Dev, via this upgrade script, as that's what you'll need to do on
Production.
And in fact, I first came up with this "put all upgrades into
upgrade.sql" scheme in exactly that situation (the
Muniversal project, which I worked on for roughly 10 months, roughly 8
months of which were after we'd launched a Production site).
Whatever you do, don't make changes to your data model one way on Dev, and then expect to come up with a whole different way of implementing those changes on your existing Production site! That's really asking for trouble.
In general, when making all data model changes as alterations to the
live database via upgrade.sql on Muniversal, I found that
it was not unusual to run into some problems when upgrading Staging,
but most of the time the upgrade to Production went quite smoothly.
In other words, one pass of testing was usually enough to work the
bugs out.
(Note: If there exists a tool that can version and diff live relational data models as well as CVS can version and diff text files, all without losing any data in the tables, I'd like to hear about it!)
We use one or two UNIX machines, three AOLservers, and three database users. All development takes place on the Development server. Developers check their changes into CVS continually. The development machine also hosts the staging server, with its own AOLserver and Oracle user. The staging server has two uses: (1) as a testing area for new code, and (2) as a testbed for updating the production server.
When a developer has a stable version of a file, he does:
$ cvs-tag-staging.sh myfile.tcl
and the file is tagged as ready for the staging server. (Note: this tags the last committed version, but doesn't commit your current changes.) When any developer does:
$ cvs-up-staging.sh
all files tagged for Staging are moved to the Staging server. (The
cvs-tag-staging.sh and cvs-up-staging.sh scripts
are located in "/web/mysite/bin/".)
You can also use the cvs-not-tag.tcl script to tell you which files have not been tagged for Staging.
With this process, tagged code is moved continually from dev to
staging. Any non-CVS changes that need to go to Staging along with
the code will be done and noted in the upgrade.sql file
by the person moving the code to Staging.
Watchdog is installed on the staging server, so any errors are sent via email to the development team.
When it is time to do a Production release, we tag all the files on Staging with a Production release tag, and then update Production.
So the process of releasing changes from Staging to Production is very similar to the process for releasing from Dev to Staging, but it differes in these important ways:
TODO: [Basically, there's not that much to it, as you use only CVS commands excluseively - I never found the need for any anciallary shell scripts as I did witht the One Branch method. See the articles and books listed above.]
TODO:
[Interpreted and Database-backed (argues for share CVS also makes
the stuff you just imported onto the vendor branch rev. 1.1.1 on the
trunk. But if you already have a revision 1.1.23 of the
foo.sql file, say, then importing a revised foo.sql onto
the vendor branch doesn't do anything to the
foo.sql on the trunk - you have to merge and commit
first.
So to import code that needs to be merged in with existing code, the steps are:
-kk.
Note that:
Here are some examples:
Upgrading from ACS 4.2beta to 4.2:
$ cvs import -b 1.1.1 -m "Importing ACS 4.2 (not beta)." mysite ArsDigita acs-4-2-R20010417
$ cd ~ ; cvs checkout -kk -d mysite-tmp mysite
$ cd mysite-tmp
$ cvs up -kk -j acs-4-2-R20010417
$ cvs commit
$ cd /web/mysite-dev/
$ cvs -q up
Importing Extop's (Joe Bank's <jbank@arsdigita.com>) enhanced ACS Developer Support package:
$ cvs import -b 1.1.3 mysite/packages/acs-developer-support Extop Extop-production-update-20010518
$ cd ~
$ cvs checkout -kk -d mysite-tmp mysite
$ cd mysite-tmp/packages/acs-developer-support
$ cvs up -kk -d -j Extop-production-update-20010518
$ cvs commit -m "Merged in changes from Extop-production-update-20010518"
$ cd /web/mysite-dev/packages/acs-developer-support
$ cvs up -d
Importing ACS-Notification tweaks from Proteome:
$ cvs import -b 1.1.5 mysite/packages/acs-notification Proteome proteome-final
$ cvs checkout -kk -d mysite-tmp mysite
$ cd mysite-tmp/packages/acs-notification
$ cvs up -kk -d -j proteome-final
$ cvs commit
$ cd /web/mysite-dev/packages/acs-notification
$ cvs up -d
Importing AOLserver nsodbc driver v. 1:
$ cvs import -b 1.1.11 aol3-src/aolserver/nsodbc AOLserver nsodbc_v1
$ cd ~/tmp
$ cvs checkout -kk aol3-src
$ cd aol3-src/aolserver/nsodbc/
$ cvs up -kk -d -j nsodbc_v1
$ cvs commit -m "Merged in nsodbc_v1 changes from www.aolserver.com."
$ cd /web/aol3-src/aolserver/nsodbc/
$ cvs up -d
Occasionally you may get just one or two extensively changed files
from someone. Here's such an example. Below, I start with onle the
single new version of the table-display-procs.tcl file in
the current directory. Then:
$ cvs import -b 1.1.5 mysite/packages/acs-tcl/tcl Proteome proteome-final
$ cd ~ ; cvs checkout -kk -d mysite-tmp mysite
$ cd mysite-tmp/packages/acs-tcl/tcl
$ cvs up -kk -d -j proteome-final table-display-procs.tcl
$ cvs commit table-display-procs.tcl
$ cd /web/mysite-dev/packages/acs-tcl/ ; cvs up table-display-procs.tcl
Remember that in CVS, branching is really just a special kind of tagging. The (only?) difference, is that a "branch tag" is placed against a special revision number of the file - see the Cederqvist manual about this
root-of tag on the trunk.
branch tag.
Steps 2 - 4 are the meat of the braching work. We give examples of them here:
Let's place the two tags. Since we're always taging the trunk with a
root-of tag, we should always use this root-of tag to
tell CVS where to put the branch tag. E.g.:
$ cd /web/myproject-dev
$ cvs tag root-of_release-2000-08-30
$ cvs tag -r root-of_release-2000-08-30 -b release-2000-08-30_branch
If you want to use rtag instead of tag, it's almost exactly the same, except that you also have to specify the module or repository root, in this case 'cvs-module':
$ cd /web/myproject-dev
$ cvs rtag root-of_release-2000-08-30 cvs-module
$ cvs rtag -r root-of_release-2000-08-30 -b release-2000-08-30_branch cvs-module
Now, checkout the new branch onto Staging:
$ cd /web/myproject-staging
$ cvs update -kk -d -r release-2000-08-30_branch
Finally, when you're ready to upgrade Production to the new Staging branch, you use the exact same update command:
$ cd /web/myproject
$ cvs update -kk -d -r release-2000-08-30_branch
fix-n tag.
This identifies your batch of fixes.
cvs update or cvs checkout to
merge the files from Staging to Dev.
-j (join) switch with a tag name to actually
make the merge happen. When you merge the first set of fixes
on a Staging branch back to Dev, you only need one -j tag, to tell CVS
what to join into the current working directory. E.g.:
$ cd /web/myproject-dev
$ cvs up -R -kk -j release-2000-08-08_fix-1
When you are merging your second set of fixes, you need to
use two -j tags, to tell CVS "merge changes from between
these two tags". If you don't, CVS will try to merge the
fix-1 changes in again, causing lots of bogus
conflicts. So for the second and subsequent batches of fixes, do
e.g.:
$ cd /web/myproject-dev
$ cvs up -R -kk -j release-2000-08-08_fix-1 -j release-2000-08-08_fix-2
-n switch in a command like:
$ cd /web/myproject-dev
$ cvs -nq up -R -kk -j release-2000-08-08_fix-1 -j release-2000-08-08_fix-2
does not show you everything that the command would do for
real without the -n switch to stop it, you can't get a
list of the files that will have conflicts prior to actually doing it.
It might be nice to have a script which uses 'cvs status -v' to generate a list of all files which DON'T have the same revision number on two tags.
Now, if you don't mind not knowing what files are going to be affected
by the merge, you can simply go ahead and use the above command
without the -nq, resolve all conflicts, and commit.
But, if you're not sure, you probably don't want to merge directly from Staging back to Dev like the above. Instead, checkout a new working copy of the Dev trunk and do the merge there. (This means that you will be merging the fixes from Staging back into the version in the repository on Dev, not the - possibly modified - files in the Dev working copy.)
You can do the checkout of a new Dev working copy and the merge from Staging all in one step. E.g.:
or:$ cd ~/
$ cvs -d ls.arsdigita.com:/cvsweb co -kk -j root-of_release-2000-08-30 -j release-2000-08-30_fix-1 cvs-module > cvs-co.log(Note that the first
-jtag is probably unnecesary.)
$ cd ~/
$ cvs -d ls.arsdigita.com:/cvsweb co -kk -j release-2000-08-08_fix-1 -j release-2000-08-08_fix-2 cvs-module
Then, fix any conflicts, commit as usual when you're done, and delete
the extra working copy.
[It might be nice to have a script to conveniently parse out all the conflict messages from the output of the above merge command, to give a concise list of the conflicts that need to be addressed. But in practice, I just do 'cvs -nq up' to find the files with conflicts, write down the list, and then resolve them one at a time.]
Finally, remember, your fixes are now all in the CVS repository for the Dev trunk now, but you still need to check them out into the primary Dev working area. E.g., do something like:
$ cd /web/myproject-dev
$ cvs up
And if necessary, then resolve any conflicts and commit. (You should
never have conflicts at this stage unless you had un-checked in
changes in your Dev tree.)
cvs update -j to merge.
$ cd /web/myproject-dev
$ cvs tag -R release-2000-09-07
$ cd /web/myproject-staging
$ cvs -q up -d -R -kk -j release-2000-09-07
TODO:
Note that there may be something wrong or non-optimal with the
above, because when I did it, I got lots of totally bogus but rather
complicated conflicts. E.g., doing:
$ cd /web/myproject-staging
$ find . \( -name "*.tcl" -o -name "*.sql" \) -exec grep '<<<' {} /dev/null \;
found several conflicts in each of the following files:
./www/trading/order-enter-n-4.tcl
./www/doc/sql/myproj-upgrade-db.sql
./www/doc/sql/myproj-orders_pb.sql
./www/doc/sql/myproj-orders_ph.sql
Beware of sticky tags!
[TODO: Consider writing more about the dangers and uses of sticky tags.]
The -A will remove all sticky tags, but although this may
fix your im
And we can again go to the branch checked out on Staging, and confirm
that everything is back the way it started:
$ cd /web/staging/www/bannerideas
$ cvs stat more.tcl
===================================================================
File: more.tcl Status: Up-to-date
Working revision: 1.1.1.2
Repository revision: 1.1.1.2 /cvsweb/myproj/www/bannerideas/more.tcl,v
Sticky Tag: release-2000-08-30_branch (branch: 1.1.1.2.4)
Sticky Date: (none)
Sticky Options: (none)
| atp@piskorski.com | $Id: cvs-conventions.html,v 1.4 2001/08/29 07:08:54 andy Exp $ |