This chapter is a guide to using and modifying Coda source code at CMU-SCS. It is mainly intended to help new members of the project come up to speed. But it may also help sites outside CMU that receive a Coda distribution to devise procedures appropriate to their environments.
All files relating to Coda, with the sole exception of RCS files (see Section
XXX), are stored in /coda/project/coda.
Let's call this $CODA
.
The directories under $CODA
correspond to different releases of Coda, such as
alpha,
beta,
and so on. A release is a complete set of mutually consistent
sources, libraries and binaries. "Mutually consistent" means, for example, that the
Venus and server from a release will work with each other. It also means that you can
freely use the libraries and include files to compile new binaries in that
release. @ux[There are no guarantees of any kind across releases.]
Before a new instance of a release is made, careful thought must be given as to how current users of that release will be affected. Will all such users have to be upgraded in one fell swoop? Or can they switch over at their own leisure? As the deployed Coda system grows, upgrading everyone in one fell swoop gets harder. After all, some of the users may be in hiding with their disconnected laptops!
By convention, the omega
release corresponds to the version of Coda in production
use. Most users Venii, as well as the servers on production machines,
are from this release. Hence the value of /usr/coda/SYMLINK
on those users workstations
is $CODA/omega
.
The alpha
release of software corresponds to a lightly tested version of the system.
Individual developers have done a significant amount of testing of alpha
software, but
it hasnt yet been stressed heavily, nor has it been tested in real use. Hence alpha
server code should not be run on servers that hold real user data. They should only be
run on the alpha
servers (also known as "test" servers). Similarly for other alpha
software. When you are doing code development, you
usually pick up include files and libraries from the alpha
release. Thus, the
private versions of software you build is, at best, of alpha
quality.
The beta
release corresponds to a version of the system that is expected to be released
soon for production use.
A small number of users, typically members of the Coda project, depend upon
this release. Actual use by such users is part of the testing of the beta
release. Eventually, when we have enough confidence in the stability and correctness of
the beta
release, we promote it omega.
beta
is really a symlink to a volume mount point beta-<unique>
,
where <unique>
is an identifier made up by the makebeta
script described below.
Upgrading the beta
release consists of creating a new volume, mounting it
at beta-<unique>
, populating it using the steps described in Section
XXX,
and finally changing the beta
symlink to point to beta-<unique>
.
The old beta
volume is preserved for a while, until is clear
that the new release hasnt triggered any serious problems. It can then be purged
to reclaim space, or recycled for a future beta release.
omega
is also a symlink to a volume mount point beta-<unique>
.
After a beta
release has been running robustly for a while, it is upgraded
to omega
by merely changing the value of the omega
symlink. No recompilation
is involved.
When a beta-<unique>
release is created, the makebeta
script creates an RCS branch
named beta-<unique>
for every file in the release.
Normally no development or changes are done along this branch.
However, it provides a way to introduce an emergency fix
in a beta
or omega
release if the need ever arises.
Lets illustrate this with a specific example. Suppose I want to upgrade the module
rpc2
and build a new Venus to use the upgrade. I begin by creating two
private directories, say /coda/usr/satya/src/rpc2
and /coda/usr/satya/src/venus
,
and populating them with source files from the alpha
release of these two modules.
Section
XXX tells you how this step actually gets done. Now I proceed to modify files in
rpc2
and to compile and test the module using standalone test programs.
Then I modify files in venus
to use the upgrade and then build a Venus.
When building Venus, I must make sure that the version of rpc2
used is the one I just built.
Section
XXX tells you how to do this. Now I have a Venus that I can test.
As bugs are found in the new Venus, I iterate the above procedure.
When I am confident that my changes to rpc2
and venus
are right, I update the alpha
release of these two modules.
The process of updating the alpha
release is known as installation.
Installation is the point at which work done by a Coda project member
becomes visible to others in the project. Prior to this point, all work is done in
that users own private directory.
Installation always occurs at the granularity of entire modules. In other words, one never
installs an individual library, include file, etc. Rather the entire source code for a module and
all relevant files compiled from it are installed together. Modified files are automatically
checked into RCS as part of the installation procedure.
At some future time, these changes along with many others that were installed
will make it into a beta
release. Section
XXX shows you how
this is done. After some use as beta
, it will be promoted to omega
.
Each release has the structure shown in Figure XXX. Note that this layout is identical for all releases.
Underlined names are symlinks to machine-specific directories. On a 386
machine for example, bin
is a symlink to i386_mach/bin
.
Although only two machine-specific directories (pmax_mach
and
i386_mach
) are shown, there can be many more.
The source tree for the release is in src
. A copy of the header
files from src
is in include
. Both src
and include
are machine-independent. For each
supported machine type, there is a directory containing binaries (bin
) and libraries (lib
).
The source tree in a release is fully self-contained. In other words, if you started out with empty
include,
lib,
and bin
directories, you could completely populate them by compiling
the source code in src
. The only exception to this are the files in include-special,
lib-special,
src-special,
and bin-special
. These directories contain a very small number of files
that have to be copied in by hand. The sources for these files are not in src
.
The src
directory in each release is organized as in Figure
XXX. The MAKECODA
script simplifies the compilation of the entire release. The Makeconf
file defines in
a single spot many key variables and paths used by the makefiles in individual modules. Those
makefiles inherit these definitions automatically, when the CMU-SCS make
is used.
The SOURCEME
file contains a minimal set of environment definitions. By sourcing this
file before you compile anything, you can be sure that you arent obtaining binaries, libraries
etc. from non-standard places. This is especially important if your .login
or .cshrc
files define elaborate PATH, CPATH, LPATH
variables.
Underlined names are symlinks to RCS directories. For example,
RCSLINK
is a symlink to /afs/cs/project/coda/rcs
;
auth2/RCS
is a symlink to ../RCSLINK/auth2
; and so on.
Indirecting via RCSLINK
makes it simple to relocate the RCS
directories without changing lots of individual symlinks. Such relocation
might happen, for example, when Coda sources are used outside CMU.
The structure of a typical module is shown in Figure XXX.
In this particular module, rpc2
, the underlined name RCS indicates a
symlink to ../RCSLINK/rpc2
. Indirection via RCSLINK
makes
relocating the RCS directories a simple matter.
Notice that only the source files are located in this directory; there are no object
files. The CMU-SCS make
facility puts all compilation targets elsewhere, thus
allowing the source directory to be readonly. This has two advantages. First, the
source directories are uncluttered. Second, it simplifies building Coda for multiple
machine types, since the target directories can be different for each machine type.
Also notice the presence of multiple makefiles in this module. Although this is not
characteristic of all Coda modules, it is typical of some. The true dependencies are
capture in Makefile.real
, and the others, like Makefile.coda
and Makefile.misc
invoke Makefile.real
after defining environment variables appropriately. This simplifies
the use of these modules outside Coda. If you are compiling by hand, you have to say
"make -f Makefile.coda <target>
", rather than just "make <target>
".
Most Coda modules rely on files installed by other Coda modules. It is therefore important to install modules in the correct order. Otherwise you could get yourself into real trouble. If you are compiling from scratch, the out-of-order installations will simply fail. But if you are modifying an existing release, you could end up with mysterious bugs because obsolete versions of header files and libraries may be used.
You are @ux[strongly] urged to use the script MAKECODA,
described in Section
XXX.
The correct precedences of modules are wired into the script, so you dont have to
deal with them.
For the curious, the correct order of compilation of modules is given below. In principle, modules of the same precedence (i.e. in the same set) can be compiled in parallel. But I havent actually tried that yet.
scripts
Miscellaneous scripts, including alphaci
on which intalls of all other modules rely mlwp
Lightweight process package dir
Directory package used by Venus and server sys
Miscellaneous routines sunrpc
Sun Microsystems public domain XDR code and interface for device driver/venus interaction igmp
Internet multicast support for old RFC
(dummied out currently; someone should fix these to use the new RFC) util
Utility routines rpc2
RPC package camstuff
Header files that allow runtime choice of RVM or VM for persistence on servers blurb
Program to adjust copyrights pdbstuff
Protection database management rp2gen
Stub generator for RPC2 comm
Communication layer above RPC2 for connection management (not yet in use) libal
Access list package. vicedep
Header files and RPC2 interface definition files put here to break circular dependencies fail
Network failure and variable speed emulator auth2
Authentication server login
Implements clog, cunlog, ctokens, etc. cfs
The VFS driver; most of this code is linked into the Mach kernel vv
Version vector routines mond
Coda usage data collector resolve
Library used by repair (should get integrated into repair
) vol
Volume package used by server res
VM-based directory resolution algorithms repair
Repair tool venus
Cache manager volutil
Volume utilities vtools
Miscellaneous Venus tools rvmres
RVM-based directory resolution algorithms vice
Server code update
The daemon which updates server databases norton
A Coda server, RVM debugger asr
Application Specific Resolver package egasr
ASR examples@end)annote
To compile Coda "out of the box" you need the following compilation tools:
make
The makefiles in the Coda sources exploit many of the enhancements
made by CMU to Unix make. Conditional macro expansion, the ability to place targets
in a different place from the sources, and the inheritance of variable definitions from
Makeconf are the three major features we rely on.long
to represent enum
.
The RT C compilers default behavior of using char
to represent small-valued enums will
cause insidious problems in RPC2 communication. On CMU-SCS machines this is done by defining the
environment variable PLAINC
to be /usr/misc/.hc/bin/hc2
. The MAKECODA
script does this
automatically for you, but you must remember to do this manually if you arent using it. (Some .login
s have the variable ccC
defined to hc
; you must delete this definition
also).It should be possible to modify the code to use other versions of C++
(such as g++
), or to use standard Unix make. But we havent
tried this, and dont plan to.
Before you can compile Coda, you need to populate the {include,lib,bin,src}-special
directories.
These contain files that are (a) needed but arent in the Coda sources or (b) standard Mach header files, with slight modifications. The MAKECODA
script contains an up-to-date list of what these files should be. Here is
a list that was current at the time of writing this document:
RVM is a lightweight transactional package that is used on servers and clients. It is a package that is independent of Coda. The current set of files from this package are:
include-special/{rvm.h,rvm_lwp.h,rvm_statistics.h,
rvm_segment.h,rds.h}
lib-special/{librvm.a,librvmlwp.a,libseg.a,librds.a}
bin-special/{rdsinit,rvmutl}
src-special/{Makeconf,Makefile,READ_ME,plumber,rds,
rvm,seg}
These files pertain to a file-tracing
facility, dfstrace
, used at CMU. To function as a
trace-driven simulator, Venus requires the following files:
include-special/tracelib.h
lib-special/libtrace.a
If you don't intend to use Venus as a simulator, you could construct a dummy libtrace.a
with
empty routines to avoid unresolved references. You do need tracelib.h
though.
malloc
The Coda makefiles allow you to build versions of venus
and codasrv
that use a special malloc
to help detect memory leaks. The following files are needed
for this:
include-special/newplumb.h
lib-special/{libplumber.a,libnewplumb.a}
If you don't plan to build the plumbing facilities, you can just create zero-length files with
these names to keep the MAKECODA
script happy.
The changes in these files had to be made
because of compilation errors
from C++, or (as in the case of assert.h
) to define different behavior for standard macros:
include-special/{assert.h,cthreads.h,setjmp.h,sys/inode.h,
i386/fpreg.h,i386_mach/endian.h}
Once you have taken care of the prerequisites and special files, you can compile a release.
The simplest way to do this is to run the MAKECODA
script in src
.
This script takes one required and two optional arguments.
The required argument is OBJECTDIR
, which is the pathname of
the directory where the object files should be placed. It is sensible to specify a different
directory for each machine type, and the @@sys
facility of AFS and Coda lets you do this.
So, for example, to compile the beta
release, I would do the following:
cd /coda/project/coda/beta/src
./MAKECODA OBJECTDIR=/coda/usr/satya/OBJS/@@sys
The MAKECODA
script will first check to make sure that all necessary
special files are present. If any are missing it will prompt you. It then
goes through the Coda modules in the correct order and does a make install
on each. The usual checkin to RCS that is done by
alphaci
as part of make install
(see Section
XXX) is supressed.
If all goes well, everything in Coda will be compiled, and the bin
, lib
,
and include
directores will be populated. You will have to repeat this once
for each machine type.
Sometimes, you will run into a problem part-way through MAKECODA
. After you
have fixed the problem, youd probably like to continue where you left off rather
than redoing everything from the beginning. Hence MAKECODA
lets you specify
the name of the module to start from. Here is an example:
./MAKECODA OBJECTDIR=/tmp/@@sys FIRSTMODULE=vol
Finally, the pathname of the root of the release you are compiling is specified by
the variable ROOT
in MAKECODA
. You can change this by editing MAKECODA
,
and this is in fact what the makebeta
script does for you when you create a new release.
But you can also override it on the command line thus:
cd /tmp
MAKECODA OBJECTDIR=/tmp/@@sys ROOT=/coda/project/coda/alpha
Once the alpha
release has been built, you can start code
development. We have already seen the general procedure in Section
XXX. Lets look at an example in more detail.
Suppose I am working on the module vtools
and need to change
the files cmon.c
and codacon.c
. Heres what I would do:
# Create a scratch directory for my work
mkdir /coda/usr/satya/src
cd /coda/usr/satya/src
# Set up links to RCS directory, and root of release
ln -s /afs/cs/project/coda/rcs/coda-1.0 RCSLINK
ln -s /coda/project/coda/alpha/src SRCROOT
ln -s SRCROOT/Makeconf
# Create directory for this module
mkdir vtools
cd vtools
ln -s ../RCSLINK/vtools RCS
# Lock and checkout the file (s) to be modified
rcsco -l cmon.c codacon.c
# Now edit cmon.c and codacon.c
# Then compile this module
# Source the standard environment file to make sure your
# compilation environment (PATH, LPATH, CPATH, etc.) is set up right
source ../SRCROOT/SOURCEME
make OBJECTDIR=/coda/usr/satya/OBJS/@@sys cmon codacon
# Test cmon & codacon, then iterate on edit/debug cycle above
# Now you are ready to install your changes.
# You must first create a file called RCSMSG, and enter text
# in it that will become the RCS log message for the checkin.
# It will also be posted to the changelog bboard, so that
# others in the group will know of your installation
echo "Fixes to annoying bugs ... blah blah blah ..." > RCSMSG
make OBJECTDIR=/coda/usr/satya/OBJS/@@sys install
# You are now done!
# The install step automatically released the write locks on cmon.c
# and codacon.c
# Repeat the install step for each of the other supported platforms.
Notice that many other files may be need for compilation, but you dont have to check them out. This is because the CMU-SCS make knows to check out any needed files automatically into the compilation target area.
The automatic checkin is done by a script called alphaci
that is invoked
as the last step of make install
. alphaci
will give you an error if
the RCSMSG
file is missing; it moves it to RCSMSG.old
once it has
checked in files. alphaci
is smart enough to discover new files that
arent mentioned in RCS, and prompts you to ask if you want them checked in
too. Often you may not want this, because the files in question were just
test files created for debugging. alphaci
assumes that only writable
files should be checked in; it prompts you about what to do with files that
are not writable.
The RCSMSG file is also used by alphaci
for posting on the changelog bboard (cmu.cs.proj.coda.changelog
at CMU). You should follow the posts on this
bboard closely, so that you are aware of changes to Coda modules by other
project members. Here are some typical posts from the bboard:
03-Feb-93 18:56 M Satya Installed scripts for IBMRT
Posted by alphaci from STRAUSS.CODA.CS.CMU.EDU
No files checked into RCS
03-Feb-93 18:50 M Satya Installed scripts for I386
Posted by alphaci from WEBER.CODA.CS.CMU.EDU
No files checked into RCS
03-Feb-93 18:22 M Satya Installed scripts for PMAX
Posted by alphaci from MOZART.CODA.CS.CMU.EDU
Files checked into RCS: Makefile makebeta restartserver restore.sh
RCS message follows:
Created new script, makebeta, to make a clone of the entire alpha
source tree, and to create branches in RCS for all files.
Notice how the installations for machine types I386 and IBMRT caused no RCS
files to be checked in. This is because the installation for the first machine
type, PMAX, did the checkin.
As we have seen earlier, there are 3 important releases of software:
alpha,
beta,
and omega
. Servers are also classified
by this scheme. At the time of writing this document, there were
3 omega
servers (rossini
, puccini
, and scarlatti
),
3 beta
servers (grieg
, haydn
, and wagner
), and
4 alpha
servers (schumann
, gershwin
, mahler
, and vivaldi
).
The omega
servers hold real data, so software let loose on them should
have been tested very well. Having to reinitialize and to restore many gigabytes on
the omega
servers because of storage corruption is not an experience
you are likely to forget! The beta
servers hold some real data, but
the number of users depending on that data is restricted to a few Coda project
members. Further, that data is of a kind that can be easily reconstructed.
The alpha
servers are used by project members for testing.
Obviously, only one person can test their software on a server at a time, so you
should coordinate use of the servers with the other members.
When the alpha
release has diverged substantially from beta
,
and is relatively stable, we will decide to make a new beta
release.
One member of the Coda project will serve as release coordinator for the promotion.
The first step in this process is for the release coordinator to post on
cmu.cs.proj.coda.general
, asking project members to checkin changes and to drop
all RCS locks on the main line of development by a certain deadline.
As soon as possible you should release all mainline locks. It is ok to
hold on to locks on private branches. Once the deadline expires, the release
coordinator will feel free to break mainline locks.
In the second step, the release coordinator runs the makebeta
script to
create a clone of the current state of alpha
. makebeta
first
synthesizes a unique identifier of the form date_xxxxx where xxxxx is
the number of seconds since midnight. The name of the new release is then
beta-<unique identifier>
; for example, beta-3Feb1993_43696
.
This name is used to tag a new RCS branch for every file in the release,
so that emergency fixes are possible long after the alpha
release has
diverged from this beta
release.
The makebeta
script works as follows:
makebeta
checks to make
sure the directory is empty before proceeding, to avoid accidental clobbers.
For brevity, we refer to this directory as $BETA
.makebeta
looks at /coda/project/coda/alpha/src
($ALPHA/src
, for short)
and determines the names of the modules in it.$ALPHA
modules in $BETA
. In making this copy,
it actually checks out the files (unlocked) from RCS, and compares them with $ALPHA
.
This is a sanity check, to ensure that RCS and $ALPHA
are in sync. One side effect
of this comparison is to detect RCS locks.$ALPHA
, makebeta
pauses and asks the user to go take a look at whats wrong.
It then asks the user whether to proceed.makebeta
creates a new RCS branch, named after this release, and re-checks
out files from the branch so that $Header$
in those files is correctly set.MAKECODA
and Makeconf
at the top level of $BETA
are edited by makebeta
so that all references to $ALPHA
are changed to $BETA
.makebeta
copies over all the special directories ({lib,bin,src,include}-special
).The above procedure is quite slow, since many RCS interactions are involved. It usually takes
a few hours. If all goes well, makebeta
requires little babysitting.
Note that the script does no locking. In other words, the $ALPHA
tree better remain
frozen for the entire duration of the promotion.
In the third step, the release coordinator cd
s into $BETA/src
, and runs MAKECODA
once for each machine type. Note that he doesnt have to specify anything other than OBJECTDIR
,
since makebeta
has already set ROOT
correctly in the MAKECODA
script. This step
guarantees that the binaries in $BETA
were indeed compiled from the sources in $BETA
.
In the final step, the release coordinator makes the symlink beta
point to $BETA
.
This is the "commit" step that blesses the newly-cloned release as the beta
release.
After the beta
release has been stressed for some time,
it will be promoted to omega
. This is a much simpler procedure than the
alpha
to beta
promotion. The release coordinator merely has to make the
symlink omega
point to the beta-xxxx
where xxxx
is the unique
identity of the release being promoted. This "commit" step blesses what was
hitherto beta
as omega
Since Coda is distributed outside CMU, it is important that
every source file contain a copyright notice. The blurb
program
simplifies adding and changing copyright notices. blurb
expects
to find the copyright notice is at the very beginning of a file, so make
sure you dont move it when modifying a file.
The only modules without Coda copyright notices are sunrpc,
which is public-domain code from Sun, and cfs,
which is mostly
Mach kernel code.
Some files in Coda are derived from the 1986 version of AFS-2.0. Since AFS-2.0 is owned by IBM, these files have an IBM copyright notice in addition to the Coda copyright notice.
When you modify a file or add a new file, you should pay attention to the copyright notices. Here are some simple rules to follow:
A simple way to think of this is that some files in Coda are "tainted" (i.e., they are derived from AFS-2.0). Tainted files can infect other files if enough of their innards are copied. Existing untainted files, and new files, stay untainted forever. In general, do your best to keep the number of tainted files to a minimum. Note that this discipline regarding copyrights is not intended to be onerous or constraining --- just common sense and a little self-discipline.
1. Branch overview
One common problem in managing large software projects comes
to play when there are several people working on the same sets
of files. RCS helps by providing per-file locking to guarantee
that no two users modify the same file at the same time. If
several users are modifying the same line of development,
however, this locking does nothing to guarantee that one users
changes in one file wont interfere with another users changes
in another file.
To combat this problem, RCS provides the notion of a "branch".
A branch is a separate line of development carried out in parallel
to the main line of development. Version found on branches have
more than two digits in their version numbers. All digits make up
the version number of an element on a branch. For example, version
1.2.1.1 is an element in the first branch off of version 1.2.
As branch numbers are hard to deal with, you are recommended to
assign symbolic names to branches.
2. How to use branch
For example, a typical command to create and name a branch would be
rcsci -b -nKUDO_STATISTICS foo.c
where KUDO_STATISTICS is a symbolic name of branch
* This command will create a branch named KUDO_STATISTICS
off the last version on the mainline of foo.c
If you would like to specify the version number of which the first
branch element is off, a command would be
rcsci -b -r1.2 -nKUDO_STATISTICS foo.c
where 1.2 is the version number of which the first
branch element is off
The name is assigned to the branch (not the branched element), so
the subsequent commands such as
rcsco -l -rKUDO_STATISTICS foo.c
will refer to the latest element in that line of development.
3. What I checked in using branch
I checked in the following files using branch with the symbolic name
"KUDO_STATISTICS".
vicedep/mond.rpc2
venus/ fso.h
fso0.c
fso1.c
hdb.c
sighand.c
venus.c
venus.private.h
venusresov.h
venusutil.c
venusvol.c
venusvol.h
venusvm.c
venusvm.h
vol_vsr.c
vproc.c
The purpose of the modifications is to collect session statistics
and send them to the mond data collector. Currently, venii built
with these files are running on Brahms and faust, safely (I think so).
<TO BE COMPLETED>