A Coda cell is a set of servers which all believe one member of the
set to be the master, or SCM server. All modifications to important
Coda databases should be made on the SCM, otherwise the SCM plays the
role of an ordinary Coda server. The
updateclt
/updatesrv
daemons will then propagate
changes from the SCM to the other servers.
The first server setup must be the SCM. This chapter is divided into
three sections: installing the binaries, configuring the system using
the configuration script vice-setup
(for both SCM and non-SCM
servers) and finally, a detailed look at what needs to be done to
setup a server manually. Before beginning, we have provided some
overview information to assist in setting up the server.
To help ensure that data is not lost or left in inconsistent state between server restarts, Coda uses Recoverable Virtual Memory (RVM) which is a transaction system that maintains the system state of the Coda server meta data. RVM is a data logging transaction system which writes modifications into the RVM log and upon truncation or restart incorporates such modifications into the RVM data file. Both log and data file for RVM are ideally held on raw partitions.
NOTE: this should not be confused with Virtual Memory.
Upon startup, the Coda servers use RVM to restore the Coda system state. To efficiently use this feature, you should have dedicated disk partitions for optimal performance the log partition ideally on its own disk. However, a disk sharing the log partition with other disk partitions or the use of a log file may be used at corresponding loss of optimal performance. See section XXX for more details on RVM.
Coda servers require a minimum of 2 disk partitions for optimal
performance (one raw partition for the RVM Log, one raw partition for
RVM Data and one regular UNIX filesystem to hold file data store in Coda),
data security and protection from accidental deletion. For additional
performance gains the RVM Log partition should be on its own disk to
avoid head movement latencies which reduces disk seek times on log
operations. Optionally, /vice
can be a separate partition for
the same reasons it is advantages to have /var
as a separate
partition.
However, other configuration may be used such as having
the RVM Data and Log information stored as regular UNIX files at a loss in
performance and data security. Also, if more than one Storage Area Data
is needed on a Coda Server (the default directory is called /vicepa
),
the additional storage areas must be separate partitions (different
partition from /vicepa
the default, initial storage area for data
under Coda) and mounted as /vicepb
, for example.
The table below shows a possible partitioning of disks on Coda servers with their respective purpose, mount points, typical sizes and consistency check program. Please note that the sizes of these partitions were taken from one of the Coda servers at CMU-SCS: the actual sizes may vary at other installations.
Partition | Storage Purpose | Mounted | Typical Size | Whether fscked |
hda2 | RootandUser File System | / | 650MB | Yes |
hda5 | Var file system | /var | 100MB | Yes |
hda3 | Vice File System | /vice | 300MB | Yes |
hdc1 | RVM Log | Not | 12MB | No |
sda1 | RVM Data | Not | 130MB | No |
sda2 | Coda FS Data0 | /vicepa | 1.6GB | Yes |
sda3 | Coda FS Data1 | /vicepb | 1.6GB | Yes |
sda5 | Coda FS Data2 | /vicepc | 1.6GB | Yes |
|
Currently, the binary and source packages are available via ftp from
( ftp://ftp.coda.cs.cmu.edu/pub/coda/<platform>)
as well as the Coda Project's local Coda servers
(/coda/project/release/objs
/<platform>)
for CMU users. The platform layout for finding the Coda binaries relative
to the base directory indicated above is:
linux/<coda-release>/{i386,sparc,coda-fs-module,SRPMS} # For Linux
freebsd/<coda-release>/<OS_RELEASE_VERSION>/i386 # For FreeBSD
netbsd/<coda-release>/<OS_RELEASE_VERSION>/i386 # For NetBSD
Where <coda_release> refers to the Coda version and
<OS_RELEASE_VERSION> is the version of the Operating System.
For example, if you want Coda release 4.3.13 for i386 FreeBSD 2.2.5,
the correct platform directory is: fbsd/4.3.13/2.2.5/i386/
. For
i386 Linux, it is: linux/4.3.13/i386/
, and for i386 NetBSD 1.3,
it is: netbsd/4.3.13/1.3/i386/
. For more information on the layout,
please check the document LAYOUT
available in the top-level of
the locations listed above for obtaining Coda. However, the following
files are needed by the various platforms:
coda-debug-server-<release>.i386.rpm
coda-doc-<release>.i386.rpm
coda-debug-backup-<release>.i386.rpm
Note that the Coda Project supports the glibc (GNU C Library) version of Red Hat (5.X and higher). Support for the older Linux standard Lib C is no longer provided although Coda is known to run on many other Linux systems.
coda-server-<release>.tgz
coda-doc-<release>.tgz
Currently, Linux, NetBSD and FreeBSD are supported (CMU-MACH is no longer supported). For Linux, the Red Hat Package Management tool is supported. The package system of FreeBSD 2.X and higher as well as NetBSD 1.3+ is now supported.
TIP: Always check the INSTALL.<platform> on ( ftp://ftp.coda.cs.cmu.edu/pub/coda/<platform>) for last minute changes and updates that have not yet found their way into the manual.
Currently, server excutables for the supported platforms are installed into the following directories:
installs files/binaries in /etc
, /usr/bin
and /usr/sbin
.
installs files/binaries in /usr/local/etc
,
/usr/local/bin
and /usr/local/sbin
installs files/binaries in /usr/pkg/etc
,
/usr/pkg/bin
and /usr/pkg/sbin
NOTE:Please make sure your PATH environment variable is set properly
for your platform. That is, ensure FreeBSD has /usr/local/{bin,sbin}
and NetBSD has /usr/pkg/{bin,sbin}
in its default search paths.
Linux installs Coda into the default system paths, so Linux shouldn't need
its default seach path modified.
There is one directory used by all platfroms: /vice
which is
used to store the supporting Coda configuration and Coda databases. It
does not need to be in the PATH.
Specific steps to install the binaries and documentation are:
rpm -Uvh coda-debug-server-<release>.i386.rpm
rpm -Uvh coda-doc-<release>.i386.rpm
The installation procedure is identical for both platforms (we assume you are using GNU tar which is the default tar for both platforms):
pkg_add coda-server-<OS_RELEASE_VERSION>-<coda_release>.tgz
tar zxfv coda-doc-<release>.tgz -C /usr/share
Please note that NetBSD uses a different location for NetBSD
packages (/usr/pkg instead of /usr/local). Please make
sure /usr/pkg/{bin,sbin} are in your paths under NetBSD.
Server setup is similar for all platforms.
The /vice
directory will be created by the setup script automatically
if it does not already exist. However, if your root partition does not have
enough additional space (e.g. the SrvLog file can become huge +10MB under
certain circumstances), you will need to create a partition
to be mount as /vice
, format the partition as normal UNIX file system,
create the mount point /vice
, add the partition and mount information
to /etc/fstab
and mount it by hand BEFORE running the setup
script.
Alternatively, /vice
may be a symbolic link to a directory
on an existing partition with enough space but it also must be created
before running the setup script, vice-setup
. Sub-directories
needed under /vice
will be created by vice-setup
.
Also note that the setup of an SCM differs in some important
ways from a non-SCM setup. So, when vice-setup
asks:
Will this machine will be the SCM? it is very important
this question is answered correctly.
Only one SCM should be setup per Coda Cell and Coda does not support multiple-homing cells at this time.
While vice-setup
sets up things that are common for
both the SCM and non-SCM servers, the setup of scripts called
by answering yes to the Is this the SCM? are:
vice-setup
vice-setup-scm
vice-setup-user
vice-setup-rvm
vice-setup-srvdir
vice-setup
vice-setup-rvm
vice-setup-srvdir
in the order listed above.
If this is the first (or only) server being setup for a Coda cell, it must
be setup as the ``SCM'' by answering yes to the question when asked by
vice-setup
.
The SCM coordinates the sync'ing of Coda database and global configuration
files needed to keep track of data stored in Coda. The SCM also coordinates
the authentication of a Coda user.
If you are adding a non-SCM server which keeps copies of databases, but does not distribute them to the rest of the machines comprising a Coda Cell answer no to the question.
Other than the distinctions indicated above, the SCM plays no special role in a Coda cell and can actually be down for a short time without denying service in a multiple-machine cell site. However, is extremely important to only have one SCM
NOTE: that running
a server seriously dips into system virtual memory. Running a
Coda server, a Coda client and X11, the Coda Group
as observed that a slightly over 64MB of available VM is
needed. The command top
gives information
on memory, cpu and process usage. Therefore, we recommend
not running a Coda client on a Coda server and only run X when
performing server maintenance if X is convenient to use.
Whether setting up an SCM Coda server or adding a server to an existing Coda Cell, make sure the following items are taken care before running the setup script:
/vicepb
, additional space
must be a separate partition from /vicepa
.vice-setup-rvm
script). /vice
,
having alternative measures setup as outlined in
Before Begining.Once you have the above items available, run:
vice-setup
This provides several useful ``canned'' configuration for setting up
a Coda server. It does a number of things ``behind the scenes'' as well
(such as setting up the directory /vice
if it doesn't exist
and populating it with the sub-directories backup db srv vol bin
)
while asking the following questions:
This must be _exactly_ eight characters in length. This requirement is due to a bug that hasn't been completely fix yet.
This must be _exactly_ eight characters in length and different that the auth2 token. This requirement is due to a bug that has not been completely fixed yet.
Answering yes to this question creates the file
/vice/srv/STARTFROMBOOT
which must be present for either rc.vice
or vice.init
to start the server at boot time. Removal of this file will prevent the Coda
server from starting automatically at boot time.
For the first Coda machine being setup in a Coda cell, the answer must be "yes". If you are adding a server to an existing Coda cell, the answer must be "no".
If you have answered ``no'' to the above question, you will be asked for the HOSTNAME of an existing SCM:
If you have answered ``yes'' to the above question, you will see the message:
An initial administrative user "admin" with password "changeme" now exists
. and pick up at the question Are you ready
to setup up RVM?
If you have answered ``no'' to the above question, you will be asked for the HOSTNAME of an existing SCM:
This question is only asked if the machine is not the SCM. The hostname entered must be that of the SCM which distributes the Coda cells'database and global configuration files.
Answer yes to this question or the setup will abort.
RVM is the Recoverable Virtual Memory used to keep a copy of Coda Virtual
memory on local disk.
While this can be a file, it is strongly recommended that it
be a small partition on a dedicated disk.
If you have set aside, for example, /dev/sdc1 under Linux as
the log partition, enter the partitions' device name at the prompt. If you
are setting up a *BSD system, you must enter the raw partition name.
For example, if you are going to use /dev/sd2e
you must enter
the raw device name /dev/rsd2e
.
The maximum template size we provide for and have tested is 30MB.
The log partition keeps transaction records for Coda volume updates
that may not have been writen to the Coda data partition yet. We
have not extensively tested values above 30MB and don't recommend more
than this. 12M is a good value for a small to medium sized server. The
value may be entered as, 12M
or 12288
for 12 mega-bytes.
Please see section
The Data Partition for a more detailed explanation
This allows you to specify a partition or file. We strongly recommend
a partition which can reside with other system partitions. Remember,
if you use a partition, you are using it "raw" . it does not contain
a file system of any kind. Again, if you are using Linux, you may
enter a block device name such as /dev/sdc1
. If you are
using a *BSD system, you must enter a raw device such as
/dev/rsd2e
.
This specifies the the size of the RVM data partition. It must
be typed exactly as 22M
, 44M
, 90M
, or
130M
. These are associated with default parameters that
must be feed to codasrv
when it is started. The following table
provides the log size to total data storage conversion:
22M
is good for up to 500MB44M
is good for up to 1.0GIG90M
is good for up to 2.2GIG130M
is good for up to 3.3GIG
if you have entered erronous entries for RVM log and RVM data partitions, you could damage your system. Be sure you entered correct information!
y
to commit or N
to abort.
Older development versions of coda require /vicepa
to be present.
This version does not. You give a name other than the default we
suggest. However, we strongly advise not calling it /code
as
that can be confused with venus.
Once you successfully complete vice-setup
:
you are ready to start the update server, the update client and the auth server, as well as the fileserver by typing:
/etc/rc.d/init.d/update.init start
/etc/rc.d/init.d/auth2.init start
/etc/rc.d/init.d/codasrv.init start
/usr/local/sbin/auth2 &
/usr/local/sbin/updateclnt -h `cat /vice/db/scm` -q coda_udpsrv &
/usr/local/sbin/updatesrv -p /vice/db &
/usr/pkg/sbin/auth2 &
/usr/local/sbin/updateclnt -h `cat /vice/db/scm` -q coda_udpsrv &
/usr/local/sbin/updatesrv -p /vice/db &
One final step is needed to make the SCM server function. Use the root volume you specified in vice-setup in place of your-root-volume. This step creates the volume, which will be mountable by the client software.
createvol_rep your-root-volume E0000100 /vicepa
Once you have done this, check that the server has started by viewing the log:
xterm -e tail -f /vice/srv/SrvLog &
You should start auth2, updatesrv and updateclnt first by typing:
/etc/rc.d/init.d/update.init start
/etc/rc.d/init.d/auth2.init start
/usr/local/sbin/auth2 -chk &
/usr/local/sbin/updateclnt -h `cat /vice/db/scm` -q coda_udpsrv &
/usr/local/sbin/updatesrv -p /vice/db &
/usr/pkg/sbin/auth2 -chk &
/usr/pkg/sbin/updateclnt -h `cat /vice/db/scm` -q coda_udpsrv &
/usr/pkg/sbin/updatesrv -p /vice/db &
This will cause the new server to first obtain the current server
files such as /vice/db/ROOTVOLUME
and /vice/db/VRList
as well as others needed for this server to participate in the
Coda cell. Once these files have been retrieved, you may start the
server manually for the first type by typing:
/usr/sbin/startserver &
/usr/local/sbin/startserver &
/usr/pkg/sbin/startserver &
Once you have done this, check that the server has started by viewing the log:
xterm -e tail -f /vice/srv/SrvLog &
If the codasrv
process started correctly, you should see
``File Server started'', if not you need look at the messages in the SrvLog
to determine what went wrong. If the messages are not helpful, you
should kill the server (NOTE:The server is designed to continue to
stay up so gdb can be attached for debugging):
kill -9 `cat /vice/srv/pid`
and then you can restart the server with:
startserver -d 10 &
However, assuming that the server came up, also make sure updatesrv
and updateclnt are running by using either top
or ps
.
you need the package bc (binary calculator on the server). you can either obtain from our ftp site or a mirror site containing your operating system and contrib software.
E0000100 is the Volume Storage Group set up for you by vice-setup when setting up an SCM. With more servers you can define other groups in /vice/db/VSGDB.
Once this is complete, you should setup a client
machine and run Venus and point venus
point to this server.
Setting up
a client machine with venus
is covered in the next chapter.
Once you have tested and verified the Coda server works from a client,
you are ready to adduser more volumes and users!
NOTE: A current limitation of the current vice-setup-user script is
that the initial admin account needed is assigned a Coda uid of 500.
In order for a client to authenticate a user, that user must exist
with the same corresponding user-id in the UNIX password file.
If you need to change the admin account uid to suite local uid space,
please edit the script vice-setup-user
, change the default
Coda uid to the needed value and re-run the script. This only applies
if you are setting up the SCM.
Otherwise, if this is a non-SCM server, you might consider setting up a singlely replicated volume on the new server to test it.
The sections contained here describe what vice-setup
does for
you. This information is useful for those who either wish to customize
our default vice-setup script or wish to have a custom server setup
outside the scope of Coda's setup script..
RVM initialization requires the selection of several parameters, each
of which involve tradeoffs. Although the
RVM log and data can be kept as regular UFS files, this is not
recommended: Raw partitions have much stronger data consistency
guarantees as well performance advantages.
It is probably best to plan out the RVM partitions on
paper first, taking into consideration both the effect on RVM
performance as well as overall disk usage if you choose not to
use the values provided in the vice-setup
script.
The size of the log device is based on available space and issues
involving truncation. Larger logs provide a longer accessible history
of operations, are truncated less frequently, but each truncation will
take a longer period of time. Shorter logs truncate more often, but
each truncation takes less time. Log size is also strongly related to
server startup time as well. We use a 90M log size, on a
storage size roughly 3.2GIG spread between two UNIX partitions.
(We suggest leaving a little space at the
end of the RVM log partition for safety, as RVM automatically adds
about one extra page to the amount you specify). Our vice-setup
script provides for several default values, the largest of which is
130MB. If you decide to follow the recommendation and use a dedicated
partition, creating a 130MB partition will leave plenty of room to expand
even if your initial use only suggests a size of 22M, for example..
The log is initialized with rvmutl. At rvmutls prompt, use the command i, and then specify the size of the log segment. In specifying the size, you can use M for megabyte and K for kilobyte. For example, to initialize a log on partition 0g to eight megabytes:
# @B(rvmutl)
* i
Enter name of log file or device: /dev/sdc1
Enter length of log data area: 22M
* q
The data segment contains the meta-data of the system such as volume headers, Coda directories, vmond lists, resolution logs, etc. The size of the data segment depends on the amount of disk space for file data, i.e. the size of the /vicep? partitions. As a rule of thumb, you will need about 3-5% of the total file data space for recoverable storage. We currently use 4% as a good value under most circumstances. In our systems the data segment is 90Meg for approximately 3.2 gigabytes of disk space. By making it smaller, you can reduce server startup time. However, if you run out of space on the RVM Data partition, you will be forced to reinitialize the system, a costly penalty. So, plan accordingly.
In initializing (or reinitializing), you need to pick several parameters.
The first is the starting address of the recoverable segment in your
address space. On our servers we start the RVM segment at 0x20000000
for both Linux and BSD based systems on Intel architecture.
The second is the amount of space to give the recoverable heap. The
heap will obviously grow over use, so plan accordingly. Our heap
space is 0x1000000. We suggest that for the last parameters you use
1Meg (0x100000) for the static area, use 80 free lists (or nlists),
and a chunksize of 32. These numbers will work well
with the internal structure of the fileserver and are provided as the
default vales in the setup script vice-setup
.
To perform the data initialization, run the program rdsinit. rdsinit takes two parameters, the names of the RVM log and data devices. For example, to initialize one of our Intel based servers:
# rdsinit /dev/hdc1 /dev/sdb1
Enter the length of the device /dev/hdc1:119070700
Going to initialize data file to zero, could take awhile.
done.
rvm_initialize succeeded.
starting address of rvm: 0x20000000
heap len: 0x1000000
static len: 0x100000
nlists: 80
chunksize: 32
rds_zap_heap completed successfully.
rvm_terminate succeeded.
NOTE: the use of the decimal value for the length of the device and the use of hex values for the address and lengths of the next three values.
The update monitor is used to propagate changes to the Coda server databases
to all servers from the SCM. Client update processes run on all the Coda
servers and connect to a server udpate process running on the SCM.
The server process uses the file /vice/db/files
to determine which
files should be kept consistent on all the servers. See the
updateclnt(8) and updatesrv(8) man pages for more details.
Create the file /vice/db/files
on the SCM. Currently our
/vice/db/files
looks like this:
VLDB
auth2.pw
auth2.tk
pro.db
servers
hosts
vice.pdb
vice.pcf
volutil.tk
VRDB
files
VSGDB
dumplist
scm
ROOTVOLUME
Coda uses an authentication database that is separate from the UNIX password file. This database is maintained by the SCM. When someone authenticates to Coda, their password is checked against this database and that person is issued a token if they successfully authenticate. This section describes how to initialize the authentication database.
If the server you are installing is the SCM, you must set up the initial
authentication database. Make sure you add at least one user to the
System:Administrators
group. You need to do the following:
user.coda
can be
found in the directory /vice/db
as
well as in Appendix
XXX.user.coda
file to the group
System:Administrators
. An example of group.coda
can be
found in the directory /vice/db
as
well as in Appendix
XXX. /usr/sbin/pwd2pdb -u user.coda -g group.coda > vice.pdb
/vice/bin/pcfgen vice.pdb
which will create vice.pcf.uid<TAB>actual cleartext password<TAB>any desired info
.
The information field could be the full name of the
user. Name this file /vice/db/passwd.coda
. Then run the command
initpw -k "drseuss " </vice/db/passwd.coda > /vice/db/auth2.pw
This command produces the file auth2.pw
in the format:
uid<TAB>encrypted password<TAB>Desired info
.
Note: "drseuss " is an encryption key to prevent accidental disclosure
of the passwords to the system administrators. In future versions of
coda we will store a sites specific encryption key in a configuration file.
Note that all future changes of the password must be made through the
au(1) program, and not by repeating this procedure since that would
overwrite any passwords changed by users after initialization.
New passwords must be exactly 8
characters long. To add additional users, please see Section
XXX. /vice/db/auth2.tk
./vice/db/volutil.tk
, used for secure communication among
the servers. This file should contain a single word which will be used to
encrypt messages between the server and server applications. The file
should be present on all servers. The UpdateMon presently takes care of
propagating them from the SCM.
On servers other than the SCM, the Update monitor will
ensure that the proper database files are propogated from the SCM.
However, you must copy the file /vice/db/volutil.tk
to the new server for the Update monitor to work.
Every combination of servers setup expects to use as a VSG which
must be given a multicast address in /vice/db/VSGDB
.
This file must be created by hand; please refer to VSGDB (5)
in Appendix
XXX for more details.
We have provided a script called startserver
which reads in the file /vice/srv.conf
which contains the
startup information needed by codasrv
. The syntax of
/vice/srv.conf
is:
-rvm <log_partition> <data_partition> <size_of_log_partition>
The vice-setup-rvm
contains the canned default value
for the corresponding value entered through vice-setup
.
Should you deviate from one of these canned default values, you will
need to either create or modify the /vice/srv.conf
file
to indicate the correct data size of the data partition. For example,
one of CMU's Coda servers contains:
-rvm /dev/hdc1 /dev/sda1 119070700
You should be careful to specify the correct partitions for the RVM log, data segment, and size of the data segment.
Once this file has been correctly edited, you can start the server with startserver&.
If you are installing any server other than the SCM, please use the instructions in section XXX to create new volumes on this server.
The root of the Coda file system must reside on the /vicepa partition for correct functioning. Since the SCM probably also acts as a server, place the root of the Coda file system in the /vicepa partition on the SCM. Create the root volume by using either one of the following commands
% createvol_rep codaroot <VSG_ENTRY> /vicepa
% createvol codaroot <hostname> /vicepa
the former is for replicated root volume, where <VSG_ENTRY> is
the entry in the VSGDB
for the volume storage group for this
volume; the later is for non-replicated root volume, where
<hostname> is the name of the host which hosts the root volume.
Since it is likely that the root will be cloned and read-only
replicated, you might wish to create this volume as singly replicated
for simplicity.
These commands will create the /vice/db/VLDB
, /vice/db/VRDB
,
/vice/vol/VolumeList
, /vice/vol/BigVolumeList
,
and the /vice/vol/AllVolume
files.
Create or modify the file /vice/db/ROOTVOLUME
to contain the
volume name (E.g. codaroot
) corresponding to the root volume as
listed in the file /vice/vol/VRList
.
You may wish for the root to be readonly. To do this, first create a root
volume as above. Start a venus which can talk to this server. Setup the
directory structure you want in your root volume. Once you are
satisfied with the structure, clone the volume and
dump it to a disk file with the the volutil
program. Now restore
the dump file to the servers in the desired VSG for the readonly volume.
Be sure to specify the volumeId and the volume Name for the new volume:
% volutil restore <filename> /vicepa <newrootname> <volid>
Then modify the /vice/db/ROOTVOLUME
file to contain the new
rootvolume name, <newrootname>.