| ProjectNorton/AfsOnNetBSD |
UserPreferences |
| The Project List | FrontPage | RecentChanges | TitleIndex | WordIndex | SiteNavigation | HelpContents | moin.sf.net |
Use OpenAFS for the server side, and for the various afs commands: fs, vos, bos, etc. The OpenAFS client stuff (afsd), however, does not (currently) work, so use Arla for afsd and friends.
While I use afs heavily at work, I do little of what you would consider administrative stuff. At best, I'm a talented amateur, at worse, several terabytes of data go wondering off into the aether. Some of the stuff here may seem goofy to seasoned AFS administrators. In particular, I've done no work on making the Arla and OpenAFS trees of installed programs be in some sane merged binary --- things are all off in completely separate trees here (no --with-transarc-paths here ....)
You have been warned....
Download openafs-1.3.77 (later versions will probably work, but this is what I used). These are the compile options I used:
./configure --enable-largefile-fileserver --enable-fast-restart --enable-bitmap-later \ --disable-kernel-module --enable-debug --enable-bos-new-config --enable-namei-fileserver \ --prefix=/usr/local/openafs
XXX Put explanations of config flags here.
Do the make and make install dance.
Download arla-0.38rc1 (again, later versions will probably work, but this is what I used). Here are the compile options I used:
./configure --with-krb4-lib=/usr/lib --with-krb4-include=/usr/include/kerberosIV \ --with-sys=/usr/src/sys --with-krb5-lib=/usr/lib --with-krb5-include=/usr/include/krb5 \ --prefix=/usr/local/arla
XXX Put explanations of config flags here
Note: You will need to have the NetBSD system src wherever --with-sys points to.
Essentially, you will be following the "Unix Quickstart Guide" available on the OpenAFS web site, with some changes: skip over the kaserver stuff, since you will be using the heimdal kdc to get kerberos tickets and various other things to munge those into AFS tokens.
I don't use AFS' built in ntp stuff, so I've set up my machines to all sync to a common time source. You will want to do this, or use the AFS ntp stuff, because kerberos requires times on machines to be within a certain range to work (typically 5 minutes). -- XXX actually startup afsd without ntp service
This assumes that you have the heimdal kdc (or some other kdc, your choice) up and running and get use it to create principals.
You will need to create two principals: one for the AFS server itself, and at least one for someone who will have administrative rights in AFS.
My kerberos realm is TPROA.NET. Replace accordingly.
You will need to pick at least one principal in kerberos to have administrative rights in AFS. Typically, you would use some instance of your normal principal, i.e. user/admin or user/afs. I'm using tkula/afs@TPROA.NET since I'm using tkula/admin@TPROA.NET for kerberos stuff only (and this is what we do at work). You can do as you like. Remember that AFS is still based on v4 kerberos, so to it, your instance will look like tkula.afs (i.e. replace the / with a . -- XXX Is this still true?).
Creating my /afs instance. The -p tkula/admin flag to kadmin tells it to use that principal to do stuff in kadmin.
afs-1# kadmin -p tkula/admin kadmin> add tkula/afs Max ticket life [1 day]: Max renewable life [1 week]: Principal expiration time [never]: Password expiration time [never]: Attributes []: tkula/afs@TPROA.NET's Password: Verifying - tkula/afs@TPROA.NET's Password:
Current best practice is to create an AFS kerberos principal with the form afs/<cell name>@TPROA.NET, i.e. afs/tproa.net@TPROA.NET for the cell tproa.net. This way you can have multiple cells use the same kerberos realm (i.e. you could create a afs/test.tproa.net@TPROA.NET for the cell test.tproa.net).
See
http://grand.central.org/twiki/bin/view/AFSLore?topic=KerberosAFSInstall for details:
kadmin> add --random-key afs/tproa.net
Max ticket life [1 day]:unlimited
Max renewable life [1 week]:unlimited
Principal expiration time [never]:
Password expiration time [never]:
Attributes []:
kadmin> get afs/tproa.net
Principal: afs/tproa.net@TPROA.NET
Principal expires: never
Password expires: never
Last password change: never
Max ticket life: unlimited
Max renewable life: unlimited
Kvno: 1
Mkvno: 0
Policy: none
Last successful login: never
Last failed login: never
Failed login count: 0
Last modified: 2004-12-27 20:33:26 UTC
Modifier: tkula/admin@TPROA.NET
Attributes:
Keytypes(salttype[(salt-value)]): des-cbc-crc(pw-salt), des-cbc-md4(pw-salt), des-cbc-md5(pw-salt), des3-cbc-sha1(pw-salt)
No idea what the lifetimes should be, I'm just guessing here. Also note that I did not delete any of the keytypes --- it seems to work fine.
afs-1# ktutil -k /tmp/afskeytabfile.krb5 list /tmp/afskeytabfile.krb5: Vno Type Principal 1 des-cbc-crc afs/tproa.net@TPROA.NET 1 des-cbc-md4 afs/tproa.net@TPROA.NET 1 des-cbc-md5 afs/tproa.net@TPROA.NET 1 des3-cbc-sha1 afs/tproa.net@TPROA.NET
Convert the key:
afs-1# ktutil -v copy FILE:/tmp/afskeytabfile.krb5 AFSKEYFILE:/tmp/KeyFile copying FILE:/tmp/afskeytabfile.krb5 to AFSKEYFILE:/tmp/KeyFile copying afs/tproa.net@TPROA.NET, keytype des-cbc-crc, kvno 1 copying afs/tproa.net@TPROA.NET, keytype des-cbc-md4, kvno 1 copying afs/tproa.net@TPROA.NET, keytype des-cbc-md5, kvno 1 copying afs/tproa.net@TPROA.NET, keytype des3-cbc-sha1, kvno 1
Now, I had to make the file /usr/afs/etc/ThisCell contain my cell name. Someone else I know doesn't have to do this, so I suspect that some knob in krb5.conf configures this (or it looks for some other file location I have yet to find) but I am not sure which one.
Make the directory /usr/local/openafs/etc/openafs/server and drop /tmp/KeyFile there, chmod it to 700.
bos manages all other AFS processes. Start it:
cd /usr/local/openafs mkdir -p var/openafs/logs sbin/bosserver &
The bosserver will background and start up some other process to actually run, so you will see something like:
afs-1# sbin/bosserver & [1] 5040 afs-1# [1] Done sbin/bosserver
but if you ps and look for the bosserver it will be running.
bin/bos setcellname afs-1.tproa.net tproa.net -localauth
The -localauth flag bypasses the normal bos authentication stuff, and says "just read from KeyFile and use it", so as long as you are logged in as root on that machine and can read KeyFile, you should be able to run commands.
afs-1# bin/bos create afs-1.tproa.net buserver simple /usr/local/openafs/libexec/openafs/buserver -cell tproa.net -localauth afs-1# bin/bos create afs-1.tproa.net ptserver simple /usr/local/openafs/libexec/openafs/ptserver -cell tproa.net -localauth afs-1# bin/bos create afs-1.tproa.net vlserver simple /usr/local/openafs/libexec/openafs/vlserver -cell tproa.net -localauth
This creates the backup, protection and volume location servers on our first machine.
Here we skip the steps in the IBM Quick Start guide that creates the afs key, and admin entries, since we did that above and are not using the kaserver in AFS. You can verify that bos can list the keys, however:
afs-1# bin/bos listkeys afs-1.tproa.net -cell tproa.net -localauth key 1 has cksum 1337504214 Keys last changed on Mon Dec 27 15:39:20 2004. All done.
This is where you tell the pt server about your administrative account. The pts command, however, does not have a -localauth flag, and while you could use pt_util to munch the prdb file manually, I couldn't get pt_util to work, nor could I find any real explanation of what I shove to it. So, stop everything, kill bosserver, and restart it, temporarily, with -noauth:
afs-1# bin/bos shutdown afs-1.tproa.net -localauth afs-1# bin/bos status afs-1.tproa.net -localauth Instance buserver, temporarily disabled, currently shutdown. Instance ptserver, temporarily disabled, currently shutdown. Instance vlserver, temporarily disabled, currently shutdown. afs-1# ps ax | grep bosserver 15697 ?? S 0:00.08 sbin/bosserver afs-1# kill -HUP 15697 afs-1# ps ax | grep bosserver afs-1# sbin/bosserver -noauth & [1] 6927 afs-1# [1] Done sbin/bosserver -noauth afs-1# ps ax | grep bosserver 9363 ?? S 0:00.02 sbin/bosserver -noauth
Create the pts entries:
afs-1# bin/pts createuser -name tkula.afs -cell tproa.net -id 10137 -noauth User tkula.afs has id 10137 afs-1# bin/pts adduser tkula.afs system:administrators -cell tproa.net -noauth afs-1# bin/pts membership tkula.afs -cell tproa.net -noauth Groups tkula.admin (id: 10137) is a member of: system:administrators
Note that I used tkula.afs, not tkula/afs
Shutdown everything again, and restart bos to have localauth:
afs-1# bin/bos shutdown afs-1.tproa.net -localauth afs-1# bin/bos status afs-1.tproa.net -localauth Instance buserver, temporarily disabled, currently shutdown. Instance ptserver, temporarily disabled, currently shutdown. Instance vlserver, temporarily disabled, currently shutdown. afs-1# ps ax | grep bosserver 9363 ?? S 0:00.03 sbin/bosserver -noauth afs-1# kill -HUP 9363 afs-1# sbin/bosserver & [1] 12759 afs-1# [1] Done sbin/bosserver afs-1# ps ax | grep bosserver 6041 ?? S 0:00.02 sbin/bosserver
afs-1# bin/bos adduser afs-1.tproa.net tkula.afs -cell tproa.net -localauth afs-1# bin/bos listusers afs-1.tproa.net -cell tproa.net -localauth SUsers are: tkula.afs
XXX
From Tracy: You need to install the LWP threaded versions of the fileserver instead of the pthread version. The fileserver.LWP version is what you find in <openafs>/src/viced/fileserver, the pthread version is from <openafs>/src/tviced/fileserver. You have to do this because currently the pthread version of the fileserver doesn't quit properly, which makes bos kinda a pain to use to shutdown the fileserver.
afs-1# bin/bos create afs-1.tproa.net fs fs \
/usr/local/openafs/libexec/openafs/fileserver \
/usr/local/openafs/libexec/openafs/volserver \
/usr/local/openafs/libexec/openafs/salvager \
-cell tproa.net -localauth
afs-1# bin/bos status afs-1.tproa.net fs -long -localauth
Bosserver reports inappropriate access on server directories
Instance fs, (type is fs) currently running normally.
Auxiliary status is: file server running.
Process last started at Mon Dec 27 16:50:31 2004 (2 proc starts)
Command 1 is '/usr/local/openafs/libexec/openafs/fileserver'
Command 2 is '/usr/local/openafs/libexec/openafs/volserver'
Command 3 is '/usr/local/openafs/libexec/openafs/salvager'
The 'inappropriate access' thing is probably some directory permissions problems. XXX Look at these.
Mon Dec 27 21:31:59 2004: unhappy with /usr/local/openafs/var/openafs which is a dir that should have at least rights 700, at most rights 770 , owned by root
Make sure the fileserver process can see your /vicep partitions:
afs-1# sbin/vos listpart afs-1.tproa.net -localauth
The partitions on the server are:
/vicepa
Total: 1
afs-1# sbin/vos create afs-1.tproa.net /vicepa root.afs -cell tproa.net -localauth Volume 536870912 created on partition /vicepa of afs-1.tproa.net
Create the following script as /etc/rc.d/arlad (provided by
Tracy Di Marco White):
#!/bin/sh
#
# PROVIDE: arlad
# REQUIRE: beforemountlkm
. /etc/rc.subr
name="arlad"
rcvar=$name
command="/usr/arla/libexec/${name}"
command_args="-z /dev/nnpfs0"
start_precmd="/usr/arla/sbin/mount_nnpfs /dev/nnpfs0 /afs"
stop_postcmd="/sbin/umount /afs"
required_files="/dev/nnpfs0"
required_dirs="/afs"
load_rc_config $name
run_rc_command "$1"
Make this executable. You will also need to modify the paths to point to the proper location. Since eventually I'll get around to putting everything in some sane location, I've left it as is.
Edit /usr/local/arla/etc/CellServDB to add your cell. You can just glom on the bit in the server's version to the end:
>tproa.net #The People's Republic of Ames 209.234.76.70 #afs-1.tproa.net
In fact, it is best to do this, since the format of this file is a bit particular
Edit /usr/local/arla/etc/ThisCell to make your cell the default cell.
/usr/arla/bin/nnpfs_mod.o - nnpfs_mod /usr/arla/sbin/nnpfs_makedev /var/db/nnpfs_sym BEFOREMOUNT
mknod /dev/nnpfs0 c 165 0
Change:
lkm=YES
in /etc/rc.conf, and add
arlad=YES
in the same file.
Restart.
/usr/local/openafs/sbin/bosserver &
If your afs database servers (the ones listed in CellServDB) are also slave kdc (or a master one, for that matter), and you are running a heimdal kdc, you can get tokens directly by setting enable-kaserver in the [kdc] section of /etc/krb5.conf file. I think there is also a kaforwarder, which you run on your db servers to forward requests to your kdc -- XXX look for this
Once again, kinit looks in a particular spot to find ThisCell. I put mine in /usr/arla/etc/ThisCell.
XXX Really, find a krb5.conf configuration for this nonsense. Or merge the trees together so that stuff looks in sane places for stuff for both server and client.
cd /usr/local/openafs/
bin/fs setacl /afs system:anyuser rl
afs-1# sbin/vos create afs-1.tproa.net /vicepa root.cell Volume 536870915 created on partition /vicepa of afs-1.tproa.net afs-1# bin/fs mkmount /afs/tproa.net root.cell afs-1# bin/fs setacl /afs/tproa.net system:anyuser rl
bin/fs mkmount /afs/.tproa.net root.cell -rw
afs-1# sbin/vos addsite afs-1.tproa.net /vicepa root.afs Added replication site afs-1.tproa.net /vicepa for volume root.afs afs-1# sbin/vos addsite afs-1.tproa.net /vicepa root.cell Added replication site afs-1.tproa.net /vicepa for volume root.cell afs-1# bin/fs examine /afs File /afs (536870912.1.1) contained in volume 536870912 Volume status for vid = 536870912 named root.afs Current disk quota is 5000 Current blocks used are 4 The partition has 18455764 blocks available out of 18455810 afs-1# bin/fs examine /afs/tproa.net/ File /afs/tproa.net/ (536870915.1.1) contained in volume 536870915 Volume status for vid = 536870915 named root.cell Current disk quota is 5000 Current blocks used are 2 The partition has 18455764 blocks available out of 18455810 afs-1# sbin/vos release root.afs Released volume root.afs successfully afs-1# sbin/vos release root.cell Released volume root.cell successfully
Now you can add other stuff, see the AFS documentation on suggestions for that
http://www.openafs.org/pages/doc/QuickStartUnix/auqbg002.htm#ToC_91