user_manage: The All-Purpose Web Server User
Management Script

Description

For obscure reasons, there isn't a satisfactory remote tool for the
Apache Web server that allows authorized users to change their
passwords remotely. As a result, the onerous task of managing the
password and group files falls on the Webmaster.
user_manage was written to fill this need. In addition
to its basic role as a password changer, this script allows the
Webmaster to add, edit and delete users and groups, all via a
form-based interface.

user_manage handles most variants of the Apache "Basic"
and "Digest" authentication scheme. Using one simple interface, it
manages each of the following types of password and group files:

Human readable text files.

DBM database files.

Berkeley DB database files.

SQL databases.

In addition to running as a CGI script, user_manage can
be used from the command-line to manipulate users and groups,
replacing both dbmmanage and htpasswd with a
simple and consistent interface. Furthermore, by allowing you to
define security "realms" consisting of paired password and group
files, it avoids operator errors that often result in the incorrect
password file being modified.

Many of user_manage's functions will also work with other
Unix Web servers, including NCSA httpd, and the CERN server. There is
support for Netscape user (but not group) databases. This script
has not been tested with Windows NT servers.

System Requirements

This set of Perl modules, written by Doug MacEachern and
enhanced by myself, provides core
file manipulation routines. For your convenience, it is
included with this distribution. The modules must be installed
in the Perl library directory as described below.

This will unpack the distribution in a directory named
HTTPD-User-Manage-X.X, where X.X is the
current version number.

Install the HTTPD::* Modules

Change to the distribution directory
(HTTPD-User-Manage-X.X), and type the following:

% perl Makefile.PL
% make
% make test
% make install

This will copy the HTTPD modules to the correct location within the
Perl library directory. You may need to be root in order to do this.

The HTTPD modules are useful in their own right. During the
installation process, Perl will have created man pages for them which
you can read with the perldoc command. Read the manual
page for HTTPD::Realm first, then HTTPD::RealmManager.

Install the user_manage Script

Copy user_manage into your server's cgi-bin
directory, or another directory that allows CGI scripts. Make sure
that it is executable and that the top line points to the correct
location of Perl on your system, e.g.

#!/usr/local/bin/perl

Work Out Permissions for the Password and Group Files

If you plan to run user_manage as a CGI script, then
you'll need to worry about file permissions. When the script is
executed by the server, it runs with the server's effective user ID,
usually an unprivileged user such as "nobody". However, this user
usually does not have permission to write to the server's password and
group files. Thus the script will fail. In order for the script to
be able to update password and group files, not only must it have
write access to the files themselves, but to the directory that
contains them. This last requirement is because the script creates
lock and other temporary files within the directory.

Several workable configurations are possible:

Make user_manage Run as a SUID Script

Perl is capable of safely running scripts set-user-id (suid). When
the script runs, Perl temporarily changes to a different user ID and
executes the script. user_manage has been written to be
safe when running in this way.

Designate a directory that will hold the various password and group
files, for example /etc/httpd/security. Make it owned and writable by
a specially-designated "web administrator" account, for example "www".
Now, running as root, change the ownership of user_manage
to "www" and set its "s" bit:

In order for suid to work, this feature needs to be configured into
Perl when it is compiled. You may need to reinstall Perl if suid was
not enabled, particularly if Perl came preinstalled on your system.
There is also a bug in perl 5.003 when running under some versions of
OSF/1 (Digital Unix) that causes suid scripts to exit silently without
producing any output. To fix this bug, apply the patch located in the
file suid.patch included with this
distribution to the Perl source tree and recompile.

Make user_manage Run as a SGID Script

An alternative to running the script suid is to set the set-group-id
bit instead. This will cause the script to execute with the
permissions of its owning group. You can then arrange for the
password and group files to reside within a group-writable directory,
such as one belonging to a web administrators' "www" group.

Under this scheme it may be convenient to have the script create new
password and group files with group writable permissions. To do this,
change the script global $CREATE_MODE to "0664". (See Configuring the Script.

Keep the Password and Group Files in a Directory writable by
"nobody"

If you prefer not to run user_manage as a suid or
sgid script, you can place the password and group files in a directory
that is owned and writable by the Web server's effective user ID
and/or group. If you wish for the password files to be modifiable
from the command line, you should also make this directory group
writable by a group that you belong to.

While this strategy is effective, it has the disadvantage that it
gives the web server and all other CGI scripts the ability to modify
the password and group files. A remote user exploiting a security
hole in the server or one of its CGI scripts could then take advantage
of this fact to alter the password file.

Restrict Access to the Script

Because you don't want everyone in the world to view and edit
your server's password files, user_manage must be
placed under some kind of access restrictions. You can do this
either by placing the script in a restricted directory to use
the server's access restrictions, or by allowing the script to
do its own user authentication.

There are several global variables towards the top of the script that
will need to be modified to suit your site. Open the script with a
text editor and make the changes if need be.

$CONFIG_FILE

This variable is set to the full pathname of the script's realm configuration file. This file
holds the definitions of all the security realms at your site.
It's set to '/etc/httpd/conf/realms.conf' by default.
Change it according to your preferences (I like
to keep the config file in the same place as the other web
server configuration files).

This global defines the name of an administrator's group. When
a user who belong to this group invokes the script remotely, she
is granted special privileges to add, edit and remove users.
The default is "administrators". Set this to an empty string to
disable this feature entirely.

$PROTECT_ADMINS

This global controls whether changes to administrators' accounts
are restricted. When set to 1, the password of an administrator's
account may only be changed by himself. Membership in the
privileged $ADMIN_GROUP may only be granted or revoked by the
webmaster using conventional command line tools, but no longer by
this script. Administrators thus cannot deny each other access, and
they can't grant privileged access to others without the webmaster's
consent.

This is the default group to assign newly-created users to,
"users" by default. Set it to an empty string to disable
automatic group assignment.

$REQUIRE_ACCESS_CONTROL

This global affects the manner in which the script interacts
with Apache's access control facilities. Ordinarily the script
first checks the environment to determine
whether the user has already provided a user name and
password in order to access the script. If it finds this
to be the case, it takes the current user name directly from the
environment and allows the user to change her password. If access
to the script has not been restricted, it performs its own
access control by prompting the user for a valid user name and password.

With this variable set to a non-zero value, the script will
refuse to run at all unless it is in a password-protected
directory. Both GET and POST methods must be restricted.
Here's a typical excerpt from Apache's access.conf
file:

Placing the script under access control provides a small amount
of additional security at the cost of an extra layer of
complexity. It is ordinarily unnecessary.

$CREATE_MODE

These are the file permissions to use for newly-created password
and group files. By default, this variable is set to mode 0644,
which grants the owner read/write permission and others
read-only permission. If you wish to grant edit access to a
group of local users, such as designated web administrators, you
might want to change this value to 0664, to give the group
read/write access.

$STTY

When this script is run from the command line, it uses the
"stty" program to turn off command echo so that you can type in
passwords invisibly. The location of this program may differ from
system to system. If you encounter problems with this part of
the script, check the location of stty and correct the value of
this variable.

Rather than force you to remember the paths to individual password
files, user_manage uses a "security realm" scheme to
group related password and group files under meaningful nicknames.
Before you can use this scheme, you must create a configuration file
and save it in the location indicated by the $CONFIG_FILE global.

A sample configuration file, realms.conf is
included in this distribution. Its format is similar to that used by
Apache for its access control file. Blank lines and lines beginning
with the "#" character are ignored. The file should contain a series
of realm definitions, each beginning with a <Realm> directive
and ending with a </Realm> directive. For example:

This example shows the definitions for two security realms, one named
"main" and the other named "development". Each realm section contains
the three directives Users, Groups, and
Type:

Required Directives in the <Realm> Section

Directive

Example Parameters

Description

Users

/etc/httpd/passwd

Path to the user password file

Groups

/etc/httpd/group

Path to the group file

Type

DBM

Type of file to use

The Users and Groups directives should
contain absolute path names (except when using SQL databases for
authentication; see below). Relative path names
will be rejected. Type may be one of Text,
DBM, DB, or SQL. These
correspond to Apache's flat file, DBM file, DB file, and mSQL
authentication formats respectively. Please be careful that the user
and group files that you declare in Apache's access.conf file
correspond to the type declared in the user_manage
configuration file:

For the purposes of compatibility with future versions of Apache,
user_manage recognizes and works with other types of
DBM-like databases as well, including GDBM,
ODBM, and SDBM, as well as any SQL database
for which there is a Perl driver.

The first realm defined in the configuration file becomes the default,
unless otherwise specified by a Default directive (see the next section).

Other Configuration File Directives

In addition to the required realm directives, there are several optional
ones that customize the security realm in various ways. The complete
list of directives is given in the table below:

Full List of configuration file directives

Directive

Example Param

Description

Authentication

Basic

Authentication scheme

Database

www@capricorn.com

Location of SQL db

Default

none

Default realm

Driver

mSQL

DBI SQL driver

Fields

name age paid

Additional user fields

Groups

/etc/httpd/group

Path to group database

GroupType

SQL

Database type

Server

NCSA

Type of server

Type

DBM

Database type

Users

/etc/httpd/passwd

Path to user database

UserType

DB

Database type

Authentication

This directives specifies the type of authentication to use. It
can be either "Basic" or "Digest." Digest, part of the
HTTP/1.1 protocol, is much more secure than basic
authentication, but is implemented by few browsers currently.

Database

This directive is valid for SQL databases only and indicates where the
authentication database can be found. It should be in the format
database@host. If the hostname is omitted, "localhost" is
assumed. For mSQL databases, performance will be much better if
database and Web server are on the same machine because in this case,
the client and server use a Unix socket to communicate rather than a
TCP/IP socket.

Driver

For SQL databases only, this directive specifies what DBD (database
driver) module to use. It defaults to "mSQL". You can use any
database for which a DBD module is available. You must also, of
course, compile and configure the Web server to correctly use the
driver.

Default

If this directive is present, the current realm becomes the default to
use when no realm is explicitly indicated. If no section in the
configuration file contains this directive, the first defined realm
becomes the default. It is a fatal error for Default to appear in
more than one section.

Fields

This directive lists other fields that can be
found in the user table. These fields can then be read and set
by user_manage, both in its command line and CGI
script forms.

The Fields directive specifies a list of field names of the form
name[:type][width]. Only the name is
required. The field type and width are optional hints that help
user_manage format the field values correctly on
the Web page. The type can be one of "i", for an integer value,
"s" for a string
value and "f" for a floating point number. If not specified, the
field is assumed to be of type string. The field value must be an
integer.

In the example below we define three fields named "Name", "Age" and
"Paid". The first is a string value of default length. The second is
an integer. The third is a string of length one (it's assumed to be a
"Y" or "N"):

Fields Name Age:i Paid:s1

Many more fields may be present in SQL databases than are listed
in the Fields directive. The Fields directive just
tells the user_manage script which fields should be made visible to
the user interface.

Groups

Path to the file that holds the database of groups. Both
relative and absolute
paths are accepted, but absolute paths are preferred. This
directive has a special format when using SQL databases, see below.

GroupType

The type of the group database only, allowing you to use separate
types for the user and group databases. May be any of DBM, DB,
SQL, NDBM, GDBM, ODBM or SDBM. If not provided, the value of Type is
used instead.

Server

Web servers differ slightly in the format of the users and groups
databases. This directive indicates which server you are using.
Recognized values include "apache", "ncsa", "cern" and "netscape."
Example:

Server cern

If no server is specified, "apache" is assumed. If your server is not
on this list, try "ncsa".

Type

The database type. May be any of DBM, DB, SQL, NDBM, GDBM, ODBM
or SDBM. This directive applies to both the user and group
databases.

Users

Path to the file that holds the database of users, their
passwords and other information. Both relative and absolute
paths are accepted, but absolute paths are preferred. This
directive has a special format when using SQL databases, see below.

UserType

The type of the user database only, allowing you to use separate
types for the user and group databases. May be any of DBM, DB,
SQL, NDBM, GDBM, ODBM or SDBM. If not provided, the value of Type is
used instead.

user_manage can also communicate with various SQL
databases using Tim Bunce's DBI database interface (ODBC is
not currently supported, but a future version may do so).
The Apache Web server can use the inexpensive mSQL and MySQL databases for access control.
Future versions of Apache may be able to access other SQL databases as
well.

When using SQL databases, the Users and Groups directives in the
realms configuration file have a different format. Instead of
pointing to a physical file, these directives contain information
about what table and field the user, password and group information is
stored in. A typical Users directive looks like this:

Users table=users uid=name password=pass

The directive consists of three tags. The "table" tag designates the
database table that contains the user information. "uid" indicates
the field that contains the user name, and "password" names the field
that contains the user's password. For performance reasons, the user
name field should be declared the primary index.

The Groups directive has a similar structure:

Groups table=groups group=grp

"table" contains the name of the database table in which the group
information can be found. "group" contains the name of the field in
which the group name is stored. You do not have to provide a "uid"
tag in this case, because it is automatically inherited from the uid
field in the Users directive. If you do provide a "uid" tag, it must
be the same as the corresponding field in the users table (this is an
Apache limitation; not a user_manage limitation). Note
that ANSI SQL forbids you from using the word "group" as a field name.
Some databases, including mySQL will not allow you to create a table
with a column named "group."

In this example, agnes belongs to groups named "users," "authors,"
and "engineers." lstein belongs to "authors" and "administrators,"
while phillip and wanda each belong to "users" only.

You may use the same table for both users and their groups. However
if you want to assign a user to more than one group, then you must use
separate tables. This is because each user record must be unique in
the user table, whereas the same user will appear once in the group
table for each group that he or she belongs to. Another implication
of this is that the user ID field in the group table cannot be a
primary key, otherwise it would be forced to be unique.

As in the Fields directive, you can provide optional field widths for
each of the field declarations in the Users and Groups directives.
The field widths are used as hints by the CGI script version of
user_manage to format the fill-out form nicely. Follow
the field name with a colon and the field width, as in:

Users table=users uid=name:30 password=pass:13

There is no need to declare the field type since they must be strings,
but it doesn't hurt to do so.

You will also need to declare the location of the database and its DBI
driver using Database and Driver directive. If these directives do
not appear, they default to "www@localhost" and "mSQL" respectively.

Unlike other database types, user_manage does not
automatically create SQL database tables for you. You will need to
create the tables manually before you use user_manage for
the first time. If you wish, the user_manage "setup"
command will output the necessary SQL table creation commands for you.

This link invokes the user_manage script with the
single parameter realm set to the security realm you
wish to modify. If you don't provide this argument, the first realm
defined in the script's configuration file
will be assumed.

To create a button that does the same thing, adapt the following
fragment of HTML:

When the user selects this link, she will be presented with a page
similar to the one shown here, which prompts her to type in her name
and password.

Password prompt

After the script confirs that the user's name and correct password are
found in the password file, the user will be prompted to enter a new
password with a page like the one shown here:

New password prompt

The user enters her new password twice in order to avoid typing
errors. The script then confirms that the password was successfully
changed.

Password confirmation screen

If an error occurred while updating the password file, the user will
be given a generic error message. A more specific error message
containing diagnostic output will be found in the server's error log.

Managing Users Remotely

If the user who accesses the password changing script belongs to the
special group "administrators" (or whatever is set in the $ADMIN_GROUP global), a different
screen will appear similar to the screenshot shown here. This screen
contains a pop-up menu of all currently defined users (it turns into a
scrolling list when the number of user exceeds eight). It also
contains a blank textfield for adding entirely new users. From this
screen you can edit the passwords and groups of existing users, define
new users, and delete users entirely.

Administrator's user management screen

Adding a New User

To add a new user, type his or her user name into the text field
labeled New User and press Edit/Add. A screen like
this one will appear:

Adding a new user

The checkboxes labeled Set Groups contains all the groups
currently defined. Toggle all the groups that you wish the user to
belong to. If a group isn't already defined, you can type in its name
in the text field labeled Other:. Enter and reenter the
password for this user in the text fields labeled Password
Enter and Confirm. When you are satisifed, press the
Set Values button. A confirmation will appear at the top
of the page, and the script will display the "User Edit" page
described in the next section. You can re-edit the user (to define
new groups, for example), edit a different user, or add another new
user.

If any additional user fields are defined in the configuration file,
they will appear as a series of text fields in a section labeled
"Other Information". In the example shown here there are three
labeled "name", "age" and "paid."

If there are more than five groups defined, the list of checkboxes
will turn into a scrolling list in order to save space.

Editing an Existing User

To edit an existing user, just select the user from the user list and
press Edit/Add. This will bring up the page shown below.
The top portion of the screen contains a duplicate of the user
selection page, while the bottom portion contains the entry for the
selected user. Edit the user's assigned groups and/or passwords and
press Set Values. If you inadvertently make a change
that you do not want to keep, press Reset Values to
revert to the original values.

The screenshot below also shows how the popup menu of existing users
turns into a scrolling list when the number becomes large.

Editing an existing user

Deleting a User

Select the user you wish to delete from the user list and press the
Delete button.

Managing Users from the Command Line

user_manage can be used from the command line to
add, edit, and delete users. It's also a handy way to view the
contents of a DB or DBM file.

The syntax for invoking the script from the command line is:

user_manage [realm] commandargument1 argument2 argument3

The first argument to the command is the security realm. You can, if
you wish, omit the realm name entirely. If you do so, the script
will default to the first realm defined in its configuration file.
This is followed by a command indicating the action you want the
script to take. Following this are zero or more additional arguments,
the meaning of which depend on the command that's being issued.

Here's a summary of the commands and their arguments:

Commands recognized by user_manage

Command

Arguments

Description

setup

(none)

First-time setup
for a database

add

user password [group1,group2...] [field1=value1,field2=value2...]

Add or edit a user's password & groups

delete

user1 user2...

Delete named users

edit

user password [group1,group2...] [field1=value1,field2=value2...]

A synonym for "add"

realms

(none)

Concise list of all defined realms

group

user [group1,group2...]

Set the user's groups

group

user [field1=value1,field2=value2...]

Set the user's values

view

user1 user2...

View users' entries

view

(none)

View all users

format

(none)

Create a formatted
entry for access.conf

Initial Setup of a Security Realm

If you issue the "setup" command, user_manage will
create a new database for you containing a single
administrative user. It will prompt you for the administrator's name,
and the name of the administrative group. For example:

$ user_manage -r development setup
Pick a name for the administrative group [administrators]:
Pick a name for the administrative account: lstein
New password: ********
Re-type new password: *******
Added lstein to database development in group administrators.

If the database already exists, its other user entries will not be
erased. However, if the name you chose for the administrative account
already exists in the database it will be overwritten.

If you are using an SQL database, "setup" will not make any changes
directly to the database. Instead, it prints out a series of SQL
commands suitable for feeding to the database via a command-line or
batch tool. For example:

Viewing the Contents of a Security Realm

To get the listing of a security realm, issue the "view" command.
With no user names, this will dump out the entire contents of the passwd
and groups files in a convenient tabular file. To see the entries on
selected users, list their names. For example:

Note that the passwords appear in encrypted form. All passwords are
encrypted before being placed into the password
file. Once encrypted, there is no easy way of recovering the original
password.

Adding a New User

Issue the add command, giving the name of the new user, her
initial password, a comma-delimited list of groups you want her to
belong to, and zero or more field=value pairs. If you don't specify
any groups, the script will assign the user to the default group
defined by the $DEFAULT_GROUP global variable. If
you specify an empty group using any of "-", '' or "", the user will
be assigned to no groups.

If the realm has additional fields defined by a Fields directive, you
can set this information here by passing user_manage a
comma-delimited series of field name = value pairs. Be careful if
field values contain embedded white space or shell metacharacters. If
this is the case, you'll need to protect the entire series of
name=value pairs with quotation marks. Field names that are not
explicitly listed in the configuration file will be silently ignored.

If you do not enter the user name or password on the command line, you
will be prompted for it. It's actually safer not to type passwords
on the command line as they can be easily intercepted by other users
of the system.

In each of these examples, the realm has been omitted, allowing
user_manage to use the default realm.

Changing the Password of an Existing User

Use the add command as described above. You can also change
the group assignments at the same time. For your convenience, the
command edit is also available. However it is identical in
functionality to add.

Changing Group Assignments without Affecting the Password

If you wish to change a user's group assignments without resetting her
password, issue the group command with the user's name and
the list of groups to assign her to (separated either by white space
or by commas). Use any of "-", '' or "" to assign the user to
no groups.

Example:

$ user_manage group joseph users,authors,administrators
Groups set for joseph.

If you omit the user name or list of groups, the program will prompt
you for their values.

Changing User Information without Affecting the Password or Groups

If you wish to change a user's additional field information without
resetting the other information, issue the info command with
the user's name and a comma-delimited set of field_name=value pairs.
If you do not provide the field pairs, the program will prompt for
them. Fields that are not listed in a Fields directive in the
configuration file will be silently ignored.

Formatting a .htaccess or access.conf Entry

The format command will produce an access control entry
suitable for cutting and pasting into your Web server's access control
file. The basic entry allows access to all valid users. You should
modify this entry to reflect your intent.

This can be a real time-saver, particularly if you have trouble
remembering the spelling of all those directives!

Restrictions on Using user_manage from the Command Line

As explained earlier, file permissions are a big issue when running
user_manage as a CGI script. One solution is to make
the script run as set-group-id or set-user-id. When run from the
command line, the script will disable the sgid and suid bits and run
with the effective ID of the user launching it. This prevents local
users from wantonly changing the password files without appropriate
permissions.

The one exception to this rule is that if the user's UNIX login name
matches the name of a user who belongs to the "administrators" group,
the script will honor the sgid and suid bits. This mimics the
behavior of the script when it's running as a CGI program.

Bugs and Caveats

File Locking

Because user_manage is used as a CGI script, it is
possible that several users will try to update the password and group
files simultaneously. In order to prevent files from becoming
scrambled by this, the script implements a form of file locking that
allows only one user to have write access to the files at any time.

The file locking used by this script is based on the Unix fcntl()
call, which does not work correctly across NFS
mounted volumes. Because of this, the password and group files must
reside on one of the web server's local disks. For the same reason,
if you use user_manage from the command line, you
should do it from the web server's host machine.

If the script exits prematurely for some reason, it may leave lock
files lying around. These appear as files with the extension ".TMP"
in the directory holding the password and group files. You can safely
ignore these files or delete them. Their presence will not adversely
affect the script's operation. If the script consistently fails to
clean up lock files, please contact me. There's probably a bug.

Another thing to be aware of is that the Apache server itself doesn't
honor the file locks. If a user updates a password or group file at
the same time that Apache is trying to read it, the server may read
partial or outdated information. This may cause rare intermittent
user authentication errors. (The same is also true of the htpasswd
and dbmmanage scripts, which don't use any sort of locking at all).

Apache Doesn't Recognize Security Realms

Security realms are a good way of organizing your site's access
policies by keeping related password and group files together. Apache
would benefit from this concept, but currently doesn't. You're still
responsible for maintaining correct paths to the password and group
files in access.conf.

An Apache module that works with Doug MacEachern's mod_perl is in the
works to correct this deficiency

No Support for Netscape Group Files

The Netscape server stores group information using a scheme that is
somewhat more complex than other servers do. No support is currently
provided for these group files. You can still define an
administration group to manage users remotely with this tool, but the
groups are not honored by the Netscape server.

Don't Use This Script to Change /etc/passwd or /etc/group !!

The file formats are different. You will destroy your system if you
try it.

The Script Doesn't Handle NIS, POP or System Passwords

It's designed for updating passwords on the Web only. If you want to
allow users to change their POP, NIS or system passwords via a Web
interface, you're going to have to roll your own.

Modification History

Version 1.56 - May 6, 1999

Added support for authenticating to SQL databases.

Version 1.51 - December 30, 1997

Fixed support for the Netscape server and added caveats about
not supporting Netscape groups.

Version 1.50 - December 30, 1997

Fully integrated with Doug MacEachern's HTTPD::* modules.

Handles SQL databases.

Allows you to get and set additional field information.

Version 1.00 - January 15, 1997

First release

Version 1.01 - March 8, 1997

Changed group list from scrolling list to checkbox group if
small enough.

Turned off suid and sgid bits if run by unprivileged user from
command line.

Author and Distribution Information

user_manage was written by Lincoln D. Stein. It
can be freely distributed and modified, so long as this documentation
accompanies it and the following copyright statement is displayed in
the source code:

Copyright 1997-2000 Lincoln D. Stein. All rights reserved.
See the accompanying HTML file for usage and distribution
information. The master version can be found at:
http://www.genome.wi.mit.edu/ftp/pub/software/WWW/passwd/