3.6 Security

Security in Condor is a broad issue, with many aspects to consider.
Because Condor's main purpose is to allow users to run arbitrary code
on large numbers of computers, it is important to try to limit who can
access a Condor pool and what privileges they have when using the
pool. This section covers these topics.

There is a distinction between the
kinds of resource attacks Condor can prevent,
and the kids of attacks Condor cannot prevent.
Condor cannot
prevent security breaches of users that can elevate their privilege to
the root or administrator account.
Condor does not run user jobs in
sandboxes (standard universe jobs are a partial exception to this),
so Condor cannot prevent malicious actions by user jobs.
An example of a malicious job is
one that launches a distributed denial of service attack.
Condor assumes that users are trustworthy.
Condor can prevent unauthorized access to the Condor pool,
to help ensure that only trusted users have access to the pool.
In addition, Condor provides encryption and
integrity checking, to ensure that data (both Condor's data and user
jobs' data) has not been examined or tampered with.

Broadly speaking, aspects of security in
Condor may be categorized and described:

Authentication

A proper identification of a user is accomplished by the
process of authentication.
It attempts to distinguish between real users and impostors.
By default, Condor's authentication uses the user id (UID)
to determine identity,
but Condor can use a variety of authentication mechanisms,
including the stronger authentication methods Kerberos and GSI.

Authorization

Authorization specifies who is allowed
to do what.
Some users are allowed to submit jobs,
while other users are allowed Condor administration privileges.
Condor provides authorization on either a per-user or on
a per-machine basis.

Privacy

Condor may encrypt data sent across the network, which
(normally) prevents others from viewing the data.
With persistence and sufficient computing power,
decryption is possible.
Condor can encrypt the data sent for internal communication,
as well as user data, such as files and executables.
Encryption operates on network
transmissions: unencrypted data is stored on disk.

Integrity

The man-in-the-middle attack tampers with data
without the awareness of either side of the communication.
Condor's integrity check sends additional cryptographic data
to verify that network data transmissions have not been
tampered with.
Note that the integrity information is only for network
transmissions: data stored on disk does not have this integrity
information.

Users

Because authorization or capability in an operating system is
based on a process owner, which process owner (or user) a job or
Condor daemon is run as is critical.
The Condor system prefers that Condor daemons are run as the user
root, while other common operations are to be owned by a
separate user. This is usually the condor user, but any user
can be chosen.
See Section 3.6.10 for more detail.

3.6.1 Condor's Security Model

At the heart of Condor's security model is the notion that
the execution of individual commands are subject to access
checks.
For example, the job submission command (condor_ submit) can be restricted
in such a way that only a certain set of users are allowed to use it.
A user outside of this authorized list of users would be prevented from
submitting jobs for execution in Condor.

A possible solution, but one that does not scale well,
maintains a list of authorized users for
every individual operation that Condor supports.
Instead, Condor commands are
categorized into groups called access levels,
based on the type of operation performed.
The user of a specific commands must be authorized at
an access level.
For example,
the condor_ status command requires the READ access level.
Actions that accomplish management tasks,
such as shutting down or restarting of a daemon
require an ADMINISTRATOR access level.
See
Section 3.6.4
for a full list of Condor's access levels and their meanings.

There are two sides to any communication or command invocation in Condor.
One side is identified as the client,
and the other side is identified as the daemon.
The client is the party that initiates the command,
and the daemon is the party that processes the command and responds.
In some cases it is easy to distinguish the
client from the daemon, while in other cases it is not as easy.
Condor tools such as condor_ submit and condor_ config_val are client tools.
They send commands to daemons and act as clients in all their communications.
For example, the condor_ submit command communicates
with the condor_ schedd.
Behind the scenes, Condor daemons also communicate with each other;
in this case the daemon initiating the command plays the role of the client.
For instance,
the condor_ negotiator daemon acts as a client when contacting the
condor_ schedd daemon to initiate matchmaking.
Once a match has been found,
the condor_ schedd daemon acts as a client and contacts the
condor_ startd daemon.

All the basic ideas of Condor's security model
are implemented using configuration.
Commands in Condor are executed over TCP/IP network connections.
While network communication enables Condor to manage
resources that are distributed across an organization (or beyond),
it also brings in security challenges.
Condor must have ways of ensuring that commands are being sent
by trustworthy users.
Jobs are operating on sensitive data must be allowed to be
encrypted such that the data is not seen by outsiders.
Jobs may need assurance that data has not been tampered with.
These issues
can be addressed with Condor's authentication,
encryption, and integrity features.

3.6.2 Security Negotiation

Because of the wide range of environments and security demands necessary,
Condor must be flexible.
Configuration provides this flexibility.
The process by which Condor determines the security settings that will
be used when a connection is established is called
security negotiation.
Security negotiation's primary purpose is to determine which
of the features of authentication, encryption, and integrity checking
will be enabled for a connection.
In addition, since Condor supports multiple
technologies for authentication and encryption,
security negotiation also
determines which technologies is chosen for the connection.

Security negotiation is a completely separate process from
matchmaking, and should not be confused with any specific function of
the condor_ negotiator daemon.

3.6.2.1 Configuration

The macro names that determine what features will be used during
client-daemon communication follow the pattern:

SEC_<context>_<feature>

The <feature>
portion of the macro name determines which security feature's
policy is being set.
<feature> may be any one of

AUTHENTICATION
ENCRYPTION
INTEGRITY
NEGOTIATION

The <context> component of the security policy macros can be
used to craft a fine-grained security policy based on the type of
communication taking place.
<context> may be any one of

Security negotiation resolves various client-daemon combinations
of desired security features in order to set a policy.

As an example, consider Frida the scientist.
Frida wants to avoid authentication when possible.
She sets

SEC_DEFAULT_AUTHENTICATION = OPTIONAL

The condor_ schedd that she would like to submit to, however, is
operated by a security-conscious system administrator who dutifully
sets:

SEC_DEFAULT_AUTHENTICATION = REQUIRED

When Frida submits her jobs, Condor's security negotiation will
determine that authentication will be used and allows the command to
continue.

Table 3.6.2 shows how security negotiation
resolves various client-daemon combinations of security feature policy
settings.
Within the table, Yes means the feature will be
used during communication.
No means it will not.
Fail means that the policy settings are incompatible and the communication
cannot continue.
Frida's (client) setting of
OPTIONAL along with the condor_ schedd's (daemon) setting of
REQUIRED results in authentication being enabled for their
communications.

Table 3.1:
How negotiation resolves whether a given security
feature will be used.

Daemon Setting

NEVER

OPTIONAL

PREFERRED

REQUIRED

NEVER

No

No

No

Fail

Client

OPTIONAL

No

No

Yes*

Yes*

Setting

PREFERRED

No*

Yes

Yes

Yes

REQUIRED

Fail

Yes

Yes

Yes

*These combinations will result in failure for the
NEGOTIATION feature

It is important to note that these features are not completely
independent of each other. In particular, no other security features
are possible unless NEGOTIATION is used. In addition, neither
ENCRYPTION or INTEGRITY may be used on a connection
unless AUTHENTICATION is enabled (since authentication is
needed to provide a secure key exchange).

Setting SEC_CLIENT_<feature> will determine the policy for all
outgoing commands. Policy for incoming commands (i.e for the daemon
side) takes a more fine-grained approach that allows different
settings depending on the access level of the command being
received. For example, it may be desirable to have all administrative
actions (ADMINISTRATOR=level commands) require authentication
but not be so strict regarding inquiries on pool status
(READ-level commands). Such a policy could include the
settings:

Finally, the DEFAULT value for <context> will be checked
by Condor if none of the specific policy settings (CLIENT, READ,
WRITE, etc.) are present. This makes it possible to set a baseline
policy with a single macro setting.

3.6.2.2 Configuration for the
Negotiation of Security Methods

Deciding what security features will be enabled for a connection is
just one part of security negotiation. Given a specific feature,
Condor can support a variety of methods, or technologies, for
enabling that feature. Currently, authentication and encryption both
support multiple methods.

Since multiple protocols are available for the authentication and
encryption features of Condor, communicating daemons must have a way
to decide what protocol to use. This is decided as part of security
negotiation.

Configuration macros that determine the policy used for negotiating
methods to use for authentication and encryption, respectively, are of
the forms:

SEC_<context>_AUTHENTICATION_METHODS
SEC_<context>_CRYPTO_METHODS

The <context> part has the same meaning as it does for the
feature policy macros discussed above, allowing policy to be tailored
depending on the type of communication.

These macros can be set to a comma- or space-delimited list of
possible methods to use. See the individual sections on authentication
(3.6.3) and encryption
(3.6.5) for all possible names.

For example, a client may be configured with:

SEC_CLIENT_AUTHENTICATION_METHODS = FS, GSI

and a daemon the client is trying to contact with:

SEC_DEFAULT_AUTHENTICATION_METHODS = GSI

Negotiation will determine that GSI authentication is the only
compatible choice. If there are multiple compatible authentication
methods, negotiation will determine the list of acceptable methods and
they will be tried in order until one succeeds. The next section will
discuss authentication in more detail.

3.6.2.3 Access Level Descriptions

Authorization is granted based on specified access levels.
Access levels are granted for users by configuration settings.
The following describes the various access levels provided
by Condor.

READ

This access level
access can obtain or read information about Condor.
Examples that require only READ access are
viewing the status of the pool with condor_ status,
checking a job queue with condor_ q,
or viewing user priorities with condor_ userprio.
READ access does not allow any
changes, and it does not allow job submission.

WRITE

This access level is
required to send (write) information to Condor. Examples that
require WRITE access are job submission with
condor_ submit and advertising a machine so it appears in the pool
(this is usually done automatically by the condor_ startd daemon).
Note that WRITE access does not imply READ access.
They are separate access levels, and a user may be given
WRITE access without READ access.

ADMINISTRATOR

This
access level has additional Condor
administrator rights to the pool. It includes the ability to
change user priorities (with the command condor_ userprio -set),
as well as the ability to turn Condor on and off
(as with the commands condor_ on and condor_ off).

CONFIG

This access level is
required to modify a daemon's configuration using
the condor_ config_val command.
By default, this level of access can
change any configuration parameters of a Condor pool,
except those specified in
the condor_config.root configuration file.

OWNER

This level of access is
required for commands that the owner of a machine (any local user)
should be able to use, in addition to the Condor administrators.
An example that requires the OWNER access level is
the condor_ vacate command.
The command causes the condor_ startd daemon to vacate any
Condor job currently running on a machine.
The owner of that machine should be able to cause the removal
of a job running on the machine.

NEGOTIATOR

This
access level is used specifically to verify that commands are
sent by the condor_ negotiator daemon.
The condor_ negotiator daemon runs on the central manager of
the pool.
Commands requiring this access
level are the ones that tell the condor_ schedd daemon to begin
negotiating, and those that tell an available condor_ startd daemon
that it has been matched to a condor_ schedd with jobs to run.

3.6.2.4 Example of Daemon-Side Security Configuration

A configuration file is provided when Condor is installed.
No security features are enabled within the configuration as
distributed.
Included as comments within the configuration file is an example
suggesting settings that enable security features.
Here is that example of the daemon-side portion.

This set of configuration macros forces security features
to be used at all times.
All communication is authenticated with Kerberos, unless the client
does not use Kerberos, but supports File System (FS) authentication,
in which case FS authentication is used.
All communication is both encrypted and integrity checked to make sure
that messages are not modified or corrupted.
The encryption is preferably with triple DES, but Blowfish will be
used if the client does not use 3DES, but does use Blowfish.

Security negotiation configuration variables are not included in this
suggested configuration.
The default (when undefined) will be to set

SEC_DEFAULT_NEGOTIATION = OPTIONAL

Note that this example configuration requires that all Condor daemons be
version 6.3.3 or later, since previous versions will not have
the ability to do secure communication.

As an example, the macro defined in the configuration file
for a daemon as

SEC_WRITE_AUTHENTICATION = REQUIRED

signifies that the daemon must authenticate the client for
any communication that requires the WRITE access level.
If the daemon's configuration contains

SEC_DEFAULT_AUTHENTICATION = REQUIRED

and does not contain any other security configuration for
AUTHENTICATION, then this default defines the daemon's needs
for authentication over all access levels.
Where a specific macro is present, its value takes
precedence over any default given.

If authentication is to be done, then the communicating parties
must negotiate a mutually acceptable method of
authentication to be used.
A list of acceptable methods may be provided by the client, using the
macros

SEC_DEFAULT_AUTHENTICATION_METHODS
SEC_CLIENT_AUTHENTICATION_METHODS

A list of acceptable methods may be provided by the daemon, using the
macros

The methods are
given as a comma-separated list of acceptable values.
These variables list the authentication methods that are available
to be used.
The ordering of the list gives preference;
the first item in the list indicates the highest preference.
The values will be

GSI
KERBEROS
PASSWORD
FS
FS_REMOTE
NTSSPI
CLAIMTOBE
ANONYMOUS

As an example, the macro

SEC_DEFAULT_AUTHENTICATION_METHODS = KERBEROS, NTSSPI

indicates that either Kerberos or Windows authentication may be used,
but Kerberos is preferred over Windows.
Note that if the client and daemon agree that multiple authentication
methods may be used, then they are tried in turn. For instance, if
they both agree that Kerberos or NTSSPI may be used, then Kerberos
will be tried first, and if there is a failure for any reason, then
NTSSPI will be tried.

By default, all of the SEC_<access-level>_AUTHENTICATION are
OPTIONAL,
except for operations which modify the job queue (such as
condor_ q and condor_ rm) which are REQUIRED.

By default on Unix, the authentication methods are FS, KERBEROS,
GSI, while on Windows they are NTSSPI, KERBEROS, GSI.

3.6.3.1 GSI Authentication

The GSI (Grid Security Infrastructure) protocol provides
an avenue for Condor to do
PKI-based (Public Key Infrastructure) authentication using X.509
certificates.
The basics of GSI are well-documented elsewhere, such as
http://www.globus.org/.

A simple introduction to this type of authentication
defines Condor's use of terminology,
and it illuminates the needed items that Condor must access to
do authentication.
Assume that
A authenticates to B.
In this example, A is the client, and B is the server.
For Condor's purposes, both the client and the server
will either be a daemon
or a user (running a Condor command such as condor_ status).
This example's one-way authentication implies that B
is verifying the identity of A,
using the certificate A provides,
together with B's own set of trusted CAs (Certification Authorities).
Client A provides its certificate (or proxy) to server B.
B does two things:
B checks that the certificate is valid,
and B checks to see that the CA that signed A's certificate
is one that B trusts.

For this specific authentication protocol,
an X.509 certificate is required.
Files with predetermined names hold a certificate,
a key, and optionally, a proxy.
A separate directory has one or more files, each of which contains
a trusted CA.

Allowing Condor to do this GSI authentication
requires knowledge of the locations of
the client-side certificate and the server-side list of
trusted CAs.
For a Condor daemon, these locations are determined
by configuration or by default locations.
For a user, these locations are determined by the
pre-set values of environment variables or by default locations.

Server-side Configuration

For a Condor daemon, the certificate may be a single host certificate,
and all Condor daemons on the same machine may share the same certificate.
In some cases, the certificate can also be copied to other machines,
where local copies are necessary.
This can only be done in cases where a single host certificate can
match multiple host names, something that is beyond the scope of this
manual.
The certificates must be protected by access rights to files,
since the password file is not encrypted.

The specification of the location of the necessary files
through configuration uses the following precedence.

Configuration variable GSI_DAEMON_DIRECTORY gives the complete
path name to the directory that contains the certificate, key,
and directory with trusted CAs.
Condor uses this directory as follows in its construction of the following
configuration variables:

If the GSI_DAEMON_DIRECTORY is not defined,
or when defined,
the location may be overridden with specific configuration
variables that specify the complete path and filename of
the certificate with

GSI_DAEMON_CERT

the key with

GSI_DAEMON_KEY

a proxy with

GSI_DAEMON_PROXY

the complete path to the directory containing the list of trusted CAs with

GSI_DAEMON_TRUSTED_CA_DIR

The default location assumed is /etc/grid-security.
Note that this implemented by setting the value of
GSI_DAEMON_DIRECTORY.

Here is an example portion of the configuration file that would
enable and require GSI authentication,
along with a minimal set of other variables to make it work.
Note that the last entry (GSI_DAEMON_NAME) in this example
must be on a single line;
this example is broken onto two lines for formatting reasons.

The
SEC_DEFAULT_AUTHENTICATION macro specifies that
authentication is required for all communications.
This single macro covers all communications, but could be
replaced with a set of macros that require authentication for
only specific communications.

In this example, only the GSI method can be used.
If another methods is acceptable (perhaps KERBEROS, then
placing this method first within the list will cause Condor to give
preference to this method over GSI, or vice-versa.

The macro GSI_DAEMON_DIRECTORY is specified
to give
Condor a single place to find the daemon's certificate.
This path may be a directory or a shared file system such as AFS.
Alternatively, this path name can point to
local copies of the certificate stored
in a local file system.

User side Set Up

The user specifies the location of a certificate, proxy, etc.
in one of two ways:

Environment variables give the location of necessary items.

X509_USER_PROXY gives the path and file name of the proxy.
This proxy will have been created using the grid-proxy-init program,
which will place the proxy in the /tmp
directory with the file name being determined by the format:

/tmp/x509up_uXXXX

The specific file name is given by substituting the XXXX
characters with the UID of the user.
Note that when a valid proxy is used, the certificate and key locations
are not needed.

X509_USER_CERT gives the path and file name of the
certificate. It is also used if a proxy location has been checked,
but the proxy is no longer valid.

X509_USER_KEY gives the path and file name of the
key. Note that most keys are password encrypted, such that knowing
the location could not lead to using the key.

X509_CERT_DIR gives the path to the directory
containing the list of trusted CAs.

Without environment variables to give locations of necessary
certificate information,
Condor uses a default directory for the user.
This directory is given by

$(HOME)/.globus

When a daemon acts as the client within authentication,
the daemon needs a listing of those from which it
will accept certificates.

The macro GSI_DAEMON_NAME configuration macro
provides daemons with a distinguished name to use for
X.509 authentication.
This name is specified with the following format

GSI_DAEMON_NAME = /C=?/O=?/O=?/OU=?/CN=<daemon_name@domain>

A complete example that has the question marks filled in and the
daemon's user name filled in is given in the
example configuration above.

Condor will also need a way to map an X.509 distinguished
name to a Condor user id.
This is done in an administrator-maintained file called an X.509 map file,
mapping from X509 Distinguished Name (DN) to Condor user id.
It is similar to a Globus grid map file except that it is only used for
mapping to a user id, not for authorization.
Information about authorization can be found in
Section 3.6.4.
Entries (lines) in the file each contain two items.
The first item in an entry is the
X.509 certificate subject name, and it is enclosed in quotes
(using the character ").
The second item is the Condor user id.
The two items in an entry are separated by tab or space character(s).
Here is an example of an entry in an X.509 map file.
Entries must be on a single line; this example is broken
onto two lines for formatting reasons.

Condor finds the map file in one of three ways.
If the configuration variable GRIDMAP is defined,
it gives the full path name to the map file.
When not defined,
Condor looks for the map file in

$(GSI_DAEMON_DIRECTORY)/grid-mapfile

If GSI_DAEMON_DIRECTORY is not defined,
then the third place Condor looks for the map file is given by

/etc/grid-security/grid-mapfile

3.6.3.2 Kerberos Authentication

If Kerberos is used for authentication, then
the configuration variable
KERBEROS_MAP_FILE
is used to define a path to an administrator-maintained file that
contains Kerberos domain (called a realm) to Condor UID domain mapping.
The configuration syntax is

KERBEROS_MAP_FILE = /path/to/etc/condor.kmap

Lines within the map file have the syntax

KERB.REALM = UID.domain.name

Here are two lines from a map file to use as an example:

CS.WISC.EDU = cs.wisc.edu
ENGR.WISC.EDU = ee.wisc.edu

If a KERBEROS_MAP_FILE
configuration variable is defined and set,
then all permitted realms must be explicitly mapped.
If no map file is specified, then Condor assumes that the
Kerberos realm is the same as the Condor UID domain.

The configuration variable
CONDOR_SERVER_PRINCIPAL
defines the name of a Kerberos principal.
If CONDOR_SERVER_PRINCIPAL is not defined,
then the default value used is "host".
A principal specifies a unique name to which a set of credentials
may be assigned.

Condor takes the specified (or default) principal and appends
a slash character, the host name, an '@' (at sign character),
and the Kerberos realm.
As an example, the configuration

CONDOR_SERVER_PRINCIPAL = condor-daemon

results in Condor's use of

condor-daemon/the.host.name@YOUR.KERB.REALM

as the server principal.

Here is
an example of configuration settings that use Kerberos for
authentication and require authentication of all communications
of the write or administrator access level.

Kerberos authentication requires access to various files that
usually are only accessible by the root user.
At this time,
the only supported way to use KERBEROS authentication is to start
daemons Condor as root.

3.6.3.3 Password Authentication

The password method provides mutual authentication through the use of
a shared secret. This is often a good choice when strong security is
desired,
but an existing Kerberos or X.509 infrastructure is not in place.
Password authentication is available on both Unix and Windows.
It currently can only be used for daemon-to-daemon authentication.
The shared secret in this context is referred to as
the pool password.

Before a daemon can use password authentication, the pool password
must be stored on the daemon's local machine.
On Unix, the password
will be placed in a file defined by the configuration variable
SEC_PASSWORD_FILE. This file will be accessible only by the
UID that Condor is started as. On Windows, the same secure password
store that is used for user passwords will be used for the pool
password (see section 6.2.3).

will prompt for the pool password and store it on the local machine,
making it available for daemons to use for authentication. The
condor_ master must be running for this command to work.

In addition, storing the pool password to a given machine requires
CONFIG-level access. For example, if the pool password should
only be set locally, and only by root, the following would be placed in
the global configuration file.

ALLOW_CONFIG = root@mydomain/$(IP_ADDRESS)

It is also possible to set the pool password remotely, but this is
recommended only if it can be done over an encrypted channel. This is
possible on Windows, for example, in an environment where common
accounts exist across all the machines in the pool. In this case,
ALLOW_CONFIG can be set to allow the Condor administrator (who
in this example has an account condor common to all machines in
the pool) to set the password from the central manager as follows.

ALLOW_CONFIG = condor@mydomain/$(CONDOR_HOST)

The Condor administrator then executes

condor_store_cred -c -n host.mydomain add

from the central manager to store the password to a given machine.
Since the condor account exists on both the central manager and
host.mydomain, the NTSSPI authentication method can be used to
authenticate and encrypt the connection. condor_ store_cred will
warn and prompt for cancellation, if the channel is not encrypted for
whatever reason (typically because common accounts do not exist or
Condor's security is misconfigured).

When a daemon is authenticated using a pool password, its security
principle is condor_pool@$(UID_DOMAIN), where
$(UID_DOMAIN) is taken from the daemon's configuration. The
ALLOW_DAEMON and ALLOW_NEGOTIATOR configuration
variables for authorization
should restrict access using this name. For example,

This configuration allows remote DAEMON-level and
NEGOTIATOR-level access, if the pool password is known. Local
daemons authenticated as condor@mydomain are also allowed
access. This is done so local authentication can be done using
another method such as FS.

3.6.3.4 File System Authentication

This form of authentication utilizes the ownership of a file
in the identity verification of a client.
A daemon authenticating a client requires the client to write
a file in a specific location (/tmp).
The daemon then checks the ownership of the file.
The file's ownership verifies the identity of the client.
In this way, the file system becomes the trusted authority.
This authentication method is only appropriate for clients and daemons
that are on the same computer.

3.6.3.5 File System Remote Authentication

Like file system authentication,
this form of authentication utilizes the ownership of a file
in the identity verification of a client.
In this case,
a daemon authenticating a client requires the client to write
a file in a specific location,
but the location is not restricted to /tmp.
The location of the file is specified by the configuration
variable FS_REMOTE_DIR.

3.6.3.6 Windows Authentication

This authentication is done only among Windows machines using
a proprietary method.
The Windows security interface SSPI is used to enforce NTLM
(NT LAN Manager).
The authentication is based on challenge and response, using the user's
password as a key.
This is similar to Kerberos.
The main difference
is that Kerberos provides an access token that typically grants
access to an entire network, whereas NTLM authentication only
verifies an identity to one machine at a time.
NTSSPI is best-used in a way similar to file system authentication in
Unix, and probably should not be used for authentication between two
computers.

3.6.3.7 Claim To Be Authentication

Claim To Be authentication accepts any identity claimed by the client.
As such, it does not authenticate.
It is included in Condor and in the list of authentication methods
for testing purposes only.

3.6.3.8 Anonymous Authentication

Anonymous authentication causes authentication to be skipped entirely.
As such, it does not authenticate.
It is included in Condor and in the list of authentication methods
for testing purposes only.

3.6.4 Authorization

Authorization protects resource usage by granting or denying
access requests made to the resources.
It defines who is allowed to do what.

Authorization is defined in terms of users.
An initial implementation provided authorization
based on hosts (machines), while the current implementation
relies on user-based authorization.
Section 3.6.8
on Setting Up IP/Host-Based Security in Condor describes the
previous implementation.
This IP/Host-Based security still exists, and it can be used,
but significantly stronger and more flexible
security can be achieved with the newer
authorization based on fully qualified user names.
This section discusses user-based authorization.

Unlike authentication, encryption, and integrity checks,
which can be configured by both client and server,
authorization is used only by a server.
The authorization portion of the security of a Condor pool is
based on a set of configuration macros.
The macros list which user/daemon will be authorized
to issue what request given a specific access level.

These configuration macros define a set of users that will be
allowed to (or denied from) carrying out various Condor commands.
Each access level may have its own list of authorized users.
A complete list of the authorization macros:

Each macro is defined by a comma-separated list of fully qualified
users.
Each
fully qualified user
is described using the following format:

username@domain/hostname

The information to the left of the slash character describes
a user within a domain.
The information to the right of the slash character describes
one or more machines from which the user would be issuing a command.
This host name may take the form of either a fully qualified host name
of the form

bird.cs.wisc.edu

or an IP address
of the form

128.105.128.0

An example is

zmiller@cs.wisc.edu/bird.cs.wisc.edu

Within the format, wild card characters (the asterisk, *) are allowed.
The use of wild cards is limited to one wild card on either side
of the slash character.
A wild card character used in the host name is further limited
to come at the beginning of a fully qualified host name
or at the end of an IP address.
For example,

*@cs.wisc.edu/bird.cs.wisc.edu

refers to any user that comes from cs.wisc.edu,
where the command is originating from the machine
bird.cs.wisc.edu.
Another valid example,

zmiller@cs.wisc.edu/*.cs.wisc.edu

refers to commands coming from any machine within the
cs.wisc.edu domain, and issued by zmiller.
A third valid example,

*@cs.wisc.edu/*

refers to commands coming from any user within the
cs.wisc.edu domain
where the command is issued from any machine.
A fourth valid example,

*@cs.wisc.edu/128.105.*

refers to commands coming from any user within the
cs.wisc.edu domain
where the command is issued from machines within the network that match
the first two octets of the IP address.

If the set of machines is specified by an IP address,
then further specification using a net mask
identifies a physical set (subnet) of machines.
This physical set of machines is specified using the form

network/netmask

The network is an IP address.
The net mask takes one of two forms.
It may be a decimal number which refers to the number of leading
bits of the IP address that are used in describing a subnet.
Or, the net mask may take the form of

a.b.c.d

where a,
b,
c, and
d
are decimal numbers that each specify an 8-bit mask.
An example net mask is

255.255.192.0

which specifies the bit mask

11111111.11111111.11000000.00000000

A single complete example of a configuration variable that uses
a net mask is

ALLOW_WRITE = joesmith@cs.wisc.edu/128.105.128.0/17

User joesmith within the
cs.wisc.edu domain is given write authorization
when originating from machines that match their leftmost
17 bits of the IP address.

This flexible set of configuration macros could used to define
conflicting authorization.
Therefore, the following protocol defines the precedence of the
configuration macros.

1. DENY_* macros take precedence over ALLOW_* macros
where there is a conflict.
This implies that if a specific user is both denied and granted authorization,
the conflict is resolved by denying access.

2. If macros are omitted, the default behavior is to grant
authorization for every user.

3.6.4.1 Example of Authorization Security Configuration

An example of the configuration variables for the user-side
authorization is derived from the necessary access levels
as described in
Section 3.6.2.

This example configuration presumes that the condor_ collector
and condor_ negotiator daemons are running on the same machine.

This example configuration authorizes
any user in the
cs.wisc.edu domain to
carry out a request that requires the
READ access level
from any machine.
Any user in the
cs.wisc.edu domain may
carry out a request that requires the
WRITE access level
from any machine in the
cs.wisc.edu domain.
Only the user called condor-admin may
carry out a request that requires the
ADMINISTRATOR access level
from any machine in the
cs.wisc.edu domain.
The administrator, logged into any machine within
the cs.wisc.edu domain is authorized at the
CONFIG access level.
Only the negotiator daemon, running as
condor on the machine defined by the
NEGOTIATOR_HOST macro is authorized
with the
NEGOTIATOR access level.
And, the last line of the example presumes that there is a
user called condor, and that the daemons have all been started
up as this user.

In the local configuration file for each host, the host's
owner should be authorized
as the owner of the machine.
An example of the entry in the local configuration file:

ALLOW_OWNER = username@cs.wisc.edu/hostname.cs.wisc.edu

In this example the owner has a login of
username, and the machine's name is represented by
hostname.

3.6.5 Encryption

Encryption provides privacy support between two communicating parties.
Through configuration macros, both the client and the daemon
can specify whether encryption is required for further communication.

The client uses one of two macros to enable or disable encryption:

SEC_DEFAULT_ENCRYPTION
SEC_CLIENT_ENCRYPTION

For the daemon, there are seven macros to enable or disable encryption:

As an example, the macro defined in the configuration file
for a daemon as

SEC_CONFIG_ENCRYPTION = REQUIRED

signifies that any communication that changes a daemon's configuration
must be encrypted.
If a daemon's configuration contains

SEC_DEFAULT_ENCRYPTION = REQUIRED

and does not contain any other security configuration for
ENCRYPTION, then this default defines the daemon's needs
for encryption over all access levels.
Where a specific macro is present, its value takes
precedence over any default given.

If encryption is to be done, then the communicating parties
must find (negotiate) a mutually acceptable method of
encryption to be used.
A list of acceptable methods may be provided by the client, using the
macros

SEC_DEFAULT_CRYPTO_METHODS
SEC_CLIENT_CRYPTO_METHODS

A list of acceptable methods may be provided by the daemon, using the
macros

The methods are
given as a comma-separated list of acceptable values.
These variables list the encryption methods that are available
to be used.
The ordering of the list gives preference;
the first item in the list indicates the highest preference.
Possible values are

3DES
BLOWFISH

3.6.6 Integrity

An integrity check assures that the messages between communicating parties
have not been tampered with.
Any change, such as addition, modification, or deletion can
be detected.
Through configuration macros, both the client and the daemon
can specify whether an integrity check is required of further communication.

The client uses one of two macros to enable or disable an integrity check:

SEC_DEFAULT_INTEGRITY
SEC_CLIENT_INTEGRITY

For the daemon, there are seven macros to enable or disable an integrity check:

As an example, the macro defined in the configuration file
for a daemon as

SEC_CONFIG_INTEGRITY = REQUIRED

signifies that any communication that changes a daemon's configuration
must have its integrity assured.
If a daemon's configuration contains

SEC_DEFAULT_INTEGRITY = REQUIRED

and does not contain any other security configuration for
INTEGRITY, then this default defines the daemon's needs
for integrity checks over all access levels.
Where a specific macro is present, its value takes
precedence over any default given.

A signed MD5 checksum is currently the only available method
for integrity checking.
Its use is implied whenever integrity checks occur.
If more methods are implemented, then there will be further
macros to allow both the client and the daemon to specify
which methods are acceptable.

3.6.7 Security Sessions

To set up and configure secure communications in Condor,
authentication, encryption, and integrity checks can be used.
However, these come at a cost: performing strong authentication can
take a significant amount of time, and generating the cryptographic
keys for encryption and integrity checks can take a significant amount
of processing power.

The Condor system makes many network connections between different
daemons.
If each one of these was to be authenticated,
and new keys were generated for each connection,
Condor would not be able to scale well.
Therefore, Condor uses the concept of sessions to cache
relevant security information for future use and greatly speed up the
establishment of secure communications between the various Condor
daemons.

A new session is established the first time a connection is made from one daemon to another.
Each session has a fixed lifetime after which it will expire and
a new session will need to be created again.
But while a valid session exists, it can be re-used as many times as
needed, thereby preventing the need to continuously re-establish secure connections.
Each entity of a connection will have access to a session key that proves the
identity of the other entity on the opposing side of the connection.
This session key is exchanged securely using
a strong authentication method, such as Kerberos or GSI.
Other authentication methods, such as NTSSPI,
FS_REMOTE, CLAIMTOBE, and
ANONYMOUS, do not support secure key exchange.
An entity
listening on the wire may be able to impersonate the client or server
in a session that does not use a strong authentication method.

Establishing a secure session requires that either the encryption or the integrity options be enabled.
If the encryption capability is enabled, then the session will be restarted using the session key
as the encryption key.
If integrity capability is enabled, then the checksum includes the session key even
though it is not transmitted.
Without either of these two methods enabled,
it is possible for an
attacker to use an open session to make a connection to a daemon and
use that connection for nefarious purposes.
It is strongly recommended that if you have authentication turned
on, you should also turn on integrity and/or encryption.

The configuration parameter SEC_DEFAULT_NEGOTIATION will allow
a user to set the default level of secure sessions in Condor.
Like other security settings, the possible values for this parameter can be
REQUIRED, PREFERRED, OPTIONAL,
or NEVER.
If you disable sessions and you have authentication turned
on, then most authentication (other than commands like
condor_ submit) will fail because Condor requires sessions when you
have security turned on.
On the other hand, if you are not using strong security in Condor, but
you are relying on the default host-based security, turning off
sessions may be useful in certain situations. These might include debugging problems
with the security session management or slightly decreasing the memory
consumption of the daemons, which keep track of the sessions in use.

Session lifetimes for specific daemons are already properly configured in the default installation
of Condor.
Condor tools such as condor_ q and condor_ status create a
session that expires after one minute.
Theoretically they should not create a session at all,
because the
session cannot be reused between program invocations, but this is
difficult to do in the general case.
This allows a very small window of time for any possible attack,
and it helps
keep the memory footprint of running daemons down,
because they are not keeping track of all of the sessions.
The session durations may be manually tuned
by using macros in the configuration file,
but this is not recommended.

3.6.8 Host-Based Security in
Condor

This section describes the mechanisms for setting up Condor's
host-based security.
This is now an outdated form of implementing security
levels for machine access.
It remains available and documented for purposes of backward compatibility.
If used at the same time as the user-based authorization,
the two specifications are merged together.

The host-based security paradigm allows control over which machines can
join a Condor pool, which machines can find out information about
your pool, and which machines within a pool can perform
administrative commands. By default, Condor is configured to allow
anyone to view or join a pool. It is recommended that this parameter is changed
to only allow access from machines that you trust.

This section discusses how the host-based security works inside Condor.
It lists the different levels of access and what
parts of Condor use which levels.
There is a description of how to configure
a pool to grant or deny certain levels of access to various
machines.
Configuration examples and the settings of configuration variables
using the condor_ config_val command complete this section.

Inside the Condor daemons or tools that use DaemonCore (see
section 3.9 for details), most
tasks are accomplished by sending commands to another Condor daemon.
These commands are represented by an integer value to specify which command
is being requested, followed
by any optional information that the protocol requires at that point
(such as a ClassAd, capability string, etc).
When the daemons start up,
they will register which commands they are willing to accept, what to
do with arriving commands, and the access level required for
each command.
When a command request is received by a daemon, Condor identifies the access level
required and checks the IP address of the sender to verify that
it satisfies the allow/deny settings
from the configuration file.
If permission is granted, the command request is honored;
otherwise, the request will be aborted.

Settings for the access levels in the global
configuration file will affect all the machines in the pool.
Settings in a local configuration file will only affect the specific machine.
The settings for a given machine determine what other hosts can send
commands to that machine.
If a machine foo is to be given
administrator access on machine bar, place foo in
bar's configuration file access list (not the other way around).

The following are the various access levels that commands within
Condor can be registered with:

READ

Machines with READ
access can read information from the Condor daemons. For example, they can
view the status of the pool, see the job queue(s), and view user
permissions. READ access does not allow a machine to
alter any information, and does not allow
job submission. A machine listed
with READ permission will be unable join a Condor pool; the machine can
only view information about the pool.

WRITE

Machines with
WRITE access can write information to the Condor daemons.
Most important for granting a machine with this access is that the machine
will be able to join a pool since they are allowed to send ClassAd
updates to the central manager.
The machine can talk to the other machines
in a pool in order to submit or run jobs.
In addition, any machine with
WRITE access can request the condor_ startd daemon to perform
periodic checkpoints on an executing job. After the
checkpoint is completed, the job will continue to execute and the
machine will still be claimed by the original condor_ schedd daemon.
This allows users on the machines where they submitted their jobs
to use the condor_ checkpoint command to get their jobs to
periodically checkpoint, even if the users do not have an account on the
machine where the jobs execute.

IMPORTANT: For a machine to join a Condor pool, the machine must
have both WRITE permission ANDREAD permission.
WRITE permission is not enough.

ADMINISTRATOR

Machines
with ADMINISTRATOR access are granted additional Condor
administrator rights to the pool. This includes the ability to
change user priorities (with the command userprio -set),
and the ability to turn Condor on and off
(with the command condor_ off <machine>).
It is recommended that few machines be granted administrator access in a pool;
typically these are the machines that are used by Condor and system
administrators as their primary workstations,
or the machines running as the pool's central manager.

IMPORTANT: Giving ADMINISTRATOR privileges to a machine
grants administrator access for the pool to
ANY USER on that machine. This includes any
users who can run Condor jobs on that machine.
It is recommended that ADMINISTRATOR access is granted with due diligence.

OWNER

This level of access is
required for commands that the owner of a machine (any local user)
should be able to use, in addition to the Condor administrators.
For example, the condor_ vacate command causes the
condor_ startd daemon to vacate any running Condor job.
It requires OWNER permission,
so that any user logged into a local machine
can issue a condor_ vacate command.

NEGOTIATOR

This
access level is used specifically to verify that commands are
sent by the condor_ negotiator daemon.
The condor_ negotiator daemon runs on the central manager of
the pool.
Commands requiring this access
level are the ones that tell the condor_ schedd daemon to begin
negotiating, and those that tell an available condor_ startd daemon
that it has been matched to a condor_ schedd with jobs to run.

CONFIG

This access level is
required to modify a daemon's configuration using
the condor_ config_val command.
By default, machines with this level of access are able
to change any configuration parameter, except those specified in
the condor_config.root configuration file.
Therefore, one should exercise extreme caution before
granting this level of host-wide access.
Because of the implications caused by CONFIG privileges,
it is disabled by default for all hosts.

Starting with version 6.3.2, Condor provides a mechanism for more
fine-grained control over the configuration settings that can be
modified remotely with condor_ config_val.

ADMINISTRATOR and NEGOTIATOR access default to
the central manager machine.
OWNER access defaults to the local machine, as well as
any machines
given with ADMINISTRATOR access.
CONFIG access is not granted to any machine
as its default.
These defaults are sufficient for most pools, and should not be changed without
a compelling reason.
If machines other than the default are to have to have OWNER
access, they probably should also have ADMINISTRATOR access.
By granting machines ADMINISTRATOR access, they
will automatically have OWNER access, given how
OWNER access is set within the configuration.

This example configuration presumes that the condor_ collector
and condor_ negotiator daemons are running on the same machine.

For each access level, an ALLOW or a DENY may be added.

If you have an ALLOW, it means "only allow these machines". No
ALLOW means allow anyone.

If you have a DENY, it means "deny these machines". No DENY
means to deny nobody.

If you have both an ALLOW and a DENY, it means allow the
machines listed in ALLOW except for the machines listed in DENY.

Exclusively for the CONFIG access,
no ALLOW means allow no one.
Note that this is different than the other ALLOW configurations.
It is different to enable more stringent security where
older configurations are used, since
older configuration files would not have a
CONFIG configuration entry.

Multiple machine entries
in the configuration files
may be separated by either a space or a comma.
The machines may be listed by

Host names with a wild card ``*'' character (only one ``*'' is
allowed per name) - for example: *.cs.wisc.edu, sol*.cs.wisc.edu

To resolve an entry that falls into both allow and deny:
individual
machines have a higher order of precedence than wild card entries, and
host names with a wild card have a higher order of precedence than IP
subnets.
Otherwise, DENY has a higher order of precedence than ALLOW.
(this is how most people would intuitively expect it to work).

In addition, the above access levels may be specified on a
per-daemon basis, instead of machine-wide for all daemons.
Do this with the subsystem string (described in
section 3.3.1 on Subsystem Names),
which is one of: STARTD, SCHEDD, MASTER, NEGOTIATOR,
or COLLECTOR.
For example, to grant different read access for the condor_ schedd:

HOSTALLOW_READ_SCHEDD = <list of machines>

The following is a list of registered commands that daemons will
accept. The list is ordered by daemon.
For each daemon, the commands are grouped by the access level
required for a daemon to accept the command from a
given machine.

ALL DAEMONS:

WRITE

The command sent as a result of condor_ reconfig to reconfigure a daemon.

ADMINISTRATOR

The command sent as a result of reconfig -full
to perform a full reconfiguration on a daemon.

STARTD:

WRITE

All commands that relate to a condor_ schedd daemon claiming
a machine, starting jobs there, or stopping those jobs.

The command that condor_ checkpoint sends to periodically checkpoint
all running jobs.

READ

The command that condor_ preen sends to request the
current state of the condor_ startd daemon.

OWNER

The command that condor_ vacate sends to cause
any running jobs to stop running.

NEGOTIATOR

The command that the condor_ negotiator daemon sends to
match a machine's condor_ startd daemon with a given condor_ schedd
daemon.

NEGOTIATOR:

WRITE

The command that initiates a new negotiation
cycle. It is sent by the condor_ schedd when new jobs are submitted
or a condor_ reschedule command is issued.

READ

The command that can retrieve the current state
of user priorities in the pool (sent by the condor_ userprio command).

ADMINISTRATOR

The command that can set the current
values of user priorities (sent as a result of the userprio -set
command).

COLLECTOR:

WRITE

All commands that update the condor_ collector daemon with new ClassAds.

READ

All commands that query the condor_ collector daemon for ClassAds.

SCHEDD:

NEGOTIATOR

The command that the condor_ negotiator sends to
begin negotiating with this condor_ schedd to match its jobs with available
condor_ startds.

WRITE

The command which condor_ reschedule sends to
the condor_ schedd to get it to update the condor_ collector with a current ClassAd
and begin a negotiation cycle.

The commands that a condor_ startd sends to the condor_ schedd when it must vacate
its jobs and release the condor_ schedd's claim.

The commands which write information into the job queue (such as
condor_ submit and condor_ hold).
Note that for most commands which attempt to write to the job queue, Condor
will perform an additional user-level authentication step.
This additional user-level authentication prevents, for example, an
ordinary user from removing a different user's jobs.

READ

The command from any
tool to view the status of the job queue.

MASTER: All commands are registered with ADMINISTRATOR
access:

restart

: Master restarts itself (and all its children)

off

: Master shuts down all its children

off -master

: Master shuts down all its children and exits

on

: Master spawns all the daemons it is configured to spawn

This section provides examples of configuration settings.
Notice that ADMINISTRATOR access is
only granted through a HOSTALLOW setting to explicitly grant access to
a small number of machines. We recommend this.

Let any machine join your pool.
Only the central manager has
administrative access (this is the default that ships with Condor)

Only allow machines at NCSA and the U of I Math department join the
pool, EXCEPT do not allow lab machines to do so.
Also, do not
allow the 177.55 subnet (perhaps this is the dial-in subnet).
Allow anyone to view pool statistics. The machine named
bigcheese administers the pool (not the central manager).

Only allow machines at NCSA and UW-Madison's CS department to
view the pool. Only NCSA machines and the machine raven.cs.wisc.edu can join
the pool.
(Note: the machine raven has the read access it needs through the
wild card setting in HOSTALLOW_READ).
This example also shows
how to use ``\'' to continue a long list of machines
onto multiple lines, making it more readable (this works for all
configuration file entries, not just host access entries)

Allow anyone except the military to view the status of the
pool, but only let machines at NCSA view the job queues.
Only NCSA machines can join the pool.
The central manager, bigcheese, and
biggercheese can perform most administrative functions.
However, only biggercheese can update user priorities.

A new security feature introduced in
Condor version 6.3.2 enables more fine-grained control over the
configuration settings that can be modified remotely with the
condor_ config_val command.
The manual page for condor_ config_val on
page details how to use
condor_ config_val to modify configuration settings remotely.
Since certain configuration attributes can have a large impact on the
functioning of the Condor system and the security of the machines in a
Condor pool, it is important to restrict the ability to change
attributes remotely.

For each security access level described,
the Condor
administrator can define which configuration settings a host at that
access level is allowed to change.
Optionally, the administrator can define separate lists of settable
attributes for each Condor daemon, or the administrator
can define one list that is used by all daemons.

For each command that requests a change in configuration setting,
Condor searches all the different possible security access
levels to see which, if any, the request satisfies.
(Some hosts can qualify for multiple access levels. For example, any
host with ADMINISTRATOR permission probably has
WRITE permission also).
Within the qualified access level,
Condor searches for the list of attributes that may be modified.
If the request is covered by the list,
the request will be granted.
If not covered, the request will be refused.

The default configuration shipped with Condor is exceedingly
restrictive.
Condor users or administrators cannot set
configuration values from remote hosts with condor_ config_val.
Enabling this feature requires a change to the
settings in the configuration file.
Use this security feature carefully.
Grant access only for attributes which you need to be able to modify
in this manner, and grant access only at the most restrictive
security level possible.

The most secure use of this feature allows Condor users to set
attributes in the configuration file which are not used by Condor
directly.
These are custom attributes published by various Condor
daemons with the <SUBSYS>_ATTRS setting described in
section 3.3.5 on page .
It is secure to grant access only to modify attributes that are used by Condor
to publish information.
Granting access to modify
settings used to control the behavior of Condor is
not secure.
The goal is to
ensure no
one can use the power to change configuration attributes to compromise
the security of your Condor pool.

The control lists are defined by configuration settings that contain
SETTABLE_ATTRS in their name.
The name of the control lists have the following form:

<SUBSYS>_SETTABLE_ATTRS_PERMISSION-LEVEL

The two parts of this name that can vary are
PERMISSION-LEVEL and the <SUBSYS>.
The PERMISSION-LEVEL can be any of the security access levels
described earlier in this section.
Examples include WRITE, OWNER, and CONFIG.

The <SUBSYS> is an optional portion of the name.
It can be used to
define separate rules for which configuration attributes can be set
for each kind of Condor daemon (for example, STARTD, SCHEDD, MASTER).
There are many configuration settings that can be defined differently
for each daemon that use this <SUBSYS> naming convention.
See section 3.3.1 on
page for a list.
If there is no daemon-specific value for a given daemon, Condor will
look for SETTABLE_ATTRS_PERMISSION-LEVEL.

Each control list is defined by a comma-separated list of attribute
names which should be allowed to be modified.
The lists can contain wild cards characters (`*').

Some examples of valid definitions of control lists with explanations:

SETTABLE_ATTRS_CONFIG = *

Grant unlimited access to modify configuration attributes
to any request that came from a machine in the CONFIG access
level.
This was the default behavior before Condor version 6.3.2.

SETTABLE_ATTRS_ADMINISTRATOR = *_DEBUG, MAX_*_LOG

Grant access to change any configuration setting that ended
with ``_DEBUG'' (for example, STARTD_DEBUG) and any
attribute that matched ``MAX_*_LOG'' (for example,
MAX_SCHEDD_LOG) to any host with ADMINISTRATOR
access.

STARTD_SETTABLE_ATTRS_OWNER = HasDataSet

Allows any request to modify the HasDataSet
attribute that came from a host with OWNER access.
By default, OWNER covers any request originating from the
local host, plus any machines listed in the ADMINISTRATOR
level.
Therefore, any Condor job would qualify for OWNER access to the
machine where it is running.
So, this setting would allow any process running on a given host,
including a Condor job, to modify the HasDataSet variable for
that host.
HasDataSet is not used by Condor, it is an invented attribute
included in the STARTD_ATTRS setting in order for this
example to make sense.

3.6.9 Using
Condor w/ Firewalls, Private Networks, and NATs

This topic is now addressed in more detail in
section 3.7, which explains network communication in
Condor.

3.6.10 User Accounts in Condor

On a Unix system,
UIDs (User IDentification numbers) form part of an operating system's
tools for maintaining access control.
Each executing program has a UID,
a unique identifier of a user executing the program.
This is also called the real UID.
A common situation has one user executing the program owned
by another user.
Many system commands work this way, with a user (corresponding
to a person) executing a program belonging to (owned by) root.
Since the program may require privileges that root has which
the user does not have, a special bit in the program's
protection specification (a setuid bit) allows the program
to run with the UID of the program's owner, instead of the
user that executes the program.
This UID of the program's owner is called an effective UID.

Condor works most smoothly when its daemons run as root.
The daemons then have the ability to switch their
effective UIDs at will.
When the daemons run as root,
they normally leave their effective UID and GID (Group IDentification)
to be those of user and group condor.
This allows access to the log files without
changing the ownership of the log files.
It also allows access to these files when
the user condor's home directory resides on an NFS server.
root can not normally access NFS files.

If there is no condor user and group on the system, an
administrator can specify which UID and GID the Condor daemons should
use when they do not need root privileges in two ways:
either with the CONDOR_IDS environment variable or the
CONDOR_IDS configuration file setting.
In either case, the value should be the UID integer, followed by a
period, followed by the GID integer.
For example, if a Condor administrator does not want to create a
condor user, and instead wants their Condor daemons to run as
the daemon user (a common non-root user for system daemons to
execute as), the daemon user's UID was 2, and group
daemon had a GID of 2, the corresponding setting in the Condor
configuration file would be CONDOR_IDS = 2.2.

On a machine where a job is submitted,
the condor_ schedd daemon
changes its effective UID to root
such that it has the capability to start up a condor_ shadow daemon
for the job.
Before a condor_ shadow daemon is created,
the condor_ schedd daemon
switches back to root,
so that it can start up the condor_ shadow daemon with the (real) UID
of the user who submitted the job.
Since the condor_ shadow runs as the owner of the job,
all remote system calls are performed under the owner's UID
and GID.
This ensures that as the job executes,
it can access only files that its owner could access if the job
were running locally, without Condor.

On the machine where the job executes, the
job runs either as the submitting user or as user nobody,
to help ensure that the job cannot access local resources or do harm.
If the UID_DOMAIN matches,
and the user exists as the same UID in password files
on both the submitting machine and on the execute machine,
the job will run as the submitting user.
If the user does not exist in the execute machine's
password file and SOFT_UID_DOMAIN is True,
then the job will run under the submitting user's UID anyway (as
defined in the submitting machine's password file).
If SOFT_UID_DOMAIN is False,
and UID_DOMAIN matches,
and the user is not in the execute machine's password file,
then the job execution attempt will be aborted.

3.6.10.1 Running Condor as Non-Root

While we strongly recommend starting up the Condor daemons as root,
we understand that it is not always possible to do so.
The main problems
appear when one Condor installation is shared by many users on a
single machine, or if machines are set up to only execute
Condor jobs. With a submit-only installation for a
single user, there is no need for (or benefit from) running as
root.

What follows are the effects on the various parts of Condor of running
both with and without root access.

condor_ startd

If you're setting up a machine to run Condor
jobs and don't start the condor_ startd as root, you're basically
relying on the goodwill of your Condor users to agree to the policy
you configure the condor_ startd to enforce as far as starting, suspending,
vacating and killing Condor jobs under certain conditions. If you
run as root, however, you can enforce these policies regardless of
malicious users. By running as root, the Condor daemons run with a
different UID than the Condor job that gets started (since the
user's job is started as either the UID of the user who submitted
it, or as user nobody, depending on the UID_DOMAIN
settings). Therefore, the Condor job cannot do anything to the
Condor daemons. If you don't start the daemons as root, all
processes started by Condor, including the end user's job, run with
the same UID (since you can't switch UIDs unless you're root).
Therefore, a user's job could just kill the condor_ startd and
condor_ starter as soon as it starts up and by doing so, avoid
getting suspended or vacated when a user comes back to the machine.
This is nice for the user, since they get unlimited access to the
machine, but awful for the machine owner or administrator. If you
trust the users submitting jobs to Condor, this might not be a
concern. To ensure, however, that the policy you choose is
effectively enforced by Condor, the condor_ startd should be
started as root.

In addition, some system information cannot be obtained without
root access on some platforms (such as load average on IRIX). As a
result, when running without root access, the condor_ startd must
call other programs (for example, uptime) to get this
information. This is much less efficient than getting the
information directly from the kernel (which is what we do if we're
running as root). On Linux and Solaris, we can get this
information directly without root access, so this is not a concern
on those platforms.

If you cannot have all of Condor running as root, at least consider
whether you can install the condor_ startd as setuid root. That
would solve both of these problems. If you cannot do that, you
could also install it as a setgid sys or kmem program (depending on
whatever group has read access to /dev/kmem on your system),
and that would at least solve the system information problem.

condor_ schedd

The biggest problem running the condor_ schedd
without root access is that the condor_ shadow processes which it
spawns are stuck with the same UID the condor_ schedd has. This
means that users submitting their jobs must go out of their way
to grant write access to user or group condor (or whoever the
condor_ schedd is running as) for any files or directories their jobs
write or create. Similarly, read access must be granted to their
input files.

Consider installing condor_ submit as a setgid condor
program so that at least the stdout, stderr and
UserLog files get created with the right permissions. If
condor_ submit is a setgid program, it will automatically set
it's umask to 002, and create group-writable files. This
way, the simple case of a job that only writes to stdout
and stderr will work. If users have programs that open
their own files, they will need to know and set the proper permissions
on the directories they submit from.

condor_ master

The condor_ master is what spawns the
condor_ startd and condor_ schedd. To have both running
as root, have the condor_ master run as root.
This happens
automatically if you start the master from your boot scripts.

condor_ negotiator and condor_ collector

There is no need to have either of these daemons running as root.

condor_ kbdd

On platforms that need the condor_ kbdd (Digital
Unix and IRIX) the condor_ kbdd must run as root. If it is
started as any other user, it will not work. You might consider
installing this program as a setuid root binary if you cannot run
the condor_ master as root. Without the condor_ kbdd, the
startd has no way to monitor mouse activity at all, and the only
keyboard activity it will notice is activity on ttys (such as
xterms, remote logins, etc).

If you do choose to run Condor as non-root, then you may choose almost any
user you like. A common choice is to use the condor user; this
simplifies the setup because Condor will look for its configuration
files in the condor user's directory. If you do not
select the condor
user, then you will need to ensure that the configuration is set
properly so that Condor can find its configuration files.

If users will be submitting jobs as a user different than the user
Condor is running as (perhaps you are running as the condor user and
users are submitting as themselves), then users have to be careful to
only have file permissions properly set up to be accessible by the
user Condor is using. In practice, this means creating world-writable
directories for output from Condor jobs. This creates a potential
security risk, in that any user on the machine where the job is
submitted can alter the data, remove it, or do other undesirable
things. It is only acceptable in an environment where users can trust
other users.

Normally, users without root access who wish to use Condor on their
machines create a condor home directory somewhere within their
own accounts and start up the daemons (to run with the UID of the
user). As in the case where the daemons run as user condor, there is
no ability to switch UIDs or GIDs. The daemons run as the UID and GID
of the user who started them. On a machine where jobs are submitted,
the condor_ shadow daemons all run as this same user. But if other
users are using Condor on the machine in this environment, the
condor_ shadow daemons for these other users' jobs execute with the
UID of the user who started the daemons. This is a security risk,
since the Condor job of the other user has access to all the files and
directories of the user who started the daemons. Some installations
have this level of trust, but others do not. Where this level of
trust does not exist, it is best to set up a condor account and group,
or to have each user start up their own Personal Condor submit
installation.

When a machine is an execution site for a Condor job, the Condor job
executes with the UID of the user who started the condor_ startd
daemon. This is also potentially a security risk, which is why we do
not recommend starting up the execution site daemons as a regular
user. Use either root or a user
(such as the user condor) that exists
only to run Condor jobs.

3.6.10.2 Running Jobs as the Nobody User

Under Unix, Condor runs jobs either as the user that submitted the jobs,
or as the user called nobody.
Condor uses user nobody if the value of the UID_DOMAIN
configuration variable of the
submitting and executing machines are different.

When Condor cleans up after a executing a vanilla universe job,
it does the best that it can by
deleting all of the processes started by the job.
Unfortunately, it is possible to fool Condor,
and leave processes behind after Condor has cleaned up.
If the job is running as user nobody,
it is possible for it to leave a lurker process lying in wait
for the next job run as nobody.
The lurker process may prey maliciously on the next nobody user job,
wreaking havoc.

Condor could prevent this problem by simply killing all processes run by
the nobody user, but this would annoy many system administrators.
The nobody user is often used for non-Condor system processes.

Condor provides a two-part solution to this difficulty.
First, create user accounts specifically for Condor to use instead
of user nobody.
These can be low-privilege accounts,
as the nobody user is.
Create one of these accounts for each
virtual machine per computer,
so that distinct users can be used for concurrent processes.
This prevents malicious behavior between
processes running on distinct virtual machines.
Section 3.13.7 details virtual machines.
For a sample machine with two virtual machines,
create two users that are intended only to be used by Condor.
As an example, call them nobody1 and nobody2.
Tell Condor about these users
with the VMx_USER configuration variables,
where x is replaced with the
virtual machine number. In this example:

VM1_USER = nobody1
VM2_USER = nobody2

Reconfigure Condor, so that Condor will make use of these users
instead of the nobody user.
One more change is required to prevent lurker processes:
tell Condor that these accounts are intended only to be used by Condor,
so Condor can kill all the processes belonging to these users upon
job completion.
The configuration variable EXECUTE_LOGIN_IS_DEDICATED
is introduced and set to True for this purpose.

EXECUTE_LOGIN_IS_DEDICATED = TRUE

Notes:

If UID_DOMAIN is not set in the configuration, do not
set EXECUTE_LOGIN_IS_DEDICATED.
In this case, lurker processes are not a concern,
and other processes that a user may have running would be killed
improperly.

This only applies to vanilla universe and Java universe jobs.
Standard universe jobs are not a concern,
because they are not allowed to create new processes.

On Windows, VMx_USER will only work if the credential
of the specified user is stored on the execute machine
using condor_ store_cred.
See the condor_ store_cred
manual page (in section 9)
for details of this command.

3.6.10.3 Working Directories for Jobs

Every executing process has a notion of its current working directory.
This is the directory that acts as the base for all file system
access.
There are two current working directories for any Condor job:
one where the job is submitted and a
second where the job executes.
When a user submits a job,
the submit-side current working directory is
the same as for the user when the condor_ submit command
is issued.
The initialdir submit command may change this,
thereby allowing different jobs to have different working
directories.
This is useful when submitting large numbers of jobs.
This submit-side current working directory remains unchanged for the
entire life of a job.
The submit-side current working directory is also
the working directory of the condor_ shadow daemon.
This is particularly relevant for standard universe jobs,
since file system
access for the job goes through the condor_ shadow daemon, and
therefore all accesses behave as if they were executing without
Condor.

There is also an execute-side current working directory.
For standard universe jobs,
it is set to the
execute subdirectory of Condor's home directory.
This directory is world-writable, since a Condor job usually runs as user
nobody.
Normally, standard universe jobs would never access this directory,
since all I/O system calls are passed back to the
condor_ shadow daemon on the submit machine.
In the event, however,
that a job crashes and creates a core dump file, the execute-side
current working directory needs to be accessible by the job
so that it can write the core file.
The core file is moved back to the submit machine,
and the condor_ shadow daemon is informed.
The condor_ shadow daemon
sends e-mail to the job owner announcing the core file, and provides a
pointer to where
the core file resides in the submit-side current working directory.