next up previous contents
Next: 4.2 An Introduction to Up: 4. Miscellaneous Concepts Previous: 4. Miscellaneous Concepts


4.1 An Introduction to Condor's ClassAd Mechanism

ClassAds are a flexible mechanism for representing the characteristics and constraints of machines and jobs in the Condor system. ClassAds are used extensively in the Condor system to represent jobs, resources, submitters and other Condor daemons. An understanding of this mechanism is required to harness the full flexibility of the Condor system.

A ClassAd is is a set of uniquely named expressions. Each named expression is called an attribute. Figure 4.1 shows an example of a ClassAd with ten attributes.

Figure 4.1: An example ClassAd
\begin{tex2html_preform}\begin{verbatim}MyType = ''Machine''
...dAvg<=0.3 && KeyboardIdle>15*60\end{verbatim}\end{tex2html_preform}

ClassAd expressions look very much like expressions in C, and are composed of literals and attribute references composed with operators. The difference between ClassAd expressions and C expressions arise from the fact that ClassAd expressions operate in a much more dynamic environment. For example, an expression from a machine's ClassAd may refer to an attribute in a job's ClassAd, such as TARGET.Owner in the above example. The value and type of the attribute is not known until the expression is evaluated in an environment which pairs a specific job ClassAd with the machine ClassAd.

ClassAd expressions handle these uncertainties by defining all operators to be total operators, which means that they have well defined behavior regardless of supplied operands. This functionality is provided through two distinguished values, UNDEFINED and ERROR, and defining all operators so that they can operate on all possible values in the ClassAd system. For example, the multiplication operator which usually only operates on numbers, has a well defined behavior if supplied with values which are not meaningful to multiply. Thus, the expression 10 * "A string" evaluates to the value ERROR. Most operators are strict with respect to ERROR, which means that they evaluate to ERROR if any of their operands are ERROR. Similarly, most operators are strict with respect to UNDEFINED.

4.1.1 Syntax

ClassAd expressions are formed by composing literals, attribute references and other sub-expressions with operators. Literals

Literals in the ClassAd language may be of integer, real, string, undefined or error types. The syntax of these literals is as follows:
A sequence of continuous digits (i.e., [0-9]). Additionally, the keywords TRUE and FALSE (case insensitive) are syntactic representations of the integers 1 and 0 respectively.

Two sequences of continuous digits separated by a period (i.e., [0-9]+.[0-9]+).

A double quote character, followed by an list of characters terminated by a double quote character. A backslash character inside the string causes the following character to be considered as part of the string, irrespective of what that character is.

The keyword UNDEFINED (case insensitive) represents the UNDEFINED value.

The keyword ERROR (case insensitive) represents the ERROR value. Attributes

Every expression in a ClassAd is named by an attribute name. Together, the (name,expression) pair is called an attribute. An attributes may be referred to in other expressions through its attribute name.

Attribute names are sequences of alphabetic characters, digits and underscores, and may not begin with a digit. All characters in the name are significant, but case is not significant. Thus, Memory, memory and MeMoRy all refer to the same attribute.

An attribute reference consists of the name of the attribute being referenced, and an optional scope resolution prefix. The three prefixes that may be used are MY., TARGET. and ENV.. The semantics of supplying a prefix are discussed in Section 4.1.2. Operators

The operators that may be used in ClassAd expressions are similar to those available in C. The available operators and their relative precedence is shown in figure 4.2.
Figure 4.2: Relative precedence of ClassAd expression operators
\begin{tex2html_preform}\begin{verbatim}- (high precedence)
* / ...
\vert\vert (low precedence)\end{verbatim}\end{tex2html_preform}

The operator with the highest precedence is the unary minus operator. The only operators which are unfamiliar are the =?= and =!= operators, which are discussed in Section 4.1.2.

4.1.2 Evaluation Semantics

The ClassAd mechanism's primary purpose is for matching entities who supply constraints on candidate matches. The mechanism is therefore defined to carry out expression evaluations in the context of two ClassAds which are testing each other for a potential match. For example, the condor_negotiator evaluates the Requirements expressions of machine and job ClassAds to test if they can be matched. The semantics of evaluating such constraints is defined below. Literals

Literals are self-evaluating, Thus, integer, string, real, undefined and error values evaluate to themselves. Attribute References

Since the expression evaluation is being carried out in the context of two ClassAds, there is a potential for namespace ambiguities. The following rules define the semantics of attribute references made by ad A that is being evaluated in a context with another ad B:
If the reference is prefixed by a scope resolution prefix,

If the reference is not prefixed by a scope resolution prefix,

Finally, if the reference refers to an expression that is itself in the process of being evaluated, there is a circular dependency in the evaluation. The value of the reference is ERROR. Operators

All operators in the ClassAd language are total, and thus have well defined behavior regardless of the supplied operands. Furthermore, most operators are strict with respect to ERROR and UNDEFINED, and thus evaluate to ERROR (or UNDEFINED) if either of their operands have these exceptional values.

4.1.3 ClassAds in the Condor System

The simplicity and flexibility of ClassAds is heavily exploited in the Condor system. ClassAds are not only used to represent machines and jobs in the Condor pool, but also other entities that exist in the pool such as checkpoint servers, submitters of jobs and master daemons. Since arbitrary expressions may be supplied and evaluated over these ads, users have a uniform and powerful mechanism to specify constraints over these ads. These constraints may take the form of Requirements expressions in resource and job ads, or queries over other ads. Requirements and Ranks

This is the mechanism by which users specify the constraints over machines and jobs respectively. Requirements for machines are specified through configuration files, while requirements for jobs are specified through the submit command file.

In both cases, the Requirements expression specifies the correctness criterion that the match must meet, and the Rank expression specifies the desirability of the match (where higher numbers mean better matches). For example, a job ad may contain the following expressions:

Requirements = Arch=="SUN4u" && OpSys == "SOLARIS251"
Rank         = TARGET.Memory + TARGET.Mips
In this case, the customer requires an UltraSparc computer running the Solaris 2.5.1 operating system. Among all such computers, the customer prefers those with large physical memories and high MIPS ratings. Since the Rank is a user specified metric, any expression may be used to specify the perceived desirability of the match. The condor_negotiator runs algorithms to deliver the ``best'' resource (as defined by the Rank expression) while satisfying other criteria.

Similarly, owners of resources may place constraints and preferences on their machines. For example,

    Friend        = Owner == "tannenba" || Owner == "wright"
    ResearchGroup = Owner == "jbasney" || Owner == "raman"
    Trusted       = Owner != "rival" && Owner != "riffraff"
    Requirements  = Trusted && ( ResearchGroup || LoadAvg < 0.3 &&
                         KeyboardIdle > 15*60 )
    Rank          = Friend + ResearchGroup*10
The above policy states that the computer will never run jobs owned by users ``rival'' and ``riffraff,'' while the computer will always run a job submitted by members of the research group. Furthermore, jobs submitted by friends are preferred to other foreign jobs, and jobs submitted by the research group are preferred to jobs submitted by friends.

Note: Because of the dynamic nature of ClassAd expressions, there is no a priori notion of an integer valued expression, a real valued expression, etc. However, it is intuitive to think of the Requirements and Rank expressions as integer valued and real valued expressions respectively. If the actual type of the expression is not of the expected type, the value is assumed to be zero. Querying with ClassAd Expressions

The flexibility of this system may also be used when querying ClassAds through the condor_status and condor_q tools which allow users to supply ClassAd constraint expressions from the command line.

For example, to find all computers which have had their keyboards idle for more than 20 minutes and have more than 100 MB of memory:

% condor_status -const 'KeyboardIdle > 20*60 && Memory > 100'

Name       Arch     OpSys        State      Activity   LoadAv Mem  ActvtyTime

amul.cs.wi SUN4u    SOLARIS251   Claimed    Busy       1.000  128   0+03:45:01
aura.cs.wi SUN4u    SOLARIS251   Claimed    Busy       1.000  128   0+00:15:01
balder.cs. INTEL    SOLARIS251   Claimed    Busy       1.000  1024  0+01:05:00
beatrice.c INTEL    SOLARIS251   Claimed    Busy       1.000  128   0+01:30:02
                     Machines Owner Claimed Unclaimed Matched Preempting

    SUN4u/SOLARIS251        3     0       3         0       0          0
    INTEL/SOLARIS251       21     0      21         0       0          0
    SUN4x/SOLARIS251        3     0       3         0       0          0
           SGI/IRIX6        1     0       0         1       0          0
         INTEL/LINUX        1     0       1         0       0          0

               Total       29     0      28         1       0          0

The similar flexibility exists in querying job queues in the Condor system.

next up previous contents
Next: 4.2 An Introduction to Up: 4. Miscellaneous Concepts Previous: 4. Miscellaneous Concepts