Protecting Your Directories with Access-Control Files

In Chapter 1 you were introduced to a couple of files that have a major impact on how your server allows access to directories and files. During that introduction, you were promised further details about these very important files. In this section, you will learn the details of these files and other files on your server that protect your server and allow you to do your job as a CGI programmer. These configuration files provide access control for the ncSA server. One of the primary files that impacts who can access your files and how that access is allowed is called the global access-control file and usually is named access.conf, which appropriately stands for access configuration file.

The global access-control file provides per-directory access control for the entire server. The various commands for this file can define identical control for the entire document root and server root directory trees or allow individual control over each directory within a selected directory tree.

The Directory Directive

The Directory directive controls which directories are affected by the commands it contains. The syntax of Directory looks very similar to an HTML tag, although this is not an HTML directive. The syntax is an open tag of <DIRECTORY DIRECTORY_PATH>, followed by a series of ncSA configuration directives (see Table 12.1), and closed with the </DIRECTORY> command.

The ncSA development team calls these types of commands sectioning directives. All sectioning directives begin with an opening directive that includes one argument-in this case, the directory path information. The information given in the opening directive affects all other directives between the opening and closing sectioning directives.

The directory path must be a physical path on the server. Aliases are not allowed. You can use wildcards in the DIRECTORY_PATH syntax. The directory path affects all subdirectories below the directory path and so also may be called a directory tree. If I want to control access to my cgi-bin directory and any subdirectories under it, I can begin with a Directory directive in the global access-control file that looks like this: <DIRECTORY /usr/local/BSN/http/accn.com/cgi-bin>

Then you can place the configuration directives next before a closing </DIRECTORY> command. The configuration directives between the opening <DIRECTORY DIRECTORY_PATH> command and the closing </DIRECTORY> command only affect the directory tree defined by the DIRECTORY_PATH-in this case, /usr/local/BSN/http/accn.com/cgi-bin

You can have as many Directory directives as you want in your global access-control file, but you cannot nest Directory directives.

The AllowOverride Directive

The global access-control file defines global access control for directory trees on your server, but you learned in Chapter 7 "Building an Online Catalog," that you also can set up per-directory access-control files, usually called .htaccess. Your capability to use per-directory access-control files is limited by the options declared along with the AllowOverride directive. Someone chose really great names for the ncSA configuration commands because the AllowOverride directive does just that: It allows the Directory directives in the global access-control file to be overruled or overridden by per-directory access-control files (.htaccess). The AllowOverride directive is the only access-control file command that can be used only in the global access-control file or global directory access-control file. All other configuration directives defined here also can be used in the per-directory access-control file.

If your job is system security, you might be a little concerned by this. Do you want all the users on your system to be able to override everything you set up in the global access-control file? That's really your decision. One thing you might consider is setting up a very restrictive document root directory but allowing overrides to all your restrictions. Then the people overriding your global access-control file must be very aware of how to run a server and you will never hear from them, or, as someone needs a special privilege, you can find out what she is doing and advise her of security precautions. This is a nice compromise, but you might feel that it gives your users too much control and requires too much work on your part in answering user questions.

The AllowOverride directive gives you several options, which can be None or All; or any combination of Options, FileInfo, AuthConfig, or Limit. The meanings of None and All are relatively clear. An AllowOverride None command means that per-directory access-control files are not allowed to override any of the directives in the global access-control file. An AllowOverride All command means that the per-directory access-control file can override any configuration directive of the global access-control file. Other than these two mutually exclusive options, you can choose what you want your users to be able to override by just adding an AllowOverride option. Table 12.2 summarizes the AllowOverride options.

The details of the configuration commands that can be overridden are covered in this chapter. The AllowOverride directive is valid only in the global access-control file. If no AllowOverride directive is included in the global access-control file, the default is All.

The Options Directive

The Options directive inside the global access-control file determines whether you can use CGI commands inside a directory tree. Each of the rich set of ncSA server features is controlled per directory by the Options directive. Server Side Include commands (SSIs), automatic indexing, and symbolic link following can be selectively applied to any directory tree on your server.

Suppose that you want to allow all your users to execute CGI programs; you want neat users, however, so that you will have at least some idea of where their CGI programs are located. You can allow any user to execute CGI programs, but only within a local user cgi-bin directory, by putting the following Directory directive in your global access-control file (assuming that all your users are under the user directory): <DIRECTORY /usr/*/cgi-bin> OPTIONS ExecCGI </DIRECTORY>

Just as with the AllowOverride directive, multiple directives can be added to the Options directive. The command in the example does not allow indexing, SSIs, or symbolic link following. This command also can be used in the per-directory access-control file and is a good candidate for your cgi-bin directory, especially if you have the Options All directive set in your global access-control file. The Options command has the same All or None possibilities as the AllowOverride directive. The default for the Options directive if it is not included in your global access-control file is Options ALL. Table 12.3 summarizes the parameters of the OPTIONS directive.

The Limit Directive

The Limit directive controls what type of request headers can be used in a directory and controls access to the directory by domain name, IP address, individual users, or a group of users. The syntax of the Limit directive is very similar to the Directory directive. Like the Directory directive, the Limit directive is a sectioning directive. Therefore, all the commands between the opening and closing Limit directive are affected by the opening directive. The Limit directive syntax follows: <LIMIT HTTP-REQUEST-METHOD(S)> followed by the <LIMIT> directives order, deny,   allow, require and closed with </LIMIT>

The Limit directive uses the allow, deny, and require commands to restrict access to a directory completely or by use of user authentication. The commands for limiting directory access are described next. Before you learn about the order, deny, allow, and require commands, take a look at the HTTP method request data in the opening Limit directive. Not only does the Limit directive define who can access a directory, but it also defines how that user can access that directory. The first HTTP request header is always the method request. The method can be Get, Post, Head, Delete, Put, Unlink, or Link. The Limit directive is supposed to limit access to a directory based on the HTTP method request by defining the valid request methods in the opening Limit directive. Currently, you can use only the Get and Post methods in the opening Limit directive.

The allow from Directive

This two-word directive works with the order and deny from directives. The allow from directive can be used only within a Limit section. The allow from directive tells the server which machines (hosts) can have access to a particular directory. You can define the machine name by its IP address or domain name. You can define a complete IP or domain name, fully restricting the use to that one address, or you can use any portion of the IP or domain name. If you use a partial domain name, the value is interpreted from right to left. If you want to restrict access to a particular directory to all domains that are part of the military network, for example, you can create a Limit section like this: <LIMIT GET POST> order deny,allow deny from all allow from .mil </LIMIT>

Each of the commands works together to tell the server how to determine who can have access to this directory. When a user is denied access because of the Limit directive, he gets a status code of 403, FORBIDDEN, as shown in Figure 12.1.

Figure 12.1 : Access is forbidden because of the Limit directive.

The domain or hostname continues to work restrictively from right to left. If you want to restrict all access to only people logged in through the Texas A&M University network (my alma mater), your Limit directive would look like this: <LIMIT GET POST> order deny,allow deny from all allow from .tamu.edu </LIMIT>

You can continue to restrict access to a fully qualified domain name by completely defining the hostname and leaving off the leading period (.). Because domain names can contain any number of subdomains before them, I'll stop here.

The allow from directive determines IP address restriction from left to right instead of right to left, as with domain names and hostnames. The fully qualified IP address for my server is 199.170.89, which is followed by an actual connection address. So an individual connection IP address might be 199.170.89.69. You don't want to restrict access this far, because only one particular dial-up line would be able to access the restricted directory.

If you want to restrict all users of the system to your own server IP address, however, you would define a Limit directive that looks like this: <LIMIT GET POST> order deny,allow deny from all allow from 199.170.89 </LIMIT>

The less restrictive you want to be, the shorter the IP definition becomes. The allow from command can be repeated on several lines and can include several domains and IP addresses on a single line. The Limit sections can be combined into the following Limit directive, for example: <LIMIT GET POST> order deny,allow deny from all allow from .mil .tamu.edu 199.170.89 </LIMIT>

Then, if your Web visitors meet any of the allow from conditions, they are allowed to Get and Post to URIs in the directory controlled by the Limit directive. The allow from directive accepts one more parameter, which you might have guessed by now: the all parameter. This works just as you would expect; it allows anyone into this directory. Why would you want to use this command at all? It would seem that if you are going to allow everyone into a directory, you don't need a Limit directive at all, much less an allow from directive. Typically, the allow from all directive is used along with the deny from directive, which is described next.

The deny from Directive

The deny from directive works exactly as you would expect it to: It denies access to the directory based on the IP and domain names/hostnames identified in the deny from directive list. I hope you take a moment to thank the ncSA gang that defined all these commands. They actually make sense, unlike many other things in life. As shown earlier, the deny from directive usually works together with the allow from directive, but in reverse order, of course. Suppose that you are a University of Texas fan and you want to keep out all those dadgum AGGIES and military types. (Dadgum is the diminutive term for $#@!, which I can't use here.) Just take the earlier Limit directive and turn it around: <LIMIT GET POST> order allow,deny allow from all deny from .mil .tamu.edu </LIMIT>

Now anyone can use the directory except AGGIES and people from the military network. The syntax and capabilities of the deny from directive are the same as the allow from directive, so I refer you to the previous section for any further detail.

The order Directive

The order directive tells the server which set of allow or deny directives to interpret first. Because you can put multiple lines of allow and deny directives inside a Limit section, the order directive is required to tell the server which set of commands overrides the other. The default order is deny, allow. Because later commands override earlier commands, the order can be important. In the default order, the server first interprets all deny from directives and then parses the allow from directives. The all from directives override any previous deny from directives.

You should use the order directive based on how you are trying to limit access to a directory. If you want everyone to have access except a few hackers you might have caught in the past, set the order to order allow,deny. This way, you can allow everyone in and exclude just the few who create problems. On the other hand, if you want to limit access to your directory to just a select few Web Heads, switch the order command to order deny,allow. Then use the deny from all directive with allow from to permit only those you want to allow into your directory.

The default order is deny, allow, and the default restrictions are to allow any domain or IP address that you don't explicitly deny. You can change this default behavior by using the order mutual-failure directive. This changes the default behavior to deny any host not specifically named in an allow from directive. All hosts who are allowed access to the directory contents must explicitly be named in the allow from directive. You can include explicit deny from directives, but deny from all is implied.

The require Directive

You have been exposed to the require directive before. In Chapter 7 you learned how to set up a password-protected directory. The require directive is used to begin the username/password authentication scheme and works with several other commands. These commands-AuthName, AuthType, AuthUserFile, and AuthGroupFile-are not enclosed by the Limit sectioning directive and are discussed next. The require directive will not work without the prior setup of these commands, however.

The require directive, when placed inside a Limit sectioning directive, tells the server to return to the client a 401, Unauthorized access, status code and begin the authenticate sequence. In addition, the require directive defines what type of authenticated users can attempt to access this directory. All users of this directory must be authenticated by the authorization scheme defined outside the Limit section, but the defining of who is even allowed to authenticate himself is controlled by the require directive.

It's easy to think of the require directive as another form of the allow from directive, because it works in a very similar manner. The allow directive works with domain names/hostnames and IP addresses, and the require directive works with a password file that contains usernames. The allow from directive has an all parameter that allows any domain, host, or IP address. The require directive has a valid-user parameter that allows any authenticated user from the AuthUserFile username/password file access to the directory. An authenticated user is someone who has entered a valid username/password in response to an HTTP response header of WWW-Authenticate. The allow from directive allows partial or fully qualified domains and IP addresses. The require directive allows groups of authenticated users or fully qualified usernames, with the require group groupname1 groupname2 ...

and require user username1 username2 username3 ...

directives. Table 12.4 summarizes the three parameters of the require directives. You can have multiple require directives within a Limit section, just as you can with the allow from directive, as shown in this example: <LIMIT GET POST> require user sherry scott eric require group aggies deny from .utexas.edu deny from .mil </LIMIT>

The directives inside the Limit section are additive. This Limit section therefore is very restrictive. Only the three users-sherry, scott, and eric-can access this directory, and then only if their usernames are part of the aggies group and they are not using either a server from the utexas.edu domain or the .mil domain. And this is only after they pass the authenticate scheme. Remember that the require directives in the Limit section are additive. Table 12.4 lists the require parameters.

Setting Up Password Protection

You learned about password protection in Chapter 7. This section covers the details that weren't covered earlier. Password protection is part of the global access-control file directive set, which can be applied on a per-directory access basis using a per-directory access-control file such as .htaccess, as can most of the directives of the global access-control file.

Directory password protection is made up of a password file, created by the htpasswd command, groupname files, the require directive, and a group of authenticate directives. Each of these pieces can be applied by using the global access-control file on a per-directory basis or by using the per-directory access-control file method defined in Chapter 7.

The htpasswd Command

In Chapter 7 you learned that the password file is created by a program distributed with the ncSA server called htpasswd. This program creates the initial password file in the directory you defined in the initial creation command. The syntax for the htpasswd command follows: htpasswd [-c] filename username

The filename should include a relative or absolute path to the password file if the password file is not in the current directory. Each time you use this command, you must supply a relative path to the password file. The htpasswd command prompts you for the username and then that user's password, verifying the password entry by requesting a second confirmation entry. Each time you use the htpasswd command, there is an assumption that you are changing an existing password or creating a new username/password pair. The htpasswd command uses the UNIX crypt algorithm to encrypt the entered password. The password file is a simple text file, and you can edit it using any text editor on your system. There is no built-in mechanism to delete users, so if you want to remove someone from the username/password list, you must manually edit the file and delete the username password pair. Listing 12.2 shows a typical username/password file. For further details on how to use the htpasswd command, refer to Chapter 7.

01: dialup-30.austin.io.com - - [08/Oct/1995:15:05:48 -0500] "GET 
 /phoenix HTTP/1.0" 302 -
02: dialup-30.austin.io.com - - [08/Oct/1995:15:25:17 -0500] "GET /phoenix/
 index.shtml HTTP/1.0" 200 2860
03: crossnet.org - - [08/Oct/1995:19:56:45 -0500] "HEAD / HTTP/1.0" 200 0
04: dialup-2.austin.io.com - - [09/Oct/1995:07:54:56 -0500] "GET /leading-rein/
 orders HTTP/1.0" 401 -
05: dialup-48.austin.io.com - - [10/Oct/1995:11:07:59 -0500] "POST /cgiguide/
 chap7/reg1.cgi HTTP/1.0" 200 232
06: dialup-48.austin.io.com - - [10/Oct/1995:11:08:26 -0500] "POST /cgiguide/
 chap7/reg1.cgi HTTP/1.0" 200 232
07: onramp1-9.onr.com - - [10/Oct/1995:11:11:40 -0500] "GET / HTTP/1.0" 200 1529
08: onramp1-9.onr.com - - [10/Oct/1995:11:11:43 -0500] "GET 
 /accn.jpg HTTP/1.0" 200 20342
09: onramp1-9.onr.com - - [10/Oct/1995:11:11:46 -0500] "GET 
 /home.gif HTTP/1.0" 200 1331
10: dialup-3.austin.io.com - - [12/Oct/1995:08:04:27 -0500] "GET /cgi-bin/
 env.cgi?SavedName=+&First+Name=Eric&Last+Name=Herrmann&Street=&City=&State=&
11: zip=&Phone+Number=%28999%29+999-9999+&Email+Address=&simple=
12: +Submit+Registration+ HTTP/1.0" 200 1261

1: [Fri Oct 13 11:21:41 1995] access to /leading-rein/orders failed for dialup-
 10.austin.io.com, reason: user eric: password mismatch
2: [Fri Oct 13 11:31:07 1995] access to /leading-rein/orders failed for dialup-
 10.austin.io.com, reason: user eric: password mismatch
3: [Fri Oct 13 11:31:20 1995] access to /leading-rein/orders failed for dialup-
 10.austin.io.com, reason: user eric: password mismatch
4: [Fri Oct 13 11:31:23 1995] access to /leading-rein/orders failed for dialup-
 10.austin.io.com, reason: user eric: password mismatch
5: [Fri Oct 13 11:31:26 1995] access to /leading-rein/orders failed for dialup-
 10.austin.io.com, reason: user eric: password mismatch