In the 1960s Bell Laboratories participated in a venture to create a powerful new operating system called Multics (Multiplexed Information and Computing System). Later, however Bell Labs withdrew from this project, but the idea was still appealing and was taken up by Ken Thompson to create a multi-user environmental operating system in assembler. This was the birth of UNIX. In 1973 Dennis Ritchie rewrote the LJNIX operating system in the C programming language for portability.

The UNIX operating system is one of the most heavily used mainframe operating systems to date. There are a number of elements that have lead to the popularity of the UNIX operating system in the world today. The most notable factors are its portability among hardware platforms and the interactive programming environment that it offers to its users.

Since its birth in 1969, Ken Thompson's LNIX has undergone many transformations. There are now two main factions of UNIX: AT&T's System V and BSD (Berkeley Software Distribution).

There are many versions of UNIX including: Cpix, Genix, ldris, PC-IX, Perpos, Solaris, Ultrix, Venix, XENIX, etc.

Although UNIX is widespread throughout the world and quite commonly used in government and other institutes containing sensitive information, it was not designed with security in mind. This does not necessarily mean that UNIX does not have any security measures.

The intention of Ken Thompson was to assist programmers by creating a more communal environment, and not one of privacy. As programmers working on projects like to share information, sub-routines, etc., this eased the process and minimised time for them. The places that UNIX was installed on first (outside their own laboratories) were educational institutes and as these environments were similar there was no need for greater security. Furthermore the transition was easy.

Later on, UNIX came out of the educational field and entered the commercial world where it was installed for use in the government and the business environment, where the openness of its former environment no longer existed.

In this section, I will cover some of the more fundamental security risks in the UNIX system. I will discuss some common causes of UNIX system compromise in areas such as file protection, password security, networking and the concept of trusted hosts. My experience is on Solaris 2.3, SunOS 4.1.3 and OSF/I variants of UNIX and consequently this chapter's contents will reflect that!

File System

The file system in UNIX is divided into 3 categories: directories, ordinary files and special files (see table below) The system's files are laid out in a tree like fashion. Below are some important files on UNIX for one of my systems:

"/", Is an abbreviation for the root directory. It is the top directory of the system and all other directories are referenced in relation to it. As it is at the top of the hierarchy, it owns all the files below it as well. This is why the root account is so powerful.

The files /dev/mem, /dev/kmem and /dev/drum should never be readable by the world. An unscrupulous user can edit his or her UID and gain privileges, if it is wnteable by the world. If your system does not have these then these files should be owned by root and chmod with the value 600.

With few exceptions, all the other devices should be owned by root. An exception to this is the terminal, which is changed to be owned by the user currently logged on it.

In the UNIX operating system, files and directories are organised in a tree like structure with specific access modes. Every file on a multi-user system must have an owner. The owner of the file and directory has access to that file through the use of file protection modes. The setting of these modes, through permission bits (as octal digits), is the basis of UNIX's file system security (as with most computer systems).

Directory protection is a commonly overlooked component of file security in the UNIX system. Many system administrators and users are unaware of the fact (or do not care) that "publicly writeable directories provide the most opportunities for compromising UNIX system security." Administrators tend to make these open for users to move around and access public files and utilities. This can be disastrous, since files and other sub-directories within writeable directories can be moved out and replaced with different versions (Trojans), even if the contained files are unreadable or unwritable by others.

As a general rule, therefore, read and write access should be removed from all but system administrative directories (although this may not be always practicable).

UNIX file types

-rw-rw-rw- 1 im other 2236 Aug 2 16:51 shadpass.c

drwx ------ 2 im other 512 Aug 2 17:02 bin

File and directory permission bits set

The file bits on the file shadpass.c show that the owner has read and write permission, the group has read and write permission and others also have read and write permission. The directory gives a secure directory not readable, writeable, or executable by group and others (only accessible by the owner).

When you set the file permissions, any permissions that you do not give a file or directory are deleted (automatically not set). In addition there exists a fourth permission bit (as depicted in the table) available in each set of permission bits. This bit has a different control, depending on which set of bits it is influencing.

The fourth permission bit is set in the owner permission mode. This means that if the program (applies to executable files only) containing this bit set is executed then the program is executed as though the user owns the program, along with the normal permissions. A program that changes its user ID is called a SUID program (SUID). Leaving your terminal unattended can leave it susceptible to the simple SUID shell attack:

% cp /bin/csh /tmp/.text

% chmod 4777 /tmp/.text

These commands will create a shell with the SUID bit set. Whenever a perpetrator wants to assume your identity (thereby granting all your privileges), all that is needed is to execute this file (which is in the trnp directory).

This behaves exactly as the SUID bit except it is relevant to the group ID status of a file and allows a file to change its group number (SGID). Again it is totally meaningless to non executable files.

This is brought over from older versions of UNIX and has little relevance today. Later versions of UNIX have connected a new meaning to the sticky bit. If this bit is set on a directory then other users may not delete or rename files in the directory. It is particularly useful for setting on directories such as /tmp or anonymous FTPs' incoming directories as it prevents ordinary users deleting each other's files.

Although these features appear to offer substantial access control to UNIX system files, they can have drawbacks. There is always the chance that root (system administrator account) may have left a writeable file for others that is also set with the SUID bit on. With some modification, or by exploiting system bugs, an executable file like this would enable a user to become a superuser (SUID set to 0). Within a short period of time an unlawful person could completely compromise system security and make it inaccessible, even to other superusers.

Since the file protection mechanism is so important in the UNIX operating system, it stands to reason that the proper setting of permission bits is required for overall security. Besides user ignorance, the most common area of file compromise involves the default setting of permission bits at file creation. In some systems the default is octal 644, meaning that only the owner can write and read to a file, while all others can read the file.

However, in cases where sensitive data is present, the access for reading by others should be turned off. The UMASK value does in fact satisfy this requirement, allowing you to control which bits are set when a file or directory is created.

UMASK

When a file is created on the operating system (whether it is a binary or text file) it is created with certain permission bits set. These permission bits can be controlled since it is rarely desirable to allow group and world to have read and or write permission on the created file. UMASK derives its name from chmod (that is to say that chmod specifies which bits to turn on, and the UMASK variable specifies which bits to turn off).

By inserting this UMASK variable in a user's .profile, .cshrc or .login file, the default will be overwritten by the new settings at file creation. The chmod utility can be used to modify permission settings on files and directories.

The common default value on most systems is 022, this means that any created file should not be writeable by group and world users. If like me, you would like to turn off all group and world permissions so that any created file is not writeable, readable or executable by others then you should set the UMASK to 077. The UMASK value is calculated as follows:

We know that the base file creation mode is 0666 and the base creation mode for directories is 0777. Now to come to the value of 077 for the UMASK (for files), we look up which bits we do not want to assign. In this case they are read, write and execute permissions for the group and others. This, from Table 3, is 40+20+10+4+2+1=077.

Shells are the programs that take care of the complex task of interfacing with you and the low level functions of the operating system. They are not the operating system itself, they are the interface between you the user and the operating system. A shell waits for you to type a command and then executes it. Shells come in two flavours. The first class is the Bourne shell (named after Steve Bourne) which includes sh, ksh, bash and zsh. There is also another type of Bourne shell, the restricted shell (rsh). (Not to be confused with the remote shell command rsh) This shell prevents a user from executing commands outside a designated search path, changing directories and changing shell variables. It is used for accounts that a system administrator wishes to restrict. The second class are those derived from the C shell and include csh and tcsh.

You can tell what type of shell you are using by checking to see if your UINIX displays a "$" prompt. If it does then you have the Boume shell. If on the other hand your UNIX displays the "%" symbol as the prompt, then you are using the C shell. Shells also use various hidden files or "." files. In the C shell they are .cshrc (start-up), .login (executed when you login),.Iogout (this is executed when you logout) and .history (commands used in a session are saved here - useful to see what commands another user has been using).

Along with the normal characters, UNIX has characters that are interpreted in a special way by the operating system (the shell really).

STTY

Displays and sets your terminal characteristics. To display your current setting's type

PS

From the above we are interested in the following, the PID which is the Process Identification Number (range is 1-30,000), the TTY column lists the ten-ninal from which the process was started from, the UID tells us who owns the process and COMD is the command that being processed.

You can also list the processes of another user by issuing ps -u <user> We can kill a process:

% kill -9 2052 3450

Here we kill the bottom two processes using their respective PIDS.

LS

This command lists the files within a directory. Useful for detecting file system problems, a good practice to adopt is to look around various system directories with the aim of finding files that could be potential Trojans.

There are hidden files in UNIX, they normally have the prefix of "." such as.login, a normal Is does not show these, to make these appear you have to use the -a qualifier thus:

% ls -a

Beware of file names such as "..[TAB]","..." as these could be hidden directories containing Trojans or warez. There is normally a ".." and a "." file, but these are OK as they are simply used to take control of a directory.

The meaning of these fields is (from left to right):

Permission-bits, links, username, group file belongs to, bytes taken, date created, time last renewed and the name of the file or directory.

SU

Super User. With this command you assume the identity of another user. Without any parameters, this will create a super user shell. This is not the same as logging into their account and you still need the password for the account you want to su to (you would use the "-" switch to instruct su to use the user's environment) unless you are root, in which case you can su to any account without supplying a password.

This command is well monitored in the /var/adm/sulog and as such this file should be carefully reviewed. Suppose that you have access to user Fred's account and then switched to the root account, the /var/adm/sulog entry would look similar to:

SU 03/05/94 18:00 + tty 10 fred-root

This will tell you that user Fred su'd to the root account on 03/05/94 at 18:00 hours.

RSH

This command allows a user to maintain a remote shell, the principal operation of rsh is that it allows a user to execute a command remotely. When the command (If you omit this command, then rsh will proceed to use rlogin to gain access to the remote system instead.) has been terminated on the remote host rsh will also terminate. It uses trust to acquire the remote connection to execute a command. You must specify the hosts in either /etc/hosts.equiv or in your .rhosts file. You first specify the hostname followed by:

-1 username By default rsh will attempt to use the current username on the remote system to execute the command. You can specify this switch to instruct it to use another username on the host computer to execute the command for you.

-n Instruct rsh to redirect its input to /dev/null.

The shell that is executed on the remote computer is specified in the password file (discussed later in this chapter).

To hide against the system monitoring commands intruders use tricks such as rsh hostname -1 username csh -i or will occasionally edit log files.

W