Pipes

Sometimes, instead of reading input from a file, it's necessary to read input which comes from an program. For example, many operating system commands produce output which is not in a very convenient format; one of the important uses of languages like python is to be able to extract useful information from such commands, and present it in an appropriately formatted way. Similarly, it is very useful to be able to have python send its output to a command. For example, you may need to perform a statistical analysis by having python gather data and then send it to a program to perform the analysis. When you read output from a command or send output to a command the input stream to your program is known as a pipe. In python, you can open a pipe using the popen function of the os module So somewhere near the beginning of your program you need a line like

import os

to import the entire os module or

from os import popen

to bring just popen into the local namespace. In the examples that follow, I'll assume that the entire module has been imported. While this may not be the most efficient route, one benefit is that it's always very clear where a function like popen is coming from, since it needs to be refered to as os.popen.

One benefit of object-oriented programming is that, if the object returned by os.popen is designed to support the same methods as a file object, then the only difference between working with files and working with pipes is that different functions are used to initially create the objects on which these methods are invoked. This is exactly the way a pipe object is designed. Furthermore, the arguments to os.popen are exactly the same as those to the built-in open command, except that the first argument to os.popen is interpreted as an operating system command instead of a file name.

For example, suppose we want to read the output of the UNIX df command, which provides information about the space being used on a computer's hard drives. The output of the command might look something like this:

Filesystem         1024-blocks  Used Available Capacity Mounted on
/dev/hda1            1398534 1102892   223370     83%   /
/dev/hdb1            2970455 2060577   756261     73%   /new
/dev/hdb2            2970487 2540561   276307     90%   /new1

The goal of the program would be to add together the three columns which provide the amount of information about the drives and print a total for each column, along with an overall percentage of the capacity of the drives.

After opening a pipe to the command with os.popen, we can break apart each line and extract the numbers, add them together, and report the desired information:

import os,sys

try:
        df = os.popen('df -k','r')
except IOError:
        stderr.write('Couldn\'t run df command\n')
        sys.exit(1)

tot = used = avail = 0
while 1:
        line = df.readline()
        if not line : break
        line = line[:-1]
        if line[:10] == 'Filesystem' : continue
        parts = line.split()[1:]
        tot = tot + int(parts[0])
        used = used + int(parts[1])
        avail = avail + int(parts[2])

print 'Total: %d  Used: %d   Avail: %d  Capacity %2.0f%%' % \
              (tot,used,avail,float(used)/float(tot)  * 100.)

If the program was stored in a file called df.py, and was made executable, it would produce the following output:

% df.py
Total: 7339476  Used: 5704030   Avail: 1255938  Capacity 78%