A large part of the power and flexibility of Linux/UNIX-like Operating Systems comes from the design of it's commands and programs. Programs are usually designed to operate efficiently by performing a single function. Programs can be generated by combining multiple basic commands. One command operates to produce data that is used to run the next set of commands.

This guide explains the basic difference between the different types of data streams with which a command line operator may come into contact.

Data Streams

There are three main types of streams used or produced by programs in UNIX-Like Operating Systems. There are nonstandard input and nonstandard output, but we will not be discussing those types of data streams.

stdin (Standard Input) (handle 0 used in redirects) This is usually text data typed from a keyboard, but can be from the stanard output of another program.

stdout (Standard Output) (handle 1 used in redirects) This is the data that is generated by a command or program which is the purpose of running commands or programs.

stderr (Standard Error) (handle 2 used in redirects) This is a secondary form of output which usually includes errors or alerts. 

Redirects

We can redirect output of commands into files or redirect the contents as input for a command. We do this by using something called an operator. The most common redirection operators are >, >>, <, <>, 2>, 2>>, and &>.

Operators

> Redirects stdout to a specified location. If the file exists, it will be overwritten. If it does not exist, it will be created.

>> Redirects stdout to a specified location. If the file exists, it will be appended. If it does not exist, it will be created.

< Redirects the contents of the file as to be used as stdin.

<> Redirects the contents of the file as to be used as stdin and redirects stdout to a specified location in a single command. If the output file exists, it will be overwritten. If it does not exist, it will be created.

2> Redirects the stderr to a specified location. If the file exists, it will be overwritten. If it does not exist, it will be created.

2>> Redirects stderr to a specified location. If the file exists, it will be appended. If it does not exist, it will be created.

&> Redirects the concatinated stdout and stderr to a specified location. If the file exists, it will be overwritten. If it does not exist, it will be created.

Format

command > stdout

command >> stdout

command < stdin

command < stdin > stdout

command 2> stderr

command 2>> stderr

command &> sdtouterr

Examples

echo servername > /etc/hostname

ls -al >> filename

sort < unsortedfile > sortedfile

Pipes

Pipes are used frequently to take the stdout (Standard Output) of a command or programm and use it as the stdin (Standard Input) of the subsequent command or program. The pipe is represented by the "|" (vertical bar) character. The vertical bar can usually be found on a QWERTY keyboard sharing a key with the "\" (backslash) immediately above the Enter/Return key. The pipe can be used for an arbitrary number of commands but here is an example of using 3 commands.

Example

command1 | command2 | command3

netstat | grep ESTABLISHED

The grep command is the most frequently used command with the pipe. It allows a user to filter results to narrow the output using words present in the output. Often the output of Linux commands can be very verbose. The grep command can be used to shorten a command to show only the important information.

Redirecting Multiple Output

You can redirect standard output to multiple places using the command tee. This means you can output to your screen and a file at the same time.

Example

ls -al | tee filename

netstat | grep ESTABLISHED | tee established.txt

This would allow you to see the result of the ls -al command, but would also write the output to the file.

This is only a primer for the use of data streams, redirects, pipes, and redirecting multiple outputs. You will likely encounter these operators throughout the use of command line and other tutorials due to their importance.