Tutorial: Unix Fundamentals
James Bunton on Jul 31st 2007
Unix is a powerful system, well worth getting to know for the productivity benefits you can obtain, and also for the experience of learning to use a new operating system. This tutorial provides an introduction to get you from no knowledge of Unix to having a basic skillset good enough for day to day tasks, in a position for you to learn more.
The Fundamentals
In Unix, there is the tools philosophy. That is, there are lots of little programs that do one job, and do it well. These programs have short, often cryptic names, such as ls => list files, cp => copy. This makes them easier to type, and you’ll be typing them a lot!
You type these commands at a shell. The one you’ll be using is called bash, however there are others, such as tcsh or zsh. The shell gives you a prompt that will look something like this:
user@host:~$
-bash-2.05b$
When you see this prompt, it means the shell is ready for you to type a command. Generally when a command finishes successfully, nothing will be displayed but another prompt. For the purposes of this tutorial, we will abbreviate the prompt to simply a dollar sign: $ You can exit a shell by typing exit, logout, or pressing CTRL-D.
For almost all of these commands, you can specify flags, or options to modify their behaviour. For example, the ‘ls’ command will not show hidden files by default. Hidden files are files that start with a period, eg ‘.bashrc’. If you specify the ‘-a’ option, then ‘ls’ will show these files. Eg,
$ ls -a
These flags are also known as parameters, or arguments, and these terms are used interchangeably.
For some commands it may make sense to combine flags. In this case, you can either list them one after the other…
$ ls -l -a
… or you can specify them in the same argument.
$ ls -la
Unix commands may also take file or directory names as parameters. The ‘cat’ command will print the contents of a file to the screen, it needs to be told which file to print.
$ cat MyFile.txt
You can pass flags along with other parameters. Eg, the ‘-n’ option to cat tells it to precede each line with a line number.
$ cat -n MyFile.txt
It is considered good style to put all flag parameters before any others. In fact, many older Unix systems require this.
If you’re running a command that is taking a while, or you want to stop it for any reason, you can press CTRL-C to kill it (quit the program).
On a Unix system, everybody has their own account. Each account is given a home folder for that person to place all of the files. You can only access the files in your own home folder. You can refer to your home folder by the shorthand ~, or somebody elses home by the shorthand ~myfriend
Everything in Unix is case sensitive. That is, file.txt and fiLe.txt are different files, and can contain completely different content. This also goes for command names, as well as all flags and other parameters.
As you know, directories can contain files, or other directories in a hierarchical manner. If we have a folder called “projects”, with a folder called “mygame” inside, we can refer to the “mygame” folder as follows:
projects/mygame
The slash character must be a forward slash, and is referred to as the directory separator. This is the only character that cannot be present in a Unix filename. If there is a file called “game.java” inside the “mygame” folder, then we can refer to it as:
projects/mygame/game.java
These two examples are called paths, to a directory and a file respectively.
For various reasons, it is also not a great idea to have spaces in Unix filenames. You can, but generally you shouldn’t.
A simple text editor you can use in these examples is nano. It is worth putting in the time to learn a more advanced editor such as vim at some point though. Nano is a very simple editor to use. The commands it supports are displayed at the bottom of the screen. CTRL-O to save (write out), and CTRL-X to exit.
Basic Commands
This section provides a brief synopsis of the most frequently used commands, as well as a few examples of how to use them and what their usage looks like. For more details, you should read the manual page. Like this:
$ man somecommand
You should also try out all of these commands on some files you have. Don’t try these on Word documents or anything like that, these commands are designed to run on plain text files.
Command – ls
Lists the contents of a directory, optionally displaying detailed information about files or folders
List the files in the current directory
$ ls
Project.java Project.class
Display a detailed listing of these files
$ ls -l
-rw------- 1 username group 223 Jul 20 20:28 Project.java
-rw------- 1 username group 223 Jul 20 20:32 Project.class
List the files in a directory, including hidden files
$ ls -a
.hiddenFile Project.java Project.class
Combine the above two options
$ ls -la
-rw------- 1 username group 100 Jul 20 20:28 .hiddenFile
-rw------- 1 username group 223 Jul 20 20:28 Project.java
-rw------- 1 username group 5723 Jul 20 20:32 Project.class
Display the contents of the specified directory
$ ls documents
Report.txt Letter.txt
Command – cd
Change directory. This command allows you to change the current directory that you’re working in.
Change into the documents directory
$ cd documents
Change into the directory above the current, that is, change into the parent directory. Eg, say there is a folder “projects”, and inside that a folder “mygame”
$ cd projects/mygame
$ cd ..
Go up two directories at a time.
$ cd ../..
Command – mv
Move or rename a file or folder.
Rename file.txt into junk.txt
$ mv file.txt junk.txt
Move file.txt into a folder called trash
$ mv file.txt trash
Move documents/report.txt into a folder called trash
$ mv documents/report.txt trash
$ cd documents
$ mv report.txt ../trash
Rename the documents directory to be called MyDocuments
$ mv documents MyDocuments
Command – mkdir
Make a new empty directory
$ mkdir nameofdir
Command – cp
Copy files or folders
Copy file.txt to backup.txt
$ cp file.txt backup.txt
Copy file.txt into a new folder
$ mkdir mystuff
$ cp file.txt mystuff
Copy a directory
$ cp -a mydir newdir
Command – rm
Removes files and/or directories (BE CAREFUL!)
Remove a file
$ rm file.txt
Remove a directory
$ rm -r dir
Remove a directory without prompting for confirmation (!!! No way to recover the files once they’re gone !!!)
$ rm -rf dir
Command – cat
Displays the files listed on the command line on the screen (concatenate).
Display one file
$ cat file.txt
contents of file here
$ cat file.txt otherfile.txt
contents of file here
contents of other file here
Command – tail
Displays the last lines in a file
Display the last 10 lines in a file
$ tail file.txt
last 10 lines in the file go here
Displays the last 1 line in a file
$ tail -n 15 file.txt
last line of file
Command – head
Similar to tail, but displays the top of the file
$ head file.txt
first 10 lines of file
Command – wc
Displays statistics about a file: line count, word count, character count.
Display file statistics
$ wc file.txt
8 22 262 file.txt
Display just the line count
$ wc -l file.txt
8
Command – grep
Search for text in a file or files
Search for the word apple in file.txt
$ grep apple file.txt
line containing apple is here
Search for the word apple in all files in a directory
$ grep -R apple somedir
file.txt: line containing apple is here
otherfile.txt: also have apple in this line
Searching for the word apple in a file that doesn’t contain it prints nothing
$ grep apple file2.txt
Command – echo
Prints whatever arguments you pass to it on the screen
Display some text on the screen
$ echo Hello World
Hello World
Display text with special formatting characters
$ echo -e 'Hello\nWorld'
Hello
World
Command – diff
Displays the difference between two files
If we have file1.txt looking like this:
apple
banana
orange
grape
and file2.txt looking like this:
apple
orange
grape
We then run diff to compare them:
$ diff -u file1.txt file2.txt
--- file1.txt 2007-07-23 23:01:39.000000000 +1000
+++ file2.txt 2007-07-23 23:02:59.000000000 +1000
@@ -1,4 +1,3 @@
apple
-banana
orange
grape
If two files that are the same are compared, nothing is printed
$ diff -u file1.txt file1.txt
Command – scp
You can use SSH to copy files from one location to another. The syntax is very simple to the cp command, except one of the source or destination can be on a remote machine.
Copy the file thesis.txt to a directory projects on the server congo4.
$ scp thesis.txt userXYXY@congo4:projects/
Copy the entire projects directory on congo4 to the current directory locally.
$ scp -r userXYXY@congo4:projects/ .
Pipes and Redirection
Everything in Unix is a file. From the perspective of a program, this includes the keyboard and the screen.
All commands that you run from the shell have three special files available to them. Standard input (stdin), standard output (stdout) and standard error (stderr). stdin in is what you type on the keyboard, stdout and stderr you see displayed on the screen.
That is, when you type a command like
$ ls -l
and see the file listing displayed on the screen, that is because the program ‘ls’ is writing it to the file stdout. Perhaps you instead wish it to be written to a file called listing.txt You could redirect stdout from the program to a file, like this:
$ ls -l > listing.txt
Try out the program ‘bc’ on the command line. It is a calculator. You can type simple equations with integers and it will print the results. It reads questions as input from stdin (one per line), and writes outputs to stdout. Edit a file and put a few questions, like ‘2+2′ on it, then run this command.
$ bc < questions.txt
You should see your answers printed. You can even do this to have your answers written to a file.
$ bc < questions.txt > answers.txt
What if we want to check if any of the answers in the file were ‘4′? We could use the grep command to find out like this:
$ grep 4 answers.txt
If there is a line in the answers file that contains the character 4, then it will be printed.
We can make this more exact by using a regular expression (which is a separate topic that you should look up) like this:
$ grep '^4$' answers.txt
That will only match a 4 on a line by itself. Now this will print out all the lines that contain a 4.
Say we wanted to count how many of these answers were 4?
$ grep '^4$' answers.txt > count.txt
$ wc -l count.txt
You may be starting to see the kind of uses that you can put this to now. However it might seem a little messy to have to create all these temporary files just to get the result we want. There is a better way, we use what’s known as a pipe. If we pipe two programs together, then the shell will connect the stdout of one to the stdin of the other. We can do all of the above in one command like so:
$ bc < questions.txt | grep '^4$' | wc -l
Notice that both ‘grep’ and ‘wc’ will read from stdin if you don’t specify a file as a parameter. This form is much shorter, and more in the Unix spirit.
Useful Tips
You can scroll through the history of commands by pressing up and down. That is, press up to see the last command you ran, up again to see the next last and so on.
CTRL-R searches through previously typed commands. Eg, if you know you typed a really long ssh command and want to find it again, press CTRL-R, type ssh and you will see the most recently typed ssh command. Press CTRL-R to see the one before that, and so on.
All your history is stored in the file ~/.bash_history
If you use vi/vim then you can make the bash command line use those keybindings by running this command:
$ cp ~suits/pub/dotfiles/.inputrc ~/.inputrc
Printing
Print a file to printer (SIT printers are named in the form “pr1-<room number>” in general):
$ lpr -P<printer> <filename>
eg.
$ lpr -Ppr1-117 MyClass.java
Note that does not wrap or format the given file in any way to make it fit on the page. If you wish to print out code, with syntax highlighting and wrapping, use the following command:
$ a2ps -E -s2 -t4 -P<printer> <filename>
To check the print queue for a given printer, the command is
$ lpq -P<printer>
Each person with an SIT account has a print quota allocated to them for the duration of a semester. To check your current print quota
$ prquota
Filed in Tutorials | No responses yet