sudo mount -t cifs -o user=abcd9999,servern=ITCASESEC,uid=foo,gid=1001 //itcasesec/abcd9999 /mnt/abcd9999

where your SIT username is “abcd9999″, your local username is “foo”, and the GID of “foo” is “1001″. The folder /mnt/abcd999 should already exist (or change it to where ever you would like to mount the folder).

SVN is the preferred way because it also gives you version control. For some subjects you may be given an SVN repository, but unfortunately this isn’t alway available.

If you just want to share the files, the chmod a+r FILENAME will allow this to happen, but your directories above might need to be publicly executable (chmod a+x DIRNAME).

Another option is the pair netfile/netget.

For netfile type:

netfile USERNAME FILENAMES

to send files to a particular user.

That user just types

netget

to view the files they have been sent. They can then choose to accept the file by entering y. Note though, that this overrides local files.

You will need an SFTP client.

Windows: FileZilla

Mac OSX: Cyberduck

All Linux distros should be able to connect by SFTP without additional software. Try typing sftp://user@host into the location bar of the file manager.
Connnect using

Host: ftp.ug.it.usyd.edu.au
Username Your SIT username
Password: Your SIT password
Port: 22
Protocol: SFTP

Make sure you connect using SFTP (port 22) rather than FTP (port 21). FTP is very insecure and will transmit your username and password unencrypted over the network.

With any of the above clients you should be able to bookmark the URL so you don’t have to type the hostname every time. Once again, make sure you choose SFTP for security. You can optionally save the password locally.

Firefox settings

In Firefox 2.0, go to Edit->Preferences->Advanced->Network->Connection Settings

Put in the proxy details (host www-cache.it.usyd.edu.au, port 8000) and check the box marked “Use the same proxy for all protocols”. At the bottom there is a list of domains to not use a proxy, add:
, .usyd.edu.au
Alternatively you can use the autoconfigure URL:
http://www.it.usyd.edu.au/proxy.pac

Unix Environment Variables

See Command Line Proxy Tutorial for more details.

Unix Environment

To use the School of IT proxies you want to set the following environment variables:
export http_proxy="http://www-cache.it.usyd.edu.au:8000"
export no_proxy="127.0.0.1, localhost, .usyd.edu.au"

sudo environment vars

sudo clears all environment variables before executing the command for security. This means if you run something like
$ sudo aptitude update
it won’t work because http_proxy gets cleared. You can fix this by doing the following. Be careful!
$ sudo visudo
Add this line:

Defaults env_keep="http_proxy https_proxy ftp_proxy no_proxy"

Mac OS X Location Settings

If you use Mac OS X and change locations frequently it’d be nice if your command line utilities picked up the proxy settings from System Preferences.

Have a look at http://delx.net.au/projects/proxyconf

To set up your personal webspace allowed on your School of IT (SIT) account, log into a congo machine either at uni or via SSH and type the following commands:

$ mkdir -p ~/lib/html
$ chmod 711 ~
$ chmod -R 711 ~/lib
$ cd ~/lib/html

Create your webpage in this directory (~/lib/html), say it is called index.html. Now to make it publicly visible on the internet, type the following:

$ chmod a+r index.html

Then assuming your Usyd username is abcd1234, open your Internet browser and go to http://www.ug.it.usyd.edu.au/~abcd1234/

The instructions in the SSH Tutorial are for use within any unix environment. It’s a good idea to read this anyway, as chances are you’ll be logging in to a Unix SSH server.

SSH Public/Private Keys

In a standard installation of PuTTY, there is a tool named PuTTYgen. This is the alternative tool to the Unix ssh-keygen command which generates a public/private key pair.

Generating the public/private key pair

When PuTTYgen is run, you will be presented with a dialog box that asks you to move your mouse over the it to generate some randomness. All you need to do at this stage is move your mouse so that the cursor moves over the dialog box. Continue to move the cursor until the progress bar reaches 100%.

Afterwards, click on the “Save private key” button to output your private key to a .ppk file. Store this file in a safe location on your computer. Click on the “Save public key” button to output your public key to file. The contents of this file is equivalent to the id_rsa.pub file generated using ssh-keygen, which should be appended to the ~/.ssh/authorized_keys file on the SSH server.

Using the key/pair

To use a generated key pair when you connect to a server, you will need to reference the private key (.ppk file) when you configure the PuTTY SSH session.

Starting PuTTY, you will see a hierarchy of settings on the left hand side of the window. The settings page that we are concerned with is located under Connection > SSH > Auth. When you have selected “Auth”, you will notice that the settings page on the right hand side will change. Click on the “Browse” button for “Private key file for authentication”, then locate and select the .ppk file you have generated earlier. After the key is selected, your connection to the server will then attempt the public/private key authentication method.

Optionally, you may provide a default username to enter when you connect to the server under “Login details” on the page at Connection > Data.

Web Proxy Port Forwarding

See the background information on the SSH Port Forwarding.

Choose a host to connect to using SSH, e.g. congo1.ug.it.usyd.edu.au (or just congo1) on the standard port 22.

Go to Connection->SSH->Tunnels and set: Source port: 8000 Destination www-cache.it.usyd.edu.au:8000 Don’t forget to click add!

Click open at the bottom to start the connection. Now you can set your browser to proxy through localhost:8000 and it’ll go through the SIT web cache.

Please read this page in it’s entirety. It’s very useful to understand how these techniques work.

Background

SIT means School of Information Technologies. This tutorial will only be useful if you’re an SIT student. UCC means University Computing Centre. They do lots of things, including supplying wireless networks for people to connect to throughout the university.

If you connect to a wireless network in a building like Wentworth, chances are it’s the UCC network. They supply a rather annoying (to me) VPN client which you’re supposed to use to access external sites through their wireless.

A proxy, for our purposes, means a computer that has greater network access than you, and can make requests on your behalf and return the result.

Procedure

The UCC wireless gives you limited connectivity without their VPN client. Thankfully this connectivity includes the ability to SSH to computers on the SIT network.

The SIT runs a web proxy, which requires an SIT username/password and allows access to web pages on the internet. The server is www-cache.it.usyd.edu.au and runs on port 8000. These are the settings you use when you’re directly connected to the SIT network, eg using a computer in the labs.

Note that your SIT internet quota is separate and in addition to your UCC quota. So if you’ve run out of your SIT web quota, you can still connect to the internet using VPN, and keep surfing the net with your UCC internet quota.

You cannot connect directly to the SIT web proxy from the wireless, and you cannot SSH directly to the SIT web proxy. However you can SSH to one of the congos, say congo1. SSH has a nice feature called TCP tunneling. Basically what it does is allow you to forward connections to a port on your computer (localhost) to a port on a remote computer, through an SSH connection. Here’s a diagram:

You connect to congo1 from your laptop.

[laptop]  <-ssh connection->  [congo1]

You want to use congo1 as a proxy to connect to the SIT web proxy. Nice eh?

         (forwarded connection
          piggybacks over SSH)
[laptop]  <-ssh connection->  [congo1]
                                | (forwarded connection)
             [www-cache.it.usyd.edu.au]

Details

Individual forwarding command:

ssh -L 1234:desthost:4321 userXYXY@proxyhost

To set this up, you run one command. See above for details on the options.
ssh -L 8000:www-cache.it.usyd.edu.au:8000 userXXXX@congo1.ug.it.usyd.edu.au
Now you have an SSH tunnel. Whenever you connect to localhost:8000 the connection will be forwarded through congo1 to www-cache.it.usyd.edu.au:8000

So you need to configure your web browser to use as the proxy: Host: localhost Port: 8000

This can be simplified by using the ssh config file (see see SUITS tutorial) instead of typing out this long command.

Host sitproxy
    HostName congo1.ug.it.usyd.edu.au
    User userXXXX
    LocalForward 8000 www-cache.it.usyd.edu.au:8000

Which can then be used via

$ ssh sitproxy


Jabber (Google Talk) Quick HOWTO

The SIT network allows unproxied outgoing connections on port 443 to all hosts. This port is usually used for HTTPS, that’s secure HTTP. Many Jabber servers (google talk, jabber.org.au) also listen on this port.

You can take advantage of this by proxying your Jabber connections from the UCC wireless through one of the congos. Run this command for google talk, or modify it as appropriate for your Jabber server.

ssh -L 5222:talk.google.com:443 userXXXX@congo1.ug.it.usyd.edu.au

Then go into your Jabber client and set it to connect to a specific host/port. Host: localhost Port: 5222

Postgrad Account

For those with a postgraduate account, you can connect through a postgraduate server, avoiding the proxy/download limit issues.

Forwarding your UniKey Email

UniKey email is your university-wide account. In 2009, the uni has moved student email accounts to a new service running on Outlook Live. Your email address is now unikey@uni.sydney.edu.au. Here are some tips from SUITS to get the most out of your new email account.

To transfer your email from your old account to your new one, follow these instructions.

If you like to automatically forward your uni email to a non-uni account, you can redirect all emails by following these instructions.

To pick up your uni email via POP, the POP server is outlook.com, port 995 (if you enable SSL). The username to provide is unikey@uni.sydney.edu.au. For more advanced setups including SMTP, take a look at this.

(This post contains info put together from Edmund’s blog and Enoch’s blog.)

Forwarding IT Email

If you have an account with SIT, then you’ll have an IT email address. While not much emails are sent this this address (unless you tell people to), you might get the odd email every now and then.

$ ssh USERNAME@mail.ug.it.usyd.edu.au
$ mailforward -a ADDRESS

The Fundamentals

In Unix, there is the tools philosophy. That is, there are lots of little programs that do one job, and do it well. These programs have short, often cryptic names, such as ls => list files, cp => copy. This makes them easier to type, and you’ll be typing them a lot!

You type these commands at a shell. The one you’ll be using is called bash, however there are others, such as tcsh or zsh. The shell gives you a prompt that will look something like this:
user@host:~$
-bash-2.05b$
When you see this prompt, it means the shell is ready for you to type a command. Generally when a command finishes successfully, nothing will be displayed but another prompt. For the purposes of this tutorial, we will abbreviate the prompt to simply a dollar sign: $ You can exit a shell by typing exit, logout, or pressing CTRL-D.

For almost all of these commands, you can specify flags, or options to modify their behaviour. For example, the ‘ls’ command will not show hidden files by default. Hidden files are files that start with a period, eg ‘.bashrc’. If you specify the ‘-a’ option, then ‘ls’ will show these files. Eg,
$ ls -a
These flags are also known as parameters, or arguments, and these terms are used interchangeably.

For some commands it may make sense to combine flags. In this case, you can either list them one after the other…
$ ls -l -a
… or you can specify them in the same argument.
$ ls -la
Unix commands may also take file or directory names as parameters. The ‘cat’ command will print the contents of a file to the screen, it needs to be told which file to print.
$ cat MyFile.txt
You can pass flags along with other parameters. Eg, the ‘-n’ option to cat tells it to precede each line with a line number.
$ cat -n MyFile.txt
It is considered good style to put all flag parameters before any others. In fact, many older Unix systems require this.

If you’re running a command that is taking a while, or you want to stop it for any reason, you can press CTRL-C to kill it (quit the program).

On a Unix system, everybody has their own account. Each account is given a home folder for that person to place all of the files. You can only access the files in your own home folder. You can refer to your home folder by the shorthand ~, or somebody elses home by the shorthand ~myfriend

Everything in Unix is case sensitive. That is, file.txt and fiLe.txt are different files, and can contain completely different content. This also goes for command names, as well as all flags and other parameters.

As you know, directories can contain files, or other directories in a hierarchical manner. If we have a folder called “projects”, with a folder called “mygame” inside, we can refer to the “mygame” folder as follows:

  projects/mygame

The slash character must be a forward slash, and is referred to as the directory separator. This is the only character that cannot be present in a Unix filename. If there is a file called “game.java” inside the “mygame” folder, then we can refer to it as:

  projects/mygame/game.java

These two examples are called paths, to a directory and a file respectively.

For various reasons, it is also not a great idea to have spaces in Unix filenames. You can, but generally you shouldn’t.

A simple text editor you can use in these examples is nano. It is worth putting in the time to learn a more advanced editor such as vim at some point though. Nano is a very simple editor to use. The commands it supports are displayed at the bottom of the screen. CTRL-O to save (write out), and CTRL-X to exit.

Basic Commands

This section provides a brief synopsis of the most frequently used commands, as well as a few examples of how to use them and what their usage looks like. For more details, you should read the manual page. Like this:
$ man somecommand
You should also try out all of these commands on some files you have. Don’t try these on Word documents or anything like that, these commands are designed to run on plain text files.

Command – ls

Lists the contents of a directory, optionally displaying detailed information about files or folders

List the files in the current directory
$ ls
Project.java Project.class
Display a detailed listing of these files
$ ls -l
-rw------- 1 username group 223 Jul 20 20:28 Project.java
-rw------- 1 username group 223 Jul 20 20:32 Project.class
List the files in a directory, including hidden files
$ ls -a
.hiddenFile Project.java Project.class
Combine the above two options
$ ls -la
-rw------- 1 username group 100 Jul 20 20:28 .hiddenFile
-rw------- 1 username group 223 Jul 20 20:28 Project.java
-rw------- 1 username group 5723 Jul 20 20:32 Project.class

Display the contents of the specified directory
$ ls documents
Report.txt Letter.txt

Command – cd

Change directory. This command allows you to change the current directory that you’re working in.

Change into the documents directory
$ cd documents
Change into the directory above the current, that is, change into the parent directory. Eg, say there is a folder “projects”, and inside that a folder “mygame”
$ cd projects/mygame
$ cd ..
Go up two directories at a time.
$ cd ../..

Command – mv

Move or rename a file or folder.

Rename file.txt into junk.txt
$ mv file.txt junk.txt
Move file.txt into a folder called trash
$ mv file.txt trash
Move documents/report.txt into a folder called trash
$ mv documents/report.txt trash
$ cd documents
$ mv report.txt ../trash
Rename the documents directory to be called MyDocuments
$ mv documents MyDocuments

Command – mkdir

Make a new empty directory
$ mkdir nameofdir

Command – cp

Copy files or folders

Copy file.txt to backup.txt
$ cp file.txt backup.txt
Copy file.txt into a new folder
$ mkdir mystuff
$ cp file.txt mystuff
Copy a directory
$ cp -a mydir newdir

Command – rm

Removes files and/or directories (BE CAREFUL!)

Remove a file
$ rm file.txt
Remove a directory
$ rm -r dir
Remove a directory without prompting for confirmation (!!! No way to recover the files once they’re gone !!!)
$ rm -rf dir

Command – cat

Displays the files listed on the command line on the screen (concatenate).

Display one file
$ cat file.txt
contents of file here
$ cat file.txt otherfile.txt
contents of file here
contents of other file here

Command – tail

Displays the last lines in a file

Display the last 10 lines in a file
$ tail file.txt
last 10 lines in the file go here
Displays the last 1 line in a file
$ tail -n 15 file.txt
last line of file

Command – head

Similar to tail, but displays the top of the file
$ head file.txt
first 10 lines of file

Command – wc

Displays statistics about a file: line count, word count, character count.

Display file statistics
$ wc file.txt
8 22 262 file.txt

Display just the line count
$ wc -l file.txt
8

Command – grep

Search for text in a file or files

Search for the word apple in file.txt
$ grep apple file.txt
line containing apple is here
Search for the word apple in all files in a directory
$ grep -R apple somedir
file.txt: line containing apple is here
otherfile.txt: also have apple in this line
Searching for the word apple in a file that doesn’t contain it prints nothing
$ grep apple file2.txt

Command – echo

Prints whatever arguments you pass to it on the screen

Display some text on the screen
$ echo Hello World
Hello World
Display text with special formatting characters
$ echo -e 'Hello\nWorld'
Hello
World

Command – diff

Displays the difference between two files

If we have file1.txt looking like this:
apple
banana
orange
grape
and file2.txt looking like this:
apple
orange
grape
We then run diff to compare them:
$ diff -u file1.txt file2.txt
--- file1.txt 2007-07-23 23:01:39.000000000 +1000
+++ file2.txt 2007-07-23 23:02:59.000000000 +1000
@@ -1,4 +1,3 @@
apple
-banana
orange
grape
If two files that are the same are compared, nothing is printed
$ diff -u file1.txt file1.txt

Command – scp

You can use SSH to copy files from one location to another. The syntax is very simple to the cp command, except one of the source or destination can be on a remote machine.

Copy the file thesis.txt to a directory projects on the server congo4.
$ scp thesis.txt userXYXY@congo4:projects/

Copy the entire projects directory on congo4 to the current directory locally.
$ scp -r userXYXY@congo4:projects/ .

Pipes and Redirection

Everything in Unix is a file. From the perspective of a program, this includes the keyboard and the screen.

All commands that you run from the shell have three special files available to them. Standard input (stdin), standard output (stdout) and standard error (stderr). stdin in is what you type on the keyboard, stdout and stderr you see displayed on the screen.

That is, when you type a command like
$ ls -l
and see the file listing displayed on the screen, that is because the program ‘ls’ is writing it to the file stdout. Perhaps you instead wish it to be written to a file called listing.txt You could redirect stdout from the program to a file, like this:
$ ls -l > listing.txt
Try out the program ‘bc’ on the command line. It is a calculator. You can type simple equations with integers and it will print the results. It reads questions as input from stdin (one per line), and writes outputs to stdout. Edit a file and put a few questions, like ‘2+2′ on it, then run this command.
$ bc < questions.txt
You should see your answers printed. You can even do this to have your answers written to a file.
$ bc < questions.txt > answers.txt
What if we want to check if any of the answers in the file were ‘4′? We could use the grep command to find out like this:
$ grep 4 answers.txt
If there is a line in the answers file that contains the character 4, then it will be printed.

We can make this more exact by using a regular expression (which is a separate topic that you should look up) like this:
$ grep '^4$' answers.txt
That will only match a 4 on a line by itself. Now this will print out all the lines that contain a 4.

Say we wanted to count how many of these answers were 4?
$ grep '^4$' answers.txt > count.txt
$ wc -l count.txt
You may be starting to see the kind of uses that you can put this to now. However it might seem a little messy to have to create all these temporary files just to get the result we want. There is a better way, we use what’s known as a pipe. If we pipe two programs together, then the shell will connect the stdout of one to the stdin of the other. We can do all of the above in one command like so:
$ bc < questions.txt | grep '^4$' | wc -l
Notice that both ‘grep’ and ‘wc’ will read from stdin if you don’t specify a file as a parameter. This form is much shorter, and more in the Unix spirit.

Useful Tips

You can scroll through the history of commands by pressing up and down. That is, press up to see the last command you ran, up again to see the next last and so on.

CTRL-R searches through previously typed commands. Eg, if you know you typed a really long ssh command and want to find it again, press CTRL-R, type ssh and you will see the most recently typed ssh command. Press CTRL-R to see the one before that, and so on.

All your history is stored in the file ~/.bash_history

If you use vi/vim then you can make the bash command line use those keybindings by running this command:
$ cp ~suits/pub/dotfiles/.inputrc ~/.inputrc

Printing

Print a file to printer (SIT printers are named in the form “pr1-<room number>” in general):
$ lpr -P<printer> <filename>
eg.
$ lpr -Ppr1-117 MyClass.java
Note that does not wrap or format the given file in any way to make it fit on the page. If you wish to print out code, with syntax highlighting and wrapping, use the following command:
$ a2ps -E -s2 -t4 -P<printer> <filename>
To check the print queue for a given printer, the command is
$ lpq -P<printer>
Each person with an SIT account has a print quota allocated to them for the duration of a semester. To check your current print quota
$ prquota