First a note about notation:
The first step in using our machines is being able to log in to them.
First, you’ll need an SSH client program, such as PuTTy. Bodeen already has both PuTTy and a Windows SSH client ("SSH Secure Shell") installed.
If you are using PuTTy:
If you are using SSH Secure Shell:
Much easier: just open a Terminal window, then type:
You will be prompted for a password. Your password will be the same as your NetID password for Caesar, Canvas, etc.
When you log into Quest for the first time, you may be prompted for a default email address. Enter your email address and press enter. You may also be told that you do not have an ssh key set up yet. Your screen should look something like this:
ssh keys can be used to login to Quest without a password. However, they require you to connect from the same computer every time, and are not worth our trouble right now. So, press enter several times until you see your command prompt:
The first thing you probably wondered is, where did my slick interface with the Start Menu and Taskbar and Recycle Bin go? You don’t get them here; you have to navigate everything from the command line. Never fear! A typical Unix root file system (located at /) like the one on quest looks like this:
Because you are not a system administrator (a stroke of luck for you, I promise), you will be living and playing exclusively in the home directory, where all your files will be saved. In fact, when you log in, you are taken to the directory
Let’s start by making an example folder called "wahoo". To do that, type (mkdir = "make directory")
Now, how do I see that directory? (ls = "list")
You should see your new folder. Fine, but how do I get into it? (cd = "change directory")
Now what is in this directory?
Nothing! Because we are in a new, and empty, directory. Very boring, so let’s make a file in “wahoo.”
Now list directory contents again.
You can copy this file:
or move it (there is no such thing as renaming a file in linux; the operation is called moving. "moving" is slightly different from "renaming"):
If you want to go up one folder in the file system hierarchy,
Also, at any time, if you want to go to the root of your home directory (even plain "cd" works the same way),
Also, at any time, if you want to go back to the directory you were in:
You can also type the exact path of where you want to go.
This command, when issued from any location on the machine will take you to the wahoo folder in your home directory.
Another very important tool is Unix’s “auto-complete” feature. Say you had a folder called verylongfoldernameyoudontwanttotype. You could get there by typing
(or however much of the directory name was required to distinguish it from the other files in the current directory) and then pressing the
Even in the well-controlled sandbox theory, things go wrong more often than right. We like to say “Garbage in, garbage out.” You will want to clean up the garbage once in a while. Say you want to delete the wahoo folder, because your WAHOO code simulation didn’t quite work out. Try:
Tricked you: it didn’t work! You can delete files that way (but not directories):
This will wipe out the boring file. However, to delete directories, you need to modify the rm command with the –r flag, which stands for recursive (meaning it deletes everything in that directory permanently):
Be very careful with rm. There’s no Recyle Bin/Trash here from which you can recover your files. A deletion is permanent!
Deleting individual files and folders can get tedious when you really have a lot of garbage to get rid of. Do:
Now, you could delete these creatively named files one at a time, or you can use a wildcard in your rm command:
Unfortunately, none of the codes we will be using will provide you with a beautiful interface in which you can click the “calculate my final project” button. They are all text- and command line-based. Unix has several well-known text editors, and the one we will be explaining is vi (but feel free to use emacs, nano, etc. if you’re more comfortable with them). To get an example file to play with, do
This is an example input file for the PWSCF DFT code we’ll be using later in the quarter. To see it in the vi text editor,
and then press TAB to use that cool autocomplete we learned earlier!
In vi you can move around the file by using the up and down arrow keys. That suffices for small files, but for big ones you could be there all day. You will also want to use:
0 or ^ - go to the beginning of the current line (ignoring leading white-space)
u - undo changes
That’s all well and good, but what if you want to edit the file? By default, when you open something in vi, you’re in “NORMAL” mode and you can’t make changes. Pressing “
Now, edit the line:
so that it says:
to change the lattice parameter we would be using for Cu. How do you save the file? In the NORMAL mode,(note, the "
What if you mess up? It’s easy to do in vi; if you start trying to enter commands while in edit mode you can wipe out lots of stuff really fast. Just quit without saving with
You can find much more information about vi through vi's built in help system. You can access the help system by typing
If you want to know more about how vi/vim works and/or practice commands/editing/etc. once you login to Quest, do
This opens up a vim tutorial that goes through most useful basic features of vim. Vim is an *extremely* powerful editor once you learn its tricks and trades. Going through the entire tutorial should take you only 30 minutes or so and is highly recommended!
Sometimes, you would probably only want to look at what is inside the file without requiring to edit it. You could do
to see the contents of a file, navigate through it using arrow keys (or Vi navigation keys!) and use "q" to quit.
To see only the top "N" lines of a file, do
Similarly, to see the last "N" lines of a file, do
The important aspects of this .q file are:
Here you request how many CPUs you want to run on (total number of cores is ppn*nodes, with the maximum number of Processors Per Node (ppn) =12), the maximum time this job should be allowed to run (hours:minutes:seconds) and what allocation this run should be charged to (e20438; you don't ever need to change this line for this class unless you intend to use your own/your group's allocation on Quest).
NOTE: you can only set walltimes that are so long. Take a look here: http://www.it.northwestern.edu/research/user-services/quest/queues.html to understand what walltimes can be set for what number of cores and what priority each case would have (remember that a lower priority number means a higher priority!).
In general, going through the "Getting Started" section here: http://www.it.northwestern.edu/research/user-services/quest/index.html is very useful and highly recommended!
mpirun -np $NPROCS -machinefile $PBS_NODEFILE pw.x < cu_test.pw.in
This is where you tell the computer what program you’re running and what input file it should use. The bolded text should never needs to be changed. The underlined text is the part that you will edit to change what you want the job to do. Here: - "pw.x" is the name of the executable that needs to be run
- "<" tells the executable that the name of the input file follows it (this "redirection" is executable dependent! may not always work)
- "cu_test.pw.in" is the input file for the executable
In addition, the following could have been added to the line to redirect the output:
- ">" redirects the standard output from the executable (not the standard error; take a look here to learn more: https://en.wikipedia.org/wiki/Standard_streams) into the name of the file following it
- "cu_test.pw.out" is the name of the output file
So, the line would look something like this:
mpirun -np $NPROCS -machinefile $PBS_NODEFILE pw.x < cu_test.pw.in > cu_test.pw.out
To submit the job, you would do:
When your job is successfully submitted, usually the job scheduler prints the ID of the submitted job, e.g., 186554 or some number like that (useful to track jobs when you have many in the queue/running).
To see a list of (everyone’s) jobs running and waiting,
To see only yours, try
To see a list of all the jobs running in our class, try
To delete one of your jobs (either running or waiting),
If your job is not running or waiting in the queue, this command should give you the following error:
If you don't redirect the output to a file inside your batch script (using "> [output_filename]"; see above) the default name of the output file is usually "[name_of_the_batch_script].o.[job_ID]", e.g., "cu_test.q.o.186554" for the example above.
Congratulations! You're now an expert on using a Linux machine!
“You’ve taken your first step into a larger world”