If you would like to look at a book I suggest:
    www.tldp.org/LDP/Bash-Beginners-Guide/Bash-Beginners-Guide.pdf

HOME

By default your HOME directory is the directory in which you "land" when you log in. The path of this directory is usually:
     /home/username
It is also stored in the HOME environment variable (see further down):
     echo $HOME

In general there is not much space in your HOME but therefore, at UQAM, a backup of everything you keep under your HOME is done every night and kept for a week. The backup of Friday nights is even kept for a month.
3 hours. We also keep daily and weekly backups for up to 4 weeks. You if you ever remove something by accident or if the machine /or disk unit crashes, there will still be a copy of everything you had in your HOME.
Therefore, it is strongly suggested that you keep everything which is small and importent important in your HOME! Like programs, scripts, configuration files.If ever you need to get something back from the backup, talk to Nadjet Labassi.
If you need to store larger amounts of data, talk to Katja Winger

Check out the following link to find out how to retrieve data from the home backup.

Shell

A shell is a UNIX system command processor. It's Basically, it is a command language to tell the system what to do. There are two major shell (command interpreter) families: 
    Bourne, Korn, Bash Shell
    C Shell
   
The syntax of UNIX shell commands can vary from one shell to another.
At UQAM we use the Bourne Again Shell (bash) for the environment but often Korn Shell (ksh) for scripting. Therefore all commands below are bash commands.

...

Syntax of a

...

shell command

        command-name [ -option(s) filename(s) or arguments ]

Everything in square brackets [] is optional.

Example: ls -l filename

The first word of the command line is usually the command name. This is followed by the options/keys, if any, then the file and/or directory names. Options are usually preceded by a single or double dash, '-' or '--', and you –'.  One may use more than one option per command which usually can get combined. For example:

      You can also most of the time combine options:     $  ls -l -t -r
is the same as
    $        ls -lrt

In case you have a file name or an argument starting with a '-' you can use '--'. The command will NOT interpret anything following a '--' as an option/key.
For example if you are using 'grep' to look for '-a' in a file you could type:
    $  grep -iw -- -a filename

For more detailed explanations about Program Argument Syntax Conventions, please click on the here.

The examples on this reference card use bold case for command names and options and italics for arguments and filenames.

Documentation of specific commands - man & whatis

      whatis command    - displays whatis command - display a one-line summary about command
      man command         - display displays on-line manual pages about command

'man' can be use on almost all of the UNIX shell commands listed below .

Important Note about UNIX/Linux Commands

to get the complete description of the command.
To quit 'man' press 'q'.

Shell commands are case sensitive

UNIX/Linux is case sensitive. Type commands exactly as shown; most UNIX shell commands are lower case. File and directory names can be lower, upper, or mixed case but must be typed exactly as listed.

...

Most important shell commands

Quota and disk space usage

On most systems your home quota (the amount of data you can keep under your home) is limited. The following commands help to check quotas and sizes of directories and file systems:

List Files and Directories

Check where you are

      pwd : display the name of present working directory - tells you where you are

      present working directory
true_path directory-name : display the "true" name of directory-name, not the links. This command is a local add-on.

List Files and Directories

      ls directory/file-name(s) : list contents of directory

Here are some of the options I find most useful options of the 'ls' command:

...

Understand the "long list (-l)" output:

Example:

...

Image Added

See 'chmod' below about how to change the permissions.

Change Directory

      cd : to change to your home directory
      cd directory-name : to change to another directory
      cd - : to change back to the previous directory

Directory Abbreviations

      ~ : (tilde) home directory of the user
      ~username : another user's home directory
      . : current or working directory
      .. : parent of working directory

Make (or Create) Directory

      mkdirdirectory-name : create a directory called directory-name

options:
      -p  : parents: no error if existing, create parent directories as needed

Move (or Rename) Files and Directories

      mv  present-file-/directory-name  new-file-/directory-name  : rename a file
      mv  source-file-/directory-name(s)  destination-directory       : move one or multiple files and/or directories into another directory

options:
      -i  : interactive mode -> must confirm overwrites

Copy Files

      cp source-file-name  destination-file-name                            : copy a file into another filename
      cp source-file-/directory-name(s)  destination-directory  : copy one or more files/directories into another directory

options:
      -i  : interactive mode →  must confirm overwrites
      -r  : recursive. Copies the directory and all its files and sub-directories and their files etc.

Link Files

      ln -s target link_name : symbolically links link_name to target (can be used for files or directories)

Remove (or Delete) File and Directories

      rm filename                    : remove a file
      rm -r directory-name  : remove a directory with all its content
      rmdir directory-name : removes only empty directories

options:
      -i : interactive mode →  prompt for confirmation

Look at a ASCII/text Files

Create/Modify ASCII/text Files

To add an empty file you can use the command 'touch':

      touch filename

To edit a file it is best to use an editor. For example:

      emacs Reference Card (more intuitive)
      vi Reference Card (available on all Linux systems)

Special characters

General

Directory abbreviations

Command I/O

Wildcards

Wildcards are "place holder" for one or many characters. A number of characters are interpreted by shell

See 'chmod' below about how to change the permissions.

Change Working Directory

...

Directory Abbreviations

...

Make (or Create) Directory

...

Move (or Rename) Files and Directories

...

Copy Files

...

Link Files

ln -s target link_name : symbolically links link_name to target (can be file or directory)

Remove (or Delete) File

...

Look at a ASCII/text File

Wild cards

A number of characters are interpreted by the Unix shell before any other action takes place. These characters are known as wildcard characters. Usually these characters are used in place of filenames or directory names.

Here are some examples for the usage of wildcards:

• cat c* : displays any file whose name begins with c including the file c, if it exists.
• ls *.c : lists all files that have a .c extension.
• cp ../rmt? .  : copies every file in the parent directory that is four characters long and begins with rmt to the working directory. (The names will remain the same.)
• ls rmt[34567] : lists every file that begins with rmt and has a 3, 4, 5, 6, or 7 at the end.
• ls rmt[3-7] : does exactly the same thing as the previous example.
• ls ~ : lists your home directory.
• ls

...

• ~username : lists the home directory of the guy with the user

...

• name username.

Change File & Directory Access Permissions

(These are the permission you see with the 'ls -l' command - see above.)

      chmod [ who option permission ] filename
who file-/directory-name(s)

'who' can be any combination of:
    u (user)
    g (group)
    o (other)
    a (all) (i.e. ugo)

'option' adds or takes away permission, and can be:
    + (add permission),
    - (remove permission), or
    = (set to exactly this permission).

'permission' can be any combination of
    r (read)
    w (write)
    x (execute)

Example: chmod a+x filename - makes filename executable by everyone.

Display a message

echo

      echo"message" : diaplaysdisplays/prints 'message'.

Examples:
     $ echo Hello
             Hello
     $ echo "Hello there"
             Hello there

Variables

printf

      printf : produces output  according to a format.

Example:
      printf '%3.3d\n' $(( 1 + 15 ))
          016

'\n' gives a new line.
For more information check: man 3 printf

Quotes: single vs. double

Single quotes preserve all special characters and treat everything inside them as a literal string, including variables and commands. On the other hand, double quotes allow variable expansion and command substitution within the string and interpret some special characters as escape characters.

Examples:

      a="Hello there"
      echo $a
         Hello there
      echo "$a"
         Hello there
      echo '$a'
         $a

Variables

You can create variables and assign values to them.
There are two kinds of variables: local variables and environment variables.
The You can create variables and assign values to them.
There are two kinds of variables: local variables and environment variables.
The difference between the two is that environment variables are passed down to child processes (sub shell, scripts, programs)
There are already several environment variables set which you can see with the command:
    env

A variable name always needs to start with a letter.
To get the content of a variable the variable needs to be preceded with a dollar sign ($)

Examples:
    $  person=Alex     Alex          #  Never put a space before or after the equal sign!
    $  echo person
            person
    $  echo $person
            Alex
    $  echo $personis
   
    $  echo ${person}is
            Alexis
    $  echo "$person"
            Alex                         # => double quotes evaluate
    $  echo '$person'
            $person                   # => single quotes: No evaluation done !!!

Environment variables and the "export" command

If you just set 'variable=word' the variable will only be known in the current shell/script in which it is set.
If you "export" the variable becomes an environment variable and will also be know in all scripts which get executed in this shell/script.
For example ifyou export a variable in your file '~/.profile_usr' resp. '~/.profile.d/.interactive_profile' it will be available in all sub shells.

      export variable=wordcontent

The value of the variable is set to word 'content'.

Example:
    $ export CMCLNG=english
or
    $ CMCLNG=english
    $ export CMCLNG
   

Alias

      alias alias-string='command-string'
Alias
And alias abbreviates a command string with an alias string. For multi-command strings, enclose commands in single quotes.

Examples:
    $       alias data='cd ~data/Validation'
    $       alias vo='voir -iment'
    $       alias rd=r.diag
    $       alias rg='r.diag ggstat'

If you put aliases in your file '~/.profile.d/.interactive_profile' they will be available in all terminals/windows after a new login.

Date

      date : display time and date
Check 'man date' for different format options.

Evaluate a command

You can evaluate a command and store the answer in a variable.
For example if you want to store the current date in a variable you can use:
    $ 
      current_date=`date`
or
    $  current_date=$(date)
    $  echo $current_date

Arithmetics

...

or the newer method:
      current_date=$(date)
      echo $current_date 

Arithmetic calculations

Arithmetic calculation can be done within double round brackets. Examples:

      year=2013
      next_year=$((  year + 1  ))
or
      (( next_year = year + 1 ))
      echo $next_year
          2014

% : modulo / remainder
      echo $(( 10 % 3 ))
          1

Just be careful and never try arithmetic operations on numbers starting with a zero!!!

History

      history : lists the most recent commands

Exit/Close a terminal or process (script, command)

The easiest way to exit a script/process is by pressing the keys Ctrl-c.
The easiest way to close a terminal is by pressing the keys Ctrl-d.

Escape character

      \ : back slash, escape character

'\' prevents the following character from being "interpreted". There are several "special characters" (see above) in shell which have a certain "meaning" and function as little "commands".

Examples:
      person=Alex
      echo $person
          Alex                     => Evaluation done !!!
      echo \$person
          $person              => No evaluation done !!! - Because of the back slash
      echo "\$person"
          $person              => No evaluation done !!! - Because of the back slash
      echo '$person'
          $person              => No evaluation done !!! - Because of the single quots

When used as the very last character in a line it will suppresses the "new line".

Formatted print

...

History

...

Exit/Close a window or exit a script

...

Escape character

\ : back slash, escape character
Unfortunately '\' has different meanings depending on where/how it gets used.
Examples:
    $  person=Alex
    $  echo $person
    Alex
    $  echo \$person
    $person              => No evaluation done !!!
    $  echo "\$person"
    $person              => No evaluation done !!!
    $  echo '\$person'
    Alex                   => Evaluation done !!!
When used as the very last character in a line it will surpress the "new line".
    $ cat > text_file << EOF
    > Hello
    > How are you?
    > EOF
    $ cat text_file
    Hello
    How are you?
    $ cat > text_file << EOF
    > Hello, \
    > how are you?
    > EOF
    $ cat text_file
    Hello, how are you?

Command I/O

      >   :command output redirection (create new)
      >> : command output redirection(append)
      <   :command input redirection (from file)
      << : command input (from script orstandard or standard input)

Example:

...

...

...

...

...

...

null device

/dev/null or the null device is a special "file" that discards all data written to it but reports that the write operation succeeded.

File Operations

Search for Patterns in Files

      grep  search-string filename [filename...] (s) : to find and type out lines containing the string in a file

Most useful search options:
    -i   : case insensitive search
    -w : whole words only
    -v  : invert match; type out lines containing the string in a file
Options:
    -i : case insensitive search
    -w : whole words only
    -v : type out lines that do NOT contain the string (invert the search)
    -c : count the number of lines that contain a match

Search for files

lines that do NOT contain the string (invert the search)
    -c  : count the number of lines that contain a match

Search for files & directories

      find directory-name [-name (part_of_)name]

Useful for finding particular files or directories. 'find' descends the directory tree beginning at directory-name and locates files/directories that meet the search criteria
'find' is a very powerful and useful command. Use 'man find' to find out more.

Most useful search options:

For example, it can find pathname -name search-string
Useful for finding particular files. find descends the directory tree beginning at each pathname and locates files that meet the specified conditions. Wild cards can be used in the search string.
find is a very powerful and useful command. Use 'man find' to find out more.
It can for example also be used in combination with 'grep':
    $
     find . -name '*.scrsh' -exec grep case {} \;

The above command will print all lines containing the word 'case' of files with end on '.scrsh'.

Counting words/lines/characters in a file

      wc filename(s) : counts the number of words, lines, and characters in a file
Options
Most useful search options:
          -l   : count lines lines
          -c  : count characters characters
          -w : count words words

Pipe

      | : (pipe) redirect standard output of first command to standard input of second command

Example:
    $   ls -1 | wc -l : count the number of files in a directory

Compare files

      difffilename1 filename2    : compares contents of two files on a line-by-line basis

      xxdiff filename1 filename2 [filename3]  : graphically compares contents of two or three files on a line-by-line basis. Can also be used to edit files!
      xxdiff directory1 directory2    : graphically compares contents of two directories
xxdiff can also be used to edit files!

      tkdifffilename1 filename2    : graphically compares contents of two files on a line-by-line basis. Use on guillimin onstead of 'xxdiff'.

File Transfer

File Transfer

While 'cp' can only be used to copy files and directories on the same machine, there are tools that can copy files and directories from one machine to another.

rsync

'rsync' stands for remote sync. It is a remote and local file and directory synchronization tool.

Examples:
rsync : a fast, versatile, remote (and local) file/directory-copying tool
Usage:
        rsync  origin  destination
      rsync  user@origin username@remote_origin_machine:origin  destination
      rsync  origin  user@destination  origin  username@remote_destination_machine:destination
'origin' and 'destination' can be file(s) and/or directories.
Only the origin or the destination machine can be a nother machine, not both.
Options:

...

files and/or directories.

Only either the origin or the destination machine can be another machine, not both.
The 'username@' only needs to be given for the remote machine and only if the username is different from the one on the local machine.
The default directory on the remote machine is the home directory. If the remote file/directory is not in the home directory a path need to be specified, either from the home directory or a full path.

Most useful options:

scp

scp : secure copy (remote file copy program)

Usage: similar to rsync aboveOptions

Most useful options:

ftp / sftp

Also see:
sftp : secure file transfer program
ftp : Internet file transfer program

wget

wget : web downloader
GNU Wget is a free utility for non-interactive download of files from the Web.

Example:
    $        wget www.tldp.org/LDP/Bash-Beginners-Guide/Bash-Beginners-Guide.pdf

File archiving and compressions

tar

"Tape archive" : tape archive, can create, add to, list, and retrieve files form an archive file (a collection of files or even whole directory trees archived together as one file).Options:

Most useful options:

Examples:
Create     Create       tar-file:      tar -cvf   cvzf  tar-filename.tar  filenames_to_archive
    Unarchive tar-file:   tar -xvf   xvzf  tar-filename.tar

gzip

...

/ gunzip

Compress : compress resp. expand files.

    gzip gzip      filename         :      compresses the file filename. Whenever possible, each file is replaced(!) by one with the same name and the extension .gz
    gunzip filename.gz   : uncompresses the gz-file again.
Good to save space!

zip

...

/ unzip

Package : package and compress resp. unarchive and uncompress (archive) files and directories.

    zip      zip-file.zip file file/directory_list  : archives and compresses all files/directories in file_list into zip-file.zip. Leaves original files untouched.
    unzip zip-file.zip                                      : unarchives and uncompresses all files/directories in zip-file.zip.
    unzip zip-file.zip aaa filename                   : unarchives and uncompresses only file 'filename' aaa fromzip-file.zip. Wildcards can get used.

Cursor movements in command line

emacs mode:   

...

...

...

...

...

...

...

...

...

Conditional structures

if-then-else

...

'elif expression ; thenelif' and 'else' are optional!!!expressions can look like this

Example 'expressions':

Arithmetic expression examples ( a=2 ): 
    (( a == 3 )) 
    (( b != 3 ))
    (( $a >= 3 ))
In Arithmetic expressions '(( ... ))' it is not necessary to put the '$' infront in front of variables since strings are not allowed so all strings are assumed to be variables anyways.

String expressions examples ( abc="Bonjour" ):
equal:
    [[ "$abc" == "Hello" ]]
not equal:
    [[ "$abc" != "Hello" ]]

The more common but less recomended recommended way is:
    [ $a -le 3 ]
resp.
    [ "$abc" -nq "Hello" ]

Note: It is mandatory to keep a 'space' just before and after the double and single square brackets!!!

and: &&
or: ||

If you want to know more about the comparison operators try:
    $      man test
or
    $      man bash            -> CONDITIONAL EXPRESSIONS

case-statement

A case statement is a conditional control structure that allows a selection to be made between several sets of program statements. It is a popular alternative to the if-then-else statement when you need to evaluate multiple different choices.

...

...

...

The pipe character '|' serves here as an "or".
The last line of each statement needs to be finished with ';;' at the end.

Examples:

...

...

...

...

...

...

Loops

Loops can be used to execute (parts of) code repeatedly.

for-loop

Loop over different fixed elements

...

...

Examples:

Loop over numbers:

...

Loop over files:

while-loop

Repeat statement(s) while 'condition' is true.

...

Example:

You One can also use a while-loop to read lines from an ASCII file.
Example:

...

...

...

break

...

& continue

To interrupt a loop you can use 'break' and 'continue'.
      break       interrupts the loop
      continue interrupts the current cycle of the loop

Manipulating Variables

Example string:
     $       string="ABC_abc_123"String

Get string length:

     $       echo ${#string}

Remove patterns from string

...

...

...

...

...

...

Look at these examples and you will understand the meaning of the above:
    $      string="ABC_abc_123"
    $      echo ${string#*_}     
             abc_123                                             123                                           => removes everything before first occurance of '_'
    $      echo ${string##*_}
             123                                                    => removes everything before last occurance of '_'
    $      echo ${string%_*}
             ABC_abc                                            abc                                          => removes everything after last occurance of '_'
    $      echo ${string%%_*}
             ABC                                                   => removes everything after first occurance of '_'
    $      echo ${string:s:n}
             abc                                                    => gets n characters, starting from position s

cut

Cut out/print part of the content of a variable:
cut prints selected parts of a string.
Options
Most useful options:

Examples:
    $      abc="I am hungry"
    $      echo $abc | cut -c 6-9
             hung                                                => returns characters 6-9
    $      echo $abc | cut -c 6-
             hungry                                            => returns characters 6 to end
    $      echo $abc | cut -d' ' -f 3
             hungry                                            => returns 3rd element with ' ' used as seperator
    $    abc="I am hungry. Are you?"
    $      echo $abc | cut -d. -f 2
              Are you?                                        => returns 2nd element with '.' used as seperator

set

When you call calling set without options but with one or more arguments, it sets the values of the command line argument variables ($1-$9$n) to its arguments.
Examples:
    $      abc="I am hungry"
    $      set $abc
    $      echo $3
             hungry

When options are specified, they set or unset shell attributes.
Options:

For more options check man page: man set
Examples:
    $      abc="I am hungry"
    $      set -x
    $      abc="I am hungry"
      + abc='I am hungry'

& -> Send job in background

You can send a job in the background by adding a '&' at the end of the command line.
This is useful to get the prompt back - beeing being able to continue using the window - when calling a program whic which opens a nother another window, like for example Matlab, emacs, xrec.
Example:
    $ matlab      emacs &

In case you forgot to add the '&' you can still send an already launched job into the background with 'Ctrl-z' followed by 'bg' (for background):
Example:
    $      emacs
         Ctrl-z
            [1]+  Stopped                 emacs
    $ bg
         bg
        [1]+ emacs &

Check and kill running processes

ps reports a snapshot of the current processesOptions

Most useful options:

Try for example:
     ps -fu $USER

Since the lines can get pretty long you can also pipe the output into less to see the full lines, for example:
    $      ps -fu $USER | less

Also have a look at the 'STIME' to see how old the processes are.

If ever you need to kill one of these jobs you can use 'kill' followed by the process ID, 'PID' (second column).
If a normal 'kill' does not work try 'kill -9' followed by the 'PID'.:
    kill -9 PID