Basic Linux Skills 1

*************************************************************
        .--.
       |o_o |
       |:_/ |     Let's learn some Basic Linux Skills 
      //   \ \                     by
     (|     | )   Pete Vargas Mas <pete.vargasmas@gmail.com>
    /'\_   _/`\
    \___)=(___/
*************************************************************                
This work is licensed under a
Creative Commons Attribution-NoDerivatives 4.0 
International License.
*************************************************************
Why is this here?

Hello! Since you are here reading this, I am guessing that you
are interested in learning a few new useful skills that will
help you with accessing a computer running Linux. My intention
here is to provide you with some basic generic Linux skills to 
help you along. This is not a definitive guide, as there are a
lot of larger better books out there that can teach you a lot 
more, and thousands of websites as well. But, I do like to share
and teach, so here you go.
**************************************************************

What is Linux?

Linux is an operating system invented by Linus Benedict Torvalds 
in August of 1991, in Helsinki Finland. Initially intended to be 
just for fun, Linux soon joined forces with the GNU Project to 
become a complete operating system. In December of 1992, Linux 
version 0.99 was released under the GNU General Public License.
Linux version 1.0 was released in March of 1994.

What is a Linux Distribution?

A Linux distribution is a combination of a Linux Kernel, 
GNU utilities, and third-party software. Examples of Linux 
distributions include projects such as Debian Linux, company 
sponsored distributions such as Fedora, Ubuntu, and SuSe, 
and commercial distributions such as Red Hat.

==================================================================

In Linux, everything is represented by a file. A file can be 
data, an executable command, it may represent a hardware device, 
a directory, or a logical communication channel. Therefore we 
can use the name of a file as an option to a Linux command, at 
the prompt or Command Line.

File and directory paths in Linux use the forward slash "/" to 
separate directory names in a path.

A PATH shows you how to get to a FILE. There are two types 
of paths:

ABSOLUTE - Starts at the root directory, and shows us the 
way to the file - for example: 
     /usr/share/doc/package/readme.txt

RELATIVE - Starts in our current working directory (CWD), and 
shows us the way to the file:
     ../../doc/readme.txt 
(from the current working directory, go up 2 directories and 
down into the doc directory, where there is a file called
readme.txt)

Note:  .. means the directory above our current working directory
        . means the current working directory
       (The pwd command will tell you what directory you are in)

/ represents the root directory. All sub-directories descend from 
root like the branches of an upside-down tree. 

                   /
                   |
----------------------------------------------------------------
 |       |             |          |       |                    |
bin     boot          root       usr     etc                  sbin
         |             |          |       |-------------|
        grub          bin       local     |             |
                                          |             |
                                        sysconfig      ssh

Avoid confusion: root is also the username of the "Administrator" 
or "Super User". "/root" is the root users home directory.

==================================================================
Commands for finding your way around the file system:
==================================================================
pwd               "print working directory", i.e. show the name
                  of the current directory.
cd                Change current directory to your HOME directory.
                  cd ~ also works.
cd /usr/share     Change current directory to /usr/share.
cd doc            Change current directory to "doc" which is a
                  sub-directory of the current
                  directory.
cd ..             Change current directory to the parent directory
                  of the current directory. That single space needs
                  to be there.
cd ../..          Change current directory to the directory two levels
                  up from the current directory.
cd $TESTER        Change current directory to the directory defined by
                  the environment variable 'TESTER'.
cd ~bob           Change the current directory to the user bob's home
                  directory (if you have permission).

Get a directory listing:

ls            list a directory in short format (file/directory 
              names only)
ls -l         list a directory in long (verbose) format, including
              metadata
ls -al        verbose listing which includes hidden files 
              (file whose name begins with a period)
ls -ld /dir/  verbose listing of a directory named /dir/ (not 
              the contents, but the directory itself)

==================================================================
Directory Listing Example with file permissions (perm):
$ ls -l 
drwxr-xr-x    3 bob      user        1024 Dec 12 08:45 MY_EARNINGS
-rw-r--r--    1 alice    user      661332 Nov  6 11:18 work1.tar.gz
^ ^  ^  ^     ^   ^       ^           ^      ^    ^      ^
| |  |  |     |   |       |           |      |    |      |  
| |  |  |     | owner   group       size   date  time    name 
| |  |  |     number of links to file or directory contents
| |  |  perm for Other: r = read, w = write, x = execute -=no perm
| |  perm for members of Group: r = read, w = write, x = exec -=no perm
| perm for Owner of file: r = read, w = write, x = execute -=no perm
type of file: - = normal file, d = directory, l = symbolic link, and others...

Note: Other is also referred to as "Everybody" or "The World". It means ANY user that can access the system.

==================================================================
File permissions and attributes:
==================================================================
File permissions determine which user or process is allowed to
access a file or directory, and what type of access they are allowed. There are three basic permissions: read (r), write (w), 
and execute (x). There are also three "special" permissions: 
Set User ID (SUID), Set Group ID (SGID) and Sticky Bit.

Attributes are metadata (metadata is information about information) such as the file size, creationdate/time, and owner/group membership.

OCTAL: Each permission setting has a numerical value: 
     r=4, w=2, and x=1
Adding up these values gives us a number to represent the user
permissions, one for the group, and one for other (which means
everyone else).

-------------------            For Example:
|  U  |  G  |  O  |            rwxr-x----
|user |group|other|  
|-----|-----|-----|            User = rwx = 4+2+1 = 7
|r|w|x|r|w|x|r|w|x|            Group = r-x = 4+0+1 = 6
| | | | | | | | | |            Other = --- = 0+0+0 = 0
|4|2|1|4|2|1|4|2|1|
-------------------            

If we want to give everyone read-only permission to the file, we
must change the permissions for Other:

Other r-- = 4+0+0 = 4

Using the 3-digit octal number created above, we can use the chmod 
command to change the permissions of a file or directory:

   chmod 764 testfile1
   changes the permissions of the file testfile1 to rwxr-xr--

NOTE: Directories must have the execute bit in order for users to 
be able to cd to, and get directory listings of, the directory itself.

An additional fourth octal digit can be used to specify additional 
permissions for a file or directory. These additional bits are known 
as the "sticky bits":

SUID, SGID and Sticky bit are modified by adding a fourth octal 
digit to the permissions value. Each of the three types of sticky bits 
have their own value.

SUID (set user id) = 4    
SGID (set group id) = 2    
Sticky = 1  

Each sticky bit adds a specific functionality:

SUID set on a file:  Normally an executable file will be executed 
                     under the same userid as the user executing 
                     the file. Setting the SUID bit will cause the 
                     file to be executed under the userid of the 
                     actual owner of the file.                    

SUID on a directory: Any file saved to a directory with the SUID
                     bit set, will be saved with the userid 
                     owning the file set the same as the userid 
                     that owns the directory, instead of being 
                     set the same as the userid of the user 
                     saving the file.

SGID set on a file:  Will execute the file under the groupid 
                     of the group the file belongs to, instead 
                     of the group the executor of the file 
                     belongs to.

SGID on a directory: Any file saved to a directory with the SGID
                     bit set, will have the same group membership
                     set as whatever the directory itself has 
                     set, instead of the group of the userid 
                     saving the file.

Sticky bit:          This bit is used in group directories so 
                     that only the owner of the file, and the 
                     root user, are allowed to delete the file. 
                     The /tmp directory in Linux systems has 
                     this bit set. 

NOTE: Setting SUID on a script owned by root and set to rwx for others is a bad idea, because anyone can edit and execute the script as root!
Keep in mind that when a user attempts to execute a program
(executable file) in Linux, the program will run with the 
permissions of the user executing the file, unless you have set
special permissions as shown above. Do not forget this.


==================================================================
How to create a group shared directory:
==================================================================

In order to create a shared (or group) directory, create a 
group name, set the directory to that group name, add the
users requiring access to the group, and set the directory
permissions so that Group has rwx permissions on the directory, 
and the SGID bit is set. Anyone saving a file in that directory 
will save it with the name of the shared group, and it will be
accessible by all members of that group. To stop users from 
deleting other users files, set the Sticky bit on the directory.

Here's an example of what the directory listing would look like 
with the "specialbits" set:

drwsrwsr-t. 2 root root 4096 Jan  4 14:20 shared-dir
   ^  ^  ^--sticky bit
   |  |-----SGID
   |--------SUID

If the underlying execute permission is set, the two "s" and 
one "t" characters would appear in upper case (S, S, T). 
Conversely, if the underlying permission is not set, the special 
bit designator will appear in lower case. 

So for example, to set SUID and SGID for a file with rwxr-xr-x 
(755) permissions:

SUID + SGID = 4 + 2 = 6

Stick that 6 in front of the other permissions and we get 6755: 

chmod 6755 filename.txt

Replacing the "6" with a "0" removes the special bit, as in the 
following example:

chmod 0755 filename.txt

If you have trouble remembering how to assign permissions using 
the octal method, another method exists, called the "Symbolic 
method".

The symbolic method uses letters to represent the permissions. 

User=u, Group=g and Other=o
Read=r, Write=w and Execute=x

For example:

To change the permissions of testfile1 to rwxr-xr-- use:        chmod u=rwx,g=rx,o=r testfile1

You must be the owner of the file/directory (or be the root 
user), before you can change any permissions or attributes 
of a file/directory. 

In order to change the ownership or group membership of a 
file or directory, use the following commands:

To change the userid that owns a file:   chown bob testfile1

To change the group who owns a file:     chgrp wheel testfile1


==================================================================
Moving, renaming, and copying files:
==================================================================
The cp command copies files:

Ex: cp file1 file2       file1 will be copied to file2. 
                         If file2 doesn't exist, it will 
                         be created. If file 2 does exist, 
                         it will be overwritten.

The mv command moves files: 

mv file1 file2           file1 will be copied to file2. 
                         The original file1 gets deleted. 
                         The effective behavior is like a
                         filename rename.
mv file1 ~/AAA/          file1 will be copied to a subdirectory
                         called "AAA" in the current users 
                         home directory.

The rm command removes one or more files:

rm file1                file1 will be deleted

The rm command by itself cannot be used to delete a directory. However, the -r option (recursive) and -f option (force) will 
allow rm to delete a directory. 

rm -rf /data            
The /data, and all files/directories under it in 
the directory tree, will be deleted. Be very careful
with this command, as it will allow you to delete 
the root (/) directory, which basically renders your
system un-bootable and unusable. 

The following commands are for managing directories:

rmdir dir1              
dir1 (and other listed directories) will be deleted if 
they are empty
mkdir dir1              
create (make) directory dir1
mkdir -p /data/goomba   
create a directory named "goomba", including 
all implied directories in the path. In this
example, if the directory /data does not exist, 
it will be created.
rmdir dir1 [dir2...]    
remove (delete) an empty directory

=========================
Viewing and editing files
=========================

file filename     
The file command will examine the first two bytes 
of a file and determine what type of file we have.
This is useful for identifying which file is in ascii 
format and can be accessed/edited with the following 
commands.

cat filename      
The cat (conCATenate) command dumps the contents of 
a file to the screen in ascii. This is a quick and 
dirty way to see the contents of a file. 
less filename     
The less command will dump a file to the screen., 
one screen-full at a time. Use arrow buttons, 
spacebar, or PgUp/PgDn to advance to the next page. 

vi filename       
Edit a file using the vi text editor. More on vi below.   

head filename     
Show the first 10 lines of a file.
head -n  filename 
Show the first n lines of a file.

tail filename     
Show the last 10 lines of a file.
tail -n filename  
Show the last n lines of a file.
 
tail -f filename  
Show the last 10 lines of a file, but don't stop, show 
updates to the file. Useful when watching files that 
update frequently, such as logs.

==================================================================
Shell Environment variables
==================================================================

A shell is a command-line interpreter that provides a 
text-based user interface to the operating system. A user 
can direct the operation of the system by entering commands 
as text for a command line interpreter to execute, or by 
creating text scripts (shell scripts) containing one or more 
such commands. All of the above listed commands are used in 
a shell. The default shell in most linux systems is /bin/bash 
(aka the BASH Shell, aka the Bourne Again SHell). 

Your shell can store values in variables for later use, by 
using "environmental variables". For example, under the 
bash shell:

export TEST1=/usr/local/test              
Defines the variable TEST1
with the value /usr/local/test
export LD_LIBRARY_PATH=$TEST1/Linux/lib   
Defines the variable LD_LIBRARY_PATH with 
the value of TEST1 with /Linux/lib appended, 
or /usr/local/test/Linux/lib 

These environmental variables are made available to any process/command that runs in the bash shell.
By prefixing $ (the dollar character) to the variable 
name, you can evaluate the variable in any command:

cd $TEST1         
Changes your present working directory to the value of TEST1
echo $TEST1       
Prints out the value of TEST1, or /usr/local/test

When the command is executed, bash replaces the name of the variable with the value of the variable. 

=========================
Interactive Shell History
=========================

BASH remembers what commands you typed, and you can use the 
up-arrow and down-arrowkeys to access your previous commands,
edit them, and re-execute them.

history      Show my command line history
history -c   Clear my command line history

Press <Ctrl>+<r> at the command prompt to search backwards
through the command line history.

Use !! to repeat the last command. For example, lets say you 
execute the command "iptables -L" and receive an error message
saying you are not allowed to do that because you are not the 
root user. But, you are allowed to use the sudo command. Instead 
of logging in as root and retyping the command, remain logged 
in as a unprivileged user and use:  sudo !!

BASH will interpret !! as meaning the previous command executed,
which was "iptables -L", and the command will execute with root
permissions. If you are asked to enter a password, be aware that 
the system is asking for the password of the currently logged 
in user. Not the password of the root user. This is how the 
system verifies that the command is being executed by the 
current user, and not someone who just happened to walk by 
and see your unattended terminal.

==================================================================
Filename Completion
==================================================================
BASH allows you to type in a partial filename, and double-tap 
the TAB key, which will cause bash try to auto-complete the 
filename (assuming it's unique), or show you possible matches 
in the command path for what you typed.

Use the command "man bash" to see the BASH shell man (manual) 
page.
==================================================================
Input/Output Redirection:
==================================================================
STDIN = Standard Input (to a file or process)
STDOUT = Standard Output (from a file or process)
STDERR = Standard Error (from a process)

Standard Input is generally understood to be the keyboard. 
However, STDIN can also come from a data file or another 
command.

Standard Output is generally understood to be the screen or
console. STDOUTcan be diverted to another file, or to another
process and become the STDIN forthat other process.

The ">" and ">>" directives are used to direct where output 
is to go.

grep string filename > newfile       
Redirects the STDOUT of the grep command to a file 
called 'newfile'. 
grep string filename >> existfile    
Appends the output of the grep command 
to the end of a file called 'existfile'.

If you use ">" to redirect the stdout of a command to an 
existing file, the current contents of the file will be 
overwritten by the new data from the redirected command.

If you use ">>" to redirect the stdout of a command to a 
file that does not exist, the file will be created and the 
stdout of the command will be placed in the new file. 

Note:  The name of the "grep" command comes from the "ed" 
command g/re/p (globally search a regular expression and 
print), which has the same effect: doing a global search 
with the regular expression and printing all matching lines.
The "ed" command is a line-oriented text editor (use vi 
instead).

The redirection directives, > and >> can be used on the 
output of most commands to direct their output to a file. 
Be careful! If the file already exists, > will override the 
file and replace all the contents with the new input.

Pipes:

The pipe symbol "|" is used to redirect the STDOUT of one 
command to the STDIN of another. In other words, you want 
to use the output of one command as the input to another 
command.

For example:

ls -l | less                 This commands takes the output 
                             of the long format directory list
                             command "ls -l" and pipes it 
                             through the less command (also 
                             known as a filter). In this case 
                             a very long list of files can be 
                             viewed a page at a time.

cat file1 | sort | tail      Dump file1 to STDOUT, but pipe the
                             output to the sort command, which
                             orders the lines in file1 into
                             alphabetical order, then pipe
                             the STDOUT of the sort command 
                             into the STDIN of the tail 
                             command, which will output the
                             results of the sort command one 
                             page at a time.

Combining this with the redirection directives, we can do something 
like this:

cat file1 | sort > output.txt

This sequence will dump file1 to stdout, piping the output 
to the sort command, which orders the lines into alphabetical 
order, then redirects the output of the sort command into a 
disk file called "output.txt",which we can then examine at 
our leisure. 
               

==================================================================
Command Substitution
==================================================================

You can also use the output of one command as an input to 
another command by using command substitution, which is invoked 
by enclosing the command in parenthesis with a $ symbol in 
front, like in this example:

       cat $(find /root/ -name rhel*)

The find command will execute, and look in the /root/ directory
for a file that starts with "rhel". It will replace the substituted
command with the output of the find command, and execute the cat
command. 

Assuming there is only one file that begins with "rhel", the above
example is interpreted by bash as: 

       cat /root/rhel5_hosts  

==================================================================
Searching for strings in files with the grep command
==================================================================
grep "string" filename    Prints all the lines in filename that 
                          contain "string".
grep -v "string" filename Prints all the lines in filename that 
                          do NOT contain "string".
ls -l | grep bobo         Executes the ls command to obtain a long 
                          list of the current working directory, and
                          pipes the stdout of the ls command to the 
                          stdin of the grep command. The grep
                          command takes all that input, and only
                          outputs any lines containing the string
                          "bobo".

If you want to grep for a string that contains Regular Expressions,
use egrep. More on regular expressions later in this document.

======================================
Search for files with the find command
======================================

The find command is used as a search tool for searching the
directory structure for a file or directory with the specified
attributes.

find search_path -name filename   

Examples:
find . -name aaa.txt    Finds all the files named aaa.txt in the
                        current directory or any subdirectory tree. 
find / -name testfile   Find any file named 'testfile' starting 
                        at / and descending thru the entire
                        directory tree. 
find /usr/local/bin -name "*goober*"       
                        Find all files whose names contain the
                        string 'goober' which exist within the
                        '/usr/local/bin' directory tree. 

Refer to the find command man page for more info.

For example:

find . -type f -exec chmod 755 {} \;   

Explanation:
Starting in the current directory and all sub-directories,  
look for any files of type "file" and execute the chmod 
command, changing the permissions of the found file to 755. 
The string "{}" gets replaced by the result of the find command.
The command specified by the -exec option must always end 
with " \;". Note that there is a mandatory space between 
the end of the command and the "\;". 

==================================================================
Reading and writing archive files with tar 
==================================================================

Tar (Tape ARchiver) is a command that is used to package one 
or more files into a single archive file. This utility was
originally intended for writing data files to magnetic tape, 
and then extended to write data to disk files.
The tar command is similar to the zip command on Windows 
systems, except that the tar command does not perform any 
file compression. We will invoke a separate utility to compress
files.

Archive files created with the tar command have file names that 
end with ".tar". In order to produce compressed tar files, we 
will use a command line option to instruct tar to use a specific
type of file compression. This compression will also cause the
filename of the resulting compressed tar file to end in ".gz" 
or ".bz2".

If the tar filename ends in .tar.gz the tar is compressed 
with gzip.
If the tar filename ends in .tar.bz2 the tar file is compressed 
with bzip2.

If the tar file came from an older UNIX system and has the 
extension .tar.Z, it most likely was compressed using the 
UNIX "compress" command. It's also (probably) a very old file!

Unlike most linux commands, tar does not require a "-" before 
the single letter command line option(s).

Command line Options:
v = verbose
c = create
x = extract
z = compress/uncompress with gunzip
j = compress/uncompress with bunzip2 (better but slower than gzip)
f = name of the tar file
t = show contents of the file without extracting

Examples:

tar xvjf filename.tar.bz2
Extract all the files from a bzip2 compressed tar archive 
called filename.tar.bz2 and be verbose about it.

tar cvzf mybackup.tar.gz /root/bin/*
Create a gzip compressed archive file named mybackup.tar.gz 
that contains the contents of the /root/bin/ directory and 
be verbose about it.

tar tvf mybackup.tar
Display a list of all the files contained in the mybackup.tar 
file (which is not in a compressed format), and be verbose 
about it. This command is just for looking at the contents of 
the tar file. No files will be extracted.

==================================================================
Help: Using the man and apropos commands
==================================================================

Most commands have a MANual page with instructions on how to 
use the command. Man pages are accessed by using the man 
command with the name of the command for which we seek help.
Curiously enough, the following command works:   man man

The standard Linux man pages are divided into sections as follows: 

MANUAL SECTIONS: 
1  User Commands
2  System Calls
3  C Library Functions
4  Devices and Special Files
5  File Formats and Conventions
6  Games et. Al.
7  Miscellanea
8  System Administration tools and Deamons (The default 
   if no section is specified)

man <command>     Display the man page for <command>
man man           Display the man page for the man command
man 1 ls          Display the Section 1 (User Commands) man 
                  page for the ls command
man 5 test.conf   Display the section 5 information (file 
                  formats) for the configuration file of 
                  the "test" command                  
apropos build     Shows a list of all the man pages
                  whose descriptions contain the word "build"
man apropos       Display the man page for the apropos command

==================================================================
The VI Editor
==================================================================

The vi editor is the most commonly available text editor on 
Linux and UNIX systems. Some people love it, and some people 
hate it. The learning curve can be somewhat steep, but as long 
as you learn enough vi to perform basic necessary tasks, you'll 
be ok.
These basic tasks are: 
   open (or create) a file
   edit the file (add/remove/change text in the file)
   save the file, and exit the editor
   exit the editor without saving the file


To open a file, use the vi command followed by the name of the 
file, asin this example:  vi textfile1
If the filename specified does not exist, it will be created 
the first time you save the file. 

Once you open a file in VI, you are in command mode. If you 
enter a different mode, and wish to return to command mode, 
press the <esc> key.

Commands in VI are single character and case sensitive.

To edit the file, you must enter Insert or Append mode:
i     Insert before current cursor position
I     Insert at beginning of current line
a     Insert (append) after current cursor position
A     Append to end of line

Any of these commands will put you into "edit mode". Once in 
edit mode, type as you normally would. Pressing the <ESC> key 
will get you out of edit mode and back into command mode.

Deletion of text:
x     Delete a single character at the cursor position
dd    Delete the current line (where the cursor is) and put 
      it in a buffer
ndd   Delete n lines and put them in a buffer
J     Attaches the next line to the end of the current line (removes 
      the carriage return).

Replace and Undo:
r     Replace 1 character at the current cursor position
nr    Replace n characters starting at the current cursor 
      position
u     Undo last command

Cut and Paste:
yy    Yank the current line into a buffer (refered to 
      as "yank-yank")
nyy   Yank n lines into a buffer
p     Put the contents of the buffer after the current line
P     Put the contents of the buffer before the current line

Moving the cursor around:
Most of the time the arrow keys will work the way you expect 
them to work. There are also standard vi commands from way 
back before we had proper keyboard mappings (i.e. the "dark 
ages").
^d       Page down (^ represents the <Ctrl> key)
^u       Page up
:n       Position cursor at line n  (The ":" is a prompt. 
         Press <shift> + <:> to get there)
:$       Position cursor at end of file
^g       Display current line number
h,j,k,l  Left,Down,Up, and Right respectively. 
               
String substitution:

At the command prompt (press <shift> + <:> to get there), 
you can use regular expressions (regex) to search the file 
in a similar fashion to what can be done with the sed command 
at the BASH command line.

:n1,n2:s/string1/string2/g       Substitute string2 for string1 
                                 on lines n1 to n2. If g is 
                                 included (global), all instances 
                                 of string1 on every line
                                 are substituted. If g is not
                                 included, only the first 
                                 matching instance is substituted.

(Very) Basic Regular Expressions (RegEx):
    ^ matches start of line
    . matches any single character
    * matches any character
    $ matches end of line

Special characters (such as the forward slash or the double 
quote) can be "escaped" with \ 
Escaping a character tells the shell to treat the special
characters literally, as opposed to interpreting them to change
the meaning of the expression. 

For example, to match the string "/usr/local/test" 
use "\/usr\/local\/test" 

Examples:

:1,$:s/dog/cat/g      Substitute 'cat' for 'dog', every instance
                      for the entire file - lines 1 to $ 
                      (end of file)

:23,25:/Bob/Alice/    Substitute 'Bob' for 'Alice' on lines
                      23 through 25. Only the first instance 
                      on each line is substituted.

Save your file and quit:

Press <shift>+<:> and a prompt will appear in the lower 
left-hand corner of the screen. These commands cannot be 
entered when you are in an edit mode. Press <ESC> to exit 
from an editing mode, and return to command mode.

:w                  Write the current file.
:w new.file         Write the file to the name 'new.file'.
:w! existing.file   Overwrite an existing file with the file 
                    currently being edited. 
:wq                 Write the file and quit vi.
:q                  Quit vi.
:q!                 Quit without saving any changes.


TO BE CONTINUED....