Sunday, 21 October 2018
Sunday, 20 May 2018
Useful Unix Commands for Developers -3
man bash
man bash | wc -l
info bash | wc -l
http://goo.gl/S2Jj2n
execute the script
Make scripts executable with chmod u+x
chmod +x file.sh
The bash time command
Bash has a builtin time command to report how much time a process consumed
time find / -name File_name
Variables in Bash
for a shell script to get a copy of a shell variable, it needs to be "exported":
export mynewvar or declare -x mynewvar
you can export and assign in the same statement:
export var2="var2 value".
export -f myFunc will export the function myFunc.
a=1
(a=2)
echo $a
prints 1
a=1
{a=2}
echo $a
prints 2
see built in commands
$enable
see bash keywords
$compgen -k
Bash startup
.bash_profile is read when bash is invoked as alogin shell.
.bashrc is executed when a new shell is started.
if you extend an exported variable, like PATH, in .bashrc, it will grow with each nested shell invocation.
PATH=$PATH:/usr/local/bin
This would keep adding /usr/local/bin to the end of PATH within nested shells.
Aliases and functions should normally be defined in .bashrc.
Sourcing Scripts:
source example.sh, or . example.sh
it is "dot space" as a short way to source a script.
The shell executed the script in the shell's own process instead of in a new process.
Sourcig is common way to import variable assignments or functions
The sourced script is executed within the calling shell's process.
Wroking with Aliases
The alias command allows for short command alternatives: alias ll="ls -l"
some people use an alias to give an alternative, more familiar name or to fix a common typo to a Linux command.
$source ./script_file.sh
Note: what this script contains we can use variable in further.
Working with Aliases
alias copy=cp
alias rn=mv
alias mroe=more
mroe myfile
ls -l | mroe
List defined aliases by simply typing alias
Unset an alias with the unalias command
Using the Echo Command
Built into bash and doesn't start a new process
-n ->don't print a trailing newline
-e ->enable backslash escaped characters like \n and \t
-E ->disable backslash escaped characters in case they were enabled by default
Ex:e
echo Hello World
echo -n Good to see you "\n\n"
echo Thanks
echo -e Hi "\t\t\t" There "\n\n"
echo -E Bye "\t\t\t" For now "\n\n"
ls * => would list contents of directories
echo * =>would show file and directory names
Use file redirection techniques to send the output to other files, such as stderr:
echo 'Warning will Robinson!' >&2
Local Variables and Typeset
Variables can be created in function that will not be available outside of it.
The typeset command makes variables local, can provide a type, or can provide formatting.
typeset -i x
#x must be an integer
Arithmetic is faster for variables defined to be integers.
Let allows for convenient arithmetic:
let x++; let y=x**2; let x=x*3; let x*=5, ....
The Declare Command
declare -l uppercase values in the variable are converted to lowercase.
declare -u lowercase values in the variable are converted to uppdercase.
declare -r variable is made read-only.
declare -a MyArray will make MyArray an indexed array.
declare -A MyArray2 will make MyArray2 an associative array.
function f1 {
typeset x
x=7
y=8
}
x=1
y=2
echo x is $x
echo y is $y
f1
echo x is $x
echo y is $y
output:
x is 1
y is 2
x is 1
y is 8
-------------------
declare -l lstring="ABCdef"
declare -u ustring="ABCdef"
declare -r readonly="A Value"
declare -a Myarray
declare -A Myarray2
echo lstring = $lstring
echo ustring = $ustring
echo readonly = $readonly
readonly="New Value"
Myarray[2]="Second Value"
echo 'Myarray[2]= ' ${Myarray[2]}
Myarray2["hotdog"]="baseball"
echo 'Myarray2[hotdog]= ' ${Myarray2["hotdog"]}
Output:
lstring = abcdef
ustring = ABCDEF
readonly = A Value
declare.sh: line 11: readonly: readonly variable
Myarray[2]= Second Value
Myarray2[hotdog]= baseball
The Read Command
Read a line into a variable or multiple variables
read a b---reads first word into a and the rest into b
convenient for a while loop
While Loops
While
command list1
do
command list
done
#loops while command list 1 succeeds
Ex:
while ((x<10))
do
echo loop $x;
date >date.$x
((x++))
done
-------------
while
read a b
do
echo a is $a b is $b
done <data_file
-------------
ls -l | while
read a b c d
do
echo owner is $c
done
--------------
For Loops
for <var> in <list>
do
command list
done
for i dog cat elephant
do
echo i is $i
done
----------------
seq 1 5
#prints 12345
for num in `seq 1 5`
#loops over 1 2 3 4 5
generate sequence with {A..Z}, {1..10}
for d in $(<data_file)
#loops over space/newline
#separated data in data_files
-----
for j in *.c
#making a list with file globbing
----
for f in $(find . -name *.c)
#using a command to generate a list
-----------
export a=first
export b=second
export c=third
echo a is '['$a']' b is '['$b']' c is '['$c']'
read a b <data_file
echo a is '['$a']' b is '['$b']' c is '['$c']'
data_file:
dog cat rooster
elephant hen rabbit snake
carrot lettuce pineapple banana pizza
output:
a is [first] b is [second] c is [third]
a is [dog] b is [cat rooster] c is [third]
--------
ls -l /etc | while
read a b c d
do
echo owner is $c
done
----------------
nl for.sh => nl number line
1 #!/bin/bash
2 for i in dog cat hotdog
3 do
4 echo i is $i
5 done
6 for i in `seq 3 5`
7 do
8 echo i in seq is $i
9 done
10 for i in {N..P}
11 do
12 echo i in letter list is $i
13 done
14 for d in $(<data_file)
15 do
16 echo d in data_file is $d
17 done
18 for f in $(find /etc 2>/dev/null | grep grub)
19 do
20 echo grub named things are $f
21 done
Functions:
Give a name to a sequence of statements that will execute within the shell, not in a new process
function NAME {
function body........
}
commonly used to organize code in a shell program
function printhello{
echo Hello
}
printhello
#shell memorizes the function like it's a new command
Return Command:
fucntions return when there are no more statements or when a return statement is executed.
function myFunc{
echo starting
return
echo this will not be executed
}
The return command:
functions produce results by writing output like commands do.
hvar=$(printhello)
The Exit command:
exit <value> sets the exit status, represented by $? to <value>
exit terminates the shell process
exit in a function terminates the whole shell program, not just the function.
Ex:
function myfunc {
echo starting myfunc
return
echo this will not be executed
}
myfunc
n=$(myfunc)
echo n is $n
Output:
starting myfunc
n is starting myfunc
---------------------
Redirection and pipes:
Processes normally have three files open:
0 =>stdin, 1 =>stdout, 2 =>stderr
Command > stdout-here 2> stderr-here < stdin-from-here
Command &> file
#file gets stdout and stderr from command, file is created or overwritten
Redirection and Pipes:
command | command2
#command2's stdin comes from command's stdout
command 2>&1 | command2
#gets stdout and stderr from command
command |& command2
#alternative way for command2 to get command's stdout and stderr as its stdin
command >> file
#appends stdout of command to end of file
command &>> file
#appends stdout and stderr of command to end of file
Here Documents: <<
Here documents are a way to embed input for standard input inside of a script.
They avoid having to create a new file just to hold some input values.
sort <<END
cherry
banana
apple
orange
END
Open and Close file descriptiors:
exec N< myfile
#opens file descriptor N for reading from file myfile
exec N> myfile
#opens file descriptor N for writing to myfile
exec N<> myfile
#opens file descriptor N for reading & writing with myfile
exec N>&- or exec N<&-
#closes file descriptor N
Use lsof to see what file descriptors for a process are open
exec 7>/tmp/myfile7
lsof -p $$
# $$ is shell's PID
echo Just '>' ---------------------------------------
find /etc -name grub >grub.out
echo Doing '2>' ---------------------------------------
find /etc -name grub 2>errs.out
echo Doing '&>' ---------------------------------------
find /etc -name grub &>both.out
> only standard output
2> only standard error
&> both standard output and errors
find /etc -name grub |& grep grub
|& => first command success ouput without errors passes to input to right side of pipe command
echo hi >myfile
replace content myfile with hi
echo cheese >>myfile
append data at bottom of myfile.
Sorting:
sort <<END
<content>
END
Ex:
sort <<END
cherry
banana
apple
orange
END
while
read a b c
do
echo a: $a b:$b c:$c
done <<EOF
one two three four
five six seven eight nine ten
eleven twelve
EOF
The Case Statements:
case expression in
pattern 1)
command list;;
pattern 2)
command list;;
...
esac
ex:
case $ans in
yes|YES|y|Y|y.x ) echo "will do!";;
n*|N*) echo "will not do!";;
*) echo "Oops!";;
esac
The If-Then-Else Statement
if
command list # last result is used
then
command list
[else
command list]
fi
EX:
if
grep -q important myfile
then
echo myfile has important stuff
else
echo myfile does not have important stuff
fi
Tests in Bash
The builtin test is used to check various conditions and set the return code with the result.
Loops and conditionals often use the result of test
An alternative to tet is [[]] or (())
ex: Test Example
if
test -f afile
if [[ -f bfile ]] #instead of 'test' [[]]
if
test $x -gt 5
Test Operator
Numeric Comparision:
[[ ex1 -eq ex2 ]] [[ ex1 -nq ex2 ]]
[[ ex1 -lt ex2 ]] [[ ex1 -gt ex2 ]]
[[ ex1 -le ex2 ]] [[ ex1 -ge ex2 ]]
or
(( ex1 == ex2 )) (( ex1 != ex2 ))
(( ex1 < ex2 )) (( ex1 > ex2 ))
(( ex1 <= ex2 )) (( ex1 >= ex2 ))
(( ex1 && ex2 )) (( ex1 || ex2 ))
(( expr?expr:expr))
test -d X => success if X is a directory
test -f X => success if X is a regular non-empty file
test -s X => success if X exists non-empty
test -x X => success if you have x permission on X
test -w X => success if you have w permission on X
test -r X => success if you have r permission on X
Ex:
x=01
y=1
echo comparing $x and $y
if
[ $x == $y ]
then
echo ==
else
echo not ==
fi
if
[ $x -eq $y ]
then
echo eq
else
echo not eq
fi
if
((x==y))
then
echo '(())' ==
else
echo not '(())' ==
fi
Ex: 2
if
test -x /bin/ls
then
if
[ ! -w /etc/hosts ]
then
if
echo about to look for foobar
grep -q foobar /etc/passwd
then
echo foobar found in /etc/passwd
else
echo foobar not found
fi
fi
else
echo Oh no, /bin/ls not executable
fi
Filters:
In Linux, a program is a filter if it reads from stdin and writes to stdout.
Filters can be used in pipes.
Filters provide the powerful means of combining the input and output of a sequence of commands to get the kind of report that you want.
The Head and Tail Commands
head prints the first n lines of a file or stdin.
tail prints the last n lines of a file or stdin
ls -l |head -5 #first 5 lines of ls -l
ls -l |tail -7 #last 7 lines of ls -l
ls -l |head -10 |tail -5 #lines 6-10
The Head and Tail Commands
wc(word count) prints line, word, and char counts
wc -l prints the number of lines
ls |wc -l prints number of entries in directory.
$./makeoutput.sh >output & => end & uses to run process background
$tail -n2 -f output
#!/bin/bash
for i in {1..100}
do
read a b c d e <<END
$(date)
END
echo $d
sleep 1
done
The Command sed:
is a stream editor, which means it is not interactive
works great as a filter
is ideal for batch editing tasks
Usually applies its editing to all lines in the input
with the -i option, change a file instead of echoing the modified file to stdout
Using sed Substitute:
sed 's/old/new/' myfile
Substitute the first occurrence of old on each line for new in the file myfile and display the result on stdout
old is a pattern and can be regular expression
The / is the usual character to separate the old from the new.
the file myfile will not be changed; the new version is echoed to stdout
No options are required for simple substitutions.
Ex:
sed 's/@home/@domicile/; s/truck/lorrie/'
sed -e 's/[xX]/Y/' -e 's/b.*n/blue/'
sed -f sedscript -n sed4
date | sed 's/J/j/'
sed '1,5p'
sed '/alpha/s/beta/gamma/'
sed '/apple/,/orange/d'
sed '/important/!s/print/throw_away/'
The awk Language:
A pattern matching language
An interpreted programming language that orks as a filter
Good for report writing
Handy for short algorithmic kinds of processing
processes a line at a time like sed
breaks each line into fields, $1, $2, etc.,
Fields are delimited by the values in the variable FS, normally white space.
$0 is the entire line(record)
EX:
$ps -el | \
awk '/pts/||$8~/35/{printf("%5d %5d %s\n", $4, $5, $14)}'
Ex:
sed2:
s/a/A/
s/B/BBB/
$ cat sometext:
now we have
some words and
fruit like apple cherry orange peach
and BIG things like
cruise ship, skyscraper, Bigfoot
$sed -f sed2 sometext
now we hAve
some words And
fruit like Apple cherry orange peach
And BBBIG things like
cruise ship, skyscrAper, BBBigfoot
Script Parameters and {}:
Parameters to a shell program: $1, $2
called "positional parameters"
to reference multidigit use {},e.g., ${10}
$0 is the path to the program itself:
For ex, echo Usage: $0 arg1 .....
Shift moves $2 into $1,$3 into $2, etc.
It is sometimes handy or required to use{} with named variable,. e.g., echo ${abc}DEF.
x=abc
abc=def
echo ${!x} prints def. indirection!
Unset or Null Variables:
${variable <OPR> value}
x=${var: -Hotdog}
:- if var unset/null , return value; otherwise, return value of var
:= if var unset/null var is assigned value & returned
:? Display an error and exit script if var unset/null
:+ if var unset/null return nothing; otherwise, return value
String Operations:
${var:offset}-value of var string at offset
${var:offset:len}-value of vr starting at offset upto length len
${#var}-length of var
${var#pre}-remove matching prefix
${var%post}-remove suffix-
Prefix and postfix -handy for processing filenames/paths
ex:#!/bin/bash
echo arg1 is $1 arg 11 is ${11}
shift
echo now arg1 is $1 arg 11 is ${11}
echo program is $0
$bash pos.sh {A..Z}
arg1 is A arg 11 is K
now arg1 is B arg 11 is L
program is pos.sh
ex:
x=abc
abc="Start Of Alphabet"
echo x is $x
echo abc is $abc
echo '${!x}' is ${!x}
output
x is abc
abc is Start Of Alphabet
${!x} is Start Of Alphabet
Ex:
#!/bin/bash
unset x
a=${x:-Hotdog}
echo a is $a
echo x is $x
a=${x:=Hotdog}
echo a is $a
echo x is $x
unset x
${x:?}
echo Will not get here
output:
a is Hotdog
x is
a is Hotdog
x is Hotdog
./unsetnull.sh: line 13: x: parameter null or not set
Ex:
#!/bin/bash
s="a string with words"
sub=${s:4}
echo sub is $sub
sub=${s:4:3}
echo sub is $sub
echo length of s is ${#s}
output:
sub is ring with words
sub is rin
length of s is 19
Ex:
p="/usr/local/bin/hotdog.sh"
echo whole path is $p
echo Remove prefix ${p#/*local/}
echo Remove suffix ${p%.sh}
cmd=${p#*/bin/}
cmd2=${cmd%.sh}
echo the command without .sh is $cmd2
output:
whole path is /usr/local/bin/hotdog.sh
Remove prefix bin/hotdog.sh
Remove suffix /usr/local/bin/hotdog
the command without .sh is hotdog
Advanced Bash:
Using Coprocesses:
a coprocess is a background process where your shell gets file descriptors for the process's stdin and stdout. Implemented with pipe
We need a script that is a filter.
#!/bin/bash
while
read line
do
echo $line | tr "ABC" "abc"
done
Using Coprocesses:
coproc ./mycoproc.sh
echo BANANA >&"${coproc[1]}"
cat <&"${COPROC[0]}"
or
coproc my { ./mycoproc.sh; }
echo BANANA >&"${my[1]}"
cat <&"${my[0]}"
Debugging script:
tee
ex: cmd | tee log.file | ...
man bash | wc -l
info bash | wc -l
http://goo.gl/S2Jj2n
execute the script
Make scripts executable with chmod u+x
chmod +x file.sh
The bash time command
Bash has a builtin time command to report how much time a process consumed
time find / -name File_name
Variables in Bash
for a shell script to get a copy of a shell variable, it needs to be "exported":
export mynewvar or declare -x mynewvar
you can export and assign in the same statement:
export var2="var2 value".
export -f myFunc will export the function myFunc.
a=1
(a=2)
echo $a
prints 1
a=1
{a=2}
echo $a
prints 2
see built in commands
$enable
see bash keywords
$compgen -k
Bash startup
.bash_profile is read when bash is invoked as alogin shell.
.bashrc is executed when a new shell is started.
if you extend an exported variable, like PATH, in .bashrc, it will grow with each nested shell invocation.
PATH=$PATH:/usr/local/bin
This would keep adding /usr/local/bin to the end of PATH within nested shells.
Aliases and functions should normally be defined in .bashrc.
Sourcing Scripts:
source example.sh, or . example.sh
it is "dot space" as a short way to source a script.
The shell executed the script in the shell's own process instead of in a new process.
Sourcig is common way to import variable assignments or functions
The sourced script is executed within the calling shell's process.
Wroking with Aliases
The alias command allows for short command alternatives: alias ll="ls -l"
some people use an alias to give an alternative, more familiar name or to fix a common typo to a Linux command.
$source ./script_file.sh
Note: what this script contains we can use variable in further.
Working with Aliases
alias copy=cp
alias rn=mv
alias mroe=more
mroe myfile
ls -l | mroe
List defined aliases by simply typing alias
Unset an alias with the unalias command
Using the Echo Command
Built into bash and doesn't start a new process
-n ->don't print a trailing newline
-e ->enable backslash escaped characters like \n and \t
-E ->disable backslash escaped characters in case they were enabled by default
Ex:e
echo Hello World
echo -n Good to see you "\n\n"
echo Thanks
echo -e Hi "\t\t\t" There "\n\n"
echo -E Bye "\t\t\t" For now "\n\n"
ls * => would list contents of directories
echo * =>would show file and directory names
Use file redirection techniques to send the output to other files, such as stderr:
echo 'Warning will Robinson!' >&2
Local Variables and Typeset
Variables can be created in function that will not be available outside of it.
The typeset command makes variables local, can provide a type, or can provide formatting.
typeset -i x
#x must be an integer
Arithmetic is faster for variables defined to be integers.
Let allows for convenient arithmetic:
let x++; let y=x**2; let x=x*3; let x*=5, ....
The Declare Command
declare -l uppercase values in the variable are converted to lowercase.
declare -u lowercase values in the variable are converted to uppdercase.
declare -r variable is made read-only.
declare -a MyArray will make MyArray an indexed array.
declare -A MyArray2 will make MyArray2 an associative array.
function f1 {
typeset x
x=7
y=8
}
x=1
y=2
echo x is $x
echo y is $y
f1
echo x is $x
echo y is $y
output:
x is 1
y is 2
x is 1
y is 8
-------------------
declare -l lstring="ABCdef"
declare -u ustring="ABCdef"
declare -r readonly="A Value"
declare -a Myarray
declare -A Myarray2
echo lstring = $lstring
echo ustring = $ustring
echo readonly = $readonly
readonly="New Value"
Myarray[2]="Second Value"
echo 'Myarray[2]= ' ${Myarray[2]}
Myarray2["hotdog"]="baseball"
echo 'Myarray2[hotdog]= ' ${Myarray2["hotdog"]}
Output:
lstring = abcdef
ustring = ABCDEF
readonly = A Value
declare.sh: line 11: readonly: readonly variable
Myarray[2]= Second Value
Myarray2[hotdog]= baseball
The Read Command
Read a line into a variable or multiple variables
read a b---reads first word into a and the rest into b
convenient for a while loop
While Loops
While
command list1
do
command list
done
#loops while command list 1 succeeds
Ex:
while ((x<10))
do
echo loop $x;
date >date.$x
((x++))
done
-------------
while
read a b
do
echo a is $a b is $b
done <data_file
-------------
ls -l | while
read a b c d
do
echo owner is $c
done
--------------
For Loops
for <var> in <list>
do
command list
done
for i dog cat elephant
do
echo i is $i
done
----------------
seq 1 5
#prints 12345
for num in `seq 1 5`
#loops over 1 2 3 4 5
generate sequence with {A..Z}, {1..10}
for d in $(<data_file)
#loops over space/newline
#separated data in data_files
-----
for j in *.c
#making a list with file globbing
----
for f in $(find . -name *.c)
#using a command to generate a list
-----------
export a=first
export b=second
export c=third
echo a is '['$a']' b is '['$b']' c is '['$c']'
read a b <data_file
echo a is '['$a']' b is '['$b']' c is '['$c']'
data_file:
dog cat rooster
elephant hen rabbit snake
carrot lettuce pineapple banana pizza
output:
a is [first] b is [second] c is [third]
a is [dog] b is [cat rooster] c is [third]
--------
ls -l /etc | while
read a b c d
do
echo owner is $c
done
----------------
nl for.sh => nl number line
1 #!/bin/bash
2 for i in dog cat hotdog
3 do
4 echo i is $i
5 done
6 for i in `seq 3 5`
7 do
8 echo i in seq is $i
9 done
10 for i in {N..P}
11 do
12 echo i in letter list is $i
13 done
14 for d in $(<data_file)
15 do
16 echo d in data_file is $d
17 done
18 for f in $(find /etc 2>/dev/null | grep grub)
19 do
20 echo grub named things are $f
21 done
Functions:
Give a name to a sequence of statements that will execute within the shell, not in a new process
function NAME {
function body........
}
commonly used to organize code in a shell program
function printhello{
echo Hello
}
printhello
#shell memorizes the function like it's a new command
Return Command:
fucntions return when there are no more statements or when a return statement is executed.
function myFunc{
echo starting
return
echo this will not be executed
}
The return command:
functions produce results by writing output like commands do.
hvar=$(printhello)
The Exit command:
exit <value> sets the exit status, represented by $? to <value>
exit terminates the shell process
exit in a function terminates the whole shell program, not just the function.
Ex:
function myfunc {
echo starting myfunc
return
echo this will not be executed
}
myfunc
n=$(myfunc)
echo n is $n
Output:
starting myfunc
n is starting myfunc
---------------------
Redirection and pipes:
Processes normally have three files open:
0 =>stdin, 1 =>stdout, 2 =>stderr
Command > stdout-here 2> stderr-here < stdin-from-here
Command &> file
#file gets stdout and stderr from command, file is created or overwritten
Redirection and Pipes:
command | command2
#command2's stdin comes from command's stdout
command 2>&1 | command2
#gets stdout and stderr from command
command |& command2
#alternative way for command2 to get command's stdout and stderr as its stdin
command >> file
#appends stdout of command to end of file
command &>> file
#appends stdout and stderr of command to end of file
Here Documents: <<
Here documents are a way to embed input for standard input inside of a script.
They avoid having to create a new file just to hold some input values.
sort <<END
cherry
banana
apple
orange
END
Open and Close file descriptiors:
exec N< myfile
#opens file descriptor N for reading from file myfile
exec N> myfile
#opens file descriptor N for writing to myfile
exec N<> myfile
#opens file descriptor N for reading & writing with myfile
exec N>&- or exec N<&-
#closes file descriptor N
Use lsof to see what file descriptors for a process are open
exec 7>/tmp/myfile7
lsof -p $$
# $$ is shell's PID
echo Just '>' ---------------------------------------
find /etc -name grub >grub.out
echo Doing '2>' ---------------------------------------
find /etc -name grub 2>errs.out
echo Doing '&>' ---------------------------------------
find /etc -name grub &>both.out
> only standard output
2> only standard error
&> both standard output and errors
find /etc -name grub |& grep grub
|& => first command success ouput without errors passes to input to right side of pipe command
echo hi >myfile
replace content myfile with hi
echo cheese >>myfile
append data at bottom of myfile.
Sorting:
sort <<END
<content>
END
Ex:
sort <<END
cherry
banana
apple
orange
END
while
read a b c
do
echo a: $a b:$b c:$c
done <<EOF
one two three four
five six seven eight nine ten
eleven twelve
EOF
The Case Statements:
case expression in
pattern 1)
command list;;
pattern 2)
command list;;
...
esac
ex:
case $ans in
yes|YES|y|Y|y.x ) echo "will do!";;
n*|N*) echo "will not do!";;
*) echo "Oops!";;
esac
The If-Then-Else Statement
if
command list # last result is used
then
command list
[else
command list]
fi
EX:
if
grep -q important myfile
then
echo myfile has important stuff
else
echo myfile does not have important stuff
fi
Tests in Bash
The builtin test is used to check various conditions and set the return code with the result.
Loops and conditionals often use the result of test
An alternative to tet is [[]] or (())
ex: Test Example
if
test -f afile
if [[ -f bfile ]] #instead of 'test' [[]]
if
test $x -gt 5
Test Operator
Numeric Comparision:
[[ ex1 -eq ex2 ]] [[ ex1 -nq ex2 ]]
[[ ex1 -lt ex2 ]] [[ ex1 -gt ex2 ]]
[[ ex1 -le ex2 ]] [[ ex1 -ge ex2 ]]
or
(( ex1 == ex2 )) (( ex1 != ex2 ))
(( ex1 < ex2 )) (( ex1 > ex2 ))
(( ex1 <= ex2 )) (( ex1 >= ex2 ))
(( ex1 && ex2 )) (( ex1 || ex2 ))
(( expr?expr:expr))
test -d X => success if X is a directory
test -f X => success if X is a regular non-empty file
test -s X => success if X exists non-empty
test -x X => success if you have x permission on X
test -w X => success if you have w permission on X
test -r X => success if you have r permission on X
Ex:
x=01
y=1
echo comparing $x and $y
if
[ $x == $y ]
then
echo ==
else
echo not ==
fi
if
[ $x -eq $y ]
then
echo eq
else
echo not eq
fi
if
((x==y))
then
echo '(())' ==
else
echo not '(())' ==
fi
Ex: 2
if
test -x /bin/ls
then
if
[ ! -w /etc/hosts ]
then
if
echo about to look for foobar
grep -q foobar /etc/passwd
then
echo foobar found in /etc/passwd
else
echo foobar not found
fi
fi
else
echo Oh no, /bin/ls not executable
fi
Filters:
In Linux, a program is a filter if it reads from stdin and writes to stdout.
Filters can be used in pipes.
Filters provide the powerful means of combining the input and output of a sequence of commands to get the kind of report that you want.
The Head and Tail Commands
head prints the first n lines of a file or stdin.
tail prints the last n lines of a file or stdin
ls -l |head -5 #first 5 lines of ls -l
ls -l |tail -7 #last 7 lines of ls -l
ls -l |head -10 |tail -5 #lines 6-10
The Head and Tail Commands
wc(word count) prints line, word, and char counts
wc -l prints the number of lines
ls |wc -l prints number of entries in directory.
$./makeoutput.sh >output & => end & uses to run process background
$tail -n2 -f output
#!/bin/bash
for i in {1..100}
do
read a b c d e <<END
$(date)
END
echo $d
sleep 1
done
The Command sed:
is a stream editor, which means it is not interactive
works great as a filter
is ideal for batch editing tasks
Usually applies its editing to all lines in the input
with the -i option, change a file instead of echoing the modified file to stdout
Using sed Substitute:
sed 's/old/new/' myfile
Substitute the first occurrence of old on each line for new in the file myfile and display the result on stdout
old is a pattern and can be regular expression
The / is the usual character to separate the old from the new.
the file myfile will not be changed; the new version is echoed to stdout
No options are required for simple substitutions.
Ex:
sed 's/@home/@domicile/; s/truck/lorrie/'
sed -e 's/[xX]/Y/' -e 's/b.*n/blue/'
sed -f sedscript -n sed4
date | sed 's/J/j/'
sed '1,5p'
sed '/alpha/s/beta/gamma/'
sed '/apple/,/orange/d'
sed '/important/!s/print/throw_away/'
The awk Language:
A pattern matching language
An interpreted programming language that orks as a filter
Good for report writing
Handy for short algorithmic kinds of processing
processes a line at a time like sed
breaks each line into fields, $1, $2, etc.,
Fields are delimited by the values in the variable FS, normally white space.
$0 is the entire line(record)
EX:
$ps -el | \
awk '/pts/||$8~/35/{printf("%5d %5d %s\n", $4, $5, $14)}'
Ex:
sed2:
s/a/A/
s/B/BBB/
$ cat sometext:
now we have
some words and
fruit like apple cherry orange peach
and BIG things like
cruise ship, skyscraper, Bigfoot
$sed -f sed2 sometext
now we hAve
some words And
fruit like Apple cherry orange peach
And BBBIG things like
cruise ship, skyscrAper, BBBigfoot
Script Parameters and {}:
Parameters to a shell program: $1, $2
called "positional parameters"
to reference multidigit use {},e.g., ${10}
$0 is the path to the program itself:
For ex, echo Usage: $0 arg1 .....
Shift moves $2 into $1,$3 into $2, etc.
It is sometimes handy or required to use{} with named variable,. e.g., echo ${abc}DEF.
x=abc
abc=def
echo ${!x} prints def. indirection!
Unset or Null Variables:
${variable <OPR> value}
x=${var: -Hotdog}
:- if var unset/null , return value; otherwise, return value of var
:= if var unset/null var is assigned value & returned
:? Display an error and exit script if var unset/null
:+ if var unset/null return nothing; otherwise, return value
String Operations:
${var:offset}-value of var string at offset
${var:offset:len}-value of vr starting at offset upto length len
${#var}-length of var
${var#pre}-remove matching prefix
${var%post}-remove suffix-
Prefix and postfix -handy for processing filenames/paths
ex:#!/bin/bash
echo arg1 is $1 arg 11 is ${11}
shift
echo now arg1 is $1 arg 11 is ${11}
echo program is $0
$bash pos.sh {A..Z}
arg1 is A arg 11 is K
now arg1 is B arg 11 is L
program is pos.sh
ex:
x=abc
abc="Start Of Alphabet"
echo x is $x
echo abc is $abc
echo '${!x}' is ${!x}
output
x is abc
abc is Start Of Alphabet
${!x} is Start Of Alphabet
Ex:
#!/bin/bash
unset x
a=${x:-Hotdog}
echo a is $a
echo x is $x
a=${x:=Hotdog}
echo a is $a
echo x is $x
unset x
${x:?}
echo Will not get here
output:
a is Hotdog
x is
a is Hotdog
x is Hotdog
./unsetnull.sh: line 13: x: parameter null or not set
Ex:
#!/bin/bash
s="a string with words"
sub=${s:4}
echo sub is $sub
sub=${s:4:3}
echo sub is $sub
echo length of s is ${#s}
output:
sub is ring with words
sub is rin
length of s is 19
Ex:
p="/usr/local/bin/hotdog.sh"
echo whole path is $p
echo Remove prefix ${p#/*local/}
echo Remove suffix ${p%.sh}
cmd=${p#*/bin/}
cmd2=${cmd%.sh}
echo the command without .sh is $cmd2
output:
whole path is /usr/local/bin/hotdog.sh
Remove prefix bin/hotdog.sh
Remove suffix /usr/local/bin/hotdog
the command without .sh is hotdog
Advanced Bash:
Using Coprocesses:
a coprocess is a background process where your shell gets file descriptors for the process's stdin and stdout. Implemented with pipe
We need a script that is a filter.
#!/bin/bash
while
read line
do
echo $line | tr "ABC" "abc"
done
Using Coprocesses:
coproc ./mycoproc.sh
echo BANANA >&"${coproc[1]}"
cat <&"${COPROC[0]}"
or
coproc my { ./mycoproc.sh; }
echo BANANA >&"${my[1]}"
cat <&"${my[0]}"
Debugging script:
tee
ex: cmd | tee log.file | ...
Sunday, 13 May 2018
Useful Unix Commands for Developers -2
rmdir
more book.txt page by page
head book.txt head
tail book.txt head
~ users home directory
~- previous directory
touch {apple,banana,cherry,durian} =>creates these files
touch file_{1..100} =>create 100 files
echo {1..10..2} => 2 represents intervel
echo {1..10..3}
echo {A..Z}
echo {A..z}
echo {w..d..2}
ls | more
input from commandline environment and output to it and error arises something goes wrong.
cp -v ../otherfolder 1>../success.txt 2>../error.txt
success will go to the success.txt and error goes to error.txt
here we are coping results to file instead of console.
----------
Redirecting output (bother success and error to log file
cp -v * ../otherfoler &> ../log.txt
ignoring output of a command "/dev/null"
ls > /dev/null
grep -i break-in auth.log | awk {'print $12'}
man awk
man sed =>used for string manipulation
ping example.com
ping -c 1 example.com | grep 'bytes from'
ping -c 1 example.com | grep 'bytes from' | cut -d = -f 4
f =>field 4
d => delimiter
#!/bin/bash
Hashbang or shebang, path to the bash executable
nana my.sh
#!/bin/bash
#this is a basic bash script
ls
ctrl o
ctrl s
bash my.sh
chmod +x my.sh
./my.sh
If you put your script file to /usr/bin then the file will run directly by just typing $my.sh instead of $./my.sh
sudo cp my.sh /usr/bin
echo no-quoates => If you write a statement with no quotes, Bash goes along and interprets things as it finds them.
echo single-quoates =>There's also single quotes or strong quotes, where nothing inside of the quotes gets interpreted. So everything comes out literally, even if I put a variable inside
echo double-quoates =>Bash display something literally inside of double quotes, you could escape it with backslash
Adding attributes to variables
declare -i d=420 # d is an integer
declare -i e=555 #e is read-only
declare -l f="LOLCats" # f is lolcats
declare -u g="LOLCats" #g is LOLCATS
Built-in variables
echo $HOME => returns user home directory[Linux: /home/phani]
echo $PWD =>returns current directory [Linux: /home/phani]
echo $MACHTYPE =>returns machine type [Linux: x86_4-pc-linux-gnu]
echo $HOSTNAME =>returns system name [Linux: orion]
echo $BASH_VERSION => returns version of Bash[Linux:4.2.25(1)-release]
echo $SECONDS=> returns the number of seconds the Bash session has run
echo $0 =>returns the name of the script
d=$(pwd)
echo $d
a=$(ping -c l example.com | grep 'bytes from' | cut -d = -f 4)
echo "the ping was $a"
Arithmetic operations
val=$(( expression ))
exponentiation - $a ** $b
multiplication - $a * $b
division - $a / $b
modulo - $a % $b
addition - $a + $b
subtraction $a - $b
increment => ((e++))), ((e--)) , ((e+=5)) , ((e/=3)) , ((e-=5))
echo $e
if we use
echo $e+=5 => returns 49 concatenation if e=4
echo $((e+=5)) => reruns 9 if e=4
Note: Bash math only works with integers, not floating point numbers.
If you need to get a floating point value you can use the BC program with the predefined math routines to return that.
g=$(echo 1/3 | bc -l )
echo $g => 0.333333333333
Comparison Operations
[[ expression ]]
1: FALSE
0: TRUE
It is only for string comparison :
less than => [[ $a < $b ]]
greater than [[ $a > $b ]]
less than or equal [[ $a <= $b ]]
greater than or equal [[ $a >= $b ]]
equal [[ $a == $b ]]
not equal [[ $a != $b ]]
Ex:
[[ "cat" == "cat" ]] echo $?
[[ "cat" = "cat" ]] echo $?
For Number comparison:
less than => [[ $a -lt $b ]]
greater than [[ $a -gt $b ]]
less than or equal [[ $a -le $b ]]
greater than or equal [[ $a -ge $b ]]
equal [[ $a -eq $b ]]
not equal [[ $a -ne $b ]]
Logic operations
Logical AND => [[ $a && $b ]]
Logical OR => [[ $a || $b ]]
Logic NOT => [[ !$a ]]
String null Value
is null? => [[ -z $a ]]
is not null? => [[ -n $a ]]
Ex: a="" , b="abc"
[[ -z $a && -n $b ]] =>echo $?
Working with Strings
a="hello"
b="Unix"
c=$a$b
echo $c => hellowunix
get the length of each of those strings, a and c
echo ${#a} => 5
echo ${#c} => 9
certain piece or a substring from an existing string
use a colon followed by an index number to start from. Keep in mind that the first character is at index 0
d=${c:3} , echo $d => lowunix
e=${c:3:4} => lowu
getting four characters at the end of the string
d=${c: -4} , echo $d => unix
e=${c: -4:3} => uni
replace text in the string with some other text.
In this case the string fruit, followed by one slash, the term I want to replace, in this case banana, another slash, and the string I want to replace
the first instance of the search term with. In this case, durian.
Ex:
fruit="apple banana banana cherry"
echo ${fruit/banana/durian} => apple durian banana cherry =>replace only one instance
echo ${fruit//banana/durian} => apple durian banana cherry =>replace all instance
echo ${fruit/#apple/durian} => durian banana banana cherry =>replace begining of string if it exist
echo ${fruit/%cherry/durian} => apple banana banana durian =>replace ending of string if it exist
Exploring some handy helpers: date and printf
date => Mon Jun 10 10:00:10 UTC 2018
date +"%d-%m-%Y" => 10-06-2018
date +"%H-%M-%S"
printf "Name:\t%s\nID:\t%04d\n" "Phani" "11"
Name: Phani
ID: 0011
today=$(date +"%d-%m-%Y")
time=$(date +"%H-%M-%S")
printf -v d "Current User:\t%s\nDate:\t\t%s @ %s\n" $USER $today $time
echo $d
Current user: phani
Date: 10-06-2018 @ 10:11:12
Note: -v used to ouput the value to a variable as "d"
Working with arrays
a=()
b=("java" "c language" "dot net")
echo ${b[2]} => dot net Note: arras are index based start with 0.
setting array value
b[5]="hadoop"
But if you want to add something to the end of an array, you can use the plus equals operator. Make sure you put the value in parentheses.
b+=("hive")
echo ${b[@]} => represents whole array
echo ${b[@]: -1} => hadoop
Associative arrays[Available only bash4 and above versions]
declare -A myarray
myarray[color]=blue
myarray["office building"]="HQ West" [Note: if we have space use quote]
echo ${myarray["office building"]} is ${myarray[color]}
=> HQ West is blue
Reading and writing text files
echo "xome text" > file.txt
$>file.txt =>remove everything in file
echo "some more text" >> file.txt =>append text in file
reading files
i=1
while read f; do
echo "Line $i: $f"
((i++))
done < file.txt
freespace = $(df -h | grep -E "\/$" | awk '{print $4}')
Conditional statement
if [ expression ]
if [[ expression ]] =>if you are using regular expression
if (( expression )) =>integer comparison
if expression; then => no brackets
if expression
then
echo "true"
elif expression2; then
echo "true"
else
echo "false"
fi
i=0
while [$i -le 10 ]; do
echo i:$i
((i++))
done
j=0
until [$j -ge 10 ]; do
echo j:$j
((j+=i))
done
For Loop:
for i in 1 2 3
do
echo $i
done
for i in {1..100..2} note : bash4 and above
do
echo $i
done
for (( i=1; i<=10; i++ ))
do
echo $i
done
arr = ("c" "java" "dotnet")
for i in ${arr[@]}
do
echo $i
done
Associate array
declare -A arr
arr["name"]="phani"
arr["id"]="1478"
for i in "${!arr[@]}" =>! represents key
do
echo "$i: ${arr[$i]}"
done
Command substitution
for i in $(ls)
do
echo "$i: ${arr[$i]}"
done
case Statement
a="java"
case $a in
dotnet) echo "feline";;
java|hadoop) echo "canine"
*)echo "No match!";;
esac
functions
function numberthings{
i=1
for f n $@; do
echo $i: $f
((i+=1))
done
}
numberthings $(ls)
numberthings java c dontnet hadoop
more book.txt page by page
head book.txt head
tail book.txt head
~ users home directory
~- previous directory
touch {apple,banana,cherry,durian} =>creates these files
touch file_{1..100} =>create 100 files
echo {1..10..2} => 2 represents intervel
echo {1..10..3}
echo {A..Z}
echo {A..z}
echo {w..d..2}
ls | more
input from commandline environment and output to it and error arises something goes wrong.
cp -v ../otherfolder 1>../success.txt 2>../error.txt
success will go to the success.txt and error goes to error.txt
here we are coping results to file instead of console.
----------
Redirecting output (bother success and error to log file
cp -v * ../otherfoler &> ../log.txt
ignoring output of a command "/dev/null"
ls > /dev/null
grep -i break-in auth.log | awk {'print $12'}
man awk
man sed =>used for string manipulation
ping example.com
ping -c 1 example.com | grep 'bytes from'
ping -c 1 example.com | grep 'bytes from' | cut -d = -f 4
f =>field 4
d => delimiter
#!/bin/bash
Hashbang or shebang, path to the bash executable
nana my.sh
#!/bin/bash
#this is a basic bash script
ls
ctrl o
ctrl s
bash my.sh
chmod +x my.sh
./my.sh
If you put your script file to /usr/bin then the file will run directly by just typing $my.sh instead of $./my.sh
sudo cp my.sh /usr/bin
echo no-quoates => If you write a statement with no quotes, Bash goes along and interprets things as it finds them.
echo single-quoates =>There's also single quotes or strong quotes, where nothing inside of the quotes gets interpreted. So everything comes out literally, even if I put a variable inside
echo double-quoates =>Bash display something literally inside of double quotes, you could escape it with backslash
Adding attributes to variables
declare -i d=420 # d is an integer
declare -i e=555 #e is read-only
declare -l f="LOLCats" # f is lolcats
declare -u g="LOLCats" #g is LOLCATS
Built-in variables
echo $HOME => returns user home directory[Linux: /home/phani]
echo $PWD =>returns current directory [Linux: /home/phani]
echo $MACHTYPE =>returns machine type [Linux: x86_4-pc-linux-gnu]
echo $HOSTNAME =>returns system name [Linux: orion]
echo $BASH_VERSION => returns version of Bash[Linux:4.2.25(1)-release]
echo $SECONDS=> returns the number of seconds the Bash session has run
echo $0 =>returns the name of the script
d=$(pwd)
echo $d
a=$(ping -c l example.com | grep 'bytes from' | cut -d = -f 4)
echo "the ping was $a"
Arithmetic operations
val=$(( expression ))
exponentiation - $a ** $b
multiplication - $a * $b
division - $a / $b
modulo - $a % $b
addition - $a + $b
subtraction $a - $b
increment => ((e++))), ((e--)) , ((e+=5)) , ((e/=3)) , ((e-=5))
echo $e
if we use
echo $e+=5 => returns 49 concatenation if e=4
echo $((e+=5)) => reruns 9 if e=4
Note: Bash math only works with integers, not floating point numbers.
If you need to get a floating point value you can use the BC program with the predefined math routines to return that.
g=$(echo 1/3 | bc -l )
echo $g => 0.333333333333
Comparison Operations
[[ expression ]]
1: FALSE
0: TRUE
It is only for string comparison :
less than => [[ $a < $b ]]
greater than [[ $a > $b ]]
less than or equal [[ $a <= $b ]]
greater than or equal [[ $a >= $b ]]
equal [[ $a == $b ]]
not equal [[ $a != $b ]]
Ex:
[[ "cat" == "cat" ]] echo $?
[[ "cat" = "cat" ]] echo $?
For Number comparison:
less than => [[ $a -lt $b ]]
greater than [[ $a -gt $b ]]
less than or equal [[ $a -le $b ]]
greater than or equal [[ $a -ge $b ]]
equal [[ $a -eq $b ]]
not equal [[ $a -ne $b ]]
Logic operations
Logical AND => [[ $a && $b ]]
Logical OR => [[ $a || $b ]]
Logic NOT => [[ !$a ]]
String null Value
is null? => [[ -z $a ]]
is not null? => [[ -n $a ]]
Ex: a="" , b="abc"
[[ -z $a && -n $b ]] =>echo $?
Working with Strings
a="hello"
b="Unix"
c=$a$b
echo $c => hellowunix
get the length of each of those strings, a and c
echo ${#a} => 5
echo ${#c} => 9
certain piece or a substring from an existing string
use a colon followed by an index number to start from. Keep in mind that the first character is at index 0
d=${c:3} , echo $d => lowunix
e=${c:3:4} => lowu
getting four characters at the end of the string
d=${c: -4} , echo $d => unix
e=${c: -4:3} => uni
replace text in the string with some other text.
In this case the string fruit, followed by one slash, the term I want to replace, in this case banana, another slash, and the string I want to replace
the first instance of the search term with. In this case, durian.
Ex:
fruit="apple banana banana cherry"
echo ${fruit/banana/durian} => apple durian banana cherry =>replace only one instance
echo ${fruit//banana/durian} => apple durian banana cherry =>replace all instance
echo ${fruit/#apple/durian} => durian banana banana cherry =>replace begining of string if it exist
echo ${fruit/%cherry/durian} => apple banana banana durian =>replace ending of string if it exist
Exploring some handy helpers: date and printf
date => Mon Jun 10 10:00:10 UTC 2018
date +"%d-%m-%Y" => 10-06-2018
date +"%H-%M-%S"
printf "Name:\t%s\nID:\t%04d\n" "Phani" "11"
Name: Phani
ID: 0011
today=$(date +"%d-%m-%Y")
time=$(date +"%H-%M-%S")
printf -v d "Current User:\t%s\nDate:\t\t%s @ %s\n" $USER $today $time
echo $d
Current user: phani
Date: 10-06-2018 @ 10:11:12
Note: -v used to ouput the value to a variable as "d"
Working with arrays
a=()
b=("java" "c language" "dot net")
echo ${b[2]} => dot net Note: arras are index based start with 0.
setting array value
b[5]="hadoop"
But if you want to add something to the end of an array, you can use the plus equals operator. Make sure you put the value in parentheses.
b+=("hive")
echo ${b[@]} => represents whole array
echo ${b[@]: -1} => hadoop
Associative arrays[Available only bash4 and above versions]
declare -A myarray
myarray[color]=blue
myarray["office building"]="HQ West" [Note: if we have space use quote]
echo ${myarray["office building"]} is ${myarray[color]}
=> HQ West is blue
Reading and writing text files
echo "xome text" > file.txt
$>file.txt =>remove everything in file
echo "some more text" >> file.txt =>append text in file
reading files
i=1
while read f; do
echo "Line $i: $f"
((i++))
done < file.txt
freespace = $(df -h | grep -E "\/$" | awk '{print $4}')
Conditional statement
if [ expression ]
if [[ expression ]] =>if you are using regular expression
if (( expression )) =>integer comparison
if expression; then => no brackets
if expression
then
echo "true"
elif expression2; then
echo "true"
else
echo "false"
fi
i=0
while [$i -le 10 ]; do
echo i:$i
((i++))
done
j=0
until [$j -ge 10 ]; do
echo j:$j
((j+=i))
done
For Loop:
for i in 1 2 3
do
echo $i
done
for i in {1..100..2} note : bash4 and above
do
echo $i
done
for (( i=1; i<=10; i++ ))
do
echo $i
done
arr = ("c" "java" "dotnet")
for i in ${arr[@]}
do
echo $i
done
Associate array
declare -A arr
arr["name"]="phani"
arr["id"]="1478"
for i in "${!arr[@]}" =>! represents key
do
echo "$i: ${arr[$i]}"
done
Command substitution
for i in $(ls)
do
echo "$i: ${arr[$i]}"
done
case Statement
a="java"
case $a in
dotnet) echo "feline";;
java|hadoop) echo "canine"
*)echo "No match!";;
esac
functions
function numberthings{
i=1
for f n $@; do
echo $i: $f
((i+=1))
done
}
numberthings $(ls)
numberthings java c dontnet hadoop
Sunday, 15 April 2018
Use Full Git Details
| Use Full Git Details | ||
| Installing Git | Linux: Apt-get install git-core | |
| Windows: http://msysgit.github.com | ||
| Git –version | ||
| Configuring Git | system level configuration | git config --global user.name "phani git" it config --global user.email "venkataphanikumar.r@gmail.com" git config --global –list git config --global core.editor vim git config --global help.autocorrect 1 git config --global color.ui auto git config --global color.ui auto cat ~/.gitconfig git config --global --unset core.editor |
| git config –sytem | ||
| stored in /etc/gitconfig or c:\programFiles\Git\etc\gitconfig | ||
| User-level configuration | ||
| git config –global | ||
| Stored in ~/.gitconfig or c:\Users\name\.gitconfig | ||
| Repository-level configuration | ||
| git config –global | ||
| stored in .git/config in each repo | ||
| Creating a local repository Adding files Comimting changes Viewing history Viewing a diff Working copy, staging, and repository Deleting files Cleaning the working copy Ignoring files with .gitignore |
Initiate link to git git init Initialized empty Git repository in /home/phani/code-git/.git/ echo "Hello, Git" > README.txt git status Untracked files: README.txt Add new file to git repo git add README.txt git status Changes to be committed : README.txt Changes to be committed git commit Added README.txt git log nano README.txt Modify the file in second time git status Changes not staged for commit ; README.txt Update file in git repo git add -u git status Changes to be committed: README.txt git log git commit -m "updated README.txt" git log Changes between 2 commits git diff a5b807..9a3dd7 (or) Git diff HEAD~1..HEAD [HEAD= latest version, HEAD~1 = latest -1 version] (or) Git diff HEAD~1.. Add new files to stating git add file1.txt file2.txt (or) Git add -A Git commit -m “adding new 2 files” [-m = comments] Remove file rm file2.txt git status Changes not staged for commit: git add -u git status git add file3.txt Mv file1.txt new_file_name.txt git add -A (-A=add all) git commit -m "Reorganized" [commit for stating] Revert our local changes to latest version in git git checkout README.txt HARD, SOFT RESET git reset –hard git reset --soft HEAD~1 Cleaning the working copy git clean -f Ignoring files with .gitignore |
creating a local repository Adding, updating , and deleting files Commiting changes Viewing history and diffs Working copy, staging, and repository Cleaning the working copy Ignoring files with .gitignore |
| Cloning a remote repository Listing remote repositories Feching changes from a remote Merging changes Pulling from a remote Pushing changes remotely Working with tags |
git clone https://github.com/jquery/jquery.git git log –onelineoption [comments for all commits will display] git log --oneline |wc -l git log --oneline –graph git shortlog git shortlog -sne git show HEAD [Last commit made] git log --oneline git show 2b5f5d5 git remote git remove -v Git protocols http(s) 80/443 – read-write pwd for auth firewall friendly Git 9418 –read-only anonymous only Ssh 22 – read-write SSH key for auth File n/a – read write local only git branch git branch -r git tag git remote add origin https://github.com/venkataphanikumarr/GitFundamentals.git Git fetch git fetch original git log original/master git merge origin/master git branch -r git fetch; git merge original/master (or) get pull git push Tags are 3types git tag v1.0 git tag -a v1.0_with_message git tag -s v1.0_signed git push –tags |
Cloning a remote repository Fetching and pulling from a remote repository Pushing changes Working with tags |
| Working with local branches Stashing changes Merging branches Rebasing commits Cherry-picking commits Working with remote branches |
git log --graph –oneline git log --graph --oneline --all –decorate git config --global alias.lga "log --graph --oneline --all –decorate" git lga cat ~/.gitconfig Git branch feature1 git checkout feature1 echo "Feature1" >> README.txt git commit -am "added feature1" git lga Note: branches follow commits as you add aditional commits to that branch Tags: stay on same commit (friendly name for perticular commit) Git checkout master git branch fix1 561e0ef git checkout fix1 echo "Fixing bug #1234" >> README.txt git commit -am "fix bug 1234" git branch -m fix1 bug1234 (rename the branch) git branch -D bug1234 (delete the branch) Bring back deleted branch (only available 30 days) git reflog git branch bug1234 8335b38 git checkout bug1234 Git stash Git stash pop GIT MERGE BRANCH git merge bug1234 git mergetoo |
Local and remote branches Stashing changes Merging, rebashing, and cherry-picking |
Sunday, 8 April 2018
Basic Unix Commands Developer Should Know
1. Java Installation and setting Java Environment variables.
sudo apt-get update
sudo apt-get install oracle-java8-installer
sudo apt install oracle-java8-set-default
sudo -H gedit /etc/environment
add below lines:
JAVA_HOME="/usr/lib/jvm/java-8-oracle"
JRE_HOME="/usr/lib/jvm/java-8-oracle/jre"
2. Maven Installation
cd /home/phani/Softwares/
wget http://www-eu.apache.org/dist/maven/maven-3/3.3.9/binaries/apache-maven-3.3.9-bin.tar.gz
sudo tar -xvzf apache-maven-3.3.9-bin.tar.gz
sudo mv apache-maven-3.3.9 maven =>rename
cd /etc/profile.d/
sudo gedit maven-env.sh
export MAVEN_HOME=/home/phani/Softwares/maven
export M2_HOME=/home/phani/Softwares/maven
export PATH=$PATH:$M2_HOME/bin
Commands:
$ cat filename
$ cat > file1.txt => creat file and append text and Ctrl+d
$ cat sample1.txt sample2.txt > sample3.txt => concatenate these two files and can save to another file named sample3.txt
$ cat -n file1.txt =>line number
$ cat file2.txt> file1.txt => To copy the content of one file to another file, you can use the greater than ‘>’ symbol with the cat command.
$ cat sample1.txt >> sample2.txt =>To append the contents of one file to another, you can use the double greater than ‘>>’ symbol with the cat command.
$gedit filename =>opens in text pad
cp /tmp/mozilla_phani0/eclipse-jee-helios-SR1-linux-gtk-x86_64.tar.gz .
tar xvf file.tar
x = eXtract, this indicated an extraction
v= verbose (optional) the files with relative locations will be displayed.
f = from/to file ... (what is next after the f is the archive file)
Remove files/folders
rm -rf /path/to/directory/*
-f - stands for "force" which is helpful when you don't want to be asked/prompted if you want to remove an archive, for example.
-r - stands for "recursive" which means that you want to go recursively down every folder and remove everything.
Duplicate copy folder
cp eclipse eclipse_bk
Creating Link (shortcut desktop)
ln -s source_file destination_path
Open Shell: xterm
echo $BASH
command,argument , options
ls -a list all files incuding hiden
ls -l list files in long format
ls -l-a combine both a and l options
ls -la combine both a and l options
ls -la /bin list all in /bin in long format
Order of options does not matter
The file name has to be at the end
Getting Help
Manual Pages => command: man
man ls
man cd
man man
Using the manual pages
use space to move down a page
move back a page with 'b'
search with '/'
exit with 'q'
drwxr
d -directory
History
ctrl+p previous
ctrl+n next
File Management
less FileName
reset =>used for reset the terminal
file fileName =>used to get the file information
file * => getting all files and folder information
mv =>move
rm -r images =>remove folder and its content recrusively
touch =>creating empty file
cat
best for small files -no paging
may mess up your terminal
use reset to fix terminal
less
pager with lots of features
see 'man less'
use space to move down a page
move back a page with b
search with '/'
exit with 'q'
Call the correct program directly
firefox index.html
opera index.html
you have to know what is installed on your system
File names
filenames can contain just about anything
except /
hidden files start with a dot
Case sensitivity
Extensions(.exe,.zip) are optional
file =>command show types
Absolute paths
start with /
relative to the root
/
/bin/bash
Relative paths
don't start with a/
resolved relative to current working direcctory
/Users/reinder/library
ls -R
cp source target
cp a dir/b
cp a b c dir
cp dir/a .
cp * dir
Copying directories
Use cp with the -R switch
Copies everything in the directory recursively
cp -R source_dir target_dir
cp -R dir1 dir2 dir3 target_dir
cp -R dir1 file1 dir2 file2 target_dir
ls -R folder =>show the nested files in the foler
moving files
mv a b =>rename
mv a dir/b
use it to move directories as well
mv a b c dir
mv * dir
Deleting Files
rm =>delete files perminently
rm a
rm a b c
rm dir/a dir/b
rm *
rmdir
will remove empty directories only
rm -r
will recursively remove all the directories in the folder
safety first: the -i switch
prompt before overwriting or deleting files
use the -i switch
cp -i and mv -i =>will ask you before overwriting files
rm -i will ask before deletion
Combine it with other options
cp -Ri
rm -ri
Wildcards
ls a*
ls *at*
ls -d m*n =>only directory
? => 1 character match
[acd7_]
Matches one of the characters in the list
Above would match a,c,d,7 or _
[^ax2] matches anything but a,x,2
range:[a-z],[0-9],[A-C3-5]
rm important_document-v[2-4].doc
rm important_document-v1[^789].docs
rm important_document-v?.docs
Brace Expansion
Generate Strings
Does not have to match existing filenames
Syntax: pre{list,of,strings}post
touch {a,b,c}.txt => touch a.txt b.txt c.txt
mv file.{txt,jpg} dir/
touch {a..c}{1..3}.txt =>touch a1.txt a2.txt...c3.txt
touch file{ab,b,c}{1,2,3}.{jpg,txt,zip}
Brace expansion comes before wildcard expression
mv *{txt,jpg} Documents => mv *txt *jpg Documents
mv filea?.{jpg,txt}a => mv filea?.jpg filea?.txt a
Output Redirection
Redirecting standard output stream
>
saves the output of a command to a file
ls > listing.txt
cat> story.txt =>ctrl+D to save
This will overwrite existing files
>>
Appends the output to the end of a file
echo "buy milk" >> shopping.txt =>append by mil to shopping.txt end of the file
PipeLine
cmd1 | cmd2 => output of cmd1 pass to input to cmd2
grep 1978 oscars.tsv
grep 1978 oscars.tsv | sort
grep 1978 oscars.tsv | sort > 1978_films.txt
cut -f 3 oscars.tsv => cut is used to sect specific column
cut -f 3 oscars.tsv | grep 4
cut -f 3 oscars.tsv | grep 4 | wc -l => wc-word count -l lines
Command substitution
Replace a command with its output
Ouput will become a part of the command line
Put command between $()
echo "hello,$(whoami)"
echo "Buy milk">"notes$(date).txt"
Note the use of double quotes
Keep command substitution intact
Older form uses backticks
echo "you are currently on 'hostname'"
echo "you are currently on $(hostname)"
Terminal and the Command Line
Bash: text in, text out
Terminal
Handles key pressed
Draws text
Fonts
Colors
Scrolling
Copy/past
copy/past => select the text click on the middle mouse it will past the selected text
Movement Keys:
ctrl-a start of line
ctrl-e end of the line
ctrl-f forward 1 char ->right arrow
ctrl-b back 1 char ->left arrow
alt-f forward 1 word -> command-Left
alt-b back 1 word ->command right
Deletion
Ctrl-D delete a char de
ctrl-H deelete a char backword
alt-D delete a word
Ctrl-w deelete a world backword ->Alt Backspace
ctrl-K delete reset of line
Ctrl-U delete from start of line
Ctrl-C break => end a running program
Ctrl-D end of transmission cat > x
ctrl-R search back in history
Editors
Nano
Tiny,simple,fast
vi (vim)
vi fileName => Press I to insert Esc for command mode=> :wq! to save and exit
:w save
:q to exit
:q! exit without saving
pre-installed on linux
Emacs
Very full featured
Pre installed on some linux distributors
Sort:
sort -k2 math_grades => -k is key column name ex: k2 second comumn
sort -nk2 match_grades
sort -rnk2 match_grades =>r reverse, n number sort
sort studentlist | uniq -c
sort studentlist | uniq -c |sort -nr
-r reverse sort
-n sorts numerically
-k sorts by fields
sort -k2
space-seperated fields by default
filter out repeated lines: uniq
Sort attendance | uniq
-c counts lines
Head & Tail:
Head:
Show first 10 lines of input by default
-n gives number of lines
head -n 1
Tail:
Show last 10 lines of input by default
-n gives number of lines
tail -n 1
tail -f follows newly appended data
ls -lS | head => first 10 lines
ls -lS | head -n 1 => first line
ls -lrS | tail -n 2 =>end of the input use tail , S-size of file
ls -lrS | tail -n 2| tail -n 1
tail auth.log =>last couple of entries on a file
tail -f auth.log => f-follow option
wc story.txt =>word count , 1st result no.of lines, no.of words, no.of bites
wc -l story.txt => -l lines
ls | wc -l
ls -a | wc -l
grep hello story.txt
grep steve *grades
grep -i steve *grades => -i ignore case
grep -v lecture math_attendance | sort |uniq => -v rule
grep -Ev "^$" math_attaendance | sort | uniq => -Ev extended regular expression
ls -l | grep -v "^d"
Grep:
grep searchs for a text in a file
Or multiple files. Ex: grep string*
-i makes search case-insensitive
-c counts occurences
-I shows line number of occurences
-v inverts the search (filter)
Can use regular expressions
find /usr
find /usr -name emacs
find . -name '*.txt'
find . -name '*.txt' -exec grep -l curious {} \; => -l file name having curious
Find:
powerful tool for searching files
find dir - lists all files in dir
find dir -name f
Lists all files named f in dir
Match Expression: find dir -name '*txt'
for More: http://goo.gl/Rid5
Search and Replace text (tr only reads from standard input)
tr S s physics_grades
cat physics_grades | tr S s
tr S s < physics_grades
Seperating date with semicolon in text file
grep \; oscars.tsv
tr \\t \; < oscars.tsv > oscars.csv
Advanced tools
sed
Stream editor
Transform text
Replace words
Most common use: to replace "old" with "new"
send 's/old/new/g'
awk
complete programming language
very useful for column-oriented files
perl
python
sed 's/curiouser/stranger/g' demo/alice > alice2
awk '{ total += s2 } END {print total/NR}' math_grades
Processing data:
sort -nk2 -t\; oscars.cst
sort -nk2 -t\; oscars.cst | head
cut -f 2 -d\; oscars.cst => -d delimeter ,-f field
cut -f 2,3 -d\; oscars.cst => -d delimeter ,-f field
paste *grades
sort physics_grades > phys_sorted
tr S s < physics_grades > physics_grades2
paste math_sorted phy_sorted
join math_sorted phys_sorted
Column-based date
sort -k to sort on a specific column
use -t to specify delimiter
cut to select a column from the input
cut -f 2 shows only second column
-d for delimiter
paste
puts lines of input files next to each other in the output
join
joins input files based on matching keys
Editing text files => Nano, vi, emacs
Sorting => sort, uniq
Head and tail
Searching and filtering => grep
Replacing text => tr,sd
Finding files => find
Advanced tools => awk,sd,perl,python
Column-based files => cut,paste,join
Input redirection with <
Multiple commands on one line => ;
Jobs and Process
cp /volumes/data-2T/Movie.avi .
fg
cp /volumes/data-2T/Movie.avi .
bg
cp /volumes/data-2T/Movie.avi .
./delayed_ls & => & means run backround
./delayed_ls > output_file &
rm -i oscars.tsv &
fg
rm -i oscar.tsv
Job Controls:
^Z suspend a running job
fg => send job to foreground
bg => send job to background
Background Jobs
Use & at end of line
Cannot read input from the user
If the program tries, bash will suspend it and let you know
Tip: only use wiht programs that don't need user interaction
Output willl mess up your screen
Tip: redirect output to file
find . > all_files &
./delayed_ls &
[1] 23837 => first one is jobid and second is process id
Killing:
Foreground program: ^C
End any program with kill
Can only end processes you own
Kill by job id:
kill %2
kill %cp
(fg and bg work with job id's as well)
Kill by process id
kill 6543
hard kill with -KILL:
kill -KILL 6543
pkill => match process by part of name
jobs => shows list of jobs
kill %job_id
ps -e => gives all list of running processses
kill process_no
pkill =>dangerious
Inspecting Processes:
Jobs => shows bash jobs for current shell
ps
display processes running under current shell
Displaying all processes
ps -e
Include owner
ps -ef
Process memory and CPU usage:
top => show all top running processsor
press ut then phani
kki
Background jobs => &, ^Z , bg,fg
Inspect processes => jobs,ps,top
End processes => kill, xkill, pkill
Bash Startup Files
.profile => loaded for login shell
.bashrc => loaded for non-login shell
Tip: read .bashrc from .profile => source ~/.bashrc
echo $PATH
PATH = "$PATH:~/bin" => always append at end
export EDITOR="nano" => Export variables that are needed in subprocess
sudo apt-get update
sudo apt-get install oracle-java8-installer
sudo apt install oracle-java8-set-default
sudo -H gedit /etc/environment
add below lines:
JAVA_HOME="/usr/lib/jvm/java-8-oracle"
JRE_HOME="/usr/lib/jvm/java-8-oracle/jre"
2. Maven Installation
cd /home/phani/Softwares/
wget http://www-eu.apache.org/dist/maven/maven-3/3.3.9/binaries/apache-maven-3.3.9-bin.tar.gz
sudo tar -xvzf apache-maven-3.3.9-bin.tar.gz
sudo mv apache-maven-3.3.9 maven =>rename
cd /etc/profile.d/
sudo gedit maven-env.sh
export MAVEN_HOME=/home/phani/Softwares/maven
export M2_HOME=/home/phani/Softwares/maven
export PATH=$PATH:$M2_HOME/bin
Commands:
$ cat filename
$ cat > file1.txt => creat file and append text and Ctrl+d
$ cat sample1.txt sample2.txt > sample3.txt => concatenate these two files and can save to another file named sample3.txt
$ cat -n file1.txt =>line number
$ cat file2.txt> file1.txt => To copy the content of one file to another file, you can use the greater than ‘>’ symbol with the cat command.
$ cat sample1.txt >> sample2.txt =>To append the contents of one file to another, you can use the double greater than ‘>>’ symbol with the cat command.
$gedit filename =>opens in text pad
cp /tmp/mozilla_phani0/eclipse-jee-helios-SR1-linux-gtk-x86_64.tar.gz .
tar xvf file.tar
x = eXtract, this indicated an extraction
v= verbose (optional) the files with relative locations will be displayed.
f = from/to file ... (what is next after the f is the archive file)
Remove files/folders
rm -rf /path/to/directory/*
-f - stands for "force" which is helpful when you don't want to be asked/prompted if you want to remove an archive, for example.
-r - stands for "recursive" which means that you want to go recursively down every folder and remove everything.
Duplicate copy folder
cp eclipse eclipse_bk
Creating Link (shortcut desktop)
ln -s source_file destination_path
Open Shell: xterm
echo $BASH
command,argument , options
ls -a list all files incuding hiden
ls -l list files in long format
ls -l-a combine both a and l options
ls -la combine both a and l options
ls -la /bin list all in /bin in long format
Order of options does not matter
The file name has to be at the end
Getting Help
Manual Pages => command: man
man ls
man cd
man man
Using the manual pages
use space to move down a page
move back a page with 'b'
search with '/'
exit with 'q'
drwxr
d -directory
History
ctrl+p previous
ctrl+n next
File Management
less FileName
reset =>used for reset the terminal
file fileName =>used to get the file information
file * => getting all files and folder information
mv =>move
rm -r images =>remove folder and its content recrusively
touch =>creating empty file
cat
best for small files -no paging
may mess up your terminal
use reset to fix terminal
less
pager with lots of features
see 'man less'
use space to move down a page
move back a page with b
search with '/'
exit with 'q'
Call the correct program directly
firefox index.html
opera index.html
you have to know what is installed on your system
File names
filenames can contain just about anything
except /
hidden files start with a dot
Case sensitivity
Extensions(.exe,.zip) are optional
file =>command show types
Absolute paths
start with /
relative to the root
/
/bin/bash
Relative paths
don't start with a/
resolved relative to current working direcctory
/Users/reinder/library
ls -R
cp source target
cp a dir/b
cp a b c dir
cp dir/a .
cp * dir
Copying directories
Use cp with the -R switch
Copies everything in the directory recursively
cp -R source_dir target_dir
cp -R dir1 dir2 dir3 target_dir
cp -R dir1 file1 dir2 file2 target_dir
ls -R folder =>show the nested files in the foler
moving files
mv a b =>rename
mv a dir/b
use it to move directories as well
mv a b c dir
mv * dir
Deleting Files
rm =>delete files perminently
rm a
rm a b c
rm dir/a dir/b
rm *
rmdir
will remove empty directories only
rm -r
will recursively remove all the directories in the folder
safety first: the -i switch
prompt before overwriting or deleting files
use the -i switch
cp -i and mv -i =>will ask you before overwriting files
rm -i will ask before deletion
Combine it with other options
cp -Ri
rm -ri
Wildcards
ls a*
ls *at*
ls -d m*n =>only directory
? => 1 character match
[acd7_]
Matches one of the characters in the list
Above would match a,c,d,7 or _
[^ax2] matches anything but a,x,2
range:[a-z],[0-9],[A-C3-5]
rm important_document-v[2-4].doc
rm important_document-v1[^789].docs
rm important_document-v?.docs
Brace Expansion
Generate Strings
Does not have to match existing filenames
Syntax: pre{list,of,strings}post
touch {a,b,c}.txt => touch a.txt b.txt c.txt
mv file.{txt,jpg} dir/
touch {a..c}{1..3}.txt =>touch a1.txt a2.txt...c3.txt
touch file{ab,b,c}{1,2,3}.{jpg,txt,zip}
Brace expansion comes before wildcard expression
mv *{txt,jpg} Documents => mv *txt *jpg Documents
mv filea?.{jpg,txt}a => mv filea?.jpg filea?.txt a
Output Redirection
Redirecting standard output stream
>
saves the output of a command to a file
ls > listing.txt
cat> story.txt =>ctrl+D to save
This will overwrite existing files
>>
Appends the output to the end of a file
echo "buy milk" >> shopping.txt =>append by mil to shopping.txt end of the file
PipeLine
cmd1 | cmd2 => output of cmd1 pass to input to cmd2
grep 1978 oscars.tsv
grep 1978 oscars.tsv | sort
grep 1978 oscars.tsv | sort > 1978_films.txt
cut -f 3 oscars.tsv => cut is used to sect specific column
cut -f 3 oscars.tsv | grep 4
cut -f 3 oscars.tsv | grep 4 | wc -l => wc-word count -l lines
Command substitution
Replace a command with its output
Ouput will become a part of the command line
Put command between $()
echo "hello,$(whoami)"
echo "Buy milk">"notes$(date).txt"
Note the use of double quotes
Keep command substitution intact
Older form uses backticks
echo "you are currently on 'hostname'"
echo "you are currently on $(hostname)"
Terminal and the Command Line
Bash: text in, text out
Terminal
Handles key pressed
Draws text
Fonts
Colors
Scrolling
Copy/past
copy/past => select the text click on the middle mouse it will past the selected text
Movement Keys:
ctrl-a start of line
ctrl-e end of the line
ctrl-f forward 1 char ->right arrow
ctrl-b back 1 char ->left arrow
alt-f forward 1 word -> command-Left
alt-b back 1 word ->command right
Deletion
Ctrl-D delete a char de
ctrl-H deelete a char backword
alt-D delete a word
Ctrl-w deelete a world backword ->Alt Backspace
ctrl-K delete reset of line
Ctrl-U delete from start of line
Ctrl-C break => end a running program
Ctrl-D end of transmission cat > x
ctrl-R search back in history
Editors
Nano
Tiny,simple,fast
vi (vim)
vi fileName => Press I to insert Esc for command mode=> :wq! to save and exit
:w save
:q to exit
:q! exit without saving
pre-installed on linux
Emacs
Very full featured
Pre installed on some linux distributors
Sort:
sort -k2 math_grades => -k is key column name ex: k2 second comumn
sort -nk2 match_grades
sort -rnk2 match_grades =>r reverse, n number sort
sort studentlist | uniq -c
sort studentlist | uniq -c |sort -nr
-r reverse sort
-n sorts numerically
-k sorts by fields
sort -k2
space-seperated fields by default
filter out repeated lines: uniq
Sort attendance | uniq
-c counts lines
Head & Tail:
Head:
Show first 10 lines of input by default
-n gives number of lines
head -n 1
Tail:
Show last 10 lines of input by default
-n gives number of lines
tail -n 1
tail -f follows newly appended data
ls -lS | head => first 10 lines
ls -lS | head -n 1 => first line
ls -lrS | tail -n 2 =>end of the input use tail , S-size of file
ls -lrS | tail -n 2| tail -n 1
tail auth.log =>last couple of entries on a file
tail -f auth.log => f-follow option
wc story.txt =>word count , 1st result no.of lines, no.of words, no.of bites
wc -l story.txt => -l lines
ls | wc -l
ls -a | wc -l
grep hello story.txt
grep steve *grades
grep -i steve *grades => -i ignore case
grep -v lecture math_attendance | sort |uniq => -v rule
grep -Ev "^$" math_attaendance | sort | uniq => -Ev extended regular expression
ls -l | grep -v "^d"
Grep:
grep searchs for a text in a file
Or multiple files. Ex: grep string*
-i makes search case-insensitive
-c counts occurences
-I shows line number of occurences
-v inverts the search (filter)
Can use regular expressions
find /usr
find /usr -name emacs
find . -name '*.txt'
find . -name '*.txt' -exec grep -l curious {} \; => -l file name having curious
Find:
powerful tool for searching files
find dir - lists all files in dir
find dir -name f
Lists all files named f in dir
Match Expression: find dir -name '*txt'
for More: http://goo.gl/Rid5
Search and Replace text (tr only reads from standard input)
tr S s physics_grades
cat physics_grades | tr S s
tr S s < physics_grades
Seperating date with semicolon in text file
grep \; oscars.tsv
tr \\t \; < oscars.tsv > oscars.csv
Advanced tools
sed
Stream editor
Transform text
Replace words
Most common use: to replace "old" with "new"
send 's/old/new/g'
awk
complete programming language
very useful for column-oriented files
perl
python
sed 's/curiouser/stranger/g' demo/alice > alice2
awk '{ total += s2 } END {print total/NR}' math_grades
Processing data:
sort -nk2 -t\; oscars.cst
sort -nk2 -t\; oscars.cst | head
cut -f 2 -d\; oscars.cst => -d delimeter ,-f field
cut -f 2,3 -d\; oscars.cst => -d delimeter ,-f field
paste *grades
sort physics_grades > phys_sorted
tr S s < physics_grades > physics_grades2
paste math_sorted phy_sorted
join math_sorted phys_sorted
Column-based date
sort -k to sort on a specific column
use -t to specify delimiter
cut to select a column from the input
cut -f 2 shows only second column
-d for delimiter
paste
puts lines of input files next to each other in the output
join
joins input files based on matching keys
Editing text files => Nano, vi, emacs
Sorting => sort, uniq
Head and tail
Searching and filtering => grep
Replacing text => tr,sd
Finding files => find
Advanced tools => awk,sd,perl,python
Column-based files => cut,paste,join
Input redirection with <
Multiple commands on one line => ;
Jobs and Process
cp /volumes/data-2T/Movie.avi .
fg
cp /volumes/data-2T/Movie.avi .
bg
cp /volumes/data-2T/Movie.avi .
./delayed_ls & => & means run backround
./delayed_ls > output_file &
rm -i oscars.tsv &
fg
rm -i oscar.tsv
Job Controls:
^Z suspend a running job
fg => send job to foreground
bg => send job to background
Background Jobs
Use & at end of line
Cannot read input from the user
If the program tries, bash will suspend it and let you know
Tip: only use wiht programs that don't need user interaction
Output willl mess up your screen
Tip: redirect output to file
find . > all_files &
./delayed_ls &
[1] 23837 => first one is jobid and second is process id
Killing:
Foreground program: ^C
End any program with kill
Can only end processes you own
Kill by job id:
kill %2
kill %cp
(fg and bg work with job id's as well)
Kill by process id
kill 6543
hard kill with -KILL:
kill -KILL 6543
pkill => match process by part of name
jobs => shows list of jobs
kill %job_id
ps -e => gives all list of running processses
kill process_no
pkill =>dangerious
Inspecting Processes:
Jobs => shows bash jobs for current shell
ps
display processes running under current shell
Displaying all processes
ps -e
Include owner
ps -ef
Process memory and CPU usage:
top => show all top running processsor
press ut then phani
kki
Background jobs => &, ^Z , bg,fg
Inspect processes => jobs,ps,top
End processes => kill, xkill, pkill
Bash Startup Files
.profile => loaded for login shell
.bashrc => loaded for non-login shell
Tip: read .bashrc from .profile => source ~/.bashrc
echo $PATH
PATH = "$PATH:~/bin" => always append at end
export EDITOR="nano" => Export variables that are needed in subprocess
Subscribe to:
Posts (Atom)