Presentation for my speech at Li

1. Advanced Shell Scripting
recalls, sed, awk and training
LUG Roma 3
Alessandro Manfredi http://www.lugroma3.org

2. Shell ?
sh, bash, ksh, csh, zsh, ...

3. How this works
YOU will propose tasks for training
( So, start thinking about them! )

4. Recalls
Should be familiar ...
$PS1 : pipes | and | filters

5. ‘ test ‘ conditions
-a, -o, -n : ***, **, ***
-eq, -ne : ******, *** ******
-gt, -lt, -ge, -le : ******* ****, ***** ****
-nt, -ot : ***** ****, ***** ****
-e, -f, -d : ******, ** ****, ** * *********
-b, -c : ******, *********
-r -w -x : ** ********, ** ********, ** **********
-s : **** ******* **** ****

6. Control structures
if [ condition ]; then action; else action; fi
for i in {1..n}; do action $i; done
for ((c=0;c<10;c++)); do echo $c; done
while read line; do action $line ; done < infile
case $var in 1|3|5) echo 1 ;; *) echo 0 ;; esac
Meanings
$?, $$, $#, $@, $1, $2, $3

7. Quoting
Double quotes (weak):
“ $this is going to be interpreted ”
Single quotes (strong):
‘ this is not going to be interpreted ‘
Backquotes ( or backticks ):
` /this/is/going/to/be/executed/with/its args `
Variables expansion:
“ ${ /the/result/may/be/quoted! } ”

8. Order
Brace Expansion e.g.> ./run_Script_{1..5}.sh
Tilde Expansion e.g.> echo ~ ~root ~xyz
Variable Expansion e.g.> echo $var{1..5}
Arithmetic Expansion e.g.> $(( $value+3*4/2 ))
Command Substitution e.g.> echo “Today is `date` "
Word Splitting e.g.> command arg1 arg2
Pathname Expansion e.g.> ls -lah | grep *.avi

9. Redirections
> and >> ( without descriptors STDOUT is assumed )
1 is STDOUT, 2 is STDERR
e.g.: command 1> stdout.txt 2> stderr.txt
2>&1 means “send STDERR to STDOUT”
1>&2 means “send STDOUT to STDERR”
>& and &> without descriptors redirect both 1 and 2

10. Vars
var=”whatever”
length=${#var}
substring=${var:from:length}
...
I/O
read var, read -p “Prompt : “ var
cat
echo $whatever
...

11. Functions
$ function greet {
> echo “Hello $1”
>}
$ greet “ Mr.`whoami` ”

12. and another endless tools list...
(e)grep [...] uniq
tail / head tee
tr, tr -d wc
cut bc
getopt(s) wget
sort nc
...

13. sed
non-interactive stream editor

14. Sed in pills
sed ‘3d’ - delete the 3rd line of the stream
sed ‘/^$/d’ - delete blank lines
sed -n ‘/pattern/p/ - print all pattern-matching lines
(by default all lines are also echoed, -n to suppress)
sed ‘s/find/replace/g’ - substitute all matching with a
replacement
sed ‘s/quote/”&”/’ - use & to copy the matched
pattern
echo "ff66" | sed 's/([a-z]*)(.*)/21/' - use 1 2
etc to copy portions of the matched pattern

15. How to use sed
cat file | sed ‘s/search/replace/g’
sed -e ‘cmd’ -i .ext file ( In place editing. If .ext is
given, original files will be moved to file.ext )
sed ‘cmd’ <old_file >new_file

16. Exercise
What is this ?
find . -print | sed -e 's;[^/]*/;|____;g;s;____|; |;g'
( hint: ; may be used as separators, like / or _ )

17. AWK

18. AWK
cat file | awk [opt] ‘pattern {action} [pattern {action}]’
input is parsed in records
foreach record
foreach pattern
if record match pattern -> action
default action : print $0
$0 is the entire record
$1 is the 1st field, $2 the 2nd, etc.

19. AWK
Patterns: BEGIN, END, /regex/, /regex/ in ( &&, ||, ! )
“” , $3==”value” , $4!=”value” , VAR==”value”
Actions: {var=0}, {print “Field2=”$2}, {print 3*2},
{var+=1}, {action1 ; action2}
{ for(i=1; i<=NF; i++){printf "-%s- ", $i } print "X" }
sub(/regex/,”replace”) , gsub(/regex/,”replace”,var)
Options: -v VAR=”value”, -F “FS”
Default variables: FS, RS, NR, NF

20. Training
... your turn !