====== Shell Scripts usage ======
\\
===== Websites =====
Color codes : http://misc.flogisoft.com/bash/tip_colors_and_formatting\\
Text to ascii generator : http://patorjk.com/software/taag/#p=display&f=Graffiti&t=Type%20Something\\
===== Function to check exit code =====
function test {
"$@"
local status=$?
if [ $status -ne 0 ]; then
echo "error with $1" >&2
fi
return $status
}
test command1
From http://stackoverflow.com/questions/5195607/checking-bash-exit-status-of-several-commands-efficiently
===== Ascii Art Bash =====
The aim here was to use the following ascii art :
P___----....
! __
' ~~ ---.#..__ ` ~ ~ - - . .:
` ~~--. .F~~___-__.
; , .- . _!
, ' ; ~ .
, ____ ; ' _ ._ ;
,_ . - '___#, ~~~ ---. _, . ' .#' ~ .;
=---==~~~ ~~~==--__ ; '~ -. ,#_ .'
' `~=.; ` /
' ' '.
' '
\ ' ' '
`.`\ ' . ; ,
\ ` ' ' ;
; ' ' '
/_ ., / __...---./ '
',_, __.--- ~~;#~ --..__ _'.-~;# // `.'
/ / ~~ .' . #; ~~ /// #; // /
/ ' . __ . ' ;#;_ . ////.;#;./ ; /
\ . / ,##' / _ /. '(/ ~||~\'
\ ` - . /_ . -==- ~ ' / (/ ' . ;;. ',
/' . ' -^^^...--- ``(/' _ ' '' `,;
##,. .#...( ' .c c .c c c. '.. ;; ../
%%#%;,..##.\_ ,;###;,. ;;.:##;,. raf
%%%%########%%%%;,.....,;%%%%%%;,.....,;%%%%%%%%%%%%%%%%%%%%............
Into the bashrc at login, but only in interactive mode, and using colors. To use colors, you need to use **echo -e**, however, you will face difficulties with special characters. An excellent way around I found (do not remember where) was to use variables to bypass the problem. The only variable to modify is `, change them by ' :
if [ "$PS1" ]; then
echo
echo
toto=" P___----...."
echo -e "$toto"
toto=' ! __'
echo -e "$toto"
toto=" ' ~~ ---.#..__ ' ~ ~ - - . .:"
echo -e "$toto"
toto=" ' "
toto0="_"
toto1=" ~~--. .F~~___-__."
echo -e "$toto\e[1m\e[38;5;202m$toto0\e[0m$toto1"
toto=" ; "
toto0="/_| _ / _ _"
toto1=" , .- . _! "
echo -e "$toto\e[1m\e[38;5;202m$toto0\e[0m$toto1"
toto=" , "
toto0="( |//)()/ (-"
toto1=" ' ; ~ ."
echo -e "$toto\e[1m\e[38;5;202m$toto0\e[0m$toto1"
toto=" , ____ ; ' _ ._ ; "
echo -e "$toto"
toto=" ,_ . - '___#, ~~~ ---. _, . ' .#' ~ .;"
echo -e "$toto"
toto=" =---==~~~ ~~~==--__ ; '~ -. ,#_ .'"
echo -e "$toto"
toto=" ' '~=.; ' /"
echo -e "$toto"
toto=" ' ' '. "
echo -e "$toto"
toto=" ' ' "
echo -e "$toto"
toto=" \ ' ' '"
echo -e "$toto"
toto=" '.'\ ' . ; ,"
echo -e "$toto"
toto=" \ ' ' Experimental ' ;"
echo -e "$toto"
toto=" ; ' Cluster ' '"
echo -e "$toto"
toto=" /_ ., / __...---./ '"
echo -e "$toto"
toto=" ',_, __.--- ~~;#~ --..__ _'.-~;# // '.'"
echo -e "$toto"
toto=" / / ~~ .' . #; ~~ /// #; // /"
echo -e "$toto"
toto=" / ' . __ . ' ;#;_ . ////.;#;./ ; /"
echo -e "$toto"
toto=" \ . / ,##' / _ /. '(/ ~||~\'"
echo -e "$toto"
toto=" \ ' - . /_ . -==- ~ ' / (/ ' . ;;. ', "
echo -e "$toto"
toto=" /' . ' -^^^...--- ''(/' _ ' '' ',;"
echo -e "$toto"
toto="##,. .#..."
toto0="( ' .c c .c c c. '"
toto1=".."
toto2=" ;; "
toto3=".."
toto4="/ "
echo -e "\e[1m\e[38;5;27m$toto\e[0m$toto0\e[1m\e[38;5;27m$toto1\e[0m$toto2\e[1m\e[38;5;27m$toto3\e[0m$toto4"
toto="%%#%;,..##."
toto0="\_ "
toto1=",;###;,."
toto2=" ;;"
toto3=".:##;,. raf"
echo -e "\e[1m\e[38;5;27m$toto\e[0m$toto0\e[1m\e[38;5;27m$toto1\e[0m$toto2\e[1m\e[38;5;27m$toto3\e[0m"
toto="%%%%########%%%%;,.....,;%%%%%%;,.....,;%%%%%%%%%%%%%%%%%%%%............"
echo -e "\e[1m\e[38;5;27m$toto\e[0m"
echo
echo
fi;
Add this into your .bashrc or in the /etc/bashrc for all users, and you obtain the following result :
{{ :software:system:allhailstoamber.png |}}
===== Ascii symbols =====
All :
๑•ิ.•ั๑ ๑۩۞۩๑ ♬✿.｡.:* ★ ☆ εїз℡❣·۰•●○●ōゃ ♥ ♡๑۩ﺴ ☜ ☞ ☎ ☏♡ ⊙◎ ☺ ☻✖╄ஐﻬ ► ◄ ▧ ▨ ♨ ◐ ◑ ↔ ↕ ▪ ▫ ☼ ♦ ▀ ▄ █▌
▐░ ▒ ▬♦ ◊ ◦ ☼ ♠♣ ▣ ▤ ▥ ▦ ▩ ◘ ◙ ◈ ♫ ♬ ♪ ♩ ♭ ♪ の ☆ → あ ぃ ￡ ❤＃ ＠ ＆ ＊ ❁ ❀ ✿ ✾ ❃ ✺ ❇ ❈ ❊ ❉ ✱ ✲ ✩ ✫
✬ ✭ ✮ ✰ ☆ ★ ✪ ¤ ☼ ☀ ☽ ☾ ❤ ♡ ღ☻ ☺ ❂ ◕ ⊕ ☉ Θ o O ♋ ☯ ㊝ ⊙ ◎◑ ◐ ۰ • ● ▪ ▫ ｡ ﾟ ๑ ☜ ☞ ☂ ♨ ☎ ☏ × ÷ ＝ ≠
≒ ∞ ˇ ± √ ⊥▶ ▷ ◀ ◁ ☀ ☁ ☂ ☃ ☄ ★ ☆ ☇ ☈ ☉ ☊ ☋ ☌ ☍ ☑ ☒☢ ☸ ☹ ☺ ☻ ☼ ☽ ☾ ♠ ♡ ♢ ♣ ♤ ♥ ♦ ♧ ♨ ♩ ✙ ✈ ✉ ✌ ✁
♝ ♞♯♩♪♫♬♭♮ ☎ ☏ ☪ ♈ ♨ ₪ ™ ♂✿ ♥ の ↑ ↓ ← → ↖ ↗ ↙ ↘ ㊣ ◎ ○ ● ⊕ ⊙ ○ △ ▲ ☆ ★ ◇ ◆ ■ □ ▽ ▼ § ￥ 〒 ￠ ￡ ※
♀ ♂ &⁂ ℡ ↂ░ ▣ ▤ ▥ ▦ ▧ ✐✌✍✡✓✔✕✖ ♂ ♀ ♥ ♡ ☜ ☞ ☎ ☏ ⊙ ◎ ☺ ☻ ► ◄ ▧ ▨ ♨ ◐ ◑ ↔ ↕ ♥ ♡ ▪ ▫ ☼ ♦ ▀ ▄ █ ▌ ▐
░ ▒ ▬ ♦ ◊ ◘ ◙ ◦ ☼ ♠ ♣ ▣ ▤ ▥ ▦ ▩ ◘ ◙ ◈ ♫ ♬ ♪ ♩ ♭ ♪ ✄☪☣☢☠░ ▒ ▬ ♦ ◊ ◦ ♠ ♣ ▣ ۰•● ❤ ●•۰► ◄ ▧ ▨ ♨ ◐ ◑ ↔ ↕
▪ ▫ ☼ ♦♧♡♂♀♠♣♥❤☜☞☎☏⊙◎ ☺☻☼▧▨♨◐◑↔↕▪ ▒ ◊◦▣▤▥ ▦▩◘ ◈◇♬♪♩♭♪の★☆→あぃ￡Ю〓§♤♥▶¤๑⊹⊱⋛⋌⋚⊰⊹ ๑۩۩.. ..۩۩๑
๑۩۞۩๑ ✲ ❈ ✿ ✲ ❈ ➹ ~.~ ◕‿- ❣ ✚ ✪ ✣ ✤ ✥ ✦❉ ❥ ❦ ❧ ❃ ❂ ❁ ❀ ✄ ☪ ☣ ☢ ☠ ☭ღღღ ▶ ▷ ◀ ◁ ☀ ☁ ☂ ☃ ☄ ★
☆ ☇ ☈ ⊙ ☊ ☋ ☌ ☍ⓛⓞⓥⓔ๑•ิ.•ั๑ ๑۩۞۩๑ ♬✿ ☉♡ ♢ ♣ ♤ ♥ ♦ ♧ ♨ ♩ ✙✈ ✉ ✌ ✁ ✎ ✐ ❀ ✰ ❁ ❤ ❥ ❦❧ ➳ ➽
εїз℡❣·۰•●○●ゃōゃ♥ ♡๑۩ﺴ ☜ ☞ ☎ ☏♡ ⊙◎ ☺ ☻✖╄ஐﻬ ► ◄ ▧ ▨ ♨ ◐ ◑ ↔ ↕ ▪ ▫ ☼ ♦ ▀ ▄ █▌ ▐░ ▒ ▬♦ ◊ ◦ ☼ ♠♣ ▣ ▤ ▥ ▦
▩ ◘ ◙ ◈ ♫ ♬ ♪ ♩ ♭ ♪ の ☆ → あ ぃ ￡ ❤ ❁ ❀ ✿ ✾ ❃ ✺ ❇ ❈ ❊ ❉ ✱ ✲ ✩ ✫ ✬ ✭ ✮ ✰ ☆ ★ ✪ ¤ ☼ ☀ ☽ ☾ ❤ ♡ ღ☻
☺ ❂ ◕ ⊕ ☉ Θ o O ♋ ☯ ㊝ ⊙ ◎ ◑ ◐ ۰ • ● ▪ ▫ ｡ ﾟ ๑ ☜ ☞ ☂ ♨ ☎ ☏▶ ▷ ◀ ◁ ☀ ☁ ☂ ☃ ☄ ★ ☆ ☇ ☈ ☉ ☊ ☋ ☌ ☍ ☑
☒☢ ☸ ☹ ☺ ☻ ☼ ☽ ☾ ♠ ♝ ♞♯♩♪♫♬♭♮ ☎ ☏ ☪ ♈ ♨ ºº ₪ ¤ 큐 « »™ ♂✿ ♥ の ↑ ↓ ← → ↖ ↗ ↙ ↘ ㊣ ◎ ○ ● ⊕ ⊙ ○ △ ▲
☆ ★ ◇ ◆ ■ □ ▽ ▼ § ￥〒 ￠ ￡ ※ ♀ ♂ © ® ⁂ ℡ ↂ░ ▣ ▤ ▥ ▦ ▧ ✐✌✍✡✓✔✕✖ ♂ ♀ ♥ ♡ ☜ ☞ ☎ ☏ ⊙ ◎ ☺ ☻ ► ◄ ▧ ▨
♨ ◐ ◑ ↔ ↕ ♥ ♡ ▪ ▫ ☼ ♦ ▀ ▄ █ ▌ ▐ ░ ▒ ▬ ♦ ◊ ◘ ◙ ◦ ☼ ♠ ♣ ▣ ▤ ▥ ▦ ▩ ◘ ◙ ◈ ♫ ♬ ♪ ♩ ♭ ♪ ✄☪☣☢☠㊊㊋㊌㊍㊎㊏ ㊐㊑
㊒㊓㊔㊕㊖㊗㊘㊜㊝㊞㊟㊠㊡㊢ ㊣㊤㊥㊦㊧㊨㊩㊪㊫㊬㊭㊮㊯㊰✗✘✚✪✣✤✥✦✧✩✫✬✭✮✯✰ ✱✲✳❃❂❁❀✿✾✽✼✻✺✹✸✷
✶✵✴❄❅❆❇❈❉❊❋❖☀☂☁【】┱ ┲ ❣ ✪ ✣ ✤ ✥ ✦ ❉ ❥ ❦ ❧ ❃ ❂ ❁ ❀ ✄ ☪ ☣ ☢ ☠ ☭ ♈ ➸ ✓ ✔ ✕ ✖ .: ◢ ◣ ◥ ◤ ▽ ▧
▨ ▣ ▤ ▥ ▦ ▩ ◘ ◙ ▓ ▒ ░ ™ ℡ 凸 の ๑۞๑ ๑۩ﺴ ﺴ۩๑ o(‧”’‧)o ❆ べò⊹⊱⋛⋋ ⋌⋚⊰⊹ ⓛⓞⓥⓔ ☀ ☼ ☜ ☞ ⊙® ◈ ♦ ◊ ◦ ◇ ◆
εїз❃❂❁❀✿✾✽✼✻✺✹✸✷ ✶✵✴❄❅❆❇❈❉ ❊❋❖❤❥❦❧↔ ↕ ▪ → ︷╅╊✿ (¯`•._.• •._.•´¯)(¯`•¸•´¯) ❤`•.¸¸.•´´¯`••
.¸¸.•´¯`•.•●•۰• ••.•´¯`•.•• ••.•´¯`•.••—¤÷(`[¤* *¤]´)÷¤——(•·÷[ ]÷·•)— ①②③④⑤⑥⑦⑧⑨⑩ ⑪⑫⑬⑭⑮⑯⑰⑱⑲⑳ ⒶⒷⒸⒹⒺⒻ
ⒼⒽⒾⒿⓀⓁ ⓂⓃⓄⓅⓆⓇ ⓈⓉⓊⓋⓌⓍ ⓎⓏ ⓐⓑⓒⓓⓔⓕ ⓖⓗⓘⓙⓚⓛ ⓜⓝⓞⓟⓠⓡ ⓢⓣⓤⓥⓦⓧ ⓨⓩ(⊙▂⊙✖ )(づ ￣ ³￣)づ
( c//”-}{-*\x)(-’๏_๏’-)(◐ o ◑ )(⊙…⊙ )๏[-ิ_•ิ]๏(•ิ_•ิ)(•ิ_•ิ) (/•ิ_•ิ)(︶︹︺)
Arrows:
← ↑ → ↓ ↔ ↕ ↖ ↗ ↘ ↙ ↚ ↛ ↜ ↝ ↞ ↟
↠ ↡ ↢ ↣ ↤ ↥ ↦ ↧ ↨ ↩ ↪ ↫ ↬ ↭ ↮ ↯
↰ ↱ ↲ ↳ ↴ ↵ ↶ ↷ ↸ ↹ ↺ ↻ ↼ ↽ ↾ ↿
⇀ ⇁ ⇂ ⇃ ⇄ ⇅ ⇆ ⇇ ⇈ ⇉ ⇊ ⇋ ⇌ ⇍ ⇎ ⇏
⇐ ⇑ ⇒ ⇓ ⇔ ⇕ ⇖ ⇗ ⇘ ⇙ ⇚ ⇛ ⇜ ⇝ ⇞ ⇟
⇠ ⇡ ⇢ ⇣ ⇤ ⇥ ⇦ ⇧ ⇨ ⇩ ⇪ ⇫ ⇬ ⇭ ⇮ ⇯
⇰ ⇱ ⇲ ⇳ ⇴ ⇵ ⇶ ⇷ ⇸ ⇹ ⇺ ⇻ ⇼ ⇽ ⇾ ⇿
⟰ ⟱ ⟲ ⟳ ⟴ ⟵ ⟶ ⟷ ⟸ ⟹ ⟺ ⟻ ⟼ ⟽ ⟾ ⟿
⤀ ⤁ ⤂ ⤃ ⤄ ⤅ ⤆ ⤇ ⤈ ⤉ ⤊ ⤋ ⤌ ⤍ ⤎ ⤏
⤐ ⤑ ⤒ ⤓ ⤔ ⤕ ⤖ ⤗ ⤘ ⤙ ⤚ ⤛ ⤜ ⤝ ⤞ ⤟
⤠ ⤡ ⤢ ⤣ ⤤ ⤥ ⤦ ⤧ ⤨ ⤩ ⤪ ⤫ ⤬ ⤭ ⤮ ⤯
⤰ ⤱ ⤲ ⤳ ⤴ ⤵ ⤶ ⤷ ⤸ ⤹ ⤺ ⤻ ⤼ ⤽ ⤾ ⤿
⥀ ⥁ ⥂ ⥃ ⥄ ⥅ ⥆ ⥇ ⥈ ⥉ ⥊ ⥋ ⥌ ⥍ ⥎ ⥏
⥐ ⥑ ⥒ ⥓ ⥔ ⥕ ⥖ ⥗ ⥘ ⥙ ⥚ ⥛ ⥜ ⥝ ⥞ ⥟
⥠ ⥡ ⥢ ⥣ ⥤ ⥥ ⥦ ⥧ ⥨ ⥩ ⥪ ⥫ ⥬ ⥭ ⥮ ⥯
⥰ ⥱ ⥲ ⥳ ⥴ ⥵ ⥶ ⥷ ⥸ ⥹ ⥺ ⥻ ⥼ ⥽ ⥾ ⥿
Geometric shapes:
■ □ ▢ ▣ ▤ ▥ ▦ ▧ ▨ ▩ ▪ ▫ ▬ ▭ ▮ ▯
▰ ▱ ▲ △ ▴ ▵ ▶ ▷ ▸ ▹ ► ▻ ▼ ▽ ▾ ▿
◀ ◁ ◂ ◃ ◄ ◅ ◆ ◇ ◈ ◉ ◊ ○ ◌ ◍ ◎ ●
◐ ◑ ◒ ◓ ◔ ◕ ◖ ◗ ◘ ◙ ◚ ◛ ◜ ◝ ◞ ◟
◠ ◡ ◢ ◣ ◤ ◥ ◦ ◧ ◨ ◩ ◪ ◫ ◬ ◭ ◮ ◯
◰ ◱ ◲ ◳ ◴ ◵ ◶ ◷ ◸ ◹ ◺ ◻ ◼ ◽ ◾ ◿
===== Ascii japanese emoticons =====
╮(─▽─)╭ ¯\_(ツ)_/¯ ٩(-̮̮̃-̃)۶
ヽ（´ー｀）┌ ┗┐ヽ(′Д、`*)ﾉ┌┛ ヽ( ´¬`)ノ
┗┃・ ■ ・┃┛ ヾ(´A｀)ノﾟ ヽ（・＿・；)ノ
ヽ( ￣д￣;)ノ ＼（〇_ｏ）／ ヽ(。_°)ノ
＼(;´□｀)/ ヾ(*´ー`)ノ ヽ(‘ー`)ノ
ヽ(ー_ー )ノ ヽ(´～｀；） ┐(‘～`；)┌
ヽ（*ω。）ノ (;´・`)> （＾～＾）
٩(-̮̮̃•̃)۶ ┐(￣ー￣)┌ t(ツ)_/¯
┐(￣ヮ￣)┌ ╰( ´◔ ω ◔ `)╯ ಠ_ಠ
ლ(ಠ_ಠლ) ¯\(°_o)/¯ ╮(╯▽╰)╭
\\
===== Simulating kghostview with okular =====
When using tools like KCacheGrind, you will encounter difficulties because KGhostView has been replaced by Okular. To bypass it, and make your tools use Okular: (note that this "bypass" can be used with all programs, including compilers!)
mkdir $HOME/bin
touche $HOME/bin/kghostview
edit the file, using vim for example :
vim $HOME/bin/kghostview
add :
#!/bin/bash
okular "$@"
save and close, and make it executable :
chmod +x $HOME/bin/kghostview
add it to your PATH :
export PATH=$HOME/bin/:$PATH
Now you can use KCacheGrind or other tools that need to use KGhostView, they will use okular instead.
===== Redirection =====
Redirect everything to file, even when using make or configure :
> output.txt 2>&1
To add a string at the end of file (for example, adding an export at the end of your bashrc to make it permanent) :
echo 'export PATH=/soft/gnuplot-4.4/bin:$PATH' >> ~/.bashrc
General redirection table :
^ Character ^ Action ^
| > | Redirect standard output |
| >& | Redirect standard output and standard error |
| < | Redirect standard input |
| >! | Redirect standard output; overwrite file if it exists |
| >&! | Redirect standard output and standard error; overwrite file if it exists |
| > > | Append standard output |
| > > filename 2>&1 | Append standard output and standard error |
===== Multiple tasks =====
To wget multiple of files at once, if the files have the same names and differ only by a numerous, and if you are using bash > 4.0 :
wget http://theadress.com/thegile{001..030}.dat
===== Loops and if =====
Read a file line per line, line is stored in $line, and work on values (users), a progress is displayed :
while read line
do
compteur=$(expr $compteur + 1)
echo -ne "User $compteur/$nb_ldap_users\r"
# Get ids
user_uid=$(id "$line" | awk -F'(' '{print $2;}' | awk -F')' '{print $1;}')
user_gid=$(id "$line" | awk -F'(' '{print $3;}' | awk -F')' '{print $1;}')
# Get home from passwd
user_home=$(getent passwd "$line" | cut -d: -f6)
# Get real home position
# First check if home exist
ls -l "$user_home" 2>> errors > /dev/null
if [ $? -eq 0 ];then
user_home_real=$(ls -l "$user_home" | awk -F' ' '{print $10;}')
# Get real home uid and gid and check them
user_home_real_uid=$(ls -ld $user_home_real | awk -F' ' '{print $3;}')
user_home_real_gid=$(ls -ld $user_home_real | awk -F' ' '{print $4;}')
# check
if [ "$user_uid" != "$user_home_real_uid" ]; then
echo "Bad home uid for user $line. Real : $user_uid, got : $user_home_real_uid " >> errors
fi
if [ "$user_gid" != "$user_home_real_gid" ]; then
echo "Bad home gid for user $line. Real : $user_gid, got : $user_home_real_gid " >> errors
fi
else
echo "Could not find home for user $line" >> errors
fi
# Write result
echo $line $user_gid $user_home $user_home_real $user_home_real_uid $user_home_real_gid >> result
done < ldap_users
echo -ne '\n'
Now, using the same, to get the size of all repertories listed in a text file :
rm size.txt; compteur=0
# Get informations on user
while read line
do
compteur=$(expr $compteur + 1)
echo -ne "User $compteur/516\r"
du -ch -sB K "$line" | grep total | awk -F' ' '{print $1;}' >> size.txt
done < home_users_nr_d
echo -ne '\n'
All size are noted like "132K\n", easy to manipulate after.
===== Basis =====
==== sed and awk ====
In a shell, you can 'pipe' output to another command. tutu.dat contains :
Y X
1.2 3.4
2.3 3.5
-0.6 3.6
To extract the value 2.3 :
grep 3.4 tyty
Result :
1.2 3.4
To take only the first value (1.2), ask awk to keep only the first value of the line :
grep 3.4 tyty | awk -F' ' '{print $1;}'
Result :
1.2
If the separator is different: echo "1.2/1.3" | awk -F'/' '{print $1;}', it still gives 1.2.
To sum all values of the first row : two ways, remove the first line when piping, or remove all lines using symbol Y for example.
First way :
sed '1d' tyty
Result :
1.2 3.4
2.3 3.5
-0.6 3.6
(of course, replace 1d with 2d for second line, or with '5,10d' for lines 5 and 10, etc)
Second way :
grep -v 'Y' tyty
Result :
1.2 3.4
2.3 3.5
-0.6 3.6
Note that to add another string, use \| :
grep -v 'Y\|1.2' tyty
Result :
2.3 3.5
-0.6 3.6
Then, pipe it on awk to recover only first row :
grep -v 'Y' tyty | awk -F' ' '{print $1;}'
Result :
1.2
2.3
-0.6
Then, you want all of this to be the same as an addition expression, compatible with bc command (the gnu calculator).
grep -v 'Y' tyty | awk -F' ' '{print $1;}' | paste -sd+
Result :
1.2+2.3+-0.6
Of course, using -sd* will give you 1.2*2.3*-0.6, etc.
Last step is to pipe it to bc :
grep -v 'Y' tyty | awk -F' ' '{print $1;}' | paste -sd+ | bc
Result :
2.9
Tip: do no try to merge steps. You can do very shorts commands that do the same thing. Very useful to show off, but bad idea to explain to others, to recover a single step, or simply to remember how it works few months later.
To put this into a variable :
myvalue=$( grep -v 'Y' tyty | awk -F' ' '{print $1;}' | paste -sd+ | bc )
echo $myvalue should give 2.9.
==== Loops ====
In bash, there are multiple ways to use loops, by I prefer to use the way like C :
#!/bin/bash
for (( c=1; c<=5; c++ ))
do
echo "Welcome $c times"
done
#!/bin/bash
for (( c=2; c<=10; c=c+2 ))
do
echo "Welcome $c times"
done
etc. Use if statement to break the loop :
if [[ $myvalue == 0 ]]
then
echo "Test $c Passed"
break
else
echo "Test $c Failed, Restart"
fi
In this example, we can start a program, and recover its exit statue to check if it executed and exited normally. Then, we break the loop if yes, or we restart it if no.
for (( c=1; c<=30; c++ )) # 30 Tests
do
grep -i "$c" Test
RETVAL=$?
if [[ $RETVAL == 0 ]]
then
echo "Test $c Passed"
break
else
echo "Test $c Failed, Restart"
fi
done
$? contains the value of the exist status of the last command.
==== Arguments ====
If you want to pass variables/arguments to launched/son scripts, and recover variables from them.
Consider the following script, in myscript.sh (do not forget to do a chmod +x to execute it):
In_Buffer="$@"
val1=$( echo $In_Buffer | awk '{print $1;}')
val2=$( echo $In_Buffer | awk '{print $2;}')
val3=$( echo $In_Buffer | awk '{print $3;}')
echo $val1 $val2 $val3
./myscript.sh
give nothing.
./myscript.sh 10 13 16 9
returns :
10 13 16
The son's script has recovered the values. They can be seen has subroutine/function arguments.
Now, to return values, just do a final echo in the son script, using a special word, let's say "SON_RETURN".
In_Buffer="$@"
val1=$( echo $In_Buffer | awk '{print $1;}')
val2=$( echo $In_Buffer | awk '{print $2;}')
val3=$( echo $In_Buffer | awk '{print $3;}')
echo $val1 $val2 $val3
sumval=$(echo "$val1+$val2+$val3" | bc)
echo '\nSON_RETURN' $sumval
Now, using the command you get :
./myscript.sh 10 13 16
10 13 16
\nSON_RETURN 39
You can then recover this result to a variable :
returnval=$(./myscript.sh 10 13 16)
then
echo -e $returnval | grep 'SON_RETURN'
SON_RETURN 39
So a simple awk will allow you to extract 39 or other returned variables. Note that the -e argument to echo allow the \n character to be used has a return. Without it, the grep could not work properly.
==== plot ====
Now, I want to use gnuplot to plot things into png files. You can use everything seen before to create gnuplot scripts, and launch them.
Suppose we want to plot the tyty file seen above.
Create a file "gnuplotscript", and add :
# Set png terminal and png size
set terminal png size 640,480
# Set output file name
set output 'myfile.png'
# Ask gnuplot to use first line as title name
set key autotitle columnhead
# Plot (exit automaticaly after)
plot 'tyty' with line
Now, use gnuplot :
gnuplot gnuplotscript
And it will output your graph in png.
===== Monitor using bash =====
I present here a small project I worked on. The idea was to be able to monitor multiple systems, using only simple user accounts and without installing anything on remote machines. This script had to render everything in HTML, and had to actualise information every 15 minutes.
==== Acquire information ====
First, we need to acquire infos on the system :
Date :
current_date=$(date +"%s")
Network :
- ipv4 and ipv6 in and out :
ipv4_in=$( echo "`egrep -v bond\|lo\|face\|Inter /proc/net/dev | awk -F' ' '{print $2;}' | paste -sd+ | bc` * 8" | bc )
ipv4_out=$( echo "`egrep -v bond\|lo\|face\|Inter /proc/net/dev | awk -F' ' '{print $10;}' | paste -sd+ | bc` * 8" | bc )
ipv6_in=$( echo "`egrep Ip6InOctets /proc/net/snmp6 | awk -F' ' '{print $2;}'` * 8" | bc )
ipv6_out=$( echo "`egrep Ip6OutOctets /proc/net/snmp6 | awk -F' ' '{print $2;}'` * 8" | bc )
totalip_in=$( echo "$ipv4_in + $ipv6_in" | bc )
totalip_out=$( echo "$ipv4_out + $ipv6_out" | bc )
totalip_in=$( echo "$totalip_in*0.001" | bc ) # in Kilooctets (from Octets)
totalip_out=$( echo "$totalip_out*0.001" | bc ) # in Kilooctets (from Octets)
Flux if you now the previous get :
difftotalip_in=$( echo "scale=4;( $totalip_in - $previous_totalip_in ) / ( $current_date - $previous_date )" | bc) # in Ko/s
difftotalip_out=$( echo "scale=4;( $totalip_out - $previous_totalip_out ) / ( $current_date - $previous_date )" | bc) # in Ko/s
Ssh :
Number of correct connexions for the user spheniscus :
grep -i "spheniscus" /var/log/auth.log | wc -l
Number of invalid user authentications and failed password :
grep -i "Invalid user" /var/log/auth.log | wc -l
grep -i "Failed password" /var/log/auth.log | wc -l
I/O :
Total HDD I/O :
totalio_write=$(echo "`grep -i "sda " /proc/diskstats | awk '{print $10;}'` * 512 * 0.001" | bc) # In Ko
totalio_read=$(echo "`grep -i "sda " /proc/diskstats | awk '{print $6;}'` * 512 * 0.001" | bc) # In Ko
Flux if you now the last get :
difftotalio_write=$( echo "scale=4;( $totalio_write - $previous_totalio_write ) / ( $current_date - $previous_date )" | bc) # in Ko/s
difftotalio_read=$( echo "scale=4;( $totalio_read - $previous_totalio_read ) / ( $current_date - $previous_date )" | bc) # in Ko/s
CPU :
cpu_idle=$( echo "`top -b -n 5 | grep -i "Cpu(s)" | tr -d '%id,' | awk '{print $2;}' | awk 'NR==5'`");
cpu_used=$( echo "100 - $cpu_idle" | bc )
Memory :
MemTotal=$( echo "`grep -i "MemTotal" /proc/meminfo | tr -d 'kB' | awk '{print $2;}'`")
MemFree=$( echo "`grep -i "MemFree" /proc/meminfo | tr -d 'kB' | awk '{print $2;}'`")
Cached=$( echo "`grep -i "Cached" /proc/meminfo | tr -d 'kB' | awk '{print $2;}' | awk 'NR==1'`")
Buffers=$( echo "`grep -i "Buffers" /proc/meminfo | tr -d 'kB' | awk '{print $2;}'`")
MemUsed=$( echo "$MemTotal - $MemFree - $Buffers - $Cached" | bc )
MemCached=$( echo "$Buffers + $Cached" | bc )
MemUsed=$( echo "$MemUsed*0.001" | bc ) # in MegaBytes
MemCached=$( echo "$MemCached*0.001" | bc ) # in MegaBytes
MemFree=$( echo "$MemFree*0.001" | bc ) # in MegaBytes
SwapTotal=$( echo "`grep -i "SwapTotal" /proc/meminfo | tr -d 'kB' | awk '{print $2;}'`")
SwapFree=$( echo "`grep -i "SwapFree" /proc/meminfo | tr -d 'kB' | awk '{print $2;}'`")
SwapUsed=$( echo "$SwapTotal - $SwapFree" | bc )
SwapUsed=$( echo "$SwapUsed*0.001" | bc ) # in MegaBytes
==== ssh launch ====
We then launch these information gathering scripts using ssh, and using input and output system seen in the basis section of this page (to keep in memory previous records and date, to evaluate flux for example). I assume you already have a key authentication system for ssh, and that you do not have passphrase (bad idea on a server, but ok four your monitoring station).
The script will be (file name : ssh_llcheck.bash):
# /bin/bash
In_Buffer="$@"
previous_date=$( echo $In_Buffer | awk '{print $1;}')
previous_totalip_in=$( echo $In_Buffer | awk '{print $2;}')
previous_totalip_out=$( echo $In_Buffer | awk '{print $3;}')
previous_totalio_write=$( echo $In_Buffer | awk '{print $4;}')
previous_totalio_read=$( echo $In_Buffer | awk '{print $5;}')
current_date=$(date +"%s")
#
# NETWORK
#
ipv4_in=$( echo "`egrep -v bond\|lo\|face\|Inter /proc/net/dev | awk -F' ' '{print $2;}' | paste -sd+ | bc` * 8" | bc )
ipv4_out=$( echo "`egrep -v bond\|lo\|face\|Inter /proc/net/dev | awk -F' ' '{print $10;}' | paste -sd+ | bc` * 8" | bc )
ipv6_in=$( echo "`egrep Ip6InOctets /proc/net/snmp6 | awk -F' ' '{print $2;}'` * 8" | bc )
ipv6_out=$( echo "`egrep Ip6OutOctets /proc/net/snmp6 | awk -F' ' '{print $2;}'` * 8" | bc )
totalip_in=$( echo "$ipv4_in + $ipv6_in" | bc )
totalip_out=$( echo "$ipv4_out + $ipv6_out" | bc )
totalip_in=$( echo "$totalip_in*0.001" | bc ) # in Kilooctets (from Octets)
totalip_out=$( echo "$totalip_out*0.001" | bc ) # in Kilooctets (from Octets)
difftotalip_in=$( echo "scale=4;( $totalip_in - $previous_totalip_in ) / ( $current_date - $previous_date )" | bc) # in Ko/s
difftotalip_out=$( echo "scale=4;( $totalip_out - $previous_totalip_out ) / ( $current_date - $previous_date )" | bc) # in Ko/s
#
# HARDWARE
#
totalio_write=$(echo "`grep -i "sda " /proc/diskstats | awk '{print $10;}'` * 512 * 0.001" | bc) # In Ko
totalio_read=$(echo "`grep -i "sda " /proc/diskstats | awk '{print $6;}'` * 512 * 0.001" | bc) # In Ko
difftotalio_write=$( echo "scale=4;( $totalio_write - $previous_totalio_write ) / ( $current_date - $previous_date )" | bc) # in Ko/s
difftotalio_read=$( echo "scale=4;( $totalio_read - $previous_totalio_read ) / ( $current_date - $previous_date )" | bc) # in Ko/s
cpu_idle=$( echo "`top -b -n 5 | grep -i "Cpu(s)" | tr -d '%id,' | awk '{print $5;}' | awk 'NR==5'`");
cpu_used=$( echo "100 - $cpu_idle" | bc )
MemTotal=$( echo "`grep -i "MemTotal" /proc/meminfo | tr -d 'kB' | awk '{print $2;}'`")
MemFree=$( echo "`grep -i "MemFree" /proc/meminfo | tr -d 'kB' | awk '{print $2;}'`")
Cached=$( echo "`grep -i "Cached" /proc/meminfo | tr -d 'kB' | awk '{print $2;}' | awk 'NR==1'`")
Buffers=$( echo "`grep -i "Buffers" /proc/meminfo | tr -d 'kB' | awk '{print $2;}'`")
MemUsed=$( echo "$MemTotal - $MemFree - $Buffers - $Cached" | bc )
MemCached=$( echo "$Buffers + $Cached" | bc )
MemUsed=$( echo "$MemUsed*0.001" | bc ) # in MegaBytes
MemCached=$( echo "$MemCached*0.001" | bc ) # in MegaBytes
MemFree=$( echo "$MemFree*0.001" | bc ) # in MegaBytes
SwapTotal=$( echo "`grep -i "SwapTotal" /proc/meminfo | tr -d 'kB' | awk '{print $2;}'`")
SwapFree=$( echo "`grep -i "SwapFree" /proc/meminfo | tr -d 'kB' | awk '{print $2;}'`")
SwapUsed=$( echo "$SwapTotal - $SwapFree" | bc )
SwapUsed=$( echo "$SwapUsed*0.001" | bc ) # in MegaBytes
# First echo, returned variables
echo "$current_date $totalip_in $totalip_out $totalio_write $totalio_read"
# Middle echo, to cut the returned line
echo "Next"
# Second echo, to be written in the Log File
echo "$( date +%Y-%d-%m-%H:%M ) $cpu_used $difftotalip_in $difftotalip_out $MemUsed $MemCached $MemFree $SwapUsed $totalio_write $totalio_read"
Now, you can use an upper script that will call this script for multiple machines, and store information in directories (one per server) :
#!/bin/bash
#
# VARIABLES
#
#TallGeese
TallGeese_previous_date="0"
TallGeese_previous_totalip_in="0"
TallGeese_previous_totalip_out="0"
TallGeese_previous_totalio_write="0"
TallGeese_previous_totalio_read="0"
TallGeese_repertory_name="TallGeese"
TallGeese_login="spheniscus"
TallGeese_ip="tallgeese.fr"
#
# MAIN LOOP
#
while true
do
echo "Check"
# ssh the script
Out_buffer=$(ssh " $TallGeese_login@$TallGeese_ip" 'bash -s' < ssh_llcheck.bash $TallGeese_previous_date $TallGeese_previous_totalip_in \
$TallGeese_previous_totalip_out $TallGeese_previous_totalio_write $TallGeese_previous_totalio_read)
# store values for next check
TallGeese_previous_date=$( echo $Out_buffer | awk '{print $1;}')
TallGeese_previous_totalip_in=$( echo $Out_buffer | awk '{print $2;}')
TallGeese_previous_totalip_out=$( echo $Out_buffer | awk '{print $3;}')
TallGeese_previous_totalio_write=$( echo $Out_buffer | awk '{print $4;}')
TallGeese_previous_totalio_read=$( echo $Out_buffer | awk '{print $5;}')
# log values
echo $Out_buffer | awk 'match($0,"Next"){print substr($0,RSTART+5)}' >> "./$TallGeese_repertory_name/Log.dat"
# plot
cd "$TallGeese_repertory_name"
gnuplot plotscript.gmp
cd ..
echo "Check OK, Sleep"
sleep 10s
done
==== Ploting results ====
The script to plot values is plotscript.gmp :
set terminal png size 642,462 small
set key autotitle columnheader
set timefmt "%Y-%d-%m-%H:%M"
set xdata time
set format x "%H:%M"
set grid
set output 'CPU.png'
plot 'Log.dat' using 1:2 w l linewidth 3 lc rgb "#f7d301"
set output 'Net.png'
plot 'Log.dat' using 1:3 w l linewidth 3 lc rgb "#f7d301", '' using 1:4 w l linewidth 3 lc rgb "#1cac77"
set output 'Memory.png'
plot 'Log.dat' using 1:7 w l linewidth 3 lc rgb "#f7d301", '' using 1:6 w l linewidth 3 lc rgb "#1f75fe", '' using 1:5 w l linewidth 3 lc rgb "#ff2d2e"
set style fill transparent solid 0.5 noborder
quit
And I first prepared the file log.dat in the directory of the server, by adding at top :
Date %CPU NetTrafic_in NetTrafic_out Memory_used Memory_cached Memory_free Swap_used IO_write IO_read
(Important, do not forget to pass line after the first one.)
This gives you png like that, that you can incorporate into an HTML display based system :
{{ :software:shell_scripts:cpu.png?500 |}}{{ :software:shell_scripts:memory.png?500 |}} {{ :software:shell_scripts:net.png?500 |}}