Difference between revisions of "Shell"

From Wiki at Neela Nurseries
Jump to navigation Jump to search
m (Minor formatting.)
m (Bash scriptlets : comment the script)
 
(9 intermediate revisions by the same user not shown)
Line 4: Line 4:
  
 
This local page for Bash shell scripts and notes on shell scripting.
 
This local page for Bash shell scripts and notes on shell scripting.
 +
 +
== [[#top|^]] Shell Resources ==
 +
 +
*  https://tldp.org/guides.html . . . look for Mendel Cooper bash guide here.
 +
 +
*  https://www.shellcheck.net/ . . . run shell scripts online here for error checking purposes.
  
 
== [[#top|^]] Bash Built-in Variables ==
 
== [[#top|^]] Bash Built-in Variables ==
Line 43: Line 49:
 
<!-- odne komentar -->
 
<!-- odne komentar -->
  
−
// ***** ***** ***** ***** ***** ***** ***** ***** ***** ***** ***** *****
+
== [[#top|^]] Variables ==
  
−
== [[#top|^]] Variable assignment and multiple variable assignments ==
+
An interesting use of the shell with variable assignment and multiple variable assignments:
  
 
* https://www.baeldung.com/linux/bash-multiple-variable-assignment
 
* https://www.baeldung.com/linux/bash-multiple-variable-assignment
  
−
// ***** ***** ***** ***** ***** ***** ***** ***** ***** ***** ***** *****
+
Bash 'declare' built-in or keyword, makes intended variable type explicit, if not truly lasting during script run time:
 +
 
 +
*  https://unix.stackexchange.com/questions/510220/what-is-declare-in-bash
 +
 
 +
Comparing bash 'set' and 'declare' built-ins produces almost identical output, at least sometimes:
 +
 
 +
<pre>
 +
ted@localhost1:~$ declare >> 1.txt
 +
ted@localhost1:~$ set >> 2.txt
 +
ted@localhost1:~$ ls -l 1.txt 2.txt
 +
-rw-rw-r-- 1 ted ted 88439 Jan 15 14:01 1.txt
 +
-rw-rw-r-- 1 ted ted 88434 Jan 15 14:01 2.txt
 +
ted@localhost1:~$ diff 1.txt 2.txt
 +
126c126
 +
< _=--color=auto
 +
---
 +
> _=declare
 +
ted@localhost1:~$
 +
</pre>
 +
 
 +
=== [[#top|^]] shell arrays ===
 +
 
 +
Shell arrays can be indexed or associated (pairs of keys and values).  A good introduction to these things is here:
 +
 
 +
* https://www.gnu.org/software/bash/manual/html_node/Arrays.html
 +
 
 +
== [[#top|^]] Tests ==
 +
 
 +
* https://unix.stackexchange.com/questions/717758/grouping-of-bash-conditionals
 +
 
 +
<!--
 +
if [[ $exec_rc -eq 1 && (-z $keymap_actm || -n $modmap_actm) ]]; then . . .
 +
-->
  
 
== [[#top|^]] Examples ==
 
== [[#top|^]] Examples ==
Line 457: Line 495:
 
     echo "[WRN] Extra arguments are ignored."
 
     echo "[WRN] Extra arguments are ignored."
 
fi
 
fi
 +
</pre>
  
 +
== [[#top|^]] Bash scriptlets ==
 +
 +
The following scripts are especially short, sometimes just one command.  They could be implemented as shell aliases but are presented here as independent scripts.
 +
 +
<i>
 +
Script:  show most recently updated git branches
 +
</i>
 +
<pre>
 +
#!/bin/bash
 +
# How to use:  name this script something short, for example `gbl`, and put it in the shell's search path.
 +
# Where to use:  in a local git repository `gbl` shows the ten most recently modified git branchs.
 +
git branch --sort=committerdate | tail
 +
exit $?
 
</pre>
 
</pre>
  
Line 511: Line 563:
 
When the old-style backquote form of substitution is used, backslash retains its literal meaning except when followed by $, `, or \. The first backquote not preceded by a backslash terminates the command substitution. When using the $(command) form, all characters between the parentheses make up the command; none are treated specially.
 
When the old-style backquote form of substitution is used, backslash retains its literal meaning except when followed by $, `, or \. The first backquote not preceded by a backslash terminates the command substitution. When using the $(command) form, all characters between the parentheses make up the command; none are treated specially.
  
 +
-->
 +
 +
<!--
 +
2026-01-18 domingo
 +
-->
 +
 +
<!--
 +
*************************************************************************************************************************
 +
 +
#!/bin/bash
 +
 +
function script_help()
 +
{
 +
    echo "Help for append-tests.sh:"
 +
    echo
 +
    echo "-h --help              . . . this help message"
 +
    echo "-c --create-test-files . . . create small files for append tests"
 +
    echo "-f --file-list        . . . specify file from which to read list"
 +
    echo "                              files to process"
 +
    echo
 +
}
 +
 +
function create_test_files()
 +
{
 +
    for file in a.txt b.txt c.txt; do
 +
        cat <<'EOL' > $file
 +
aaaa
 +
bbbb
 +
cccc
 +
EOL
 +
    done
 +
}
 +
 +
function debug_message()
 +
{
 +
    if [[ "$2" != "--quiet" ]]; then
 +
        echo $1
 +
    fi
 +
}
 +
 +
# @note This function updates the variable $file_size.
 +
# TODO [ ] Amend function to return a filesize and let caller update any
 +
#  variables in need.
 +
 +
function parse_file_size()
 +
{
 +
    file_size=`ls -l $1 | cut -d " " -f 5`
 +
    debug_message "(f5) parse_file_size : $1 has size $file_size"
 +
}
 +
 +
# ----------------------------------------------------------------------
 +
# - remainder of block handling -
 +
# ----------------------------------------------------------------------
 +
 +
function write_partial_block()
 +
{
 +
    debug_message "(f4) write_partial_block : handling byte count of $fsize,"
 +
    debug_message "(f4) write_partial_block : \$idx holds $idx,"
 +
 +
    for i in `seq $idx`
 +
    do
 +
        if [ $MODE = "umode" ]; then
 +
            echo -n $CHAR_1 >> $DATA_FILE_1
 +
        else
 +
            echo -n '*' >> $DATA_FILE_1
 +
        fi
 +
    done
 +
}
 +
 +
# ----------------------------------------------------------------------
 +
# - block handling -
 +
# ----------------------------------------------------------------------
 +
 +
function handle_block()
 +
{
 +
    byte_count=$(seq $BS)
 +
    debug_message "(f3) handle_block : at byte index $idx, writing $BS bytes . . ." --quiet
 +
 +
    for i in $byte_count
 +
    do
 +
        if [ $MODE = "umode" ]; then
 +
            echo -n $CHAR_1 >> $DATA_FILE_1
 +
        else
 +
            echo -n '*' >> $DATA_FILE_1
 +
        fi
 +
    done
 +
}
 +
 +
function iterate_blocks()
 +
{
 +
    debug_message "(f2) iterate_blocks : at byte index $idx,"
 +
 +
    while [[ $idx -ge $BS ]]
 +
    do
 +
        handle_block
 +
        (( idx -= $BS ))
 +
        debug_message "(f2) iterate_blocks : remaining bytes index (count) now $idx" --quiet
 +
    done
 +
 +
    if [[ $idx -ge 0 && $idx -lt $BS ]]; then
 +
        debug_message "(f2) iterate_blocks : byte index now holds partial block size $idx"
 +
        write_partial_block $idx
 +
    fi
 +
}
 +
 +
function write_pattern()
 +
{
 +
# Write first byte:
 +
    if [ $MODE = "umode" ]; then
 +
        echo -n $CHAR_1 > $DATA_FILE_1
 +
    else
 +
        echo -n '*' > $DATA_FILE_1
 +
    fi
 +
 +
    # When a file is greater than our selected block size we interate over
 +
    # blocks of the file first:
 +
    if [ $file_size -gt $BS ]
 +
    then
 +
        echo "(f1) write_pattern : file larger than $BS bytes."
 +
        iterate_blocks
 +
    fi
 +
}
 +
 +
# ----------------------------------------------------------------------
 +
# - SECTION - akin to int main
 +
# ----------------------------------------------------------------------
 +
 +
DATA_FILE_1=1.dat
 +
DATA_FILE_2=2.dat
 +
# block size
 +
BS=1024
 +
 +
# Prefer ASCII values 42 and 85 . . .
 +
CHAR_1="U"
 +
CHAR_2="A"
 +
MODE="umode"
 +
 +
echo "Append script starting,"
 +
 +
if [ $# -gt 0 ]
 +
then
 +
    echo "in first argument detected filename: $1"
 +
else
 +
    echo "$0 expects a filename to work with!"
 +
    echo "exiting early . . ."
 +
    echo
 +
    exit -1
 +
fi
 +
 +
OPTS=$(getopt -o cf:h --long help,create-test-files,file-list: -n append.sh -- "$@")
 +
 +
if [ $? -ne 0 ]; then
 +
    echo "Failed to parse options" >&2
 +
    exit 1
 +
fi
 +
 +
# Reset the positional parameters to the parsed options
 +
eval set -- "$OPTS"
 +
 +
## Process the options
 +
while true; do
 +
  case "$1" in
 +
    -h | --help)
 +
      script_help
 +
      EXIT_AFTER_OPTION=1
 +
      shift
 +
      ;;
 +
    -c | --create-test-files)
 +
      create_test_files
 +
      shift
 +
      ;;
 +
    --)
 +
      # debug_message "got additional arguments '$@'"
 +
      shift
 +
      # debug_message "clearing options and args separator have '$@'"
 +
      break
 +
      ;;
 +
    *)
 +
      echo "$0 Failed to parse options, exiting early!"
 +
      exit 1
 +
      ;;
 +
  esac
 +
done
 +
 +
DATA_FILE_1=$1
 +
 +
# DEBUG BLOCK BLOCK
 +
file_size=`ls -l $DATA_FILE_1 | cut -d " " -f 5`
 +
debug_message "(0) main : file $DATA_FILE_1 has size $file_size"
 +
parse_file_size $DATA_FILE_1
 +
# DEBUG BLOCK END
 +
 +
# TODO [ ] account for files of size one byte.
 +
 +
# - STEP - check for file size greater than one byte:
 +
b=`expr $file_size \> 1`
 +
if [ $b ]
 +
then
 +
    fsize=$file_size
 +
    (( fsize -= 1 ))
 +
else
 +
    echo "file has size of zero or one."
 +
    return 0
 +
fi
 +
 +
idx=$file_size
 +
write_pattern
 +
 +
debug_message ""
 +
 +
debug_message "main : switching to asterisk mode,"
 +
MODE="asterisk"
 +
idx=$file_size
 +
debug_message "main : index idx assigned value $idx"
 +
 +
# DEBUG BLOCK BEGIN
 +
file_size=`ls -l $DATA_FILE_1 | cut -d " " -f 5`
 +
debug_message "(0) main : file $DATA_FILE_1 has size $file_size"
 +
parse_file_size $DATA_FILE_1
 +
# DEBUG BLOCK END
 +
 +
write_pattern
 +
 +
debug_message ""
 +
 +
echo "Test of file append done, returning $?"
 +
 +
exit $?
 +
***************************************************************************************************************************
 
-->
 
-->

Latest revision as of 16:28, 23 January 2026

Shell Scripting

^ OVERVIEW

This local page for Bash shell scripts and notes on shell scripting.

^ Shell Resources

^ Bash Built-in Variables

Some links to useful articles discussing `bash` built-in variables:

Bash's built-in variables with short names, as listed in Kernigan and Ritchie's "Unix Programming" book and at the first linked article above: 

$0
    The first element passed to the shell is the command name.
$n
    The nth argument passed on the command line. If n ≥ 10, then the syntax must be ${n}.
$*
    All the arguments on the command line. The values are separated by the first character in the shell variable IFS: (${1} … ${n}). See also: the IFS entry in Other Shell Variables.
$@
    All the arguments on the command line. The values are individually quoted: ("${1}" … "${n}").
$#
    The number of command-line arguments.
$?
    The exit value of the last executed command.
$_
    The last argument of the previous command.
$!
    The process ID of the most recent background process.


^ Getopt and Getopts

The shell page section to cover `getopt` and `getopts`, two libraries or facilities available to Bash shell scripts to process command line arguments.

An introduction to `getopt` at Labex dot io:


^ Variables

An interesting use of the shell with variable assignment and multiple variable assignments:

Bash 'declare' built-in or keyword, makes intended variable type explicit, if not truly lasting during script run time:

Comparing bash 'set' and 'declare' built-ins produces almost identical output, at least sometimes: 

ted@localhost1:~$ declare >> 1.txt
ted@localhost1:~$ set >> 2.txt
ted@localhost1:~$ ls -l 1.txt 2.txt
-rw-rw-r-- 1 ted ted 88439 Jan 15 14:01 1.txt
-rw-rw-r-- 1 ted ted 88434 Jan 15 14:01 2.txt
ted@localhost1:~$ diff 1.txt 2.txt
126c126
< _=--color=auto
---
> _=declare
ted@localhost1:~$

^ shell arrays

Shell arrays can be indexed or associated (pairs of keys and values). A good introduction to these things is here:

^ Tests


^ Examples

Some example scripts

^ rename badly named files

How to produce a usable filename which contains <ESC> characters, and to rename that badly named file: 

 $ ls -i
 9704871 $'\033\033'   9703029  CMakeLists.txt  10899590  dts          9708845  samples
 $ ls -q `find . -inum 9704871`
 './'$'\033\033'
 $ mv './'$'\033\033' betterfilename

^ Git diff-tree helper script

A helper script for calling `git diff-tree`, which can be used to determine at which project commit one or more files have changed: 

#!/bin/bash

CMD_GIT=/usr/bin/git

$CMD_GIT log --oneline | head | cut -d " " -f 1

echo "2024-01-31 git diff-tree helper script in progress . . ."
echo "got $# arguments, arg one single quoted is '$1'"

hashes=`$CMD_GIT log --oneline | head -n ${1} | cut -d " " -f 1`
echo "In $PWD found git commit hashes:"
echo " " $hashes

#i=1
#for hash in $hashes; do echo "("$i")" $hash; (( i++ )); done

#echo "Files changed between commit pairs youngest pairings to oldest:
#for hash in $hashes; do echo "Files changed in git commit:"; git diff-tree -r $hash; echo; done
for hash in $hashes
    do echo "Files changed in git commit:"
    git log -1 --oneline $hash
    git diff-tree -r $hash
    echo
done

echo "done"

exit 0

Script to test compilation of series of git commits: 

#!/bin/bash

CMD_GIT=/usr/bin/git

VETTED_COMMITS_FILE="branch-commit-vetting-report.txt"
current_branch="NOT_SET"
hashes="NOT_SET"

function usage()
{
    echo "How to use vet-branch.sh:":
    echo
    echo "  $ vet-branches.sh n"
    echo
    echo "Argument 'n' is a number of commits to test in current git branch."
    echo
}

#
# script starting point, akin to int main
#

echo "$0: starting"
if [ $# -lt 1 ] 
then
    echo "$0 called with too few arguments!"
    usage
    exit -1  
fi

current_branch=`$CMD_GIT branch --show-current`
hashes=`$CMD_GIT log --oneline | head -n $1 | cut -d " " -f 1`

echo "Vetting $1 commits of branch ${current_branch}."
echo "Asked to test commits: $hashes"
echo

date >> $VETTED_COMMITS_FILE
echo "Testing compilation of $1 commits start at tip of '$current_branch':" >> $VETTED_COMMITS_FILE

for hash in $hashes
    do  
    $CMD_GIT checkout --quiet $hash
    echo
    echo "At commit $hash, vetting script in progress"
    echo "---------------------------------------------"
    ./scripts/docker-helper.sh build_both_debug
    echo "$hash build result: $?" >> $VETTED_COMMITS_FILE
done

echo
echo "Restoring git checkout to starting branch:"
$CMD_GIT checkout $current_branch
echo "$0: done."

exit #?

^ shell logging helper

Bash and related shell programming definitely have their limitations, with numeric and mathematical handling, name spaces and more. Despite these shortfalls we still want to debug shell programs. Noted here is an idea to collect some logging shell functions in a script which can be sourced by other shell scripts.

Code excerpt, not complete: 

#!/bin/bash

# @brief Bash script fragment, collection of functions to aid with shell script
#  debugging.
#
# @note Script adopts the notion of log levels as implemented in Zephyr RTOS.
#   From most critical to most verbose these levels are:
#
#   Error, Warning, Info, Debug
#
#  ERR
#  WRN
#  INF
#  DBG

function set_script_log_level()
{

}

# @brief This function expects:
#
# $1 . . . message to print
# $2 . . . log level used to filter printing

function diag()
{

}


^ aliases fragment

function set_custom_shell_aliases()
{
## - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
## Note:  from recent experimentation and reading, shell alias setting
##  does not lend itself to being sourced in a script other than
##  .bashrc, at least in the case of bash shell environment.  For this
##  reason restoring custom shell aliases to dot-bashrc-amendments.sh
##  file - TMH
## - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

    echo "setting custom shell aliases . . ."

    alias gw='mv ${HOME}/.ssh/config ${HOME}/.ssh/archive; ln -s ${HOME}/.ssh/config--work ${HOME}/.ssh/config; echo "switching ssh auth to github work account"'
    alias gp='mv ${HOME}/.ssh/config ${HOME}/.ssh/archive; ln -s ${HOME}/.ssh/config--personal ${HOME}/.ssh/config; echo "switching ssh auth to github personal account"'
# for safety:
    alias rm='rm -i'

    alias xscr='xscreensaver-command -lock'
 
# 2024-01-04 THU
    alias vu='ps -u ted x | grep " vi"'

# 2024-12-16 MON - list FTDI like devices
    alias lsf='ls -l /dev/ttyUSB*; ls -l /dev/ttyACM*'

# 2025-02-05 WED - git log --all --graph --oneline
    alias gla='git log --all --graph --oneline'

# 2025-10-26 SUN
    alias xscr='xscreensaver-command -lock'
    alias lgrep='grep -nr --exclude-dir=build $@'

    echo ". . . done."
}

^ to source versus to call

A script fragment which acts as an entry point of a script and treats the absence of all arguments as sign that the script is being sourced, rather than called: 

SCRIPT_NAME=`echo $0 | sed -e 's,^.*/,,'`

SCRIPT_NAME_WHEN_SOURCED="check-path-existence.sh"

EXPECTED_ARG_COUNT=3

if [ $# -eq $EXPECTED_ARG_COUNT ]; then
    SCRIPT_STATUS=0
elif [ $# -eq 0 ]; then
    echo "[INF] $SCRIPT_NAME_WHEN_SOURCED received no arguments, assuming we "
    echo "       are being sourced and not called."
    echo "[INF] (this means we won't run most of this script's code.)"
    SCRIPT_STATUS=300
elif [[ $# -le $EXPECTED_ARG_COUNT && $# -ne 0 ]]; then
    echo "[ERR] $SCRIPT_NAME requires $EXPECTED_ARG_COUNT arguments, got $#."
    SCRIPT_STATUS=400
    help
elif [[ $# -gt $EXPECTED_ARG_COUNT && $# -ne 0 ]]; then
    echo "[WRN] $SCRIPT_NAME expects $EXPECTED_ARG_COUNT arguments, got $#."
    echo "[WRN] Extra arguments are ignored."
fi

^ Bash scriptlets

The following scripts are especially short, sometimes just one command. They could be implemented as shell aliases but are presented here as independent scripts.

Script: show most recently updated git branches 

#!/bin/bash
# How to use:  name this script something short, for example `gbl`, and put it in the shell's search path.
# Where to use:  in a local git repository `gbl` shows the ten most recently modified git branchs.
git branch --sort=committerdate | tail
exit $?

^ Bash helper functions

Common helper functions in bash scripts: 

function help()
{
    # echo script usage
}

function exit_early()
{
    # echo script name, echo status of last command
}

function set_local_vars()
{
    # set variables particular to given script, but note they're global
    # (good to rename this function.)
}

Some shell scripts to provide handy localhost commands, for embedded work on a Linux host:

  1. !/bin/bash

grep -nr --exclude-dir=.git --exclude-dir=build --exclude-dir=docker_cache --exclude-dir=twister-out* $1 | grep $1

  1. exit $?

^ To Research

[ ] Look up `git ls-files` and its options.

[ ] Review `xargs` called with the dash zero option.

Interesting header file from Zephyr RTOS 3.4.0:

`zephyr/include/zephyr/toolchain/xcc_missing_defs.h`

Shell command substitution backticks versus braces . . .



Retrieved from "https://wiki.neelanurseries.com/index.php?title=Shell&oldid=8771"