Difference between revisions of "Shell"

From Wiki at Neela Nurseries
Jump to navigation Jump to search
m (Update extended git-tree helper script.)
(Shell command substitution backticks versus braces)
 
(3 intermediate revisions by the same user not shown)
Line 32: Line 32:
  
 
<!-- odne komentar -->
 
<!-- odne komentar -->
 +
 +
== [[#top|^]] Getopt and Getopts ==
 +
 +
The shell page section to cover `getopt` and `getopts`, two libraries or facilities available to Bash shell scripts to process command line arguments.
 +
 +
An introduction to `getopt` at Labex dot io:
 +
 +
*  https://labex.io/tutorials/shell-bash-getopt-391993
 +
 +
<!-- odne komentar -->
 +
 
== [[#top|^]] Rename Badly Named Files ==
 
== [[#top|^]] Rename Badly Named Files ==
  
Line 140: Line 151:
 
</pre>
 
</pre>
  
<!--
+
== [[#top|^]] Shell Logging Helper ==
************************************************************************************************************
+
 
-->
+
Bash and related shell programming definitely have their limitations, with numeric and mathematical handling, name spaces and more.  Despite these shortfalls we still want to debug shell programs.  Noted here is an idea to collect some logging shell functions in a script which can be sourced by other shell scripts.
 +
 
 +
Code excerpt, not complete:
 +
 
 +
<pre>
 +
#!/bin/bash
 +
 
 +
# @brief Bash script fragment, collection of functions to aid with shell script
 +
#  debugging.
 +
#
 +
# @note Script adopts the notion of log levels as implemented in Zephyr RTOS.
 +
#  From most critical to most verbose these levels are:
 +
#
 +
#  Error, Warning, Info, Debug
 +
#
 +
#  ERR
 +
#  WRN
 +
#  INF
 +
#  DBG
 +
 
 +
function set_script_log_level()
 +
{
 +
 
 +
}
 +
 
 +
# @brief This function expects:
 +
#
 +
# $1 . . . message to print
 +
# $2 . . . log level used to filter printing
 +
 
 +
function diag()
 +
{
  
<!--
+
}
 +
</pre>
 +
<!-- ************************************************************************************************************ --> <!--
 
#!/bin/bash
 
#!/bin/bash
  
 +
# TODO [ ] Add sanity checking and modestly vigorous search for system
 +
#  git and file utilities:
 
CMD_GIT=/usr/bin/git
 
CMD_GIT=/usr/bin/git
 +
CMD_TREE=/usr/bin/tree
  
 
# Changed file filter, may differ on various work stations:
 
# Changed file filter, may differ on various work stations:
Line 157: Line 204:
 
     echo
 
     echo
 
     echo "  $ $0 <count> [-u|-d]"
 
     echo "  $ $0 <count> [-u|-d]"
     echo "  $ $0 <count> [--show-dirs --make-dirs] <dest_dit> # <- both long options required here"
+
     echo "  $ $0 <count> [--show-dirs --make-dirs] <dest_dir> # <- both long options required here"
 
     echo "  $ $0 <count> -c <dest_dir>"
 
     echo "  $ $0 <count> -c <dest_dir>"
 
     echo
 
     echo
 
     echo "count . . . count of commits to review from tip of git branch"
 
     echo "count . . . count of commits to review from tip of git branch"
 +
    echo
 
     echo "-u    . . . optionally summarize file changes to list of unique files changed"
 
     echo "-u    . . . optionally summarize file changes to list of unique files changed"
 +
    echo
 +
    echo "-d    . . . output essentially like -u.  To be updated."
 +
    echo
 +
    echo "Examples:"
 +
    echo
 +
    echo "  $ helper-git-diff-tree-extended.sh 5"
 +
    echo
 +
    echo "...to report per commit a list of files changed and some git details"
 +
    echo "including git's internal file hashes.  This helpful in highlighting"
 +
    echo "files deleted and newly created files, whose commit hashes before and"
 +
    echo "after the given commit respectively end as all zeroes (file deletion)"
 +
    echo "and begin as all zeroes (file creation)."
 +
    echo
 +
    echo "As a side note, file renaming in git involves a file deleation"
 +
    echo "operation followed immediately by a file creation operation."
 +
    echo
 +
    echo "  $ helper-git-diff-tree-extended.sh 5 -u"
 +
    echo
 +
    echo "...to show a list of relative paths ending in names of files changed"
 +
    echo "anywhere in the commit series of the latest 'count' commits of the"
 +
    echo "local repo at its current commit."
 +
    echo
 
}
 
}
  
Line 269: Line 339:
 
             (( i++ ))
 
             (( i++ ))
 
         else
 
         else
             cp -pv $file $scratch_dir//$file
+
             cp -pv $file $scratch_dir/$file
 
         fi
 
         fi
 
     done
 
     done
 +
 +
    echo
 +
    echo "Summary of files changed in commit series, and their final"
 +
    echo "relative locations:"
 +
    echo
 +
    $CMD_TREE $scratch_dir
 
}
 
}
  
Line 290: Line 366:
 
elif [ ${2} == "-d" ] ; then
 
elif [ ${2} == "-d" ] ; then
 
     show_dirs_per_changed_file ${1}
 
     show_dirs_per_changed_file ${1}
elif [[ ${3} == "--show-dirs"  &&  ${4} == "--make-dirs" ]] ; then
+
elif [[ ${2} == "--show-dirs"  &&  ${3} == "--make-dirs" ]] ; then
     recreate_directory_paths ${1} ${2}
+
     recreate_directory_paths ${1} ${4}
 
elif [ ${2} == "-c" ] ; then
 
elif [ ${2} == "-c" ] ; then
 
     copy_relpaths_and_file ${1} ${3}
 
     copy_relpaths_and_file ${1} ${3}
Line 300: Line 376:
 
exit $?
 
exit $?
 
-->
 
-->
 
+
<!-- ************************************************************************************************************ -->
<!--
 
************************************************************************************************************
 
************************************************************************************************************
 
-->
 
 
 
<!--
 
 
 
-->
 
 
 
 
<!-- odne komentar -->
 
<!-- odne komentar -->
 
 
== [[#top|^]] To Research ==
 
== [[#top|^]] To Research ==
  
Line 321: Line 387:
  
 
`zephyr/include/zephyr/toolchain/xcc_missing_defs.h`
 
`zephyr/include/zephyr/toolchain/xcc_missing_defs.h`
 +
 +
Shell command substitution backticks versus braces . . .
 +
 +
*  https://stackoverflow.com/questions/22709371/backticks-vs-braces-in-bash
  
 
<!-- odne komentar -->
 
<!-- odne komentar -->
  
 
<!--
 
<!--
 +
https://stackoverflow.com/questions/22709371/backticks-vs-braces-in-bash
  
#!/bin/bash
+
# later on in the man page:
 
+
When the old-style backquote form of substitution is used, backslash retains its literal meaning except when followed by $, `, or \. The first backquote not preceded by a backslash terminates the command substitution. When using the $(command) form, all characters between the parentheses make up the command; none are treated specially.
export AWS_PROFILE=prod
 
echo "starting, exporting AWS_PROFILE=$AWS_PROFILE . . ."
 
 
 
# aws s3api put-object --bucket edge-fota-images --key img-EPN687-MMMM.bin --body /home/ted/projects-sandbox/workspace-for-nexus/nexus-gateway-firmware/build/img-MMMM.bin
 
 
 
fname1=op-and-location--image-MMMM--pr-316.json
 
fname2=op-and-location--image-SSSS--pr-316.json
 
fname3=img-EPN687-MMMM-PR-310.bin
 
fname4=img-EPN687-SSSS-PR-310.bin
 
 
 
filenames="$fname1 $fname2 $fname3 $fname4"
 
 
 
i=1
 
 
 
for filename in $filenames; do
 
    echo "($i) to S3 uploading $filename . . ."
 
    (( i++ ))
 
    aws s3api put-object --bucket edge-fota-images \
 
    --key $filename \
 
    --body $filename
 
done
 
 
 
echo "done."
 
 
 
exit $?
 
  
 
-->
 
-->

Latest revision as of 19:05, 3 October 2025

Shell Scripting

^ OVERVIEW

This local page for Bash shell scripts and notes on shell scripting.

^ Bash Built-in Variables

Some links to useful articles discussing `bash` built-in variables:

Bash's built-in variables with short names, as listed in Kernigan and Ritchie's "Unix Programming" book and at the first linked article above:

$0
    The first element passed to the shell is the command name.
$n
    The nth argument passed on the command line. If n ≥ 10, then the syntax must be ${n}.
$*
    All the arguments on the command line. The values are separated by the first character in the shell variable IFS: (${1} … ${n}). See also: the IFS entry in Other Shell Variables.
$@
    All the arguments on the command line. The values are individually quoted: ("${1}" … "${n}").
$#
    The number of command-line arguments.
$?
    The exit value of the last executed command.
$_
    The last argument of the previous command.
$!
    The process ID of the most recent background process.


^ Getopt and Getopts

The shell page section to cover `getopt` and `getopts`, two libraries or facilities available to Bash shell scripts to process command line arguments.

An introduction to `getopt` at Labex dot io:


^ Rename Badly Named Files

How to produce a usable filename which contains <ESC> characters, and to rename that badly named file:

 $ ls -i
 9704871 $'\033\033'   9703029  CMakeLists.txt  10899590  dts          9708845  samples
 $ ls -q `find . -inum 9704871`
 './'$'\033\033'
 $ mv './'$'\033\033' betterfilename

^ Git Diff-tree Helper Script

A helper script for calling `git diff-tree`, which can be used to determine at which project commit one or more files have changed:

#!/bin/bash

CMD_GIT=/usr/bin/git

$CMD_GIT log --oneline | head | cut -d " " -f 1

echo "2024-01-31 git diff-tree helper script in progress . . ."
echo "got $# arguments, arg one single quoted is '$1'"

hashes=`$CMD_GIT log --oneline | head -n ${1} | cut -d " " -f 1`
echo "In $PWD found git commit hashes:"
echo " " $hashes

#i=1
#for hash in $hashes; do echo "("$i")" $hash; (( i++ )); done

#echo "Files changed between commit pairs youngest pairings to oldest:
#for hash in $hashes; do echo "Files changed in git commit:"; git diff-tree -r $hash; echo; done
for hash in $hashes
    do echo "Files changed in git commit:"
    git log -1 --oneline $hash
    git diff-tree -r $hash
    echo
done

echo "done"

exit 0

Script to test compilation of series of git commits:

#!/bin/bash

CMD_GIT=/usr/bin/git

VETTED_COMMITS_FILE="branch-commit-vetting-report.txt"
current_branch="NOT_SET"
hashes="NOT_SET"

function usage()
{
    echo "How to use vet-branch.sh:":
    echo
    echo "  $ vet-branches.sh n"
    echo
    echo "Argument 'n' is a number of commits to test in current git branch."
    echo
}

#
# script starting point, akin to int main
#

echo "$0: starting"
if [ $# -lt 1 ] 
then
    echo "$0 called with too few arguments!"
    usage
    exit -1  
fi

current_branch=`$CMD_GIT branch --show-current`
hashes=`$CMD_GIT log --oneline | head -n $1 | cut -d " " -f 1`

echo "Vetting $1 commits of branch ${current_branch}."
echo "Asked to test commits: $hashes"
echo

date >> $VETTED_COMMITS_FILE
echo "Testing compilation of $1 commits start at tip of '$current_branch':" >> $VETTED_COMMITS_FILE

for hash in $hashes
    do  
    $CMD_GIT checkout --quiet $hash
    echo
    echo "At commit $hash, vetting script in progress"
    echo "---------------------------------------------"
    ./scripts/docker-helper.sh build_both_debug
    echo "$hash build result: $?" >> $VETTED_COMMITS_FILE
done

echo
echo "Restoring git checkout to starting branch:"
$CMD_GIT checkout $current_branch
echo "$0: done."

exit #?

^ Shell Logging Helper

Bash and related shell programming definitely have their limitations, with numeric and mathematical handling, name spaces and more. Despite these shortfalls we still want to debug shell programs. Noted here is an idea to collect some logging shell functions in a script which can be sourced by other shell scripts.

Code excerpt, not complete:

#!/bin/bash

# @brief Bash script fragment, collection of functions to aid with shell script
#  debugging.
#
# @note Script adopts the notion of log levels as implemented in Zephyr RTOS.
#   From most critical to most verbose these levels are:
#
#   Error, Warning, Info, Debug
#
#  ERR
#  WRN
#  INF
#  DBG

function set_script_log_level()
{

}

# @brief This function expects:
#
# $1 . . . message to print
# $2 . . . log level used to filter printing

function diag()
{

}

^ To Research

[ ] Look up `git ls-files` and its options.

[ ] Review `xargs` called with the dash zero option.

Interesting header file from Zephyr RTOS 3.4.0:

`zephyr/include/zephyr/toolchain/xcc_missing_defs.h`

Shell command substitution backticks versus braces . . .