Difference between revisions of "User:Ted"

From Wiki at Neela Nurseries
Jump to: navigation, search
m (→‎↑ ↓ Electronics)
m (→‎^ C programming notes)
Line 775: Line 775:
  
 
Click here -> [[c_programming_notes|C programming notes]] <- for separate wiki article.
 
Click here -> [[c_programming_notes|C programming notes]] <- for separate wiki article.
 +
 +
Typedef and type defining function pointers:
 +
 +
<ul>
 +
*  2019-06-11 https://www.learncpp.com/cpp-tutorial/78-function-pointers/
 +
</u>
  
  

Revision as of 00:46, 12 June 2019

-- 2017-07-21 शुक्रवार --
User Ted's wiki page at Neela Nurseries


OVERVIEW

This wiki document a starting point of Ted's notes on Linux use and configuration, Open Source Software and web development. Recent summer 2017 efforts of Ted's are focused on several high-level programming language and web development frameworks, which can be studied and used separately but are often glued together to achieve meaningful tasks and end-user tools and interfaces. This personal page of Ted's notes on Neela Nurseries wiki not yet well organized, but here as a quick stash point for holding useful references, and a starting point for more complete and formal documentation.


^ Quick Links



Contents


- 2017-10-05 - Ted noting here that would be good to have a technical glossary to make easily accessible all the technical terms which arise in daily work and exploration of technologies . . .


- 2017-10-19 - TO-DO list: 

  *  [✓] complete LFS 8.1 exercise
  *  [ ] configure xserver to run stand-along without window manager (seems to be working on 1604 spare LTS host . . .)
  *  [ ] set up two mysql servers on one host
  *  [ ] write "best photo contribution" practices page for ASI
  *  [ ] code "re-use or reference text block" module for MediaWiki  <-- this needs further explanation - TMH 
  *  [ ] in local Neela Nurseries PHP code base, amend hybrid building of nav menus to include highlighting parent page of given viewed page
  *  [ ] in OpenCart 2.x find way to customize top-of-page information and links
  *  [ ] in OpenCart 2.x find way to format product lists as compact lists, like Gmail inbox or Windows 'list view' of files


↑ ↓ GNU Make and GCC Tools


2018-07-18 NOTE: this section to be moved to separate GCC and Make article . . . - TMH


Using macros with gcc:

Code libraries and shared objects, expressing paths to them:

gcc.gnu.org - 3.15 Specifying subprocesses and switches passed to them

gcc how to show macros and pre-processor defines:



↑ ↓ Mediawiki Issues and Notes

- 2018-01-22 - While editing sidebar of Mediawiki instance at Neela, Ted noticing a difference in Mediawiki's treatment of hyperlinks in the sidebar. Links to external sites don't appear, and links to internal pages require single square brace enclosure to appear with alternate text as a link label . . . 

<!--
** [https://buildroot.org/downloads/manual/manual.html Buildroot manual]
<a href="https://buildroot.org/downloads/manual/manual.html">Buildroot manual</a>

- 2018-01-22 MON -
** https://wiki.neelanurseries.com/index.php/Nn_embedded_linux_notes#.E2.86.91_.E2.86.93_Buildroot|Buildroot notes . . . only 'notes' shows as non-link text - TMH
-->


^ File uploads

- 2018-06-29 FRI - Issue adding .txt type files to whitelist of Mediawiki installation uploadable files: 

Action failed

Could not open lock file for "mwstore://local-backend/local-public/e/ec/Asi-officers-list-2018.txt".

Return to Main Page.

200px|thumb|left|png type image to test file uploads of allowed image type . . .
media:arrow-up-prelight.png


Following link may have helpful information:

- 2018-07-25 WED -


^ Mediawiki layout topics like multi column layout

- 2018-11-30 FRI -



↑ ↓ Web Site Building Blocks - Summer 2017 to 2018 Work


↑ ↓ Open Source License Types



↑ ↓ Linux Packages (Separate Article)

Linux software package notes a separate wiki document on Neela wiki. - TMH


- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

↑ ↓ Bash Shell Scripting

Bash (Bourne shell) Quick Reference: 

Shell Built-in Variables:

   $0    Name of given shell script itself
   $1    Value of first command line parameter (similarly $2, $3, ...)
   $#    In given shell script the number of command line parameters received
   $*    All of the command line parameters
   $-    Options given to the shell
   $?    Return (or holds) the exit status of the last command
   $$    Process id of given script (really the id of the shell running the script)


Shell test constructs:

SYNTAX      NAME OF SHELL TEST                                NOTES

[           built-in test                                     considers its arguments as comparison expressions or file tests

[[...]]     in Bash => 2.02 the extended test command

((...))     arithmetic tests                                  example:  (( 0 && 1 ))  # Logical AND

References:


Shell scripting and use of built-in shell commands is a practical knowledge to employ, when working in Unix-like environments. One common task is to find all the instances of a given file or program. The locate command can perform this kind of search, but it's results don't show whether the file instances differ. To check at the rough level of file size, we can use a one-line shell script technique involving shell piping, to "long list" the results of the locate command, like this . . . 

  $ for file in `locate tavrasm | grep 'asm$'`; do ls -l ${file}; done

Hmm strange, the above command calls `grep` with a pattern that ends in the shell end-of-line anchoring character $, and appears to filter for results of `locate` which end in 'asm'. But on 2017-09-12 needed to add a "one or more wildcard" character pattern to limit search results for instances of `locate` results ending in 'qemu': 

    $ locate qemu | grep '.*qemu$'

Why the apparent difference in command invocation? Need to test . . . - TMH


Command to break out first ten $PATH environment variable to individual paths each shown on a line of the shell: 

    $ for path in `echo $PATH | cut -d':' -f1-10 --output-delimiter=' '`; do echo $path; done

2018-04-23 - Monday, command to obtain summary of directories containing 1GB or more data: 

    $ du -h . | grep [0-9]G



↑ ↓ edit point - shell variable quoting

- 2017-10-19 THU -

Running into issues when need to expand shell variable in single quotes pair . . .

Stack Exchange, Unix forum post 178411
TLPD Advanced Bash Scripting Guide, chapter 4.2 example 4-3 double quotes preserve white space


WIKI WISH LIST - While adding to wiki, Ted noting that would be useful to have relative-depth wiki section markers. The standard wiki section markers '== ==', '=== ===' and similar have fixed depth. That is, '== ==' is always a top-level or first-level section in a wiki document, '=== ===' is always a sub-subsection of wiki documents, and '==== ====' a sub-sub-section. When section gets re-factored into its own article, would be handy to have those section markers rise up to levels of top-most and successive section markers. - TMH



↑ ↓ example shell script - back up several databases

 
Here is a simple shell script to call mysqldump utility and back up several databases. Couple of things Ted wants to add to this script include script variables to hold back-up filename prefix and infix patterns, and an option to compress the MYSQL dump files . . .

Figure x - shell script to back-up multiple MYSQL databases 


#!/bin/bash


DATABASE_LIST="information_schema mysql drupal_8p0 phpmyadmin wiki_database"

user="root"
pass_phrase_for_mysql="mysql_user_password"
options_extra="--skip-lock-tables"

response="n"

mode_interactive="n"



echo "shell script starting,"



for database in ${DATABASE_LIST};
do echo "backing up alta-ubuntu database $database . . .";

#    command="mysqldump --databases $database -u$user -p$pass_phrase_for_mysql $options_extra >> au-database-back-up--${database}.sql"
    command="mysqldump --databases $database -u$user -p$pass_phrase_for_mysql $options_extra"
    redirect="au-database-back-up--${database}.sql"

    if [ $mode_interactive = 'y' ]; then

        echo "build command '$command' and database back-up filename'$redirect',"
        echo "full command will be '$command' > '$redirect',"
        echo "trying running this command and redirect? [y/N/q]  yes, no, 'q' to quit"

        read response
        if [ $response = 'Y' -o $response = 'y' ]; then
            ${command} > $redirect
        elif [ $response = 'N' -o $response = 'n' ]; then
            echo "skipping present command . . ."
        elif [ $response = 'Q' -o $response = 'q' ]; then
            echo "stopping script '$0' and exiting."
            break
        fi
        echo

    else

        echo "$0:  backing up database '$database' to file '$redirect' . . ."
        ${command} > $redirect
        ls -l $redirect

    fi

done


echo "done."

exit 0

↑ ↓ show kernel version script

Following script 'showkversion' can be used in a Linux kernel source tree to determine the kernel's version number, based on the number elements defined in the kernel's top level makefile.

Figure x - simple bash script to show kernel version in root of kernel sources tree 

#!/bin/bash

## - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
## Started 2017-12-15 FRI - script to parse and show Linux kernel
##  version from top-level makefile in set of kernel sources.  Patterns
##  to `grep` chosen based on kernel version identifiers in first
##  three lines of typical kernel top-level makefile.  Example:
##
##  $ head -n 6 Makefile
##  VERSION = 4
##  PATCHLEVEL = 9
##  SUBLEVEL = 66
##  EXTRAVERSION =
##  NAME = Roaring Lionus
##
##
## - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -


script_name="showkversion"


if [ ]; then
    head -n 5 Makefile | grep ^VERSION | cut -d " " -f 3
    head -n 5 Makefile | grep ^PATCH | cut -d " " -f 3
    head -n 5 Makefile | grep ^SUBLEVEL | cut -d " " -f 3
fi

MAJOR=$(head -n 5 Makefile | grep ^VERSION | cut -d " " -f 3)
MINOR1=$(head -n 5 Makefile | grep ^PATCH | cut -d " " -f 3)
MINOR2=$(head -n 5 Makefile | grep ^SUBLEVEL | cut -d " " -f 3)

KERNEL_REVISION="${MAJOR}.${MINOR1}.${MINOR2}"

echo
echo "Kernel version in present kernel sources tree, per makefile, is ${KERNEL_REVISION}"
echo


exit 0



↑ ↓ Other useful shell tutorial references



↑ ↓ Regex tutorials



↑   ↓ `sed` stream editor

Unix `sed` stream editor, examples and references:

<Example 1 - rename multiple files where a substring need be changed:</i> 

   $ for file in `ls ./2109*.*`; do mv $file `ls -1 $file | sed s/2109/2019/` ; done

References on `sed`:


↑   ↓ Other Computer Programming Languages

↑ ↓ Python Scripting Language



↑ ↓ Expect Utility by Don Libes

Installing 'expect' on Ubuntu 16.04.x LTS host, typical dependencies: 

ted@localhost:~/projects/shell/expect-work$ sudo apt install expect

Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following additional packages will be installed:
  libfontenc1 libxaw7 libxmu6 libxmuu1 libxtst6 libxv1 libxxf86dga1 tcl-expect tcl8.6 tk8.6 x11-utils xbitmaps xterm
Suggested packages:
  tcl-tclreadline mesa-utils xfonts-cyrillic
The following NEW packages will be installed:
  expect libfontenc1 libxaw7 libxmu6 libxmuu1 libxtst6 libxv1 libxxf86dga1 tcl-expect tcl8.6 tk8.6 x11-utils xbitmaps xterm
0 upgraded, 14 newly installed, 0 to remove and 5 not upgraded.
Need to get 1,373 kB of archives.
After this operation, 4,110 kB of additional disk space will be used.
Do you want to continue? [Y/n] y
  .
  .
  .


Some 'expect' utility references:

Good short examples covering range of tasks here at Robert Elder blog page, noting md5sum expect script example . . .

How to capture specific results during an expect script's execution:


- 2018-01-30 - Tuesday, picking up from last week researching how to redirect expect output and its spawned processes' output with filtering to log file. The first reference here at Stackoverflow repeats the useful lindex syntax. There is also mention of the expect manual and a syntax $::env(action) which can be expressed in an expect script to access an environment variable that is part of the controlling shell . . .

How to use expect's log_file key word in an expect script:

From the manual page for expect version 5.45: 

           log_file [args] [[-a] file]
             If a filename is provided, log_file will record a transcript of the session (beginning at that point) in
             the file.  log_file will stop recording if no argument is given.  Any previous log file is closed.

             Instead  of  a  filename,  a Tcl file identifier may be provided by using the -open or -leaveopen flags.
             This is similar to the spawn command.  (See spawn for more info.)

             The -a flag forces output to be logged that was suppressed by the log_user command.

             By default, the log_file command appends to old files rather than truncating them, for  the  convenience
             of  being  able  to  turn  logging off and on multiple times in one session.  To truncate files, use the
             -noappend flag.

             The -info flag causes log_file to return a description of the most recent non-info arguments given.


Also from the expect manual page, Expect Hints section: 

    EXPECT HINTS
       There are a couple of things about Expect that may be non-intuitive.  This section attempts to address some of
       these things with a couple of suggestions.

       A common expect problem is how to recognize shell prompts.  Since these are customized differently by  differ-
       ently  people and different shells, portably automating rlogin can be difficult without knowing the prompt.  A
       reasonable convention is to have users store a regular expression describing their prompt (in particular,  the
       end  of it) in the environment variable EXPECT_PROMPT.  Code like the following can be used.  If EXPECT_PROMPT
       doesn't exist, the code still has a good chance of functioning correctly.

           set prompt "(%|#|\\$) $"          ;# default prompt
           catch {set prompt $env(EXPECT_PROMPT)}

           expect -re $prompt

       I encourage you to write expect patterns that include the end of whatever you expect to see.  This avoids  the
       possibility  of  answering a question before seeing the entire thing.  In addition . . .


The expect program has some built-in string buffers, and one of these may hold the latest characters received from a spawned process. Here is a man page excerpt which shows use of one of these expect buffers: 

           disconnect
             disconnects a forked process from the terminal.  It continues running in the background.  The process is
             given its own process group (if possible).  Standard I/O is redirected to /dev/null.

             The following fragment uses disconnect to continue running the script in the background.

                 if {[fork]!=0} exit
                 disconnect
                 . . .

             The following script reads a password, and then runs a program every hour that demands a  password  each
             time  it is run.  The script supplies the password so that you only have to type it once.  (See the stty
             command which demonstrates how to turn off password echoing.)

                 send_user "password?\ "
                 expect_user -re "(.*)\n"
                 for {} 1 {} {
                     if {[fork]!=0} {sleep 3600;continue}
                     disconnect
                     spawn priv_prog
                     expect Password:
                     send "$expect_out(1,string)\r"
                     . . .
                     exit
                 }


Further looking for expect's built-in string buffers and variables . . .

Pattern matching syntax in Don Libe's expect . . .


From another source, Tcl's string match command: 

if [string match "f*b*" "foobar"] {
   puts "match"
else {
   puts "no match"
}



↑ ↓ edit point - Tool command language Tcl 

"Practical programming
in Tcl and Tk"
ISBN 0-13-038560-3


In using Don Libes' expect utility or program, looks like it will be important and helpful to understand Tool Command Language Tcl to at least some fluency. Here appears to be a good starting point tutorial for Tcl:


Tcl puts built-in command, channels and code to open log file, 

Append a log message to a file:

set chan [open my.log a]
set timestamp [clock format [clock seconds]]
puts $chan "$timestamp - Hello, World!"
close $chan


Tcl regular expressions:


Tcl reserved words: 

after		append		array		auto_execok
auto_import	auto_load	auto_load_index	auto_qualify
binary		Bgerror		break		catch
cd		Clock		close		concat
continue	Dde		default		else
elseif		Encoding	eof		error
eval		Exec		exit		expr
fblocked	Fconfigure	fcopy		file
fileevent	Flush		for		foreach
format		Gets		glob		global
history		If		info		interp
join		Lappend		lindex		linsert
list		Llength		load		lrange
lreplace	Lsearch		lsort		namespace
open		Package		pid		pkg_mkIndex
proc		Puts		pwd		read
regexp		Regsub		rename		resource
return		Scan		seek		set
socket		Source		split		string
subst		Switch		tclLog		tell
time		Trace		unknown		unset
update		Uplevel		upvar		variable
vwait		While



↑ ↓ edit point - Tcl pattern matching and anchoring, symbols ^ and $ wrt data not lines

From the Tcl manual page on-line here is a clue about pattern matching in Tcl scripts, which may explain why we are struggling to obtain matched substrings from sequences of bytes from expect-spawned processes . . . 

        Note that in many editors, the ^ and $ match the beginning and end of lines respectively. However, because expect is not line oriented, these characters match the beginning and end of the data (as opposed to lines) currently in the expect matching buffer. (Also, see the note below on "system indigestion.")


Tcl regular expression syntax:


Expect variables, using global and upvar keywords in Tcl scripts:


Excerpt on Tcl pattern matching from Inguza web site: 

2   Expect
The expect suite actually consist just of a few commands: expect, send, spawn and interact and is an extension to TCL.
Before the expect command can be used a program must be spawned, see expect commands below.
expect \
  {match1} {action1} \
  {match2} {action2} \
  {match3} {action3} \
  {match4} {action4}
It is possible that no such output is encountered within the time period defined by the timeout (variable name "timeout" in seconds, default to 2). In this case expect stops waiting and continues with the next command in the script.
You can use '*' to match any characters. You can also match using [] syntax and similar.
expect "\[a-f0-9]"  ; # strongly preferred as \n and similar is not taken as litteral charactesr \ and n.
expect {a-f0-9}     ; # can also be used
A note on \ characters. The pattern matcher translate \x to x but this is done once before the pattern matcher. So you need to type \\n to match n. This is especially important for * and ? characters.
expect "*" ;# match anything (*)
expect "\n" ;# match linefeed
expect "\*" ;# match anything
expect "\\*" ;# match * character
expect "\\\*" ;# match * character
expect "\\\\*" ;# match \ followed by anything
expect "\\\\\*" ;# match \ followed by anything
expect "\\\\\\*" ;# match \ followed * character
The [ character is special to both tcl and the pattern matcher so it is especially messy.
proc xy {} { return "n*w" }
expect "[xy]" ;# match n followed by anything followed by w
expect "\[xy]" ;# match x or y
expect "\\[xy]" ;# match n followed by anything followed by w
expect "\\\[xy]" ;# match [xy]
expect "\\\\[xy]" ;# match \ followed by n followed ...
expect "\\\\\[xy]" ;# match sequence of \ and then x or y.
The expect command can also take flags. The default flag is -gl for glob pattern matching. It can also take the flag -re for regular expression matching.
expect {
  -re "a*" { action_when_a_followed_by_any_a }
  "b*" { action_when_b_followed_by_anything }
  -gl "-re" { action_when_real_dash_r_e }
  eof { action_on_end_of_file }
  timeout { action_on_command_timeout }
  default { action_on_command_timeout_or_eof }
  -re "(abc)*" { action_on_any_number_of_a_b_c }
  -re "-?(0|\[1-9]\[0-9]*)?\\.?\[0-9]*" { action_on_float_or_integer }
}
You can also pass the pattern as a variable.
set v2pats "pat2 act2"
set v3pats "pat3 act3"
expect pat1 act1 $v2pats $v3pats
Observe that the following expect string is wrong as $v2pats and $v3pats is considered as two different arguments. It will try to match $v2pats as a pattern and $v3pats as an action. Instead build up a string and use the -brace flag. You can not use double quotes for that though, unless you use eval and then the -brace flag is not necessary. You also need to protect the pattern and action inside a [list x] or an extra brace if they contain whitespaces.
2.1   Parentheses for feedback
In the previous section, parantheses were used to group subpatterns together. They can also play another role. When a regular expression successfully matches a string each part of the string that matches a paranthensized subpattern is stored in "expect_out(1,string)" the second in "expect_out(2,string)" etc up to "expect_out(9,string)". The entire pattern matched is stored in "expect_out(0,string)". Everthing before the pattern and the pattern matched is stored in "expect_out(buffer)". The last two assignments work in glob pattern too.
"a*((ab)*|b)"
How is this passed to expect_out? It is quite simple. The entire string withing the first parenthesis is stored in 1 (which means up to end of string). The second fount parenthesis is stored in 2. Look for the left parenthesis to determine where the data is put.
2.2   Flags
-re	
-gl	
-nocase	Ignore case. Do not use -nocase with uppercase characters in the pattern as it will never match.
-notransfer	Prevent expect from removing matching characters from the internal buffer. The characters can be matched repeatedly as long as this option is associated with the pattern. It can be abbreviated with "-n" when expect is running interactively.
-brace	Expect normally see the first argument as a pattern to match, but sometimes you want to give a list of patterns and actions to expect. You can then use -brace to expand the list before it is handled by expect.
expect -brace { pat1 act1 pat2 act2 }
This can be useful when building up lists of pattern to match with corresponding actions to do.
-i n	Use an alternative spawn_id. This applies to all patterns after the -i flag.
expect {
 -i $ftp
 "ftp>" "exit"
 eof "exit"
}
You can also expect values from multiple spawned processes at the same time. At most one action can be executed just as expect normally do.
expect {
 -i $ftp "ftp> " { ftp_action }
 -i $shell $prompt { shell_action }
}
There is an input buffer associated with each spawn id. So any output from ftp is kept separate from thta of the shell in the example above. When output appear on terminal it will be mixed, unless you expect one at a time or set log_user 0 and then wrote output explictly using send_user $expect_out(buffer). The process that did not match will keep its buffer until matched by next expect command.
When the expect command match something it record the spawn_id to expect_out(spawn_id) so you can know which process that it matched against.
You can also set the spawn_id definition to a list of spawn_ids.
expect {
  -i "$host1 $host2 $host3" $prompt {
    an_action $expect_out(spawn_id)
  }
  eof exit
  -i $host1 another_pattern {host1_action}
}
This example will do an_action if $prompt is matched agains any of the $hostnr, exit if end of file is matched against same list and do host1_action if matchina another_pattern from host1.
2.3   Special patterns
Note that these patcherns will only match if none of the '-gl', '-re' or '-ex' flags has been used.
eof	Match when end of file is reached.
timeout	Match when the timeout value has been reached.
default	Match when end of file or timeout is reached.
full_buffer	Match when no other pattern match and expect would otherwise throw away part of the input to make room fore more. When this matches, all of the unmatched input is moved to expect_out(buffer).
null	
By default null characters are removed from the buffer before expect matching is done. This can be disabled using the remove_nulls command below. If it is disabled null characters will be matched using this special keyword. Null characters can not be matched inside a pattern as expect first look for null characters and then do pattern matching. If a null character is found the characters before it is skilled. These caracters can be found in expect_out(buffer).
Since the null character is used internally to represent the end of a string unanchored patterns cannot be matched past a null. This is not a problem since null pattern can always be listed last.


↑ ↓ Rust programming language

There are two editions of the Rust Programming book. The Rust home page recommends to start by reading the second edition:


^ C programming notes

Click here -> C programming notes <- for separate wiki article.

Typedef and type defining function pointers:

Retrieved from "https://wiki.neelanurseries.com/index.php?title=User:Ted&oldid=2536"