Difference between revisions of "User:Ted"

From Wiki at Neela Nurseries
Jump to: navigation, search
m (→‎^ Linux Packages)
m (→‎= GRUB2 - Grand Unified Boot Loader)
Line 228: Line 228:
 
<!-- comment -->
 
<!-- comment -->
  
−
=== GRUB2 - Grand Unified Boot Loader ==
+
=== GRUB2 - Grand Unified Boot Loader ===
  
  
 
*  https://unix.stackexchange.com/questions/252936/grub2-boot-to-a-second-another-hard-disk
 
*  https://unix.stackexchange.com/questions/252936/grub2-boot-to-a-second-another-hard-disk
−
*
+
* https://help.ubuntu.com/community/Grub2/CustomMenus
 +
 
  
  

Revision as of 18:23, 28 November 2017

-- 2017-07-21 शुक्रवार --
Ted's page on Neela Nurseries Wiki


^ OVERVIEW

This wiki document a starting point of Ted's notes on Linux use and configuration, Open Source Software and web development. Recent summer 2017 efforts of Ted's are focused on several high-level programming language and web development frameworks, which can be studied and used separately but are often glued together to achieve meaningful tasks and end-user tools and interfaces. This personal page of Ted's notes on Neela Nurseries wiki not yet well organized, but here as a quick stash point for holding useful references, and a starting point for more complete and formal documentation.

- 2017-10-05 - Ted noting here that would be good to have a technical glossary to make easily accessible all the technical terms which arise in daily work and exploration of technologies . . .


- 2017-10-19 - TO-DO list: 

  *  complete LFS 8.1 exercise
  *  configure xserver to run stand-along without window manager
  *  set up two mysql servers on one host
  *  write "best photo contribution" practices page for ASI
  *  code "re-use or reference text block" module for MediaWiki
  *  in local Neela Nurseries PHP code base, amend hybrid building of nav menus to include highlighting parent page of given viewed page
  *  in OpenCart 2.x find way to customize top-of-page information and links
  *  in OpenCart 2.x find way to format product lists as compact lists, like Gmail inbox or Windows 'list view' of files



^ Web Site Building Blocks - Summer 2017 Work

 
The following tools and web and programming frameworks are all part of Ted's volunteer efforts with ASI web site, and in-progress study of how to configure and customize shopping carts, article and document management pages, and easy-to-read easy-to-adjust web page layouts using CSS and third party, open source frameworks:


  • Linux package selection for constrained systems
  • MYSQL data base install and config (set up)
  • PHP set up
  • Apache2 set up
  • SSL Self-signed certificate creation and config
  • Sub-domain creation under Ubuntu 16.04 LTS and similar Linux releases
  • Mediawiki work . . .
  • OpenCart 2.x install and config, 'Wish List' and 'To Do List' . . .


    ^ Cascading Style Sheets (CSS)


    ^ 2017-11-27 - Web Page Fonts

    ^ 2017-11-28 - Javascript and JS Frameworks



    These are the first topics on which Ted wants to gather together notes. Hard to remember all details regarding pitfalls encountered, solutions found, and ideas for improving the configuration and use experience of these softwares . . . - TMH




    ^ Linux Packages

    - 2017-07-31 सोमवार -


    This section a starting point for notes on Linux packages, namely Debian and Ubuntu packages. Focus here is on userland software, and which packages are needed to set up various tools sets in a working Unix / Linux environment . . .

    • libreoffice . . .
      - 2017-10-11 WED -
      libreoffice-base
      libreoffice-calc
      libreoffice-dbg
      libreoffice-draw
      libreoffice-help-en-gb
      libreoffice-help-es
      libreoffice-impress
      libservlet2.5-java
    • postfix . . . see section below
    • gpm
    • ctags
    • vim-doc
    • vim-scripts
    • xfonts-cyrillic
    • fonts-sil-gentium-basic
    • devhelp
    • distcc . . . distributed compiling of C and C++ programs across multiple machines

    - 2017-10-12 THU -

    • xfractint
    • radeontop



    ^ Postfix


    ^ Linux packages for Raspberry Pi system


    ^ Logitech wireless devices and unifying receiver


    ^ sudo

    Ted noting sudo manual page URL here, in order to revisit and read about sudo -e as well as sudo invoked with the -l and -v options. Also noting there is an environment variable sometimes set, variable named $SUDO_USER.


    ^ parted partition editor

    Some excerpts from parted use on LFS volume . . . 

    
(parted) print /dev/sdb                                                   
Model: ATA ST3160812AS (scsi)
Disk /dev/sdb: 160GB
Sector size (logical/physical): 512B/512B
Partition Table: msdos

Number  Start   End     Size    Type      File system  Flags
 1      32.3kB  107MB   107MB   primary   ext4         boot
 2      107MB   7789MB  7682MB  extended
 5      107MB   7789MB  7682MB  logical   ext4

   .
   .
   .

    Well one thing Ted noting is that parted cannot resize file systems, which are the structured data which overlay and live in partitions. The partition is one thing, mapped in a partition table somewhere on the physical disk or volume. The file system on a given partition is another thing, contained and bounded within that partition. Resizing the partition doesn't resize the file system. So which tool can deal with resizing a partition on which an ext4 file system exists? And then permit a next, independent step to resize the file system there? - TMH



    GRUB2 - Grand Unified Boot Loader



    ^ Bash Shell Scripting

    Shell scripting and use of built-in shell commands is a really practical knowledge to employ, when working in Unix-like environments. One common task is to find all the instances of a given file or program. The locate command can perform this kind of search, but it's results don't show whether the file instances differ. To check at the rough level of file size, we can use a one-line shell script technique involving shell piping, to "long list" the results of the locate command, like this . . . 


      $ for file in `locate tavrasm | grep 'asm$'`; do ls -l ${file}; done


    Hmm strange, the above command calls `grep` with a pattern that ends in the shell end-of-line anchoring character $, and appears to filter for results of `locate` which end in 'asm'. But on 2017-09-12 needed to add a "one or more wildcard" character pattern to limit search results for instances of `locate` results ending in 'qemu': 


    $ locate qemu | grep '.*qemu$'


    Why the apparent difference in command invocation? Need to test . . . - TMH


    edit point - shell variable quoting

    - 2017-10-19 THU -

    Running into issues when need to expand shell variable in single quotes pair . . .

    Stack Exchange, Unix forum post 178411
    TLPD Advanced Bash Scripting Guide, chapter 4.2 example 4-3 double quotes preserve white space


    WIKI WISH LIST - While adding to wiki, Ted noting that would be useful to have relative-depth wiki section markers. The standard wiki section markers '== ==', '=== ===' and similar have fixed depth. That is, '== ==' is always a top-level or first-level section in a wiki document, '=== ===' is always a sub-subsection of wiki documents, and '==== ====' a sub-sub-section. When section gets re-factored into its own article, would be handy to have those section markers rise up to levels of top-most and successive section markers. - TMH



    example shell script - back up several databases

     
    Here is a simple shell script to call mysqldump utility and back up several databases. Couple of things Ted wants to add to this script include script variables to hold back-up filename prefix and infix patterns, and an option to compress the MYSQL dump files . . .

    Figure x - shell script to back-up multiple MYSQL databases 

    
#!/bin/bash


DATABASE_LIST="information_schema mysql drupal_8p0 phpmyadmin wiki_database"

user="root"
pass_phrase_for_mysql="mysql_user_password"
options_extra="--skip-lock-tables"

response="n"

mode_interactive="n"



echo "shell script starting,"



for database in ${DATABASE_LIST};
do echo "backing up alta-ubuntu database $database . . .";

#    command="mysqldump --databases $database -u$user -p$pass_phrase_for_mysql $options_extra >> au-database-back-up--${database}.sql"
    command="mysqldump --databases $database -u$user -p$pass_phrase_for_mysql $options_extra"
    redirect="au-database-back-up--${database}.sql"

    if [ $mode_interactive = 'y' ]; then

        echo "build command '$command' and database back-up filename'$redirect',"
        echo "full command will be '$command' > '$redirect',"
        echo "trying running this command and redirect? [y/N/q]  yes, no, 'q' to quit"

        read response
        if [ $response = 'Y' -o $response = 'y' ]; then
            ${command} > $redirect
        elif [ $response = 'N' -o $response = 'n' ]; then
            echo "skipping present command . . ."
        elif [ $response = 'Q' -o $response = 'q' ]; then
            echo "stopping script '$0' and exiting."
            break
        fi
        echo

    else

        echo "$0:  backing up database '$database' to file '$redirect' . . ."
        ${command} > $redirect
        ls -l $redirect

    fi

done


echo "done."

exit 0



    ^ Aboriginal Linux - project now defunct, new project is 'makeroot'

    Ok, looks like Robert Landley's Aboriginal Linux project has been superceded by mkroot. Ladnley's newer makeroot project sources are available at GitHub:



    Having trouble carrying out the basic build of Aboriginal Linux with armv6l as target . . . 

    
   .
   .
   .

Confirmed e2fsprogs-1.42.13.tar.gz
Confirmed zlib-1.2.7.tar.bz2
Confirmed squashfs-4.2.tar.gz
=== Got all source.

real    0m2.240s
user    0m1.188s
sys     0m0.176s
=== toybox (host host-tools)
Snapshot 'toybox'...
scripts/genconfig.sh
cc -o kconfig/conf kconfig/conf.c kconfig/zconf.tab.c -DKBUILD_NO_NLS=1 \
                -DPROJECT_NAME=\"ToyBox\"
kconfig/conf -D /dev/null Config.in > /dev/null
scripts/make.sh
Generate headers from toys/*/*.c...
generated/newtoys.h Library probe.......
Make generated/config.h from .config.
generated/flags.h generated/globals.h generated/help.h
Compile toybox................................................................................................................................................................generated/obj/nsenter.o: In function `unshare_main':
nsenter.c:(.text.unshare_main+0x152): undefined reference to `setns'
collect2: error: ld returned 1 exit status
make: *** [toybox] Error 1

Exiting due to errors (host host-tools toybox)


    Robert Landley also has a presence on www.patreon.com/landley, where he talks about the over-arching goals in his programming and software systems level work.



    ^ Linux From Scratch (Separate Article)

    Linux From Scratch Book 8.1 - Build and Notes



    ^ QEMU Emulator

    Overview

    Notes on QEMU, an emulator which can help with building and configuring Linux systems to run on embedded computers and development boards. Ted noting 2017-09-08 that to compile today's latest QEMU source, a 2.10.x release, needed to install Debian stretch packages pkg-config, libglib2.0-dev, dhautoreconf. Here are some links to downloading QEMU project sources, a manual for using QEMU, and an article about emulating a RaspberryPi system by using QEMU:

    References:


    So got QEMU 2.10.0 sources and dependencies installed, ran `configure` and `make` steps, and let the build process run for an hour or two. But in the end could not find any `qemu` executable. Why is the emulator itself apparently missing? Building latest QEMU under Linux . . .


    So I'm following the above article author's typical build steps, and run into this message part-way down those steps, at the step of calling `make`: 

    user@host-6:~/Downloads/qemu/qemu-2.10.0/bin/debug/native$ make
Makefile:21: *** This is an out of tree build but your source tree (/home/veris/Downloads/qemu/qemu-2.10.0) seems to have been used for an in-tree build. You can fix this by running "make distclean && rm -rf *-linux-user *-softmmu" in your source tree.  Stop.
user@host-6:~/Downloads/qemu/qemu-2.10.0/bin/debug/native$


    Ok interesting, I know that QEMU is a complicated project with a complex build process, but clearly I don't know what I'm doing. Following the above 'distclean' command given in the error message from QEMU's makefile, I'm now able to begin an "out of tree" build as described in the "Building QEMU" article at qemu.org just above. Tired already however of QEMU project builds taking more than an hour to complete. Here is one of many articles describing how to invoke `make` to build only limited parts of QEMU project, for example just the pieces needed to emulate ARM type systems:



    Ted to test this type of build soon . . .

    On a different note, QEMU build has completed and the stock RaspberryPi Debian "Stretch" image is coming up. Needed to install a VNC client to interact with the emulated RaspberryPi operating and software, stored in the downloaded .img file from Rpi's web site. Here's a link to a forum post which clued Ted into Remmina VNC client:



    - 2017-11-21 Tuesday -

    The version of qemu which installed from Debian's package list didn't seem to work. It's been some weeks now since trying that instance, but qemu 2.10.0 works to bring up a command line image built for RaspberryPi. Here are the versions of the two qemu instances:

    Figure x - locating qemu instances, determining their versions 

    
user@localhost:~$ which qemu
/usr/bin/qemu

user@localhost:~$ /usr/bin/qemu --version
QEMU emulator version 1.1.2 (Debian 1.1.2+dfsg-6+deb7u23), Copyright (c) 2003-2008 Fabrice Bellard

user@localhost:~$ ls /opt/qemu
qemu-system-arm

user@localhost:~$ ls -l /opt/qemu/qemu-system-arm
lrwxrwxrwx 1 root root 83 Sep 12 16:37 /opt/qemu/qemu-system-arm -> /home/user/Downloads/qemu/qemu-2.10.0/bin/debug/native/arm-softmmu/qemu-system-arm

user@localhost:~$ /opt/qemu/qemu-system-arm --version
QEMU emulator version 2.10.0
Copyright (c) 2003-2017 Fabrice Bellard and the QEMU Project developers

user@localhost:~


    - 2017-11-22 Wednesday -

    Yesterday could not bring up raspbian image in qemu, same image which booted and appeared, and permitted shell based login and shell use a month ago on the local host. Looking at a new on-line reference for running raspbian images in qemu: 


      *  https://ownyourbits.com/2017/02/06/raspbian-on-qemu-with-network-access/


    This above article looks really interesting, appears to go to a second installment of the article where the developers there are building some kind of cloud-based or cloud involved web, database, mail and PHP servers. But on my end having trouble with a kernel panic this morning when invoking qemu-system-arm with a large set of options . . . ok, just located a different invocation of qemu-system-arm version 2.10.0, noting here in following code figure:


    Figure x - emulate-rpi shell script 

    
#!/bin/bash
# Starts raspberry pi image in configuration mode


##----------------------------------------------------------------------
## - SECTION - script variables
##----------------------------------------------------------------------

EMULATOR=/home/veris/Downloads/qemu/qemu-2.10.0/bin/debug/native/arm-softmmu/qemu-system-arm



##----------------------------------------------------------------------
## - SECTION - original emulator invocation, everything hard-coded
##----------------------------------------------------------------------

# qemu-system-arm -kernel ./qemu-rpi-kernel/kernel-qemu -cpu arm1176 -m 256 -M versatilepb -no-reboot -serial stdio -append "root=/dev/sda2 panic=1 rootfstype=ext4 rw init=/bin/bash" -hda rpi.img

# qemu-system-arm -kernel ./qemu-rpi-kernel/kernel-qemu -cpu arm1176 -m 256 -M versatilepb -no-reboot -serial stdio -append "root=/dev/sda2 panic=1 rootfstype=ext4 rw" -hda rpi.img



##----------------------------------------------------------------------
## - SECTION - invocation, emulator as variable:
##----------------------------------------------------------------------

# ${EMULATOR} -kernel ./qemu-rpi-kernel/kernel-qemu -cpu arm1176 -m 256 -M versatilepb -no-reboot -serial stdio -append "root=/dev/sda2 panic=1 rootfstype=ext4 rw init=/bin/bash" -hda rpi.img

# Start RaspberryPi in fully functional mode:
${EMULATOR} -kernel ./qemu-rpi-kernel/kernel-qemu -cpu arm1176 -m 256 -M versatilepb -no-reboot -serial stdio -append "root=/dev/sda2 panic=1 rootfstype=ext4 rw" -hda rpi.img






## - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
## 2017-09-08 FRI - this script copied from http://embedonix.com/articles/linux/emulating-raspberry-pi-on-linux/
## - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

exit 0


    The final token rpi.imgL points to localhost:/home/user/Downloads/raspberrypi/2017-08-16-raspbian-stretch-lite.img which we downloaded from https://www.raspberrypi.org/downloads/raspbian/. The working qemu invocation doesn't open qemu's native window, but puts to standard out that is the same shell in which qemu invoked, a couple of lines of output from the booting kernel image. Then using remmina we connect via "VNC server running on ::1:5900".

    In summary shell script in latest figure successfully invokes qemu-system-arm 2.10.0 built from sources about August 2017, and boots up Raspbian Stretch lite image and kernel named kernel-qemu-4.4.34-jessie. Next figure is screenshot showing successful login, and last login message . . . though emulation was and is very slow on host system!


    Figure x - emulated raspbian stretch lite screenshot, Remmina window

    media:z.png





    ^ RaspberryPi Kernel Compilation

    - 2017-11-25 SAT - These notes are of first efforts to cross-compile Linux kernel and device tree blob, on Intel i386/i686 host system, for target ARM system specifically RaspberryPi 2 dev board. Here are instructions for compiling and for cross-compiling a Linux kernel for RaspberryPi target system . . .



    Back in 2017 February this year, February 24th just after Embedded Linux Conference, looks like Ted stepped through some of the instructions at this Raspberry Pi dot org documentation page. Ted noting there are two directories on compiling host in /mnt with this timestamp: 


    
ted@localhost:/mnt$ ls -l
total 44
drwxr-xr-x 2 root root 4096 Oct 24  2016 dvd-rom
drwxr-xr-x 2 root root 4096 Feb 24  2017 ext4
drwxr-xr-x 2 root root 4096 Feb 24  2017 fat32
   . . .


    When issuing the cross-compile command that task seemed to complete quickly, giving the following messages which total less than two hundred lines: 


    
ted@rangari:~/projects/raspberrypi/linux$ make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- bcm2709_defconfig
#
# configuration written to .config
#
ted@rangari:~/projects/raspberrypi/linux$ make -j 2 ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- zImage modules dtbs
scripts/kconfig/conf  --silentoldconfig Kconfig
  CHK     include/config/kernel.release
  CHK     include/generated/uapi/linux/version.h
  CHK     include/generated/utsrelease.h
  HOSTCC  scripts/dtc/dtc.o
  HOSTCC  scripts/dtc/flattree.o
  HOSTCC  scripts/dtc/fstree.o
  HOSTCC  scripts/dtc/data.o
  HOSTCC  scripts/dtc/livetree.o
  HOSTCC  scripts/dtc/treesource.o
  HOSTCC  scripts/dtc/srcpos.o
make[1]: 'include/generated/mach-types.h' is up to date.
  CHK     include/generated/timeconst.h
  CHK     include/generated/bounds.h
  CHK     include/generated/asm-offsets.h
  CALL    scripts/checksyscalls.sh
  HOSTCC  scripts/dtc/checks.o
  HOSTCC  scripts/dtc/util.o
  HOSTCC  scripts/dtc/dtc-lexer.lex.o
  HOSTCC  scripts/dtc/dtc-parser.tab.o
  HOSTLD  scripts/dtc/dtc
  HOSTCC  scripts/genksyms/genksyms.o
  HOSTCC  scripts/genksyms/parse.tab.o
  HOSTCC  scripts/mod/mk_elfconfig
  HOSTCC  scripts/genksyms/lex.lex.o
  MKELF   scripts/mod/elfconfig.h
  HOSTCC  scripts/mod/modpost.o
  HOSTLD  scripts/genksyms/genksyms
  HOSTCC  scripts/kallsyms
  HOSTCC  scripts/mod/file2alias.o
  HOSTCC  scripts/pnmtologo
  HOSTCC  scripts/conmakehash
  HOSTCC  scripts/mod/sumversion.o
  HOSTCC  scripts/recordmcount
  HOSTLD  scripts/mod/modpost
  HOSTCC  scripts/sortextable
  HOSTCC  usr/gen_init_cpio
  CHK     include/generated/compile.h
  GEN     usr/initramfs_data.cpio.gz
  HOSTCC  arch/arm/vdso/vdsomunge
  AS      usr/initramfs_data.o
  LD      usr/built-in.o
  MUNGE   arch/arm/vdso/vdso.so.dbg
  OBJCOPY arch/arm/vdso/vdso.so
  AS      arch/arm/vdso/vdso.o
  LD      arch/arm/vdso/built-in.o
  GZIP    kernel/config_data.gz
  CHK     kernel/config_data.h
  HOSTCC  lib/gen_crc32table
  HOSTCC  lib/raid6/mktables
  TABLE   lib/raid6/tables.c
  CC [M]  lib/raid6/tables.o
  LD [M]  lib/raid6/raid6_pq.o
  GEN     lib/crc32table.h
  CC      lib/crc32.o
  LD      lib/built-in.o
  LOGO    drivers/video/logo/logo_linux_clut224.c
  LOGO    drivers/video/logo/logo_linux_mono.c
  LOGO    drivers/video/logo/logo_superh_mono.c
  LOGO    drivers/video/logo/logo_superh_vga16.c
  LOGO    drivers/video/logo/logo_linux_vga16.c
  LOGO    drivers/video/logo/logo_blackfin_vga16.c
  LOGO    drivers/video/logo/clut_vga16.c
  LOGO    drivers/video/logo/logo_spe_clut224.c
  LOGO    drivers/video/logo/logo_mac_clut224.c
  LOGO    drivers/video/logo/logo_superh_clut224.c
  LOGO    drivers/video/logo/logo_sun_clut224.c
  LOGO    drivers/video/logo/logo_parisc_clut224.c
  LOGO    drivers/video/logo/logo_blackfin_clut224.c
  LOGO    drivers/video/logo/logo_dec_clut224.c
  LOGO    drivers/video/logo/logo_m32r_clut224.c
  LOGO    drivers/video/logo/logo_sgi_clut224.c
  CC      drivers/video/logo/logo_linux_clut224.o
  LD      drivers/video/logo/built-in.o
  LD      drivers/video/built-in.o
  LD      drivers/built-in.o
  LINK    vmlinux
  LD      vmlinux.o
  MODPOST vmlinux.o
  GEN     .version
  CHK     include/generated/compile.h
  UPD     include/generated/compile.h
  CC      init/version.o
  LD      init/built-in.o
  KSYM    .tmp_kallsyms1.o
  KSYM    .tmp_kallsyms2.o
  LD      vmlinux
  SORTEX  vmlinux
  SYSMAP  System.map
  OBJCOPY arch/arm/boot/Image
  Building modules, stage 2.
  Kernel: arch/arm/boot/Image is ready
  Kernel: arch/arm/boot/Image is ready
  GZIP    arch/arm/boot/compressed/piggy.gzip
  AS      arch/arm/boot/compressed/piggy.gzip.o
  LD      arch/arm/boot/compressed/vmlinux
  OBJCOPY arch/arm/boot/zImage
  Kernel: arch/arm/boot/zImage is ready
  MODPOST 1540 modules
  LD [M]  lib/raid6/raid6_pq.ko
ted@rangari:~/projects/raspberrypi/linux$


    At this point it looks like we may have enough to attempt the qemu invocation which was partially working off site west side, since couple days ago we were missing what appeared to be only the kernel when invoking qemu on build box rangari . . .



    ^ Python Scripting Language



    ^ Other Things To Explore


    Here are some technical and science topics to explore, perhaps on a rainy day . . .


    3D Graphics, mathematics behind and programming:


    Creative Commons and other Open Source licensed materials:


    Dublin Core:


    Javascript and JSON:


    PC Video, Bochs VBE Extensions:


    Perl programming related:


    Unix and Linux building blocks:

    • nss, Name Service Switch
    • "Burrows-Wheeler block sorting text compression algorithm", noted 2017-09-29 FRI from LFS 8.1 chapter 6.21 - TMH


    Webvanta:


    Virtual server image creation:


    Github Users Sharing Their Work:




    ^ References


    Reproducible builds, byte-wise reproducible software:


    Server Side services . . .


    Couple of MediaWiki and publishing issues to look into, which came up during wiki configuration:

    Some Hindi language and UTF-8 encoding references, to be factored to a dedicated wiki page later:

    Linux programs and utilities:


    Computer hardware and firmware:


    QEMU



    - - - top of page - - -


Retrieved from "https://wiki.neelanurseries.com/index.php?title=User:Ted&oldid=386"