Difference between revisions of "Nn embedded linux notes"

From Wiki at Neela Nurseries
Jump to: navigation, search
m
m
Line 8: Line 8:
  
  
 +
 +
<b>Overview</b>
 +
 +
This wiki document is a collection of Ted's notes on embedded Linux studies, with a starting point of building a recipe for a Linux system to run on a [https://en.wikipedia.org/wiki/Raspberry_Pi#Specifications RaspberryPi 2] target hardware platform.  Near term goal is to build a customized Linux 4.x kernel and test that it runs on a RaspberryPi dev board.  Long term goal has two parts:  &nbsp;(1)  build a more useful Linux system with working C and C++ toolchain, networking stack, web server, e-mail and database server support, an X server or comparable graphics and desktop support, and  &nbsp;(2)  target other hardware platforms beyond RaspberryPi 2 board.
 +
 +
As of 2017-12-11 these notes are a mess, so Ted reviewing work so far and here starting summary of what's important, what has worked, what best steps known today to take going forward.  First the major pieces we are finding and using:
 +
 +
<ul>
 +
  <li> cross-compiling toolchain, e.g. Linaro gcc<br />
 +
  <li> QEMU system hardware emulator<br />
 +
<ul>
 +
 +
The cross-compiler and related tools we use to build our kernel and 'target' a.k.a. 'guest' system softwares.  The guest system in our case is a Linux kernel and additional programs compiled to run together as a complete system on an ARM based processor, in the target hardware which is the RaspberryPi 2 board.  We cross compile in order to get the much faster compilation times available on a typical cerca 2010 Intel desktop computer.  The QEMU software allows us to experiment, jumping through some hoops, with our ARM-targeted kernels and softwares, without a physical RaspberryPi board present.  We also get to skip the step of creating a system image and flashing that to SD card.  Knowing how to create an SD-card-ready image, however, is important.  We'll return to this we hope sooner than later, once we have seen our cross-compiled system pieces run in normal expected ways in QEMU's target hardware emulating environment.
 +
 +
Our target system software pieces for a basic Linux operating system include:
 +
 +
<ul>
 +
  <li>Linux kernel 4.x<br />
 +
 +
</ul>
  
  

Revision as of 16:42, 11 December 2017

Wiki Main Page | User Page of Ted | Google search engine


Embedded Linux Notes
 this wiki article created 2017-12-11 Monday


Overview

This wiki document is a collection of Ted's notes on embedded Linux studies, with a starting point of building a recipe for a Linux system to run on a RaspberryPi 2 target hardware platform. Near term goal is to build a customized Linux 4.x kernel and test that it runs on a RaspberryPi dev board. Long term goal has two parts:  (1) build a more useful Linux system with working C and C++ toolchain, networking stack, web server, e-mail and database server support, an X server or comparable graphics and desktop support, and  (2) target other hardware platforms beyond RaspberryPi 2 board.

As of 2017-12-11 these notes are a mess, so Ted reviewing work so far and here starting summary of what's important, what has worked, what best steps known today to take going forward. First the major pieces we are finding and using:

  • cross-compiling toolchain, e.g. Linaro gcc
  • QEMU system hardware emulator
      The cross-compiler and related tools we use to build our kernel and 'target' a.k.a. 'guest' system softwares. The guest system in our case is a Linux kernel and additional programs compiled to run together as a complete system on an ARM based processor, in the target hardware which is the RaspberryPi 2 board. We cross compile in order to get the much faster compilation times available on a typical cerca 2010 Intel desktop computer. The QEMU software allows us to experiment, jumping through some hoops, with our ARM-targeted kernels and softwares, without a physical RaspberryPi board present. We also get to skip the step of creating a system image and flashing that to SD card. Knowing how to create an SD-card-ready image, however, is important. We'll return to this we hope sooner than later, once we have seen our cross-compiled system pieces run in normal expected ways in QEMU's target hardware emulating environment. Our target system software pieces for a basic Linux operating system include:
      • Linux kernel 4.x



      ^ RaspberryPi Kernel Compilation


      - 2017-11-25 SAT - These notes are of first efforts to cross-compile Linux kernel and device tree blob, on Intel i386/i686 host system, for target ARM system specifically RaspberryPi 2 dev board. As of 2017-DEC-09 Saturday, four or five custom kernel configurations with rpi target chosen / in mind have failed, for reasons such as "enermation or pound define value not an integer" and "alignment type of variable x not available [on target processor]". Ted noting that the Linux kernel stable release with all its options and supported target hardware platforms and peripherals is heavyweight, not by default configured to be small or simple as possible. Now also looking into Tiny Kernel wiki page and home page of Tiny Core Linux.

      Here are instructions for compiling and for cross-compiling a Linux kernel for RaspberryPi target system:



      Here are limited instructions on how to configure the Linux kernel for RaspberryPi target systems:



      Back in 2017 February this year, February 24th just after Embedded Linux Conference, looks like Ted stepped through some of the instructions at this Raspberry Pi dot org documentation page. Ted noting there are two directories on compiling host in /mnt with this timestamp: 


      ted@localhost:/mnt$ ls -l make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- menuconfig
total 44
drwxr-xr-x 2 root root 4096 Oct 24  2016 dvd-rom
drwxr-xr-x 2 root root 4096 Feb 24  2017 ext4
drwxr-xr-x 2 root root 4096 Feb 24  2017 fat32
   .
   .
   .

      When issuing the cross-compile command that task seemed to complete quickly, giving the following messages which total less than two hundred lines: 


      ted@rangari:~/projects/raspberrypi/linux$ make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- bcm2709_defconfig
#
# configuration written to .config
#
ted@rangari:~/projects/raspberrypi/linux$ make -j 2 ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- zImage modules dtbs
scripts/kconfig/conf  --silentoldconfig Kconfig
  CHK     include/config/kernel.release
  CHK     include/generated/uapi/linux/version.h
  CHK     include/generated/utsrelease.h
  HOSTCC  scripts/dtc/dtc.o
  HOSTCC  scripts/dtc/flattree.o
  HOSTCC  scripts/dtc/fstree.o
  HOSTCC  scripts/dtc/data.o
  HOSTCC  scripts/dtc/livetree.o
  HOSTCC  scripts/dtc/treesource.o
  HOSTCC  scripts/dtc/srcpos.o
make[1]: 'include/generated/mach-types.h' is up to date.
  CHK     include/generated/timeconst.h
  CHK     include/generated/bounds.h
  CHK     include/generated/asm-offsets.h
  CALL    scripts/checksyscalls.sh
  HOSTCC  scripts/dtc/checks.o make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- menuconfig
  HOSTCC  scripts/dtc/util.o
  HOSTCC  scripts/dtc/dtc-lexer.lex.o
  HOSTCC  scripts/dtc/dtc-parser.tab.o
  HOSTLD  scripts/dtc/dtc
  HOSTCC  scripts/genksyms/genksyms.o
  HOSTCC  scripts/genksyms/parse.tab.o
  HOSTCC  scripts/mod/mk_elfconfig
  HOSTCC  scripts/genksyms/lex.lex.o
  MKELF   scripts/mod/elfconfig.hhttps://www.raspberrypi.org/forums/viewtopic.php?f=66&t=178806
  HOSTCC  scripts/mod/modpost.o
  HOSTLD  scripts/genksyms/genksyms
  HOSTCC  scripts/kallsyms
  HOSTCC  scripts/mod/file2alias.o
  HOSTCC  scripts/pnmtologo
  HOSTCC  scripts/conmakehash make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- menuconfig
  HOSTCC  scripts/mod/sumversion.o
  HOSTCC  scripts/recordmcount
  HOSTLD  scripts/mod/modpost
  HOSTCC  scripts/sortextable
  HOSTCC  usr/gen_init_cpio
  CHK     include/generated/compile.h
  GEN     usr/initramfs_data.cpio.gz
  HOSTCC  arch/arm/vdso/vdsomunge
  AS      usr/initramfs_data.o
  LD      usr/built-in.o
  MUNGE   arch/arm/vdso/vdso.so.dbg
  OBJCOPY arch/arm/vdso/vdso.so
  AS      arch/arm/vdso/vdso.o
  LD      arch/arm/vdso/built-in.o
  GZIP    kernel/config_data.gz make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- menuconfig
  CHK     kernel/config_data.h
  HOSTCC  lib/gen_crc32table
  HOSTCC  lib/raid6/mktables
  TABLE   lib/raid6/tables.c
  CC [M]  lib/raid6/tables.o
  LD [M]  lib/raid6/raid6_pq.o
  GEN     lib/crc32table.h
  CC      lib/crc32.o
  LD      lib/built-in.o
  LOGO    drivers/video/logo/logo_linux_clut224.c
  LOGO    drivers/video/logo/logo_linux_mono.c
  LOGO    drivers/video/logo/logo_superh_mono.c make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- menuconfig
  LOGO    drivers/video/logo/logo_superh_vga16.c make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- menuconfig
  LOGO    drivers/video/logo/logo_linux_vga16.c
  LOGO    drivers/video/logo/logo_blackfin_vga16.c
  LOGO    drivers/video/logo/clut_vga16.c
  LOGO    drivers/video/logo/logo_spe_clut224.c
  LOGO    drivers/video/logo/logo_mac_clut224.c
  LOGO    drivers/video/logo/logo_superh_clut224.c
  LOGO    drivers/video/logo/logo_sun_clut224.c
  LOGO    drivers/video/logo/logo_parisc_clut224.c
  LOGO    drivers/video/logo/logo_blackfin_clut224.c
  LOGO    drivers/video/logo/logo_dec_clut224.c
  LOGO    drivers/video/logo/logo_m32r_clut224.c
  LOGO    drivers/video/logo/logo_sgi_clut224.c
  CC      drivers/video/logo/logo_linux_clut224.o
  LD      drivers/video/logo/built-in.o
  LD      drivers/video/built-in.o make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- menuconfig
  LD      drivers/built-in.o
  LINK    vmlinux
  LD      vmlinux.o
  MODPOST vmlinux.o
  GEN     .version
  CHK     include/generated/compile.h
  UPD     include/generated/compile.h
  CC      init/version.o
  LD      init/built-in.o
  KSYM    .tmp_kallsyms1.o
  KSYM    .tmp_kallsyms2.o
  LD      vmlinux
  SORTEX  vmlinux
  SYSMAP  System.map
  OBJCOPY arch/arm/boot/Image
  Building modules, stage 2.
  Kernel: arch/arm/boot/Image is ready
  Kernel: arch/arm/boot/Image is ready
  GZIP    arch/arm/boot/compressed/piggy.gzip
  AS      arch/arm/boot/compressed/piggy.gzip.o
  LD      arch/arm/boot/compressed/vmlinux
  OBJCOPY arch/arm/boot/zImage
  Kernel: arch/arm/boot/zImage is ready
  MODPOST 1540 modules
  LD [M]  lib/raid6/raid6_pq.ko
ted@rangari:~/projects/raspberrypi/linux$


      At this point it looks like we may have enough to attempt the qemu invocation which was partially working off site west side, since couple days ago we were missing what appeared to be only the kernel when invoking qemu on build box rangari . . .

      So we can launch Raspbian for the RaspberryPi 2 board, using a kernel we've built on an i386 architecture host server, using the default bcmp_2907 (mis-spelled) kernel config file. But we can't fully compile a kernel for an ARM / Rpi target architecture when we make any kernel configuration changes via `make menuconfig`. The script which successfully starts our qemu-system-arm version 2.10.0 is: 


      #!/bin/bash

qemu-system-arm -kernel ./kernel-qemu-4.4.34-jessie -cpu arm1176 -m 256 -M versatilepb -serial stdio -append "root=/dev/sda2 rootfstype=ext4 rw" -hda ./2017-09-07-raspbian-stretch-lite.img -redir tcp:5022::22 -no-reboot

exit 0


      Right now this script is located along side several already compiled, ARM-targeted kernels which Ted believes are from the git available cross-compile Rpi project by one 'dhruvvyas90' . . . Dhruvvyas90 qemu-rpi-kernel. 

      ted@localhost:/var/local/ted/projects/raspberrypi/qemu_vms$ ls
2017-09-07-raspbian-stretch-lite.img  kernel-qemu-4.1.7-jessie   kernel-qemu-4.4.21-jessie  README.md
kernel-qemu-3.10.25-wheezy            kernel-qemu-4.4.12-jessie  kernel-qemu-4.4.26-jessie  start-qemu.sh
kernel-qemu-4.1.13-jessie             kernel-qemu-4.4.13-jessie  kernel-qemu-4.4.34-jessie  tools
ted@localhost:/var/local/ted/projects/raspberrypi/qemu_vms$


      - Summary rpi kernel compilations -

      As of 2017 Dec 2 we can using QEMU 2.10.0 boot and use Raspbian image and kernel from Raspberry Pi on-line store. We can also substitute alternate kernel in call to QEMU system emulator, a kernel from . . .



      ^ edit point - custom Rpi-targeted kernel config issues

      - 2017-11-30 - 

      In file included from ./include/uapi/linux/stddef.h:1:0,
                 from ./include/linux/stddef.h:4,
                 from ./include/uapi/linux/posix_types.h:4,
                 from ./include/uapi/linux/types.h:13,
                 from ./include/linux/types.h:5,
                 from fs/xfs/xfs_linux.h:21,
                 from fs/xfs/xfs.h:32,
                 from fs/xfs/xfs_super.c:19:
In function ‘xfs_check_ondisk_structs’,
    inlined from ‘init_xfs_fs’ at fs/xfs/xfs_super.c:1974:2:
./include/linux/compiler.h:518:38: error: call to ‘__compiletime_assert_119’ declared with attribute error: XFS: sizeof(xfs_dir2_sf_entry_t) is wrong, expected 3
  _compiletime_assert(condition, msg, __compiletime_assert_, __LINE__)
                                      ^
./include/linux/compiler.h:501:4: note: in definition of macro ‘__compiletime_assert’
    prefix ## suffix();    \
    ^
./include/linux/compiler.h:518:2: note: in expansion of macro ‘_compiletime_assert’
  _compiletime_assert(condition, msg, __compiletime_assert_, __LINE__)
  ^
./include/linux/bug.h:54:37: note: in expansion of macro ‘compiletime_assert’
 #define BUILD_BUG_ON_MSG(cond, msg) compiletime_assert(!(cond), msg)
                                     ^
fs/xfs/xfs_ondisk.h:22:2: note: in expansion of macro ‘BUILD_BUG_ON_MSG’
  BUILD_BUG_ON_MSG(sizeof(structname) != (size), "XFS: sizeof(" \
  ^
fs/xfs/xfs_ondisk.h:119:2: note: in expansion of macro ‘XFS_CHECK_STRUCT_SIZE’
  XFS_CHECK_STRUCT_SIZE(xfs_dir2_sf_entry_t,  3);
  ^
scripts/Makefile.build:293: recipe for target 'fs/xfs/xfs_super.o' failed
make[2]: *** [fs/xfs/xfs_super.o] Error 1
scripts/Makefile.build:544: recipe for target 'fs/xfs' failed
make[1]: *** [fs/xfs] Error 2
Makefile:991: recipe for target 'fs' failed
make: *** [fs] Error 2

real	17m33.542s
user	16m10.988s
sys	0m53.000s


      Ok as of 2017-12-08 and a few days ago Ted realizing that the `make menuconfig` type call to configure a Linux kernel for a RaspberryPi/Arm target architecture must be of the form: 

         $ make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- menuconfig

      Now again calling the version of this command with options to compile kernel, device tree binary, and modules: 

         $ make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- zImage modules dtbs


      ^ edit point - further references for kernel configuration wrt RaspberryPi target

      Ted found these references in the README.md file of QEMU Rpi Tools project. Local directory is qemu-rpi-kernel/. Invoking `git status .` shows this to be a git repository. Invoking `git remote -v` shows this repository to be a clone of: 

      ted@localhost:~/projects/qemu/qemu-rpi-kernel$ git remote -v
origin  https://github.com/dhruvvyas90/qemu-rpi-kernel.git (fetch)
origin  https://github.com/dhruvvyas90/qemu-rpi-kernel.git (push)
ted@localhost:~/projects/qemu/qemu-rpi-kernel$ #


      In the READ.md file the Rpi kernel project author who doesn't identify him or herself lists the following four references together, and there are a couple more refs beyond these mentioned near the end of READ.md: 

      https://web.archive.org/web/20131210001638/http://xecdesign.com/compiling-a-kernel/
https://web.archive.org/web/20131209235952/http://xecdesign.com/compiling-qemu/
https://web.archive.org/web/20131210001407/http://xecdesign.com/working-with-qemu/
https://web.archive.org/web/20131210001526/http://xecdesign.com/qemu-emulating-raspberry-pi-the-easy-way/



      ^ Rpi kernel configuration further references



      ^ general kernel configuration and compile reference

      - 2017-12-05 - Ted adding section on general articles, some non-Rpi articles and reference materials which explain how to configure a kernel and then how to invoke the build process and point `make` to the new custom configuration file. This so far not clear from the instructions at Raspberry Pi's articles https://www.raspberrypi.org/documentation/linux/kernel/building.md and https://www.raspberrypi.org/documentation/linux/kernel/configuring.md. These references found at Google via search phrase "invoking make after custom kernel configuration":




      ^ Buildroot

      Per Medicine Yeh's tutorial setting out to compile and use buildroot project, to create Linux based root file system to run under QEMU. Mid-way messages from `make` call on buildroot project downloaded from git repository: 

      /usr/bin/install -m 0644 support/misc/target-dir-warning.txt /var/local/ted/projects-on-slash-var/buildroot/buildroot/output/target/THIS_IS_NOT_YOUR_ROOT_FILESYSTEM
>>> skeleton-init-sysv  Extracting
>>> skeleton-init-sysv  Patching
>>> skeleton-init-sysv  Configuring
>>> skeleton-init-sysv  Building
>>> skeleton-init-sysv  Installing to target
rsync -a --ignore-times --exclude .svn --exclude .git --exclude .hg --exclude .bzr --exclude CVS --chmod=u=rwX,go=rX --exclude .empty --exclude '*~' package/skeleton-init-sysv//skeleton/ /var/local/ted/projects-on-slash-var/buildroot/buildroot/output/target/
>>> skeleton  Extracting
>>> skeleton  Patching
>>> skeleton  Configuring
>>> skeleton  Building
>>> skeleton  Installing to target
>>> linux-headers 4.14.4 Downloading
--2017-12-09 23:11:28--  https://cdn.kernel.org/pub/linux/kernel/v4.x/linux-4.14.4.tar.xz
Resolving cdn.kernel.org (cdn.kernel.org)... 151.101.53.176, 2a04:4e42:d::432
Connecting to cdn.kernel.org (cdn.kernel.org)|151.101.53.176|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 100784788 (96M) [application/x-xz]
Saving to: ‘/var/local/ted/projects-on-slash-var/buildroot/buildroot/output/build/.linux-4.14.4.tar.xz.VsSH6c/output’


      Tail message from `make` on buildroot project: 

      chmod a+x /var/local/ted/projects-on-slash-var/buildroot/buildroot/output/build/buildroot-fs/fakeroot.fs
rm -f /var/local/ted/projects-on-slash-var/buildroot/buildroot/output/target/THIS_IS_NOT_YOUR_ROOT_FILESYSTEM
PATH="/var/local/ted/projects-on-slash-var/buildroot/buildroot/output/host/bin:/var/local/ted/projects-on-slash-var/buildroot/buildroot/output/host/sbin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/sbin:/usr/sbin:/opt/cross/bin:/opt/cross/x-tools/arm-unknown-linux-gnueabi/bin:/home/ted/bin:/usr/local/mysql/bin:/usr/lib/xscreensaver:/etc/init.d:/var/opt/sam-ba_cdc_cdc_linux" /var/local/ted/projects-on-slash-var/buildroot/buildroot/output/host/bin/fakeroot -- /var/local/ted/projects-on-slash-var/buildroot/buildroot/output/build/buildroot-fs/fakeroot.fs
rootdir=/var/local/ted/projects-on-slash-var/buildroot/buildroot/output/target
table='/var/local/ted/projects-on-slash-var/buildroot/buildroot/output/build/buildroot-fs/device_table.txt'
/usr/bin/install -m 0644 support/misc/target-dir-warning.txt /var/local/ted/projects-on-slash-var/buildroot/buildroot/output/target/THIS_IS_NOT_YOUR_ROOT_FILESYSTEM

real    48m56.055s
user    53m27.352s
sys     4m47.896s
ted@rangari:/var/local/ted/projects/buildroot/buildroot$



      - - - top of page - - -


Retrieved from "https://wiki.neelanurseries.com/index.php?title=Nn_embedded_linux_notes&oldid=529"