Difference between revisions of "Nn embedded linux notes"

From Wiki at Neela Nurseries
Jump to: navigation, search
m (→‎^ edit point - drhuvvyas and bcm2709_defconfig comparison)
m (→‎↑ ↓ Extra Notes on QEMU)
 
(79 intermediate revisions by the same user not shown)
Line 3: Line 3:
  
  
−
<font size="3">Embedded Linux Notes<br /></font>
+
<font size="4">Embedded Linux Notes<br /></font>
 +
primarily Linux kernel config to target Raspberry Pi board<br />
 
this wiki article created 2017-12-11 Monday<br />
 
this wiki article created 2017-12-11 Monday<br />
 
</center>
 
</center>
 +
 +
 +
 +
<b>Overview</b>
 +
 +
This wiki document is a collection of Ted's notes on embedded Linux studies, with a starting point of building a recipe for a Linux system to run on a [https://en.wikipedia.org/wiki/Raspberry_Pi#Specifications RaspberryPi 2] target hardware platform.  Near term goal is to build a customized Linux 4.x kernel and test that it runs on a RaspberryPi dev board.  Long term goal has two parts:  &nbsp;(1)  build a more useful Linux system with working C and C++ toolchain, networking stack, web server, e-mail and database server support, an X server or comparable graphics and desktop support, and  &nbsp;(2)  target other hardware platforms beyond RaspberryPi 2 board.
 +
 +
- 2017-12-17 - Sunday evening, Ted noting that working kernel of drhuvvyas90 based on Linux kernel 4.4.34, while our kernel sources are at release 4.4.50.
 +
 +
- 2018-01-08 - Monday evening, only a single instance of QEMU, a specific Linux kernel and jessie-stretch-lite image boot to a prompt and usable system.  Ted adding [[#.5E_What_Works|'what works']] section . . .
  
  
Line 13: Line 24:
  
  
−
<b>Overview</b>
+
=== [[#top|&uarr;]] [[#references|&darr;]] Rpi project commands quick reference ===
 +
 
 +
Here are some commands we're using often to configure and to compile the Linux kernel for RaspberryPi 2 target board:
 +
 
 +
<pre>
 +
 
 +
# Run interactive ncurses-based kernel config script:
 +
 
 +
    $ make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- menuconfig
 +
 
 +
 
 +
# Run kernel config script pulling up latest config to modify a few things:
 +
 
 +
    $ make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- menuconfig oldconfig
 +
 
 +
 
 +
# Build kernel, kernel modules and device tree binary as per RaspberryPi kernel compilation tutorial:
 +
 
 +
    $ make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- zImage modules dtbs
 +
 
 +
 
 +
# Same command as previous but use two processor cores on host system and capture time taken to compile everything:
 +
 
 +
    $ time { make -j 2 ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- zImage modules dtbs; }
 +
 
 +
</pre>
 +
 
 +
 
 +
 
 +
Same commands in shorter form, culled from history:
 +
 
 +
<pre>
 +
  959  make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- mrproper
 +
  960  make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- bcm2835_defconfig
 +
  961  make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- menuconfig oldconfig
 +
  962  time { make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- zImage modules dtbs; }
 +
</pre>
 +
 
 +
Noting here also that on most Linux systems the command <code>nproc</code> gives the number of processor cores available to run tasks in parallel.  The shell phrase <code>-j $(nproc)</code> can be inserted in calls to make, to speed compilation of large projects like compilation of the Linux kernel.  Noting also however that some projects specifically advice against compiling project elements (source files) in parallel, as there are sometimes dependency issues and problems which arise from this type of parallel tasking.
 +
 
 +
 
 +
 
 +
<!-- comment -->
 +
 
 +
=== [[#top|&uarr;]] [[#references|&darr;]] Goals And Present Steps To Take ===
 +
 
 +
- 2017-12-17- At this point Ted looking at kernel configurations for drhuvvyas90's Jessie 4.4.34 kernel for Rpi, versus configuration settings enabled by bcm2709_defconfig.  Working through exercise to [[drhuvvyas90-4-4-34-kconfig-versus-bnc2709-kconfig|adjust bcm2709_defconfig settings]] ten at a time, to sync them with the 4.4.34 config which fully boots in QEMU.  There are hundreds of kernel options so changing one at a time and test running the kernel via a QEMU invocation is impractically slow.  Starting out with ten changes at a time and seeing how that goes . . .
 +
 
 +
 
 +
 
 +
<!-- comment -->
  
−
This wiki document is a collection of Ted's notes on embedded Linux studies, with a starting point of building a recipe for a Linux system to run on a [https://en.wikipedia.org/wiki/Raspberry_Pi#Specifications RaspberryPi 2] target hardware platform.  Near term goal is to build a customized Linux 4.x kernel and test that it runs on a RaspberryPi dev board.  Long term goal has two parts:  &nbsp;(1)  build a more useful Linux system with working C and C++ toolchain, networking stack, web server, e-mail and database server support, an X server or comparable graphics and desktop support, and  &nbsp;(2)  target other hardware platforms beyond RaspberryPi 2 board.
+
=== [[#top|&uarr;]] [[#references|&darr;]] These Rpi QEMU notes rough!!! ===
  
 
As of 2017-12-11 these notes are a mess, so Ted reviewing work so far and here starting summary of what's important, what has worked, what best steps known today to take going forward.  First the major pieces we are finding and using:
 
As of 2017-12-11 these notes are a mess, so Ted reviewing work so far and here starting summary of what's important, what has worked, what best steps known today to take going forward.  First the major pieces we are finding and using:
Line 47: Line 108:
 
<!-- comment -->
 
<!-- comment -->
  
−
== [[#top|^]] New section - Trouble points ==
+
== [[#top|&uarr;]] [[#references|&darr;]] Trouble points ==
  
  
Line 190: Line 251:
  
 
- 2017-12-14 . . . Ok looks like the file <code>./linux/arch/arm/mm/Kconfig</code> must be patched in order to avoid the [[#arm926_dma_clean_range|two kernel compile errors]] we see today and saw yesterday.
 
- 2017-12-14 . . . Ok looks like the file <code>./linux/arch/arm/mm/Kconfig</code> must be patched in order to avoid the [[#arm926_dma_clean_range|two kernel compile errors]] we see today and saw yesterday.
 +
 +
 +
 +
<!-- comment -->
 +
 +
 +
 +
== [[#top|&uarr;]] [[#references|&darr;]] Raspberry Pi hardware ==
 +
 +
 +
Quoting from the forum post at:
 +
 +
*  https://raspberrypi.stackexchange.com/questions/67705/raspberry-pi-2-revisions
 +
 +
<i>
 +
<pre>
 +
"To check what revision you have open a terminal window and run:
 +
 +
cat /proc/cpuinfo
 +
 +
You should get an output that looks like the following:
 +
 +
Processor      : ARMv6-compatible processor rev 7 (v6l)
 +
BogoMIPS        : 847.05
 +
Features        : swp half thumb fastmult vfp edsp java tls
 +
CPU implementer : 0x41
 +
CPU architecture: 7
 +
CPU variant    : 0x0
 +
CPU part        : 0xb76
 +
CPU revision    : 7
 +
Hardware        : BCM2708
 +
Revision        : 0002
 +
Serial          : 000000000abc0ab1"
 +
</pre>
 +
</i>
 +
 +
 +
A related forum post which compares RaspberryPi 2 revision 1.1 and revision 1.2:
 +
 +
*  https://www.raspberrypi.org/forums/viewtopic.php?f=63&t=162139
 +
  
  
 
<!-- comment -->
 
<!-- comment -->
  
−
== [[#top|^]] RaspberryPi Kernel Compilation ==
+
== [[#top|&uarr;]] [[#references|&darr;]] RaspberryPi Kernel Compilation ==
  
 
<br />
 
<br />
Line 372: Line 474:
  
  
−
=== <i>[[#top|^]] edit point - custom Rpi-targeted kernel config issues</i> ===
+
=== <i>[[#top|&uarr;]] [[#references|&darr;]] edit point - custom Rpi-targeted kernel config issues</i> ===
  
  
Line 433: Line 535:
 
     $ make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- zImage modules dtbs
 
     $ make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- zImage modules dtbs
  
−
=== <i>[[#top|^]] RaspberryPi kernel configuration references </i> ===
+
=== <i>[[#top|&uarr;]] [[#references|&darr;]] RaspberryPi kernel configuration references </i> ===
  
  
Line 460: Line 562:
 
<!-- comment -->
 
<!-- comment -->
  
−
=== [[#top|^]] Finding kernel version ===
+
=== [[#top|&uarr;]] [[#references|&darr;]] Finding kernel version ===
  
  
Line 481: Line 583:
 
<!-- comment -->
 
<!-- comment -->
  
−
=== [[#top|^]] Main Line Kernel versus RaspberryPi Kernel Project ===
+
=== [[#top|&uarr;]] [[#references|&darr;]] Main Line Kernel versus RaspberryPi Kernel Project ===
  
 
A number of the latest kernel releases, both stable and testing releases are not yet supported by the Raspberry Pi team -- if there is a team -- which develops the patches and detailed necessary configurations to build 4.x Linux kernels for the fullest Rpi board support possible:
 
A number of the latest kernel releases, both stable and testing releases are not yet supported by the Raspberry Pi team -- if there is a team -- which develops the patches and detailed necessary configurations to build 4.x Linux kernels for the fullest Rpi board support possible:
Line 493: Line 595:
 
<!-- comment -->
 
<!-- comment -->
  
−
=== [[#top|^]] general kernel configuration and compile reference ===
+
=== [[#top|&uarr;]] [[#references|&darr;]] general kernel configuration and compile reference ===
  
 
- 2017-12-05 - Ted adding section on general articles, some non-Rpi articles and reference materials which explain how to configure a kernel and then how to invoke the build process and point `make` to the new custom configuration file.  This so far not clear from the instructions at Raspberry Pi's articles https://www.raspberrypi.org/documentation/linux/kernel/building.md and  https://www.raspberrypi.org/documentation/linux/kernel/configuring.md.  These references found at Google via search phrase "invoking make after custom kernel configuration":
 
- 2017-12-05 - Ted adding section on general articles, some non-Rpi articles and reference materials which explain how to configure a kernel and then how to invoke the build process and point `make` to the new custom configuration file.  This so far not clear from the instructions at Raspberry Pi's articles https://www.raspberrypi.org/documentation/linux/kernel/building.md and  https://www.raspberrypi.org/documentation/linux/kernel/configuring.md.  These references found at Google via search phrase "invoking make after custom kernel configuration":
Line 508: Line 610:
 
<!-- comment -->
 
<!-- comment -->
  
−
=== [[#top|^]] Finding kernel configurations - extract-ikconfig and flag CONFIG_IKCONFIG ===
+
=== [[#top|&uarr;]] [[#references|&darr;]] Finding kernel configurations - extract-ikconfig and flag CONFIG_IKCONFIG ===
  
 
A booted Linux kernel which exports its configuration settings to <code>/proc/config.gz</code> can be studied in detail from its configuration standpoint.  Tools to use in this case include <code>fdisk</code> on the unmounted root file system which got booted up to a working system with the given kernel.  During that run session the kernel will populate <code>/proc/config.gz</code>.  On the unmounted root file system image use <code>fdisk</code> to determine partition count, file system type and block size, and starting block of the rootfs.  Then mount that partition within the rootfs image file with an offset equal to (block size * starting block).  Example of this at Azeria Labs RaspberryPi under Qemu tutorial.
 
A booted Linux kernel which exports its configuration settings to <code>/proc/config.gz</code> can be studied in detail from its configuration standpoint.  Tools to use in this case include <code>fdisk</code> on the unmounted root file system which got booted up to a working system with the given kernel.  During that run session the kernel will populate <code>/proc/config.gz</code>.  On the unmounted root file system image use <code>fdisk</code> to determine partition count, file system type and block size, and starting block of the rootfs.  Then mount that partition within the rootfs image file with an offset equal to (block size * starting block).  Example of this at Azeria Labs RaspberryPi under Qemu tutorial.
Line 533: Line 635:
 
Ok we enabled one new kernel configuration as a built-in part of the kernel rather than its apparent default module [M] setting from the bcm2709_defconfig settings.  Recompiled the rpi-4.4.y kernel sources and in a short time just a minute or so have a kernel from which we can extract full configuration using <code>extract-ikconfig</code>.  We now have two configuration summary files, one from drhuvvyas90's 4.4.34 kernel for QEMU and Rpi, and one from our kernel based on the rpi-4.4.y branch of RaspberryPi kernel project on Github.  The files are a couple thousand lines long or longer . . . it may be also that some kernel config flags in drhuvvyas90's project are not found in . . .
 
Ok we enabled one new kernel configuration as a built-in part of the kernel rather than its apparent default module [M] setting from the bcm2709_defconfig settings.  Recompiled the rpi-4.4.y kernel sources and in a short time just a minute or so have a kernel from which we can extract full configuration using <code>extract-ikconfig</code>.  We now have two configuration summary files, one from drhuvvyas90's 4.4.34 kernel for QEMU and Rpi, and one from our kernel based on the rpi-4.4.y branch of RaspberryPi kernel project on Github.  The files are a couple thousand lines long or longer . . . it may be also that some kernel config flags in drhuvvyas90's project are not found in . . .
  
−
=== [[#top|^]] <i>edit point - drhuvvyas and bcm2709_defconfig comparison </i> ===
+
=== [[#top|&uarr;]] [[#references|&darr;]] <i>edit point - drhuvvyas and bcm2709_defconfig comparison </i> ===
  
 
This is a brute force approach to finding the key differences between two kernel configs, the first one which boots to a fully operational Linux system in QEMU versus a second kernel which fails to boot.  First Ted noting the config file sections in each kernel configuration in next table just below these sentences.  Easy to see that bcm2709_defconfig enables many more kernel options, and an alternate machine type compared with drhuvvyas90's pre-built rpi kernel for QEMU:
 
This is a brute force approach to finding the key differences between two kernel configs, the first one which boots to a fully operational Linux system in QEMU versus a second kernel which fails to boot.  First Ted noting the config file sections in each kernel configuration in next table just below these sentences.  Easy to see that bcm2709_defconfig enables many more kernel options, and an alternate machine type compared with drhuvvyas90's pre-built rpi kernel for QEMU:
Line 550: Line 652:
 
| bgcolor="cce0ff" valign="top" | bcm2709_defconfig
 
| bgcolor="cce0ff" valign="top" | bcm2709_defconfig
 
| bgcolor="cce0ff" valign="top" |
 
| bgcolor="cce0ff" valign="top" |
−
|-
 
 
|-
 
|-
 
| bgcolor="e6e6e6" valign="top" |
 
| bgcolor="e6e6e6" valign="top" |
Line 580: Line 681:
  
 
758:# Network testing . . . . . . . . . . . . . . . . . . .
 
758:# Network testing . . . . . . . . . . . . . . . . . . .
 +
  
  
Line 591: Line 693:
 
810:# Bus devices
 
810:# Bus devices
 
822:# User Modules And Translation Layers
 
822:# User Modules And Translation Layers
−
867:# Mapping drivers for chip access636:# Xtables matches
+
867:# Mapping drivers for chip access
 
758:# Network testing
 
758:# Network testing
 
784:# Device Drivers
 
784:# Device Drivers
Line 600: Line 702:
 
877:# Self-contained MTD device drivers
 
877:# Self-contained MTD device drivers
 
886:# Disk-On-Chip Device Drivers
 
886:# Disk-On-Chip Device Drivers
−
924:# Misc devices
+
924:# Misc devices . . . . . . . . . . . . . . . . . . . .
 
960:# Texas Instruments shared transport line discipline
 
960:# Texas Instruments shared transport line discipline
 
966:# Altera FPGA firmware download module
 
966:# Altera FPGA firmware download module
Line 633: Line 735:
 
822:# User Modules And Translation Layers
 
822:# User Modules And Translation Layers
 
867:# Mapping drivers for chip access
 
867:# Mapping drivers for chip access
−
877:# Self-contained MTD device drivers
 
−
886:# Disk-On-Chip Device Drivers
 
−
924:# Misc devices
 
−
960:# Texas Instruments shared transport line discipline
 
−
966:# Altera FPGA firmware download module
 
−
971:# Intel MIC Bus Driver
 
−
979:# Intel MIC Host Driver
 
−
983:# Intel MIC Card Driver
 
−
991:# Intel MIC Coprocessor State Management (COSM) Drivers
 
−
1120:# Distributed Switch Architecture drivers
 
−
1281:# Host-side USB support is needed for USB Network Adapter support
 
−
1290:# Enable WiMAX (Networking options) to see the WiMAX drivers
 
−
1298:# Input device support
 
−
1308:# Userland interfaces
 
−
1319:# Input Device Drivers
 
−
1370:# Hardware I/O ports
 
−
1385:# Character devices
 
−
1408:# Serial drivers
 
−
1428:# Non-8250 serial port support
 
−
1508:# External I2C/SMBus adapter drivers
 
−
1514:# Other I2C/SMBus bus drivers
 
−
1540:# Enable PHYLIB and NETWORK_PHY_TIMESTAMPING to see the additional clocks.
 
−
1551:# Memory mapped GPIO drivers
 
−
1594:# Sonics Silicon Backplane
 
−
1600:# Broadcom specific AMBA
 
−
1605:# Multifunction device drivers
 
−
1692:# Graphics support
 
−
1699:# Frame buffer Devices
 
−
1722:# Frame buffer hardware drivers
 
−
1765:# Console display driver support
 
−
1882:# Special HID drivers
 
−
2067:# Virtio drivers
 
−
2073:# Microsoft Hyper-V guest support
 
−
2081:# Hardware Spinlock drivers
 
−
2085:# Clock Source drivers
 
−
2099:# Generic IOMMU Pagetable Support
 
−
2105:# Remoteproc drivers
 
−
2110:# Rpmsg drivers
 
−
2145:# Performance monitor support
 
−
2151:# Android
 
−
2166:# Firmware Drivers
 
−
2171:# File systems
 
−
2208:# Caches
 
−
2229:# Pseudo filesystems
 
−
2355:# Kernel hacking
 
−
2366:# Compile-time checks and compiler options
 
−
2386:# Memory Debugging
 
−
2400:# Debug Lockups and Hangs
 
−
2415:# Lock Debugging (spinlocks, mutexes, etc...)
 
−
2473:# Runtime Testing
 
−
2517:# Security options
 
−
2531:# Crypto core or helper
 
−
2560:# Authenticated Encryption with Associated Data
 
−
2569:# Block modes
 
−
2581:# Hash modes
 
−
2589:# Digest
 
−
2610:# Ciphers
 
−
2630:# Compression
 
−
2640:# Random Number Generation
 
−
2658:# Certificates for signature checking
 
−
2665:# Library routines
 
−
 
−
 
−
 
−
2:# Automatically generated file; DO NOT EDIT.
 
−
3:# Linux/arm 4.4.50 Kernel Configuration
 
−
30:# General setup
 
−
76:# Timers subsystem
 
−
186:# Kernel Performance Events And Counters
 
−
269:# Partition Types
 
−
315:# System Type
 
−
350:# Broadcom BCM2709 Implementations
 
−
355:# Processor Type
 
−
371:# Processor Features
 
−
407:# Bus support
 
−
414:# Kernel Features
 
−
493:# Boot options
 
−
543:# Floating point emulation
 
−
547:# At least one emulation must be selected
 
−
557:# Userspace binary formats
 
−
567:# Power management options
 
−
585:# Networking options
 
−
677:# Core Netfilter Configuration
 
−
726:# Xtables combined modules
 
−
733:# Xtables targets
 
−
760:# Xtables matches
 
−
1765:# Console display driver support
 
−
1882:# Special HID drivers
 
−
2067:# Virtio drivers
 
−
2073:# Microsoft Hyper-V guest support
 
−
2081:# Hardware Spinlock drivers
 
−
2085:# Clock Source drivers
 
−
2099:# Generic IOMMU Pagetable Support
 
−
2105:# Remoteproc drivers
 
−
2110:# Rpmsg drivers
 
−
2145:# Performance monitor support
 
−
2151:# Android
 
−
2166:# Firmware Drivers
 
−
2171:# File systems
 
−
2208:# Caches
 
−
2229:# Pseudo filesystems
 
−
2355:# Kernel hacking
 
−
2366:# Compile-time checks and compiler options
 
−
2386:# Memory Debugging
 
−
2400:# Debug Lockups and Hangs
 
−
2415:# Lock Debugging (spinlocks, mutexes, etc...)
 
−
2473:# Runtime Testing
 
−
2517:# Security options
 
−
2531:# Crypto core or helper
 
−
2560:# Authenticated Encryption with Associated Data
 
−
2569:# Block modes
 
−
2581:# Hash modes
 
−
2589:# Digest
 
−
2610:# Ciphers
 
−
2630:# Compression
 
−
2640:# Random Number Generation
 
−
2658:# Certificates for signature checking
 
−
2665:# Library routines
 
−
 
−
 
−
 
−
2:# Automatically generated file; DO NOT EDIT.
 
−
3:# Linux/arm 4.4.50 Kernel Configuration
 
−
30:# General setup
 
−
76:# Timers subsystem
 
−
186:# Kernel Performance Events And Counters
 
−
269:# Partition Types
 
−
315:# System Type
 
−
350:# Broadcom BCM2709 Implementations
 
−
355:# Processor Type
 
−
371:# Processor Features
 
−
407:# Bus support
 
−
414:# Kernel Features
 
−
493:# Boot options
 
−
543:# Floating point emulation
 
−
547:# At least one emulation must be selected
 
−
557:# Userspace binary formats
 
−
567:# Power management options
 
−
585:# Networking options
 
−
677:# Core Netfilter Configuration
 
−
726:# Xtables combined modules
 
−
733:# Xtables targets
 
−
760:# Xtables matches
 
 
877:# Self-contained MTD device drivers
 
877:# Self-contained MTD device drivers
 
886:# Disk-On-Chip Device Drivers
 
886:# Disk-On-Chip Device Drivers
Line 880: Line 839:
 
1417:# Self-contained MTD device drivers
 
1417:# Self-contained MTD device drivers
 
1428:# Disk-On-Chip Device Drivers
 
1428:# Disk-On-Chip Device Drivers
 +
 +
 +
 +
 +
 +
 
1501:# Misc devices
 
1501:# Misc devices
 
1537:# Texas Instruments shared transport line discipline
 
1537:# Texas Instruments shared transport line discipline
Line 1,031: Line 996:
  
  
 +
<i>Table x - Comparing kernel configurations of drhuvvyas90 and bcm2709_defconfig:</i>
  
−
<i>Table x - Comparing kernel configurations of drhuvvyas90 and bcm2709_defconfig:</i>
 
  
−
<p>
 
 
{| style="width:100%"
 
{| style="width:100%"
 
|-
 
|-
Line 1,233: Line 1,197:
  
 
|}
 
|}
−
</p><!-- bounding paragraph tags don't avert the weird wrapping of next wiki section about this table -->
+
<!-- </p> --><!-- bounding paragraph tags don't avert the weird wrapping of next wiki section about this table -->
  
 
<br /><!-- HTML line break also doesn't avert the wrapping of next wiki section about the inner table -->
 
<br /><!-- HTML line break also doesn't avert the wrapping of next wiki section about the inner table -->
Line 1,249: Line 1,213:
 
<!-- comment -->
 
<!-- comment -->
  
−
=== [[#top|^]] linux/arch/arm/configs and bcm2709_defconfig versus versatile_defconfig ===
+
=== [[#top|&uarr;]] [[#references|&darr;]] linux/arch/arm/configs and bcm2709_defconfig versus versatile_defconfig ===
  
 
Today is 2017-12-13 Wednesday, and yesterday we determined that the one difference between bootable QEMU + kernel + rootfs and non-booting QEMU + kernel + rootfs is the compiled kernel file.  So somethings are wrong with our kernel configuration.  We retrieved the kernel configuration from the drhuvvyas90 pre-compiled Rpi kernel named <code>kernel-qemu-4.4.34-jessie</code> by mounting the "jessie stretch lite" rootfs image from RaspberryPi downloads page, and copying /proc/config.gz from that rootfs.  We could not extract any config information from our compiled kernel, but we likely left disabled a kernel config option mentioned in https://stackoverflow.com/questions/14958192/getting-config-from-linux-kernel-image which would permit running script <code>extract-ikconfig</code> on a compressed kernel image.
 
Today is 2017-12-13 Wednesday, and yesterday we determined that the one difference between bootable QEMU + kernel + rootfs and non-booting QEMU + kernel + rootfs is the compiled kernel file.  So somethings are wrong with our kernel configuration.  We retrieved the kernel configuration from the drhuvvyas90 pre-compiled Rpi kernel named <code>kernel-qemu-4.4.34-jessie</code> by mounting the "jessie stretch lite" rootfs image from RaspberryPi downloads page, and copying /proc/config.gz from that rootfs.  We could not extract any config information from our compiled kernel, but we likely left disabled a kernel config option mentioned in https://stackoverflow.com/questions/14958192/getting-config-from-linux-kernel-image which would permit running script <code>extract-ikconfig</code> on a compressed kernel image.
Line 1,266: Line 1,230:
 
<!-- comment -->
 
<!-- comment -->
  
−
=== [[#top|^]] kernel - specific configuration build results ===
+
=== [[#top|&uarr;]] [[#references|&darr;]] kernel - specific configuration build results ===
  
  
Line 1,299: Line 1,263:
 
user 21m5.491s
 
user 21m5.491s
 
sys 1m21.729s
 
sys 1m21.729s
−
veris@alta-spare-6:/var/local/raspberrypi/linux$
+
 
 +
user@localhost:/var/local/raspberrypi/linux$
 
</pre>
 
</pre>
  
Line 1,319: Line 1,284:
 
user 68m30.589s
 
user 68m30.589s
 
sys 5m9.499s
 
sys 5m9.499s
−
veris@alta-spare-6:/var/local/raspberrypi/rpi-linux$ exit
+
 
 +
user@localhost:/var/local/raspberrypi/rpi-linux$ exit
 
exit
 
exit
 
Script done, file is z--kernel-config-using-bcm2709-defconfig--try-2.txt
 
Script done, file is z--kernel-config-using-bcm2709-defconfig--try-2.txt
−
veris@alta-spare-6:/var/local/raspberrypi/rpi-linux$
+
 
 +
user@localhost:/var/local/raspberrypi/rpi-linux$
 
</pre>
 
</pre>
  
Line 1,330: Line 1,297:
 
<!-- comment -->
 
<!-- comment -->
  
−
== [[#top|^]] Buildroot ==
+
== [[#top|&uarr;]] [[#references|&darr;]] Buildroot ==
 +
 
 +
 
 +
[https://buildroot.org The Buildroot Project] is a many-years standing build system project targeted at embedded Linux system development.  Per [https://buildroot.org/downloads/manual/manual.html Buildroot's documentation] this project shares much of the functionality and capacity of project and toolset known as Yocto, which is also a software project build management system.  Buildroot however [https://buildroot.org/downloads/manual/manual.html#_cross_compilation_toolchain does not support Yocto-generated toolchains], apparently because to quote Buildroot's manual those toolchains contain non- tool chain parts beyond a given compiler, binutils and C and C++ libraries.
 +
 
 +
Before reading far in these notes it is good to know a couple of important design scope matter about the Buildroot scripts and project.  From chapter 10 of the Buildroot stable manual as of 2018 January:
 +
 
 +
 
 +
<b>
 +
*  Buildroot is not designed to create systems with compilers and toolchains on them
 +
*  Buildroot is not a [Linux] distribution generator
 +
</b>
 +
 
 +
 
 +
These points appear in the Frequently Asked Questions section of Buildroot's manual, and nearly all the things which Buildroot is not come out of the small-memory, small hardware resource nature of the embedded systems which Buildroot developers in general work to support.  This said, Buildroot is still a highly useful and configurable tool -- set of scripts and more -- for building small, networked and functional Linux systems.
 +
 
 +
The same chapter 10 of the manual lists some alternative tools and projects to Buildroot, which include in their scope the adding of toolchains to target systems, systems being boot-loader, kernel, rootfs compiled and configured for a target board or hardware.  Some of these alternate tools include [https://www.yoctoproject.org/ Yocto] and [https://www.yoctoproject.org/ Openembedded].
 +
 
 +
 
 +
 
 +
An early search for ways to build a kernel and root filesystem led to [https://medicineyeh.wordpress.com/2016/03/29/buildup-your-arm-image-for-qemu/ Medicine Yeh's tutorial], but following his build steps did not yield any kernel which booted in QEMU.  There are other buildroot projects available on-line and in the public domain, including a [https://github.com/gamaral/rpi-buildroot buildroot Rpi project by Guillermo Amaral] and a [http://www.jumpnowtek.com/rpi/Raspberry-Pi-Systems-with-Buildroot.html buildroot project involving Qt5 support] by Jumpnow Technologies LLC.
 +
 
 +
For now working through Buildroot's on-line manual.
 +
 
 +
 
 +
<b>Click here [[buildroot_early_notes|Early notes related to Buildroot]] for notes from first discovery of [https://medicineyeh.wordpress.com/2016/03/29/buildup-your-arm-image-for-qemu/Medicine Yeh's work].</b>
 +
 
  
−
Per [https://medicineyeh.wordpress.com/2016/03/29/buildup-your-arm-image-for-qemu/ Medicine Yeh's tutorial] setting out to compile and use buildroot project, to create Linux based root file system to run under QEMU.  Mid-way messages from `make` call on buildroot project downloaded from git repository:
 
  
−
<pre>
+
- 2018-01-17 WED -
−
/usr/bin/install -m 0644 support/misc/target-dir-warning.txt /var/local/ted/projects-on-slash-var/buildroot/buildroot/output/target/THIS_IS_NOT_YOUR_ROOT_FILESYSTEM
+
Left this line of study a couple weeks ago, not finding a root filesystem which would boot or allow our QEMU set-up to progress beyond the initial few or no messages, seemingly hanging. Putting this study on hold as we need to understand Buildroot system better. Working through Buildroot manual before returning here.  - TMH
−
>>> skeleton-init-sysv  Extracting
 
−
>>> skeleton-init-sysv  Patching
 
−
>>> skeleton-init-sysv  Configuring
 
−
>>> skeleton-init-sysv  Building
 
−
>>> skeleton-init-sysv  Installing to target
 
−
rsync -a --ignore-times --exclude .svn --exclude .git --exclude .hg --exclude .bzr --exclude CVS --chmod=u=rwX,go=rX --exclude .empty --exclude '*~' package/skeleton-init-sysv//skeleton/ /var/local/ted/projects-on-slash-var/buildroot/buildroot/output/target/
 
−
>>> skeleton  Extracting
 
−
>>> skeleton  Patching
 
−
>>> skeleton  Configuring
 
−
>>> skeleton  Building
 
−
>>> skeleton  Installing to target
 
−
>>> linux-headers 4.14.4 Downloading
 
−
--2017-12-09 23:11:28-- https://cdn.kernel.org/pub/linux/kernel/v4.x/linux-4.14.4.tar.xz
 
−
Resolving cdn.kernel.org (cdn.kernel.org)... 151.101.53.176, 2a04:4e42:d::432
 
−
Connecting to cdn.kernel.org (cdn.kernel.org)|151.101.53.176|:443... connected.
 
−
HTTP request sent, awaiting response... 200 OK
 
−
Length: 100784788 (96M) [application/x-xz]
 
−
Saving to: ‘/var/local/ted/projects-on-slash-var/buildroot/buildroot/output/build/.linux-4.14.4.tar.xz.VsSH6c/output’
 
−
</pre>
 
  
 +
- 2018-01-19 FRI -
 +
Buildroot manual is long, and an important section to remember and return to is titled "[https://buildroot.org/downloads/manual/manual.html#make-tips Make Tips]".  This is section 8 in the stable manual published at [https://buildroot.org Buildroot dot org].  In particular this section talks about how to invoke `make` on a Buildroot project when major attributes such as target platform architecture change.  This manual section worth noting and revisiting.
  
−
Tail message from `make` on buildroot project:
 
  
−
<pre>
+
<!-- comment -->
−
chmod a+x /var/local/ted/projects-on-slash-var/buildroot/buildroot/output/build/buildroot-fs/fakeroot.fs
+
 
−
rm -f /var/local/ted/projects-on-slash-var/buildroot/buildroot/output/target/THIS_IS_NOT_YOUR_ROOT_FILESYSTEM
+
=== [[#top|&uarr;]] [[#references|&darr;]] Buildroot manual sections ===
−
PATH="/var/local/ted/projects-on-slash-var/buildroot/buildroot/output/host/bin:/var/local/ted/projects-on-slash-var/buildroot/buildroot/output/host/sbin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/sbin:/usr/sbin:/opt/cross/bin:/opt/cross/x-tools/arm-unknown-linux-gnueabi/bin:/home/ted/bin:/usr/local/mysql/bin:/usr/lib/xscreensaver:/etc/init.d:/var/opt/sam-ba_cdc_cdc_linux" /var/local/ted/projects-on-slash-var/buildroot/buildroot/output/host/bin/fakeroot -- /var/local/ted/projects-on-slash-var/buildroot/buildroot/output/build/buildroot-fs/fakeroot.fs
 
−
rootdir=/var/local/ted/projects-on-slash-var/buildroot/buildroot/output/target
 
−
table='/var/local/ted/projects-on-slash-var/buildroot/buildroot/output/build/buildroot-fs/device_table.txt'
 
−
/usr/bin/install -m 0644 support/misc/target-dir-warning.txt /var/local/ted/projects-on-slash-var/buildroot/buildroot/output/target/THIS_IS_NOT_YOUR_ROOT_FILESYSTEM
 
  
−
real    48m56.055s
+
[https://buildroot.org/downloads/manual/manual.html#full-rebuild 8.2] &nbsp;Full rebuilds, when and how to do them<br />
−
user    53m27.352s
+
[https://buildroot.org/downloads/manual/manual.html#_offline_builds 8.4] &nbsp;Building off-line<br />
−
sys    4m47.896s
+
8.8 Graphing package dependencies<br />
−
ted@rangari:/var/local/ted/projects/buildroot/buildroot$
+
8.9 Graphing build duration<br />
−
</pre>
+
8.10 Graphing file system size contribution of packages<br />
 +
[https://buildroot.org/downloads/manual/manual.html#_advanced_usage 8.12.5] &nbsp;package-specific make targets<br />
 +
[https://buildroot.org/downloads/manual/manual.html#_advanced_usage 8.12.6] &nbsp;override package source directory to avoid re-downloading sources on every rebuild<br />
  
  
Line 1,377: Line 1,349:
 
<!-- comment -->
 
<!-- comment -->
  
−
== [[#top|^]] Putting The Pieces Together - QEMU, kernel, rootfs ==
+
== [[#top|&uarr;]] [[#references|&darr;]] Putting The Pieces Together - QEMU, kernel, rootfs ==
  
 
<br />
 
<br />
Line 1,413: Line 1,385:
 
<!-- comment -->
 
<!-- comment -->
  
−
== [[#top|^]] QEMU references ==
+
== QEMU results and references ==
 +
 
 +
 
 +
- 2017-12-14 - Here is all we get when building kernel per bcm2709_defconfig recipe:
 +
 
 +
<pre>
 +
ted@rangari:/var/local/ted/projects/raspberrypi/qemu_vms$ ./2.sh
 +
WARNING: Image format was not specified for './2017-09-07-raspbian-stretch-lite.img' and probing guessed raw.
 +
        Automatically detecting the format is dangerous for raw images, write operations on block 0 will be restricted.
 +
        Specify the 'raw' format explicitly to remove the restrictions.
 +
pulseaudio: set_sink_input_volume() failed
 +
pulseaudio: Reason: Invalid argument
 +
pulseaudio: set_sink_input_mute() failed
 +
pulseaudio: Reason: Invalid argument
 +
</pre>
 +
 
 +
and here the command line prompt waits without response to keyboard input.  Qemu's window appears but remains blank and unresponsive too.
 +
 
 +
 
 +
 
 +
- 2017-12-17 - Sunday morning results of calling QEMU with a RaspberryPi kernel 4p4py, configured to default values of versatile_defconfig recipe.  These few lines show slightly further progress than we've gotten from bcm2709_defconfig.  Processor variant settable at ___ . . .
  
−
<br />
 
−
*  https://lists.gnu.org/archive/html/qemu-discuss/2014-09/msg00060.html  . . . example qemu-system-arm invocations
 
 
<pre>
 
<pre>
 +
ted@rangari:/var/local/ted/projects/raspberrypi/qemu_vms$ ./2.sh
 +
WARNING: Image format was not specified for './2017-09-07-raspbian-stretch-lite.img' and probing guessed raw.
 +
        Automatically detecting the format is dangerous for raw images, write operations on block 0 will be restricted.
 +
        Specify the 'raw' format explicitly to remove the restrictions.
 +
pulseaudio: set_sink_input_volume() failed
 +
pulseaudio: Reason: Invalid argument
 +
pulseaudio: set_sink_input_mute() failed
 +
pulseaudio: Reason: Invalid argument
 +
Uncompressing Linux... done, booting the kernel.
  
 +
Error: unrecognized/unsupported processor variant (0x410fb767).
 
</pre>
 
</pre>
 +
 +
 +
 +
<b>QEMU references . . .</b>
 +
 +
<br />
 +
*  https://lists.gnu.org/archive/html/qemu-discuss/2014-09/msg00060.html  . . . example qemu-system-arm invocations
  
 
*  http://www.cnblogs.com/coryxie/p/3329462.html
 
*  http://www.cnblogs.com/coryxie/p/3329462.html
 +
 
<pre>
 
<pre>
 
# qemu-system-arm -kernel arch/arm/boot/zImage -dtb rtsm_ve-cortex_a15x4.dtb -m 512 -M vexpress-a15 -serial stdio -append "console=ttyAMA0"
 
# qemu-system-arm -kernel arch/arm/boot/zImage -dtb rtsm_ve-cortex_a15x4.dtb -m 512 -M vexpress-a15 -serial stdio -append "console=ttyAMA0"
Line 1,430: Line 1,438:
 
<!-- comment -->
 
<!-- comment -->
  
−
== Extra Notes on QEMU ==
+
== [[#top|&uarr;]] [[#references|&darr;]] Extra Notes on QEMU ==
  
 
<br />
 
<br />
Line 1,438: Line 1,446:
  
 
<pre>
 
<pre>
−
$ qemu-system-arm -M versatilepb -cpu arm1176 -m 256 -serial stdio -kernel /var/local/raspberrypi/qemu-rpi-kernel/kernel-qemu-4.4.34-jessiez -dtb /var/local/raspberrypi/linux/arch/arm/boot/dts/bcm2709-rpi-2-b.dtb -append "root=/dev/sda2 rootfstype=ext4 rw init=/bin/bash console=ttyAMA0" -drive format=raw,file=/home/veris/Downloads/raspberrypi/2017-08-16-raspbian-stretch-lite.img
+
$ qemu-system-arm -M versatilepb -cpu arm1176 -m 256 -serial stdio -kernel /var/local/raspberrypi/qemu-rpi-kernel/kernel-qemu-4.4.34-jessiez -dtb /var/local/raspberrypi/linux/arch/arm/boot/dts/bcm2709-rpi-2-b.dtb -append "root=/dev/sda2 rootfstype=ext4 rw init=/bin/bash console=ttyAMA0" -drive format=raw,file=/home/${USER}/Downloads/raspberrypi/2017-08-16-raspbian-stretch-lite.img
  
 
audio: Could not init `oss' audio driver
 
audio: Could not init `oss' audio driver
Line 1,450: Line 1,458:
 
<!-- comment -->
 
<!-- comment -->
  
−
== References ==
+
== [[#top|&uarr;]] [[#references|&darr;]] What Works ==
 +
 
 +
The following qemu invocation works . . .
 +
 
 +
<pre>
 +
#!/bin/bash
 +
 
 +
qemu-system-arm -kernel ./kernel-qemu-4.4.34-jessie -cpu arm1176 -m 256 -M versatilepb -serial stdio -append "root=/dev/sda2 rootfstype=ext4 rw" -hda ./2017-09-07-raspbian-stretch-lite.img -redir tcp:5022::22 -no-reboot
 +
 
 +
exit 0
 +
</pre>
 +
 
 +
 
 +
The 4.4.34 kernel is of size 3024048 bytes and has md5sum <code>f2a26e138923aa87a97cf43ba8f4d584  kernel-qemu-4.4.34-jessie</code>.  The 2017-09-07 image file is of size 1854590976 bytes.  Because this image file contains an entire root file system, its content changes with each login session even when the kernel and the image are exercised in a qemu virtual machine.
 +
 
 +
Note also that the options passed to the kernel via qemu's -append option reflect a change that the lead webmaster and article writer at Azeria Labs described, where one must modify a file in the Raspbian Stretch Lite root filesystem.  That modification is in /etc/fstab.  References to the RaspberryPi default device name /dev/mmcblk0p1 need to be replaced with /dev/sda1.  The final number '1' may vary.
 +
 
 +
The image file <code>2017-09-07-raspbian-stretch-lite.img</code> has a /proc/config.gz file.  If we can extract and copy that file from the file system image, we could use that as a configuration or starting point of configuring a newer release of the Linux kernel . . .
 +
 
 +
 
 +
 
 +
<!-- comment -->
 +
 
 +
== [[#top|&uarr;]] [[#references|&darr;]] Finding Early Kernel Messages ==
 +
 
 +
Out of three hosts the above set of files and call to qemu only boots successfully on one host.  Some invocations have caused qemu to quickly exit, but others have left qemu running with the one message "Uncompressing Linux... done, booting the kernel.".  What is happening in the early kernel boot stages?  It looks like there are problems the kernel encounters or has before it can set up a console.  Ted looking for way to uncompress the compressed part of the kernel image, and then look at kernel symbol table using arm-linux-gnueabi-nm or similar.
 +
 
 +
<pre>
 +
user@localhost:~/projects/raspberrypi/qemu-vms$ binwalk kernel-qemu-4.4.34-jessie
 +
 
 +
DECIMAL      HEXADECIMAL    DESCRIPTION
 +
--------------------------------------------------------------------------------
 +
36            0x24            Linux kernel ARM boot executable zImage (little-endian), load address: "0x00000000", end address: "0x002E24B0"
 +
17576        0x44A8          gzip compressed data, maximum compression, from Unix, NULL date (1970-01-01 00:00:00)
 +
 
 +
user@localhost:~/projects/raspberrypi/qemu-vms$
 +
</pre>
 +
 
 +
 
 +
 
 +
<!-- comment -->
 +
 
 +
== [[#top|&uarr;]] [[#references|&darr;]] Linux System Components - Boot-loader, Kernel, Initramfs, Rootfs ==
 +
 
 +
Purpose of initramfs . . . https://wiki.gentoo.org/wiki/Custom_Initramfs.
 +
 
 +
 
 +
 
 +
 
 +
<span id="references"></span>
 +
 
 +
<!-- comment -->
 +
 
 +
== [[#top|^]] References ==
 +
 
 +
 
 +
*  https://www.raspberrypi.org/documentation/linux/kernel/building.md
  
 
*  http://sysprogs.com/VisualKernel/tutorials/raspberry/buildkernel/
 
*  http://sysprogs.com/VisualKernel/tutorials/raspberry/buildkernel/
 +
 +
 +
- 2017-12-19 -
 +
*  http://intestinate.com/pilfs/ . . . PiLFS - Linux From Scratch on the RaspberryPi (however, no cross-compiling)
 +
 +
*  http://www.jumpnowtek.com/rpi/Raspberry-Pi-Systems-with-Yocto.html
 +
 +
*  https://buildroot.org/download.html
 +
 +
*  https://free-electrons.com/doc/training/buildroot/buildroot-slides.pdf
 +
 +
 +
- 2017-12-21 - Searching for info on installing GRUB or alternate bootloader on Raspberry Pi 2 board . . .
 +
*  https://elinux.org/RPi_U-Boot
 +
 +
 +
- 2018-01-02 -
 +
*  https://wiki.gentoo.org/wiki/Raspberry_Pi/Kernel_Compilation . . . Must use patched Linux kernel from Raspberry Pi Foundation!
 +
*  http://sysprogs.com/VisualKernel//tutorials/raspberry/buildkernel/ . . . download Rpi patched kernel and Rpi tools, including cross-compiler
 +
*  https://wiki.linaro.org/PeterMaydell/QemuVersatileExpress . . . extract kernel and initrd from image file
 +
 +
- 2018-01-03 -
 +
*  https://www.raspberrypi.org/documentation/configuration/cmdline-txt.md . . . the kernel command line
 +
 +
- 2018-01-08 -
 +
*  http://billauer.co.il/blog/2014/02/uboot-linux-dtb-fdt-device-tree/ . . . U-boot and how kernel finds device tree blob, .dtb file
 +
*  https://developer.ridgerun.com/wiki/index.php/Common_Problems_-_Linux_kernel_doesnt_boot . . . U-boot can send machine ID number to kernel
 +
 +
- 2018-01-09 -
 +
*  https://www.raspberrypi.org/blog/raspbian-stretch/ . . . Raspbian "Stretch" operating system release announcement
 +
*  https://lists.gnu.org/archive/html/qemu-devel/2015-03/msg02830.html . . . qemu error "-serial stdio: cannot use stdio by multiple character devices", amend qemu's options with:
 +
-nographic -monitor none
 +
*  https://sthbrx.github.io/blog/2017/01/30/extracting-early-boot-messages-in-qemu/ . . . use qemu to examine kernel log buffer <code>__log_buf</code> for early kernel messages
 +
*  https://reverseengineering.stackexchange.com/questions/14587/decompressing-zimage . . . utilities binwalk and extract-vmlinux
 +
*  http://www.isysop.com/unpacking-and-repacking-u-boot-uimage-files/ . . . use of 'dd' to extract images within kernel image file
 +
*  https://tools.kali.org/forensics/binwalk . . . binwalk good for studying firmware image files
 +
 +
- 2018-01-10 -
 +
*  https://github.com/gamaral/rpi-buildroot Gamaral (Guillermo Amaral buildroot for RaspberryPi board . . . builds and works!
 +
*  https://github.com/gamaral/rpi-buildroot/tree/rpi/board/raspberrypi . . . genimage-*.cfg files, can these be written from scratch with enough target knowledge?
 +
 +
- 2018-01-14 -
 +
*  http://www.jumpnowtek.com/rpi/Raspberry-Pi-Systems-with-Buildroot.html RaspberryPi and Qt5 blog post of JumpNowTek
 +
*  http://doc.qt.io/qt-5/embedded-linux.html Qt5 embedded , Qt embedded
 +
*  http://www.jumpnowtek.com/ Jumpnowtek dot com embedded design and engineering
 +
 +
*  https://en.wikipedia.org/wiki/UClibc . . . uClibc small C library , standard C language library small footprint
 +
 +
  
  
  
 
<!-- comment -->
 
<!-- comment -->
 +
 +
 +
  
 
<center>
 
<center>
−
[[#top|- - - top of page - - -]]
+
[[#top|- - - top of page - - -]]<span id="bottom"></span>
 
</center>
 
</center>
−
 
  
 
<!-- EOF -->
 
<!-- EOF -->

Latest revision as of 19:50, 26 July 2018

Wiki Main Page | User Page of Ted | Google search engine


Embedded Linux Notes
 primarily Linux kernel config to target Raspberry Pi board
this wiki article created 2017-12-11 Monday


Overview

This wiki document is a collection of Ted's notes on embedded Linux studies, with a starting point of building a recipe for a Linux system to run on a RaspberryPi 2 target hardware platform. Near term goal is to build a customized Linux 4.x kernel and test that it runs on a RaspberryPi dev board. Long term goal has two parts:  (1) build a more useful Linux system with working C and C++ toolchain, networking stack, web server, e-mail and database server support, an X server or comparable graphics and desktop support, and  (2) target other hardware platforms beyond RaspberryPi 2 board.

- 2017-12-17 - Sunday evening, Ted noting that working kernel of drhuvvyas90 based on Linux kernel 4.4.34, while our kernel sources are at release 4.4.50.

- 2018-01-08 - Monday evening, only a single instance of QEMU, a specific Linux kernel and jessie-stretch-lite image boot to a prompt and usable system. Ted adding 'what works' section . . .



↑ ↓ Rpi project commands quick reference

Here are some commands we're using often to configure and to compile the Linux kernel for RaspberryPi 2 target board: 


# Run interactive ncurses-based kernel config script:

    $ make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- menuconfig


# Run kernel config script pulling up latest config to modify a few things:

    $ make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- menuconfig oldconfig


# Build kernel, kernel modules and device tree binary as per RaspberryPi kernel compilation tutorial:

    $ make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- zImage modules dtbs


# Same command as previous but use two processor cores on host system and capture time taken to compile everything:

    $ time { make -j 2 ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- zImage modules dtbs; }


Same commands in shorter form, culled from history: 

  959  make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- mrproper
  960  make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- bcm2835_defconfig
  961  make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- menuconfig oldconfig
  962  time { make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- zImage modules dtbs; }

Noting here also that on most Linux systems the command nproc gives the number of processor cores available to run tasks in parallel. The shell phrase -j $(nproc) can be inserted in calls to make, to speed compilation of large projects like compilation of the Linux kernel. Noting also however that some projects specifically advice against compiling project elements (source files) in parallel, as there are sometimes dependency issues and problems which arise from this type of parallel tasking.



↑ ↓ Goals And Present Steps To Take

- 2017-12-17- At this point Ted looking at kernel configurations for drhuvvyas90's Jessie 4.4.34 kernel for Rpi, versus configuration settings enabled by bcm2709_defconfig. Working through exercise to adjust bcm2709_defconfig settings ten at a time, to sync them with the 4.4.34 config which fully boots in QEMU. There are hundreds of kernel options so changing one at a time and test running the kernel via a QEMU invocation is impractically slow. Starting out with ten changes at a time and seeing how that goes . . .



↑ ↓ These Rpi QEMU notes rough!!!

As of 2017-12-11 these notes are a mess, so Ted reviewing work so far and here starting summary of what's important, what has worked, what best steps known today to take going forward. First the major pieces we are finding and using:

  • cross-compiling toolchain, e.g. Linaro gcc
  • QEMU system hardware emulator

The cross-compiler and related tools we use to build our kernel and 'target' a.k.a. 'guest' system softwares. The guest system in our case is a Linux kernel and additional programs compiled to run together as a complete system on an ARM based processor, in the target hardware which is the RaspberryPi 2 board. We cross compile in order to get the much faster compilation times available on a typical cerca 2010 Intel desktop computer. The QEMU software allows us to experiment, jumping through some hoops, with our ARM-targeted kernels and softwares, without a physical RaspberryPi board present. We also get to skip the step of creating a system image and flashing that to SD card. Knowing how to create an SD-card-ready image, however, is important. We'll return to this we hope sooner than later, once we have seen our cross-compiled system pieces run in normal expected ways in QEMU's target hardware emulating environment.

Our target system software pieces for a basic Linux operating system include:

  • Linux kernel 4.x
  • device tree binary (dtb file)
  • kernel modules
  • root file system

Kernel, device trees and kernel modules are all part of the official operating system space. These three pieces get compiled when configuring Linux kernel for a RaspberryPi target board, using the kernel's arch/arm/configs/bcm2709_defconfig default configuration file. Our needed root file system is a separate matter, which we must built separately. We also have a number of options for how to configure and specify what we want to build in this file system.

As of this Monday, Ted finds good notes at a page of Azeria Labs and also a post by graduate student MedicineYeh. Ted finds a little more success from Azeria Labs article, getting QEMU invocations to bring up an already-compiled QEMU Linux kernel for ARM. But Ted's custom compiled 4.x kernels haven't booted yet, and no diagnostics messages visible save four lines relating to some kind of sound card module. MedicineYeh's blog post goes into more details about root file systems and a couple of ways to build them quickly for basic testing and practice, but so far none of his example calls to QEMU have worked. These all hang and time-out without any diagnostics feedback. MedicineYeh's blog post seems to imply in its QEMU output captures that QEMU can put guest system message to the shell which invoked QEMU. But it's not clear to Ted how this would happen, and it doesn't appear to happen by default. QEMU will present the ARM-targeted kernel with a device tree that's expressed in the device tree binary file that the Linux kernel arch' ARM default configuration describes somewhere under the hood . . . that device tree is different and separate from the host's device tree, including the host system's.

Regarding QEMU passing guest kernel message to standard out, there's mention in Azeria Labs article about QEMU option and argument -serial stdio. Ted to try this once he has kernel, dtb file, kernel modules and rootfs all cross-compiled for ARM on Ubuntu host . . .




↑ ↓ Trouble points

Two trouble points as of 2017-12-11 Monday. One, we're not seeing guest system messages in any of invoking shell, QEMU window, nor Remmina VNC client window. Ted wants to see guest system diagnostics at QEMU-invoking shell prompt. Two, Ted unsure whether we're mounting a rootfs properly in our call to QEMU.


Figure x - problems building and running Linux for ARM under QEMU

  • guest kernel diagnostics not visible
  • questions about passing rootfs to QEMU properly


- 2017-12-14 THU -

Over the past couple days Ted found that most call to QEMU are passing the same parameters, and between the working qemu invocation on hand and a non-working one, narrowed the differing piece (argument to QEMU) down to the kernel. Ted's compiled kernels don't boot up, and give few or no diagnostics. The pre-compiled RPi kernels of drhuvvyas90 boot fully. Ted noting also that drhuvvyas90 supplies a kernel config-and-build script along side the pre-compiled kernels, in a diretory named 'tools'. Ted has tried to execute this script on 2017-12-13 Wednesday and 2017-12-14 Thursday but script fails at the point of applying patches to multiple Kconfig files among the Linux kernel's supported processor and system architectures. Here is the latest output from script build-kernel-qemu: 

# In /var/local/raspberrypi/qemu-rpi-kernel/tools, where qemu-rpi-kernel is a cloned git repository:

$ ./build-kernel-qemu 
Cloning into 'linux'...
p11-kit: invalid config filename, will be ignored in the future: /etc/pkcs11/modules/gnome-keyring-module
WARNING: gnome-keyring:: couldn't connect to: /run/user/1000/keyring-cUT3JX/pkcs11: No such file or directory
p11-kit: skipping module 'gnome-keyring-module' whose initialization failed: An error occurred on the device
remote: Counting objects: 6064925, done.
remote: Compressing objects: 100% (1312/1312), done.
remote: Total 6064925 (delta 1730), reused 1519 (delta 1240), pack-reused 6062373
Receiving objects: 100% (6064925/6064925), 1.69 GiB | 2.27 MiB/s, done.
Resolving deltas: 100% (5054327/5054327), done.
Checking connectivity... done.
Checking out files: 100% (56781/56781), done.
patching file arch/arm/mach-versatile/Kconfig
Hunk #1 FAILED at 4.
Hunk #2 FAILED at 12.
2 out of 2 hunks FAILED -- saving rejects to file arch/arm/mach-versatile/Kconfig.rej
patching file arch/arm/mm/Kconfig
Hunk #1 FAILED at 71.
Hunk #2 FAILED at 89.
Hunk #3 FAILED at 127.
Hunk #5 FAILED at 359.
4 out of 5 hunks FAILED -- saving rejects to file arch/arm/mm/Kconfig.rej
patching file drivers/mmc/host/Kconfig
Hunk #1 FAILED at 6.
1 out of 1 hunk FAILED -- saving rejects to file drivers/mmc/host/Kconfig.rej

$

Ted may have to forego any automated RPi kernel configuration in drhuvvyas90's helper script, but before moving on here are the contents of the three mentioned patch rejection files: 

# In file ../qemu-rpi-kernel/tools/linux/arch/arm/mach-versatile/Kconfig.rej

--- arch/arm/mach-versatile/Kconfig
+++ arch/arm/mach-versatile/Kconfig
@@ -4,7 +4,6 @@ menu "Versatile platform type"
 config ARCH_VERSATILE_PB
        bool "Support Versatile Platform Baseboard for ARM926EJ-S"
        default y
-       select CPU_ARM926T
        select MIGHT_HAVE_PCI
        help
          Include support for the ARM(R) Versatile Platform Baseboard
@@ -12,7 +11,6 @@ config ARCH_VERSATILE_PB

 config MACH_VERSATILE_AB
        bool "Support Versatile Application Baseboard for ARM926EJ-S"
-       select CPU_ARM926T
        help
          Include support for the ARM(R) Versatile Application Baseboard
          for the ARM926EJ-S.



In file "linux/arch/arm/mm/Kconfig.rej" 38L, 1463C

--- arch/arm/mm/Kconfig
+++ arch/arm/mm/Kconfig
@@ -71,7 +71,7 @@ config CPU_ARM9TDMI

 # ARM920T
 config CPU_ARM920T
-       bool "Support ARM920T processor" if (ARCH_MULTI_V4T && ARCH_INTEGRATOR)
+       bool "Support ARM920T processor" if ARCH_INTEGRATOR || ARCH_VERSATILE_PB || ARCH_VERSATILE_AB
        select CPU_32v4T
        select CPU_ABRT_EV4T
        select CPU_CACHE_V4WT
@@ -89,7 +89,7 @@ config CPU_ARM920T

 # ARM922T
 config CPU_ARM922T
-       bool "Support ARM922T processor" if (ARCH_MULTI_V4T && ARCH_INTEGRATOR)
+       bool "Support ARM922T processor" if ARCH_INTEGRATOR || ARCH_VERSATILE_PB || ARCH_VERSATILE_AB
        select CPU_32v4T
        select CPU_ABRT_EV4T
        select CPU_CACHE_V4WT
@@ -127,7 +127,7 @@ config CPU_ARM925T

 # ARM926T
 config CPU_ARM926T
-       bool "Support ARM926T processor" if (!ARCH_MULTIPLATFORM || ARCH_MULTI_V5) && (ARCH_INTEGRATOR || MACH_REALVIEW_EB)
+       bool "Support ARM926T processor" if ARCH_INTEGRATOR || MACH_REALVIEW_EB || ARCH_VERSATILE_PB || ARCH_VERSATILE_AB
        select CPU_32v5
        select CPU_ABRT_EV5TJ
        select CPU_CACHE_VIVT
@@ -359,7 +360,7 @@ config CPU_PJ4B

 # ARMv6
 config CPU_V6
-       bool "Support ARM V6 processor" if (!ARCH_MULTIPLATFORM || ARCH_MULTI_V6) && (ARCH_INTEGRATOR || MACH_REALVIEW_EB || MACH_REALVIEW_PBX || MACH_BCM2708)
+       bool "Support ARM V6 processor" if ARCH_INTEGRATOR || MACH_REALVIEW_EB || MACH_REALVIEW_PBX || MACH_BCM2708 || ARCH_VERSATILE_PB || ARCH_VERSATILE_AB
        select CPU_32v6
        select CPU_ABRT_EV6
        select CPU_CACHE_V6



In file "linux/drivers/mmc/host/Kconfig.rej" 11L, 392C

--- drivers/mmc/host/Kconfig
+++ drivers/mmc/host/Kconfig
@@ -6,7 +6,7 @@ comment "MMC/SD/SDIO Host Controller Drivers"

 config MMC_BCM2835
        tristate "MMC support on BCM2835"
-       depends on MACH_BCM2708 || MACH_BCM2709 || ARCH_BCM2835
+       depends on MACH_BCM2708 || MACH_BCM2709 || ARCH_BCM2835 || ARCH_VERSATILE_PB || ARCH_VERSATILE_AB
        help
          This selects the MMC Interface on BCM2835.

Before applying these patches the helper script cloned git repositories, two of them between one and two GB in size, into the path from which script build-kernel-qemu is called. Ted noting that some of the paths and file referenced in the script may be incorrect or fixed enough to keep the script from finding what it needs to reference and to modify. Not sure whether this observation is accurate or how many broken issues there are in the script, could be fine. But for example Ted needed to create a symlink to file linux-arm.patches initially to get the Rpi kernel configuring script to find that patch file . . . a patch file which in turn hasn't yet worked this 2017 December.

Another path to try is in drhuvvyas90's script to comment out the call to patch. What happens then? - TMH


- 2017-12-14 . . . Ok looks like the file ./linux/arch/arm/mm/Kconfig must be patched in order to avoid the two kernel compile errors we see today and saw yesterday.




↑ ↓ Raspberry Pi hardware

Quoting from the forum post at: 

"To check what revision you have open a terminal window and run:

cat /proc/cpuinfo

You should get an output that looks like the following:

Processor       : ARMv6-compatible processor rev 7 (v6l)
BogoMIPS        : 847.05
Features        : swp half thumb fastmult vfp edsp java tls
CPU implementer : 0x41
CPU architecture: 7
CPU variant     : 0x0
CPU part        : 0xb76
CPU revision    : 7
Hardware        : BCM2708
Revision        : 0002
Serial          : 000000000abc0ab1"


A related forum post which compares RaspberryPi 2 revision 1.1 and revision 1.2:



↑ ↓ RaspberryPi Kernel Compilation


- 2017-11-25 SAT - These notes are of first efforts to cross-compile Linux kernel and device tree blob, on Intel i386/i686 host system, for target ARM system specifically RaspberryPi 2 dev board. As of 2017-DEC-09 Saturday, four or five custom kernel configurations with rpi target chosen / in mind have failed, for reasons such as "enermation or pound define value not an integer" and "alignment type of variable x not available [on target processor]". Ted noting that the Linux kernel stable release with all its options and supported target hardware platforms and peripherals is heavyweight, not by default configured to be small or simple as possible. Now also looking into Tiny Kernel wiki page and home page of Tiny Core Linux.

Here are instructions for compiling and for cross-compiling a Linux kernel for RaspberryPi target system:



Here are limited instructions on how to configure the Linux kernel for RaspberryPi target systems:



Back in 2017 February this year, February 24th just after Embedded Linux Conference, looks like Ted stepped through some of the instructions at this Raspberry Pi dot org documentation page. Ted noting there are two directories on compiling host in /mnt with this timestamp: 


ted@localhost:/mnt$ ls -l make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- menuconfig
total 44
drwxr-xr-x 2 root root 4096 Oct 24  2016 dvd-rom
drwxr-xr-x 2 root root 4096 Feb 24  2017 ext4
drwxr-xr-x 2 root root 4096 Feb 24  2017 fat32
   .
   .
   .

When issuing the cross-compile command that task seemed to complete quickly, giving the following messages which total less than two hundred lines: 


ted@rangari:~/projects/raspberrypi/linux$ make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- bcm2709_defconfig
#
# configuration written to .config
#
ted@rangari:~/projects/raspberrypi/linux$ make -j 2 ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- zImage modules dtbs
scripts/kconfig/conf  --silentoldconfig Kconfig
  CHK     include/config/kernel.release
  CHK     include/generated/uapi/linux/version.h
  CHK     include/generated/utsrelease.h
  HOSTCC  scripts/dtc/dtc.o
  HOSTCC  scripts/dtc/flattree.o
  HOSTCC  scripts/dtc/fstree.o
  HOSTCC  scripts/dtc/data.o
  HOSTCC  scripts/dtc/livetree.o
  HOSTCC  scripts/dtc/treesource.o
  HOSTCC  scripts/dtc/srcpos.o
make[1]: 'include/generated/mach-types.h' is up to date.
  CHK     include/generated/timeconst.h
  CHK     include/generated/bounds.h
  CHK     include/generated/asm-offsets.h
  CALL    scripts/checksyscalls.sh
  HOSTCC  scripts/dtc/checks.o make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- menuconfig
  HOSTCC  scripts/dtc/util.o
  HOSTCC  scripts/dtc/dtc-lexer.lex.o
  HOSTCC  scripts/dtc/dtc-parser.tab.o
  HOSTLD  scripts/dtc/dtc
  HOSTCC  scripts/genksyms/genksyms.o
  HOSTCC  scripts/genksyms/parse.tab.o
  HOSTCC  scripts/mod/mk_elfconfig
  HOSTCC  scripts/genksyms/lex.lex.o
  MKELF   scripts/mod/elfconfig.hhttps://www.raspberrypi.org/forums/viewtopic.php?f=66&t=178806
  HOSTCC  scripts/mod/modpost.o
  HOSTLD  scripts/genksyms/genksyms
  HOSTCC  scripts/kallsyms
  HOSTCC  scripts/mod/file2alias.o
  HOSTCC  scripts/pnmtologo
  HOSTCC  scripts/conmakehash make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- menuconfig
  HOSTCC  scripts/mod/sumversion.o
  HOSTCC  scripts/recordmcount
  HOSTLD  scripts/mod/modpost
  HOSTCC  scripts/sortextable
  HOSTCC  usr/gen_init_cpio
  CHK     include/generated/compile.h
  GEN     usr/initramfs_data.cpio.gz
  HOSTCC  arch/arm/vdso/vdsomunge
  AS      usr/initramfs_data.o
  LD      usr/built-in.o
  MUNGE   arch/arm/vdso/vdso.so.dbg
  OBJCOPY arch/arm/vdso/vdso.so
  AS      arch/arm/vdso/vdso.o
  LD      arch/arm/vdso/built-in.o
  GZIP    kernel/config_data.gz make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- menuconfig
  CHK     kernel/config_data.h
  HOSTCC  lib/gen_crc32table
  HOSTCC  lib/raid6/mktables
  TABLE   lib/raid6/tables.c
  CC [M]  lib/raid6/tables.o
  LD [M]  lib/raid6/raid6_pq.o
  GEN     lib/crc32table.h
  CC      lib/crc32.o
  LD      lib/built-in.o
  LOGO    drivers/video/logo/logo_linux_clut224.c
  LOGO    drivers/video/logo/logo_linux_mono.c
  LOGO    drivers/video/logo/logo_superh_mono.c make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- menuconfig
  LOGO    drivers/video/logo/logo_superh_vga16.c make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- menuconfig
  LOGO    drivers/video/logo/logo_linux_vga16.c
  LOGO    drivers/video/logo/logo_blackfin_vga16.c
  LOGO    drivers/video/logo/clut_vga16.c
  LOGO    drivers/video/logo/logo_spe_clut224.c
  LOGO    drivers/video/logo/logo_mac_clut224.c
  LOGO    drivers/video/logo/logo_superh_clut224.c
  LOGO    drivers/video/logo/logo_sun_clut224.c
  LOGO    drivers/video/logo/logo_parisc_clut224.c
  LOGO    drivers/video/logo/logo_blackfin_clut224.c
  LOGO    drivers/video/logo/logo_dec_clut224.c
  LOGO    drivers/video/logo/logo_m32r_clut224.c
  LOGO    drivers/video/logo/logo_sgi_clut224.c
  CC      drivers/video/logo/logo_linux_clut224.o
  LD      drivers/video/logo/built-in.o
  LD      drivers/video/built-in.o make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- menuconfig
  LD      drivers/built-in.o
  LINK    vmlinux
  LD      vmlinux.o
  MODPOST vmlinux.o
  GEN     .version
  CHK     include/generated/compile.h
  UPD     include/generated/compile.h
  CC      init/version.o
  LD      init/built-in.o
  KSYM    .tmp_kallsyms1.o
  KSYM    .tmp_kallsyms2.o
  LD      vmlinux
  SORTEX  vmlinux
  SYSMAP  System.map
  OBJCOPY arch/arm/boot/Image
  Building modules, stage 2.
  Kernel: arch/arm/boot/Image is ready
  Kernel: arch/arm/boot/Image is ready
  GZIP    arch/arm/boot/compressed/piggy.gzip
  AS      arch/arm/boot/compressed/piggy.gzip.o
  LD      arch/arm/boot/compressed/vmlinux
  OBJCOPY arch/arm/boot/zImage
  Kernel: arch/arm/boot/zImage is ready
  MODPOST 1540 modules
  LD [M]  lib/raid6/raid6_pq.ko
ted@rangari:~/projects/raspberrypi/linux$


At this point it looks like we may have enough to attempt the qemu invocation which was partially working off site west side, since couple days ago we were missing what appeared to be only the kernel when invoking qemu on build box rangari . . .

So we can launch Raspbian for the RaspberryPi 2 board, using a kernel we've built on an i386 architecture host server, using the default bcmp_2907 (mis-spelled) kernel config file. But we can't fully compile a kernel for an ARM / Rpi target architecture when we make any kernel configuration changes via `make menuconfig`. The script which successfully starts our qemu-system-arm version 2.10.0 is: 


#!/bin/bash

qemu-system-arm -kernel ./kernel-qemu-4.4.34-jessie -cpu arm1176 -m 256 -M versatilepb -serial stdio -append "root=/dev/sda2 rootfstype=ext4 rw" -hda ./2017-09-07-raspbian-stretch-lite.img -redir tcp:5022::22 -no-reboot

exit 0


Right now this script is located along side several already compiled, ARM-targeted kernels which Ted believes are from the git available cross-compile Rpi project by one 'dhruvvyas90' . . . Dhruvvyas90 qemu-rpi-kernel. 

ted@localhost:/var/local/ted/projects/raspberrypi/qemu_vms$ ls
2017-09-07-raspbian-stretch-lite.img  kernel-qemu-4.1.7-jessie   kernel-qemu-4.4.21-jessie  README.md
kernel-qemu-3.10.25-wheezy            kernel-qemu-4.4.12-jessie  kernel-qemu-4.4.26-jessie  start-qemu.sh
kernel-qemu-4.1.13-jessie             kernel-qemu-4.4.13-jessie  kernel-qemu-4.4.34-jessie  tools
ted@localhost:/var/local/ted/projects/raspberrypi/qemu_vms$


- Summary rpi kernel compilations -

As of 2017 Dec 2 we can using QEMU 2.10.0 boot and use Raspbian image and kernel from Raspberry Pi on-line store. We can also substitute alternate kernel in call to QEMU system emulator, a kernel from . . .



↑ ↓ edit point - custom Rpi-targeted kernel config issues

- 2017-11-30 -

Took a couple days but Ted learned that following errors arose from running `make menuconfig` on a given set of kernel sources without specifying the ARM architecture nor the GNU abiehf cross-compiler: 

In file included from ./include/uapi/linux/stddef.h:1:0,
                 from ./include/linux/stddef.h:4,
                 from ./include/uapi/linux/posix_types.h:4,
                 from ./include/uapi/linux/types.h:13,
                 from ./include/linux/types.h:5,
                 from fs/xfs/xfs_linux.h:21,
                 from fs/xfs/xfs.h:32,
                 from fs/xfs/xfs_super.c:19:
In function ‘xfs_check_ondisk_structs’,
    inlined from ‘init_xfs_fs’ at fs/xfs/xfs_super.c:1974:2:
./include/linux/compiler.h:518:38: error: call to ‘__compiletime_assert_119’ declared with attribute error: XFS: sizeof(xfs_dir2_sf_entry_t) is wrong, expected 3
  _compiletime_assert(condition, msg, __compiletime_assert_, __LINE__)
                                      ^
./include/linux/compiler.h:501:4: note: in definition of macro ‘__compiletime_assert’
    prefix ## suffix();    \
    ^
./include/linux/compiler.h:518:2: note: in expansion of macro ‘_compiletime_assert’
  _compiletime_assert(condition, msg, __compiletime_assert_, __LINE__)
  ^
./include/linux/bug.h:54:37: note: in expansion of macro ‘compiletime_assert’
 #define BUILD_BUG_ON_MSG(cond, msg) compiletime_assert(!(cond), msg)
                                     ^
fs/xfs/xfs_ondisk.h:22:2: note: in expansion of macro ‘BUILD_BUG_ON_MSG’
  BUILD_BUG_ON_MSG(sizeof(structname) != (size), "XFS: sizeof(" \
  ^
fs/xfs/xfs_ondisk.h:119:2: note: in expansion of macro ‘XFS_CHECK_STRUCT_SIZE’
  XFS_CHECK_STRUCT_SIZE(xfs_dir2_sf_entry_t,  3);
  ^
scripts/Makefile.build:293: recipe for target 'fs/xfs/xfs_super.o' failed
make[2]: *** [fs/xfs/xfs_super.o] Error 1
scripts/Makefile.build:544: recipe for target 'fs/xfs' failed
make[1]: *** [fs/xfs] Error 2
Makefile:991: recipe for target 'fs' failed
make: *** [fs] Error 2

real	17m33.542s
user	16m10.988s
sys	0m53.000s


Ok as of 2017-12-08 and a few days ago Ted realizing that the `make menuconfig` type call to configure a Linux kernel for a RaspberryPi/Arm target architecture must be of the form as follows below . . . and before starting a fresh kernel configuration it is a good idea to clean up all old object files, intermediate files, old config information using: 

   $ make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- mrproper
 
   $ make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- menuconfig

Now again calling the version of this command with options to compile kernel, device tree binary, and modules: 

   $ make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- zImage modules dtbs

↑ ↓ RaspberryPi kernel configuration references

Ted found these references in the README.md file of QEMU Rpi Tools project. Local directory is qemu-rpi-kernel/. Invoking `git status .` shows this to be a git repository. Invoking `git remote -v` shows this repository to be a clone of: 

ted@localhost:~/projects/qemu/qemu-rpi-kernel$ git remote -v
origin  https://github.com/dhruvvyas90/qemu-rpi-kernel.git (fetch)
origin  https://github.com/dhruvvyas90/qemu-rpi-kernel.git (push)
ted@localhost:~/projects/qemu/qemu-rpi-kernel$ #


In the READ.md file the Rpi kernel project author who doesn't identify him or herself lists the following four references together, and there are a couple more refs beyond these mentioned near the end of READ.md: 

https://web.archive.org/web/20131210001638/http://xecdesign.com/compiling-a-kernel/
https://web.archive.org/web/20131209235952/http://xecdesign.com/compiling-qemu/
https://web.archive.org/web/20131210001407/http://xecdesign.com/working-with-qemu/
https://web.archive.org/web/20131210001526/http://xecdesign.com/qemu-emulating-raspberry-pi-the-easy-way/


- 2017-12-14 THU -

Sysprogs site with multiple kernel related tutorial in left hand column:


↑ ↓ Finding kernel version

Use of `dpkg-query -l` to find kernel version details on Linux system . . .

Quote from following article: "In a kernel source directory [invoke] `$ head Makefile -n 3` . . .

Looks like there is also kernel version information in a file in an include directory which is part of the given kernel sources: 

/var/local/raspberrypi/rpi-linux/include/config$ cat kernel.release 
4.4.50-v7+
$



↑ ↓ Main Line Kernel versus RaspberryPi Kernel Project

A number of the latest kernel releases, both stable and testing releases are not yet supported by the Raspberry Pi team -- if there is a team -- which develops the patches and detailed necessary configurations to build 4.x Linux kernels for the fullest Rpi board support possible:

Quoting one Rpi user Dougie Lawson, "You'll find 4.1.x and 4.4.x are the best supported kernels currently . . ."



↑ ↓ general kernel configuration and compile reference

- 2017-12-05 - Ted adding section on general articles, some non-Rpi articles and reference materials which explain how to configure a kernel and then how to invoke the build process and point `make` to the new custom configuration file. This so far not clear from the instructions at Raspberry Pi's articles https://www.raspberrypi.org/documentation/linux/kernel/building.md and https://www.raspberrypi.org/documentation/linux/kernel/configuring.md. These references found at Google via search phrase "invoking make after custom kernel configuration":


There are thousands of settable options in the Linux kernel sources. Ted not sure whether all the flags are visible and accessible from `make menuconfig`. This forum post discusses <code>menuconfig's search feature . . .



↑ ↓ Finding kernel configurations - extract-ikconfig and flag CONFIG_IKCONFIG

A booted Linux kernel which exports its configuration settings to /proc/config.gz can be studied in detail from its configuration standpoint. Tools to use in this case include fdisk on the unmounted root file system which got booted up to a working system with the given kernel. During that run session the kernel will populate /proc/config.gz. On the unmounted root file system image use fdisk to determine partition count, file system type and block size, and starting block of the rootfs. Then mount that partition within the rootfs image file with an offset equal to (block size * starting block). Example of this at Azeria Labs RaspberryPi under Qemu tutorial.

Alternately depending on a kernel configuration flag a script named extract-ikconfig can be used on the compressed kernel image, sometimes named zImage to extract the given kernel's configuration. According to,

the kernel config flag to enable full configuration extraction from kernel image files is named CONFIG_IKCONFIG.

From our rpi-4.4.y based kernel build today, to which Ted applied bcm2709_defconfig configuration we find no kernel configuration info extractable. Nor is there any config info extractable from the host system's kernel /boot/vmlinuz-3.2.0-4-686-pae. However the latest of drhuvvyas90's already compiled kernels produces full configuration settings: 

$ /var/local/raspberrypi/rpi-linux/scripts/extract-ikconfig $D21/kernel-qemu-4.4.34-jessie | wc
   2736    8998   70148
$

and we saw this a couple days ago. Now going through this output while using menuconfig we should be able to see, albeit slow and tediously, differences in the working Rpi kernel configuration and our own . . .

- 2017-12-15 FRI ~13:50 -

Ok we enabled one new kernel configuration as a built-in part of the kernel rather than its apparent default module [M] setting from the bcm2709_defconfig settings. Recompiled the rpi-4.4.y kernel sources and in a short time just a minute or so have a kernel from which we can extract full configuration using extract-ikconfig. We now have two configuration summary files, one from drhuvvyas90's 4.4.34 kernel for QEMU and Rpi, and one from our kernel based on the rpi-4.4.y branch of RaspberryPi kernel project on Github. The files are a couple thousand lines long or longer . . . it may be also that some kernel config flags in drhuvvyas90's project are not found in . . .

↑ ↓ edit point - drhuvvyas and bcm2709_defconfig comparison

This is a brute force approach to finding the key differences between two kernel configs, the first one which boots to a fully operational Linux system in QEMU versus a second kernel which fails to boot. First Ted noting the config file sections in each kernel configuration in next table just below these sentences. Easy to see that bcm2709_defconfig enables many more kernel options, and an alternate machine type compared with drhuvvyas90's pre-built rpi kernel for QEMU:

Table x - Comparing kernel configuration sections

  drhuvvyas90 bcm2709_defconfig 
2:# Automatically generated file; DO NOT EDIT.
3:# Linux/arm 4.4.34 Kernel Configuration
28:# General setup
70:# Timers subsystem
170:# Kernel Performance Events And Counters
240:# Partition Types
282:# System Type
316:# Versatile platform type
326:# Processor Type
343:# Processor Features
362:# Bus support
383:# Kernel Features
440:# Boot options
472:# Floating point emulation
476:# At least one emulation must be selected
484:# Userspace binary formats
494:# Power management options
515:# Networking options
566:# Core Netfilter Configuration
606:# Xtables combined modules
612:# Xtables targets
636:# Xtables matches . . . . . . . . . . . . . . . . . . .


758:# Network testing . . . . . . . . . . . . . . . . . . .







784:# Device Drivers  . . . . . . . . . . . . . . . . . . .
790:# Generic Driver Options

810:# Bus devices
822:# User Modules And Translation Layers
867:# Mapping drivers for chip access
758:# Network testing
784:# Device Drivers
790:# Generic Driver Options
810:# Bus devices
822:# User Modules And Translation Layers
867:# Mapping drivers for chip access
877:# Self-contained MTD device drivers
886:# Disk-On-Chip Device Drivers
924:# Misc devices  . . . . . . . . . . . . . . . . . . . .
960:# Texas Instruments shared transport line discipline
966:# Altera FPGA firmware download module
971:# Intel MIC Bus Driver
979:# Intel MIC Host Driver
983:# Intel MIC Card Driver
991:# Intel MIC Coprocessor State Management (COSM) Drivers
1120:# Distributed Switch Architecture drivers
1281:# Host-side USB support is needed for USB Network Adapter support
1290:# Enable WiMAX (Networking options) to see the WiMAX drivers
1298:# Input device support
1308:# Userland interfaces
1319:# Input Device Drivers
1370:# Hardware I/O ports
1385:# Character devices
1408:# Serial drivers
1428:# Non-8250 serial port support
1508:# External I2C/SMBus adapter drivers
1514:# Other I2C/SMBus bus drivers
1540:# Enable PHYLIB and NETWORK_PHY_TIMESTAMPING to see the additional clocks.
1551:# Memory mapped GPIO drivers
1594:# Sonics Silicon Backplane
1600:# Broadcom specific AMBA
1605:# Multifunction device drivers
1692:# Graphics support
1699:# Frame buffer Devices
1722:# Frame buffer hardware drivers636:# Xtables matches
758:# Network testing
784:# Device Drivers
790:# Generic Driver Options
810:# Bus devices
822:# User Modules And Translation Layers
867:# Mapping drivers for chip access
877:# Self-contained MTD device drivers
886:# Disk-On-Chip Device Drivers
924:# Misc devices
960:# Texas Instruments shared transport line discipline
966:# Altera FPGA firmware download module
971:# Intel MIC Bus Driver
979:# Intel MIC Host Driver
983:# Intel MIC Card Driver
991:# Intel MIC Coprocessor State Management (COSM) Drivers
1120:# Distributed Switch Architecture drivers
1281:# Host-side USB support is needed for USB Network Adapter support
1290:# Enable WiMAX (Networking options) to see the WiMAX drivers
1298:# Input device support
1308:# Userland interfaces
1319:# Input Device Drivers
1370:# Hardware I/O ports
1385:# Character devices
1408:# Serial drivers
1428:# Non-8250 serial port support
1508:# External I2C/SMBus adapter drivers
1514:# Other I2C/SMBus bus drivers
1540:# Enable PHYLIB and NETWORK_PHY_TIMESTAMPING to see the additional clocks.
1551:# Memory mapped GPIO drivers
1594:# Sonics Silicon Backplane
1600:# Broadcom specific AMBA
1605:# Multifunction device drivers
1692:# Graphics support
1699:# Frame buffer Devices
1722:# Frame buffer hardware drivers
1765:# Console display driver support
1882:# Special HID drivers
2067:# Virtio drivers
2073:# Microsoft Hyper-V guest support
2081:# Hardware Spinlock drivers
2085:# Clock Source drivers
2099:# Generic IOMMU Pagetable Support
2105:# Remoteproc drivers
2110:# Rpmsg drivers
2145:# Performance monitor support
2151:# Android
2166:# Firmware Drivers
2171:# File systems
2208:# Caches
2229:# Pseudo filesystems
2355:# Kernel hacking
2366:# Compile-time checks and compiler options
2386:# Memory Debugging
2400:# Debug Lockups and Hangs
2415:# Lock Debugging (spinlocks, mutexes, etc...)
2473:# Runtime Testing
2517:# Security options
2531:# Crypto core or helper
2560:# Authenticated Encryption with Associated Data
2569:# Block modes
2581:# Hash modes
2589:# Digest
2610:# Ciphers
2630:# Compression
2640:# Random Number Generation
2658:# Certificates for signature checking
2665:# Library routines

2:# Automatically generated file; DO NOT EDIT.
3:# Linux/arm 4.4.50 Kernel Configuration
30:# General setup
76:# Timers subsystem
186:# Kernel Performance Events And Counters
269:# Partition Types
315:# System Type
350:# Broadcom BCM2709 Implementations
355:# Processor Type
371:# Processor Features
407:# Bus support
414:# Kernel Features
493:# Boot options
543:# Floating point emulation
547:# At least one emulation must be selected
557:# Userspace binary formats
567:# Power management options
585:# Networking options
677:# Core Netfilter Configuration
726:# Xtables combined modules
733:# Xtables targets
760:# Xtables matches
1010:# Queueing/Scheduling
1039:# Classification
1111:# Network testing
1119:# Packet Radio protocols
1176:# IrDA protocols
1184:# IrDA options
1191:# Infrared-port device drivers
1200:# Dongle support
1228:# Bluetooth device drivers
1306:# Near Field Communication (NFC) devices
1314:# Device Drivers
1320:# Generic Driver Options
1349:# Default contiguous memory area size:
1359:# Bus devices
1376:# User Modules And Translation Layers
1411:# Mapping drivers for chip access
1417:# Self-contained MTD device drivers
1428:# Disk-On-Chip Device Drivers






1501:# Misc devices
1537:# Texas Instruments shared transport line discipline
1544:# Altera FPGA firmware download module
1550:# Intel MIC Bus Driver
1558:# Intel MIC Host Driver
1562:# Intel MIC Card Driver
1570:# Intel MIC Coprocessor State Management (COSM) Drivers
1688:# Distributed Switch Architecture drivers
1963:# WiMAX Wireless Broadband devices
1980:# Input device support
1990:# Userland interfaces
2001:# Input Device Drivers
2167:# Hardware I/O ports
2184:# Character devices
2209:# Serial drivers
2226:# Non-8250 serial port support
2277:# Multiplexer I2C Chip support
2311:# External I2C/SMBus adapter drivers
2320:# Other I2C/SMBus bus drivers
2382:# Enable PHYLIB and NETWORK_PHY_TIMESTAMPING to see the additional clocks.
2387:# Pin controllers
2405:# Memory mapped GPIO drivers
2521:# Native drivers
2659:# Watchdog Device Drivers
2679:# Sonics Silicon Backplane
2692:# Broadcom specific AMBA
2702:# Multifunction device drivers
2811:# Multimedia core support
2839:# Media drivers
2870:# Webcam devices
2934:# Analog TV USB devices
2951:# Analog/digital TV USB devices
2965:# Digital TV USB devices
3013:# Webcam, TV (analog/digital) USB devices
3029:# Supported MMC/SDIO adapters
3054:# Texas Instruments WL128x FM driver (ST based)
3070:# Media ancillary drivers (tuners, sensors, i2c, frontends)
3077:# Encoders, decoders, sensors and other helper chips
3081:# Audio decoders, processors and mixers
3104:# Video decoders
3123:# Video and audio decoders
3129:# Video encoders
3141:# Camera sensor devices
3151:# Flash devices
3159:# Video improvement chips
3165:# Audio/Video compression chips
3170:# Miscellaneous helper chips
3176:# Sensors used on soc_camera driver
3181:# Customize TV tuners
3220:# Customise DVB Frontends
3224:# Multistandard (satellite) frontends
3233:# Multistandard (cable + terrestrial) frontends
3339:# Digital terrestrial only tuners/PLL
3368:# Tools to develop new frontends
3373:# Graphics support
3399:# Display Panels
3410:# Display Interface Bridges
3417:# Frame buffer Devices
3441:# Frame buffer hardware drivers
3491:# Console display driver support
3588:# SoC Audio for Freescale CPUs
3592:# Common SoC Audio options for Freescale CPUs:
3602:# Allwinner SoC Audio support
3697:# Special HID drivers
3804:# Miscellaneous USB options
3883:# Gadget/Dual-role mode requires USB Gadget support to be enabled
4221:# Platform RTC drivers
4282:# Virtio drivers
4287:# Microsoft Hyper-V guest support
4306:# Accelerometers
4318:# Analog to digital converters
4327:# Analog digital bi-direction converters
4332:# Capacitance to digital converters
4339:# Direct Digital Synthesis
4345:# Digital gyroscope sensors
4350:# Network Analyzer, Impedance Converters
4355:# Light sensors
4363:# Magnetometer sensors
4369:# Active energy metering IC
4378:# Resolver to digital converters
4385:# Triggers - standalone
4390:# Speakup console speech
4419:# Android
4469:# Common Clock Framework
4481:# Hardware Spinlock drivers
4485:# Clock Source drivers
4506:# Remoteproc drivers
4511:# Rpmsg drivers
4525:# Extcon Device Drivers
4541:# Accelerometers
4555:# Analog to digital converters
4577:# Amplifiers
4582:# Chemical Sensors
4587:# Hid Sensor IIO Common
4597:# Digital to analog converters
4619:# Frequency Synthesizers DDS/PLL
4623:# Clock Generator/Distribution
4628:# Phase-Locked Loop (PLL) frequency synthesizers
4633:# Digital gyroscope sensors
4645:# Humidity sensors
4654:# Inertial measurement units
4662:# Light sensors
4689:# Magnetometer sensors
4699:# Inclinometer sensors
4703:# Digital potentiometers
4708:# Pressure sensors
4719:# Lightning sensors
4724:# Proximity sensors
4730:# Temperature sensors
4758:# Performance monitor support
4764:# Android
4779:# Firmware Drivers
4785:# File systems
4860:# Caches
4893:# Pseudo filesystems
5064:# Kernel hacking
5076:# Compile-time checks and compiler options
5097:# Memory Debugging
5113:# Debug Lockups and Hangs
5131:# Lock Debugging (spinlocks, mutexes, etc...)
5209:# Runtime Testing
5257:# Security options
5277:# Crypto core or helper
5309:# Authenticated Encryption with Associated Data
5318:# Block modes
5330:# Hash modes
5338:# Digest
5359:# Ciphers
5380:# Compression
5390:# Random Number Generation
5408:# Certificates for signature checking
5425:# Library routines
3 4
# 5 6


Table x - Comparing kernel configurations of drhuvvyas90 and bcm2709_defconfig:


  drhuvvyas90 bcm2709_defconfig
# Timers subsystem CONFIG_HZ_PERIODIC=y # CONFIG_HZ_PERIODIC is not set
# RCU Subsystem CONFIG_TINY_RCU=y -
# Kernel Performance Events And Counters # CONFIG_PERF_EVENTS is not set CONFIG_PERF_EVENTS=y
CONFIG_COMPAT_BRK=y -
# System Type CONFIG_MMU=y CONFIG_MMU=y
# CONFIG_ARCH_BCM2709 is not set CONFIG_ARCH_BCM2709=y <-- major difference here
CONFIG_ARCH_VERSATILE=y # CONFIG_ARCH_VERSATILE is not set
# Versatile platform type CONFIG_ARCH_VERSATILE_PB=y
CONFIG_MACH_VERSATILE_AB=y
# CONFIG_MACH_VERSATILE_DT is not set
CONFIG_PLAT_VERSATILE_CLOCK=y
CONFIG_PLAT_VERSATILE_SCHED_CLOCK=y
CONFIG_PLAT_VERSATILE=y
# Processor Type 
# CONFIG_CPU_ARM920T is not set
# CONFIG_CPU_ARM922T is not set
CONFIG_CPU_V6=y
CONFIG_CPU_32v6=y
CONFIG_CPU_ABRT_EV6=y
CONFIG_CPU_PABRT_V6=y
CONFIG_CPU_CACHE_V6=y
CONFIG_CPU_CACHE_VIPT=y
CONFIG_CPU_COPY_V6=y
CONFIG_CPU_TLB_V6=y
CONFIG_CPU_HAS_ASID=y
CONFIG_CPU_CP15=y
CONFIG_CPU_CP15_MMU=y

CONFIG_CPU_V7=y
CONFIG_CPU_32v6K=y
CONFIG_CPU_32v7=y
CONFIG_CPU_ABRT_EV7=y
CONFIG_CPU_PABRT_V7=y
CONFIG_CPU_CACHE_V7=y
CONFIG_CPU_CACHE_VIPT=y
CONFIG_CPU_COPY_V6=y
CONFIG_CPU_TLB_V7=y
CONFIG_CPU_HAS_ASID=y
CONFIG_CPU_CP15=y
CONFIG_CPU_CP15_MMU=y
<-- major difference here
# Processor Features 
# CONFIG_ARCH_PHYS_ADDR_T_64BIT is not set
CONFIG_ARM_THUMB=y
# CONFIG_CPU_ICACHE_DISABLE is not set
# CONFIG_CPU_DCACHE_DISABLE is not set
# CONFIG_CPU_BPREDICT_DISABLE is not set
CONFIG_KUSER_HELPERS=y
# CONFIG_CACHE_L2X0 is not set
CONFIG_ARM_L1_CACHE_SHIFT=5
CONFIG_ARM_DMA_MEM_BUFFERABLE=y
# CONFIG_ARM_KERNMEM_PERMS is not set
CONFIG_MULTI_IRQ_HANDLER=y
# CONFIG_ARM_ERRATA_326103 is not set
CONFIG_ARM_ERRATA_411920=y
CONFIG_ARM_ERRATA_364296=y
CONFIG_ICST=y

# CONFIG_ARM_LPAE is not set
# CONFIG_ARCH_PHYS_ADDR_T_64BIT is not set
CONFIG_ARM_THUMB=y
# CONFIG_ARM_THUMBEE is not set
CONFIG_ARM_VIRT_EXT=y
CONFIG_SWP_EMULATE=y
# CONFIG_CPU_ICACHE_DISABLE is not set
# CONFIG_CPU_BPREDICT_DISABLE is not set
CONFIG_KUSER_HELPERS=y
CONFIG_VDSO=y
CONFIG_MIGHT_HAVE_CACHE_L2X0=y
# CONFIG_CACHE_L2X0 is not set
CONFIG_ARM_L1_CACHE_SHIFT_6=y
CONFIG_ARM_L1_CACHE_SHIFT=6
CONFIG_ARM_DMA_MEM_BUFFERABLE=y
# CONFIG_ARM_KERNMEM_PERMS is not set
CONFIG_MULTI_IRQ_HANDLER=y
# Bus support 
CONFIG_PCI=y
# CONFIG_PCI_DOMAINS_GENERIC is not set
CONFIG_PCI_SYSCALL=y
# CONFIG_PCI_MSI is not set
# CONFIG_PCI_DEBUG is not set
# CONFIG_PCI_REALLOC_ENABLE_AUTO is not set
# CONFIG_PCI_STUB is not set
# CONFIG_PCI_IOV is not set
# CONFIG_PCI_PRI is not set
# CONFIG_PCI_PASID is not set

# CONFIG_PCI_DOMAINS_GENERIC is not set
# CONFIG_PCI_SYSCALL is not set
# CONFIG_PCCARD is not set
# 5 6
3 4
# 5 6
3 4
# 5 6
3 4
# 5 6


Hmm, MediaWiki page rendering engine is causing next section to wrap funny around the table of kernel config comparisons . . .

Looking up some terms, or to look up later:



↑ ↓ linux/arch/arm/configs and bcm2709_defconfig versus versatile_defconfig

Today is 2017-12-13 Wednesday, and yesterday we determined that the one difference between bootable QEMU + kernel + rootfs and non-booting QEMU + kernel + rootfs is the compiled kernel file. So somethings are wrong with our kernel configuration. We retrieved the kernel configuration from the drhuvvyas90 pre-compiled Rpi kernel named kernel-qemu-4.4.34-jessie by mounting the "jessie stretch lite" rootfs image from RaspberryPi downloads page, and copying /proc/config.gz from that rootfs. We could not extract any config information from our compiled kernel, but we likely left disabled a kernel config option mentioned in https://stackoverflow.com/questions/14958192/getting-config-from-linux-kernel-image which would permit running script extract-ikconfig on a compressed kernel image.

While studying drhuvvyas90's build-qemu-kernel shell script in file qemu-rpi-kernel/tools/build-kernel-qemu we notice reference to a default configuration file which is different from the bcm2709_defconfig default configuration named in some of our recently found tutorials on building Linux kernel for RaspberryPi 2 development boards . . .


- 2017-12-15 FRI -

From an Rpi forum post yesterday we have word -- not yet confirmed -- that as of mid- to late 2017 the Linux 4.x kernel versions are only a couple of them close to fully supported in their port or configuration works targeting RaspberryPi dev boards. Raspberry Pi kernel releases 4.1.x and 4.4.x are the most suppported kernel versions, just two out of the many 4.x kernel release candidates. With this in mind Ted returning to question of which default ARM configuration is appropriate to invoke when calling `make ... menuconfig`. Ted has seen at RaspberryPi organizations tutorial to set KERNEL=kernel7 and use bmc2709_defconfig. But yesterday in drhuvvyas90's build_kernel_qemu script Ted saw reference to use versatile_defconfig. There is a similarly named bcm2835_defconfig . . . which default ARM targeted config recipes will work to build a functional kernel that runs in QEMU system emulator and on a real world Raspberry Pi 2 board? A further forum post may help, though advice here repeats what's on Raspberry Pi's kernel building tutorial . . .



↑ ↓ kernel - specific configuration build results

- 2017-12-13 WED -

Kernel build results 1

Errors run into when building kernel following configuration 'versatile_defconfig', after issuing command make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- versatile_defconfig . . . 

  CC      lib/win_minmax.o
  AR      lib/lib.a
  EXPORTS lib/lib-ksyms.o
  LD      lib/built-in.o
  LD      virt/lib/built-in.o
  LD      virt/built-in.o
  LD      vmlinux.o
  MODPOST vmlinux.o
  GEN     .version
  CHK     include/generated/compile.h
  UPD     include/generated/compile.h
  CC      init/version.o
  LD      init/built-in.o
arch/arm/mm/built-in.o:(___ksymtab+arm926_dma_clean_range+0x0): undefined reference to `arm926_dma_clean_range'
arch/arm/mm/built-in.o:(___ksymtab+arm926_dma_inv_range+0x0): undefined reference to `arm926_dma_inv_range'
Makefile:972: recipe for target 'vmlinux' failed
make: *** [vmlinux] Error 1

real	23m22.245s
user	21m5.491s
sys	1m21.729s

user@localhost:/var/local/raspberrypi/linux$


- 2017-12-15 FRI -

Kernel build results 2 - below are tail end results of building kernel version rpi-4.4.y, after exporting KERNEL=kernel7 and invoking make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- zImage modules dtbs: 

  DTCO    arch/arm/boot/dts/overlays/w1-gpio.dtbo
  DTCO    arch/arm/boot/dts/overlays/w1-gpio-pullup.dtbo
  DTCO    arch/arm/boot/dts/overlays/wittypi.dtbo
  DTC     arch/arm/boot/dts/bcm2709-rpi-2-b.dtb
  DTC     arch/arm/boot/dts/bcm2710-rpi-3-b.dtb
  DTC     arch/arm/boot/dts/bcm2710-rpi-cm3.dtb

real	74m3.399s
user	68m30.589s
sys	5m9.499s

user@localhost:/var/local/raspberrypi/rpi-linux$ exit
exit
Script done, file is z--kernel-config-using-bcm2709-defconfig--try-2.txt

user@localhost:/var/local/raspberrypi/rpi-linux$



↑ ↓ Buildroot

The Buildroot Project is a many-years standing build system project targeted at embedded Linux system development. Per Buildroot's documentation this project shares much of the functionality and capacity of project and toolset known as Yocto, which is also a software project build management system. Buildroot however does not support Yocto-generated toolchains, apparently because to quote Buildroot's manual those toolchains contain non- tool chain parts beyond a given compiler, binutils and C and C++ libraries.

Before reading far in these notes it is good to know a couple of important design scope matter about the Buildroot scripts and project. From chapter 10 of the Buildroot stable manual as of 2018 January:


  • Buildroot is not designed to create systems with compilers and toolchains on them
  • Buildroot is not a [Linux] distribution generator


These points appear in the Frequently Asked Questions section of Buildroot's manual, and nearly all the things which Buildroot is not come out of the small-memory, small hardware resource nature of the embedded systems which Buildroot developers in general work to support. This said, Buildroot is still a highly useful and configurable tool -- set of scripts and more -- for building small, networked and functional Linux systems.

The same chapter 10 of the manual lists some alternative tools and projects to Buildroot, which include in their scope the adding of toolchains to target systems, systems being boot-loader, kernel, rootfs compiled and configured for a target board or hardware. Some of these alternate tools include Yocto and Openembedded.


An early search for ways to build a kernel and root filesystem led to Medicine Yeh's tutorial, but following his build steps did not yield any kernel which booted in QEMU. There are other buildroot projects available on-line and in the public domain, including a buildroot Rpi project by Guillermo Amaral and a buildroot project involving Qt5 support by Jumpnow Technologies LLC.

For now working through Buildroot's on-line manual.


Click here Early notes related to Buildroot for notes from first discovery of Yeh's work.


- 2018-01-17 WED - Left this line of study a couple weeks ago, not finding a root filesystem which would boot or allow our QEMU set-up to progress beyond the initial few or no messages, seemingly hanging. Putting this study on hold as we need to understand Buildroot system better. Working through Buildroot manual before returning here. - TMH

- 2018-01-19 FRI - Buildroot manual is long, and an important section to remember and return to is titled "Make Tips". This is section 8 in the stable manual published at Buildroot dot org. In particular this section talks about how to invoke `make` on a Buildroot project when major attributes such as target platform architecture change. This manual section worth noting and revisiting.


↑ ↓ Buildroot manual sections

8.2  Full rebuilds, when and how to do them
8.4  Building off-line
8.8 Graphing package dependencies
8.9 Graphing build duration
8.10 Graphing file system size contribution of packages
8.12.5  package-specific make targets
8.12.6  override package source directory to avoid re-downloading sources on every rebuild



↑ ↓ Putting The Pieces Together - QEMU, kernel, rootfs


On our day host system we need and have a cross-compiler, a guest system emulator, a cross-compiled Linux 4.x kernel, a corresponding device tree binary and a root file system . . . 


   cross-compiler       ~/tools/arm-bcm2708/gcc-linaro-arm-linux-gnueabihf-raspbian/bin
   qemu                 /usr/local/bin/qemu-system-arm
   kernel               /var/local/raspberrypi/linux/arch/arm/boot/zImage
   device tree binary   /var/local/raspberrypi/linux/arch/arm/boot/dts/[ bcm2709-rpi-2-b.dtb | bcm2836-rpi-2-b.dtb | ... ] not sure which dtb to use! - TMH
   root file system     ...

One root file system we have which is part of a successful QEMU boot of a guest Raspbian system is the ~1.85GB raspbian stretch lite image, md5 check sum noted here: 

   projects-on-slash-var/raspberrypi/qemu_vms$ md5sum /var/local/ted/downloads/raspberrypi/2017-09-07-raspbian-stretch-lite.img
   2d5f26a5c75ea7e5677542a3411851e3  /var/local/ted/downloads/raspberrypi/2017-09-07-raspbian-stretch-lite.img

And after copying and uncompressing, in the rpi image /proc/config.gz to /home/pi the image file is yet the same size byte-wise but its md5 checksum has changed: 


   /var/local/ted/projects-on-slash-var/raspberrypi/qemu_vms$ md5sum /var/local/ted/downloads/raspberrypi/2017-09-07-raspbian-stretch-lite.img
   779a317911ccee73c8a63ff62f670977  /var/local/ted/downloads/raspberrypi/2017-09-07-raspbian-stretch-lite.img


Per Azeria Labs tutorial on emulating RaspberryPi operating system and hardware, a prerequisite step which modifies a system "reach run level" time must be modified. To modify a file in a disk image file we mount the image file in a manner like, 


   $ sudo mount -v -o offset=47185920 -t ext4 /var/local/raspberrypi/file-system-images/2017-04-10-raspbian-jessie.img /mnt/raspbian
   [sudo] password for user: 
   mount: /dev/loop0 mounted on /mnt/raspbian.
   $



QEMU results and references

- 2017-12-14 - Here is all we get when building kernel per bcm2709_defconfig recipe: 

ted@rangari:/var/local/ted/projects/raspberrypi/qemu_vms$ ./2.sh
WARNING: Image format was not specified for './2017-09-07-raspbian-stretch-lite.img' and probing guessed raw.
         Automatically detecting the format is dangerous for raw images, write operations on block 0 will be restricted.
         Specify the 'raw' format explicitly to remove the restrictions.
pulseaudio: set_sink_input_volume() failed
pulseaudio: Reason: Invalid argument
pulseaudio: set_sink_input_mute() failed
pulseaudio: Reason: Invalid argument

and here the command line prompt waits without response to keyboard input. Qemu's window appears but remains blank and unresponsive too.


- 2017-12-17 - Sunday morning results of calling QEMU with a RaspberryPi kernel 4p4py, configured to default values of versatile_defconfig recipe. These few lines show slightly further progress than we've gotten from bcm2709_defconfig. Processor variant settable at ___ . . . 

ted@rangari:/var/local/ted/projects/raspberrypi/qemu_vms$ ./2.sh
WARNING: Image format was not specified for './2017-09-07-raspbian-stretch-lite.img' and probing guessed raw.
         Automatically detecting the format is dangerous for raw images, write operations on block 0 will be restricted.
         Specify the 'raw' format explicitly to remove the restrictions.
pulseaudio: set_sink_input_volume() failed
pulseaudio: Reason: Invalid argument
pulseaudio: set_sink_input_mute() failed
pulseaudio: Reason: Invalid argument
Uncompressing Linux... done, booting the kernel.

Error: unrecognized/unsupported processor variant (0x410fb767).


QEMU references . . . 


# qemu-system-arm -kernel arch/arm/boot/zImage -dtb rtsm_ve-cortex_a15x4.dtb -m 512 -M vexpress-a15 -serial stdio -append "console=ttyAMA0"



↑ ↓ Extra Notes on QEMU


Gosh frustrating, we were just practicing with calling QEMU in the following way (figure x) which follows the form of invocations we have seen in multiple tutorials and forum posts. Nearly all calls fail with a two lines "audio: Could not init `oss' audio driver . . . Could not read keymap file: 'en-us'". Ted suspected these might be errors coming from the kernel starting up in QEMU, and thought to change the kernel to an earlier release in the pre-compiled kernels of Dhruvvyas. This thought led to choosing a non-existent kernel filename and trying that out in call to QEMU. This made no difference! Same error messages, looks like QEMU doesn't get to the point of sanity checking or needing a valid kernel file name. So how can we debug this failure to boot a kernel plus root filesystem for RaspberryPi "guest" platform? What program or executing code are we actually wanting to debug?

Figure x - QEMU invocation which fails with or without kernel . . . 

$ qemu-system-arm -M versatilepb -cpu arm1176 -m 256 -serial stdio -kernel /var/local/raspberrypi/qemu-rpi-kernel/kernel-qemu-4.4.34-jessiez -dtb /var/local/raspberrypi/linux/arch/arm/boot/dts/bcm2709-rpi-2-b.dtb -append "root=/dev/sda2 rootfstype=ext4 rw init=/bin/bash console=ttyAMA0" -drive format=raw,file=/home/${USER}/Downloads/raspberrypi/2017-08-16-raspbian-stretch-lite.img

audio: Could not init `oss' audio driver
Could not read keymap file: 'en-us'

$



↑ ↓ What Works

The following qemu invocation works . . . 

#!/bin/bash

qemu-system-arm -kernel ./kernel-qemu-4.4.34-jessie -cpu arm1176 -m 256 -M versatilepb -serial stdio -append "root=/dev/sda2 rootfstype=ext4 rw" -hda ./2017-09-07-raspbian-stretch-lite.img -redir tcp:5022::22 -no-reboot

exit 0


The 4.4.34 kernel is of size 3024048 bytes and has md5sum f2a26e138923aa87a97cf43ba8f4d584 kernel-qemu-4.4.34-jessie. The 2017-09-07 image file is of size 1854590976 bytes. Because this image file contains an entire root file system, its content changes with each login session even when the kernel and the image are exercised in a qemu virtual machine.

Note also that the options passed to the kernel via qemu's -append option reflect a change that the lead webmaster and article writer at Azeria Labs described, where one must modify a file in the Raspbian Stretch Lite root filesystem. That modification is in /etc/fstab. References to the RaspberryPi default device name /dev/mmcblk0p1 need to be replaced with /dev/sda1. The final number '1' may vary.

The image file 2017-09-07-raspbian-stretch-lite.img has a /proc/config.gz file. If we can extract and copy that file from the file system image, we could use that as a configuration or starting point of configuring a newer release of the Linux kernel . . .



↑ ↓ Finding Early Kernel Messages

Out of three hosts the above set of files and call to qemu only boots successfully on one host. Some invocations have caused qemu to quickly exit, but others have left qemu running with the one message "Uncompressing Linux... done, booting the kernel.". What is happening in the early kernel boot stages? It looks like there are problems the kernel encounters or has before it can set up a console. Ted looking for way to uncompress the compressed part of the kernel image, and then look at kernel symbol table using arm-linux-gnueabi-nm or similar. 

user@localhost:~/projects/raspberrypi/qemu-vms$ binwalk kernel-qemu-4.4.34-jessie 

DECIMAL       HEXADECIMAL     DESCRIPTION
--------------------------------------------------------------------------------
36            0x24            Linux kernel ARM boot executable zImage (little-endian), load address: "0x00000000", end address: "0x002E24B0"
17576         0x44A8          gzip compressed data, maximum compression, from Unix, NULL date (1970-01-01 00:00:00)

user@localhost:~/projects/raspberrypi/qemu-vms$



↑ ↓ Linux System Components - Boot-loader, Kernel, Initramfs, Rootfs

Purpose of initramfs . . . https://wiki.gentoo.org/wiki/Custom_Initramfs.




^ References


- 2017-12-19 -


- 2017-12-21 - Searching for info on installing GRUB or alternate bootloader on Raspberry Pi 2 board . . .


- 2018-01-02 -

- 2018-01-03 -

- 2018-01-08 -

- 2018-01-09 - 

-nographic -monitor none

- 2018-01-10 -

- 2018-01-14 -





- - - top of page - - -

Retrieved from "https://wiki.neelanurseries.com/index.php?title=Nn_embedded_linux_notes&oldid=1748"