Skip to main content

RPI4 Cross Compile for libcamera-app on Mac Pro (with docker) (3): Setup toolchain

 

RPI4 Cross Compile for libcamera-app on Mac Pro (with docker) (3): Setup toolchain

Why am I doing this

After reading the camera app instructions, I thought of starting the development by modifying the libcamera-apps. Therefore, the first step is to build the original repo; after that, I can add my stuff to the official standard version.

It looks easy. The code-building instructions are well-documented on the official website, but there is a catch in my mind. I bought an expensive, neat, and tidy Macbook Pro after quitting my job as a mind-comforting geek gadget to compensate for losing my working life, so I don't want to build the code on a small RPI board. Instead, I want to build the code and develop things on my new laptop, which is why I got into this over tons of unexpected things just popping up in my sight…

Preface

  1. Create a docker container where I can cross-compile the libcamera-apps.
    1. Setup chroot
    2. Setup toolchain <-
    3. Build libcamera-apps
  2. The docker container should include the development environment I like.
    1. Setup vim
  3. Develop and debug for the self-built libcamera-apps
  4. My GitHub repository

      Goals for setup toolchain

      • Build and make install crosstool-ng
      • Gather the RPI4 system information
      • Generate the configuration and build toolchain
      • Start the Docker container and test the "sbuild-apt install"

      According to the guide for building chroot [Earthly, A. Bhattacharyea], he used a prebuilt toolchain [docker-arm-cross-toolchain], which is only for the X86 host machine; however, my laptop has an ARM-based CPU, so I have to generate the toolchain specifically for my laptop.

      Therefore, I found a reference from [Medium, S.Preston] to help me generate the toolchain. So what is the toolchain? The definition from [Crosstool-ng’s GitHub page] is “Toolchains are an essential component in a software development project. It will compile, assemble, and link the code being developed.” In the sense of cross-compilation, in our case, which is [build == host ≠ target], the toolchain needs to contain the compiler (GCC or G++), linker (ld or ldd), debugger (gdb), and other tools (readelf) that are compatible with the target machine, so that we can use the toolchain software in the host machine to build binaries for the target machine.

      I used Crosstool-ng to build the toolchain so that we will build Crosstool-ng first, make a proper setting for RPI4, build the toolchain, and put the final results in /opt/x-tools in the end.

      Build and make install Crosstool-ng

      • Run the docker container by the below setting.

        • aarch64-rpi4-bullseye-cross-compiler-2204 is the built docker image in the chroot setting page
        #!/bin/bash
_source=$SOURCE_CODE_DIR
_docker_source=/root/main
_test_source=$SOURCE_CODE_EXTRA_DATA_DIR
_docker_test_source=/root/test_data

_docker_image=aarch64-rpi4-bullseye-cross-compiler-2204:v0.1.3
_docker_volume=docker_build_rpi
_docker_volume_toolchain=rpi4-toolchain
_docker_volume_crosstool_ng=crosstoll_ng

docker run -it --rm \\\\
           --privileged=true \\\\
           --cap-add=SYS_ADMIN --security-opt apparmor:unconfined \\\\
           -v $_docker_volume:/root/build \\\\
           -v $_docker_volume_toolchain:/opt \\\\
           -v $_docker_volume_crosstool_ng:/root/crosstool_ng \\\\
           -v $_source:$_docker_source:delegated \\\\
           -v $_test_source:$_docker_test_source:delegated \\\\
           --net=host \\\\
           $_docker_image bash

      • Inside the container, do the below instruction

        wget <http://crosstool-ng.org/download/crosstool-ng/crosstool-ng-1.26.0_rc1.tar.bz2>
tar xjf crosstool-ng-1.26.0_rc1.tar.bz2
cd crosstool-ng-1.26.0_rc1
./configure --prefix=/opt/x-tools
make
make install
export PATH="/opt/x-tools/bin:$PATH"

      • Note: the version, crosstool-ng-1.26.0_rc1, is the latest version when I wrote this page, and there is no harm in using the latest stable version anytime.

      Gathering the RPI4 system information

      • Find the system and binary version

        # check the Linux version
pi@raspberrypi:~ $ uname -r
6.1.21-v8+

# check the ld version
pi@raspberrypi:~ $ ld --version
GNU ld (GNU Binutils for Debian) 2.35.2
Copyright (C) 2020 Free Software Foundation, Inc.
This program is free software; you may redistribute it under the terms of
the GNU General Public License version 3 or (at your option) a later version.
This program has absolutely no warranty.

# check the gcc version
pi@raspberrypi:~ $ gcc --version
gcc (Debian 10.2.1-6) 10.2.1 20210110
Copyright (C) 2020 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

# check the C library version
pi@raspberrypi:~ $ ldd --version
ldd (Debian GLIBC 2.31-13+rpt2+rpi1+deb11u5) 2.31
Copyright (C) 2020 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Written by Roland McGrath and Ulrich Drepper.

      • Find the multiarch patch in the PRI4

        • Copy out 129_multiarch_libpath.patch from in /usr/src/binutils/patches/ RPI OS.

        • This patch will be used in the setting later, which can make ld link the libraries based on the multiarch directory setting, or we will see the below error when cross-compiling if the multiarch setting is not set properly since the ld will only find libraries in /usr/lib instate of /usr/lib/aarch64-linux-gnu.

          • Note: aarch64-linux-gnu is called “triplets,” and it is the setting of my RPI4.
          CMake Error at /usr/share/cmake-3.22/Modules/CMakeTestCCompiler.cmake:69 (message):
  The C compiler

    "/opt/x-tools/aarch64-rpi4-linux-gnu/bin/aarch64-rpi4-linux-gnu-gcc"

  is not able to compile a simple test program.

  It fails with the following output:

    Change Dir: /root/main/cross_compile_test/build/CMakeFiles/CMakeTmp

    Run Build Command(s):/usr/bin/gmake -f Makefile cmTC_5f41d/fast && /usr/bin/gmake  -f CMakeFiles/cmTC_5f41d.dir/build.make CMakeFiles/cmTC_5f41d.dir/build
    gmake[1]: Entering directory '/root/main/cross_compile_test/build/CMakeFiles/CMakeTmp'
    Building C object CMakeFiles/cmTC_5f41d.dir/testCCompiler.c.o
    /opt/x-tools/aarch64-rpi4-linux-gnu/bin/aarch64-rpi4-linux-gnu-gcc --sysroot=/root/sys/chroots/rpi-bullseye-arm64/    -o CMakeFiles/cmTC_5f41d.dir/testCCompiler.c.o -c /root/main/cross_compile_test/build/CMakeFiles/CMakeTmp/testCCompiler.c
    Linking C executable cmTC_5f41d
    /usr/bin/cmake -E cmake_link_script CMakeFiles/cmTC_5f41d.dir/link.txt --verbose=1
    /opt/x-tools/aarch64-rpi4-linux-gnu/bin/aarch64-rpi4-linux-gnu-gcc --sysroot=/root/sys/chroots/rpi-bullseye-arm64/ CMakeFiles/cmTC_5f41d.dir/testCCompiler.c.o -o cmTC_5f41d
    /opt/x-tools/aarch64-rpi4-linux-gnu/bin/../lib/gcc/aarch64-rpi4-linux-gnu/10.5.0/../../../../aarch64-rpi4-linux-gnu/bin/ld: cannot find crt1.o: No such file or directory
    /opt/x-tools/aarch64-rpi4-linux-gnu/bin/../lib/gcc/aarch64-rpi4-linux-gnu/10.5.0/../../../../aarch64-rpi4-linux-gnu/bin/ld: cannot find crti.o: No such file or directory
    collect2: error: ld returned 1 exit status
    gmake[1]: *** [CMakeFiles/cmTC_5f41d.dir/build.make:99: cmTC_5f41d] Error 1
    gmake[1]: Leaving directory '/root/main/cross_compile_test/build/CMakeFiles/CMakeTmp'
    gmake: *** [Makefile:127: cmTC_5f41d/fast] Error 2

  CMake will not be able to correctly generate this project.
Call Stack (most recent call first):
  CMakeLists.txt:2 (project)

      Generate the configuration and build the toolchain

      • Generate config from the default example

        mkdir -p ~/crosstool_ng/raspberry_pi/toolchain_staging
cd ~/crosstool_ng/raspberry_pi/toolchain_staging
mkdir src
mkdir -p patches/binutils/2.35.1/ # the 2.35.1 for binutils will be shown on the cs-ng setting. I chose the closest version under 2.35.2, the real version on my RPI system.
cd patches/binutils/2.35.1/
scp pi@$RPI_IP:/usr/src/binutils/patches/129_multiarch_libpath.patch ./
cd ../../../
ct-ng aarch64-rpi4-linux-gnu # aarch64-rpi4-linux-gnu can be found by "ct-ng list-samples"
        • ct-ng aarch64-rpi4-linux-gnu will generate the below output.
        CONF  aarch64-rpi4-linux-gnu
#
# configuration written to .config
#

***********************************************************

Initially reported by: Bensuperpc <bensuperpc@gmail.com>
URL: <https://github.com/Bensuperpc>

Comment:
Raspberry PI 4 aarch64

***********************************************************

Now configured for "aarch64-rpi4-linux-gnu"

      • Setup the configuration

        • Use ct-ng menuconfig, and the below TUI will show up. Then, we can go to each tab to adjust the default config.

        • Tab Paths and misc options

          • Allow building as root user (the default user in a docker container is root.
          • Adjust the path below into ${CT_TOP_DIR}
          • Uncheck
          • Set patch path for multiarch

        • Tab Toolchain options

        • Tab Operating System

        • Tab Binary utilities

        • Tab C-library

        • Tab C compiler

        • Save and exit

          • All the versions I chose are based on the previous findings on RPI OS.

        • Build the toolchain

          • export DEB_TARGET_MULTIARCH=aarch64-linux-gnu
          • ct-ng build
          • It takes 20 minutes on my computer. After building, mv the result directory aarch64-rpi4-linux-gnu into /opt/x-tools , and since /opt is a docker volume outside of the image, it will last even if I start a new container without building it again.

      Check the final results

      root@docker-desktop:~# tree /opt/x-tools/aarch64-rpi4-linux-gnu -L 2
/opt/x-tools/aarch64-rpi4-linux-gnu
|-- aarch64-rpi4-linux-gnu
|   |-- bin
|   |-- debug-root
|   |-- include
|   |-- lib
|   |-- lib64
|   `-- sysroot
|-- bin
|   |-- aarch64-rpi4-linux-gnu-addr2line
|   |-- aarch64-rpi4-linux-gnu-ar
|   |-- aarch64-rpi4-linux-gnu-as
|   |-- aarch64-rpi4-linux-gnu-c++
|   |-- aarch64-rpi4-linux-gnu-c++filt
|   |-- aarch64-rpi4-linux-gnu-cc -> aarch64-rpi4-linux-gnu-gcc
|   |-- aarch64-rpi4-linux-gnu-cpp
|   |-- aarch64-rpi4-linux-gnu-ct-ng.config
|   |-- aarch64-rpi4-linux-gnu-elfedit
|   |-- aarch64-rpi4-linux-gnu-g++
|   |-- aarch64-rpi4-linux-gnu-gcc
|   |-- aarch64-rpi4-linux-gnu-gcc-10.5.0
|   |-- aarch64-rpi4-linux-gnu-gcc-ar
|   |-- aarch64-rpi4-linux-gnu-gcc-nm
|   |-- aarch64-rpi4-linux-gnu-gcc-ranlib
|   |-- aarch64-rpi4-linux-gnu-gcov
|   |-- aarch64-rpi4-linux-gnu-gcov-dump
|   |-- aarch64-rpi4-linux-gnu-gcov-tool
|   |-- aarch64-rpi4-linux-gnu-gdb
|   |-- aarch64-rpi4-linux-gnu-gdb-add-index
|   |-- aarch64-rpi4-linux-gnu-gprof
|   |-- aarch64-rpi4-linux-gnu-ld
|   |-- aarch64-rpi4-linux-gnu-ld.bfd
|   |-- aarch64-rpi4-linux-gnu-ldd
|   |-- aarch64-rpi4-linux-gnu-nm
|   |-- aarch64-rpi4-linux-gnu-objcopy
|   |-- aarch64-rpi4-linux-gnu-objdump
|   |-- aarch64-rpi4-linux-gnu-populate
|   |-- aarch64-rpi4-linux-gnu-ranlib
|   |-- aarch64-rpi4-linux-gnu-readelf
|   |-- aarch64-rpi4-linux-gnu-size
|   |-- aarch64-rpi4-linux-gnu-strings
|   `-- aarch64-rpi4-linux-gnu-strip
|-- build.log.bz2
|-- include
|-- lib
|   |-- gcc
|   `-- ldscripts
|-- libexec
|   `-- gcc
`-- share
    |-- gcc-10.5.0
    |-- gdb
    `-- licenses

      Conclusion

      Now, I have workable toolchain and chroot so that I can start to test the cross-compiling.

      Comments

      Post a Comment

      Popular posts from this blog

      RPI4 Cross Compile for libcamera-app on Mac Pro (with docker) (4): Build libcamera-apps

        RPI4 Cross Compile for libcamera-app on Mac Pro (with docker) (4): Build libcamera-apps Why am I doing this After reading the  camera app instructions , I thought of starting the development by modifying the libcamera-apps. Therefore, the first step is to build the original repo; after that, I can add my stuff to the official standard version. It looks easy. The  code-building instructions  are well-documented on the official website, but there is a catch in my mind. I bought an expensive, neat, and tidy Macbook Pro after quitting my job as a mind-comforting geek gadget to compensate for losing my working life, so I don't want to build the code on a small RPI board. Instead, I want to build the code and develop things on my new laptop, which is why I got into this over tons of unexpected things just popping up in my sight… Preface Create a docker container where I can cross-compile the libcamera-apps. Setup chroot Setup toolchain Build libcamera-apps  <- Th...
      Post a Comment
      Read more

      RPI camera → website → Chromecast → TV: (2) Streaming the video to TV

       RPI camera → website → Chromecast → TV: (2) Streaming the video to TV Why am I doing this? As an unemployed engineer, I need to find interesting things to fill my leisure time. I found there is a thing I can do to make my life more convenient to see the toys under my couch without in the crowing position of bending my knee and waist to find the toys which are "eaten by the couch," my kids said. Maybe I can use a camera on RPI to see the area under the couch and project the streaming video onto the TV by Chromecast. That is my initial thought. Devices An RPI4 A V2 camera My computer that I can run a server on. A Chromecast and TV Goal Streaming the video to TV From the previous post , I can stream camera data from RPI to my computer; this time, I need to project the camera data to my TV. There are many ways to send the video to TV. My initial thoughts are that if I can write a website to show videos, I can project the webpage using the browser's built-in button, and all t...
      Post a Comment
      Read more

      RPI4 Cross Compile for libcamera-app on Mac Pro (with docker) (1): Preface

        RPI4 Cross Compile for libcamera-app on Mac Pro (with docker) (1): Preface Why am I doing this After reading the camera app instructions , I thought of starting the development by modifying the libcamera-apps. Therefore, the first step is to build the original repo; after that, I can add my stuff to the official standard version. It looks easy. The code-building instructions are well-documented on the official website, but there is a catch in my mind. I bought an expensive, neat, and tidy Macbook Pro after quitting my job as a mind-comforting geek gadget to compensate for losing my working life, so I don't want to build the code on a small RPI board. Instead, I want to build the code and develop things on my new laptop, which is why I got into this over tons of unexpected things just popping up in my sight… Goals Create a docker container where I can cross-compile the libcamera-apps. Setup chroot Setup toolchain Build libcamera-apps The docker container should include the d...
      Post a Comment
      Read more