Linux on the LEON3

This guide shows how to build a Linux boot image for a LEON3 embedded system. All software is manually compiled from source. No pre-built binaries or magical build scripts are needed.

There are other ways to get Linux running on the LEON3. For an overview, see LEON Linux Development overview (PDF) by Daniel Hellstrom of Aeroflex Gaisler. For example, Gaisler provides pre-built toolchains and scripts to build a custom Linux environment. If you want to get started quickly, you may want to look there first. On the other hand, if you want to build everything from scratch and understand how the pieces fit together, keep reading this guide.

Our boot image will be based on Linux, uClibc and BusyBox. A root filesystem will be built into the boot image as an initramfs. The root filesystem is thus not mounted from a block device like a normal PC would do.

Builing the Linux image is a three-step process:

Build a toolchain (binutils, gcc, uClibc);
Prepare a root filesystem (busybox, scripts);
Build a Linux kernel with root filesystem in initramfs.

Necessary tools:

Knowledge of the LEON3 processor and experience with GRMON;
FPGA board with a working LEON3 and at least 32 MB RAM;
LEON3 configured with MMU enabled and APBUART serial port;
Linux PC (for example Debian 6.0) to compile the software.

WARNING: Users and processes are not isolated from each other in a secure way when Linux runs on a LEON3. The reason is that the LEON3 allows unprivileged processes to write to system management registers %asr17 and %asr19. Furthermore, on a LEON3 without FPU, unprivileged processes can crash the system by executing an FPU instruction. Finally the LEON3 multiply-accumulate instruction is not suported under Linux because the contents of %asr18 are not saved during task switches.

This guide is strongly based on Cross-Linux-From-Scratch.
Other sources are the MKLINUXIMG User Manual (PDF) by Daniel Hellstrom
and Building GCC with uClibc for XScale.

Step 1: Building a cross-compilation toolchain

The goal of this step is to build a cross-compilation toolchain which runs on a normal PC (e.g. x86) but compiles C programs for the LEON3. Later steps will use this toolchain to compile the Linux kernel and to compile system tools.

Step 1.1: Preparation

Create a directory where the cross-compilation toolchain will be installed. Add the cross-compilation toolchain to the path. Set ${TOOLS} to point to the location of the cross-compilation toolchain, for easy reference in the rest of this manual.

$ mkdir /somewhere/sparc-linux $ PATH="$PATH:/somewhere/sparc-linux/usr/bin" $ TOOLS=/somewhere/sparc-linux

Step 1.2: binutils

Download binutils binutils-2.23 from http://www.gnu.org/software/binutils/

$ tar xzvf binutils-2.23.tar.gz $ mkdir b-binutils $ cd b-binutils $ ../binutils-2.23/configure --prefix=${TOOLS}/usr --target=sparc-leon3-linux \ --with-sysroot=${TOOLS} --disable-nls --disable-multilib \ --with-cpu=v8 --with-float=soft $ make $ make install $ cd .. $ rm -r b-binutils

Step 1.3: kernel headers

Download Linux 3.8 from http://kernel.org/

$ tar xjvf linux-3.8.tar.bz2 $ cd linux-3.8 $ make mrproper $ make ARCH=sparc headers_check $ make ARCH=sparc INSTALL_HDR_PATH=${TOOLS}/usr headers_install

Step 1.4: GCC pass 1 (without libc)

The full compiler can not yet be built because it depends on libc. But libc can not be installed without a compiler. To escape this paradox, we first do a partial installation of GCC which can be used to build libc.

Download GCC-4.7.2 from http://gcc.gnu.org/

$ tar xjvf gcc-4.7.2.tar.bz2 $ mkdir b-gcc $ cd b-gcc $ ../gcc-4.7.2/configure --prefix=${TOOLS}/usr --target=sparc-leon3-linux \ --disable-nls --disable-shared --disable-multilib --disable-libgomp \ --disable-libmudflap --disable-libssp --disable-threads \ --enable-languages=c --with-cpu=v8 --with-float=soft $ make all-gcc all-target-libgcc $ make install-gcc install-target-libgcc $ cd .. $ rm -r b-gcc

Step 1.5: uClibc

Download uClibc-0.9.33.2 from http://www.uclibc.org/

$ tar xjvf uClibc-0.9.33.2.tar.bz2 $ cd uClibc-0.9.33.2 $ make menuconfig

Target Architecture:	sparc
Target Options:
Processor type:	SPARC v8
Target has MMU:	yes
Utilize MMU:	yes
Enable floating point:	yes
Target has FPU:	no
Enable C99 math:	yes
KERNEL_HEADERS:	`/somewhere/sparc-linux/usr/include`
General Library Settings:
Thread support:	native POSIX threading
SUSv3 legacy functions:	yes
SUSv3 legacy macros:	yes
SUSv4 legacy functions:	yes
Networking:
IPv6:	yes
RPC support:	yes
Full RPC:	no
DNS resolver functions:	yes
String support:
ctype argument checking:	detect and handle
Installation options:
RUNTIME_PREFIX:	`/`
DEVEL_PREFIX:	`/usr`
HARDWIRED_ABSPATH:	no
Development:
CROSS_COMPILER_PREFIX:	`sparc-leon3-linux-`

$ make $ make PREFIX=${TOOLS} install

Step 1.6: GCC pass 2 (full installation)

Now that uClibc is in place, we can do a full installation of GCC. The following commands build a compiler for C and C++. The C++ run-time library takes 600 kByte extra, so you may want to disable C++ if you are not going to use it.

$ mkdir b-gcc $ cd b-gcc $ ../gcc-4.7.2/configure --prefix=${TOOLS}/usr --target=sparc-leon3-linux \ --with-sysroot=${TOOLS} --disable-nls --enable-shared --disable-multilib \ --disable-libgomp --disable-libmudflap --enable-threads=posix \ --enable-languages="c,c++" --with-gnu-as --disable-libitm \ --with-cpu=v8 --with-float=soft $ make $ make install-strip $ cd .. $ rm -r b-gcc

The cross-compiler is now installed in ${TOOLS}/usr/bin. It can be invoked as sparc-leon3-linux-gcc on the command-line to build C programs for the LEON3.

However, some LEON3 support tools from Gaisler (mkprom, mklinuximg) expect the compiler to be named sparc-linux-gcc instead of sparc-leon3-linux-gcc. The name of the compiler is derived from the --target value during GCC configuration, and this value MUST be "sparc-leon3-linux", otherwise GCC will not work properly. We will just create aliases for the essential compiler commands so that the Gaisler tools can find them:

$ ln -s sparc-leon3-linux-gcc ${TOOLS}/usr/bin/sparc-linux-gcc $ ln -s sparc-leon3-linux-ld ${TOOLS}/usr/bin/sparc-linux-ld $ ln -s sparc-leon3-linux-objdump ${TOOLS}/usr/bin/sparc-linux-objdump $ ln -s sparc-leon3-linux-objcopy ${TOOLS}/usr/bin/sparc-linux-objcopy $ ln -s sparc-leon3-linux-readelf ${TOOLS}/usr/bin/sparc-linux-readelf

Step 2: Root filesystem

The goal of this step is to create a directory containing a complete set of system libraries, system tools and boot scripts to act as the root filesystem on the LEON3.

Step 2.1: Preparation

Create a directory where the LEON3 root filesystem will be installed. Set ${SYSROOT} to point to the location of the root filesystem for easy reference in the rest of this manual.

$ mkdir /somewhere/sysroot $ export SYSROOT=/somewhere/sysroot

The root filesystem is put together as a normal user (no root privs needed). Files may be owned by a normal user; ownership will be changed to root when the final filesystem image is built.

Step 2.2: System libraries

Some shared libraries were already installed by GCC and uClibc. These are run-time libraries, so they are needed in the root filesystem. Let's copy them there.

$ cp -a ${TOOLS}/lib ${SYSROOT} $ cp -a ${TOOLS}/usr/sparc-leon3-linux/lib/*.so* ${SYSROOT}/lib $ cd ${SYSROOT}/lib $ ln -s ld-uClibc*.so ld-linux.so.2

Step 2.3: Busybox

Download busybox-1.20.2 from http://www.busybox.net/

$ tar xjvf busybox-1.20.2.tar.bz2 $ cd busybox-1.20.2 $ make menuconfig

BusyBox Settings:
General configuration:
Don't use /usr	yes
Support Unicode	no
Build Options:
CONFIG_CROSS_COMPILER_PREFIX:	`sparc-leon3-linux-`
CONFIG_SYSROOT: (this is not the same thing as our `${SYSROOT}`)	`/somewhere/sparc-linux`
CONFIG_EXTRA_CFLAGS:	`-D_XOPEN_SOURCE`
Library Tuning:
systemd support:	no
Use clock_gettime(CLOCK_MONOTONIC):	yes
Support infiniband:	no
Print Utilities:	(disable all)
Mail Utilities:	(disable all)
Shell Utilities:
hush:	no

$ make busybox $ make CONFIG_PREFIX=${SYSROOT} install $ ln -s bin/busybox ${SYSROOT}/init

Step 2.4: Base configuration

Boot scripts and configuration files must be installed in ${SYSROOT}/etc. The easiest way to explain what is needed there is by providing the whole set of files as a download.

Download http://jorisvr.nl/files/leon3/leon3-linux-stuff-20130228.tar.gz.
This package contains boot scripts derived from Cross-Linux-From-Scratch.

$ tar xzvf leon3-linux-stuff-20130228.tar.gz $ cp -a leon3-linux-stuff-20130228/etc ${SYSROOT}

The following will be set up:

etc/inittab	(enable login on console and serial port)
etc/passwd	(just a "root" account with empty password)
etc/group	(default system groups)
etc/fstab	(mount proc filesystem)
etc/hosts	(mapping IP addresses to names)
etc/shells	(list of allowed user shells)
etc/protocols	(copied from Debian)
etc/services	(copied from Debian)
etc/issue	(welcome message)
etc/rc.d	(startup scripts for network, telnet, dropbear)

The LEON3 will boot with IP address 192.168.0.80 for the Ethernet port. Edit ${SYSROOT}/etc/rc.d/init.d/network if you want to change the IP address.

Edit ${SYSROOT}/etc/rc.d/init.d/telnetd if you want to start the telnet daemon during system boot.

Step 2.5: Zlib (optional, but required for Dropbear)

Download zlib-1.2.7 from http://www.zlib.net/

$ tar xzvf zlib-1.2.7.tar.gz $ cd zlib-1.2.7 $ CROSS_PREFIX=sparc-leon3-linux- CFLAGS=-Os ./configure --prefix=/usr --eprefix=/ $ make $ sparc-leon3-linux-strip libz.so $ make prefix=${TOOLS}/usr exec_prefix=${TOOLS}/usr install $ cp -a ${TOOLS}/usr/lib/libz*.so* ${SYSROOT}/lib

Step 2.6: Dropbear SSH (optional)

Download dropbear-2012.55 from http://matt.ucc.asn.au/dropbear/dropbear.html

$ tar xjvf dropbear-2012.55.tar.bz2 $ cd dropbear-2012.55 $ CFLAGS=-Os ./configure --prefix=/ --host=sparc-leon3-linux $ make MULTI=1 PROGRAMS="dropbear dbclient dropbearkey dropbearconvert scp" $ fakeroot make MULTI=1 PROGRAMS="dropbear dbclient dropbearkey dropbearconvert scp" \ DESTDIR=${SYSROOT} strip installdropbearmulti

Prepare an RSA host key for Dropbear:

$ mkdir ${SYSROOT}/etc/dropbear $ ssh-keygen -t rsa -N "" -f ${SYSROOT}/etc/dropbear/openssh_rsa_host_key

Step 2.7: IPtables (optional)

Download iptables-1.4.17 from http://www.netfilter.org/projects/iptables/

$ tar xjvf iptables-1.4.17.tar.bz2 $ cd iptables-1.4.17 $ ./configure --prefix=/usr --exec-prefix=/ --libdir=/usr/lib --host=sparc-leon3-linux \ --without-kernel --disable-shared $ make $ mkdir tmpinst $ make DESTDIR="$PWD/tmpinst" install-strip $ cp -a tmpinst/bin ${SYSROOT} $ cp -a tmpinst/sbin ${SYSROOT}

Step 2.8: IPerf (optional)

Download iperf-2.0.5 from http://sourceforge.net/projects/iperf/

$ tar xzvf iperf-2.0.5.tar.gz $ cd iperf-2.0.5 $ ac_cv_func_malloc_0_nonnull=yes ./configure --prefix=/ --host=sparc-leon3-linux $ make $ make DESTDIR=${SYSROOT} install-strip

(Many thanks to Marc Singer for his explanation of rpl_malloc.)

Step 2.9: Strace (optional)

Download strace-4.7 from http://sourceforge.net/projects/strace/

$ tar xJvf strace-4.7.tar.xz $ cd strace-4.7 $ ./configure --prefix=/ --host=sparc-leon3-linux $ make $ make DESTDIR=${SYSROOT} install-strip

Step 2.10: SoCat (optional)

Download socat-1.7.2.1 from http://www.dest-unreach.org/socat/

$ tar xjvf socat-1.7.2.1.tar.bz2 $ cd socat-1.7.2.1 $ ./configure --prefix=/ --host=sparc-leon3-linux $ make strip $ install socat procan filan ${SYSROOT}/bin

Step 2.11: RSync (optional)

Download rsync-3.0.9 from http://rsync.samba.org/

$ tar xzvf rsync-3.0.9.tar.gz $ cd rsync-3.0.9 $ ./configure --prefix=/ --host=sparc-leon3-linux $ make $ sparc-leon3-linux-strip rsync $ install rsync ${SYSROOT}/bin

Step 3: Linux kernel

The goal of this step is to create a Linux kernel boot image. The root filesystem, built in step 2, will be included in the Linux kernel as an initramfs image.

Step 3.1: Configure the kernel

Download the Linux kernel from kernel.org. Set ${LINUXSRC} to point to the Linux source directory, for easy reference in the rest of this manual.

$ tar xjvf linux-3.8.tar.bz2 $ export LINUXSRC=${PWD}/linux-3.8 $ cd linux-3.8

A patch must be applied to avoid crashing the kernel on FPU instructions. Without this patch, the system will boot and work fine, but any user can crash the system by executing an invalid instruction.

$ patch -p1 < ../leon3-linux-stuff-20130228/patches/linux-3.8_sparcfp.diff

A patch must be applied to add support for PS/2 keyboard and mouse devices that are attached to the LEON3 processor via APBPS2. (Patch by Daniel Hellstrom from Gaisler. More kernel patches are available in the leon-linux package from Gaisler.)

$ patch -p1 < ../leon3-linux-stuff-20130228/patches/linux-3.8_apbps2-added-GRLIB-APBPS2-PS-2-Keyboard-and-Mouse-d.patch

Configure the Linux kernel. Many defaults can be disabled to make the kernel smaller. (Optionally copy leon3-linux-stuff-20130228/linux-3.8-config to .config and use it as a starting point.)

$ make ARCH=sparc CROSS_COMPILE=sparc-leon3-linux- menuconfig

64-bit kernel:	no
General:
Support paging (swap):	no
Timers, High resolution:	no
CPU accounting:	disable all
Initramfs:	yes
Initramfs source file:	`../sysroot.cpio`
Initramfs compression:	gzip
Loadable modules:	no
Block layer:
Partitions, Adv sel:	no
IO schedulers:	disable all
Processor:
SMP:	no
Sparc Leon family:	yes
Bus options:
Support for PCI:	yes (REQUIRED!)
GRPCI:	yes
Network:
Options:
PF_KEY:	no
Large receive offload:	no
802.1d bridging:	no
IPX:	no
Appletalk:	no
QoS:	no
IrDA:	no
Device Drivers:
SCSI device support:	no
Serial ATA:	no
Multiple devices (RAID):	no
Network device:
Ethernet, Aeroflex Gaisler GRETH:	yes
Ethernet, all others:	no
USB:	disable all
Wireless:	no
Input:
Mouse, PS/2:	yes
Hardware IO, GRLIB APBPS2:	yes (keyboard IRQ 5)
Character devices:
Serial, GRLIB APBUART:	yes (console: yes)
Multimedia:	no
Graphics:
Framebuf, Aeroflex Gaisler:	yes
Framebuf, Console:	enable Sparc 8x16 font
Sound card:	no
USB:	no
Real Time Clock:	no
IOMMU:	no
File systems:	(disable most, but keep proc and tmpfs)

Step 3.2: Assemble root filesystem image

Run a script from the Linux kernel source to make a list of files that are to be included in the initramfs image. Give setuid permission to the busybox binary. Add a bunch of device nodes and mount points. Run a helper program from the Linux kernel source to build the actual CPIO archive containing the root filesystem.

$ cd ${LINUXSRC}/usr $ make gen_init_cpio $ cd ../.. $ sh ${LINUXSRC}/scripts/gen_initramfs_list.sh -u `id -u` -g `id -g` ${SYSROOT} \ | sed -e 's/^file $\/bin\/busybox .*$ 755 0 0/file \1 4755 0 0/' \ > sysroot.files $ cat <<EOF >> sysroot.files dir /dev 755 0 0 dir /proc 755 0 0 dir /sys 755 0 0 dir /tmp 755 0 0 dir /var 755 0 0 dir /root 700 0 0 nod /dev/null 666 0 0 c 1 3 nod /dev/tty1 600 0 0 c 4 1 nod /dev/ttyS0 600 0 0 c 4 64 nod /dev/tty 666 0 0 c 5 0 nod /dev/console 600 0 0 c 5 1 EOF $ ${LINUXSRC}/usr/gen_init_cpio sysroot.files > sysroot.cpio

(Or use the script leon3-linux-stuff-20130228/make_initramfs.sh to do this entire step.)

Step 3.3: Compile the kernel

$ cd ${LINUXSRC} $ make ARCH=sparc CROSS_COMPILE=sparc-leon3-linux-

Step 3.4: Create executable image for LEON3

Before the kernel can run, it must be loaded in a specific location in RAM and the MMU must be initialized. The kernel also needs the OpenBoot API to handle platform-specific functions. These functions are provided by the LEON Linux RAM loader from Gaisler.

Download mklinuximg-2.6.36-2.0.3 from http://www.gaisler.com/index.php/downloads/linux

Two patches must be applied. The first patch makes the loader work with recent versions of the Linux kernel. The second patch works around a linker issue with binutils 2.23.

$ tar xjvf mklinuximg-2.6.36-2.0.3.tar.bz2 $ cd mklinuximg-2.6.36-2.0.3 $ patch -p1 < ../leon3-linux-stuff-20130228/patches/mklinuximg-2.6.36-2.0.3_compat.diff $ patch -p1 < ../leon3-linux-stuff-20130228/patches/mklinuximg-2.6.36-2.0.3_fixld.diff $ cd ..

Run a script to compile the loader and merge it with the Linux kernel from the previous step.

$ mklinuximg-2.6.36-2.0.3/mklinuximg ${LINUXSRC}/vmlinux linux.dsu

The file linux.dsu is proper LEON3 executable. It can be loaded and executed with GRMON:

grlib> load linux.dsu grlib> run

Step 3.5: Create PROM image for LEON3 (optional)

Linux can boot directly from a flash PROM. In this case, a PROM image must be created to write into the flash device. We will use MKPROM2, the boot loader from Gaisler.

Download MKPROM2 from http://gaisler.com/index.php/downloads/compilers.
Compile MKPROM2:

$ tar xzvf mkprom2-2.0.51.tar.gz $ cd mkprom2/src $ make TARGET=sparc-leon3-linux PREFIX=${TOOLS}/mkprom2 $ cd ../..

Run MKPROM2 to create a Linux PROM image. MKPROM2 needs command-line options that are specific for the board and LEON3 configuration (type and amount of memory, system clock frequency, etc.).

$ ${TOOLS}/mkprom2/mkprom2 -baud 38400 {board options} linux.dsu -o linux.prom

The file linux.prom can be flashed into the PROM device, for example with GRMON.

-- The End