RISC-V Bytes: Cross-Platform Debugging with QEMU and GDB

June 1, 2021

This is part of a new series I am starting on the blog where we’ll explore RISC-V by breaking down real programs and explaining how they work. You can view all posts in this series on the RISC-V Bytes page.

To start of the series, we are just going to get setup to do some exploration. I am going to assume you will not primarily be using a RISC-V machine1, so we need to configure our local development environment for cross-platform compiling, emulation, and debugging.

Installing Tools

As the title of this post suggests, we are going to install the RISC-V GNU toolchain for our C compiler (gcc) and our debugger (gdb). We’ll be using the 64-bit variant of the RISC-V ISA throughout this series. If you are running Ubuntu, the easiest way to install the toolchain is by by downloading the latest release from the RISC-V GNU toolchain GitHub repository. However, there are also extensive instructions for how to compile from source in the README.md. This will provide you with all of the following tools:


riscv64-unknown-elf-addr2line      riscv64-unknown-elf-elfedit        riscv64-unknown-elf-gcc-ranlib     riscv64-unknown-elf-gprof          riscv64-unknown-elf-ranlib
riscv64-unknown-elf-ar             riscv64-unknown-elf-g++            riscv64-unknown-elf-gcov           riscv64-unknown-elf-ld             riscv64-unknown-elf-readelf
riscv64-unknown-elf-as             riscv64-unknown-elf-gcc            riscv64-unknown-elf-gcov-dump      riscv64-unknown-elf-ld.bfd         riscv64-unknown-elf-size
riscv64-unknown-elf-c++            riscv64-unknown-elf-gcc-8.3.0      riscv64-unknown-elf-gcov-tool      riscv64-unknown-elf-nm             riscv64-unknown-elf-strings
riscv64-unknown-elf-c++filt        riscv64-unknown-elf-gcc-ar         riscv64-unknown-elf-gdb            riscv64-unknown-elf-objcopy        riscv64-unknown-elf-strip
riscv64-unknown-elf-cpp            riscv64-unknown-elf-gcc-nm         riscv64-unknown-elf-gdb-add-index  riscv64-unknown-elf-objdump

So we can now compile and and run a debugger for RISC-V, but we won’t be able to actually run our programs without emulation. QEMU is one of the most popular and widely used emulation platforms in the world. It offers full-system and user-mode emulation2 for a wide variety of platforms, including RISC-V. There are instructions for installing on Linux, macOS, and Windows on the downloads page, as well as links to source and build instructions. After installing, you should have binaries for each of the following platforms:


qemu-aarch64              qemu-ga                   qemu-microblaze           qemu-mipsn32el            qemu-ppc64le              qemu-sparc                qemu-system-riscv64
qemu-aarch64_be           qemu-hppa                 qemu-microblazeel         qemu-nbd                  qemu-pr-helper            qemu-sparc32plus          qemu-system-x86_64
qemu-alpha                qemu-i386                 qemu-mips                 qemu-nios2                qemu-riscv32              qemu-sparc64              qemu-system-x86_64-spice
qemu-arm                  qemu-img                  qemu-mips64               qemu-or1k                 qemu-riscv64              qemu-storage-daemon       qemu-tilegx
qemu-armeb                qemu-io                   qemu-mips64el             qemu-ppc                  qemu-s390x                qemu-system-aarch64       qemu-x86_64
qemu-cris                 qemu-m68k                 qemu-mipsel               qemu-ppc64                qemu-sh4                  qemu-system-arm           qemu-xtensa
qemu-edid                 qemu-make-debian-root     qemu-mipsn32              qemu-ppc64abi32           qemu-sh4eb                qemu-system-i386          qemu-xtensaeb

You’ll see a few for RISC-V: qemu-riscv32, qemu-riscv64, qemu-system-riscv64. The first two are the 32-bit and 64-bit variants for user-mode emulation, and the third is the 64-bit variant for full-system emulation.

A First Example

Now that we have our tools in place, let’s use them! We’ll start by compiling a very small C program:

sum.c


#include <stdio.h>

int main()
{
    int num1 = 1;
    int num2 = 2;
    int sum = num1 + num2;

    printf("The sum is: %d", sum);
    return 0;
}

When debugging, it is helpful to compile with additional debugging symbols and information. gcc supports a variety of flags to produce debugging information, but if using gdb, the -ggdb flag will produce debug information specifically for gdb:

riscv64-unknown-elf-gcc -ggdb -static -o sum sum.c

We now have a sum binary, let’s try and run it:

./sum

This should not work because our binary is compiled with a RISC-V target. Oops! Looks like that worked? Did we use the wrong compiler? Let’s check:


$ file sum
sum: ELF 64-bit LSB executable, UCB RISC-V, version 1 (SYSV), statically linked, with debug_info, not stripped

We can see that sum is a 64-bit ELF binary compiled for RISC-V, so what’s going on here? Without knowing it, you may already have jumped ahead to our emulation step. If you are running Linux, the kernel supports a feature called binfmt_misc. This allows for an interpreter to be registered and invoked automatically for a specified binary type. When we installed QEMU, it went ahead and registered our user-mode emulators:


$ ls /proc/sys/fs/binfmt_misc/
jar               qemu-aarch64      qemu-armeb        qemu-m68k         qemu-mips64       qemu-mipsn32      qemu-ppc64        qemu-riscv32      qemu-sh4          qemu-sparc32plus  qemu-xtensaeb
python2.7         qemu-alpha        qemu-cris         qemu-microblaze   qemu-mips64el     qemu-mipsn32el    qemu-ppc64abi32   qemu-riscv64      qemu-sh4eb        qemu-sparc64      register
python3.8         qemu-arm          qemu-hppa         qemu-mips         qemu-mipsel       qemu-ppc          qemu-ppc64le      qemu-s390x        qemu-sparc        qemu-xtensa       status

This is great! However, this isn’t going to work when we are debugging, so we need to run our RISC-V gdb while pointing at qemu-riscv64. First lets start our debugging session:


$ riscv64-unknown-elf-gdb sum
(gdb) run
Don't know how to run.  Try "help target".
(gdb)

You’ll notice that if we try to run, gdb will prompt us to add a target. QEMU supports the gdbstub remote connection protocol, and we can start a gdb server in user-mode QEMU by passing -g <port>:

qemu-riscv64 -g 1234 sum

This will start QEMU, but wait for gdb to connect, which we can do by setting a remote target:


(gdb) target remote :1234
Remote debugging using :1234
0x00000000000100c6 in _start ()
(gdb)

Let’s verify our main function exists:


(gdb) disass main
Dump of assembler code for function main:
   0x0000000000010158 <+0>:     addi       sp,sp,-32
   0x000000000001015a <+2>:     sd         ra,24(sp)
   0x000000000001015c <+4>:     sd         s0,16(sp)
   0x000000000001015e <+6>:     addi       s0,sp,32
   0x0000000000010160 <+8>:     li         a5,1
   0x0000000000010162 <+10>:    sw         a5,-20(s0)
   0x0000000000010166 <+14>:    li         a5,2
   0x0000000000010168 <+16>:    sw         a5,-24(s0)
   0x000000000001016c <+20>:    lw         a4,-20(s0)
   0x0000000000010170 <+24>:    lw         a5,-24(s0)
   0x0000000000010174 <+28>:    addw       a5,a5,a4
   0x0000000000010176 <+30>:    sw         a5,-28(s0)
   0x000000000001017a <+34>:    lw         a5,-28(s0)
   0x000000000001017e <+38>:    mv         a1,a5
   0x0000000000010180 <+40>:    lui        a5,0x1c
   0x0000000000010182 <+42>:    addi       a0,a5,176 # 0x1c0b0
   0x0000000000010186 <+46>:    jal        ra,0x10332 <printf>
   0x000000000001018a <+50>:    li         a5,0
   0x000000000001018c <+52>:    mv         a0,a5
   0x000000000001018e <+54>:    ld         ra,24(sp)
   0x0000000000010190 <+56>:    ld         s0,16(sp)
   0x0000000000010192 <+58>:    addi       sp,sp,32
   0x0000000000010194 <+60>:    ret
End of assembler dump.
(gdb)

Nice, now we are ready to get to work! So keep an eye out for the next post :)

Closing Thoughts

I’m excited to get this series started. My goal is to peel back some of the layers of abstraction that we commonly find ourselves interacting with on a daily basis, while also demonstrating some of the components and design decisions of RISC-V specifically that have a chance to make it the dominant ISA of the future. If I can do a better job of helping us achieve that goal, or you just have questions of comments, send me a message @hasheddan on Twitter!


  1. Though if you are, that’s awesome! I recently got a BeagleV StarLight, which you will almost certainly be seeing in future posts in this series. ↩︎

  2. We’ll be exploring both of these in future posts. ↩︎