week 2 · day 13

The System V Calling Convention

rdi, rsi, rdx… who passes what

Easy 36 min 180 xp

After this you can

› Explain why a calling convention must exist for functions to interoperate
› State the System V AMD64 argument registers, the return register, and the order
› Distinguish caller-saved from callee-saved registers and say why the split matters
› Read a function call in assembly and identify each argument and the return value

A contract nobody can skip

Yesterday call and ret moved control in and out of a function. But a function also needs data — its arguments — and it produces a result. Where do arguments go? Which registers is a function allowed to clobber? If the caller and the callee disagree, everything breaks. So there's a contract: a fixed agreement on where arguments live, where the return value goes, and who is responsible for preserving which registers. On Linux and macOS for x86-64 that contract is the System V AMD64 ABI. Every compiled function obeys it, which is why code from different compilers and languages can call each other at all.

Where the arguments go

Integer and pointer arguments are passed in registers, in this exact order:

Arg #	Register	Note
1st	rdi	first integer/pointer argument
2nd	rsi
3rd	rdx
4th	rcx
5th	r8
6th	r9	sixth; further args go on the stack
return	rax	the function's return value comes back here

Key idea

Memorize this order — rdi, rsi, rdx, rcx, r8, r9 — because you'll use it forever: to read any call, to write assembly that calls C, and (later) to control a program during exploitation, where "put my value in rdi before this call" is a core move. The return value is always in rax. A 7th+ argument is pushed onto the stack. This is the day 7 mystery ("a and b arrived in edi/esi") made into a rule.

Reading a call

Here's add(2, 3) set up and called. Watch the convention:

calling add(2, 3)asm

1  mov edi, 2         ; 1st arg -> rdi (edi = its 4-byte view, since int)
2  mov esi, 3         ; 2nd arg -> rsi
3  call add           ; jump in; add reads edi/esi, leaves result in eax
4  ; now eax = 5      ; return value is in rax/eax

You didn't need add's source. The convention *told* you: first argument in rdi, second in rsi, result in rax. Reading calls becomes mechanical — find the movs into rdi/rsi/rdx… before a call, and you've found the arguments; look at rax after, and you've found the result.

Who preserves what: caller-saved vs callee-saved

There's a second half of the contract, and it prevents chaos when functions clobber registers. The registers are split into two groups:

Callee-saved (rbx, rbp, r12–r15): a function that uses these must save and restore them, so
the caller sees them unchanged across the call. If you hold a value in rbx across a call, it
survives.
Caller-saved (rax, rcx, rdx, rsi, rdi, r8–r11): a function may freely clobber these. If you
need a value in one of them across a call, *you* must save it first (e.g. push it), because the
callee might trash it.

Note

The split is a division of labour: someone has to save a register's old value around a call, and the ABI decides who. For callee-saved registers the *function* takes responsibility (that's why you often see push rbx in a prologue and pop rbx in the epilogue). For caller-saved, the *caller* does. Get this wrong by hand and you get corruption bugs that look like magic — the ABI is what makes separately compiled code safe to combine.

Predict first

You call memcpy(dst, src, n) — a 3-argument function. Before the call, which register holds dst, which holds src, and which holds n? And where would memcpy's return value appear?

finished reading?

Your task, you write the code

Confirm the convention

Write abi.c with a function f(int a, int b, int c, int d) that returns a - b + c - d, called from main with four distinct constants. Compile at -O0 and disassemble. In main's disassembly, confirm the four arguments are loaded into rdi, rsi, rdx, rcx (in that order) before the call, and that after the call the result is read from rax. Then find f's prologue and see whether it saves any callee-saved register. Uses only the ABI rules from this lesson.

deliverable: abi.c (+ annotated disassembly of the call site)

build & run

$ gcc -O0 -g abi.c -o abi
$ objdump -d -M intel abi

self-review before running

› You matched each of the four arguments to rdi/rsi/rdx/rcx in order
› You found the return value being taken from rax after the call
› You can state the six integer-argument registers from memory
› You can explain the difference between caller-saved and callee-saved

stretchGive f a seventh and eighth parameter and find where the 7th and 8th arguments go (hint: not registers). Then add a local you keep across a nested call and watch the compiler choose a callee-saved register (and save it in the prologue) to hold it.

Self-check

01The first six integer/pointer arguments go in, in order:

02Where does a function's return value come back?

03A callee-saved register (like rbx) means:

04Why does a calling convention need to exist at all?

0/4 correct · 0/4 checked