Instruction level countermeasure against buffer overflow attacks. Slides of paper presented at Eurosys2009 VTDS. Nurmberg, March 31st 2009

1. Instruction level countermeasure
against stack-based buffer overflows
Francesco Gadaleta, Yves Younan, Bart Jacobs
Wouter Joosen, Erik De Neve, Nils Beosier
DistriNet, Department of Computer Science
Katholieke Universiteit Leuven
Belgium
Francesco.Gadaleta@cs.kuleuven.be

2. Overview
Buffer Overflows
stack-based BOF and related works
Implementation details
Results analysis
Future work
Francesco Gadaleta Instruction level countermeasure against stack-based buffer overflow attacks March 31st, 2009

3. Buffer Overflow Attacks
Most commonly associated to unsafe
languages (C/C++)
90% of BOF vulnerabilities
reported in 2008 had a high severity rating
~3$ billion damage
(according to NIST’s National Vulnerability Database, 2008)
Many different types
of BOF attacks exist
Designed countermeasures
are often affected by considerable overhead
Francesco Gadaleta Instruction level countermeasure against stack-based buffer overflow attacks March 31st, 2009

4. Buffer Overflow Attacks
Most commonly associated to unsafe
languages (C/C++)
90% of BOF vulnerabilities
reported in 2008 had a high severity rating
~3$ billion damage
(according to NIST’s National Vulnerability Database, 2008)
Many different types
of BOF attacks exist
Designed countermeasures
are often affected by considerable overhead
Francesco Gadaleta Instruction level countermeasure against stack-based buffer overflow attacks March 31st, 2009

5. Buffer Overflow Attacks
Most commonly associated to unsafe
languages (C/C++)
90% of BOF vulnerabilities
reported in 2008 had a high severity rating
~3$ billion damage
(according to NIST’s National Vulnerability Database, 2008)
Many different types
of BOF attacks exist
Designed countermeasures
are often affected by considerable overhead
Francesco Gadaleta Instruction level countermeasure against stack-based buffer overflow attacks March 31st, 2009

6. Buffer Overflow Attacks
Most commonly associated to unsafe
languages (C/C++)
90% of BOF vulnerabilities
reported in 2008 had a high severity rating
~3$ billion damage
(according to NIST’s National Vulnerability Database, 2008)
Many different types
of BOF attacks exist
Designed countermeasures
are often affected by considerable overhead
Francesco Gadaleta Instruction level countermeasure against stack-based buffer overflow attacks March 31st, 2009

7. Buffer Overflow Attacks
Most commonly associated to unsafe
languages (C/C++)
90% of BOF vulnerabilities
reported in 2008 had a high severity rating
~3$ billion damage
(according to NIST’s National Vulnerability Database, 2008)
Many different types
of BOF attacks exist
Designed countermeasures
are often affected by considerable overhead
Francesco Gadaleta Instruction level countermeasure against stack-based buffer overflow attacks March 31st, 2009

8. Buffer Overflow Attacks
Most commonly associated to unsafe
languages (C/C++)
90% of BOF vulnerabilities
reported in 2008 had a high severity rating
~3$ billion damage
(according to NIST’s National Vulnerability Database, 2008)
Many different types
of BOF attacks exist
Designed countermeasures
are often affected by considerable overhead
Francesco Gadaleta Instruction level countermeasure against stack-based buffer overflow attacks March 31st, 2009

9. Stack-based BOF attack
Francesco Gadaleta Instruction level countermeasure against stack-based buffer overflow attacks March 31st, 2009

10. Stack-based BOF attack
It’s not a bug, it’s a language feature.
void copy(char* str)
{
char buf[80];
strcpy(buf, str);
}
Francesco Gadaleta Instruction level countermeasure against stack-based buffer overflow attacks March 31st, 2009

11. Stack-based BOF attack
void copy(char* str)
{
char buf[80];
strcpy(buf, str);
}
Francesco Gadaleta Instruction level countermeasure against stack-based buffer overflow attacks March 31st, 2009

12. Stack-based BOF attack
MACHINE CODE TRANSLATION
prologue:
pushl %ebp
mov %esp, %ebp
char buf[80];
strcpy(buf, str);
epilogue:
leave
ret
Francesco Gadaleta Instruction level countermeasure against stack-based buffer overflow attacks March 31st, 2009

13. Stack-based BOF attack
MACHINE CODE TRANSLATION
prologue:
low addresses
pushl %ebp
mov %esp, %ebp
buf
Saved Frame Pointer
char buf[80];
strcpy(buf, str); Return Address
func args *str
epilogue: high addresses
leave
ret
Francesco Gadaleta Instruction level countermeasure against stack-based buffer overflow attacks March 31st, 2009

14. Stack-based BOF attack
MACHINE CODE TRANSLATION
prologue:
low addresses
pushl %ebp
mov %esp, %ebp
buf
Saved Frame Pointer
char buf[80];
strcpy(buf, str); Return Address
Return address
func args *str
epilogue: high addresses
leave
ret
Francesco Gadaleta Instruction level countermeasure against stack-based buffer overflow attacks March 31st, 2009

15. Related work
StackShield
Vendicator, 2000
• saves a copy of RET to normal memory
• can be bypassed
RAD (Return Address Defender)
Tzi-cker Chiueh, Fu-Hau Hsu, 2001
• mprotected memory (RAR)
• high overhead (140x to 200x slow-down)
Francesco Gadaleta Instruction level countermeasure against stack-based buffer overflow attacks March 31st, 2009

16. Virtualized Environments
Why a virtualized environment?
widely deployed
technology
VMOS ... VMOS
a solution to reduce Supervisor
(Host OS)
overhead of RAD
Security
improvements
at hypervisor
HYPERVISOR level
EXCEPTION/INTERRUPTS DEVICE I/O, MEMORY MANAGEMENT
H A R D W A R E
Francesco Gadaleta Instruction level countermeasure against stack-based buffer overflow attacks March 31st, 2009

17. Design
main:
call init_callxretx
... PROTECTED PAGE
prologue: PROGRAM STACK
pushl %ebp
mov %esp, %ebp
RET
<function body>
epilogue:
leave RET
ret
Francesco Gadaleta Instruction level countermeasure against stack-based buffer overflow attacks March 31st, 2009

18. Design
main:
call init_callxretx
... PROTECTED PAGE
prologue: PROGRAM STACK
pushl %ebp
mov %esp, %ebp
instrumented code
RET
<function body>
epilogue:
leave RET
ret
Francesco Gadaleta Instruction level countermeasure against stack-based buffer overflow attacks March 31st, 2009

19. Design
main:
call init_callxretx
... PROTECTED PAGE
prologue: PROGRAM STACK
pushl %ebp
mov %esp, %ebp
instrumented code
RET
<function body>
epilogue:
instrumented code
leave RET
ret
Francesco Gadaleta Instruction level countermeasure against stack-based buffer overflow attacks March 31st, 2009

20. Design
main:
call init_callxretx
... PROTECTED PAGE
prologue: PROGRAM STACK
pushl %ebp
mov %esp, %ebp
(callx)
RET
<function body>
epilogue:
(retx)
leave RET
ret
Francesco Gadaleta Instruction level countermeasure against stack-based buffer overflow attacks March 31st, 2009

21. Implementation
traps.c x86_emulate.c
void cpu_wpswitch(int set) …
{ case 0xd0:
… read counter
vcpu->arch.guest_context.ctrlreg[0] &= ~X86_CR0_WP; increase counter
… copy RET to address(@counter)
vcpu->arch.guest_context.ctrlreg[0] |= X86_CR0_WP;
… case 0xd1:
} read counter
decrease counter
copy value(@counter) to program_stack
…
if (opcode == “xfxd0”) /*callx*/
{
...
cpu_wpswitch(1);
x86_emulate(&ctxt);
cpu_wpswitch(0);
...
}
if (opcode == “xfxd1”) /*retx*/
{
...
cpu_wpswitch(1);
x86_emulate(&ctxt);
cpu_wpswitch(0);
...
}
Francesco Gadaleta Instruction level countermeasure against stack-based buffer overflow attacks March 31st, 2009

22. Results
CPU SPEC 2000 benchmarks
Program Base r/t (s) Instr. r/t (s) Overhead
164.gzip 223 3202 14,36x
175.vpr 372 2892 7,7x
176.gcc 225 2191 8.7x
181.mcf 640 3849 5x
186.crafty 114 3676 32x
256.bzip2 307 5161 15x
300.twolf 717 4007 4.5x
Better than RAD (140x-200x)
but still poor for real life deployments
Francesco Gadaleta Instruction level countermeasure against stack-based buffer overflow attacks March 31st, 2009

23. x86 architecture virtualization issues
Hardware managed TLB
Xen context switching (dom0-domU)
TLB flush --> full page table lookup
No tagged entries
AMD SVM uses a tagged TLB (room for improvements)
Francesco Gadaleta Instruction level countermeasure against stack-based buffer overflow attacks March 31st, 2009

24. Hardware supported implementation
Special instructions added to the instruction set of the (emulated)
processor
skip software emulation of insns
Show that hardware implementation for this type of countermeasure
may be the solution
QEMU - processor emulator
changes to the (emulated) MMU
(callx)/(retx) allowed to write/read protected pages directly
Francesco Gadaleta Instruction level countermeasure against stack-based buffer overflow attacks March 31st, 2009

25. QEMU Results
Program Base r/t (s) Instr. r/t (s) Overhead
164.gzip 1368 1446 1.05x
175.vpr 2458 2606 1.06x
176.gcc 1010 1067 1.05x
181.mcf 646 701 1.07x
186.crafty 1542 1656 1.07x
256.bzip2 1638 1729 1.05x
300.twolf 2316 2399 1.03x
Let’s do some math
RAD Xen
140x 5x
200x 32x
Francesco Gadaleta Instruction level countermeasure against stack-based buffer overflow attacks March 31st, 2009

26. Future work
Different approach
(but the hw architecture has issues...)
Port to different architectures (e.g. AMD)
Protect all pointers
same idea: use special insns to handle arbitrary memory locations
Francesco Gadaleta Instruction level countermeasure against stack-based buffer overflow attacks March 31st, 2009

27. Questions
?
Francesco Gadaleta Instruction level countermeasure against stack-based buffer overflow attacks March 31st, 2009