This presentation is based on the technical hurdles we overcame when building a commercial product on the introspection capabilities of the Xen hypervisor. Mihai Dontu will relate the importance of the x86 emulator, the need for a more focused effort on its completeness and correctness, the problems encountered, and the solutions adopted. He will also approach the subject of performance, for which hypervisor features that were not meant to be in the hot path had to be punctually reworked to solve a key requirement for making a theoretical product a commercial reality.

1. HYPERVISOR-BASED SECURITY:
VICARIOUS LEARNING VIA
INTROSPEKTIONEERING, OR HOW I
LEARNED TO LOVE THE BOMB
Mihai DONȚU
Technical Project Manager, Bitdefender Linux Development Team
Xen Project Developer Summit, August 25-26, 2016

2. 08/26/2016
Outline
• Quick recap of VMI history
• Current status
• Bitdefender’s approach to a VMI-based commercial product
• Current effort
• the x86 emulator
• the Xen VMI API performance

3. 08/26/2016
Quick recap of VMI history
• 2003 – Garfinkel & Rosenblum: “A Virtual Machine Introspection Based Architecture for
Intrusion Detection”
• 2008 – Dinaburg et al.: “Ether: Malware Analysis via Hardware Virtualization Extensions”
(Xen 3.1)
• 2012 – A number of companies start working on improving Xen’s VMI capabilities
(Bitdefender is among them)
• 2016 – Bitdefender releases a commercial product using VMI, in technical preview (based
on Citrix XenServer 7.0)

4. 08/26/2016
Current status
The core API is complete for x86 (ARM?) and allows:
• receiving MSR modification events
• … CR modification events
• … user VMCALL-s
• … breakpoint events
• EPT violation events
• ability to veto events (eg. prevent setting and MSR)
• query guest vCPU-s (GPRs, TSC etc.)

5. 08/26/2016
Bitdefender’s approach to a VMI-based commercial product
Xen Hypervisor
dom0
SVA
(domU0)
domU1Memory
Introspection
Engine
domU2 domUN
SVA (Security Virtual Appliance) – privileged VM capable of introspecting all others
Memory Introspection Engine – user space application that processes all VMI events

6. 08/26/2016
Current effort
• Focused on VDI
• Few but serious bumps on the road, largely performance related
• Testing rig:
• 2 x Intel(R) Xeon(R) CPU E5-2630 v3 @ 2.40GHz (16 cores, 32 threads)
• 384GiB RAM
• 1TB SSD
• XenServer 7.0
• Standard VDI benchmarking software

7. 08/26/2016
x86 emulator
• Used to execute an instruction that would normally generate a #PF without changing the
page protection in EPT
• Very complex code
• Forked ~ 2006 in KVM and largely rewritten
• Incomplete SSE support causes problems in non-VMI projects as well (eg. QXL)
• SSE-related patches are incoming
• Two alternatives available:
• pause all vCPU-s, remove the page protection, single step, restore the page
protection, unpause all vCPU-s
• switch EPT view (altp2m), single step, switch back EPT view (this is the future)
• In Xen 4.6 (XenServer 7.0) altp2m is not yet ready

8. 08/26/2016
The Xen VMI API performance
• Benchmark shows how many VM-s the host can run until the ‘user experience’ degrades
too much
• Target: 20% penalty
• Each VM: 2 vCPU, 3 GiB RAM
• First benchmark test (w/o VMI): 96
• Second benchmark test (w/ VMI, OS kernel only): 9
• Optimizations: w/ VMI, OS kernel only:
• … + cache xc_domain_hvm_getcontext_partial(): 50
• … + filter MSR-events: 55
• … + no domain_pause() in xc_domain_hvm_getcontext_partial(): 90
• w/ VMI, OS kernel, userspace (winword.exe): 30
• … + no xc_get_vcpucontext(): 50
• … + no xc_set_vcpucontext(): 86
• … + upgrade SVA kernel from 3.8 to 4.4: 90
• w/ VMI, OS kernel, userspace (more processes): 35
• … + more xc_domain_hvm_getcontext_partial() work: 40
• … + cache xc_get_mem_access(): 43
• optimization opportunities are still being researched

9. 08/26/2016
The Xen VMI API performance (2)
• A number of API calls were not meant to be abused by VMI
(xc_domain_hvm_getcontext_partial(), xc_{set,get}_vcpucontext())
• Needlessly pause the entire domain in order to get the registers of a paused vCPU
• xen_hypercall_xen_version() shows up at the top when the event channel is stressed
# perf top -C 1
25.58% [kernel] [k] xen_hypercall_xen_version
10.25% [kernel] [k] xen_hypercall_sched_op
9.48% [kernel] [k] xen_hypercall_event_channel_op
5.80% libintrocore.so [.] IntHandleEptViolation
1.83% bdmid [.] IntrocoreManager::IntQueryGuestInfo
1.71% libpthread-2.23.so [.] pthread_mutex_lock
1.28% libpthread-2.23.so [.] pthread_mutex_unlock
1.27% libbdvmi.so [.] bdvmi::XenEventManager::waitForEvents
1.14% libbdvmi.so [.] bdvmi::StatsCollector::incStat
1.12% libintrocore.so [.] IntIcLookupInstruction
# perf top -C 0
26.34% [kernel] [k] xen_hypercall_xen_version
8.07% [kernel] [k] xen_hypercall_event_channel_op
6.52% libintrocore.so [.] IntHandleEptViolation
2.43% libintrocore.so [.] NdDecodeEx2
2.28% [kernel] [k] xen_hypercall_sched_op
2.20% bdmid [.] IntrocoreManager::IntQueryGuestInfo
2.13% libpthread-2.23.so [.] pthread_mutex_lock
1.47% libpthread-2.23.so [.] 0x000000000001055d
1.42% libpthread-2.23.so [.] pthread_mutex_unlock
1.40% libc-2.23.so [.] ioctl
• evtchn IRQ balancing has negative effects, even though CPU0 cannot take all IRQ-s

10. Q & A