Revised and extended version of the "Virtual Separation of Concerns" talk, prepared for the GI-Dissertationspreis colloquium at Dagstuhl and for the invited talk at Oregon State University in Corvallis.

1. Photo (cc) Horia Varlan
Virtual Separation of Concerns
Christian Kästner

3. Feature-Oriented
Product Lines

4. Database
Engine

5. Printer
Firmware

6. Linux
Kernel

7. 7

8. Variability = Complexity

9. 33 features
optional, independent
a unique configuration for every
person on this planet

10. 320 features
optional, independent
more configurations than estimated
atoms in the universe

11. Correctness?

12. Maintenance?
Comprehension?

13. Conditional Compilation
static int _rep_queue_filedone(...)
DB_ENV *dbenv;
REP *rep;
__rep_fileinfo_args *rfp; {
#ifndef HAVE_QUEUE
COMPQUIET(rep, NULL);
COMPQUIET(rfp, NULL);
return (__db_no_queue_am(dbenv));
#else
db_pgno_t first, last;
u_int32_t flags;
int empty, ret, t_ret;
#ifdef DIAGNOSTIC
DB_MSGBUF mb;
#endif
...
Excerpt from Oracle’s Berkeley DB

14. Objections / Criticism
Designed in the 70th and hardly evolved since
“#ifdef considered harmful”
“#ifdef hell”
“maintenance becomes a „hit or miss‟ process”
“is difficult to determine if the code being
viewed is actually compiled into the system”
“incomprehensible source texts”
“programming errors are easy “CPP makes maintenance difficult”
to make and difficult to detect”
“source code rapidly
“preprocessor diagnostics are poor” becomes a maze”

15. Academia: Compositional Approaches
Base / Platform
class Stack {
void push(Object o) {
elementData[size++] = o;
}
...
}
class Stack {
void push(Object o) {
Lock l = lock(o);
Feature: Queue elementData[size++] = o;
refines class Stack { Composition }
l.unlock();
...
void push(Object o) {
Lock l = lock(o); }
Super.push(o);
l.unlock();
}
...
}
Module
Components
Feature: Diagnostic Frameworks, Plug-ins
aspect Diagnostics {
... Feature-Modules / Mixin Layers / …
}
Aspects / Subjects, Hyper/J, Deltas

16. A CLOSER LOOK AT PREPROCESSORS

17. Separation of Concerns

18. class Stack {
void push(Object o
#ifdef SYNC Obfuscation
, Transaction txn
#endif
) {
if (o==null
#ifdef SYNC
|| txn==null
#endif
) return;
#ifdef SYNC
Lock l=txn.lock(o);
#endif
elementData[size++] = o;
#ifdef SYNC
l.unlock();
#endif
fireStackChanged();
}
}

19. Obfuscation
Femto OS

20. static int _rep_queue_filedone(...)
DB_ENV *dbenv;
REP *rep; Error-Prone
__rep_fileinfo_args *rfp; {
#ifndef HAVE_QUEUE
COMPQUIET(rep, NULL);
COMPQUIET(rfp, NULL);
return (__db_no_queue_am(dbenv));
#else
db_pgno_t first, last;
#ifdef TABLES
u_int32_t flags;
class Table {
int empty, ret,void insert(Object data, Txn txn) {
t_ret;
#ifdef DIAGNOSTIC storage.set(data, txn.getLock());
DB_MSGBUF mb; }
#endif }
// over 100 lines of add. code
#endif
} class Storage {
#endif #ifdef WRITE
boolean set(…) { ... }
#endif
}

21. Advantages
• Easy to use
• “annotate and remove”
• Already part of many languages / simple tools
• Most developers already familiar
• Flexible / Expressive
• Language-independent / Uniform
• Low Adoption Barrier
• esp. with legacy code

22. IMPROVING TOOL SUPPORT

23. [ICSE‟08, ViSPLE‟08]
Views

25. View on Feature Eval

26. View on Configuration
“Add and Eval”

27. [ICSE‟08, ICSE„10, EASE„11]

28. FeatureCommander

29. Controlled
3 Experiments
Faster?
Scaling? How Used?

30. ERROR DETECTION / PREVENTION

31. Disciplined
Annotations
#ifdef WORLD
int main() {
printf(“Hello World”);
}
#endif
int main() {
#ifdef WORLD
printf(“Hello World”);
#endif
} greet.c
printf VBYE main
Align with variability model
VWORLD
msg ε msg ε printf
Align with code structure msg

32. Disciplined Undisciplined
Annotations
#ifdef WORLD int main() {
int main() { #ifdef DYNAMIC
printf(“Hello World”); if (enabled) {
} #endif
#endif printf(“Hello World”);
#ifdef DYNAIC
int main() { }
#ifdef WORLD #endif
printf(“Hello World”); }
#endif
} int a =
#ifdef DYNAMIC
Align with variability model enabled? 3 : 4 +
#endif
Align with code structure 5;

33. static int _rep_queue_filedone(...)
DB_ENV *dbenv;
REP *rep;
__rep_fileinfo_args *rfp; {
#ifndef HAVE_QUEUE
COMPQUIET(rep, NULL);
COMPQUIET(rfp, NULL);
return (__db_no_queue_am(dbenv));
#else
db_pgno_t first, last;
u_int32_t flags;
int empty, ret, t_ret;
#ifdef DIAGNOSTIC
DB_MSGBUF mb;
#endif
// over 100 lines of add. code
}
#endif

34. Parser
Variability-Aware Type System
Static Analysis
Bug Finding
Testing
Model Checking
Theorem Proving
…

36. References
Conflicts

37. Presence Conditions
WORLD
BYE
true
true

38. Reachability: pc(caller) -> pc(target)
Conflicts: ¬(pc(def1) ˄pc(def2))
¬ (WORLD ˄BYE)
true -> (WORLD v BYE)
true -> true

39. P
Variability Model:
WORLD BYE
VM ->¬ (WORLD ˄BYE)
VM -> (true -> (WORLD v BYE))
VM -> (true -> true)

40. AST with Variability Information
true -> true
greet.c
…
printf VWORLD VBYE main
msg ε msg ε printf
¬ (WORLD ˄BYE)
true -> (WORLD v BYE)
msg
WORLD
BYE 40
Extended Lookup Mechanism

41. [ASE‟08, TOSEM„11]
Formalization: CFJ

42. Surface Complexity
Inherent
Complexity
SAT
Problem

43. Variability-Aware Analysis
Type System
Static Analysis
Bug Finding
Testing
Model Checking
Theorem Proving
…

44. CIDE open source, fosd.net/cide
http://fosd.de/cide

45. [GPCE‟09]
Automated Refactorings
• Feature Module • Annotationen
class Stack { class Stack {
int[] data; int[] data;
Base
void push(int o) { … } #ifdef UNDO
int pop() { /*...*/ } int backup;
} void undo() { /*...*/ }
#endif
refines class Stack { #ifdef TOP
Automated Transformation
Top
int top() { /*...*/ } int top() { /*...*/ }
} #endif
void push(int o) {
refines class Stack { #ifdef UNDO
int backup; backup = top();
void undo() { /*...*/ } #endif
Undo
void push(int o) { /*...*/
backup = top(); }
original(v); int pop() { /*...*/ }
} }

46. THE REAL-WORLD CHALLENGE

47. Parse and Type check
all configurations of the
entire Linux kernel
10,000 features, 6 million lines of C code

48. [OOPSLA„11]
greet.c
…
printf VWORLD VBYE main
msg ε msg ε printf
msg
AST with Variability Information

49. Macro expansion Undisciplined
needed for parsing annotations Alternative macros
?
? ?
greet.c
+ printf VWORLD VBYE + main
+ msg ε + msg ε printf
msg

50. Parsing C without Preprocessing

52. Previous Solutions
Disciplined Subset
Requires Code Preparation
Heuristics and Partial Analysis
Inaccurate, False Positives
Brute Force
Infeasible Effort

53. TypeChef
( +
2
Variability-Aware * Variability-Aware
3 * VA
Lexer )
+
Parser
4A
5¬A 2 3 4 5
Variability-Aware
Analysis
https://github.com/ckaestne/TypeChef

54. 4A (¬A 4¬A˄B +¬A˄ 6¬A
B )¬A true
( 3 + )
4¬A˄ +¬A˄ 6¬A
B B
+ 4A (¬A )¬A
3 VA 4¬A˄B +¬A˄ 6¬A
B
4 VB
+ 6
4 6

55. Variability-Aware Parsers
GNU C + cpp Java + Antenna

56. Parsing Linux
Variability in Build System (kbuild)
and with #ifdef
2.6.33.3
X86
Variability Model (kconfig)

57. 353 included header files per C file
0
8590 macros per C file
0
1387 conditional macros per C file
0
340 alternative macros per C file
0
335490 token per C file
0 2.6.33.3
X86
72 % conditional
0

58. 7665 C files (x86)
0
30 seconds per file (median)
0
85 hours total
0
4.1 % overhead (undiscipl.)
0
0
0 syntax errors 2.6.33.3
X86

59. Type Checking Linux
2.6.33.3
X86

60. SUMMARY AND PERSPECTIVE

61. Summary
• Problems: • Improvements:
• Separation of Concerns • Views
• Obfuscation • Visual representation
• Error-proneness • Variability-aware analysis
• Retained advantages:
• Remaining Problems:
• Easy to use
• Modular Type Checking
• Flexible
• Separate Compilation
• Language-independent • Black-box Reuse
• Low adoption barrier
“Virtual Separation of Concerns”

62. Perspective
• Preprocessors common in practice, despite criticism
• Neglected by researchers
• Tool support can address most problems
• Currently scaling to Linux
• Interim solution or serious alternative?
Give preprocessors a second chance!

63. Questions?
http://fosd.de/cide

64. Picture Credits
© Stuck in Customs (cc by-nc-sa 2.0)
© Horia Varlan (cc by 2.0)