1. LEXISNEXIS,
NCSTATE
OPPORTUNITIES
TIM MENZIES
COMPUTER SCIENCE,
JUNE 2015

2. SEBIG LAB : SE FOR BIG DATA
• Three year partnership
• New lab to explore SE methods for big data apps.
• Grow skill set of engineers:
• Assess different approaches to Big Data
• Validation of results
2

3. LAB PROCESSES VS
INDUSTRIAL PROCESSES
• Lab processes
• Make 10ml of
oxygen?
• Easy!
• Make 100,000 liters
per day?
• That’s another matter
3

4. INDUSTRIAL PROCESSES FOR
DATA MINING
4

5. INDUSTRIAL PROCESSES
FOR DATA MINING
5
1
23
4
5

6. EXPLORING NEW ALGORITHMS
• New ideas
• SVM
• Deep learning
• Ensembles
• etc
• Visualizations
• Parameter tuning
• Synonym discovery
• Incremental association
rule learning
6
1

7. VALIDATION STUDIES
• Independent checks of industrial results
• Optimizing validation:
• ? Mechanical Turk
• Better support tools for coding new
functionality
• Better test suites for
certifying new functionality
7
2

8. CAN WE MAKE BETTER
USE OF OLD KNOWLEDGE?
• Learning domain ontologies.
• Corpus definition.
• How to revise old knowledge?
• The privileged review problem.
• Transfer learning.
8
3

9. SUPPORT
Gather case study data
Synthetic studies
Annonymization of data
Training
• Papers
• Tutorials
• Learning information
seeking behavior
9
4

10. LESS IS MORE
• Reasoning via fewer, most representative
examples
• Active learning
• Early stopping
• Stack ranking (early stop)
10
5