This document discusses predictive coding and machine learning methods for e-discovery. It begins by explaining that predictive coding relies on supervised machine learning methods, which use human-coded training sets to classify documents as relevant or non-relevant. The document then discusses different types of supervised and unsupervised machine learning algorithms that could be used for predictive coding. It suggests that using prior cases to inform feature selection and weightings could make algorithms "smarter." Finally, it considers whether discovery costs will eventually be reduced due to scaling relationships for cost per gigabyte and long-term rates of electronic data growth.

1. @ computationalcomputationallegalstudies.com
Predictive Coding
and E-Discovery
in 2015 and
Beyond
daniel martin katz
michael j bommarito ii

2. The 2x2 Machine Learning Spectrum
Info Viz & Pattern Detection
Rates of Scaling

3. The 2x2 Machine
Learning Spectrum

4. In Order to
Understand Where
We Are Heading ...

5. in 2015 and
Beyond ...

6. it is necessary to have
insight regarding how
predictive coding
actually works

7. Predictive Coding
Relies Upon a
Particular Class of
Machine Learning
Methods

8. Predictive Coding
Relies Upon a
Particular Class of
Machine Learning
Methods

9. The Current Approach
is drawn from
the family of so called
“supervised methods”

10. What is the difference
between supervised
and unsupervised?

12. As you have likely seen ...

13. Predictive Coding

14. Develop a Training Set
using human experts

15. In the simple case,
assign objects to
two piles

16. Take This Document Set ...

17. Apply Human Coders

18. yellow = relevant
white = non-relevant
And Return This ...

19. Non RelevantRelevant

20. Key Insight ...

21. What Allows A Human
To Separate These
Two Classes of
Documents?

22. that precise human
process is what
predictive coding is
trying to mimic

23. Humans are selecting
upon features of
documents

24. to place those
documents in their
respective bins
(i.e. relevant, non-relevant)

25. features =?
text,
author,
date,
other metadata

26. supervised methods
“learn” from the
training data

27. but there are different
forms of learning by
machines ...

28. There Is Learning
Within a Matter
(i.e. learning from a
specific training set)

29. But what about using
prior matters to inform
both feature selection
and the weighting of
those features

30. In other words, it is
possible to learn from
the experience of
having processed
documents in the past

31. both inside a given
company but also
across companies ...

32. It comes from
data aggregation / reusing data

33. This is Learning and
Rule Propagation
Across Matters

34. feedback loops are the
best friends of algorithms

35. feedback loops can help
make algorithms become
much smarter ...

37. Supervised Unsupervised
Predictive
Coding
The Future
Machine
Learning
Methods
2 x 2
Informed
Naive
Basic
Clustering
Algorithm

38. Supervised
Statistical models
Bayesian, e.g., Naïve Bayes Classification
Frequentist, e.g., Ordinary Least Squares
Neural Networks (NN)
Support Vector Machines (SVM)
Random Forests (RF)
Genetic Algorithms (GA)
Semi/unsupervised
Neural Networks (NN)
Clustering
K-means
Hierarchical
Radial Basis (RBF)
Graph
Some Machine Learning Algorithms

40. Info Viz &
Pattern
Detection

41. Think about the task faced
by the intelligence
community ...

42. mountains of
information to process

43. how are those
intelligence
analysts aided?

44. Information
Visualization

45. The Visual Cortex is a very
powerful CPU ...

46. We are very good
pattern detectors ...

47. We need a mix of analytics
and viz ...

49. because there are significant
efficiency gains to be
obtained from applications of
sophisticated data
visualization techniques

50. This Next Generation of
EDiscovery Software is
viz intensive ...

52. but this is only
the beginning ...

55. including an even more
enriched notion of time
dynamics ...

58. Rates of Scaling

59. Will Discovery Costs
Eventually Be Reduced?

60. Two Scaling Relationships
that are in question ...

61. Cost Per Gig

62. “[I]n 2001, a 300 Gb legal matter would take 200 attorneys a full
year to review, at a cost of about $15 million.
In 2003, a similar-sized matter took 100 attorneys 3 weeks to
complete, at a cost of $6 million.
And in 2006, a 300 Gb investigation took 65 attorneys only 2.5
days to complete, at a cost of $2 million.
And now, cases with several hundreds of Gbs are routine.”
Improving Document Review in E-Discovery
FTI Consulting

63. Past Rate
of ESI Creation

64. Long Term
Rate of ESI Creation ?

65. Daniel Martin Katz
Michigan State University
Associate Professor of Law
@ computational
computationallegalstudies.com
reinventlaw.com
http://about.me/daniel.martin.katz