This slide deck provides an overview of my Ph.D. dissertation completed at the University of Southern California in Biomedical Engineering.

1. Inference of computational
models of tendon networks via
sparse experimentation
Manish Umesh Kurse
Apr 11, 2012
1
Brain-Body Dynamics Laboratory
Ph.D. committee: Dr. Francisco J.Valero-Cuevas, Dr. Hod Lipson,
Dr. Gerald E. Loeb, Dr. Eva Kanso

2. 2
MSMS:
Davoodi
et
al.,
2007
Measurement of internal states Injury, deformity, surgery
h4p://www.ispub.com/
Ergonomics, prosthetic design, etc.
h4p://www.anybodytech.com
Inputs Outputs
femoris (rectfem), gluteus medialis/glueteus minimus (glmed/min),
Muscles cooperate
to exert force
Solutions in muscle activation s
task-specific activation se
2
12
3
LIMB
3
1
Fy
Fx
Muscle 2
Muscle 1
Target x-for
Target y-f
Task-specific activation
Fig. 1. Three muscle ‘‘schematic model’’ conceptually illustrates the nece
region of force space, the feasible force set, is achievable given this mu
y-force. (c) The valid coordination patterns for the x and y targets can a
Kutch andValero-Cuevas, 2011
Computational modeling of musculoskeletal
systems

3. 3
Modeling : Structure based on observation + experimental
measurement of some parameters.
Drawbacks:
•
Not
possible
to
measure
all
parameters.
•
Not
validated
with
experimental
input-­‐output
data.
•
Structure
assumed
need
not
be
funcEonally
accurate
representaEon.
R
System
✓
s
R
Structure
assumed,
parameters fit.
1 R2
Infer structure and
parameters from input-
output data
(✓)

4. 4
Develop computational methods to simultaneously infer
structure and parameter values of functionally accurate
models of musculoskeletal systems directly from
experimental input-output data.
Objective
Inputs Outputs
We examined 5 different postures in 3 specimens, and 3 different posture
the final specimen. Each posture was neutral in add-abduction. The exami
postures were chosen to cover the workspace and simulate those found
everyday tasks. After positioning the finger in a specific posture, we determi
the action matrix for the finger: we applied 128 combinations of tendon tensi
representing all possible combinations of 0 and 10 N across the seven tendons,
held each combination for 3 s. The fingertip forces resulting from each coordi
tion pattern was determined by averaging the fingertip load cell readings acr
the hold period. Linear regression was performed on each fingertip fo
component using the tendon tensions as factors. In this way, the fingertip fo
vector generated by 1 N of tendon tension was determined for all muscles.
force vector generated by each muscle was scaled by an estimate for maxim
muscle force (Valero-Cuevas et al., 2000) to generate the columns of the act
matrix for each specimen and posture examined.
2.2. Action matrix for human leg model
We also studied the necessity of muscles for mechanical output fo
simplified, but plausible, sagittal plane model of the human leg (hip, knee,
ankle joints). The model contained 14 muscles/muscle groups (Kuo and Za
1993) (muscle/muscle group abbreviation in parentheses): medial and lat
gastrocnemius (gastroc), soleus (soleus), tibialis posterior (tibpost), peron
brevis (perbrev), tibialis anterior (tibant), semimembranoseus/semitendeno
biceps femoris long head (hamstring), biceps femoris short head (bfsh), rec
femoris (rectfem), gluteus medialis/glueteus minimus (glmed/min), adduc
Muscles cooperate
to exert force
2
12
3
LIMB
3
1
Fy
Fx
Kutch andValero-Cuevas, 2011

5. Tendon networks of the fingers
5
Lateral bands
Central slip
Terminal slip
Retinacular ligament
Sagittal band
Transverse fibers
Clavero et al. (2003). “Extensor Mechanism of the Fingers: MR Imaging-Anatomic Correlation”, Radiographics
Netter, F. Atlas of Human Anatomy, 3rd edition, pp 447-453

6. Computational models
6
Analytical models Anatomy-based models
s2 = ✓2
1 + 9e✓2

7. Significance of research
7
http://www.myefficientassistant.com/
http://www.punchstock.com/
‘Plant’
‘Controller’
Motor control
http://www.ispub.com/
Clinical
Wilkinson et al. 2006
Robotics and prosthetics

8. ?
Computational
models
Experimentation Mathematical
modeling
Inference algorithms
8

9. Dissertation outline
Experimental actuation of a cadaveric hand.
2
3
4
5
6
New inference approach to
learn functions of tendon
routing.
Application to the human
index finger.
Experimental validation of an existing
model.
Tendon network simulator and
sensitivity analysis.
Inference of anatomy-based models
from experimental data.
1
Analytical models Anatomy-based models
ASME SBC ’10 & IEEETBME ’12
ASB ’11
CSB ’12
ASME SBC ’09
9

10. Cadaver finger control
10
Load cells
Strings to the tendons
Motion capture markers
6 DOF Load cell
DC motors
Positon encoders
1 2 3 4 5 6
1

11. Finger tapping
11
1 2 3 4 5 6

12. Slow finger movements
12
All intact
Radial nerve palsy
Median nerve palsy
Ulnar nerve palsy
fm
Iteratively,
1 2 3 4 5 6

13. Finger equilibrium
13
⌧ = RFm
0
Neutral
equilibrium
Stable equilibrium
1 2 3 4 5 6

14. 14
NeutralStable
FDP
FDS
EI
EDC
LUM
FDI
FPI
1 2
0
1
2
3
4
Distancefromnullspace(N)
1 2
0
2
4
6
8
Coordination pattern
Tendontensions
i
Distance from null space
1 2 3 4 5 6

15. Conclusions
• Spring-based (muscle-like) control effective
to control movement.
• Simple tap requires a coordinated set of
tendon excursions.
• Neutral equilibrium in specific postures and
tendon tensions.
15
1 2 3 4 5 6

16. Computational models
16
Analytical models Anatomy-based models
s2 = ✓2
1 + 9e✓2
1 2 3 4 5 6

17. Inference of analytical functions
17
Co-authors: Dr. Hod Lipson, Dr. FranciscoValero-Cuevas
1 2 3 4 5 6
2
• Analytical functions for tendon excursions
s = f(✓)
Deshpande et al. 2009
• State of the art : Polynomial regression
s s
‘Controller’
‘Plant’
• Why? R(✓) =
@s
@✓
⌧(✓) = R(✓)Fm
R
✓
R1
R2
(✓)

18. • Can we simultaneously learn form and
parameter values from data?
• Compare accuracy with polynomial regression.
Specific Aims
18
1 2 3 4 5 6
s = f(✓)

19. 19
Schmidt and Lipson, 2009
Koza 1992
Symbolic regression
1 2 3 4 5 6

20. Robotic tendon driven system
20
s1
s2
s3
2
1
3
Position encoders
Motors keeping
tendons taut
Load cells
Motion capture
markers
Motion capture
camera
1 2 3 4 5 6
Landsmeer
model I
Landsmeer
model II
Landsmeer
model III
s = 3.6sin(0.5θ)
s = 0.6θ + 3.2(1 −
θ/2
tan(θ/2)
)
s = 1.8θ
s = f(✓1, ✓2, ✓3)

21. 21
Schmidt and Lipson, 2009
Polynomial regression
Koza 1992
Symbolic regression vs.
Linear
Quadratic
Cubic
Quartic
1 2 3 4 5 6

22. Comparing symbolic and polynomial regressions
22
2
5
10
20
2
5
10
20
2
5
10
20
Tendon 1
2
5
10
20
Tendon 2
Tendon 3
2
5
10
20
n/256
n/16
n/64
n n
2
n
4
n
8
n
16
n
32
n
64
n
128
n
256
X
X
X
XX
n/256
n/16
n/64
2
5
10
20
n n
2
n
4
n
8
n
16
n
32
n
64
n
128
n
256
n/256
n/16
n/64
n/256
n/16
n/64
n/256
n/16
n/64
n/256
n/16
n/64
Symbolic
Quartic
Linear
Quadratic
Cubic
Dataset size (n =1688) Dataset size (n =1688)
X
X
X
Cross-validation Extrapolation
RMSerror(%)
RMSerror(%)
X Error for all sizes > 5%
Min training set size < n/256
2
5
10
20
Tendon 1
Tendon 2
2
5
10
20
25%
75%
125%
25%
75%
125%
25%
75%
125%
2
5
10
20
Tendon 3
RMSerror(%)
Extrapolation by volume (%)
0 25 15075 10050 125
Symbolic
Quartic
Linear
Quadratic
Cubic
X
X
X All extrapolation errors > 5%
Achievable extrapolation > 150%
Fewer training data
points required
More extrapolatable
2

23. 23
Extrapolation by volume (%)
0
25
50
75
100
125
150
>150
n n
2
n
4
n
8
n
16
n
32
n
64
n
128
Training set size (n =1688)
Extrapolationbyvolume(%)
Symbolic
Quartic
Linear
Quadratic
Cubic
Comparing symbolic and polynomial regressions
Fewer training data
points required
More extrapolatable
Kurse et al. 2012 (in press)
1 2 3 4 5 6

24. 24
Landsmeer
model I
Landsmeer
model II
Landsmeer
model III
s = 3.6sin(0.5θ)
s = 0.6θ + 3.2(1 −
θ/2
tan(θ/2)
)
s = 1.8θ
Simulated musculoskeletal systems
Landsmeer comb. Expressions
I, I, I
Target 1.8✓1 + 1.8✓2 + 1.8✓3
Evolved 1.8✓1 + 1.8✓2 + 1.8✓3
I, II, III
Target
1.8✓1 + 3.6sin(0.5✓2) + 0.6✓3
(1.6✓3)/tan(0.5✓3) + 3.2
Evolved
1.8✓1 + 3.61sin(0.5✓2) + 1.54✓3
0.778sin(✓3)
II, II, I
Target 3.6sin(0.5✓1)+3.6sin(0.5✓2)+1.8✓3
Evolved 3.6sin(0.5✓1)+3.6sin(0.5✓2)+1.8✓3
Table 1: Target and inferred expressions with training, cross-validation and extrap
for some combinations of Landsmeer’s models I, II, III
1 2 3 4 5 6

25. Error vs. number of parameters
25
RMSerror(%)
Cross-validationExtrapolation
Symbolic
Quartic
Linear
Quadratic
Cubic
Experimental data
With no noise
Number of parameters
Simulated data
With noise added
1
2
3
0 20 40
1
2
3
5
.0001
.01
1
0 20 40
.0001
.01
1
0 20 40
1
2
5
10
1
2
5
10
1 2 3 4 5 6

26. Conclusions
• Symbolic regression outperforms polynomial
regression
• Number of training data points
• Extrapolatability
• Robustness to noise
• Number of parameters
• Insight on physics
26
1 2 3 4 5 6
Kurse et al. 2012 (in press)

27. 27
Novel method of
inference of
analytical functions
from data
Application to the
human finger
Schmidt and Lipson, 2009
s1 = f(✓1, ✓2, ✓3, ✓4)
1 2 3 4 5 6

28. Analytical functions: Index finger
28
Constant moment arm
(Linear)
Polynomial regressions
Landsmeer based models
Landsmeer
model I
Landsmeer
model II
Landsmeer
model III
s = 3.6sin(0.5θ)
s = 0.6θ + 3.2(1 −
θ/2
tan(θ/2)
)
s = 1.8θ
Landsmeer, 1961, Brook 1995
3
Co-authors: Dr. Hod Lipson, Dr. FranciscoValero-Cuevas
1 2 3 4 5 6
Eg.An et al. 1983,Valero-Cuevas et al. 1998
Eg. Franko et al. 2011
Eg. Brook et al. 1995

29. Specific aims
• Infer analytical functions for the seven tendons of
the index finger.
• Compare against polynomial regression and
Landsmeer based models.
29
1 2 3 4 5 6

30. Cadaver experimental setup
30
MoEon
capture
Load cells
Position encoders
Servo motors
Markers
1 2 3 4 5 6
s1 = f(✓1, ✓2, ✓3, ✓4)
s7 = f(✓1, ✓2, ✓3, ✓4)
.
.
.

31. 31
Schmidt and Lipson, 2009
Polynomial and
LandsmeerKoza 1992
Symbolic regression vs.
Linear
Quadratic
Cubic
Quartic
Landsmeer
1 2 3 4 5 6

32. Across trials
32
FDP FDS EIP EDC LUM FDI FPI
2
5
10
20
Symbolic
Landsmeer
Quartic
Linear
Quadratic
Cubic
2 10 50
2
5
10
20
50
FDP
2 10 50
2
5
10
20
50
FDS
2 10 50
2
5
10
20
50
EIP
2 10 50
2
5
10
20
50
EDC
2 10 50
2
5
10
20
50
LUM
2 10 50
2
5
10
20
50
FDI
2 10 50
2
5
10
20
50
FPI
Tendon Number of parameters
NormalizedRMSerror(%)
1 2 3 4 5 6

33. Across hands
33
FDP FDS EIP EDC LUM FDI FPI
2
5
10
20
2
5
10
20
50
Tendon
NormalizedRMSerror(%)
Symbolic
Landsmeer
Quartic
Linear
Quadratic
Cubic
1 2 3 4 5 6
FDP FDS EIP EDC LUM FDI FPI
2
5
10
20
Symbolic
Landsmeer
Quartic
Linear
Quadratic
Cubic
10
20
50
Tendon
NormalizedRMSerror(%)

34. Conclusions
• For subject-specific models as well as generalizable
models,
• Symbolic regression more accurate than other models.
• Error bounds on generalizability.
• Models insight on tendon routing.
34
1 2 3 4 5 6

35. Computational models
35
Analytical models Anatomy-based models
s2 = ✓2
1 + 9e✓2
1 2 3 4 5 6

36. Anatomy-based modeling
36
Co-author: Dr. FranciscoValero-Cuevas
Netter, F. Atlas of Human Anatomy, 3rd edition, pp 447-453
1 2 3 4 5 6
4
Clavero et al. 2003
Boutonniere deformity
http://www.ispub.com/
Swan-neck deformity
Mallet finger deformity

37. 37
• Widely used representation:
An-Chao normative model
(1978, 79)
TE=RB+UB
RB=0.133 RI+0.167 EDC+0.667 LU
UB=0.313 UI+0.167 EDC
ES=0.133 RI+0.313 UI+0.167 EDC+0.333 LU
Chao et al. 1978,79
1 2 3 4 5 6

38. Validation of An-Chao model
38
6 DOF loadcell
Load cells measur-
ing tendon tensions
Strings connecting
tendons to motors
Fingertip force vector
1 2 3 4 5 6

39. Validation of An-Chao normative model
39
• Large magnitude and direction errors in fingertip force
magnitude and direction.
1 2 3 4 5 6
(Sagittal plane)
FDP FDS EIP EDC LUM FDI FPI
0
20
40
60
Direrror(degrees)
FDP FDS EIP EDC LUM FDI FPI
0
200
400
600
800
1000
Magerror%
Magnitude errors Direction errors
Flex
Tap
Extend

40. 40
• Let the physics and mechanics
decide force distribution.
• Existing musculoskeletal
modeling software do not model
tendon networks.
• Environment to understand role
of components in force
transformation.
Valero-Cuevas and Lipson, 2004
1 2 3 4 5 6

41. Specific aims
• Develop a modeling environment to
represent these tendon networks.
• Study sensitivity of fingertip force output to
properties of the extensor mechanism.
41
1 2 3 4 5 6
Tendon network simulator
and sensitivity analysis
5

42. Import MRI scan of bones.
Define tendon network.
Tendon network simulator
Solve the nonlinear finite
element problem.
1 2 3 4 5 6
42

43. Iteratively,
• Node and element penetration testing.
• Apply input Forces in increments
• Solve by Newton-Raphson iteration method the displacements of
nodes, U(i), for system equilibrium :
Finite Element Method
• Assemble the internal force vector and the tangent stiffness
matrix in each element.
43

44. Tension (N)
44
Differential loading as observed by
Sarrafian, 1970, Micks 1981

45. 45

46. Validation
46
Motors
Load cells
Hemisphere
Reflective markers
Fixed nodes
Hemisphere
1 2 3 4 5 6
0 1 2 3 4 5
0
1
2
3
4
5
RF Magnitude (Model) in N
RFMagnitude(Data)inN
RF Node 1
RF Node 2
x=y line

47. Sensitivity analysis of parameters and topology
47
Tessellated
bones
i. Locations
of nodes
ii. Cross-sectional
areas
iii. Resting lengths
iv. Topology
1 2 3 4 5 6

48. 48
Fully flexed
Tap
Fully
extended
Fully flexed Tap Fully extended
Fixed
nodes
Sensitivity of fingertip force output in three postures
1 2 3 4 5 6

49. Results: Sensitivity analysis
49
0
5
10
15
20
25
30
35
40
Directiondeviation(deg)
−100
−50
0
50
100
150
200
Magnitudedeviation(%)
1 2 3 4 1 2 3 4
Network parameters Topology Network parameters Topology
Fully flexed
Tap
Fully
extended
Fingertip force direction
and magnitude
• Sensitive to:
- Posture
- Resting lengths
- Topology
• Less sensitive to
- Node positions.
- Cross-sectional
areas.
1 2 3 4 5 6

50. Conclusions
50
• Developed a novel tendon network simulator
to represent these tendon networks.
• Studied what properties the fingertip force
output is most sensitive to.

51. 51
1 2 3 4 5 6
Simultaneous inference of topology and
parameter values
Valero-Cuevas et al. 2007 Saxena et al. (in review)
Inference of anatomy-based models
Co-authors: Dr. Hod Lipson, Dr. FranciscoValero-Cuevas
6
R
✓
R1
R2
(✓)

52. 52
Specific aims
•Simultaneous inference of 3D tendon networks from
input-output data in simulation.
•Inference of models of the finger’s extensor
mechanism directly from input-output data via sparse
experimentation.
Inference of anatomy-based models
Co-authors: Dr. Hod Lipson, Dr. FranciscoValero-Cuevas
6
1 2 3 4 5 6

53. Data
530 5000 10000 15000
0.1
0.5
2
10
50
Fitness error vs iterations
TotalRFerroras%
Num evaluations
CPU 1
CPU 2
CPU N
...
?
Topology and
parameter inference
of 3D models
1 2 3 4 5 6

54. Inference of tendon networks in simulation
54
6 DOF loadcell
Load cells measur-
ing tendon tensions
Strings connecting
tendons to motors
Fingertip force vector
3 Postures,
3 sets of inputs
1 2 3 4 5 6

55. Inference parameters
55
Tessellated
bones
i. Locations of
6 nodes
(4 variables)
ii. Resting lengths of
13 elements
(7 variables)
iii. Topology : 8
elements
(4 variables)
1 2 3 4 5 6

56. Estimation-exploration algorithm
Models Tests
Bongard et al., Science, 2006
1 2 3 4 5 6
56

57. Inference using EEA
57
Test suite
Converged?
5N 5N
3N
1N1.5N
3N
Start3 Random tests
Measured data
Evolve models
No
End
Two best tests selected
Estimation Exploration
1N
3N3N
Identify most
`intelligent’
tests
(posture +
tendon
tensions)
1 2 3 4 5 6

58. Inference results
58
0
100
200
300
400
500
600
700
RMSMagerror(%)
An Chao 1979
Optimized (best 5)
Full flex Tap Full Ext
0
5
10
15
20
25
30
35
40
45
50
RMSDirerrordeg
Full flex Tap Full Ext
1 2 3 4 5 6
5
10
15
20
25 N
0

59. Conclusions
• Demonstrated for the first time the successful inference
of model topology and parameters of a complex
musculoskeletal system from experimental input-output
data.
• Inferred models are more accurate than models in the
literature.
59
1 2 3 4 5 6

60. 60
?
Computational
models
Experimentation Mathematical
modeling
Inference algorithms

61. 61
?
Computational
models
Experimentation Mathematical
modeling
Inference algorithms
http://www.myefficientassistant.com/
http://www.punchstock.com/
‘Plant’
‘Controller’
Motor control
Boutonniere deformity
http://www.ispub.com/
Swan-neck deformity
Mallet finger deformity
Clinical
Computational models
Analytical models Anatomy-based models
s2 = ✓2
1 + 9e✓2

62. Conclusions and future work
62
•Applies to other systems.
•Step towards subject-specific models inferred from
data.
R
System
✓
s
R
2
Infer structure and
parameters from input-
output data
(✓)
tension in each cord, which was fed back to the motor so that a desired amount of
tension could be maintained on each tendon. The fingertip was rigidly attached to
6 DOF load cell (JR3, Woodland, CA).
We examined 5 different postures in 3 specimens, and 3 different postures in
the final specimen. Each posture was neutral in add-abduction. The examined
postures were chosen to cover the workspace and simulate those found in
everyday tasks. After positioning the finger in a specific posture, we determined
the action matrix for the finger: we applied 128 combinations of tendon tensions
representing all possible combinations of 0 and 10 N across the seven tendons, and
held each combination for 3 s. The fingertip forces resulting from each coordina-
tion pattern was determined by averaging the fingertip load cell readings across
the hold period. Linear regression was performed on each fingertip force
component using the tendon tensions as factors. In this way, the fingertip force
vector generated by 1 N of tendon tension was determined for all muscles. The
force vector generated by each muscle was scaled by an estimate for maximum
muscle force (Valero-Cuevas et al., 2000) to generate the columns of the action
matrix for each specimen and posture examined.
2.2. Action matrix for human leg model
We also studied the necessity of muscles for mechanical output for a
simplified, but plausible, sagittal plane model of the human leg (hip, knee, and
ankle joints). The model contained 14 muscles/muscle groups (Kuo and Zajac,
1993) (muscle/muscle group abbreviation in parentheses): medial and lateral
gastrocnemius (gastroc), soleus (soleus), tibialis posterior (tibpost), peroneus
brevis (perbrev), tibialis anterior (tibant), semimembranoseus/semitendenosis/
biceps femoris long head (hamstring), biceps femoris short head (bfsh), rectus
femoris (rectfem), gluteus medialis/glueteus minimus (glmed/min), adductor
longus (addlong), iliacus (iliacus), tensor fac
(glmax). Moment arms for hip flexion, knee fle
of these muscles were obtained from a compute
et al., 2010). When necessary, multiple muscles
muscle groups. We derived a 3 Â 3 square Jaco
knee, and ankle angle to the foot position in t
orientation of the foot in space. This Jacobian m
combined with the moment arms and maxima
matrix mapping muscle activation to forces
Cuevas, 2005b), although our analysis of muscl
with respect to the endpoint forces.
2.3. Analyzing the action matrix to determine m
We used the action matrix to determine
for a given desired output force using standard
The muscle redundancy problem can be expre
(Chao and An, 1978; Spoor, 1983). These ineq
activation for each muscle lie between 0 and 1,
equal to the desired force. The inequality con
activation space called the task-specific activatio
produce the desired output force (Kuo and Z
Valero-Cuevas et al., 2000, 1998). We comput
specific activation set using a vertex enumera
1992). We then found the task-specific activat
output force for each muscle by projecting a
coordinate axes to determine the minimum and
While previous studies have used similar experi
Muscles cooperate
to exert force
Feasible force set,
one target force vector
Fy
Fx
Target x-force
Target y-f
Feasible force set
Target force vector
2
12
3
LIMB
3
1
Fy
Fx
Muscle 2
J.J. Kutch, F.J. Valero-Cuevas / Journal of Biomechanics 44 (2011) 1264–1270
Kutch andValero-Cuevas, 2011

63. Acknowledgements
63
Dr. Francisco
Valero-Cuevas
Dr. Hod
Lipson
Dr. Gerald
Loeb
Dr. Eva
Kanso Dr. Jason
Kutch
Josh Inouye
Sudarshan
Dayanidhi
Dr. Heiko
Hoffmann
Dr.Anupam
Saxena
Dr. Jae-Woong
Yi
Kornelius
Rácz
Brendan
Holt
Alex Reyes
Emily
Lawrence
Dr. Srideep
Musuvathy
John
Rocamora
Dr. Marta
Mora
Na-hyeon
Ko
Alison HuDr.
Evangelos
Theodorou
Dr. Caroline
LeClercq
Dr.Vincent
Rod Hentz
Dr. Nina
Lightdale
Dr. Isabella
Fasolla
Kari Oki
Dr.
Terrance
Sanger

64. Acknowledgements
64
The
NaEonal
Science
FoundaEon:
CAREER award,
EFRI - COPN to FVC
The National Institutes of Health
NIAMS/NICHD R01-AR050520; R01-AR052345

65. Thank you!
65