Acellera Founder Gianni De Fabritiis, and CTO Matt Harvey talk about the latest developments of high-throughput molecular dynamics both in terms of applications and methodological advances. Examples are in the context of ACEMD, a highly efficient, best-in-class graphical processing units (GPUs) centric code for running MD simulations, and its protocols. In particular, attendees will learn how the high arithmetic performance and intrinsic parallelism of the latest NVIDIA Kepler GPUs can offer a technological edge for molecular dynamics simulations. Try GPUs for free via: www.Nvidia.com/GPUTestDrive

1. ACEMD: HIGH-THROUGHPUT
MOLECULAR DYNAMICS WITH NVIDIA
KEPLER GPUS
info@acellera.com
www.acellera.com

2. - M. Harvey, G. Giupponi and G. De Fabritiis, ACEMD: Accelerating biomolecular dynamics in the
microsecond time scale, J. Chem. Theory and Comput. 5, 1632 (2009).
- M. J. Harvey and G. De Fabritiis, An implementation of the smooth particle-mesh Ewald (PME)
method on GPU hardware, J. Chem. Theory Comput., 5, 2371–2377 (2009).
www.acellera.com

3. Paradigms of molecular dynamics
High performance!
• A single or few simulations
run for very long
• Reached simulations time
of several milliseconds
• Best systems: Anton,
Desmond
• A bit easier to analyze
High-throughput!
• Very many runs of
reasonable length
(hundreds of ns)
• Reached simulations time
of several milliseconds
• Best systems: GPUs
clusters, GPUGRID.net,
Folding@home
• Complex analysis
www.acellera.com

4. Binding processes
Direct!
A+B
Intermediate!
AB
A+B
www.acellera.com
AB*
AB

5. SH2-pYEII
• T. Giorgino, I. Buch and G. De Fabritiis, J. Chem. Theory Comput.,8, 1171–
1175 (2012).
www.acellera.com

6. FAAH-AEA
•
E. Dainese, G. De Fabritiis et al. submitted (2013).
www.acellera.com

7. IN-SILICO LIGAND BINDING
Trypsin-Benzamidine
I. Buch, T. Giorgino and G. De Fabritiis,Complete reconstruction of an enzyme-inhibitor
binding process by molecular dynamics simulations, PNAS 108, 10184-10189 (2011).
www.acellera.com

8. Free ligand binding simulations!
35,000
atoms
500 trajectories
100
ns/each
50
µs of data
Beta-Trypsin/Benzamidine (3PTB)
ACEMD software
AMBER99SB ff.
Explicit solvent
www.acellera.com

9. Calculating kinetics of binding
Assuming first order kinetics
www.acellera.com
Guilliain
Singhal N et al. J Chem Phys (2004)
F and Thusius D. J Am Chem Soc (1970)

10. Characteristic transition modes
www.acellera.com

11. Trypsin-benzamidine from X-ray
a) Poses of Benzamidine on Trypsin detected through high pressure x-ray crystallography
b) and c) The native binding pose of Benzamidine on Trypsin
d) Benzamidine poses labeled from X0-X8 on the front and back side of Trypsin
www.acellera.com

12. FBDD ON FACTOR XA
With Noelia Ferruz Capapey (Universitat Pompeu Fabra)
Matt Harvey (ACELLERA), Jordi Mestres (IMIM)
www.acellera.com

13. S1
S4
[1] Lee Fielding , Dan Fletcher , Samantha Rutherford , Jasmit
Kaurand, Jordi Mestres. Exploring the active site of human
factor Xa protein by NMR screening of small molecule probes.
Royal Society of Chemistry 2003.
www.acellera.com

14. Library of
compounds
34 compounds
screened by STD-NMR
9 ligands
bound to factor Xa
4 ligands
selected for further
studies
www.acellera.com

15. Three derived by known inhibitors
www.acellera.com

16. Experimental competition
assays with TPAM
• Binding sites positions
were hypothesized from
similarity to known
ligands:
• Ligands 29, 31 at site S1,
• Ligands 10, 27 at site S4.
• Displacement for ligands
10 and 29, but only
partially to 27 and no
displacement for 31.
Ligand
Predicted Pose
Displacement
Further comments
31
S4
No
29
S1
Yes
-
27
S4/core
Yes
Only partially displaced
10
S4
Yes
Highest affinity but not
displacement
Bad fit in experimental
www.acellera.com
curves

17. METHODS
• 34 ligands built, simulated by
MD and analyzed b means of
Markov State Models
• Protein structure from human
Factor Xa (2BOK[2]) was
established as the initial
protein conformation
• Each box contained only one
randomly placed ligand giving
a final concentration of
0.0038M
• 1000 replicas of 50 ns were
run for each system for an
aggregate of 1.8 ms
simulation data.
www.acellera.com

18. Kinetics and thermodynamics
Ligand
Kd (µM)
Residence Time (ns.)
∆G (kcal/mol)
kon(s-1·M)
koff (s-1)
29
126.9 ± 56.9
74786 ± 15914
-5.35 ± 0.2
(1.17 ± 0.07) ·108
(1.46 ± 0.63) ·104
15
363.1 ± 12.6
113445 ± 3637
-4.69 ± 0.0
(2.43 ± 0.00) ·107
(8.82 ± 0.23) ·103
16
1089.9± 659.1
87491 ± 12875
-4.10 ± 0.2
(1.23 ± 2.81) ·107
(1.19 ± 0.34) ·104
31
1543.5 ± 432.3
11165 ± 2623
-3.86 ± 0.2
(6.33 ± 1.09) ·107
(9.38 ± 1.83) ·104
-3.76 ± 0.0
(7.33 ± 0.40) ·10
6
(1.27 ± 0.07) ·104
(9.26 ± 0.11) ·10
6
(1.90 ± 0.00) ·104
7
(7.49 ± 2.09) ·104
27
1736.7 ± 119.3
78895 ± 3783
10
2047.3 ± 24.1
3
2056.7 ± 699.5
19753 ± 26094
-3.72 ± 0.3
(3.68 ± 0.35) ·10
13
3045.3 ± 1375.8
48332 ± 57242
-3.51 ± 0.3
(9.38 ± 7.20) ·107
(3.80 ± 3.04) ·105
23
5404.1 ± 2531.3
9673 ± 21216
-3.34 ± 0.8
(1.35 ± 4.27) ·108
(7.23 ± 3.88) ·105
12
9199.3 ± 1026.4
1959 ± 313
-2.78 ± 0.1
(5.65 ± 0.18) ·107
(5.20 ± 0.64) ·105
-2.61 ± 1.4
(1.21 ± 1.89) ·10
7
(9.43 ± 4.53) ·103
7
(2.94 ± 0.06) ·106
21
12126 ± 10080
52773 ± 50
-3.67 ± 0.0
162406 ± 117707
11
27013 ± 2511
340 ± 7
-2.14± 0.1
(1.10 ± 0.11) ·10
28
28500 ± 1844
1092 ± 67
-2.11 ± 0.1
(3.23 ± 0.03) ·107
(9.19 ± 0.54) ·105
7
29433 ± 1331
294 ± 11
-2.09 ± 0.1
(1.16 ± 0.02) ·108
(3.40 ± 0.13) ·106
20
34626 ± 5960
1625 ± 274
-2.00 ± 0.0
(1.83 ± 0.01) ·107
(6.33 ± 1.15) ·105
-1.94 ± 0.0
(1.01 ± 0.05) ·10
8
(3.82 ± 0.06) ·106
(1.56 ± 0.27) ·10
8
(6.28 ± 0.15) ·106
7
(2.16 ± 0.38) ·106
8
37980 ± 1505
261 ± 3
32
42040 ± 11404
26
60353 ± 11699
478 ± 89
-1.67 ± 0.1
(3.59 ± 0.09) ·10
2
92553 ± 27598
1697 ± 923
-1.44 ± 0.2
(1.52 ± 1.79) ·107
(1.78 ± 2.52) ·106
33
102180 ± 93464
1142 ± 365
-1.48 ± 0.3
(1.21 ± 0.07) ·107
(1.30 ± 1.35) ·106
25
123000 ± 5773
544 ± 3
-1.24 ± 0.0
(1.50 ± 0.06) ·107
(1.84 ± 0.01) ·106
-1.18 ± 0.0
(4.44 ± 0.11) ·10
7
(6.04 ± 0.22) ·106
7
(1.85 ± 0.02) ·106
6
136133 ± 4224
159 ± 3
165 ± 6
-1.89 ± 0.1
5
137467 ± 4224
540 ± 5
-1.17 ± 0.0
(1.35 ± 0.03) ·10
1
157133 ± 3930
627 ± 0
-1.10 ± 0.0
(1.01 ± 0.00) ·107
(1.59 ± 0.00 ) ·106
19
157333 ± 805
673 ± 28
-1.10 ± 0.0
(9.44 ± 0.02) ·106
(1.49 ± 0.06) ·106
4
210333 ± 6609
169 ± 24
-0.92 ± 0.1
(2.84 ± 0.06) ·107
(5.98 ± 0.70) ·106
17
238333 ± 23561
376 ± 21
-0.88 ± 0.2
(1.21 ± 0.26) ·107
(2.67 ± 0.15) ·106
-0.77 ± 0.0
7
(2.83 ± 0.07) ·106
18
270400 ± 93196
353± 8
(1.05 ± 0.00) ·10
7
(4.60 ± 0.01) ·106
30
345600 ± 8073
217 ± 0
-0.63 ± 0.0
(1.33 ± 0.00 ) ·10
22
464667 ± 1143
605 ± 0
-0.45 ± 0.0
(3.56 ± 0.03) ·106
(1.65 ± 0.00) ·106
24
539400 ± 3960
144 ± 3
-0.37 ± 0.0
(1.28 ± 0.02) ·107
(6.90 ± 0.18) ·106
14
658867 ± 105204
90 ± 1
-0.26 ± 0.1
(1.73 ± 0.32) ·107
(1.10 ± 0.02) ·107
-0.07 ± 0.0
(8.03 ± 0.32) ·10
6
(7.12 ± 0.12) ·106
(8.59 ± 0.11) ·10
6
(1.00 ± 0.02) ·107
9
34
887000 ± 22518
1172000 ± 33704
140 ± 2
99 ± 2
0.09 ± 0.0
www.acellera.com

19. www.acellera.com

20. LIGAND 27
Experimental results!
•
•
Computational results!
Weakly displaced by TPAM.
Hypothesized to bind at S4 pocket
by similarity with Berlex compound
• Binds at the core part of the
cavity and entrance of S1
www.acellera.com

21. www.acellera.com

22. LIGAND 10
Experimental results!
•
•
Computational results!
Clearly displaced by TPAM.
Expected to bind at pocket S4 by
similarity with Rhone-Poulec Rorer
compound.
• Binds at pocket S4
• Sixth in ranking by KD
www.acellera.com

23. LIGAND 29
Experimental results!
•
•
Computational results!
Clearly displaced by TPAM.
Hypothesized to bind at S1 pocket
by similarity with DuPont
compound
• Binds at pocket S1
www.acellera.com

24. LIGAND 31
Experimental results!
•
•
•
Computational results!
KD = 30µM
Failure to be displaced by TPAM.
Expected to bind at S1 pocket
becouse of the known high affinity
of the S1 pocket for the amidine
fragment
• Binds beneath the loop
between S1 & S4
www.acellera.com

25. Ensemble view with TPAM
www.acellera.com

26. METHODS
From hardware to software
www.acellera.com

27. “Molecular simulation will mature within the next 5 years to
allow simulations at temporal scales of biological interest, thus
achieving its full potential for biological discovery”
27
www.acellera.com

28. ACEMD - History
•
Developed from CellMD (2006), 19 times a CPU
•
First CUDA GPUs released 2006
•
ACEMD released 2007
•
First fully-GPU accelerated MD application
•
First GPU implementation of Particle Mesh
Ewald doi:10.1021/ct900275y
•
Presented in ACEMD: Accelerating
Biomolecular Dynamics in the microsecond time
scale, JCTC 2009 doi:10.1021/ct9000685
www.acellera.com

29. ACEMD - Capabilities
ACEMD has all the features required for production simulations of
biomolecules:
•
•
•
•
Major force fields: CHARMM, Amber, OPLS and Martini
Common file formats: PDB, Bincoor, PRMTOP, PSF, DCD, XTC
PME or GRF electrostatics
NVE, NVT, NPT ensembles
– Langevin thermostat
– Berendsen barostat
•
•
•
•
Constraints, restraints
Powerful scripting and extension capability
Multi-host execution for replica-exchange methods
Binary distribution – no compilation necessary
www.acellera.com

30. ACEMD - Resources
User manual
Extensions developer manual
Protocols manuals
Support forum for everybody
support@acellera.com for paying users
Develop for you to allow to interface your methods
as plugins (almost always free)
• Acecloud – acemd cloud
• Metrocubo – acemd special patented hardware
•
•
•
•
•
•
www.acellera.com

31. ACEMD - Performance
DHFR
Exceptional single GPU performance
Parallel scaling up to 1.4x on 3 GPUs
(single host)
Titan
OC
System sizes up to ~1M atoms
Performance scales ~linearly with
system size and GPU speed
Does not need a GPU with fast
double-precision arithmetic
Does not need a fast CPU;
performance normally dependent on
GPU
GTX780
ns/day
Tesla
K20,
GTX680
Tesla
M2090,
GTX
580
0
•
•
50
100
150
200
250
Benchmarking conditions: DHFR model (23558 atoms) NVT cutoff 9A, PME enabled (frequency 2), dt=4fs. Langevin thermostat
System: 4 GPUs, X79 chipset, CUDA 4.2, driver 310.44, CentOS 6
www.acellera.com

32. ACEMD – Free basic download
• Optimal if you do little use of MD
• Have only single GPU machines in your lab
• Fully functional version of ACEMD on a single
GPU
• Ideal for small groups or to start on MD
• http://www.acellera.com/acemd
www.acellera.com

33. ACEMD - Extensions
• ACEMD can be extended by the user
•
TCL – coded directly in input file
•
Plugins – separate binary library
•
Pros:
•
Pros:
– Fast to develop
– Very familiar for NAMD users
•
Cons:
– Slow for numerically intensive work
– Not all features exposed
– Only one TCL extension at a time
•
– Written in C or C++
– Fast for numerically intensive work
– More advanced features than TCL
interface
– Have multiple plugins active
simultaneously
Suitable for:
– Applying point restraints
– Modification of simulation parameters
(eg temperature annealing)
•
Cons:
– Written in C or C++
• Suitable for:
– Writing complex, intensive plugins
– Interfacing with existing, third-party code
www.acellera.com

34. ACEMD – Extensions
• Simple event-based programming model
• Clean separation between ACEMD and extension
– Much easier and safer to develop for than direct sourcecode modification
• Documented API with examples
• Events:
–
–
–
–
Initialise called once at the beginning of the simulation
Calcforces called every iteration during force evaluation
Endstep called at the end of every iteration
Terminate called once at the end of the simulation
www.acellera.com

35. TCL Extension Example
•
Set thermostat parameters
•
•
Enable tclforces
Set annealing parameters
•
Frequency of calling extension
•
Calcforces – called every* iteration
–
Calculate new target temperature
–
Apply new target temperature
–
Disable extension when target temperature
reached
www.acellera.com

36. Plugin Example (1)
•
Include API definition
•
Set default values
•
Initialisation function:
–
•
Parse vlaues from arguments
passed in the input file
Calcforces called every*
iteration
–
Apply new target temperature
www.acellera.com

37. Plugin Example (2)
Compile:
Configure in input file:
www.acellera.com

38. Metrocubo
•
•
•
•
•
Patent pending
4 GPU workstation designed for ACEMD
Compact, quiet chassis
E3 Xeon CPU
Operating System and ACEMD installed
Best price/performance for MD
www.acellera.com

39. ACEMD Test Drive
• One week of access to a 4-GPU Metrocubo
• Test ACEMD with your current models
• Expert support for system setup and testing
• http://www.acellera.com/products/metrocubo/metrocubo-test-drive/
www.acellera.com

40. ACECloud
• Run ACEMD easily on Cloud resources
– No need to deal with queuing systems
– All files copies transparently
• Simple command-line interface
– optimised for managing large numbers of
simulations
– Supports many users
• Test drive now:
info@acellera.com
Patent pending
www.acellera.com

41. ACECloud
Run a simulation on the cloud:
See the progress of all simulations:
Patent pending
www.acellera.com

42. In-silico binding assays @ Acellera
• We performed the calculations on 30 ligands in
45 days (1600 GPU days using acecloud)
• Determined 4 strong fragments by residence
time and another small group as intermediates,
the others discarded
• Poses available for follow-up
• Pathway of binding
• Currently performing NMR on top molecules
www.acellera.com

43. http://htmdworkshop.wordpress.com
www.acellera.com

44. Any Questions?
Write to:
info@acellera.com
www.acellera.com

45. GPU Accelerated Apps Momentum
Key codes are GPU Accelerated!
Molecular Dynamics
"
"
"
"
"
"
"
"
"
Abalone – GPU only code
ACEMD – GPU only code
AMBER
CHARMM
DL_POLY
GROMACS
HOOMD-Blue – GPU only code
LAMMPS
NAMD
Quantum Chemistry
"
"
"
"
"
"
"
"
"
ABINIT
BigDFT
CP2K
GAMESS
Gaussian – in development
NWChem
Quantum Espresso
TeraChem – GPU only code
VASP
Check many more apps at www.nvidia.com/teslaapps

46. Test Drive K20 GPUs!
Experience The Acceleration
Run ACEMD on Tesla K20 GPU
today
Sign up for FREE GPU Test Drive
on remotely hosted clusters
www.nvidia.com/GPUTestDrive

47. Test Drive K20 GPUs!
Questions?
Experience The Acceleration
Run ACEMD on Tesla K20 GPU
today
Sign up for FREE GPU Test Drive
on remotely hosted clusters
www.nvidia.com/GPUTestDrive
Contact us
" Devang Sachdev - NVIDIA
" dsachdev@nvidia.com
" @DevangSachdev
" Acellera ltd
"
info@acellera.com
Stream other webinars from GTC
Express:
http://www.gputechconf.com/page/
gtc-express-webinar.html

48. Upcoming GTC Express Webinars
July 30 - Getting Started with GPU-accelerated Computer
Vision using OpenCV and CUDA
July 31 - NMath Premium: GPU-accelerated Math Libraries for
.NET
August 7 - Accelerating High Performance Computing with
GPUDirect RDMA
Register at www.gputechconf.com/gtcexpress

49. GTC 2014 Call for Submissions
Looking for submissions in the fields of
§ Science and research
§ Professional graphics
§ Mobile computing
§ Automotive applications
§ Game development
§ Cloud computing
Submit at www.gputechconf.com