How to quantify human visual function using multimodal approach with both functional and structural tests (with a focus here more on functional tests)

1. Petteri Teikari, PhD
http://petteri-teikari.com/
Version “Mon 9 April 2018 “
Portable Visual
Function Diagnostics
Deep learning based data-
driven ophthalmology
beyond unimodal “magical”
scalar measures

2. Future Trends
for Healthcare
at HealthtechFundingForum-Advancing Innovationin DigitalHealth September 27, 2017|WellcomeTrust, London, UK
DrVishalGulatiResponsibleforhealthcaredealsatDraperEspritplc, alistedPatientCapitalVC firm:
interpretedfor“HealthcareDesign”:
“If one would design healthcare systems now, they would not look like
the contemporary ones. … just look at how Chinese are automating their
healthcare.. and how startups that target health in general rather than
solving problems at the hospital only when people have got already sick”
Lost in Thought —
The Limitsof the Human Mind and the Future of Medicine
Ziad Obermeyer, M.D., and ThomasH. Lee,M.D.
NEngl J Med 2017; 377:1209-1211 September28, 2017
DOI: 10.1056/NEJMp1705348
If a root cause of our challenges is complexity, the solutions are
unlikely to be simple. Asking doctors to work harder or get smarter
won’t help.
There is little doubt that algorithms will transform the thinking
underlying medicine. The only question is whether this transformation
will be driven by forces from within or outside the field. If medicine
wishes to stay in control of its own future, physicians will not only
have to embrace algorithms, they will also have to excel at developing
and evaluating them, bringing machine-learning methods into the
medical domain.

3. Future Trends
for Healthcare
AIIsYourDoctor’sNextBest Friend
MikeMcCormickJan192017
https://mccormick.vc/ai-is-your-doctors-next-best-friend-2bb33e7cf4e8
Wherewillintelligentmachines affect
healthcare?
Eventuallymachine intelligencewill touch virtuallyall
aspectsof healthcare. Four areasalready beingaffected
are…
● Diagnosticsand detection: Examples:radiology,
tissueanalysis, genomicinsights, chatbots, and
patient monitoringviaexternal sensors, wearables
and implantables.
● Treatment and patient care: Examples:
personalized precision drugsand treatment plans,
remotepatient monitoring, and automated real-time
treatmentadjustments.
● Drug development: Deep learningwill augment
the pharmaceutical industry’sincreasinglycostlyR&D
processesby identifyingpatternsin molecular
interactionsat previouslyunheard oflevels of
granularityand efficiency. Machine learningwill also
better match patientstoclinical trialsleadingtobetter
patient outcomesand faster drugapprovals.
● Informatics, system-design and data
management: Machineswill bringefficiencytothe
interactionsthat takeplace within the complexweb of
stakeholdersand processesthatcomprise modern
healthcare systems.
Barriersandrisk factors
Healthcare, perhapsmorethan mostindustries, presentsseveral barriers
and risk factorsto newtechnologiesand would-bedisruptors:
Highstakes: Theliteral life-and-deathnatureof healthcaremakefor a
tinymarginof error inpatient-facing technologies.
Legal andregulatory issues: Healthcareisamong theworld’smost
heavilyregulatedindustries.
Datasecurity andaccess: Making data accessibleyet secureis
crucial.
Dataquality: Thoughthehealthcareindustryissitting on ever-growing
mountainsof data, thequalityand relevanceof thedatasets isn’talways
great,and accessing meaningful datasetscan bechallenging,particularly
forstartups.
Causalcomplexity inbiology anddisease:Ourunderstanding of
biological systemsand diseasesisincomplete. Thecellularprogression
of cancersand thecomplexityof moment-to-momentneural
interactions,forexample, areprocesseswe’refarfromfully
understanding.
Misalignedincentives: Disparatestakeholdersarenotalways
incentivizedtosharedataorplaynicelywithoneanother.
Bioethicalconsiderations:Theseissueswillbecomestickier and
moredifficultto parseasthelinesbetween biologyand technologyblur.
Forexample, whatwill betheethical implicationsof advanced genetic
engineering thatallowsfor“designer”babiescrafted to theirparents’
exactspecifications?

4. Future Trends
for Healthcare
Slowly FDA (and
regulators in
general) is waking up
to the situation
"When you start adding analytical AI
for any image analysis—think of
detecting cancer or some other
serious disease—at that point
people need to know when that
detection means something and is
real," Bakul Patel, FDA’s associate
director for digital health, says.
https://spectrum.ieee.org/the-human-os/biomedi
cal/devices/fda-assembles-team-to-oversee-ai-r
evolution-in-health
New AI Device for Diabetes Eye
Screening to Complete FDA
ClinicalTrial
IDx, an early-stage medical device company focused on
developing software-based algorithms that can identify disease in
medical images is currently conducting an FDA clinical trial to
obtain clearance for its first product, IDx-DR by the end of summer
2017. IDx-DR is a screening solution for diabetic
retinopathy. IDx also has algorithms in development for the
detection of macular degeneration, glaucoma, Alzheimer’s
disease, cardiovascular disease, and stroke risk.
http://hitconsultant.net/2017/07/06/new-ai-device-diabetes-eye-screening/

5. Future Trends
for Healthcare
Digitalizing
hospital
processes
The Hospitalof the Future isa
Network
February17,2017– JeroenTas
ChiefInnovation&StrategyOfficer atPhilips
https://www.linkedin.com/pulse/hospital-future-network-
connecting-care-continuous-health-jeroen-tas/
HowGoogleDeepMind'sStreamsapp
islayingthefoundationsforartificial
intelligence-powered healthcarein
theNHS
Wednesday 10May2017
http://www.cityam.com/264463/google-deepminds-streams-app-laying-foundation
s-artificial
DeepMind's work with the NHS to help alert doctors to
patients whose health is at risk is laying the
groundwork for delivering information that one day will
be powered by artificial intelligence.
The Google-owned British pioneer is working with the
Royal Free London NHS Trust on a
smartphone app called Streams. It currently uses an
NHS created algorithm to provide information to
clinicians relating to acute kidney injury (AKI), with
DeepMind creating the method of delivery that
includes "breaking news" style notifications. It does not
use AI despite the companies expertise in the
technology.
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5333321/
Gettingtheinteroperabilityanddatapipe
solved allowing 3rd
party“AIapps”andreally
allowingefficientdeeplearning miningofpatients
Through FHIR (“Fire API”)

6. Future Trends
for Healthcare
System Thinking
rather than module-
based optimization
reducing system
efficiency
https://hbr.org/2017/06/hospitals-are-dramatically-overpaying-
for-their-technology
-revolutionise-way-they-work-how-hospitals-could-be-rebuilt-better
CommandCenter to ImprovePatientFlow
http://www.hopkinsmedicine.org/news/articles/command-center-to-improve-patient-flow
https://www.ahcmedia.com/articles/139933-hopkins-command-center-improves-quality-with-coordination
http://www.modernhealthcare.com/article/20161126/MAGAZINE/311269980

7. Future Trends
for Healthcare
Deep learning image
analysis for just one type
images, let alone full EHR
mining is constrained by
poor data infrastructure
and bad curation with
missing segmentation and
pathology class labels.
Labeling need medical
expertise making the
process harder than just
crowdsourcing dog vs. cat
labels for example
CrowdsourcingtoEvaluateFundusPhotographsforthe
PresenceofGlaucoma Wang et al. (2017)
doi: 10.1097/IJG.0000000000000660
To assess the accuracy of crowdsourcing for grading optic
nerve images for glaucoma using Amazon Mechanical Turk
before and after training modules.
Gamificationof theelectron microscopesegmentationthrough
EyeWireprojectrun by Sebastian Seung.
A VideogameThatRecruitsPlayersto Map theBrain |WIRED
EyeWire,A Gameto Map theBrain fromMIT
Voxeleron Orion has developed a good augmented
intelligence for efficient collaboration with the AI and the
person segmenting the retinal layers (without yet gamifying
theexperience)
By ROWLAND MANTHORPE 23 Sep 2017
http://www.wired.co.uk/article/harri-valpola-curious-
ai-artificial-intelligence-third-wave
Harri Valpola, 44, is founder of
TheCuriousAI Company (co-founded
with Antti Rasmus, Timo Haanpää and Mathias
Berglund), that focuses on semi-
supervised learning a 20-person
artificial intelligence startup based in
Helsinki, which has just raised $3.67
million in funding – small change
compared to many tech funding
rounds, but an impressive sum for a
company that has no products and is
onlyinterested in research.
https://doi.org/10.1016/j.patcog.2016.09.030

8. Future Trends
for Healthcare
Rethinking medicine –
novel ways to deliver
healthcare
by ChristinaFarr April 7, 2017
https://www.technologyreview.com/s/604053/can-digital-therapeutics-be-as-good-as-drugs/
Jose Hamilton: “The "real" digital
therapy won't be a competitor to
biological therapy. But a platform where
psychological treatments (digital or
personal), biological (pills) or even
physical would be optimized,
personalized and accountable.“
To distinguishthemselvesfrom“wellness”gadgets,
digitaltherapeuticscompaniestendto carryout
clinicaltestsandsometimesseekregulatory
approvals

9. Future Trends
for Healthcare
Rethinking medicine –
how clinical
profession should
change, with the ones
not changing,
perishing away
Digital evangelists argue that intelligent machines will be able to incorporate the latest data and research
immediately, but that is both questionable and a potential weakness. Clinical trials vary in scale and quality,
and indiscriminate inclusion would inevitably lead to mistakes. Digital hardliners would argue that machines
should judge the quality of the research, but for the foreseeable future the expertise of doctors will be
essential to deciding the validity of new approaches.
So perhaps one of the most powerful effects of artificial intelligence will be, perversely, to make
healthcare more human and personal. It will remove the dependency on doctors’ fallible memory and
incomplete knowledge, and free them to use machine-generated information to work with patients to
shape their specific treatment.
11 March 2017
https://www.theguardian.com/healthcare-network/2017/mar/11/artificial-intelligence-nhs-doctor-patient-relationship
https://www.newyorker.com/magazine/2017/04/03/ai-versus-md
GeoffreyHinton now qualifies the provocation. “The role of radiologists
will evolve from doing perceptual things that could probably be done by a
highly trained pigeon to doing far more cognitive things,” he told me. His
prognosis for the future of automated medicine is based on a simple
principle: “Take any old classification problem where you have a lot of data,
and it’s going to be solved by deep learning. There’s going to
be thousands ofapplicationsofdeeplearning.”

10. Future Trends
for Healthcare
Rethinking medicine –
making doctors more
human again
https://www.technologyreview.com/s/609060/put-humans-at-the-center-of-ai/

11. Future Trends
for Healthcare
All the “digital
natives” will get
into the play
January11,2017
Nokia'svisionfordigitalhealth:FromAIanalyticstoconnectedhairbrushes.NokiaboughtFrenchhealthdevice
manufacturerWithingsearlierthisyeartotakeon the IoThealthcaremarket.What'snext forthecompany.
http://www.zdnet.com/article/nokias-vision-for-digital-health-from-ai-analytics-to-connected-hairbrushes/
January24,2017:
The Chan Zuckerberg Inititative,aphilanthropic initiativefrom Facebook CEOMark ZuckerbergandhiswifeDr.Priscilla
Chan,apediatrician,hasacquiredastartup, Meta,focusedon usingAIandmachinelearningto sift throughrecently
published scientificstudies. TheChan Zuckerberg Initiativeisalimitedliabilitycompanyfocusedon the ambitious
goal to"cure,prevent,ormanagealldiseasesby theendof thecentury." Atleast$3billionwillbeallocatedtowardthat
goal,allcomingoutofChan andZuckerberg'sFacebookshares.
http://www.mobihealthnews.com/content/chan-zuckerberg-initiative-acquires-ai-startup-meta-will-offer-its-services-free
April30,2017:
Googletocommercializeartificial intelligencetodetect diseases.LilyPeng,productmanagerofthemedical
imagingteamatGoogleResearch,sharedhowtheUStechgiantisusing deeplearningtotrain machinesto analyze
medicalimagesand automaticallydetectpathologicalcues,beitswollen bloodvesselsintheeye orcancerous
tumors,duringavideoconferencewiththeSouthKorean mediahostedby GoogleKorea.
http://m.theinvestor.co.kr/view.php?ud=20170430000162
June7,2017:
Applewantsa pieceof theartificialintelligencepie. Apple’sResearchKit,whichusesiPhonesto collect health
information and then makesthedataavailableforresearch, isshowingpromiseafterscientistspublisheddataon
seizures,asthmaattacksandheartdiseaseusingthetool.WhileApplestillfaceschallengesapplyingResearchKit’s
resultstoabroaderpopulation (mostconsumersofAppleproductsareyounger,well-off and well-educated),the
company seemsdetermined tocarveoutanichein healthcareandAIcould helpitsefforts.
http://www.healthcaredive.com/news/apple-wants-a-piece-of-the-artificial-intelligence-pie/444393/
July28, 2017:
Here'swhattomakeofAmazon'spotentialconnected healthplay …Amazon'spotentialadvantagesinthe
connectedhealthdevicemarketlikelyoutnumberit'sdisadvantages.
http://uk.businessinsider.com/amazons-healthcare-play-2017-7?r=US&IR=T
September25, 2017:
Microsoft hires Iain Buchan , worldleaderindigitalhealthcaretotakepersonalisedhealthtothenextlevel
https://news.microsoft.com/en-gb/2017/09/25/microsoft-hires-world-leader-in-digital-healthcare-to-take-personalised-health-to-the-next-level/

12. Future Trends
for Healthcare
Data is the new gold
And for the healthcare (especially
public systems), to the stay
competitive, the organizations
should think of their data
strategy along with monetization
schemes
https://hbr.org/2017/06/to-survive-health-care-data-providers-need-to-stop-selling-data
“Most data-driven healthcare IT (HCIT)
providers aren’t going to survive. Their
business models are at serious risk of failure
in the next three to five years. To beat those
odds, they need to evolve dramatically, and
fast, to a point where they are not selling
data at all.”

13. Future Trends
for Eye Care

14. Future Trends
for Eye Care
Similar slow
awakening to “machine
medicine” in
ophthalmology as in
healthcare in general
BIGDATA:CURRENTSTATUSANDFUTURE DIRECTIONSAGENDA
ARVO2017|Baltimore,MD
Organizers:MichaelF.Chiang,MD,AnneL.Coleman,MD,PhD,FARVOandSethBlackshaw,PhD
“Similarly as highlighted in the perspective by Obermeyer
and Lee (2017) in previous slide, ophthalmologist training
need to keep up-to-date with the machine learning
revolution.
With big data, the role of diagnoses have to be re-valued
as well as even more fine-grained phenotyping becomes
possible. And would a patient with DR and glaucoma be just
DR+Glaucoma, or something slightly different with these
coexisting pathologies”

15. Future Trends
for Eye Care
AI-driven drug
discovery for eye
care as well
SiliconValleyComputational
DrugStartupTakeson Glaucoma
By TeklaS.Perry-Posted 13Mar 2017
https://spectrum.ieee.org/view-from-the-valley/at-work/start-ups/silicon-valley-computational-drug-startup-takes-on-glaucoma
TwoXAR (AndrewA.Radin) announced a partnership with Santen Inc., the U.S. subsidiary of Japanese
ophthalmology company Santen Pharmaceutical, to collaborate on identifying new drug candidates for the
treatmentofglaucoma.
BenevolentAI is
currently largest
privateAI firm in
Europe
21March2017
https://www.cnbc.com/video/2017/03/21/be
nevolent-ai-is-currently-largest-private
-ai-firm-in-europe.html
http://www.wired.co.uk/article/benevolent-ai-london-unicorn-pharma-startup

16. Future Trends
for Eye Care
Battle of the Egos on
many fronts.
Optometrists want to to
upskill themselves, and
essentially make money
from surgeries and
lucrative VEGF
injections
“Both optometrists and ophthalmologists point to experiences in
Oklahoma to support their positions.
Bryant said that in Oklahoma, which has allowed expanded work
for optometrists the longest, there were only two reported
complaints for more than 25,000 procedures.
The ophthalmologists point to a research paper published last
October in the medical journal JAMA Ophthalmology (Steinetal.2016)
that
found that patients who had a certain type of laser surgery to
treat glaucoma had to go back for treatment on the same eye 35.9
percent of the time when an optometrist did the work, as opposed
to 15.1 percent of the time when an ophthalmologist did it.”
http://www.newsobserver.com/news/politics-government/state-polit
ics/article131198204.html
MAR02,2017-AAO
OptometristsinFloridaTake
BrazenStepTowardPrimary-
CareProviderStatus
Proposal sets the bar for audacious assaults
on patient safety by attempting to place
100,000-plus non-surgeons on equal footing
withophthalmologists
http://optometrytimes.modernmedicine.com/optometrytimes/news/intravitreal-
injections-optometrists
The article, “Implementation of aNurse-Delivered Intravitreal Injection Service” was published in the
June 2014 issue of Eye. The purpose of this study was “to introduce nurse-delivered intravitreal
injections to increase medical retina treatment capacity in the United Kingdom.” … “Our preliminary
results of a series of 4,000 nurse-delivered injections associated without serious vision-threatening
complication is indicative that this procedure can be safely administered by a nurse.” No cases of
post-intravitreal anti-VEGFendophthalmitisoccurred in thisstudy.

17. Future Trends
for Eye Care
Maintaining healthy
lifestyle as the most
obvious first step to
treat.
Not every approach needs
to be high-tech and
highly scalable digital
service
AerobicExerciseforNeuroprotection "Aerobic exerciseis known to
lower intraocular pressure(IOP), which weknow protects retinal ganglion
cells," says Harry A. Quigley, MD, professorand director of glaucoma
services at the WilmerEye Institute at Johns Hopkins University in Baltimore.
"And short-term studies show it may improve blood flow to theretina and
optic nerveas well."
http://www.glaucoma.org/treatment/aerobic-exercise-for-neuroprotection.php
http://dx.doi.org/10.1111/acel.12512
“These data provide new insight into the
mechanisms underlying exercise-mediated
protection of retinal cells. We found that
daily forced exercise, initiated 24 h
after an acute RGC-specific injury in
middle-aged mice, led to a substantial
improvement in RGC function and survival.”
Lifestyle,Nutrition,and
Glaucoma
LouisR Pasquale,JaeHeeKang
Journal of Glaucoma: August2009 -Volume18 - Issue6 - pp 423-428
doi: 10.1097/IJG.0b013e31818d3899
In this review, we have examined the evidence on whether
environmental factors are related to developing
glaucoma. How do we answer the questions from newly
diagnosed glaucoma patients on lifestyle behaviors and
their relation to POAG? There is even scarcer data on
lifestyle factors and their influence on disease
progression. However, rather than default to the view that
patients should simply comply with medical therapy and
follow-up recommendations (which of course is true), we
also suggest advocating for activities consistent with
overall good health such as avoidance of smoking,
moderate exercise and a diet high in fruits and vegetables.
The weight of the current medical is not sufficiently strong to
make broad recommendations regarding activities that
glaucoma patients should avoid because they elevate IOP
such as certain yoga positions, playing high wind
instruments for long periods of time, and drinking large
amountsof caffeinated coffee.

18. Future Trends
for Eye Care
Where is the
innovation happening?
According to the 2017 Centre for World University Rankings (CWUR)
rankings by subject, UCL Institute of Ophthalmology (Moorfields
Hospital) is thebest place in the world to study ophthalmology
http://cwur.org/2017/subjects.php#Ophthalmology
University College London (Moorfields)
Harvard University
Johns Hopkins University (Wilmer Eye Institute)
University of Melbourne
National University of Singapore (Singapore Eye Research Institute, SERI)
University of Sydney
University of Southern California
University of Miami (Bascom Palmer Eye Institute)
University of California, Los Angeles (UCLA Stein Eye Institute Westwood)
University of California, San Diego (Shiley Eye Institute)

19. Focus
on novel
portable
diagnostic
tools

20. de facto
“Standard”
of portable future
https://theophthalmologist.com/issues/01
16/the-eye-exams-quantum-leap/
http://dx.doi.org/10.1167/tvst.6.4.16
Test 6: Pupillometry Pupil
reactions were assessed using
simultaneous OCT capture of
the anterior segments including
the iris plane. Each eye was
stimulated independently and
sequentially with a single, bright,
250-ms flash of white light. B-
scan recordings are captured at
regular intervals of 350 ms prior
to stimulation and 4000 ms
post-stimulation. Measurements
of the pupil circumference could
subsequently be calculated to
identify pupil abnormalities and
relative afferentpupillarydefects

21. User
Experience
Portable
Diagnostics
Easier to carry around just one single
device doingmost ofthe stuff rather
than dedicated devices for each task
Inhospitalsettings,onecanthenuse
dedicateddevicesforhigherdiagnostic
capability after the portable“pre-
screening”

22. Structural
Measures

23. Fundus
Imaging
with some variants
Annidis RHAMultispectralfundusimagingsystem
http://www.annidis.com/page/technology
OptomedAurora
portablefundusimaging
http://www.annidis.com/page/technology
OptosUltra-widefundus
imaging
”High-endimaging”
http://www.nikon.com/about/technology/pr
oduct/retinal-imaging/index.htm
Do it yourself smartphone
fundus camera – DIYretCAM
https://dx.doi.org/10.4103%2F0301-4
738.194325
(a) The do it yourself smartphone fundus camera
used as a hand held the device. (b) The do it
yourself smartphone fundus camera can be held at
the condensing lens and supported with the other
hand on the camera. (c) Like in indirect
ophthalmoscopy, scleral depression is done after
stabilizing the do it yourself smartphone fundus
camera
Multi-spectralimagingfor invivoimaging
of oxygentensionand -amyloidβ
Dr. TosTJM Berendschot,
Prof.dr. Carroll ABWebers
UniversityEyeClinicMaastricht

24. Fundus
Imaging
Smartphones getting more
and more ubiquitous.
Clip-ons and embedded
electronics allow cloud-based
teleophthalmology both with
automated AI approaches and
augmented with human expert
ophthalmologists for hard-to-
reach areas
Peek Visiondeveloped a smartphoneapp and lensattachment, Peek
Retina,to capturesharp imagesof theback oftheeye.
https://www.fastcompany.com/3062154/smartphones-are-leading-the-global-charge-against-blindness
Nonmydriatic Fundus Camera Based on the
Raspberry Pi® Computer” paper in J Ophth. He
says the camera can be replicated from parts easily
foundonline for about $185.
An ophthalmology resident at the University of
Illinois at Chicago College of Medicine has
invented an inexpensive, handheld camera that
can photograph the retina without need for pupil
dilation.
https://blogs.nvidia.com/blog/2016/02/17/deep-learning-4/
. https://youtu.be/wa9OdaRMgO8
As the founder of SocialEyes,
Nicholas Bedworth is delivering
healthcare via embedded NVIDIA
(such as the JetsonTX2) at scale in
places where doctors are scarce,
and without internet for cloud
connection
EyeSpy:SocialEyesUsesDeepLearning
toSpotSeriousEyeProblems
Centralized cloud GPU inference allows higher
throughputs when good enough internet is available for example
using the V100 GPUs from NVIDIA (and the even faster generations after
that)

25. SLO
Scanning Light
Ophthalmoscope
Adaptive optics scanning light ophthalmoscopy (AOSLO) is an
emerging technology for improving in vivo imaging of the
human retinal microvasculature, allowing unprecedented
visualization of retinal microvascular structure, measurements
of blood flow velocity, and microvascular network mapping.
Human retinal microvascular imaging using
adaptive optics scanning light ophthalmoscopy
Chui et al. (2016) https://doi.org/10.1186/s40942-016-0037-8
Distribution differences of macular cones
measured by AOSLO: Variation in slope from
fovea to periphery more pronounced than
differences in total cones
Ann E. Elsner, Toco Y.P. Chui, Lei Feng, Hong Xin Song, Joel A. Papay,
Stephen A. Burns (2017) https://doi.org/10.1016/j.visres.2016.06.015
Cone density variesamongindividualsbymore than just ascalar factor.
Imaging Foveal Microvasculature: Optical
Coherence Tomography Angiography Versus
Adaptive Optics Scanning Light
Ophthalmoscope Fluorescein Angiography
Shelley Mo; Brian Krawitz; Eleni Efstathiadis; Lawrence Geyman;
Rishard Weitz; Toco Y. P. Chui; Joseph Carroll; Alfredo Dubra;
Richard B. Rosen (2016) http://dx.doi.org/10.1167/iovs.15-18932
Optical coherence tomography angiography is comparable to AOSLO FA at
imaging the foveal microvasculature except for differences in FAZ area, lumen
diameter, and some qualitative features. These results, together with its ease of
use, short acquisition time, and avoidance of potentially phototoxic blue light,
support OCTA as a tool for monitoring ocular pathology and detecting early
disease.
Photoreceptor-Based Biomarkers in AOSLO
Retinal Imaging
Katie M. Litts; Robert F. Cooper; Jacque L. Duncan; Joseph Carroll
(2017) http://dx.doi.org/10.1167/iovs.17-21868
Resolving cone inner and outer
segment structure with AOSLO.
Shown are confocal (A) and split-
detection (B) images from the
parafoveal retina of a patient
with CNGA3-associated ACHM. The
color-merged image (C) has the
confocal image displayed
in green and the split-detection image
in red.

26. OCT
with various
variants again
(2016) https://doi.org/10.1364/BOE.7.001783
https://dx.doi.org/10.1167/tvst.3.3.10
Biomedical OpticsExpressVol. 8, Issue 4, pp. 2287-2300 (2017)
https://doi.org/10.1364/BOE.8.002287

27. Intelligent
Imaging
Embed deep learning to
the device and the image
acquisition process to
minimize operator-
dependent image quality
degradations (see e.g.
OSCAR-IB study)
Think of the consumer AI-driven
camera systems as inspiration
Better for the patient and the
operator that the camera
automatically re-acquires the
images and even reconstructs the
image from partially good quality
shots rather than realizing the
suboptimal quality later when
patient have left the hospital
already
4October 2017: Google has announced a new add-
on for the Pixel 2 camera called Clips. The camera is
hands-free and works like a photographer you’ve hired
for an event. The AI captures moments for you, and
then you decide how to use the images later when you
can look through them. An AI engine snaps photos
when you arenotevenlooking or paying attention.
https://www.fastcompany.com/3059281/introducing-hover-an-ai-powered-indoor
-safe-camera-drone
Plan3D:Viewpointand
TrajectoryOptimization
for AerialMulti-View
StereoReconstruction
BenjaminHepp, MatthiasNießner,
Otmar Hilliges
(Submittedon25May2017)
https://arxiv.org/abs/1705.09314

28. Imaging
In the wild – in
“high street”
optometry and in
hospitals
OCTROLLOUTINEVERY
SPECSAVERS
ANNOUNCED
Themultiplewillensureall740ofitsUK
practiceshaveanOCTdevice installedwithin
thenexttwoyears.22May2017 by Emily McCormick
https://www.aop.org.uk/ot/industry/high-street/2017/05/22/oct
-rollout-in-every-specsavers-announced
WHAT'SGOINGONIN
TECH?
Revealingthelatesttechnologylaunchesforthe
practice07 Mar2017 by LaurenceDerbyshire
https://www.aop.org.uk/ot/industry/equipment-and-suppliers/20
17/03/07/whats-going-on-in-tech
https://youtu.be/JS550fDKyOE OT spoke to a range of equipment companies (Optos,
Heidelberg, Carl Zeiss and Bib Ophthalmic Instruments) and experts at 100% Optical to
find out about the latest technologylaunches forthe practice.
In practice opticians do not
have enough skilled staff
for image analysis, and the
whole imaging process should
be made deep learning
-driven and high quality
always independent of the
device operator
Thiscreatesopportunitiestosell
automatedimageanalysissoftwarefor
opticians, reducing their laborcosts,
improving diagnosticquality andcreating
cross-selling opportunities.
https://www.mi
vision.com.au/
taking-financi
al-control-of-
your-practice/

29. Imaging
Beyond
Retinopathies
Retina as being the
most easily measured
part of the brain,
makes sense to use
it as a proxy for
other pathologies
EyeScans toDetect
CancerandAlzheimer’s
Disease
TheHumanOS |Biomedical|Diagnostics
ByMeganScudellariPosted31Aug2017
https://spectrum.ieee.org/the-human-os/biomedical/diagnostics/eye-sca
ns-to-detect-cancer-and-alzheimers-disease
At the University of Washington, a team led by
computer scientist ShwetakPatel created a
smartphone app (BiliScreen) to screen for
pancreatic cancer with a quick selfie. Developed
over the last year and ahalf, the team recently tested
their system in a clinicalstudyof70people. They
were able to identify cases of concern with 89.7
percentsensitivityand96.8percentaccuracy.
At Cedars-Sinai and NeuroVision Imaging LLC in
California, researchers have developed a
sophisticated camera and retinal imaging approach
to detect early signs of Alzheimer’s disease (AD).
The camera capture beta-amyloid plaques being
taggedwith curcumin. This system, recentlydetailed
in a proof-of-concepttrial published in the
journal JCI Insight, relieson aspecialized ophthalmic
camerathatisnotyetavailableonasmartphone.
Photo:Dennis Wise/Universityof Washington

30. Functional
Measures

31. Ocular Blood
Flow
with Laser speckle
flowgraphy (LSFG)
or AOSLO
https://doi.org/10.1155/2017/2969064
ChangesinRetinal Vessel
Architecture andBlood Flow in
Multiple Sclerosis(P6.401)
Richard Nicholas, Adam Dubis, Ashwini
Nandoskar, Jeremy Chataway and John Greenwood
http://www.neurology.org/content/88/16_Supplement/P6.401.short
WaveformAnalysisof OcularBlood
FlowandtheEarlyDetectionof
NormalTensionGlaucoma
Shiga et al. (2013) IOVS
doi:10.1167/iovs.13-12930
Softcare Co.,Ltd.ispleased to announcethat it received 510(k) clearancefrom the U.S. Foodand
DrugAdministration (FDA)for LSFG-NAVI. http://www.softcare-ltd.co.jp/510k_clearance.html
BostonMicromachines
ApaerosRetinalImagingSystem – Small-vessel blood flow
http://www.bostonmicromachines.com/qualitative-measures-of-small-vessel-blood-flow-.html

32. Pupillometry Technically“simple” method thatcan
be deployedeasily into avirtualreality
headset or into a smartphoneclip-on
Headsetkeepsthe eye-cameradistance constant
(without themoreexpensivetelecentric lenses thatis)
,and reducing the
error ofthe samepupil size toappear
smaller/largertodistance variations.

33. Pupillometry
Evolution of the
technique
Chopraetal.(2017)
http://dx.doi.org/10.1167/tvst.6.4.16
OCT-based
pupillometry
Murray etal.(1981)
https://doi.org/10.1016/0165-0270(81)90024-8
Video-based
pupillometry
Starket al.(1959)
https://doi.org/10.1109/JRPROC.1959.287206
Iris reflectivity-based
pupillometry
850-950 nm infrared lighting for example from LEDs is used
illuminate the eye, and the pupil boundary is computed
from the video via computer vision techniques

34. Pupillometry
Components
Temporalresolution
High-freqencybiomarkersrequire
highsampling ates
Spatialresolution
Low-amplitudebiomarkersrequire
morepixelsfromcamera Blinkandartifactsingeneral
“Realpupilsize”
Measured pupilsize
-Discretization noise (in thiscase, pupil width in pixels)
-Instrumentation noise (image noise, and uncertainty
ofboundaryalgorithm)
-Physiological noise (“focusfluctuations”, and mental
processes)
-“Pupilnoise” (“pupillaryunrest” used asaindex of
alertnessfor example, Stanten et al. 1966)

35. Pupillometry
Spatial and temporal
characteristics of
pupillary size
Pupil noise (hippus, pupillary unrest) is characterized as a random noise on top of the “mean
pupil size”-signal in the frequency range of 0.05-0.3 Hz (Stark 1959; Usui andStark 1982).
Experiments have indicated skewing of the pupil noise spectrum from Gaussian white noise at
high and low pupil areas. This skewing has been attributed the multiplier gain dependence on the
expansive range nonlinearity and length-tension relationship of iris muscles (Usui and Stark 1978;
Usui andStark1982) . In practice, pupil neuromuscular dynamics shape the high frequency cutoff
for noise pulses and large and small sinusoids, whereas retinal adaptation accounts for the low-
frequency asymptotes(Stanten andStark 1966;Stark 1984).
Usui andStark 1982
Stark1959

36. Pupillometry
Spatial and temporal
characteristics of
pupillary size
Adhikari et al. (2016)
http://dx.doi.org/10.1038/srep33373
Maynard et al. (2015)
http://dx.doi.org/10.1167/iovs.15-17357
Pupil size do not change especially fast, and the modulation frequencies
(e.g. sinusoidal light stimuli) are not very high especially for melanopsin
studies. Thus, it makes more sense to have slower sampling rates and higher
resolution cameras if onecannot get theboth for reasonable price.

37. Pupillometry
Example of “spatial”
PLR use, also beyond
retinopathies
Alzheimer’sdiseaseinthehumaneye.Clinicalteststhat identifyocularand
visualinformationprocessingdeficitasbiomarkers
ChangL.Y.L.,LoweJ.,ArdilesA.,LimJ.,GreyA.C.,Robertson K.,etal..(2014)..AlzheimersDement.10,251–261.
10.1016/j.jalz.2013.06.004
Methodfollowupofabove:InfraredVideoPupillography CoupledwithSmartPhone
LEDforMeasurementofPupillary LightReflex
LilyYu-LiChang, Jason Turuwhenua, Tian Yuan Qu, JoannaM. Black, and MonicaL. Acosta
Front Integr Neurosci. 2017;11:6.Published online 2017 Mar 7. doi: 10.3389/fnint.2017.00006
with Point GreyFireflyUSB 2.0 infrared camera(752 x480, 60fps, $275)
CorrelationsBetweenHourlyPupillometerReadingsandIntracranial Pressure
Values
McNett,Molly;Moran,Cristina; Janki,Clare;Gianakis,Anastasia,JournalofNeuroscienceNursing:August2017-
Volume49-Issue4 -p229–234.10.1097/JNN.0000000000000290
Theuse anduptake of pupillometersinthe Intensive Care Unit
MatthewHaoLee,BiswadevMitra,JiunKaePui,MarkFitzgerald
AustralianCriticalCareAvailableonline17July2017 https://doi.org/10.1016/j.aucc.2017.06.003
Useof DigitalPupillometrytoMeasure SedativeResponsetoPropofol
Haddocketal(2017)TheOchsner Journal:Fall2017,Vol.17,No.3,pp.250-253
http://www.ochsnerjournal.org/doi/abs/10.1043/1524-5012-17.3.250?code=occl-site
Infrared pupillometryhelpstodetectandpredictdeliriumin thepost-
anesthesiacare unit
EricYang,MatthiasKreuzer,SeptemberHesse, ParanDavari,SimonC.Lee,PaulS.García
JournalofClinicalMonitoringandComputing(2017), https://doi.org/10.1007/s10877-017-0009-z

38. Pupillometry
Focus on non-invasive
trauma evaluation
Highlighting also the
increased use of crowdfunding
platform for scientific uses.
Waiting for the “Scientific
ICO” rollouts still...
Dr.CharleneOng,atHarvardUniversityHospitalsMassachusetts
General and Brigham and Women’s, hopes to find a safe and
effectivemethodtocatchsignsofbrainswellingearlier.

39. Pupillometry
Example of “temporal”
PLR use, also beyond
retinopathies
Pupillary Motility: Bringing Neuroscience to the Psychiatry Clinicof the Future
SimonaGraurandGregSiegle CurrentNeurologyand NeuroscienceReports August2013, 13:365
https://doi.org/10.1007/s11910-013-0365-0
Reduced Pupillary Unrest: Autonomic NervousSystemAbnormality inDiabetesMellitus
AstradurBHreidarsson andHansJorgen GGundersenDiabetes 1988Apr; 37(4):446-451
https://doi.org/10.2337/diab.37.4.446
Fatigue and cognition: Pupillary responses to problem solving inearly multiple sclerosis‐
patients
REgg, B Högl, SGlatzl, RBeer, TBerger MultipleSclerosisJournalVolume:8 issue: 3,page(s):256-260
https://doi.org/10.1191/1352458502ms793oa
Using pupilsize and heartrate to inferaffective statesduring behavioral neurophysiology
and neuropsychology experiments
SigridA.deRodezBenaventetal.| Brain andBehavior 2017
http://dx.doi.org/10.1002/brb3.717
Pupillary unrest correlateswith arousal symptoms and motorsignsin Parkinsondisease
SamayJain etal.(2011)MovementDisorders
http://dx.doi.org/10.1002/mds.23628
Comparisonofthe antidepressantsreboxetine, fluvoxamine and amitriptyline upon
spontaneous pupillary fluctuationsin healthy humanvolunteers
M.A.Phillips,P.Bitsios,E.Szabadi,C.M.Bradshaw
Psychopharmacology March2000, Volume149, Issue 1, pp72–76
https://doi.org/10.1007/s002139900334
Assessing Pain Using the VariationCoefficient ofPupillary Diameter
DavidJ. Charieretal.(2017)TheJournalof Pain Availableonline13 July2017
https://doi.org/10.1016/j.jpain.2017.06.006

40. Pupillometry
Camera selection
in practice
33 series-GigE monochromeindustrial cameras
https://www.theimagingsource.com/products/industrial-cameras/gige-monochrome/
-Windows and Linux softwareincluded
~$730
~$330
~$439
~$495
~$730
oemcameras.com
by MicahSingleton
@MicahSingleton Feb 7,2017, 11:15amEST
https://www.theverge.com/circuitbreaker/2017/2/7/14532610/sony-smartphone-camera-sensor-1000-fps
https://www.sony.net/SonyInfo/News/Press/201702/17-013E/index.html
Ultra-HighResolutionMonochrome
Cameras
http://www.adept.net.au/cameras/ultraMono.shtml
Genie NANO XL-M51005120x 5120 px, CMOS Mono, 20fps, GigE
HS-20000M 5120 x3840 px, CMOS, 32fps,10GigE
Flare 48MP30-CX 7920x 6004px, CMOS, 30.9 fps
EoSens25CXP+ 5120x 5120 px,CMOS, 80 fps, CXP-6 CoaXPress
CP80-25-M-72 5120 x5120px, CMOS,72fps, CoaXPress
PhantomVEO4K PL forfilm 4096 x2304px,CMOS, 1000 fps
IOIndustriesFlare48MPCameraDemo
https://youtu.be/WvW9532k81M
https://youtu.be/dFdU-JjypWs

41. oemcameras.com
Pupillometry
Camera Specifications
explained

42. Pupillometry
Camera Calibration
and comparison rig
If you want to become a
PLR Powerhouse lab, you
could measure the same
subjects in clinical
settings for example with
three different quality
levels and learn quality
improvement with deep
learning?
1) Very low-cost cameraphone
2a) Entry-levelindustrialcamera
2b) High-end smartphone
3) High-end industrialcamera
See similar idea for
training adeep learning
network for3D reconstruction
from indoor scans
Slide 10 of
https://www.slideshare.net/PetteriTeikari
PhD/dataset-creation-for-deep-learningbas
ed-geometric-computer-vision-problems
An exampleof “path multiplication” with a
beamsplitter fora slit lamp setup. This allows
simultaneous exam by the clinician andrecording
of theexam

43. Pupillometry
Commercial
landscape
Established:NeurOptics
NPi®-200Pupillometer
athttps://www.neurocriticalcare.org/
Open-source: PupilLabs eyetracker
withHoloLens,HTCViveandOculusadd-onsavailable
-https://doi.org/10.1145/2638728.2641695
Emerging: “ThePUPIL
Study”-Automated,
QuantitativePupil Assessment
Using BinocularOCT
Sponsor: UniversityCollege,
London (PearseKeane)
https://clinicaltrials.gov/ct2/
show/NCT03081468
Emerging:
“PupilScreen: Using
Smartphonesto
AssessTraumaticBrain
Injury. Mariakisetal.
(2017)
http://doi.org/10.1145/3
131896
Emerging: ”BrightLamp use
thecamera and torch of your
phoneto measureyour pupil's
responseto lightstimulusto
diagnoseconcussion.”
www.brightlamp.org
techcrunch.com/2017/05/16
“Wehaveatotalof8
peopleontheteam:2
computervision
specialists,regulatory,
financial,legal,app
dev,marketing,
ML/diagnostics.Allof
whichhavevery
specificrolesin
growingastartup.”
reddit.com

44. Eye
Movements
Similar “end-to-end”
pipeline as for
pupillometry (and
even the same
hardware)
One can “easily”
integrate various
measures to a single
VR headset
https://github.com/pupil-labs/hmd-eyes
Orloskyetal.(2017) https://doi.org/10.1109/TVCG.2017.2657018

45. Eye
Movements
Even Oculus joins
the Open Source
development kit
Theopensourcereleaseofthe
RiftDevelopmentKit2(DK2).
hardwarefollowsonfromourearlier
releasesof RiftDK1 and
LatencyTester.Thisincludes
schematics,boardlayout,
mechanicalCAD,artwork,and
specificationsunder aCreative
CommonsAttribution4.0license,
andfirmwareunderBSD+PATENT
licences.Wepresentaguidedtour
ofDK2for thoseinterestedin
digging indeeper.
https://developer.oculus.com/blog/open-source-release-of-rift-dk2/

46. Eye
Movements
and VR
http://www.tomshardware.com/news/virtual-reality-locomotion-oculus-connect-vr,35699.html

47. Eye
Movements
Useful in virtual
reality environments
in general
Allows foveated rendering
(LoD, level-of-detail),
which again reduces
computation requirements
for convincing VR
rendering
Foveated rendering is a process that combines eye tracking
and software to adjust the way a VR experience is rendered in
realtime. Withfoveatedrendering, thePC runningyourVivewith
7invensun eye tracker only has to render the greatest detail in
thesmallareaon whichyoureyesaredirectlyfocused.
https://uploadvr.com/7invensun-eye-tracker-for-vive/
Games like this don’t just look incredible because of ‘hyper-realism’ but because
their engineers use all sorts of tricks [LOD’ing, or Level of Detail; Mipmapping;
frustumculling,etc.] to savememory.
https://kotaku.com/horizon-zero-dawn-uses-all-sorts-of-clever-tricks-to-lo-1
794385026

48. Eye
Movements
Beyond retinopathies
http://doi.org/10.1002/mds.27105
Many high-prevalence neurological disorders involve
dysfunctions of oculomotor control and attention,
including attention deficit hyperactivity disorder
(ADHD), fetal alcohol spectrum disorder (FASD), and
Parkinson’sdisease(PD).
https://doi.org/10.1007/s00415-012-6631-2
https://doi.org/10.1016/j.bpsc.2016.12.009
https://doi.org/10.1016/j.cobeha.2016.03.008
https://doi.org/10.1016/j.bandc.2008.08.026

49. Eye
Movements
Commercial landscape
Unlockingthe potential of eye trackingtechnology
Posted Feb 19,2017 by BenDickson (@bendee983)
https://techcrunch.com/2017/02/19/unlocking-the-potential-of-eye-tracking-technology/
A newbrainhealthappfrom
Neurotrackwarns users of
memory decline
Posted Dec1,2016 by LoraKolodny (@lorakolodny)
https://techcrunch.com/2016/12/01/neurotrack-takes-brain-scans-home/
https://qz.com/604397/a-simple-five-minute-test-could-make-earlier-diagno
sis-of-alzheimers-possible/
Oculusacquireseye-tracking
startup TheEye Tribe
28Dec2016- Thestartuphasdevelopeda$99 eye tracking
devicedeveloper kits
https://techcrunch.com/2016/12/28/the-eye-tribe-oculus/
The iPhone 8 Could Soon
Usher inEye-TrackingAds
Sep5th,2017
https://www.macobserver.com/columns-opinions
/editorial/iphone-8-eye-tracking-ads/
Better drugs could also lead to “better quality of life for the patient,
which would ultimately lower health care costs,” says Samir Kaul, the
founding managing director of Khosla Ventures. Khosla is the lead
investor for Neurotrack’s $6.5 million in new funding,
announced Jan. 27. If it passes muster in ongoing studies, Neurotrack’s
online test could be a cheap, easy, non-invasive way to detect
Alzheimer’s in advance of symptoms.
The Palo Alto, California-based company is inviting physicians to offer
the eye-tracking test to their patients. It’s also developing a personalized
lifestyle program for prospective patients. The program will be based on
emerging research in Finland (FINGER) and elsewhere suggesting that
diet, exercise, cognitive training, sleep and stress management could
preserve brain health and help prevent Alzheimer’s.

50. Electro-
physiology
Electroretinography
(ERG) and electro-
oculogram (EOG)
The systems used to be
hard to use with long dark
adaptations and long setup
times.
Recently new devices such
as EvokeDx and RETEval
have made things easier in
practice
http://konanmedical.com/evokedx/
https://youtu.be/aT6dCD_5p5k
Multifocal ERG
http://vsri.ucdavis.edu/research/electrophysiology
Wearable electrooculography (EOG) goggles. The
Swiss Federal Institute of Technology (ETH) Zurich
developed these goggles to track relative eye
movements.
https://doi.org/10.1109/MPRV.2010.86

51. Electro-
physiology
Correlating function
with structure
Documenta Ophthalmologica April 2017, Volume134, Issue 2, pp 111–128|
Comparingthree differentmodesof
electroretinographyinexperimental
glaucoma: diagnosticperformance and
correlationtostructure
Laura Wilsey,Sowjanya Gowrisankaran,Grant Cull,
Christy Hardin.Claude F. Burgoyne,Brad Fortune
https://doi.org/10.1007/s10633-017-9578-x
ArqBrasOftalmol. 2017Mar-Apr;80(2):118-121.
doi:10.5935/0004-2749.20170028
Structure-functionalcorrelation
usingadaptiveoptics,OCT,and
microperimetryinacaseofoccult
macular dystrophy.
VianaKÍ, MessiasA, SiqueiraRC, RodriguesMW, Jorge R
Optical coherencetomography(1-3), adaptiveoptics(A-H), and
microperimetry (2-4) imagesof a patientwith occultmaculardystrophy.
(Arrows) Lossof continuityof theouterphotoreceptorlayerin thecentral
fovealregion.(yellowasterisk)Reduced ring photoreceptor densityin the
fovealregion.(2) Reductionof ringsensitivityin thecentral foveal region.
(Red Asterisk)Reduced photoreceptor densityin acentralfoveal region.(4)
Reduced central sensitivityinthefovea.

52. Electro-
physiology
Clinical Uses http://dx.doi.org/10.1007/BF01206208
https://doi.org/10.1159/000450958https://doi.org/10.1016/j.taap.2015.10.008
https://doi.org/10.1016/j.pharmthera.2017.02.009
http://dx.doi.org/10.3233/JAD-150798

53. Electro-
physiology
Relatively simple to
develop and integrate
to the same VR-type
of headset used for
pupillometry and eye
movement measurement
A filter setting of 1–200 Hz appears most sensitive to
detect glaucomatous damage if using a two-global-
flash mfERG: High frequencies of 100–300 Hz also
contain information that differentiates glaucoma from
normal and thus should be included in the analysis.
A 50 Hz notch filter allows grossly contaminated
waveforms to be analyzed in a meaningful manner.
With a 50 Hz filter, glaucoma patients still differed
significantly from normal.

54. Visual Evoked
Potentials
Essentially a EEG
headset with reduced
electrode count and
active dry electrodes
ThenGoggle,aPortableBrain-Computer Interfacefor AssessmentofVisualFunctionandglaucoma
diagnosis–Nakanishietal.(2017) http://dx.doi.org/10.1001/jamaophthalmol.2017.0738
REINVENT:Alow-cost,virtual
realitybrain-computerinterface
forseverestrokeupperlimb
motorrecovery
Spiceret al. (2017)
https://doi.org/10.1109/VR.2017.7892338
AVirtual-RealityBased
NeurofeedbackGame
Frameworkfor Depression
RehabilitationusingPervasive
Three-ElectrodeEEGCollector
Caietal. (2017)
https://doi.org/10.1145/3127404.3127433
HTC Vive Modified With Neurable
ReadsYourMind At SIGGRAPH
https://youtu.be/47WHqDNckI8
https://www.extremetech.com/extreme/254816-eeg-
virtual-reality-matrix-just-around-corner
A feasibilitystudyonSSVEP-basedinteractionwith
motivatingandimmersivevirtual andaugmented reality
Josef Faller, Brendan Z. Allison, ClemensBrunner, ReinholdScherer, Dieter Schmalstieg, GertPfurtscheller, ChristaNeuper
(Submitted on15Jan 2017)
https://arxiv.org/abs/1701.03981

55. Magneto-
retinography
With Diamond
Magnetometry
(instead of costly SQUID
magnetometers used in
magnetoencephalography
[MEG] for example).
At some point eventually allowing
non-contact electrical
measurements when the price goes
even well below $150k per
instrument?
Today, Matthew Dale and Gavin
Morley at the University of
Warwick in the U.K. say that
diamond sensors are poised to
revolutionize the way physicians
use magnetic field measurements
in diagnostic medicine. They map
out the state of the art in this area
and say that the business
opportunityissignificant.
https://arxiv.org/abs/1705.01994
https://www.technologyreview.com/s/607871/how-diamond-sensors-are-set-to-revolutionize-medical-diagnostics/
There are around 100 SQUID MEG systems installed worldwide, at a cost of over $1M each. The MCG
market should be much larger if the instrumentation was affordable and portable, because MCG has been
shown to be superior to ECG and hence other non-invasive approaches for the diagnosis of coronary
artery disease (CAD) [Kwongetal.2013, Fenici et al. 2005 and 2013]. CAD is the most common type of heart
diseaseandistheleadingcauseofdeathintheUnitedStatesinbothmenandwomen.
Several companies have tried and failed to commercialize SQUID-based MCG, held back by the cost
of a cryogen-based system. We estimate that 100,000 MCG systems could be sold if the functionality
were the same as existing SQUID systems and the price was below $150k. This is based on there being
over 100,000hospitalsinChina,India,theEU,JapanandtheUSA.
Diamond magnetometers are at technology readiness level (TRL) 7: the technology has been
demonstrated and is moving towards being put on sale. However, this has not yet reached the sensitivity
neededfor MCG,soanMCGsystembasedondiamondisatTRL4-5(technologydevelopment).

56. Visual Fields
Very common measurement
even though it can be
stressful for the
patient with high noise
in this psychophysical
measurement (reduced by
using log units)
Similarly, visual field
measurement fail to
detect early changes in
glaucoma as the brain
can compensate for the
neurodegeneration of
RGCs
ARVO 2017 poster for deep
learning in visual field
assessment
2846—B0449Adeep-learningbased
automaticglaucomaidentification.
Serife SedaS. Kucur1
, M. Abegg2
, S. Wolf2
, R. Sznitman1
.
1
ARTORGCenter, Universityof Bern, Bern, Switzerland;
2
DepartmentofOpthalmology, Inselspital Bern, Bern, Switzerland https://doi.org/10.1016/j.ophtha.2017.06.028
VisualFieldTestingwithHead-
MountedPerimeter‘imo’
Matsumoto et al. (2016)
https://doi.org/10.1371/journal.pone.0161974
The perimeter imo has completely isolated
optical systems for the right and left eyes.
Stimulus presentation is also independently
performedfor eacheye.
Visualfieldexaminationmethodusing
virtual realityglassescomparedwith
the Humphrey perimeter
Tsapakis et al. (2017)
doi: 10.2147/OPTH.S131160
Effect ofcognitive demand on
functionalvisualfield performancein
seniordriverswithglaucoma
Gangeddula, Viswa, et al (2017)
https://doi.org/10.3389/fnagi.2017.00286

57. Visual
Function
Towards assessing real-
life function,
simultaneously for
diagnosis and disease
progression purposes
Using virtual reality to cause a
subject to correct for perceived
motion has revealed that
glaucoma patients’ reactions are
more erratic than those of healthy
individuals
Diniz-Filho,etal.(2015)
doi: 10.1016/j.ophtha.2015.02.010
ChristopherKent,Senior EditorPUBLISHED 6JULY2015
Virtual Reality:A NewFrontierinEye
Care?
https://www.reviewofophthalmology.com/article/virtual-reality-a-new-frontier-in-eye-care
Daga,Fábio B., et al. "Wayfinding and Glaucoma: A Virtual
Reality Experiment." InvestigativeOphthalmology&Visual
Science58.9(2017):3343-3349. doi: 10.1167/iovs.17-21849
The VEHuNT consisted of a cave automatic virtual environment (CAVE) used to present an
immersiveVR environmentto studywayfinding tasks.
Usingvirtual/augmentedrealitytosimulate
visualimpairments
ByDr.PeteRJones
http://www.ucl.ac.uk/~smgxprj/projects.html

58. Vision
Disorders
Myopia, amblyopia,
etc.
Virtual reality is still far from beinga mainstream technology. But when
Facebook bought VR headset-makerOculus for $2 billion last year, itsignaled to the
world thatvirtual reality was no longer sci-fi, kicking off a frenzy ofexperimentation.
For James Blaha, who’s struggled all his lifewith strabismus (typeofamblyopia)—a
visioncondition more commonlyknown as crossedeyes—virtual realityoffereda
potential cure, and he’s builta venture-backed company See Vividly (Vivid Vision
software) based onthis promise.
https://qz.com/489048/an-entrepreneur-is-using-virtual-reality-headsets-to-try-to-c
ure-vision-disorders/
TheCure/Diagnosis and TheCause?
http://dx.doi.org/10.3109/02713683.2016.1158271
https://endmyopia.org/virtual-reality-the-next-myopia-tsunami/
https://essilorusa.com/newsroom/virtual-reality-bad-fo-the-eye

59. Gamification
of diagnosis
and Service Design
for Eyecare.
Patients can be given the
headsets for the waiting room
and make their waiting time
less boring and make the
“process” more efficient
Similarly it might be hard to
get children to be attentive.
The gamification might help the
children to focus
Amblyopia treatmentof adultswith
dichoptic training usingthe virtual
realityOculusRifthead mounted
display:preliminary results
Peter Žiak,Anders Holm, Juraj Halička, Peter Mojžišand
David P Piñero BMCOphthalmology201717:105
https://doi.org/10.1186/s12886-017-0501-8

60. Other Clinical
Eye Measures

61. Intraocular
Pressure
Boucard et al. 2016: “The classic view of
glaucomab
is that of an eye disease in
which elevated intraocular pressure (IOP)
mechanically damages the optic nerve (ON)
causing the death of retinal ganglion cells
(RGCs). Indeed, in high-pressure glaucoma
(HPG, the most common form of
glaucoma), RGC and ON damage are
associated with an elevated IOP
(>21 mmHg).[1]
However, this view cannot
be complete as glaucoma with normal
levels of IOP is commonly reported as well.
In such normal-pressure glaucoma (NPG),
damage occurs to the ON without the eye
pressure exceeding the normal range. By
definition, NPG only differs from HPG in
that the IOP is consistently below 22
mmHG.[1]
Moreover, rather than being a
disease restricted to the eye, damage of
the RCGs extends to the axons that form
the primary visual pathways.c
The patient wears the SENSIMED Triggerfish® system up to 24 hours
and assumes normal activities including sleep periods. The SENSIMED
Triggerfish® Sensor is a soft disposable silicone contact lens
embedding a micro-sensor that captures spontaneous circumferential
changes at the corneoscleral area.
http://www.sensimed.ch/
Journal ofGlaucoma. August 22, 2016.
doi: 10.1097/IJG.0000000000000517
The range of IOP fluctuation was larger in the eyes with
normal-tension glaucoma (NTG) than in the nonglaucoma
eyes. This larger fluctuation might be one of the reasons
underlying the aggravation of the visual field by NTG.
Measurements of 24-hour continuous IOP might be one of
the useful methods to distinguish NTG from nonglaucoma
eyes.
Daily variation of intraocular pressure
Measurement once in 6 months might
not capture all relevant information
IOP Transient
Cyclical strain vs. constant strain
(i.e. punching your eye with fist vs. applying
constant pressure over longer time)
IOP variability by CrawfordDowns atWorld Glaucoma Conference2017
HelsinkiFinland, at“Newfrontiersinglaucoma”session,Saturday July1, 2017
https://youtu.be/1nrV3zztisk |https://youtu.be/QNDzq5Rp5RA
IOP vs. CSFO
Intraocular pressure, cerebrospinal pressure
or trans lamina pressure difference or what?
http://dx.doi.org/10.1016/j.preteyeres.2015.01.002

62. Intraocular
Pressure
Already devices with
both FDA and CE
approvals for clinical
use
German medical device company Implandata has received CE Marking for EyeMate, the first approved IOP-
monitoringsystemtooffer24-hourpressuremeasurementsinpatientswithprimary open-angleglaucoma(POAG).
The sensor consists of eight pressure-sensitive capacitors and a circular microcoil antenna, which is coupled to
an external handheld unit that displays readings to the patient and sends real-time data to the physician over the internet.
An associated smartphoneappcan beusedto displayIOPhistoryandsetmedication alerts.
https://www.aao.org/headline/continuous-iop-monitoring-implant-approved-in-euro

63. OMICs
Genomics,
proteomics,
metabolomics,
lipidomics, etc
Progress in Retinal and EyeResearch Volume 58, May2017, Pages 89-114:15.
Characterizing the “POAGome”: A
bioinformatics-drivenapproach to
primary open-angle glaucoma
Danford etal.(2017) https://doi.org/10.1016/j.preteyeres.2017.02.001
SciRep. 2017;7:41595.https://dx.doi.org/10.1038/srep41595
AnOcular Protein Triad Can Classify
FourComplex Retinal Diseases
Kuiperetal. (2017)
In the era where molecular assessment has
improved dramatically, we aimed at the
identification of biomarkers in 175 ocular
fluids to classify four archetypical ocular
conditions affecting the retina (age-related
macular degeneration, idiopathic non-
infectious uveitis, primary vitreoretinal
lymphoma, and rhegmatogenous retinal
detachment) with one single test.
IOVS July 2017, Vol.58, BIO88-BIO98.
Omics Biomarkers in Ophthalmology
SusetteLauwen; Eiko K.deJong; Dirk J. Lefeber; AnnekeI. den Hollander
https://doi.org/10.1167/iovs.17-21809
Here, we review the application of omics
techniques in eye diseases, focusing on age-
related macular degeneration (AMD), diabetic
retinopathy (DR), retinal detachment (RD),
myopia, glaucoma, Fuchs' corneal dystrophy (FCD),
cataract, keratoconus, and dry eyes. We observe
that genomic analyses were mainly successful in
AMD research (almost half of the genomic
heritability has been explained), whereas large
parts of disease variability or risk remain
unsolved in most of the other diseases.
Expert Review ofOphthalmology Volume 11, 2016 - Issue 2
GWAS in myopia: insightsinto disease
and implicationsforthe clinic
KatieM Williams &ChristopherJ Hammond
Departmentof Ophthalmology, King’s College London, London, UK;Departmentof Twin Research &Genetic
Epidemiology, King’s College London, London, UK
In this review we focus on what a genome-wide
association study involves, what studies have
been performed in relation to myopia to date,
and what they ultimately tell us about myopia
variance and functional pathways leading to
pathogenesis. The current limitations of
genome-wide association studies are reviewed
and potential means to improve our
understanding of the genetic factors for
myopia are described.
Overview of the different layers within a biological system contributing to
multifactorial diseases and their relation to each other. For each layer, the name
of thecorresponding omicstechniqueisindicated in blueboxes.

64. Unimodal
Model
Many proof-of-concept
(PoC) published
showing that deep
learning can replace
humans for low-level
unimodal tasks such
as image analysis

65. Unimodal
Model
The typical ophthalmologic
approach to tackle
diagnostics.
Doctors are eager in finding
new scalar measures that can
quantify the pathology the best
As much as it would be nice to
have scalar variables and
simple decision trees, it might
not be realistic way to model
complex pathogenesis
Examples:
OCT BMO-MRW (Kabbara et al. 2017):
“To compare the cube and radial scan patterns of the spectral domain optical coherence
tomography (SD-OCT) for quantifying the Bruch's membrane opening minimum
rim width (BMO-MRW). The BMO-MRW diagnostic accuracy for glaucoma
detectionandratesofchangederivedfromthetwoscanpatternswerecompared.”
IOP (Chan et al. 2017):
“A UK study of 8623 Norfolk residents has found that the use of intraocular pressure (IOP)
to detect glaucoma is ‘inaccurate and probably not viable’ … The researchers also report
that no single IOP threshold provided adequate sensitivity and specificity for diagnosis of
glaucoma. … ‘The evidence around the performance of IOP for either screening or case-
findingisnot strong,’PaulFosteremphasised.

66. Unimodal
Diagnostics
JAMA. 2016;316(22):
2402-2410. doi:
10.1001/jama.2016.17216
One million anonymised eye scans from
Moorfields Eye Hospital will be used to
train an artificial intelligence (AI)
system from Google DeepMind.
A spokesperson for DeepMind
Health told Business Insider:
"DeepMind has reimbursed Moorfields for
the direct costs they have incurred de-
personalising and manually segmenting eye
scans prior to transfer … "We hope this work
will help doctors faster analyse the 3,000
eye scans Moorfields carries out every
week.”

67. Unimodal
Disease
Management
Investigative Ophthalmology & Visual Science June 2017, Vol.58, BIO141-BIO150.
doi:10.1167/iovs.17-21789

68. Unimodal
Treatment
“Personalized
precision” medicine
doi: 10.1016/j.preteyeres.2015.07.007

69. Multimodal
Model
Going beyond human
capabilities, the
next generation
models will be
incorporating “full
clinical knowledge”
of the patient

70. Multimodal
Model
Traditional numerical
methods for “small
data” problems might
not be enough for the
emergence of “network
medicine”
HollyF.Ainsworthetal.(2017):
The use of causal inference techniques to integrate omics and GWAS data has the potential
to improve biological understanding of the pathways leading to disease. Our study
demonstrates the suitability of various methods for performing causal inference under
several biologically plausible scenarios
EwenCallaway(2017):
“Biologists are likely to find that larger studies turn up more and more genetic variants
– or “hits” - that have minuscule influences on disease” - Jonathan Pritchard, Stanford
University
matrix.

71. Multimodal
Model
Personalized
precision medicine
based on multiple
measures with EHR
mining.
Example for diagnostics
Example for management
Example for treatment
In 2015, aresearch groupatMountSinai Hospitalin New York wasinspiredtoapplydeeplearning to
the hospital’s vast database of patient records. This data set features hundreds of variables on
patients, drawn from their test results, doctor visits, and so on. The resulting program, which the
researchers named Deep Patient, was trained using data from about 700,000 individuals, and
whentestedonnewrecords,itprovedincrediblygoodatpredictingdisease.
https://www.technologyreview.com/s/604087/the-dark-secret-at-the-heart-of-ai/
http://doi.org/10.1038/srep26094 https://github.com/greenelab/deep-review/issues/63
https://syncedreview.com/2017/02/26/deep-pa
tient-improving-prognosis-with-electronic-h
ealth-records-by-deep-learning/ on-demand.gputechconf.com

72. Multimodal
Model
Personalized
precision medicine
based on multiple
measures with EHR
mining.
Example for diagnostics
Example for management
Example for treatment
Thefuture ofhealth diagnostics. Currentdiagnosticsarebased on a “snapshot”in timeand limited datapoints.Inthefuture, largedatasetsacquired over
timethroughconstantmonitoring will beanalyzed to establishbaselinesand trends,enabling preventativeinterventions. Partof thisfigurereusesa drawing
previouslypublished in Swedish etal,2015.Copyright©2017ACM, Inc. Adapted fromSwedish T,Roesch K,LeeIK, Rastogi K, BernsteinS, RaskarR. EyeSelfie:
Self directed eyealignmentusing reciprocaleyeboximaging. ACM TransGraph. 2015;34(4):58.39
KarinRoesch, TristanSwedish, and RameshRaskar (2017): “Automated retinalimaging andtrend analysis– atool forhealthmonitoring”
https://dx.doi.org/10.2147/OPTH.S116265

73. Multimodal
Model
Personalized
precision medicine
based on multiple
measures with EHR
mining.
Example for diagnostics
Example for management
Example for treatment
Classificationofadvanced
stagesofParkinson’s
disease:translation into
stratifiedtreatments
JournalofNeuralTransmission August
2017, Volume124, Issue 8, pp1015–1027
RejkoKrüger, Jochen Klucken, Daniel Weiss, Lars Tönges, Pierre Kolber,
Stefan Unterecker, Michael Lorrain, Horst Baas,Thomas Müller, Peter
Riederer https://doi.org/10.1007/s00702-017-1707-x
PrecisionMedicineinPediatric
Oncology:Translating
GenomicDiscoveriesinto
OptimizedTherapies
American Association forCancerResearch June9,2017
ThaiHoaTran, AvanthiTayiShah and Mignon L. Loh
https://doi.org/10.1158/1078-0432.CCR-16-0115
Relative frequency of genomic alterations in neuroblastoma at
diagnosiscompared with relapse

74. Usability
Make it for the
clinician,
technician, eye-
selfie taker the most
easiest to use

75. Image
Management
Integrate deep
learning algorithms
to existing image
management software
The easiest for the
end-user KideSystems– Optoflow
https://www.kidesystems.com/optoflow/
DigisightPaxos
https://www.digisight.net/ds/
Breaking Down Silos AcrossSpecialties
Singapore National EyeCentre advanced itstechnology
tocapture all images and fullyintegrate withitsEMR.
http://go.merge.com/2017-Q1-OC-CS-Singapore-National-Eye-Centre_LP-
Singapore-CS.html?utm_source=CS_Singapore
https://www.aao.org/eyenet/article/image-manageme
nt-systems-what-you-need-to-know

76. Interpretability
Visualize what part
of the image, 1D
signal, the whole
“disease network” is
causing the deep
learning system to
flag up the patient
in risk
July–August,2017 Volume1,Issue4,Pages 322–327
CeciliaS.Lee,MD, DougM.Baughman,BS, Aaron Y.Lee,MD,MSCI
DepartmentofOphthalmology, University ofWashingtonSchoolofMedicine,Seattle,
Washington.http://dx.doi.org/10.1016/j.oret.2016.12.009
Anocclusiontest (ZeilerandFergus,2016) wasperformedtoidentifythe areas
contributing most to the neural network's assigning the category of AMD. Ablank
20 × 20-pixel box was systematically moved across every possible position in
theimageandtheprobabilitieswere recorded.The highestdropintheprobability
represents the region of interest that contributed the highest importance to the
deeplearningalgorithm.
Examples of identification of pathology by
the deep learning algorithm. Optical
coherence tomography images showing
age-related macular degeneration
(AMD) pathology (A, B, C) are used as
input images, and hotspots (D, E, F) are
identified using an occlusion test from the
deep learning algorithm. The intensity of
the color is determined by the drop in the
probability of being labeled AMD when
occluded.

77. Interpretability
Not just medical
algorithms benefit
from opening the
“black box”
Last month, a YouTubevideo of a conference talk in Berlin,
shared widely among artificial-intelligence researchers, offered
a possible answer. In the talk, NaftaliTishby, a computer
scientist and neuroscientist from the Hebrew University of
Jerusalem, presented evidence in support of a new theory
explaining how deep learning works. Tishby argues that deep
neural networks learn according to a procedure called the
“information bottleneck,” which he and two collaborators
firstdescribedinpurelytheoreticaltermsin1999.
The idea is that a network rids noisy input data of extraneous
details as if by squeezing the information through a bottleneck,
retaining only the features most relevant to general concepts.
Striking new computer experiments by Tishby and his student
Ravid Shwartz-Ziv reveal how this squeezing procedure
happensduringdeeplearning,atleastinthecasestheystudied.
https://youtu.be/bLqJHjXihK8

78. Extra
Resources
Shallow introduction for Deep Learning Retinal Image Analysis
https://www.slideshare.net/PetteriTeikariPhD/shallow-introduction-f
or-deep-learning-retinal-image-analysis
Shallow introduction for Deep Learning Retinal Image Analysis
https://www.slideshare.net/PetteriTeikariPhD/artificial-intelligence-
in-ophthalmology
Shallow introduction for Deep Learning Retinal Image Analysis
https://www.slideshare.net/PetteriTeikariPhD/datadriven-ophthalmol
ogy
Shallow introduction for Deep Learning Retinal Image Analysis
https://www.slideshare.net/PetteriTeikariPhD/understanding-the-inve
stors-medical-ai-startups