https://telecombcn-dl.github.io/2017-dlcv/ Deep learning technologies are at the core of the current revolution in artificial intelligence for multimedia data analysis. The convergence of large-scale annotated datasets and affordable GPU hardware has allowed the training of neural networks for data analysis tasks which were previously addressed with hand-crafted features. Architectures such as convolutional neural networks, recurrent neural networks and Q-nets for reinforcement learning have shaped a brand new scenario in signal processing. This course will cover the basic principles and applications of deep learning to computer vision problems, such as image classification, object detection or image captioning.

1. Day 5
Closing,
Course offer 17/18
& Homework
#DLUPC
[course site]

2. 2
Thank you
Eduard
Ramon
Marta
Coll
Fran
Roldan

3. 3

4. 4
Deep learning opportunities at UPC TelecomBCN
during 2017/2018 year:
Master
MET
BSc
Deep Learning (5 ECTS)
Autumn Semester 2017 Spring Semester 2018
Deep Learning for Speech,
Audio & Language (2.5 ECTS)
Intro to Deep Learning (2 ECTS)
Deep Learning for
Computer Vision (2.5 ECTS)
Introduction to Research (5,10,15 ECTS)
Reading Groups on AI & Biomedical Imaging (2.5 ECTS)
Bachelor Thesis (12, 24 ECTS)
Master Thesis (30 ECTS)

5. 5
Deep learning opportunities during 2017/2018 year:
Learn more @ ETSETB TelecomBCN
Master
MIRI,
Industry,
Visitors...
Deep Learning (5 ECTS)
Autumn Semester 2017 Spring Semester 2018
Deep Learning for Speech,
Audio & Language (2.5 ECTS)
Intro to Deep Learning (2 ECTS)
Deep Learning for
Computer Vision (2.5 ECTS)

6. 6
Deep learning opportunities during 2017/2018 year:
Learn more @ ETSETB TelecomBCN
Master
MET
BSc
Autumn Semester 2017 Spring Semester 2018
Reading Groups on AI & Biomedical Imaging (2.5 ECTS)

7. 7
● Reading & discussion group (DLMI)
● E-mail to veronica.vilaplana@upc.edu if you want to join
BSc, MSc & Phd on biomedical imaging applications
Learn more @ ETSETB TelecomBCN

8. 8
● Reading Group with public listing of videos, slides and papers.
● E-mail to xavier.giro@upc.edu if you want to join in Autumn 2017.
Learn more @ ETSETB TelecomBCN

9. 9
Deep learning specific courses during 2017/2018 year:
Learn more @ UPC TelecomBCN
Master
MET
BSc
Autumn Semester 2017 Spring Semester 2018
Introduction to Research (5,10,15 ECTS)
Bachelor Thesis
Master Thesis

10. Vision (GPI-UPC) Speech & Language (TALP-UPC)
Xavier
Giró
Elisa
Sayrol
Verónica
Vilaplana
Ramon
Morros
Javier
Ruiz
Marta
Ruiz
Costa-
jussà
Antonio
Bonfaonte
Javier
Hernando
José Adrián
Rodríguez
Fonollosa
https://imatge.upc.edu http://www.talp.upc.edu/

11. 11
Learn more @ GPI UPC
11 Professors / Associate Professors
7 Phd students
2 Technical support
https://imatge.upc.edu/web/

12. 12
Visual Reasoning
Learn more @ GPI UPC
Johnson, Justin, Bharath Hariharan, Laurens van der Maaten, Li Fei-Fei, C. Lawrence Zitnick, and Ross
Girshick. "CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoning."
CVPR 2017

13. 13
Gaze Scanpath for Saliency Prediction (2D & 360o
images)
Learn more @ GPI UPC
Output
Saliency Volume
Scan-paths
Conv
Max Pooling
Upsampling
Sigmoid
Sampling

14. 14
Learn more @ GPI UPC
X. Lin, Campos, V., Giró-i-Nieto, X., Torres, J., and Canton-Ferrer, C., “Disentangling Motion, Foreground
and Background Features in Videos”, in CVPR 2017 Workshop Brave New Motion Representations
kernel dec
C3D
Foreground
Motion
First
Foreground
Background
Fg
Dec
Bg
Dec
Fg
Dec
Reconstruction
of foreground in
last frame
Reconstruction
of foreground in
first frame
Reconstruction
of background
in first frame
uNLC
Mask
Block
gradients
Last
foreground
Kernels
share
weights

15. 15
Image synthesis
with Generative
Adversarial Networks.
Learn more @ GPI UPC
Shrivastava, Ashish, Tomas Pfister, Oncel Tuzel, Josh Susskind, Wenda Wang, and Russ Webb.
"Learning from simulated and unsupervised images through adversarial training." arXiv preprint
arXiv:1612.07828 (2016).

16. 16
Wearables, Lifelogging & Egocentric Vision
Learn more @ GPI UPC

17. 17
Affective Computing
CNN
Learn more @ GPI UPC

18. 18
Medical Imaging Segmentation
Learn more @ GPI UPC
Brain tumor
Infant brain (WM/ GM/ CSF)
White matter hyperintensitiesLiver tumor

19. 19
Generative adversarial networks in medical imaging
Synthesis
Super-resolution
Learn more @ GPI UPC
CT from MRI
High resolution 3D cardiac MRI

20. 20
Alzheimer’s Disease: prediction of preclinical AD
(collaboration with Pasqual Maragall Foundation)
Histological tissue: Classification / Feature extraction
(collaboration with Centre for Genomic Regulation)
Learn more @ GPI UPC
Associate tissue samples with histological and
pathological phenotypes

21. 21
Learn more @ GPI UPC
Multimodal People Recognition
Incremental Learning

22. 22
Learn more @ GPI UPC
“Dancing” with Deep Learning”, generating choreographies,
using LSTM and Mixture Density Models)
...our skeleton is still
working

23. 23
3D point cloud analysis
SEMANTIC SEGMENTATION
SUPER-RESOLUTION
Learn more @ GPI UPC

24. 24
Learn more @ Insight DCU

25. 25
BSc &
Master
MET
BSc
Data Analysis and Machine
Learning (7.5 ECTS)
Autumn Semester 2017 Spring Semester 2018
Bachelor Thesis
Master Thesis
Learn more @ Insight DCU
Master
MET

26. Dublin City UniversityInsight Centre for Data Analytics 26
Learn more @ Insight DCU
Kevin McGuinness Noel O’Connor Cathal Gurrin Alan E. Smeaton

27. 27
Learn more @ Insight DCU

28. ● Multi-task deep learning
● Learning generic representations
● Unsupervised and semi-supervised feature learning
● Visual attention models and applications
● Image segmentation
● Interactive computer vision
● Multimedia recommender systems (hybrid content
based and collaborative)
● Deep video analysis (tagging, genres, actions)
● Generative adversarial networks
● Model update and lifelong learning
28
Learn more @ Insight DCU

29. Some applications:
● Image and video retrieval
● Medical imaging and computer aided diagnosis
● Lifelogging
● Autonomous vehicles
● Crowd scene analysis
● Brand and logo detection
● Photo OCR
29
Learn more @ Insight DCU

30. 30
Master in Computer Vision (one and two-year tracks).
Opportunities @ UPC+UAB+UPF+UOC

31. 31
Learn more
Grup d’estudi de machine learning Barcelona

32. Learn more online
32[course site] [course site]
Self-paced online learning (video & slides available).
[course site]

33. 33
Deep Learning labs with TensorFlow and Keras by Amaia Salvador & Santi Pascual.
Learn more online

34. 34
http://cs231n.stanford.edu/
Learn more online

35. 35
http://cs224n.stanford.edu/
Learn more online

36. 36
Learn more online
Electronic Frontier Foundation: Measuring the Progress of AI Research

37. 37
Learn more online

38. 38
Learn more online
@DocXavi
@kevinmcguinness
@amaiasalvador
@ElisaSayrol
@ramonmorros_upc
@Javier_RuizHida
#DLUPC

39. 39
Learn more online
“Machine learning” sub-Reddit.

40. 40
Learn more online

41. 41
Computer vision is (finally) taking off...
...because machines have learned to see.
Learning only to see ?

42. Learning only to see ?
Nexi, del MIT Media Lab (Foto: Spencer Lowel)
42

43. Video games
Learning only to see ?
Mnih, Volodymyr, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, and Martin Riedmiller.
"Playing atari with deep reinforcement learning." arXiv preprint arXiv:1312.5602 (2013).
43

44. Human games
Learning only to see ?
44
Silver, D., Huang, A., Maddison, C.J., Guez, A., Sifre, L., Van Den Driessche, G., Schrittwieser, J., Antonoglou, I.,
Panneershelvam, V., Lanctot, M. and Dieleman, S., 2016. Mastering the game of Go with deep neural networks and tree
search. Nature, 529(7587), pp.484-489

45. Autonomous Driving
Google
Self-driving car
Learning only to see ?
45

46. Elgammal, Ahmed, Bingchen Liu, Mohamed Elhoseiny, and Marian Mazzone. "CAN: Creative Adversarial Networks,
Generating" Art" by Learning About Styles and Deviating from Style Norms." arXiv 2017.
Learning only to see ?
46
Visual arts

47. Movie Scripts
Learning only to see ?
47
Ars Technica, “Movie written by algorithm turns out to be hilarious and intense” (2016)

48. Public Health
Esteva, Andre, Brett Kuprel, Roberto A. Novoa, Justin Ko, Susan M. Swetter, Helen M. Blau, and Sebastian Thrun.
"Dermatologist-level classification of skin cancer with deep neural networks." Nature 542, no. 7639 (2017): 115-118.
Learning only to see ?
48

49. Nacho Hernandez, “Why artificial intelligence will democratize healthcare”
(TEDx Talk, 2014)
Public health
Learning only to see ?
49

50. Nancy Lublin, “The heartbreaking text that inspired a crisis helpline” (TED Talk
2015)
Mental health
Learning only to see ?
50

51. Psychological support and counseling ?
Learning only to see ?
51

52. 52
Affective computing
Rana el Kalioubi, “This app know how you feel, from the look on your face”,
TEDTalks 2015.
Learning only to see ?

53. 53
Nexi Project,
from MIT Media Lab
(Photos: Spencer Lowel)
[video]
Affective computing
Learning only to see ?

54. 54
Challenges

55. 55
Xavier Sala-i-Martin (Columbia University),
“Les conclusions del Fòrum de Davos”
(TV3, 03/02/2016) - in Catalan
Carles Boix (Princeton University),
“La quarta revolució industrial”
(Diari Ara, 08/02/2016) - in Catalan
Artificial intelligence

56. “Google’s chairman (Eric Schmidth) thinks artificial intelligence will let
scientists solve some of the world’s "hard problems," like population
growth, climate change, human development, and education.”
(Bloomberg Business, 11/01/2016)
[+info @ MIT Technology Review]
Artificial intelligence
56

57. Google’s CEO Sundar Pichai: “Era Of Computers Will End Very Soon, AI Will
Rule” (Fossbytes, 03/05/2016)
Artificial intelligence
57

58. 58
Barack Obama, Neural Nets, Self-driving cars, and the Future
of the World (Wired, June 2016)
Artificial intelligence

59. Artificial intelligence
Stephen Hawking, “Artificial intelligence could spell out the human race.”
(2014)
59

60. Jeremy Howard, “The wonderful and terrifying implications of computers
that can learn”, TEDTalks 2014.
Artificial intelligence
60

61. 61
The White House:
“How to prepare the future for the Future Intelligence” (Jun’16)
“Artificial Intelligence, Autonomy, and the Economy” (Dec’16)
“These transformations will open up new
opportunities for individuals, the economy, and
society, but they have the potential to disrupt the
current livelihoods of millions of Americans.
Whether AI leads to unemployment and
increases in inequality over the long-run
depends not only on the technology itself but
also on the institutions and policies that are in
place.”
ArtificiaI Intelligence & Human Ethics

62. 62
Kai-Fu Lee, “The Real Threat of Artificial Intelligence”. The New
York Times (24/06/2017)
ArtificiaI Intelligence & Human Ethics
Figure: Rune Fisker
“...leading to
unprecedented economic
inequalities and even
altering the global balance
of power.”

63. 63
ArtificiaI Intelligence & Human Ethics

64. 64
ArtificiaI Intelligence & Human Ethics

65. 65
ArtificiaI Intelligence & Human Ethics

66. 66
ArtificiaI Intelligence & Human Ethics

67. Big data
Internet of things - IoT
67
ArtificiaI Intelligence & Human Ethics

68. Personal data
Big data
68
ArtificiaI Intelligence & Human Ethics

69. Neil Lawrence, OpenAI won’t benefit humanity without
open data sharing (The Guardian, 14/12/2015)
Phd Comics:
“Who owns your
data ?
(Hint: it is not you)”
69
ArtificiaI Intelligence & Human Ethics

70. 70
ArtificiaI Intelligence & Human Ethics
Jitendra Malik, “What lead computer vision to deep learning ?” ACM Communications 2017.
The AI Hype
ALGORITHMS
Deep Learning
BIG DATA
Vision: ImageNet
BIG COMPUTATION
GPUs

71. 71
ArtificiaI Intelligence & Human Ethics

72. 72

73. Course photo at the stairs (exit + left)