Le téléchargement de votre SlideShare est en cours. ×

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

DataScienceConferenc1

•

30 nov. 2022

• •

Publicité

Publicité

Publicité

Publicité

Publicité

Publicité

Publicité

Publicité

Publicité

Publicité

Publicité

Publicité

Sketch of the proof:

Sketch of the proof:

Sketch of the proof:

Sketch of the proof (d=2):

Sketch of the proof (d=2):

Sketch of the proof (d=2):

Sketch of the proof (d=2):

Sketch of the proof (d=2):

Sketch of the proof (d=2):

Sketch of the proof (d=2):

Over-parametrization is necessary for robustness!

Over-parametrization is necessary for robustness!

Over-parametrization is necessary for robustness!

Curse of dimensionality!

This is huge over-parametrization!

This is huge over-parametrization!

This is huge over-parametrization!

Source: nilesjohnson.net

•Thank you!

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

SlideShares suivants

Prochain SlideShare

[DSC Europe 22] The nuts and bolts of music source separation - Stipe Kabic

[DSC Europe 22] The nuts and bolts of music source separation - Stipe Kabic

Chargement dans…3

×

Consultez-les par la suite

[DSC Europe 22] Avoid mistakes building AI products - Karol Przystalski

DataScienceConferenc1

[DSC Europe 22] The Making of a Data Organization - Denys Holovatyi

DataScienceConferenc1

[DSC Europe 22] Building the future, a founder's perspective - Ben Taylor

DataScienceConferenc1

[DSC Europe 22] Bridging the gap between AI Research and Human-Centered Produ...

DataScienceConferenc1

[DSC Europe 22] Reverse Engineering Cloud Native: Interoperability and Commun...

DataScienceConferenc1

[DSC Europe 22] Smart approach in development and deployment process for vari...

DataScienceConferenc1

[DSC Europe 22] Engineers guide for shepherding models in to production - Mar...

DataScienceConferenc1

[DSC Europe 22] Anti-Money Laundering ML Modeling approach - Gizem Akar

DataScienceConferenc1

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

30 nov. 2022

• •

Télécharger pour lire hors ligne

Données & analyses

We will present some recent fundamental developments in understanding how neural networks work, in particular why the overparametrized models with the billions or even trilions of parameters perform so well even in cases when the number of parameters outnumbers the number of training samples, contrary to classical machine learning paradigm. We will present the BLN Conjecture (2020) and demonstrate few extreme cases of Bubeck-Sellke Theorem (2021). In order to illustrate highly technical theorem and it's proof we will use an original pictorial approach to represent the behaviour of neural networks from the point of view of the higher-dimensional geometry.

DataScienceConferenc1

We will present some recent fundamental developments in understanding how neural networks work, in particular why the overparametrized models with the billions or even trilions of parameters perform so well even in cases when the number of parameters outnumbers the number of training samples, contrary to classical machine learning paradigm. We will present the BLN Conjecture (2020) and demonstrate few extreme cases of Bubeck-Sellke Theorem (2021). In order to illustrate highly technical theorem and it's proof we will use an original pictorial approach to represent the behaviour of neural networks from the point of view of the higher-dimensional geometry.

Données & analyses

Plus De Contenu Connexe

[DSC Europe 22] Avoid mistakes building AI products - Karol Przystalski

DataScienceConferenc1

[DSC Europe 22] The Making of a Data Organization - Denys Holovatyi

DataScienceConferenc1

[DSC Europe 22] Building the future, a founder's perspective - Ben Taylor

DataScienceConferenc1

[DSC Europe 22] Bridging the gap between AI Research and Human-Centered Produ...

DataScienceConferenc1

[DSC Europe 22] Reverse Engineering Cloud Native: Interoperability and Commun...

DataScienceConferenc1

[DSC Europe 22] Smart approach in development and deployment process for vari...

DataScienceConferenc1

[DSC Europe 22] Engineers guide for shepherding models in to production - Mar...

DataScienceConferenc1

[DSC Europe 22] Anti-Money Laundering ML Modeling approach - Gizem Akar

DataScienceConferenc1

[DSC Europe 22] The ways AI will change your workplace - Iliada Kothra

DataScienceConferenc1

[DSC Europe 22] Similarity search: Take your search engine to the next level ...

DataScienceConferenc1

[DSC Europe 22] Equality alert in newsrooms: Providing transparency about the...

DataScienceConferenc1

[DSC Europe 22] Reproducibility and Versioning of ML Systems - Spela Poklukar

DataScienceConferenc1

[DSC Europe 22] Lakehouse architecture with Delta Lake and Databricks - Draga...

DataScienceConferenc1

[DSC Europe 22] AIOPS – How can machine learning help in IT operations - Dam...

DataScienceConferenc1

[DSC Europe 22] AI Ethics and AI Quality By Design - Muthu Ramachandran

DataScienceConferenc1

[DSC Europe 22] Machine learning algorithms as tools for student success pred...

DataScienceConferenc1

[DSC Europe 22] Parakeet - a parrot that can repeat words very well - Tomisla...

DataScienceConferenc1

[DSC Europe 22] Disrupting Supply Chain with Data Science - Arthur Jansen

DataScienceConferenc1

[DSC Europe 22] How to enable enterprises for DevSecOps without fighting the ...

DataScienceConferenc1

[DSC Europe 22] Starting deep learning projects without sufficient amount of ...

DataScienceConferenc1

[DSC Europe 22] Avoid mistakes building AI products - Karol Przystalski

DataScienceConferenc1

69 diapositives

[DSC Europe 22] The Making of a Data Organization - Denys Holovatyi

DataScienceConferenc1

16 diapositives

[DSC Europe 22] Building the future, a founder's perspective - Ben Taylor

DataScienceConferenc1

41 diapositives

[DSC Europe 22] Bridging the gap between AI Research and Human-Centered Produ...

DataScienceConferenc1

22 diapositives

[DSC Europe 22] Reverse Engineering Cloud Native: Interoperability and Commun...

DataScienceConferenc1

43 diapositives

[DSC Europe 22] Smart approach in development and deployment process for vari...

DataScienceConferenc1

15 diapositives

Business Case _ PM.pptx

vaibhavverma749167

deeplearning-annotated.pdf

Chapter 5 Advanced Data Modeling.pptx

gartner-magic-quadrant-critical-capabilities-methodologies-evolution.pdf

FADML 09 PPC NP Completeness (1).pdf

Abhishekgarg99271

Lectures_on_Artificial_Intelligence_08.09.16.pdf

KpsMurugesan Kpsm

Presentación 2° Encuesta Nacional Ecosistema Emprendedor UGM BW.pdf

AproximacionAlFuturo

Aashish PPT.pptx

IOT Smart City PowerPoint Presentation Template Free_wowTemplates.pptx

KpsMurugesan Kpsm

Canon Project - Audience Research.pptx

watson ppt.pptx

Computer Basics.docx

Python PPT.pptx

Data_Science_Applications_&_Use_Cases.pptx

AIS 3 - EDITED.pdf

_12_ - Exponential Functions (1).pdf

QNX-Neutrino-by-Pranshu-Gupta-2003-Fall.ppt

INTRODUCTION TO STATISTICS.pptx

Business Case _ PM.pptx

vaibhavverma749167

16 diapositives

deeplearning-annotated.pdf

44 diapositives

Chapter 5 Advanced Data Modeling.pptx

40 diapositives

gartner-magic-quadrant-critical-capabilities-methodologies-evolution.pdf

22 diapositives

FADML 09 PPC NP Completeness (1).pdf

Abhishekgarg99271

34 diapositives

Lectures_on_Artificial_Intelligence_08.09.16.pdf

KpsMurugesan Kpsm

96 diapositives

[DSC Europe 22] Towards the Universal Law of Robustness for Deep Neural Networks - Luka Nenadovic

1. Sketch of the proof:
2. Sketch of the proof:
3. Sketch of the proof:
4. Sketch of the proof (d=2):
5. Sketch of the proof (d=2):
6. Sketch of the proof (d=2):
7. Sketch of the proof (d=2):
8. Sketch of the proof (d=2):
9. Sketch of the proof (d=2):
10. Sketch of the proof (d=2):
11. Over-parametrization is necessary for robustness!
12. Over-parametrization is necessary for robustness!
13. Over-parametrization is necessary for robustness!
14. Curse of dimensionality!
15. This is huge over-parametrization!
16. This is huge over-parametrization!
17. This is huge over-parametrization!
18. Source: nilesjohnson.net
19. •Thank you!