Cvetanka Eftimoska: How we can use PySpark for building and training an ML model
•
0 j'aime•270 vues
Cvetanka Eftimoska: How we can use PySpark for building and training an ML model Data Science Online Camp 2021 Website - https://dscamp.org/ Youtube - https://www.youtube.com/startuplviv FB - https://www.facebook.com/Data-Science-Camp-103012708431833
Recommandé
Recommandé
Contenu connexe
Tendances
Tendances (19)
Similaire à Cvetanka Eftimoska: How we can use PySpark for building and training an ML model
Similaire à Cvetanka Eftimoska: How we can use PySpark for building and training an ML model (20)
Plus de Lviv Startup Club
Plus de Lviv Startup Club (20)
Dernier
Dernier (20)
Cvetanka Eftimoska: How we can use PySpark for building and training an ML model
- 1. 1 How we can use PySpark for building and training an ML model
- 2. 2 About me. Cvetanka Eftimoska ⋮IWConnect, North Macedonia Senior Technical Consultant
- 3. 3 KEY FACTS ABOUT ⋮IWConnect . 350+ projects in over 20 industries and over 10 countries in Europe and USA 11 practices providing diversified solutions 4 locations: Bitola, Prilep, and Skopje, North Macedonia and Amsterdam, Netherlands. 16 years of working experience
- 4. 4 The goal of the presentation. • The concept of AI and Machine Learning • What is machine learning? • Types of machine learning • Machine learning flow • Define PySpark • PySpark Mllib • Data preprocessing • Linear regression • GBT Regressor • Example
- 5. 5 The concept of AI.
- 6. 6 What is Machine Learning?
- 7. 7
- 10. 10 PySpark Mllib
- 11. 11 Data preprocessing • Better results • Formatted Data • More meaningful data • Data that can be trusted • Inaccurate data • Missing data • Presence of noisy data • Inconsistent data
- 18. 18 Handle missing values - estimate
- 19. 19 Handle missing values - estimate 2. Iterative Imputer (Multivariate) • Models each feature with missing values as a function of other features
- 20. 20 Handle missing values - eliminate
- 21. 21 Tip
- 24. 24 OLS assumptions – multicollinearity
- 25. 25 OLS assumptions - multicollinearity
- 26. 26 OLS assumptions - linearity
- 27. 27 OLS assumptions - linearity
- 33. 33 Mean = 0 Standard deviation = 1 𝑥𝑖 − m𝑒𝑎𝑛 𝑥 stdev 𝑥 Standardization
- 35. 35 Split into train and test
- 40. 40 GBT Regressor
- 41. 41 GBT Regressor
- 42. 42 Conclusion.