Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.
Best Practices for Hyperparameter
Optimization
Alexandra Johnson
@alexandraj777
Example: Beating Vegas
Scott Clark. Using Model Tuning to Beat Vegas.
Terminology
● Optimization
● Hyperparameter
Optimization
● Hyperparameter
tuning
● Model tuning
Tune the Whole Pipeline
Optimize all Parameters at Once
TensorFlow Playground
Include Feature Parameters
Include Feature Parameters
Choosing a Metric
● Balance long-term
and short-term goals
● Question underlying
assumptions
● Example from
Microsoft
Composite Metric
Example: Lifetime Value
clicks*wclicks + likes*wlikes + views*wviews
Choose Multiple Metrics
● Balance competing
metrics
● Explore entire result
space
Image from PhD Comics
Avoiding Overfitting
Get A Suggestion
Shuffle and Split Data
Train the Model
Test the Performance
Repeat
Shuffle Train Evaluate
Report An Observation
Repeat the Entire Process
Shuffle Train Evaluate
Optimization Methods
Hand Tuning
● Hand tuning is time
consuming and
expensive
● Algorithms can
quickly and cheaply
beat expert tuning
Grid Search Random Search Bayesian Optimization
Alternatives to Hand Tuning
No Grid Search
Hyper-
parameters
Model
Evaluations
2 100
3 1,000
4 10,000
5 100,000
No Random Search
● Theoretically more
effective than grid
search
● Large variance in
results
● No intelligence
Bayesian Optimization
● Explore/exploit
● Ideal for "expensive"
optimization
● No requirements on:
convex,
differentiable,...
Alternatives to Bayesian Optimization
Genetic algorithms
Particle-based methods
Convex optimizers
Simulated annealing
To n...
Takeaways
●Optimize the entire pipeline
●Ensure generalization
●Use Bayesian optimization
Thank You!
blog.sigopt.com
sigopt.com/research
Prochain SlideShare
Chargement dans…5
×

Alexandra Johnson, Software Engineer, SigOpt at MLconf ATL 2017

409 vues

Publié le

Best Practices for Hyperparameter Optimization:
All machine learning and artificial intelligence pipelines – from reinforcement agents to deep neural nets – have tunable hyperparameters. Optimizing these hyperparameters provides tremendous performance gains, but only if the optimization is done correctly. This presentation will discuss topics including selecting performance criteria, why you should always use cross validation, and choosing between state of the art optimization methods.

Publié dans : Technologie
  • Identifiez-vous pour voir les commentaires

  • Soyez le premier à aimer ceci

Alexandra Johnson, Software Engineer, SigOpt at MLconf ATL 2017

  1. 1. Best Practices for Hyperparameter Optimization Alexandra Johnson @alexandraj777
  2. 2. Example: Beating Vegas Scott Clark. Using Model Tuning to Beat Vegas.
  3. 3. Terminology ● Optimization ● Hyperparameter Optimization ● Hyperparameter tuning ● Model tuning
  4. 4. Tune the Whole Pipeline
  5. 5. Optimize all Parameters at Once TensorFlow Playground
  6. 6. Include Feature Parameters
  7. 7. Include Feature Parameters
  8. 8. Choosing a Metric ● Balance long-term and short-term goals ● Question underlying assumptions ● Example from Microsoft
  9. 9. Composite Metric Example: Lifetime Value clicks*wclicks + likes*wlikes + views*wviews
  10. 10. Choose Multiple Metrics ● Balance competing metrics ● Explore entire result space Image from PhD Comics
  11. 11. Avoiding Overfitting
  12. 12. Get A Suggestion
  13. 13. Shuffle and Split Data
  14. 14. Train the Model
  15. 15. Test the Performance
  16. 16. Repeat Shuffle Train Evaluate
  17. 17. Report An Observation
  18. 18. Repeat the Entire Process Shuffle Train Evaluate
  19. 19. Optimization Methods
  20. 20. Hand Tuning ● Hand tuning is time consuming and expensive ● Algorithms can quickly and cheaply beat expert tuning
  21. 21. Grid Search Random Search Bayesian Optimization Alternatives to Hand Tuning
  22. 22. No Grid Search Hyper- parameters Model Evaluations 2 100 3 1,000 4 10,000 5 100,000
  23. 23. No Random Search ● Theoretically more effective than grid search ● Large variance in results ● No intelligence
  24. 24. Bayesian Optimization ● Explore/exploit ● Ideal for "expensive" optimization ● No requirements on: convex, differentiable, continuous
  25. 25. Alternatives to Bayesian Optimization Genetic algorithms Particle-based methods Convex optimizers Simulated annealing To name a few...
  26. 26. Takeaways ●Optimize the entire pipeline ●Ensure generalization ●Use Bayesian optimization
  27. 27. Thank You! blog.sigopt.com sigopt.com/research

×