SlideShare une entreprise Scribd logo
1  sur  19
Télécharger pour lire hors ligne
plyr
                         Wrap up


                       Hadley Wickham
Tuesday, 7 July 2009
1. Fitting multiple models to the same
                   data
                2. Reporting progress & dealing with
                   errors
                3. Overall structure & correspondence to
                   base R functions
                4. Plans
                5. Feedback

Tuesday, 7 July 2009
Multiple models
                       May need to fit multiple models to the
                       same data, with varying parameters or
                       many random starts.
                       Two plyr functions make this easy: rlply
                       & mlply
                       Example: fitting a neural network



Tuesday, 7 July 2009
1.0                                                                                                                                                                                         ●

                                                                                                                                                      ●                                                        ●
                                                                                                                              ●                                           ●
                                                                                                                                      ●                                           ●                    ●
                                                                                                                                                                      ●
                                                                                                                      ●                           ●       ●                                        ●
                                                                                                  ●                                                                   ●                        ●                   ●
                                                                                          ●               ●                                                                                        ●
                                                                                                                                                  ●                               ●           ●            ●
                                                                      ●                       ●                                   ●       ●                                                   ●
                                                                                                          ●               ●                                   ●                   ●
                           0.8                                                                    ●                                               ●
                                                                                      ●               ●               ●               ●                               ●               ●                            ●
                                                                  ●                                                                                                                                        ●
                                                                       ●                                                      ●                                   ●           ●           ●●
                                                                                       ●                                                                  ●                                            ●
                                                                                                                                          ●
                                                                                                                  ●           ●                   ●
                                                          ●                   ●                   ●
                                                                                                                                              ●           ●                                                    ●
                                                                                      ●                   ●                                                           ●       ●           ●
                                                                                                                          ●           ●                                                                    ●
                                                 ●                                                                                                                                             ●
                                                                      ●                       ●                                                                   ● ●
                                     ●                                ●●                                                  ●                       ● ●                                 ●
                           0.6                                                    ●                                                       ●
                                                              ●            ●                          ●           ●           ●                   ●
                                                                                                                      ●                                       ●                   ●            ●
                                                     ●                 ●                                                          ●                               ●
                                                                                       ●                      ●                           ●
                                                              ●                                                                                                                           ●
                                                                                              ●       ●                                       ●                                   ●                                        class
                                                                               ●                              ●               ●                               ●
                                                     ●                                                                                ●                                                                                     ●   A
                       y




                                                                               ●          ●               ●       ●                                       ●●                  ●
                                                  ●               ●                                                                                                                       ●
                                                                                                  ●                           ●       ● ●                                                                                   ●   B
                                                 ● ●                      ●                                                                                               ●
                                                                                                                                                                                  ●
                           0.4                                                        ●                   ●       ●
                                                                                                          ●                                   ●                               ●
                                             ●            ●                                                                           ●                   ●
                                                          ●                                                               ●
                                  ●                                                   ●                                           ●                                       ●
                                                                       ●                                      ●
                                                                                                                                                  ●       ●
                                         ●                                                                                        ●
                                                              ●            ●                                                  ●       ●                                           ●
                                 ●               ●                                                                ●                                                   ●
                                                 ●
                                                 ●
                                                                                                                                              ●                                   ●
                                                                   ●               ●                              ●           ●       ●               ●
                           0.2                                ●
                                                                                              ● ●                                                             ●
                                         ●                                     ●                                          ●
                                                                                          ●
                                                     ●●                                           ●               ●                       ●
                                  ●                                    ●
                                                                  ●                ●
                                                          ●                                               ●               ●       ●
                                             ●                                            ●
                                                              ●        ●
                                                     ●                                                        ●
                                         ●                ●                   ●               ●

                           0.0                                     ●



                                 0.0                              0.2                                 0.4                                 0.6                                     0.8                              1.0
                                                                                                                      x
Tuesday, 7 July 2009
library(nnet)
      library(ggplot2)

      w <- read.csv("wiggly.csv")
      qplot(x, y, data = w, colour = class)

      accuracy <- function(mod, true) {
        pred <- factor(predict(mod, type = "class"),
          levels = levels(true))
        tb <- table(pred, true)
        sum(diag(tb)) / sum(tb)
      }

      nnet(class ~ x + y, data = w, size = 3)

Tuesday, 7 July 2009
rlply

                       A little different to the other plyr functions:
                       first argument is number of times to run,
                       second argument is an expression (not a
                       function).
                       Automatically adds run number (.n) to
                       labels.



Tuesday, 7 July 2009
models <- rlply(50, nnet(class ~ x + y, data = w,
      size = 3, trace = FALSE))

      accdf <- ldply(models, "accuracy", true = w$class)
      accdf
      qplot(accuracy, data = accdf, binwith = 0.02)




Tuesday, 7 July 2009
mlply
                       What if we want to systematically vary the
                       input parameters?
                       mlply allows us to vary all of the
                       arguments to the applicator function, not
                       just the first argument
                       Input is a data frame of parameter values



Tuesday, 7 July 2009
wiggly_nnet <- function(...) {
        nnet(class ~ x + y, data = w, trace = FALSE, ...)
      }
      rlply(5, wiggly_nnet(size = 3))

      # Unfortunately need 2+ parameters because of bug
      opts <- data.frame(size = 1:10, maxiter = 50)
      opts

      models <- mlply(opts, wiggly_nnet)
      ldply(models, "accuracy", true = w$class)

      # expand.grid() useful if you want to explore
      # all combinations

Tuesday, 7 July 2009
Progress & errors



Tuesday, 7 July 2009
Progress bars
                       Things seem to take much less time when
                       you are regularly updated on their
                       progress.
                       Plyr provides convenient method for
                       doing this: all arguments
                       accept .progress = "text" argument




Tuesday, 7 July 2009
Error handling
                       Helper function: failwith
                       Takes a function as an input and returns a
                       function as output, but instead of
                       throwing an error it will return the value
                       you specify.
                       failwith(NULL, lm)(cty ~ displ, data = mpg)
                       failwith(NULL, lm)(cty ~ displ, data = NULL)



Tuesday, 7 July 2009
Overall structure



Tuesday, 7 July 2009
array   data frame    list   nothing

             array     aaply     adply      alply    a_ply

      data frame       daply     ddply      dlply   d_ply

                list   laply     ldply      llply    l_ply

     n replicates      raply     rdply      rlply    r_ply

       function
                       maply     mdply      mlply   m_ply
      arguments

Tuesday, 7 July 2009
No output

                       Useful for functions called purely for their
                       side effects: write.table, save, graphics.
                       If .print = TRUE will print each result
                       (particularly useful lattice and ggplot2 graphics)




Tuesday, 7 July 2009
Your turn

                       With your partner, using your collective R
                       knowledge, come up with all of the
                       functions in base R (or contributed
                       packages) that do the same thing.




Tuesday, 7 July 2009
array      data frame      list     nothing

             array      apply        adply        alply      a_ply

      data frame        daply      aggregate       by       d_ply

                list    sapply       ldply       lapply      l_ply

     n replicates      replicate     rdply      replicate    r_ply

       function
                       mapply        mdply      mapply      m_ply
      arguments

Tuesday, 7 July 2009
Plans

                       Deal better with large and larger data:
                       trivial parallelisation & on-disk data (sql
                       etc)
                       Stay tuned for details.




Tuesday, 7 July 2009
http://hadley.wufoo.com/
                 forms/course-evaluation/



Tuesday, 7 July 2009

Contenu connexe

Similaire à 04 Wrapup

Los Angeles R users group - July 12 2011 - Part 1
Los Angeles R users group - July 12 2011 - Part 1Los Angeles R users group - July 12 2011 - Part 1
Los Angeles R users group - July 12 2011 - Part 1
rusersla
 
Over Visie, Missie En Strategie
Over Visie, Missie En StrategieOver Visie, Missie En Strategie
Over Visie, Missie En Strategie
Guus Vos
 
About Vision, Mission And Strategy
About Vision, Mission And StrategyAbout Vision, Mission And Strategy
About Vision, Mission And Strategy
Guus Vos
 
研修企画書11 12term voda-カヤック
研修企画書11 12term voda-カヤック研修企画書11 12term voda-カヤック
研修企画書11 12term voda-カヤック
aiesecsfc_icx2011
 
研修企画書11-12term voda-カヤック
研修企画書11-12term voda-カヤック研修企画書11-12term voda-カヤック
研修企画書11-12term voda-カヤック
aiesecsfc_icx2011
 
Modul mulus bahagian c sjk (modul murid)
Modul mulus bahagian c sjk (modul murid)Modul mulus bahagian c sjk (modul murid)
Modul mulus bahagian c sjk (modul murid)
Anparasu
 
Modul mulus bahagian c sjk (modul guru)
Modul mulus bahagian c sjk (modul guru)Modul mulus bahagian c sjk (modul guru)
Modul mulus bahagian c sjk (modul guru)
Anparasu
 
Modul mulus bahagian c sk (modul murid)
Modul mulus bahagian c sk (modul murid)Modul mulus bahagian c sk (modul murid)
Modul mulus bahagian c sk (modul murid)
Anparasu
 
Modul mulus bahagian c sk (modul guru)
Modul mulus bahagian c sk (modul guru)Modul mulus bahagian c sk (modul guru)
Modul mulus bahagian c sk (modul guru)
Anparasu
 
正誤表 p39
正誤表 p39正誤表 p39
正誤表 p39
zafiro555
 

Similaire à 04 Wrapup (20)

02 Large
02 Large02 Large
02 Large
 
02 large
02 large02 large
02 large
 
Los Angeles R users group - July 12 2011 - Part 1
Los Angeles R users group - July 12 2011 - Part 1Los Angeles R users group - July 12 2011 - Part 1
Los Angeles R users group - July 12 2011 - Part 1
 
08 Continuous
08 Continuous08 Continuous
08 Continuous
 
08 Continuous
08 Continuous08 Continuous
08 Continuous
 
Over Visie, Missie En Strategie
Over Visie, Missie En StrategieOver Visie, Missie En Strategie
Over Visie, Missie En Strategie
 
About Vision, Mission And Strategy
About Vision, Mission And StrategyAbout Vision, Mission And Strategy
About Vision, Mission And Strategy
 
13 Bivariate
13 Bivariate13 Bivariate
13 Bivariate
 
21 Ml
21 Ml21 Ml
21 Ml
 
How People Use Facebook -- And Why It Matters
How People Use Facebook -- And Why It MattersHow People Use Facebook -- And Why It Matters
How People Use Facebook -- And Why It Matters
 
研修企画書11 12term voda-カヤック
研修企画書11 12term voda-カヤック研修企画書11 12term voda-カヤック
研修企画書11 12term voda-カヤック
 
01 intro
01 intro01 intro
01 intro
 
研修企画書11-12term voda-カヤック
研修企画書11-12term voda-カヤック研修企画書11-12term voda-カヤック
研修企画書11-12term voda-カヤック
 
Modul mulus bahagian c sjk (modul murid)
Modul mulus bahagian c sjk (modul murid)Modul mulus bahagian c sjk (modul murid)
Modul mulus bahagian c sjk (modul murid)
 
Modul mulus bahagian c sjk (modul guru)
Modul mulus bahagian c sjk (modul guru)Modul mulus bahagian c sjk (modul guru)
Modul mulus bahagian c sjk (modul guru)
 
17 Sampling Dist
17 Sampling Dist17 Sampling Dist
17 Sampling Dist
 
Modul mulus bahagian c sk (modul murid)
Modul mulus bahagian c sk (modul murid)Modul mulus bahagian c sk (modul murid)
Modul mulus bahagian c sk (modul murid)
 
Modul mulus bahagian c sk (modul guru)
Modul mulus bahagian c sk (modul guru)Modul mulus bahagian c sk (modul guru)
Modul mulus bahagian c sk (modul guru)
 
The ecological and evolutionary impacts of altered
The ecological and evolutionary impacts of alteredThe ecological and evolutionary impacts of altered
The ecological and evolutionary impacts of altered
 
正誤表 p39
正誤表 p39正誤表 p39
正誤表 p39
 

Plus de Hadley Wickham (20)

27 development
27 development27 development
27 development
 
27 development
27 development27 development
27 development
 
24 modelling
24 modelling24 modelling
24 modelling
 
23 data-structures
23 data-structures23 data-structures
23 data-structures
 
Graphical inference
Graphical inferenceGraphical inference
Graphical inference
 
R packages
R packagesR packages
R packages
 
22 spam
22 spam22 spam
22 spam
 
21 spam
21 spam21 spam
21 spam
 
20 date-times
20 date-times20 date-times
20 date-times
 
19 tables
19 tables19 tables
19 tables
 
18 cleaning
18 cleaning18 cleaning
18 cleaning
 
16 critique
16 critique16 critique
16 critique
 
15 time-space
15 time-space15 time-space
15 time-space
 
13 case-study
13 case-study13 case-study
13 case-study
 
12 adv-manip
12 adv-manip12 adv-manip
12 adv-manip
 
11 adv-manip
11 adv-manip11 adv-manip
11 adv-manip
 
11 adv-manip
11 adv-manip11 adv-manip
11 adv-manip
 
10 simulation
10 simulation10 simulation
10 simulation
 
10 simulation
10 simulation10 simulation
10 simulation
 
09 bootstrapping
09 bootstrapping09 bootstrapping
09 bootstrapping
 

Dernier

+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
?#DUbAI#??##{{(☎️+971_581248768%)**%*]'#abortion pills for sale in dubai@
 

Dernier (20)

"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ..."I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
 
AWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of TerraformAWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of Terraform
 
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot TakeoffStrategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
 
Automating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps ScriptAutomating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps Script
 
AXA XL - Insurer Innovation Award Americas 2024
AXA XL - Insurer Innovation Award Americas 2024AXA XL - Insurer Innovation Award Americas 2024
AXA XL - Insurer Innovation Award Americas 2024
 
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
 
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024
 
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
 
FWD Group - Insurer Innovation Award 2024
FWD Group - Insurer Innovation Award 2024FWD Group - Insurer Innovation Award 2024
FWD Group - Insurer Innovation Award 2024
 
Navi Mumbai Call Girls 🥰 8617370543 Service Offer VIP Hot Model
Navi Mumbai Call Girls 🥰 8617370543 Service Offer VIP Hot ModelNavi Mumbai Call Girls 🥰 8617370543 Service Offer VIP Hot Model
Navi Mumbai Call Girls 🥰 8617370543 Service Offer VIP Hot Model
 
MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024
 
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin WoodPolkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
 
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, AdobeApidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
 
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
 
ICT role in 21st century education and its challenges
ICT role in 21st century education and its challengesICT role in 21st century education and its challenges
ICT role in 21st century education and its challenges
 
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
 
2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...
 
Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...
Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...
Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...
 
Real Time Object Detection Using Open CV
Real Time Object Detection Using Open CVReal Time Object Detection Using Open CV
Real Time Object Detection Using Open CV
 
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
 

04 Wrapup

  • 1. plyr Wrap up Hadley Wickham Tuesday, 7 July 2009
  • 2. 1. Fitting multiple models to the same data 2. Reporting progress & dealing with errors 3. Overall structure & correspondence to base R functions 4. Plans 5. Feedback Tuesday, 7 July 2009
  • 3. Multiple models May need to fit multiple models to the same data, with varying parameters or many random starts. Two plyr functions make this easy: rlply & mlply Example: fitting a neural network Tuesday, 7 July 2009
  • 4. 1.0 ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● 0.8 ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● 0.6 ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● class ● ● ● ● ● ● ● A y ● ● ● ● ●● ● ● ● ● ● ● ● ● ● B ● ● ● ● ● 0.4 ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● 0.2 ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● 0.0 ● 0.0 0.2 0.4 0.6 0.8 1.0 x Tuesday, 7 July 2009
  • 5. library(nnet) library(ggplot2) w <- read.csv("wiggly.csv") qplot(x, y, data = w, colour = class) accuracy <- function(mod, true) { pred <- factor(predict(mod, type = "class"), levels = levels(true)) tb <- table(pred, true) sum(diag(tb)) / sum(tb) } nnet(class ~ x + y, data = w, size = 3) Tuesday, 7 July 2009
  • 6. rlply A little different to the other plyr functions: first argument is number of times to run, second argument is an expression (not a function). Automatically adds run number (.n) to labels. Tuesday, 7 July 2009
  • 7. models <- rlply(50, nnet(class ~ x + y, data = w, size = 3, trace = FALSE)) accdf <- ldply(models, "accuracy", true = w$class) accdf qplot(accuracy, data = accdf, binwith = 0.02) Tuesday, 7 July 2009
  • 8. mlply What if we want to systematically vary the input parameters? mlply allows us to vary all of the arguments to the applicator function, not just the first argument Input is a data frame of parameter values Tuesday, 7 July 2009
  • 9. wiggly_nnet <- function(...) { nnet(class ~ x + y, data = w, trace = FALSE, ...) } rlply(5, wiggly_nnet(size = 3)) # Unfortunately need 2+ parameters because of bug opts <- data.frame(size = 1:10, maxiter = 50) opts models <- mlply(opts, wiggly_nnet) ldply(models, "accuracy", true = w$class) # expand.grid() useful if you want to explore # all combinations Tuesday, 7 July 2009
  • 11. Progress bars Things seem to take much less time when you are regularly updated on their progress. Plyr provides convenient method for doing this: all arguments accept .progress = "text" argument Tuesday, 7 July 2009
  • 12. Error handling Helper function: failwith Takes a function as an input and returns a function as output, but instead of throwing an error it will return the value you specify. failwith(NULL, lm)(cty ~ displ, data = mpg) failwith(NULL, lm)(cty ~ displ, data = NULL) Tuesday, 7 July 2009
  • 14. array data frame list nothing array aaply adply alply a_ply data frame daply ddply dlply d_ply list laply ldply llply l_ply n replicates raply rdply rlply r_ply function maply mdply mlply m_ply arguments Tuesday, 7 July 2009
  • 15. No output Useful for functions called purely for their side effects: write.table, save, graphics. If .print = TRUE will print each result (particularly useful lattice and ggplot2 graphics) Tuesday, 7 July 2009
  • 16. Your turn With your partner, using your collective R knowledge, come up with all of the functions in base R (or contributed packages) that do the same thing. Tuesday, 7 July 2009
  • 17. array data frame list nothing array apply adply alply a_ply data frame daply aggregate by d_ply list sapply ldply lapply l_ply n replicates replicate rdply replicate r_ply function mapply mdply mapply m_ply arguments Tuesday, 7 July 2009
  • 18. Plans Deal better with large and larger data: trivial parallelisation & on-disk data (sql etc) Stay tuned for details. Tuesday, 7 July 2009
  • 19. http://hadley.wufoo.com/ forms/course-evaluation/ Tuesday, 7 July 2009