SlideShare une entreprise Scribd logo
1  sur  30
Adding Statistical Functionality to the DATA Step with PROC FCMP Stacey Christian and Jacques Rioux  SAS Institute Inc., Cary, NC Paper 326-2010
Introduction/Motivation ,[object Object],[object Object],[object Object]
Overview ,[object Object],[object Object],[object Object],[object Object]
RUN_MACRO Function in FCMP  ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
See Macro Run ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],diff=4.6
See Macro Run  in DATA Step ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],diff=4.6
Recursive Technique:   Segmenting Time Series Data ,[object Object],[object Object],[object Object],[object Object]
Top Down Algorithm ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Top Down Subroutine ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Linear Approximation Subroutine function  linear_approximation (ds_in $, var $, first_obs, last_obs); rc = run_macro(‘linear_approximation_macro’, ds_in,  first_obs, last_obs, var, error); return(error); endsub;
Linear Approximation Macro ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Recursive Technique: Results data _NULL_;  call segment_topdown("sasuser.snp", "work.segds_20",  "close", 1, 15116, 0.2);  call segment_topdown("sasuser.snp", "work.segds_15",  "close", 1, 15116, 0.15); run;
Recursive Technique:   Graphic Results  42 Piecewise Linear Segments
Recursive Technique:   Graphic Results  113 Piecewise Linear Segments
Iterative Technique ,[object Object],[object Object],[object Object],[object Object]
Iteratively Reweighted Least Squares Algorithm ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
IterativeTechnique:    DATA Step code ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Run_Regression Subroutine subroutine  run_regression ( data $, dependent $, independent $, weight $, parmData $, parmArray[*]);  outargs parmArray;  array tmpArray[1] _temporary_;  rc = RUN_MACRO ('run_regression_macro', data, parmData , dependent, independent, weight) ;  rc = read_array(parmData, tmpArray);  do i = 1 to dim(parmArray);  parmArray[i] = tmpArray[1,i];  end;  endsub;
Run_Regression Macro ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
The True Glory of Reusable Functions:  The Simulation ,[object Object],[object Object],[object Object]
The Simulation Study ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]
Simulation Results
Simulation Graphs
Meta Programming ,[object Object],subroutine create_score( data $, dependent $, independent $, scoreFunc $, library $ ); paramds = "work.params"; rc = RUN_MACRO('run_regression_macro', data, paramds, dependent, independent); rc = RUN_MACRO('create_score_func_macro', paramds, independent, scoreFunc, library); endsub;
Score Function Macro %macro create_score_func_macro; proc transpose data =&paramds out=&paramds._t; var &independent; run; proc sql noprint; select trim(_NAME_) || &quot; * &quot; || strip(put(col1,BEST12.)) into: theScore separated by &quot;  +  &quot; from &paramds._t; select trim(_NAME_) into: theArgs separated by &quot; , &quot; from &paramds._t; quit; data _NULL_;  set &paramds; call symputX (&quot;Intercept&quot;,intercept); run; <continued>
Score Function Macro - continued proc fcmp outlib=&library..score; function &scoreFunc(&theArgs); return(&Intercept + &theScore); endsub; quit; %mend create_score_func_macro;
Run Create Score Function data _NULL_; call create_score(&quot;work.mroz&quot;, &quot;lwage&quot;, &quot;educ exper age kidslt6 kidsge6&quot;, &quot;PredLWage_Full&quot;, &quot;sasuser.score&quot;); call create_score(&quot;work.mroz&quot;, &quot;lwage&quot;, &quot;educ exper age&quot;,  &quot;PredLWage_NoKids&quot;, &quot;sasuser.score&quot;); run; data _NULL_; educ = 15;  exper = 5; age = 30; kidslt6 = 2; kidsge6 = 1; PredWage_Full = exp(PredLWage_Full(educ, exper, age, kidslt6, kidsge6)); put PredWage_Full=; PredWage_NoKids = exp(PredLWage_NoKids(educ, exper, age)); put PredWage_NoKids=; run; PredWage_Full=3.4199679212  PredWage_NoKids=3.787216653
Conclusions ,[object Object],[object Object],[object Object]
Where to find more information ,[object Object],[object Object],[object Object]
Adding Statistical Functionality to the DATA Step with PROC FCMP Paper 326-2010

Contenu connexe

Tendances

Oracle - Program with PL/SQL - Lession 09
Oracle - Program with PL/SQL - Lession 09Oracle - Program with PL/SQL - Lession 09
Oracle - Program with PL/SQL - Lession 09Thuan Nguyen
 
T11_effects_of_peripheral_stim
T11_effects_of_peripheral_stimT11_effects_of_peripheral_stim
T11_effects_of_peripheral_stimtutorialsruby
 
Web Developer make the most out of your Database !
Web Developer make the most out of your Database !Web Developer make the most out of your Database !
Web Developer make the most out of your Database !Jean-Marc Desvaux
 
Oracle - Program with PL/SQL - Lession 04
Oracle - Program with PL/SQL - Lession 04Oracle - Program with PL/SQL - Lession 04
Oracle - Program with PL/SQL - Lession 04Thuan Nguyen
 
Oracle - Program with PL/SQL - Lession 11
Oracle - Program with PL/SQL - Lession 11Oracle - Program with PL/SQL - Lession 11
Oracle - Program with PL/SQL - Lession 11Thuan Nguyen
 
as400 built in function- %PARMS
as400 built in function- %PARMSas400 built in function- %PARMS
as400 built in function- %PARMSaminem_mp
 
Oracle - Program with PL/SQL - Lession 07
Oracle - Program with PL/SQL - Lession 07Oracle - Program with PL/SQL - Lession 07
Oracle - Program with PL/SQL - Lession 07Thuan Nguyen
 
The Ring programming language version 1.7 book - Part 84 of 196
The Ring programming language version 1.7 book - Part 84 of 196The Ring programming language version 1.7 book - Part 84 of 196
The Ring programming language version 1.7 book - Part 84 of 196Mahmoud Samir Fayed
 
Oracle 11g PL/SQL notes
Oracle 11g PL/SQL notesOracle 11g PL/SQL notes
Oracle 11g PL/SQL notesanilakduygu
 
Caspar's First Kernel Patch
Caspar's First Kernel PatchCaspar's First Kernel Patch
Caspar's First Kernel PatchCaspar Zhang
 
Parametrized testing, v2
Parametrized testing, v2Parametrized testing, v2
Parametrized testing, v2Brian Okken
 

Tendances (13)

Oracle - Program with PL/SQL - Lession 09
Oracle - Program with PL/SQL - Lession 09Oracle - Program with PL/SQL - Lession 09
Oracle - Program with PL/SQL - Lession 09
 
T11_effects_of_peripheral_stim
T11_effects_of_peripheral_stimT11_effects_of_peripheral_stim
T11_effects_of_peripheral_stim
 
Web Developer make the most out of your Database !
Web Developer make the most out of your Database !Web Developer make the most out of your Database !
Web Developer make the most out of your Database !
 
Oracle 11g caracteristicas poco documentadas 3 en 1
Oracle 11g caracteristicas poco documentadas 3 en 1Oracle 11g caracteristicas poco documentadas 3 en 1
Oracle 11g caracteristicas poco documentadas 3 en 1
 
Oracle - Program with PL/SQL - Lession 04
Oracle - Program with PL/SQL - Lession 04Oracle - Program with PL/SQL - Lession 04
Oracle - Program with PL/SQL - Lession 04
 
Oracle - Program with PL/SQL - Lession 11
Oracle - Program with PL/SQL - Lession 11Oracle - Program with PL/SQL - Lession 11
Oracle - Program with PL/SQL - Lession 11
 
as400 built in function- %PARMS
as400 built in function- %PARMSas400 built in function- %PARMS
as400 built in function- %PARMS
 
Oracle - Program with PL/SQL - Lession 07
Oracle - Program with PL/SQL - Lession 07Oracle - Program with PL/SQL - Lession 07
Oracle - Program with PL/SQL - Lession 07
 
The Ring programming language version 1.7 book - Part 84 of 196
The Ring programming language version 1.7 book - Part 84 of 196The Ring programming language version 1.7 book - Part 84 of 196
The Ring programming language version 1.7 book - Part 84 of 196
 
Oracle 11g PL/SQL notes
Oracle 11g PL/SQL notesOracle 11g PL/SQL notes
Oracle 11g PL/SQL notes
 
Caspar's First Kernel Patch
Caspar's First Kernel PatchCaspar's First Kernel Patch
Caspar's First Kernel Patch
 
Parametrized testing, v2
Parametrized testing, v2Parametrized testing, v2
Parametrized testing, v2
 
Estructura secuencial -garcia
Estructura secuencial -garciaEstructura secuencial -garcia
Estructura secuencial -garcia
 

En vedette

FPGA IMPLEMENTATION OF NOISE CANCELLATION USING ADAPTIVE ALGORITHMS
FPGA IMPLEMENTATION OF NOISE CANCELLATION USING ADAPTIVE ALGORITHMSFPGA IMPLEMENTATION OF NOISE CANCELLATION USING ADAPTIVE ALGORITHMS
FPGA IMPLEMENTATION OF NOISE CANCELLATION USING ADAPTIVE ALGORITHMSEditor IJMTER
 
Design of composite digital filter with least square method parameter identif...
Design of composite digital filter with least square method parameter identif...Design of composite digital filter with least square method parameter identif...
Design of composite digital filter with least square method parameter identif...IJRES Journal
 
ECG Noise cancelling
ECG Noise cancelling ECG Noise cancelling
ECG Noise cancelling salamy88
 
Machine learning basics using trees algorithm (Random forest, Gradient Boosting)
Machine learning basics using trees algorithm (Random forest, Gradient Boosting)Machine learning basics using trees algorithm (Random forest, Gradient Boosting)
Machine learning basics using trees algorithm (Random forest, Gradient Boosting)Parth Khare
 
Noice canclellation using adaptive filters with adpative algorithms(LMS,NLMS,...
Noice canclellation using adaptive filters with adpative algorithms(LMS,NLMS,...Noice canclellation using adaptive filters with adpative algorithms(LMS,NLMS,...
Noice canclellation using adaptive filters with adpative algorithms(LMS,NLMS,...Brati Sundar Nanda
 

En vedette (6)

FPGA IMPLEMENTATION OF NOISE CANCELLATION USING ADAPTIVE ALGORITHMS
FPGA IMPLEMENTATION OF NOISE CANCELLATION USING ADAPTIVE ALGORITHMSFPGA IMPLEMENTATION OF NOISE CANCELLATION USING ADAPTIVE ALGORITHMS
FPGA IMPLEMENTATION OF NOISE CANCELLATION USING ADAPTIVE ALGORITHMS
 
Design of composite digital filter with least square method parameter identif...
Design of composite digital filter with least square method parameter identif...Design of composite digital filter with least square method parameter identif...
Design of composite digital filter with least square method parameter identif...
 
Adaptive filters
Adaptive filtersAdaptive filters
Adaptive filters
 
ECG Noise cancelling
ECG Noise cancelling ECG Noise cancelling
ECG Noise cancelling
 
Machine learning basics using trees algorithm (Random forest, Gradient Boosting)
Machine learning basics using trees algorithm (Random forest, Gradient Boosting)Machine learning basics using trees algorithm (Random forest, Gradient Boosting)
Machine learning basics using trees algorithm (Random forest, Gradient Boosting)
 
Noice canclellation using adaptive filters with adpative algorithms(LMS,NLMS,...
Noice canclellation using adaptive filters with adpative algorithms(LMS,NLMS,...Noice canclellation using adaptive filters with adpative algorithms(LMS,NLMS,...
Noice canclellation using adaptive filters with adpative algorithms(LMS,NLMS,...
 

Similaire à Adding Statistical Functionality to the DATA Step with PROC FCMP

SAS Macros part 4.1
SAS Macros part 4.1SAS Macros part 4.1
SAS Macros part 4.1venkatam
 
11 Things About 11gr2
11 Things About 11gr211 Things About 11gr2
11 Things About 11gr2afa reg
 
Distributed Radar Tracking Simulation Project
Distributed Radar Tracking Simulation ProjectDistributed Radar Tracking Simulation Project
Distributed Radar Tracking Simulation ProjectAssignmentpedia
 
Distributed Radar Tracking Simulation Project
Distributed Radar Tracking Simulation ProjectDistributed Radar Tracking Simulation Project
Distributed Radar Tracking Simulation ProjectAssignmentpedia
 
Tony jambu (obscure) tools of the trade for tuning oracle sq ls
Tony jambu   (obscure) tools of the trade for tuning oracle sq lsTony jambu   (obscure) tools of the trade for tuning oracle sq ls
Tony jambu (obscure) tools of the trade for tuning oracle sq lsInSync Conference
 
Easing the Complex with SPBench framework
Easing the Complex with SPBench frameworkEasing the Complex with SPBench framework
Easing the Complex with SPBench frameworkadriano1mg
 
Data mining with caret package
Data mining with caret packageData mining with caret package
Data mining with caret packageVivian S. Zhang
 
Simple API for XML
Simple API for XMLSimple API for XML
Simple API for XMLguest2556de
 
Building Web Interface On Rails
Building Web Interface On RailsBuilding Web Interface On Rails
Building Web Interface On RailsWen-Tien Chang
 
Bti1022 lab sheet 3
Bti1022 lab sheet 3Bti1022 lab sheet 3
Bti1022 lab sheet 3alish sha
 

Similaire à Adding Statistical Functionality to the DATA Step with PROC FCMP (20)

SAS Macros part 4.1
SAS Macros part 4.1SAS Macros part 4.1
SAS Macros part 4.1
 
R console
R consoleR console
R console
 
Chapter 2
Chapter 2Chapter 2
Chapter 2
 
Memory Manglement in Raku
Memory Manglement in RakuMemory Manglement in Raku
Memory Manglement in Raku
 
sas aeroplan sample
sas aeroplan samplesas aeroplan sample
sas aeroplan sample
 
Gps c
Gps cGps c
Gps c
 
Mysql:Operators
Mysql:OperatorsMysql:Operators
Mysql:Operators
 
MySQL Operators
MySQL OperatorsMySQL Operators
MySQL Operators
 
11 Things About 11gr2
11 Things About 11gr211 Things About 11gr2
11 Things About 11gr2
 
Distributed Radar Tracking Simulation Project
Distributed Radar Tracking Simulation ProjectDistributed Radar Tracking Simulation Project
Distributed Radar Tracking Simulation Project
 
Distributed Radar Tracking Simulation Project
Distributed Radar Tracking Simulation ProjectDistributed Radar Tracking Simulation Project
Distributed Radar Tracking Simulation Project
 
Tony jambu (obscure) tools of the trade for tuning oracle sq ls
Tony jambu   (obscure) tools of the trade for tuning oracle sq lsTony jambu   (obscure) tools of the trade for tuning oracle sq ls
Tony jambu (obscure) tools of the trade for tuning oracle sq ls
 
Performance
PerformancePerformance
Performance
 
Combres
CombresCombres
Combres
 
Easing the Complex with SPBench framework
Easing the Complex with SPBench frameworkEasing the Complex with SPBench framework
Easing the Complex with SPBench framework
 
Data mining with caret package
Data mining with caret packageData mining with caret package
Data mining with caret package
 
Simple API for XML
Simple API for XMLSimple API for XML
Simple API for XML
 
Building Web Interface On Rails
Building Web Interface On RailsBuilding Web Interface On Rails
Building Web Interface On Rails
 
My java file
My java fileMy java file
My java file
 
Bti1022 lab sheet 3
Bti1022 lab sheet 3Bti1022 lab sheet 3
Bti1022 lab sheet 3
 

Adding Statistical Functionality to the DATA Step with PROC FCMP

  • 1. Adding Statistical Functionality to the DATA Step with PROC FCMP Stacey Christian and Jacques Rioux SAS Institute Inc., Cary, NC Paper 326-2010
  • 2.
  • 3.
  • 4.
  • 5.
  • 6.
  • 7.
  • 8.
  • 9.
  • 10. Linear Approximation Subroutine function linear_approximation (ds_in $, var $, first_obs, last_obs); rc = run_macro(‘linear_approximation_macro’, ds_in, first_obs, last_obs, var, error); return(error); endsub;
  • 11.
  • 12. Recursive Technique: Results data _NULL_; call segment_topdown(&quot;sasuser.snp&quot;, &quot;work.segds_20&quot;, &quot;close&quot;, 1, 15116, 0.2); call segment_topdown(&quot;sasuser.snp&quot;, &quot;work.segds_15&quot;, &quot;close&quot;, 1, 15116, 0.15); run;
  • 13. Recursive Technique: Graphic Results 42 Piecewise Linear Segments
  • 14. Recursive Technique: Graphic Results 113 Piecewise Linear Segments
  • 15.
  • 16.
  • 17.
  • 18. Run_Regression Subroutine subroutine run_regression ( data $, dependent $, independent $, weight $, parmData $, parmArray[*]); outargs parmArray; array tmpArray[1] _temporary_; rc = RUN_MACRO ('run_regression_macro', data, parmData , dependent, independent, weight) ; rc = read_array(parmData, tmpArray); do i = 1 to dim(parmArray); parmArray[i] = tmpArray[1,i]; end; endsub;
  • 19.
  • 20.
  • 21.
  • 24.
  • 25. Score Function Macro %macro create_score_func_macro; proc transpose data =&paramds out=&paramds._t; var &independent; run; proc sql noprint; select trim(_NAME_) || &quot; * &quot; || strip(put(col1,BEST12.)) into: theScore separated by &quot; + &quot; from &paramds._t; select trim(_NAME_) into: theArgs separated by &quot; , &quot; from &paramds._t; quit; data _NULL_; set &paramds; call symputX (&quot;Intercept&quot;,intercept); run; <continued>
  • 26. Score Function Macro - continued proc fcmp outlib=&library..score; function &scoreFunc(&theArgs); return(&Intercept + &theScore); endsub; quit; %mend create_score_func_macro;
  • 27. Run Create Score Function data _NULL_; call create_score(&quot;work.mroz&quot;, &quot;lwage&quot;, &quot;educ exper age kidslt6 kidsge6&quot;, &quot;PredLWage_Full&quot;, &quot;sasuser.score&quot;); call create_score(&quot;work.mroz&quot;, &quot;lwage&quot;, &quot;educ exper age&quot;, &quot;PredLWage_NoKids&quot;, &quot;sasuser.score&quot;); run; data _NULL_; educ = 15; exper = 5; age = 30; kidslt6 = 2; kidsge6 = 1; PredWage_Full = exp(PredLWage_Full(educ, exper, age, kidslt6, kidsge6)); put PredWage_Full=; PredWage_NoKids = exp(PredLWage_NoKids(educ, exper, age)); put PredWage_NoKids=; run; PredWage_Full=3.4199679212 PredWage_NoKids=3.787216653
  • 28.
  • 29.
  • 30. Adding Statistical Functionality to the DATA Step with PROC FCMP Paper 326-2010

Notes de l'éditeur

  1. Copyright © 2010, SAS Institute Inc. All rights reserved.
  2. Copyright © 2010, SAS Institute Inc. All rights reserved.
  3. Copyright © 2010, SAS Institute Inc. All rights reserved.
  4. Copyright © 2010, SAS Institute Inc. All rights reserved.
  5. Copyright © 2010, SAS Institute Inc. All rights reserved.
  6. Copyright © 2010, SAS Institute Inc. All rights reserved.
  7. Copyright © 2010, SAS Institute Inc. All rights reserved.
  8. Copyright © 2010, SAS Institute Inc. All rights reserved.
  9. Copyright © 2010, SAS Institute Inc. All rights reserved.
  10. Copyright © 2010, SAS Institute Inc. All rights reserved.
  11. Copyright © 2010, SAS Institute Inc. All rights reserved.
  12. The SEGMENT_TOPDOWN subroutine was called from the DATA step on S&amp;P 500 data from1950 to 2010. Data contained 15,116 observations Copyright © 2010, SAS Institute Inc. All rights reserved.
  13. 42 segments (with 20% error reduction threshold) Copyright © 2010, SAS Institute Inc. All rights reserved.
  14. 113 segments (with 15% error reduction threshold) Copyright © 2010, SAS Institute Inc. All rights reserved.
  15. Copyright © 2010, SAS Institute Inc. All rights reserved.
  16. Copyright © 2010, SAS Institute Inc. All rights reserved.
  17. Copyright © 2010, SAS Institute Inc. All rights reserved.
  18. Copyright © 2010, SAS Institute Inc. All rights reserved.
  19. Copyright © 2010, SAS Institute Inc. All rights reserved.
  20. Copyright © 2010, SAS Institute Inc. All rights reserved.
  21. Note we ran in fcmp instead of data step Copyright © 2010, SAS Institute Inc. All rights reserved.
  22. Copyright © 2010, SAS Institute Inc. All rights reserved.
  23. Copyright © 2010, SAS Institute Inc. All rights reserved.
  24. Copyright © 2010, SAS Institute Inc. All rights reserved.
  25. Copyright © 2010, SAS Institute Inc. All rights reserved.
  26. Copyright © 2010, SAS Institute Inc. All rights reserved.
  27. Copyright © 2010, SAS Institute Inc. All rights reserved.
  28. Copyright © 2010, SAS Institute Inc. All rights reserved.
  29. Copyright © 2010, SAS Institute Inc. All rights reserved.
  30. Copyright © 2010, SAS Institute Inc. All rights reserved.