SlideShare une entreprise Scribd logo
1  sur  17
Télécharger pour lire hors ligne
(An attempt of)
          Introduction to ad-3.4,
an automatic differentiation library in Haskell


                  3/31/2013
             Ekmett study meeting
              in Shibuya, Tokyo

                    by Nebuta

Any comments or correction to the material are welcome
About myself
Nebuta (@nebutalab)               https://github.com/nebuta

My interest in softwares:
 Programming languages (Haskell, Scala, Ruby, etc)
 Image processing, data visualization, web design
 Brainstorming and lifehack methods that take advantage of IT, etc.

My research areas:
 A graduate student, studying biophysical chemistry and quantitative
 biology (2010−)
 Imaging live cells, analyzing microscopy images by Scala on ImageJ


Where my interest in Haskell came from:
 MATLAB、ImageJで細胞の顕微鏡画像の解析 (2010年) → MATLAB,
  Javaはいまいち使いづらい → Scalaっていうイケてる言語がある
  (2011年)
  → 関数型? → Haskell 面白い!(2011年)
ad-3.4, an automatic differentiation library
What you can do
   Differentiation of arbitrary mathematical functions
   Taylor expansion
   Calculation of gradient, Jacobian, and Hessian, etc.


Dependencies
   array (≥0.2 & <0.5), base (4.*), comonad (≥3),
   containers (≥0.2 & <0.6), data-reify (0.6.*), erf (2.0.*),
   free (≥3), mtl (≥2), reflection (≥1.1.6),
   tagged (≥0.4.2.1), template-haskell (≥2.5 & <2.9)

Installation
$ sudo cabal install ad simple-reflect
                                           記号で微分するのに使う
                                           For symbolic differentiation
How to use ad-3.4
            https://github.com/ekmett/ad/blob/master/README.markdown#examples


Differentiation of a single-variable scalar function
>> :m + Numeric.AD                       ※Derivative of a
>> diff sin 0                            trigonometric function
1.0
>> :m + Debug.SimpleReflect
>> diff sin x -- x :: Expr is defined in Debug.SimpleReflect
cos x * 1 Derivative with a symbol!
>> diff (x -> if x >= 0 then 1 else 0) 0
0.0 Not delta function nor undefined.

Gradient
>> grad ([x,y] -> exp (x * y)) [x,y]
[0 + (0 + y * (0 + exp (x * y) * 1)),0 + (0 + x * (0 + exp (x
* y) * 1))]
>> grad ([x,y] -> exp (x * y)) [1,1]
[2.718281828459045,2.718281828459045]
How to use (continued)

Taylor expansion
Prelude Numeric.AD Debug.SimpleReflect>                         take 3 $ taylor exp 0 d
[exp 0 * 1,1 * exp 0 * (1 * d / 1),(0 *                         exp 0 + 1 * exp 0 * 1) *
(1 * d / 1 * d / (1 + 1))]
Prelude Numeric.AD Debug.SimpleReflect>                         take 3 $ taylor exp x d
[exp x * 1,1 * exp x * (1 * d / 1),(0 *                         exp x + 1 * exp x * 1) *
(1 * d / 1 * d / (1 + 1))]
Prelude Numeric.AD Debug.SimpleReflect>                         take 3 $ taylor exp x 0
[exp x * 1,1 * exp x * (1 * 0 / 1),(0 *                         exp x + 1 * exp x * 1) *
(1 * 0 / 1 * 0 / (1 + 1))]

•Taylor expansion is an infinite list!               Taylor expansion (general)
•No simplification, and slow in higher order terms




                                                     Exponential function
How to use (continued)

Equality of functions
>> sin x ==       sin x
True
>> diff sin       x
cos x * 1
>> diff sin       x == cos x * 1
True
>> diff sin       x == cos x * 0.5 * 2
False
Cool! (no simplification, though...)




And so on.
Cf. Mechanism of automatic differentiation
Read a Wikipedia article     http://en.wikipedia.org/wiki/Automatic_differentiation



I don’t understand it yet.
(What’s the difference from symbolic differentiation?)


                                         ??


                                  要は、合成関数の微分を
                               機械的に順次適用していく、

      (f + g)’ = f’ + g’
                             という認識で良いかと思われる
                            It seems to be mechanical, successive
                           application of rules of differentiation for
                                     composite functions.
I’ll try to appreciate the type and class organization



注意:
Githubに上がっている最新バージョン
(https://github.com/ekmett/ad)は4.0で、
Hackage(http://hackage.haskell.org/package/ad-3.4)とは違います。
ライブラリの構造が若干違うようです。



cabal unpack ad-3.4 でver. 3.4のソースをダウンロードする
か、Hackageを見て下さい。


            I’ll use ad-3.4 on Hackage, not ad-4.0 on Github
Package structure
           http://new-hackage.haskell.org/package/ad-3.4
       ここでは複数のモードの実装のうち、デ
       フォルトのモードがインポート&再エクス
       ポートされている
           クラスの定義




     いろいろな自動微分の”モード”の実装



    型の定義
The starting point for exploration: diff function
Numeric.AD.Mode.Forward                      1変数スカラー関数の微分
{-# LANGUAGE Rank2Types #-}
diff :: Num a => (forall s. Mode s => AD s a -> AD s a) -> a -> a
diff f a = tangent $ apply f a
     微分対象の関数fは(AD s a->AD s a)型として表される

Numeric.AD.Internal.Forward
{-# LANGUAGE Rank2Types, TypeFamilies, DeriveDataTypeable,
TemplateHaskell, UndecidableInstances, BangPatterns #-}
tangent :: Num a => AD Forward a -> a
tangent (AD (Forward _ da)) = da
tangent _ = 0

bundle :: a -> a -> AD Forward a
bundle a da = AD (Forward a da)

apply :: Num a => (AD Forward a -> b) -> a -> b
apply f a = f (bundle a 1)
                 この部分の型:
                 AD Forward a

                      どうやらAD型が になるようだ。
AD type
Numeric.AD.Types




                              An instance of Mode class determines
                                          the behavior.

                   e.g.) AD (Forward 10 1) :: Num a => AD Forward a


  {-# LANGUAGE CPP #-}
  {-# LANGUAGE Rank2Types, GeneralizedNewtypeDeriving,
  TemplateHaskell, FlexibleContexts, FlexibleInstances,
  MultiParamTypeClasses, UndecidableInstances #-}

  {-# ANN module "HLint: ignore Eta reduce" #-}

  newtype AD f a = AD { runAD :: f a } deriving (Iso (f a), Lifted,
  Mode, Primal)
Classes that have AD type as an instance
Numeric.AD.Types           http://hackage.haskell.org/packages/archive/ad/3.4/doc/
                                     html/Numeric-AD-Types.html#t:AD




                                 例:1変数実数スカラー関数 y = f(x)

                                (fがLiftedのインスタンス &&
                                aがFloatingのインスタンス)のとき、
                                 AD f a 
                                はFloatingのインスタンス。




                                  diffの第一引数の型
                                    AD s a -> AD s a
                                  は、以下の型に適合する*
                                     Floating a => a -> a

                                        *こういう言い方が正確か分からないが。
Let’s look at some examples of values with AD type
adtest.hs
import   Numeric.AD
import   Numeric.AD.Types
import   Numeric.AD.Internal.Classes
import   Numeric.AD.Internal.Forward

f x = x + 3
g x | x > 0 = x
    | otherwise = 0

d = diff (g . f)
GHCi
*Main> :t (g . f)
(g . f) :: (Num c, Ord c) => c -> c
*Main> :t (g . f) :: (Num a, Ord a, Mode s) => AD s a -> AD s a
(g . f) :: (Num a, Ord a, Mode s) => AD s a -> AD s a
  :: (Num a, Ord a, Mode s) => AD s a -> AD s a
*Main> :t f
f :: Num a => a -> a
*Main> :t f :: (Num a, Lifted s) => AD s a -> AD s a
f :: (Num a, Lifted s) => AD s a -> AD s a
  :: (Num a, Lifted s) => AD s a -> AD s a
*Main> :t g
g :: (Num a, Ord a) => a -> a
*Main> :t d
d :: Integer -> Integer
Let’s play around with apply and tangent
GHCi
> :l adtest.hs
[1 of 1] Compiling Main             ( adtest.hs, interpreted )
Ok, modules loaded: Main.
[*Main]
> :t apply
apply :: Num a => (AD Forward a -> b) -> a -> b
[*Main]
> :t apply f
apply f :: Num a => a -> AD Forward a Since f :: Num t0 => t0 -> t0,
[*Main]                                b in the type of apply is restricted to
> :t apply f 0                         AD Forward a
apply f 0 :: Num a => AD Forward a
[*Main]
> apply f 0
3
[*Main]
> :t tangent
tangent :: Num a => AD Forward a -> a
[*Main]
> :t tangent $ apply f 0
tangent $ apply f 0 :: Num a => a
[*Main]
> tangent $ apply f 0
1
Lifted class and Mode class
Chain rule of differentiation
                                          http://hackage.haskell.org/packages/archive/ad/3.4/doc/
Wrap a (Num a => a) value into t             html/Numeric-AD-Internal-Classes.html#t:Mode




       Definition of different modes.
       Mode is a subclass of Lifted        http://hackage.haskell.org/packages/archive/ad/3.4/doc/
                                              html/Numeric-AD-Internal-Classes.html#t:Lifted




                                        e.g.) Forward is an instance of Lifted and Mode
Lifted class defines chain rules of differentiation
                                http://hackage.haskell.org/packages/archive/ad/3.4/doc/html/
                                      src/Numeric-AD-Internal-Classes.html#deriveLifted

deriveLifted (in Numeric.AD.Internal.Classes)というTemplate
Haskellの関数がLiftedのインスタンスを作ってくれる。

A part of deriveLifted
   exp1        = lift1_ exp const
   log1        = lift1 log recip1

   sin1        = lift1 sin cos1
   cos1        = lift1 cos $ negate1 . sin1
   tan1        = lift1 tan $ recip1 . square1 . cos1
deriveNumeric generates instances of Num, etc.
                                                       http://hackage.haskell.org/packages/archive/ad/3.4/doc/html/
                                                           src/Numeric-AD-Internal-Classes.html#deriveNumeric

 deriveNumeric (in Numeric.AD.Internal.Classes)

 --   | @'deriveNumeric' f g@ provides the following instances:
 --
 --   >   instance   ('Lifted'   $f,   'Num'   a,   'Enum' a) => 'Enum' ($g a)
 --   >   instance   ('Lifted'   $f,   'Num'   a,   'Eq' a) => 'Eq' ($g a)
 --   >   instance   ('Lifted'   $f,   'Num'   a,   'Ord' a) => 'Ord' ($g a)
 --   >   instance   ('Lifted'   $f,   'Num'   a,   'Bounded' a) => 'Bounded' ($g a)
 --
 --   >   instance   ('Lifted'   $f,   'Show' a) => 'Show' ($g a)
 --   >   instance   ('Lifted'   $f,   'Num' a) => 'Num' ($g a)
 --   >   instance   ('Lifted'   $f,   'Fractional' a) => 'Fractional' ($g a)
 --   >   instance   ('Lifted'   $f,   'Floating' a) => 'Floating' ($g a)
 --   >   instance   ('Lifted'   $f,   'RealFloat' a) => 'RealFloat' ($g a)
 --   >   instance   ('Lifted'   $f,   'RealFrac' a) => 'RealFrac' ($g a)
 --   >   instance   ('Lifted'   $f,   'Real' a) => 'Real' ($g a)

Contenu connexe

Tendances

C - aptitude3
C - aptitude3C - aptitude3
C - aptitude3Srikanth
 
Functions in python
Functions in pythonFunctions in python
Functions in pythonIlian Iliev
 
Python легко и просто. Красиво решаем повседневные задачи
Python легко и просто. Красиво решаем повседневные задачиPython легко и просто. Красиво решаем повседневные задачи
Python легко и просто. Красиво решаем повседневные задачиMaxim Kulsha
 
Arrry structure Stacks in data structure
Arrry structure Stacks  in data structureArrry structure Stacks  in data structure
Arrry structure Stacks in data structurelodhran-hayat
 
Python Programming: Data Structure
Python Programming: Data StructurePython Programming: Data Structure
Python Programming: Data StructureChan Shik Lim
 
iOS와 케라스의 만남
iOS와 케라스의 만남iOS와 케라스의 만남
iOS와 케라스의 만남Mijeong Jeon
 
Python Functions (PyAtl Beginners Night)
Python Functions (PyAtl Beginners Night)Python Functions (PyAtl Beginners Night)
Python Functions (PyAtl Beginners Night)Rick Copeland
 
프알못의 Keras 사용기
프알못의 Keras 사용기프알못의 Keras 사용기
프알못의 Keras 사용기Mijeong Jeon
 
yield and return (poor English ver)
yield and return (poor English ver)yield and return (poor English ver)
yield and return (poor English ver)bleis tift
 
Fp in scala part 1
Fp in scala part 1Fp in scala part 1
Fp in scala part 1Hang Zhao
 
Introduction to Monads in Scala (1)
Introduction to Monads in Scala (1)Introduction to Monads in Scala (1)
Introduction to Monads in Scala (1)stasimus
 
Fp in scala with adts part 2
Fp in scala with adts part 2Fp in scala with adts part 2
Fp in scala with adts part 2Hang Zhao
 
Fp in scala with adts
Fp in scala with adtsFp in scala with adts
Fp in scala with adtsHang Zhao
 
FP 201 - Unit 6
FP 201 - Unit 6FP 201 - Unit 6
FP 201 - Unit 6rohassanie
 
Idiomatic Javascript (ES5 to ES2015+)
Idiomatic Javascript (ES5 to ES2015+)Idiomatic Javascript (ES5 to ES2015+)
Idiomatic Javascript (ES5 to ES2015+)David Atchley
 
Swift 함수 커링 사용하기
Swift 함수 커링 사용하기Swift 함수 커링 사용하기
Swift 함수 커링 사용하기진성 오
 

Tendances (18)

Pointers+(2)
Pointers+(2)Pointers+(2)
Pointers+(2)
 
Fp201 unit4
Fp201 unit4Fp201 unit4
Fp201 unit4
 
C - aptitude3
C - aptitude3C - aptitude3
C - aptitude3
 
Functions in python
Functions in pythonFunctions in python
Functions in python
 
Python легко и просто. Красиво решаем повседневные задачи
Python легко и просто. Красиво решаем повседневные задачиPython легко и просто. Красиво решаем повседневные задачи
Python легко и просто. Красиво решаем повседневные задачи
 
Arrry structure Stacks in data structure
Arrry structure Stacks  in data structureArrry structure Stacks  in data structure
Arrry structure Stacks in data structure
 
Python Programming: Data Structure
Python Programming: Data StructurePython Programming: Data Structure
Python Programming: Data Structure
 
iOS와 케라스의 만남
iOS와 케라스의 만남iOS와 케라스의 만남
iOS와 케라스의 만남
 
Python Functions (PyAtl Beginners Night)
Python Functions (PyAtl Beginners Night)Python Functions (PyAtl Beginners Night)
Python Functions (PyAtl Beginners Night)
 
프알못의 Keras 사용기
프알못의 Keras 사용기프알못의 Keras 사용기
프알못의 Keras 사용기
 
yield and return (poor English ver)
yield and return (poor English ver)yield and return (poor English ver)
yield and return (poor English ver)
 
Fp in scala part 1
Fp in scala part 1Fp in scala part 1
Fp in scala part 1
 
Introduction to Monads in Scala (1)
Introduction to Monads in Scala (1)Introduction to Monads in Scala (1)
Introduction to Monads in Scala (1)
 
Fp in scala with adts part 2
Fp in scala with adts part 2Fp in scala with adts part 2
Fp in scala with adts part 2
 
Fp in scala with adts
Fp in scala with adtsFp in scala with adts
Fp in scala with adts
 
FP 201 - Unit 6
FP 201 - Unit 6FP 201 - Unit 6
FP 201 - Unit 6
 
Idiomatic Javascript (ES5 to ES2015+)
Idiomatic Javascript (ES5 to ES2015+)Idiomatic Javascript (ES5 to ES2015+)
Idiomatic Javascript (ES5 to ES2015+)
 
Swift 함수 커링 사용하기
Swift 함수 커링 사용하기Swift 함수 커링 사용하기
Swift 함수 커링 사용하기
 

Similaire à Introduction to ad-3.4, an automatic differentiation library in Haskell

Lecture 5: Functional Programming
Lecture 5: Functional ProgrammingLecture 5: Functional Programming
Lecture 5: Functional ProgrammingEelco Visser
 
Refactoring to Macros with Clojure
Refactoring to Macros with ClojureRefactoring to Macros with Clojure
Refactoring to Macros with ClojureDmitry Buzdin
 
Functional programming with Scala
Functional programming with ScalaFunctional programming with Scala
Functional programming with ScalaNeelkanth Sachdeva
 
Столпы функционального программирования для адептов ООП, Николай Мозговой
Столпы функционального программирования для адептов ООП, Николай МозговойСтолпы функционального программирования для адептов ООП, Николай Мозговой
Столпы функционального программирования для адептов ООП, Николай МозговойSigma Software
 
High-Performance Haskell
High-Performance HaskellHigh-Performance Haskell
High-Performance HaskellJohan Tibell
 
Introduction to Client-Side Javascript
Introduction to Client-Side JavascriptIntroduction to Client-Side Javascript
Introduction to Client-Side JavascriptJulie Iskander
 
Matlab Functions
Matlab FunctionsMatlab Functions
Matlab FunctionsUmer Azeem
 
Functional Programming With Scala
Functional Programming With ScalaFunctional Programming With Scala
Functional Programming With ScalaKnoldus Inc.
 
Mining Functional Patterns
Mining Functional PatternsMining Functional Patterns
Mining Functional PatternsDebasish Ghosh
 
Procedure Typing for Scala
Procedure Typing for ScalaProcedure Typing for Scala
Procedure Typing for Scalaakuklev
 
Big Data Day LA 2016/ Hadoop/ Spark/ Kafka track - Iterative Spark Developmen...
Big Data Day LA 2016/ Hadoop/ Spark/ Kafka track - Iterative Spark Developmen...Big Data Day LA 2016/ Hadoop/ Spark/ Kafka track - Iterative Spark Developmen...
Big Data Day LA 2016/ Hadoop/ Spark/ Kafka track - Iterative Spark Developmen...Data Con LA
 
From Java to Scala - advantages and possible risks
From Java to Scala - advantages and possible risksFrom Java to Scala - advantages and possible risks
From Java to Scala - advantages and possible risksSeniorDevOnly
 
T3chFest 2016 - The polyglot programmer
T3chFest 2016 - The polyglot programmerT3chFest 2016 - The polyglot programmer
T3chFest 2016 - The polyglot programmerDavid Muñoz Díaz
 

Similaire à Introduction to ad-3.4, an automatic differentiation library in Haskell (20)

Lecture 3
Lecture 3Lecture 3
Lecture 3
 
Functional object
Functional objectFunctional object
Functional object
 
Practical cats
Practical catsPractical cats
Practical cats
 
Lecture 5: Functional Programming
Lecture 5: Functional ProgrammingLecture 5: Functional Programming
Lecture 5: Functional Programming
 
Refactoring to Macros with Clojure
Refactoring to Macros with ClojureRefactoring to Macros with Clojure
Refactoring to Macros with Clojure
 
C# programming
C# programming C# programming
C# programming
 
Functional programming with Scala
Functional programming with ScalaFunctional programming with Scala
Functional programming with Scala
 
Spark workshop
Spark workshopSpark workshop
Spark workshop
 
Столпы функционального программирования для адептов ООП, Николай Мозговой
Столпы функционального программирования для адептов ООП, Николай МозговойСтолпы функционального программирования для адептов ООП, Николай Мозговой
Столпы функционального программирования для адептов ООП, Николай Мозговой
 
High-Performance Haskell
High-Performance HaskellHigh-Performance Haskell
High-Performance Haskell
 
Introduction to Client-Side Javascript
Introduction to Client-Side JavascriptIntroduction to Client-Side Javascript
Introduction to Client-Side Javascript
 
Matlab Functions
Matlab FunctionsMatlab Functions
Matlab Functions
 
Functional Programming With Scala
Functional Programming With ScalaFunctional Programming With Scala
Functional Programming With Scala
 
Cpp tutorial
Cpp tutorialCpp tutorial
Cpp tutorial
 
CppTutorial.ppt
CppTutorial.pptCppTutorial.ppt
CppTutorial.ppt
 
Mining Functional Patterns
Mining Functional PatternsMining Functional Patterns
Mining Functional Patterns
 
Procedure Typing for Scala
Procedure Typing for ScalaProcedure Typing for Scala
Procedure Typing for Scala
 
Big Data Day LA 2016/ Hadoop/ Spark/ Kafka track - Iterative Spark Developmen...
Big Data Day LA 2016/ Hadoop/ Spark/ Kafka track - Iterative Spark Developmen...Big Data Day LA 2016/ Hadoop/ Spark/ Kafka track - Iterative Spark Developmen...
Big Data Day LA 2016/ Hadoop/ Spark/ Kafka track - Iterative Spark Developmen...
 
From Java to Scala - advantages and possible risks
From Java to Scala - advantages and possible risksFrom Java to Scala - advantages and possible risks
From Java to Scala - advantages and possible risks
 
T3chFest 2016 - The polyglot programmer
T3chFest 2016 - The polyglot programmerT3chFest 2016 - The polyglot programmer
T3chFest 2016 - The polyglot programmer
 

Dernier

UiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPathCommunity
 
UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7DianaGray10
 
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Will Schroeder
 
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationIES VE
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfAijun Zhang
 
NIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopNIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopBachir Benyammi
 
Designing A Time bound resource download URL
Designing A Time bound resource download URLDesigning A Time bound resource download URL
Designing A Time bound resource download URLRuncy Oommen
 
Linked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesLinked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesDavid Newbury
 
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...DianaGray10
 
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfJamie (Taka) Wang
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024D Cloud Solutions
 
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UbiTrack UK
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioChristian Posta
 
Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Brian Pichman
 
Bird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemBird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemAsko Soukka
 
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDEADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDELiveplex
 
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IES VE
 
Introduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxIntroduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxMatsuo Lab
 

Dernier (20)

UiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation Developers
 
UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7
 
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
 
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve DecarbonizationUsing IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
Using IESVE for Loads, Sizing and Heat Pump Modeling to Achieve Decarbonization
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdf
 
NIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopNIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 Workshop
 
Designing A Time bound resource download URL
Designing A Time bound resource download URLDesigning A Time bound resource download URL
Designing A Time bound resource download URL
 
Linked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond OntologiesLinked Data in Production: Moving Beyond Ontologies
Linked Data in Production: Moving Beyond Ontologies
 
20150722 - AGV
20150722 - AGV20150722 - AGV
20150722 - AGV
 
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
Connector Corner: Extending LLM automation use cases with UiPath GenAI connec...
 
201610817 - edge part1
201610817 - edge part1201610817 - edge part1
201610817 - edge part1
 
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
activity_diagram_combine_v4_20190827.pdfactivity_diagram_combine_v4_20190827.pdf
 
Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024Artificial Intelligence & SEO Trends for 2024
Artificial Intelligence & SEO Trends for 2024
 
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
UWB Technology for Enhanced Indoor and Outdoor Positioning in Physiological M...
 
Comparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and IstioComparing Sidecar-less Service Mesh from Cilium and Istio
Comparing Sidecar-less Service Mesh from Cilium and Istio
 
Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )Building Your Own AI Instance (TBLC AI )
Building Your Own AI Instance (TBLC AI )
 
Bird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemBird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystem
 
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDEADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
 
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
IESVE Software for Florida Code Compliance Using ASHRAE 90.1-2019
 
Introduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxIntroduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptx
 

Introduction to ad-3.4, an automatic differentiation library in Haskell

  • 1. (An attempt of) Introduction to ad-3.4, an automatic differentiation library in Haskell 3/31/2013 Ekmett study meeting in Shibuya, Tokyo by Nebuta Any comments or correction to the material are welcome
  • 2. About myself Nebuta (@nebutalab) https://github.com/nebuta My interest in softwares: Programming languages (Haskell, Scala, Ruby, etc) Image processing, data visualization, web design Brainstorming and lifehack methods that take advantage of IT, etc. My research areas: A graduate student, studying biophysical chemistry and quantitative biology (2010−) Imaging live cells, analyzing microscopy images by Scala on ImageJ Where my interest in Haskell came from: MATLAB、ImageJで細胞の顕微鏡画像の解析 (2010年) → MATLAB, Javaはいまいち使いづらい → Scalaっていうイケてる言語がある (2011年) → 関数型? → Haskell 面白い!(2011年)
  • 3. ad-3.4, an automatic differentiation library What you can do Differentiation of arbitrary mathematical functions Taylor expansion Calculation of gradient, Jacobian, and Hessian, etc. Dependencies array (≥0.2 & <0.5), base (4.*), comonad (≥3), containers (≥0.2 & <0.6), data-reify (0.6.*), erf (2.0.*), free (≥3), mtl (≥2), reflection (≥1.1.6), tagged (≥0.4.2.1), template-haskell (≥2.5 & <2.9) Installation $ sudo cabal install ad simple-reflect 記号で微分するのに使う For symbolic differentiation
  • 4. How to use ad-3.4 https://github.com/ekmett/ad/blob/master/README.markdown#examples Differentiation of a single-variable scalar function >> :m + Numeric.AD ※Derivative of a >> diff sin 0 trigonometric function 1.0 >> :m + Debug.SimpleReflect >> diff sin x -- x :: Expr is defined in Debug.SimpleReflect cos x * 1 Derivative with a symbol! >> diff (x -> if x >= 0 then 1 else 0) 0 0.0 Not delta function nor undefined. Gradient >> grad ([x,y] -> exp (x * y)) [x,y] [0 + (0 + y * (0 + exp (x * y) * 1)),0 + (0 + x * (0 + exp (x * y) * 1))] >> grad ([x,y] -> exp (x * y)) [1,1] [2.718281828459045,2.718281828459045]
  • 5. How to use (continued) Taylor expansion Prelude Numeric.AD Debug.SimpleReflect> take 3 $ taylor exp 0 d [exp 0 * 1,1 * exp 0 * (1 * d / 1),(0 * exp 0 + 1 * exp 0 * 1) * (1 * d / 1 * d / (1 + 1))] Prelude Numeric.AD Debug.SimpleReflect> take 3 $ taylor exp x d [exp x * 1,1 * exp x * (1 * d / 1),(0 * exp x + 1 * exp x * 1) * (1 * d / 1 * d / (1 + 1))] Prelude Numeric.AD Debug.SimpleReflect> take 3 $ taylor exp x 0 [exp x * 1,1 * exp x * (1 * 0 / 1),(0 * exp x + 1 * exp x * 1) * (1 * 0 / 1 * 0 / (1 + 1))] •Taylor expansion is an infinite list! Taylor expansion (general) •No simplification, and slow in higher order terms Exponential function
  • 6. How to use (continued) Equality of functions >> sin x == sin x True >> diff sin x cos x * 1 >> diff sin x == cos x * 1 True >> diff sin x == cos x * 0.5 * 2 False Cool! (no simplification, though...) And so on.
  • 7. Cf. Mechanism of automatic differentiation Read a Wikipedia article http://en.wikipedia.org/wiki/Automatic_differentiation I don’t understand it yet. (What’s the difference from symbolic differentiation?) ?? 要は、合成関数の微分を 機械的に順次適用していく、 (f + g)’ = f’ + g’ という認識で良いかと思われる It seems to be mechanical, successive application of rules of differentiation for composite functions.
  • 8. I’ll try to appreciate the type and class organization 注意: Githubに上がっている最新バージョン (https://github.com/ekmett/ad)は4.0で、 Hackage(http://hackage.haskell.org/package/ad-3.4)とは違います。 ライブラリの構造が若干違うようです。 cabal unpack ad-3.4 でver. 3.4のソースをダウンロードする か、Hackageを見て下さい。 I’ll use ad-3.4 on Hackage, not ad-4.0 on Github
  • 9. Package structure http://new-hackage.haskell.org/package/ad-3.4 ここでは複数のモードの実装のうち、デ フォルトのモードがインポート&再エクス ポートされている クラスの定義 いろいろな自動微分の”モード”の実装 型の定義
  • 10. The starting point for exploration: diff function Numeric.AD.Mode.Forward 1変数スカラー関数の微分 {-# LANGUAGE Rank2Types #-} diff :: Num a => (forall s. Mode s => AD s a -> AD s a) -> a -> a diff f a = tangent $ apply f a 微分対象の関数fは(AD s a->AD s a)型として表される Numeric.AD.Internal.Forward {-# LANGUAGE Rank2Types, TypeFamilies, DeriveDataTypeable, TemplateHaskell, UndecidableInstances, BangPatterns #-} tangent :: Num a => AD Forward a -> a tangent (AD (Forward _ da)) = da tangent _ = 0 bundle :: a -> a -> AD Forward a bundle a da = AD (Forward a da) apply :: Num a => (AD Forward a -> b) -> a -> b apply f a = f (bundle a 1) この部分の型: AD Forward a どうやらAD型が になるようだ。
  • 11. AD type Numeric.AD.Types An instance of Mode class determines the behavior. e.g.) AD (Forward 10 1) :: Num a => AD Forward a {-# LANGUAGE CPP #-} {-# LANGUAGE Rank2Types, GeneralizedNewtypeDeriving, TemplateHaskell, FlexibleContexts, FlexibleInstances, MultiParamTypeClasses, UndecidableInstances #-} {-# ANN module "HLint: ignore Eta reduce" #-} newtype AD f a = AD { runAD :: f a } deriving (Iso (f a), Lifted, Mode, Primal)
  • 12. Classes that have AD type as an instance Numeric.AD.Types http://hackage.haskell.org/packages/archive/ad/3.4/doc/ html/Numeric-AD-Types.html#t:AD 例:1変数実数スカラー関数 y = f(x) (fがLiftedのインスタンス && aがFloatingのインスタンス)のとき、  AD f a  はFloatingのインスタンス。 diffの第一引数の型 AD s a -> AD s a は、以下の型に適合する* Floating a => a -> a *こういう言い方が正確か分からないが。
  • 13. Let’s look at some examples of values with AD type adtest.hs import Numeric.AD import Numeric.AD.Types import Numeric.AD.Internal.Classes import Numeric.AD.Internal.Forward f x = x + 3 g x | x > 0 = x | otherwise = 0 d = diff (g . f) GHCi *Main> :t (g . f) (g . f) :: (Num c, Ord c) => c -> c *Main> :t (g . f) :: (Num a, Ord a, Mode s) => AD s a -> AD s a (g . f) :: (Num a, Ord a, Mode s) => AD s a -> AD s a :: (Num a, Ord a, Mode s) => AD s a -> AD s a *Main> :t f f :: Num a => a -> a *Main> :t f :: (Num a, Lifted s) => AD s a -> AD s a f :: (Num a, Lifted s) => AD s a -> AD s a :: (Num a, Lifted s) => AD s a -> AD s a *Main> :t g g :: (Num a, Ord a) => a -> a *Main> :t d d :: Integer -> Integer
  • 14. Let’s play around with apply and tangent GHCi > :l adtest.hs [1 of 1] Compiling Main ( adtest.hs, interpreted ) Ok, modules loaded: Main. [*Main] > :t apply apply :: Num a => (AD Forward a -> b) -> a -> b [*Main] > :t apply f apply f :: Num a => a -> AD Forward a Since f :: Num t0 => t0 -> t0, [*Main] b in the type of apply is restricted to > :t apply f 0 AD Forward a apply f 0 :: Num a => AD Forward a [*Main] > apply f 0 3 [*Main] > :t tangent tangent :: Num a => AD Forward a -> a [*Main] > :t tangent $ apply f 0 tangent $ apply f 0 :: Num a => a [*Main] > tangent $ apply f 0 1
  • 15. Lifted class and Mode class Chain rule of differentiation http://hackage.haskell.org/packages/archive/ad/3.4/doc/ Wrap a (Num a => a) value into t html/Numeric-AD-Internal-Classes.html#t:Mode Definition of different modes. Mode is a subclass of Lifted http://hackage.haskell.org/packages/archive/ad/3.4/doc/ html/Numeric-AD-Internal-Classes.html#t:Lifted e.g.) Forward is an instance of Lifted and Mode
  • 16. Lifted class defines chain rules of differentiation http://hackage.haskell.org/packages/archive/ad/3.4/doc/html/ src/Numeric-AD-Internal-Classes.html#deriveLifted deriveLifted (in Numeric.AD.Internal.Classes)というTemplate Haskellの関数がLiftedのインスタンスを作ってくれる。 A part of deriveLifted exp1 = lift1_ exp const log1 = lift1 log recip1 sin1 = lift1 sin cos1 cos1 = lift1 cos $ negate1 . sin1 tan1 = lift1 tan $ recip1 . square1 . cos1
  • 17. deriveNumeric generates instances of Num, etc. http://hackage.haskell.org/packages/archive/ad/3.4/doc/html/ src/Numeric-AD-Internal-Classes.html#deriveNumeric deriveNumeric (in Numeric.AD.Internal.Classes) -- | @'deriveNumeric' f g@ provides the following instances: -- -- > instance ('Lifted' $f, 'Num' a, 'Enum' a) => 'Enum' ($g a) -- > instance ('Lifted' $f, 'Num' a, 'Eq' a) => 'Eq' ($g a) -- > instance ('Lifted' $f, 'Num' a, 'Ord' a) => 'Ord' ($g a) -- > instance ('Lifted' $f, 'Num' a, 'Bounded' a) => 'Bounded' ($g a) -- -- > instance ('Lifted' $f, 'Show' a) => 'Show' ($g a) -- > instance ('Lifted' $f, 'Num' a) => 'Num' ($g a) -- > instance ('Lifted' $f, 'Fractional' a) => 'Fractional' ($g a) -- > instance ('Lifted' $f, 'Floating' a) => 'Floating' ($g a) -- > instance ('Lifted' $f, 'RealFloat' a) => 'RealFloat' ($g a) -- > instance ('Lifted' $f, 'RealFrac' a) => 'RealFrac' ($g a) -- > instance ('Lifted' $f, 'Real' a) => 'Real' ($g a)
  • 18. Definition of deriveNumeric deriveNumeric :: ([Q Pred] -> [Q Pred]) -> Q Type -> Q [Dec] deriveNumeric f t = do members <- liftedMembers let keep n = nameBase n `elem` members xs <- lowerInstance keep ((classP ''Num [varA]:) . f) t `mapM` [''Enum, ''Eq, ''Ord, ''Bounded, ''Show] ys <- lowerInstance keep f t `mapM` [''Num, ''Fractional, ''Floating, ''RealFloat,''RealFrac, ''Real, ''Erf, ''InvErf] return (xs ++ ys) lowerInstance :: (Name -> Bool) -> ([Q Pred] -> [Q Pred]) -> Q Type -> Name -> Q Dec lowerInstance p f t n = do #ifdef OldClassI ClassI (ClassD _ _ _ _ ds) <- reify n #else ClassI (ClassD _ _ _ _ ds) _ <- reify n #endif instanceD (cxt (f [classP n [varA]])) (conT n `appT` (t `appT` varA)) (concatMap lower1 ds) where lower1 :: Dec -> [Q Dec] lower1 (SigD n' _) | p n'' = [valD (varP n') (normalB (varE n'')) []] where n'' = primed n' lower1 _ = [] primed n' = mkName $ base ++ [prime] This looks an important part, where but a bit difficult.... base = nameBase n' h = head base prime | isSymbol h || h `elem` "/*-<>" = '!' | otherwise = '1'
  • 19. A pitfall? >> :t diff ((**2) :: Floating a => a -> a) diff ((**2) :: Floating a => a -> a) :: Floating a => a -> a >> :t diff ((**2) :: Double -> Double) <interactive>:1:7: Couldn't match expected type `ad-3.4:Numeric.AD.Internal.Types.AD s a0' with actual type `Double' Expected type: ad-3.4:Numeric.AD.Internal.Types.AD s a0 -> ad-3.4:Numeric.AD.Internal.Types.AD s a0 Actual type: Double -> Double In the first argument of `diff', namely `((** 2) :: Double -> Double)' In the expression: diff ((** 2) :: Double -> Double) You need to keep functions polymorphic for differentiation. import Numeric.AD func x = x ** 2 func2 x = x ** 3 fs = [func, func2] test = map (f -> diff f 1.0) fs So, this does not compile. (there is a GHC extension to accept this, isn’t there..?)
  • 20. Summary An interesting library that shows how to use types and typeclasses effectively and beautifully. Sorry, but I’m still unclear about how Lifted actually executes differentiation It seems to be done by Num instance of AD f a, via instance (Lifted f, Num a) => Num (AD f a)... Things that might be improved for application • Simplification of terms • Improving the performance of higher order taylor expansion. • Is automatic integration possible, maybe? These might be a good practice for seasoned Haskellers (I’m not...)
  • 21. 感想 • Haskellならではのコードの簡潔さのおかげで、ソースの どこを追っていけば良いかはわりと明確。 • 他のEkmett氏のライブラリよりは具体的な対象がはっき りしている分、少しは分かりやすい。 • とはいっても、私には若干(相当?)手に余る感じで、 深いところまでは解説出来ませんでした。知らないGHC 拡張も多し。ただ、人に説明しようとすることで少しは 理解が進んで、嬉しい。