{-# LANGUAGE Rank2Types, TemplateHaskell, BangPatterns, MultiParamTypeClasses, FunctionalDependencies, FlexibleInstances, UndecidableInstances, ScopedTypeVariables #-}------------------------------------------------------------------------------- |-- Module : Numeric.AD.Mode.Reverse-- Copyright : (c) Edward Kmett 2010-- License : BSD3-- Maintainer : ekmett@gmail.com-- Stability : experimental-- Portability : GHC only---- Mixed-Mode Automatic Differentiation.---- For reverse mode AD we use 'System.Mem.StableName.StableName' to recover sharing information from-- the tape to avoid combinatorial explosion, and thus run asymptotically faster-- than it could without such sharing information, but the use of side-effects-- contained herein is benign.-------------------------------------------------------------------------------moduleNumeric.AD.Mode.Reverse(-- * Gradientgrad,grad',gradWith,gradWith'-- * Jacobian,jacobian,jacobian',jacobianWith,jacobianWith'-- * Hessian,hessian,hessianF-- * Derivatives,diff,diff',diffF,diffF'-- * Unsafe Variadic Gradient,vgrad,vgrad',Grad)whereimportControl.Applicative((<$>))importData.Traversable(Traversable)importNumeric.AD.TypesimportNumeric.AD.Internal.ClassesimportNumeric.AD.Internal.CompositionimportNumeric.AD.Internal.Reverse-- | The 'grad' function calculates the gradient of a non-scalar-to-scalar function with 'Reverse' AD in a single pass.grad::(Traversablef,Numa)=>(foralls.Modes=>f(ADsa)->ADsa)->fa->fagradfas=unbindvs(partialArraybds$fvs)where(vs,bds)=bindas{-# INLINE grad #-}-- | The 'grad'' function calculates the result and gradient of a non-scalar-to-scalar function with 'Reverse' AD in a single pass.grad'::(Traversablef,Numa)=>(foralls.Modes=>f(ADsa)->ADsa)->fa->(a,fa)grad'fas=(primalr,unbindvs$partialArraybdsr)where(vs,bds)=bindasr=fvs{-# INLINE grad' #-}-- | @'grad' g f@ function calculates the gradient of a non-scalar-to-scalar function @f@ with reverse-mode AD in a single pass.-- The gradient is combined element-wise with the argument using the function @g@.---- > grad == gradWith (\_ dx -> dx)-- > id == gradWith constgradWith::(Traversablef,Numa)=>(a->a->b)->(foralls.Modes=>f(ADsa)->ADsa)->fa->fbgradWithgfas=unbindWithgvs(partialArraybds$fvs)where(vs,bds)=bindas{-# INLINE gradWith #-}-- | @'grad'' g f@ calculates the result and gradient of a non-scalar-to-scalar function @f@ with 'Reverse' AD in a single pass-- the gradient is combined element-wise with the argument using the function @g@.---- > grad' == gradWith' (\_ dx -> dx)gradWith'::(Traversablef,Numa)=>(a->a->b)->(foralls.Modes=>f(ADsa)->ADsa)->fa->(a,fb)gradWith'gfas=(primalr,unbindWithgvs$partialArraybdsr)where(vs,bds)=bindasr=fvs{-# INLINE gradWith' #-}-- | The 'jacobian' function calculates the jacobian of a non-scalar-to-non-scalar function with reverse AD lazily in @m@ passes for @m@ outputs.jacobian::(Traversablef,Functorg,Numa)=>(foralls.Modes=>f(ADsa)->g(ADsa))->fa->g(fa)jacobianfas=unbindvs.partialArraybds<$>fvswhere(vs,bds)=bindas{-# INLINE jacobian #-}-- | The 'jacobian'' function calculates both the result and the Jacobian of a nonscalar-to-nonscalar function, using @m@ invocations of reverse AD,-- where @m@ is the output dimensionality. Applying @fmap snd@ to the result will recover the result of 'jacobian'-- | An alias for 'gradF''jacobian'::(Traversablef,Functorg,Numa)=>(foralls.Modes=>f(ADsa)->g(ADsa))->fa->g(a,fa)jacobian'fas=row<$>fvswhere(vs,bds)=bindasrowa=(primala,unbindvs(partialArraybdsa)){-# INLINE jacobian' #-}-- | 'jacobianWith g f' calculates the Jacobian of a non-scalar-to-non-scalar function @f@ with reverse AD lazily in @m@ passes for @m@ outputs.---- Instead of returning the Jacobian matrix, the elements of the matrix are combined with the input using the @g@.---- > jacobian == jacobianWith (\_ dx -> dx)-- > jacobianWith const == (\f x -> const x <$> f x)--jacobianWith::(Traversablef,Functorg,Numa)=>(a->a->b)->(foralls.Modes=>f(ADsa)->g(ADsa))->fa->g(fb)jacobianWithgfas=unbindWithgvs.partialArraybds<$>fvswhere(vs,bds)=bindas{-# INLINE jacobianWith #-}-- | 'jacobianWith' g f' calculates both the result and the Jacobian of a nonscalar-to-nonscalar function @f@, using @m@ invocations of reverse AD,-- where @m@ is the output dimensionality. Applying @fmap snd@ to the result will recover the result of 'jacobianWith'---- Instead of returning the Jacobian matrix, the elements of the matrix are combined with the input using the @g@.---- > jacobian' == jacobianWith' (\_ dx -> dx)--jacobianWith'::(Traversablef,Functorg,Numa)=>(a->a->b)->(foralls.Modes=>f(ADsa)->g(ADsa))->fa->g(a,fb)jacobianWith'gfas=row<$>fvswhere(vs,bds)=bindasrowa=(primala,unbindWithgvs(partialArraybdsa)){-# INLINE jacobianWith' #-}diff::Numa=>(foralls.Modes=>ADsa->ADsa)->a->adifffa=derivative$f(vara0){-# INLINE diff #-}-- | The 'd'' function calculates the value and derivative, as a-- pair, of a scalar-to-scalar function.diff'::Numa=>(foralls.Modes=>ADsa->ADsa)->a->(a,a)diff'fa=derivative'$f(vara0){-# INLINE diff' #-}diffF::(Functorf,Numa)=>(foralls.Modes=>ADsa->f(ADsa))->a->fadiffFfa=derivative<$>f(vara0){-# INLINE diffF #-}diffF'::(Functorf,Numa)=>(foralls.Modes=>ADsa->f(ADsa))->a->f(a,a)diffF'fa=derivative'<$>f(vara0){-# INLINE diffF' #-}-- | Compute the hessian via the jacobian of the gradient. gradient is computed in reverse mode and then the jacobian is computed in reverse mode.---- However, since the @'grad f :: f a -> f a'@ is square this is not as fast as using the forward-mode Jacobian of a reverse mode gradient provided by 'Numeric.AD.hessian'.hessian::(Traversablef,Numa)=>(foralls.Modes=>f(ADsa)->ADsa)->fa->f(fa)hessianf=jacobian(grad(decomposeMode.f.fmapcomposeMode))-- | Compute the order 3 Hessian tensor on a non-scalar-to-non-scalar function via the reverse-mode Jacobian of the reverse-mode Jacobian of the function.---- Less efficient than 'Numeric.AD.Mode.Mixed.hessianF'.hessianF::(Traversablef,Functorg,Numa)=>(foralls.Modes=>f(ADsa)->g(ADsa))->fa->g(f(fa))hessianFf=decomposeFunctor.jacobian(ComposeFunctor.jacobian(fmapdecomposeMode.f.fmapcomposeMode))