Transcription

2 I. INRODUCION One relaively new approach o financial raing is o use machine learning algorihms o preic he rise an fall of asse prices before hey occur. An opimal raer woul buy an asse before he price rises, an sell he asse before is value eclines. or his projec, an asse raer will be implemene using recurren reinforcemen learning (RRL). he algorihm an is parameers are from a paper wrien by Mooy an Saffell. I is a graien ascen algorihm which aemps o maximize a uiliy funcion known as Sharpe s raio. By choosing an opimal parameer w for he raer, we aemp o ake avanage of asse price changes. es examples of he asse raer s operaion, boh real-worl an conrive, are illusrae in he final secion. III. UILIY UNCION: SHARPE S RAIO One commonly use meric in financial engineering is Sharpe s raio. or a ime series of invesmen reurns, Sharpe s raio can be calculae as: Average( R ) S for inerval,..., Sanar Deviaion( R ) where R is he reurn on invesmen for raing perio. Inuiively, Sharpe s raio rewars invesmen sraegies ha rely on less volaile rens o make a profi. IV. RADER UNCION he raer will aemp o maximize Sharpe s raio for a given price ime series. or his projec, he raer funcion akes he form of a neuron: anh( w x ) where M is he number of ime series inpus o he raer, he parameer vecor x, r,..., r M,, an he reurn r p p. w M, he inpu Noe ha r is he ifference in value of he asse beween he curren perio an he previous perio. herefore, r is he reurn on one share of he asse bough a ime. Also, he funcion [, ] represens he raing posiion a ime. here are hree ypes of posiions ha can be hel: long, shor, or neural. A long posiion is when. In his case, he raer buys an asse a price p an hopes ha i appreciaes by perio. A shor posiion is when. In his case, he raer sells an asse which i oes no own a price p, wih he expecaion o prouce he shares a perio. If he price a is higher, hen he raer is force o buy a he higher price o fulfill he conrac. If he price a is lower, hen he raer has mae a profi. J Mooy, M Saffell, Learning o rae via Direc Reinforcemen, IEEE ransacions on Neural Neworks, Vol, No 4, July.

3 A neural posiion is when. In his case, he oucome a ime has no effec on he raer s profis. here will be neiher gain nor loss. hus, represens holings a perio. ha is, n shares are bough (long posiion) or sol (shor posiion), where is he maximum possible number of shares per ransacion. he reurn a ime, consiering he ecision, is: R r where is he cos for a ransacion a perio. If (i.e. no change in our invesmen his perio) hen here will be no ransacion penaly. Oherwise he penaly is proporional o he ifference in shares hel. he firs erm ( r ) is he reurn resuling from he invesmen ecision from he perio. or example, if shares, he ecision was o buy half he maximum allowe (. 5 ), an each share increase r 8 price unis, his erm woul be 8, he oal reurn profi (ignoring ransacion penalies incurre uring perio ). V. GRADIEN ASCEN Maximizing Sharpe s raio requires a graien ascen. irs, we efine our uiliy funcion using basic formulas from saisics for mean an variance: We have S E[ R ] A where A E[ R ] ( E[ R ]) B A R an B R hen we can ake he erivaive of S using he chain rule: S S A B A A A S B S A B A S B B S A A S B B he necessary parial erivaives of he reurn funcion are: r sgn( ) r r r sgn( ) hen, he parial erivaives an mus be calculae:

4 anh( w x ) ( anh( w x ) ) w x ( anh( w ) x M x ) w 3 Noe ha he erivaive is recurren an epens on all previous values of. his means ha o rain he parameers, we mus keep a recor of from he beginning of our ime series. Because sock aa is in he range of - samples, his slows own he graien ascen bu oes no presen an insurmounable compuaional buren. An alernaive is o use online learning an o approximae using only he previous erm, effecively making he algorihm a sochasic graien ascen as in Mooy & Saffell s paper. However, my chosen approach is o insea use he exac expressions as wrien above. Once he S erm has been calculae, he weighs are upae accoring o he graien ascen rule wi wi S. he process is repeae for N e ieraions, where N e is chosen o assure ha Sharpe s raio has converge. VI. RAINING he mos successful meho in my exploraion has been he following algorihm:. rain parameers w M using a hisorical winow of size. Use he opimal policy w o make real ime ecisions from o N preic 3. Afer N preic preicions are complee, repea sep one. Inuiively, he sock price has unerlying srucure ha is changing as a funcion of ime. Choosing large assumes he sock price s srucure oes no change much uring samples. In he ranom process example below, an N are large because he srucure of he process is consan. If long erm rens o no appear o preic ominae sock behavior, hen i makes sense o reuce, since shorer winows can be a beer soluion han raining on large amouns of pas hisory. or example, aa for he years IBM 98-6 migh no lea o a goo sraegy for use in Dec. 6. A more accurae policy woul likely resul from raining wih aa from 4-6. VII. EXAMPLE price, p() Sharpe' raio raining ieraion igure. raining resuls for auoregressive ranom process., 75 N e he firs example of raining a policy is execue on an auoregressive ranom process (ranomness by injecing Gaussian noise ino couple equaions). In figure, he op graph is he generae price series. he boom graph is Sharpe s raio on he ime series using he parameer w for each ieraion of raining. So, as raining progresses, we fin beer values of w unil we have achieve an opimum Sharpe s raio for he given aa.

5 hen, we use his opimal w parameer o form a preicion for he nex N preic aa samples, shown below: 4 igure. Preicion performance using opimal policy from raining. N preic As is apparen from he above graph, he raer is making ecisions base on he w parameer. Of course, w is subopimal for he ime series over his preice inerval, bu i oes beer han a monkey. Afer inervals our reurn woul be %. he nex experimen, presene in he same forma, is o preic real sock aa wih some precipious rops (Ciigroup): price series, p Sharpe's raio raining ieraion, N e igure 3. raining w on Ciigroup sock aa. 6

6 5 5 reurns, r (ecisions) percen gains (%) igure 4. r (op), (mile), an percenage profi (cumulaive) for Ciigroup. Noe ha alhough he general r ) wipes ou our gains aroun = 75. policy is goo, he precipious rop in price (ownwar spike in he recurren reinforcemen learner seems o work bes on socks ha are consan on average, ye flucuae up an own. In such a case, here is less worry abou a precipious rop like in he above example. Wih a relaively consan mean sock price, he reinforcemen learner is free o play he ups an owns. he recurren reinforcemen learner seems o work, alhough i is ricky o se up an verify. One imporan rick is o properly scale he reurn series aa o mean zero an variance one, or he neuron canno separae he resuling aa poins. VII. CONCLUSIONS he primary ifficulies wih his approach res in he fac ha cerain sock evens o no exhibi srucure. As seen in he secon example above, he reinforcemen learner oes no preic precipious rops in he sock price an is jus as vulnerable as a human. Perhaps i woul be more effecive if combine wih a mechanism o preic such precipious rops. Oher changes o he moel migh be incluing sock volumes as feaures ha coul help in preicing rises an falls. Aiionally, i woul be nice o augmen he moel o incorporae fixe ransacion coss, as well as less frequen ransacions. or example, a moel coul be creae ha learns from long perios of aa, bu only perioically makes a ecision. his woul reflec he case of a casual raer ha paricipaes in smaller volume raes wih fixe ransacion coss. Because i is oo expensive for small-ime invesors o rae every perio wih fixe ransacion coss, a moel wih a perioic rae sraegy woul more financially feasible for such users. I woul probably be worhwhile o ry aaping his moel o his sor of perioic raing an see he resuls. Gol, Carl, X raing via Recurren Reinforcemen Learning, Compuaional Inelligences for inancial Engineering, 3. Proceeings. 3 IEEE Inernaional Conference on. p March 3. Special hanks o Carl for avice on algorihm implemenaion.

Profi Tes Modelling in Life Assurance Using Spreadshees PROFIT TEST MODELLING IN LIFE ASSURANCE USING SPREADSHEETS PART ONE Erik Alm Peer Millingon 2004 Profi Tes Modelling in Life Assurance Using Spreadshees

Small enu Coss an Large Business Ccles: An Exension of ankiw oel * Hirana K Nah Deparmen of Economics an Inl. Business Sam Houson Sae Universi an ober Srecher Deparmen of General Business an Finance Sam

The Transpor Equaion Consider a fluid, flowing wih velociy, V, in a hin sraigh ube whose cross secion will be denoed by A. Suppose he fluid conains a conaminan whose concenraion a posiion a ime will be

Profi Tes Modelling in Life Assurance Using Spreadshees, par wo PROFIT TEST MODELLING IN LIFE ASSURANCE USING SPREADSHEETS PART TWO Erik Alm Peer Millingon Profi Tes Modelling in Life Assurance Using Spreadshees,

Chaper 6 Ineres Raes and Bond Valuaion Definiion and Descripion of Bonds Long-erm deb-loosely, bonds wih a mauriy of one year or more Shor-erm deb-less han a year o mauriy, also called unfunded deb Bond-sricly

4. Inernaional ariy ondiions 4.1 urchasing ower ariy he urchasing ower ariy ( heory is one of he early heories of exchange rae deerminaion. his heory is based on he concep ha he demand for a counry's currency

Chaper 6: Business Valuaion (Income Approach) Cash flow deerminaion is one of he mos criical elemens o a business valuaion. Everyhing may be secondary. If cash flow is high, hen he value is high; if he

WHAT ARE OTION CONTRACTS? By rof. Ashok anekar An oion conrac is a derivaive which gives he righ o he holder of he conrac o do 'Somehing' bu wihou he obligaion o do ha 'Somehing'. The 'Somehing' can be

An empirical analysis abou forecasing Tmall air-condiioning sales using ime series model Yan Xia Deparmen of Mahemaics, Ocean Universiy of China, China Absrac Time series model is a hospo in he research

Individual Healh Insurance April 30, 2008 Pages 167-170 We have received feedback ha his secion of he e is confusing because some of he defined noaion is inconsisen wih comparable life insurance reserve

Supply Chain Managemen Using Simulaion Opimizaion By Miheer Kulkarni This problem was inspired by he paper by Jung, Blau, Pekny, Reklaii and Eversdyk which deals wih supply chain managemen for he chemical

Chaper 2 Problems 2.1 During a hard sneeze, your eyes migh shu for 0.5s. If you are driving a car a 90km/h during such a sneeze, how far does he car move during ha ime s = 90km 1000m h 1km 1h 3600s = 25m

Homework6 Soluions.7 In Problem hrough 4 use he mehod of variaion of parameers o find a paricular soluion of he given differenial equaion. Then check your answer by using he mehod of undeermined coeffiens..

Mah 344 May 4, Complex Fourier Series Par I: Inroducion The Fourier series represenaion for a funcion f of period P, f) = a + a k coskω) + b k sinkω), ω = π/p, ) can be expressed more simply using complex

Inroducion Chaper 14: Dynamic D-S dynamic model of aggregae and aggregae supply gives us more insigh ino how he economy works in he shor run. I is a simplified version of a DSGE model, used in cuing-edge

VII. THE FIRM'S INVESTMENT DECISION UNDER CERTAINTY: CAPITAL BUDGETING AND RANKING OF NEW INVESTMENT PROJECTS The mos imporan decisions for a firm's managemen are is invesmen decisions. While i is surely

Chaper Review Quesions. xplain why he heory of purchasing power pariy is ofen referred o as he law of one price. urchasing ower ariy () is referred o as he law of one price because he deerminaion of he

Page 9 Design of Inducors and High Frequency Transformers Inducors sore energy, ransformers ransfer energy. This is he prime difference. The magneic cores are significanly differen for inducors and high

CHAPTER 2 Double Enry Sysem of Accouning Sysem of Accouning \ The following are he main sysem of accouning for recording he business ransacions: (a) Cash Sysem of Accouning. (b) Mercanile or Accrual Sysem

Financial Marke Microsrucure and Trading Algorihms M.Sc. in Economics and Business Adminisraion Specializaion in Applied Economics and Finance Deparmen of Finance Copenhagen Business School 2009 Jens Vallø

AP Calculus AB 1 Scoring Guidelines The College Board The College Board is a no-for-profi membership associaion whose mission is o connec sudens o college success and opporuniy. Founded in 1, he College

Chaper H Inducance and Transien Circuis Blinn College - Physics 2426 - Terry Honan As a consequence of Faraday's law a changing curren hrough one coil induces an EMF in anoher coil; his is known as muual

Two Comparmen Body Model and V d Terms by Jeff Sark In a one-comparmen model, we make wo imporan assumpions: (1) Linear pharmacokineics - By his, we mean ha eliminaion is firs order and ha pharmacokineic

13 Solving nonhomogeneous equaions: Variaion of he consans meho We are sill solving Ly = f, (1 where L is a linear ifferenial operaor wih consan coefficiens an f is a given funcion Togeher (1 is a linear

Chaper 7. esponse of Firs-Order L and C Circuis 7.1. The Naural esponse of an L Circui 7.2. The Naural esponse of an C Circui 7.3. The ep esponse of L and C Circuis 7.4. A General oluion for ep and Naural

Even and Odd Funcions 23.3 Inroducion In his Secion we examine how o obain Fourier series of periodic funcions which are eiher even or odd. We show ha he Fourier series for such funcions is considerabl