Description

For courses in Probability and Random Processes.

Probability and Random Processes with Applications to Signal Processing, 4/e is a comprehensive treatment of probability and random processes that, more than any other available source, combines rigor with accessibility. Beginning with the fundamentals of probability theory and requiring only college-level calculus, the book develops all the tools needed to understand more advanced topics such as random sequences, continuous-time random processes, and statistical signal processing. The book progresses at a leisurely pace, never assuming more knowledge than contained in the material already covered. Rigor is established by developing all results from the basic axioms and carefully defining and discussing such advanced notions as stochastic convergence, stochastic integrals and resolution of stochastic processes.

Table of Contents

Preface

1 Introduction to Probability 1

1.1 Introduction: Why Study Probability? 1

1.2 The Different Kinds of Probability 2

Probability as Intuition 2

Probability as the Ratio of Favorable to Total Outcomes (Classical Theory) 3

Probability as a Measure of Frequency of Occurrence 4

Probability Based on an Axiomatic Theory 5

1.3 Misuses, Miscalculations, and Paradoxes in Probability 7

1.4 Sets, Fields, and Events 8

Examples of Sample Spaces 8

1.5 Axiomatic Definition of Probability 15

1.6 Joint, Conditional, and Total Probabilities; Independence 20

Compound Experiments 23

1.7 Bayes’ Theorem and Applications 35

1.8 Combinatorics 38

Occupancy Problems 42

Extensions and Applications 46

1.9 Bernoulli Trials–Binomial and Multinomial Probability Laws 48

Multinomial Probability Law 54

1.10 Asymptotic Behavior of the Binomial Law: The Poisson Law 57

1.11 Normal Approximation to the Binomial Law 63

Summary 65

Problems 66

References 77

2 Random Variables 79

2.1 Introduction 79

2.2 Definition of a Random Variable 80

2.3 Cumulative Distribution Function 83

Properties of FX(x) 84

Computation of FX(x) 85

2.4 Probability Density Function (pdf) 88

Four Other Common Density Functions 95

More Advanced Density Functions 97

2.5 Continuous, Discrete, and Mixed Random Variables 100

Some Common Discrete Random Variables 102

2.6 Conditional and Joint Distributions and Densities 107

Properties of Joint CDF FXY (x, y) 118

2.7 Failure Rates 137

Summary 141

Problems 141

References 149

Additional Reading 149

3 Functions of Random Variables 151

3.1 Introduction 151

Functions of a Random Variable (FRV): Several Views 154

3.2 Solving Problems of the Type Y = g(X) 155

General Formula of Determining the pdf of Y = g(X) 166

3.3 Solving Problems of the Type Z = g(X, Y ) 171

3.4 Solving Problems of the Type V = g(X, Y ), W = h(X, Y ) 193

Fundamental Problem 193

Obtaining fVW Directly from fXY 196

3.5 Additional Examples 200

Summary 205

Problems 206

References 214

Additional Reading 214

4 Expectation and Moments 215

4.1 Expected Value of a Random Variable 215

On the Validity of Equation 4.1-8 218

4.2 Conditional Expectations 232

Conditional Expectation as a Random Variable 239

4.3 Moments of Random Variables 242

Joint Moments 246

Properties of Uncorrelated Random Variables 248

Jointly Gaussian Random Variables 251

4.4 Chebyshev and Schwarz Inequalities 255

Markov Inequality 257

The Schwarz Inequality 258

4.5 Moment-Generating Functions 261

4.6 Chernoff Bound 264

4.7 Characteristic Functions 266

Joint Characteristic Functions 273

The Central Limit Theorem 276

4.8 Additional Examples 281

Summary 283

Problems 284

References 293

Additional Reading 294

5 Random Vectors 295

5.1 Joint Distribution and Densities 295

5.2 Multiple Transformation of Random Variables 299

5.3 Ordered Random Variables 302

Distribution of area random variables 305

5.4 Expectation Vectors and Covariance Matrices 311

5.5 Properties of Covariance Matrices 314

Whitening Transformation 318

5.6 The Multidimensional Gaussian (Normal) Law 319

5.7 Characteristic Functions of Random Vectors 328

Properties of CF of Random Vectors 330

The Characteristic Function of the Gaussian (Normal) Law 331

Summary 332

Problems 333

References 339

Additional Reading 339

6 Statistics: Part 1 Parameter Estimation 340

6.1 Introduction 340

Independent, Identically Distributed (i.i.d.) Observations 341

Estimation of Probabilities 343

6.2 Estimators 346

6.3 Estimation of the Mean 348

Properties of the Mean-Estimator Function (MEF) 349

Procedure for Getting a δ-confidence Interval on the Mean of a Normal

Random Variable When σX Is Known 352

Confidence Interval for the Mean of a Normal Distribution When σX Is Not

Known 352

Procedure for Getting a δ-Confidence Interval Based on n Observations on