3: Summary Statistics

Transcription

1 3: Summary Statistics Notation Let s start by introducing some notation. Consider the following small data set: The symbol n represents the sample size (n = 0). The capital letter X denotes the variable. (This is more compact than using descriptive labels such as AGE, for instance.) x i represents the i th value of variable X. For the data that is above, x =, x = 4, and so on. The symbol Σ ( sigma ) indicates that you should add all the values for the variable. For example, Σx i = x + x + þ + x 0 = = 90. Measures of Central Location Mean When used without specification, mean refers to the arithmetic average of a data set. To calculate the mean, add all the values in the data set and divide by the number of observations. We distinguish between the population mean and the sample mean. The population mean μ ( mu ) is: μ = x + x + L + x N N Using more compact notation: μ N = xi where Σx i represents the sum of all values in the population and N represents the population size. Illustrative example. The data set populati.sav is a complete listing of a population. It has individuals. The sum of age values in this population is 7,703. Therefore, μ = = 9.5 years. 600 Page 3. (C:\data\StatPrimer\sumstats.doc, Last printed 8//006 6:8:00 PM)

2 The sample mean ( x ; pronounced x bar ) is: x = x + x + L + x n n In more succinct notation, x n = xi Illustrative example. The data set sample.sav on the prior page has Σx i = 90 and n = 0. Therefore x = (90) = 9 years. 0 The formulas for population mean μ and sample mean x are [almost] identical. Whether you are calculating a population mean or sample mean is based on whether data represent the entire population or a sample. In practice you should always assume you are working with a sample and not the entire population. Interpretation of the mean: The mean represents the gravitational center of a distribution. This is where the distribution would balance: The mean is like a seesaw. A small child can sit farther from the center of a seesaw in order to balance a large child sitting closer to the center. Similarly, a single value sitting far off the center of a distribution can have a profound effect on the mean. The mean also tells you three things that you might want to know. It is a reflection of:. An individual value drawn at random from the sample.. An individual value drawn at random from the population. 3. The population mean. Reporting the mean: The mean should be rounded before it is reported. In general, report the mean (and other summary statistics) with one decimal more than the recording of the initial data. For example, if AGE is recorded in years, report the mean age to the nearest tenth of a year. Included units of measure with your summary statistics. The mean of our sample is 9.0 years. Page 3. (C:\data\StatPrimer\sumstats.doc, Last printed 8//006 6:8:00 PM)

3 Median The median is a different kind of average. It is the value that is greater than or equal to half of the values in the data set. It is the middle point of the data set. The median is easy to determine if you first order the data from low to high. This is an ordered array. The distance from the lowest value in the ordered array to where a given point is situated in n + the array is called depth. The median has a depth of. o When n is even, this will fall between two values. Average these values to get the median. n + o When n is odd, the depth will fall right on a value. This will be your median. Illustrative example (populati.sav). Since n = 0 in the illustrative data, the median has a depth of (0 + ) / = 5.5. This places the median between the data points 7 and median When the depth of the median falls between two values, we average them to find the median. The median in this data set is (7 + 9) / = 7.5. Illustrative example (tumorsize.sav): MRI based estimates of the volume of 5 benign tumors (cubic centimeters) are: median The median has a depth of (5 + ) / = 3. Therefore, the median is 8. The median is resistant to outliers. For instance, if the biggest value in the above set had been 0 instead of, the median would still be 8. Page 3.3 (C:\data\StatPrimer\sumstats.doc, Last printed 8//006 6:8:00 PM)

4 Mode The mode is the most frequently occurring value in a data set. AGE Freq % % 4 9.4% % % % % % Mode (most frequent value) 0 8.4% % % % % 5 9.9% 6 3.0% 7 8.% % % Total % The mode in this dataset is 9 since it appears more frequently than any other value. Note: The mode is unreliable is all but large data sets. Page 3.4 (C:\data\StatPrimer\sumstats.doc, Last printed 8//006 6:8:00 PM)

5 Comparison of the Mean, Median, and Mode The mean, median, and mode are equivalent when the distribution is unimodal and symmetrical. However, with asymmetry, the median is approximately one-third the distance between the mean and mode: The mean, median, and mode offer different advantages and disadvantages. The mean offers the advantages of familiarity and efficiency. It also has advantages when making inferences about a population mean. On the downside, the mean is markedly influenced by extreme skewness and outliers. Under circumstances of extreme skewness, the median is a more stable. An often cited example of this advantage come when considering the salary of employees, where the salary of highly paid executives skews the average income toward a misleadingly high value. Another example is the average price of homes, in which case high priced homes skew the data in a positive direction. In such circumstances, the median is less likely to be misinterpreted, and is therefore the preferred measure of central location. You can judge the asymmetry of a distribution by comparing its mean and median. When the mean is greater than the median, the distribution has a positive skew. When the mean is about equal to the median, the distribution is symmetrical. When the mean is less than the median, the distribution has a negative skew: mean > median : positive skew mean median : symmetry mean < median : negative skew In summary, the mean is preferred when data are symmetrical and do not have outliers. In other instances, the median is often preferred measure of central location. Page 3.5 (C:\data\StatPrimer\sumstats.doc, Last printed 8//006 6:8:00 PM)

6 Measures of Spread Range One simple way to measure spread is to provide the smallest (minimum) and largest (maximum) values in the data set. The difference of these values is the sample range. range = maximum! minimum The range of the sample in the stemplot below is 5! 5 = The sample range is a crude and biased measure of spread. If you use the range, you should supplement it with at least one other measure of spread. Standard Deviation and Variance The standard deviation is the most common measure of spread. A deviation is the difference between a value and the mean: x i x. We take each deviation in a data set and square it: ( x i x). This makes the negative or positive sign associated with the deviation unimportant. We then sum the squared deviations to derive a statistic known as the sum of squares (SS): SS = ( xi x) The population variance is the average sum of squares: σ SS = N The sample variance is s SS = n The sample variance divides the sum of squares by n! instead of n. When n is large, n! n, so the numerical results from the two formulas will be similar. However, when n is small, the sample variance formula will give a bigger result than the population variance formula. This is necessary to derive an unbiased estimate of the population variance. The number n is called the degree of freedom of the variance. We lose one degree of freedom when using x to estimate μ. The sample standard deviation is the square root of the variance: Page 3.6 (C:\data\StatPrimer\sumstats.doc, Last printed 8//006 6:8:00 PM)

7 s = s The one-step formula for the standard deviation is s = ( xi n x) Illustrative example. Recall the 0 ages in SAMPLE.SAV: The sample mean x is 9.0. Keep things tidy when calculating the standard deviation: Values Deviations Squared Deviation 9 = 8 8 = = 3 3 = = 4 4 = = 8 8 = = = = = = = = = = 5 5 = = 3 3 = 59 Sum of values = 90 Sum of Deviations = 0 Sum of Squares = 34 SS 34 The sample variance is s = = = 37. years. The standard deviation n 0 s = s Notes: = 37. = 5.398years. Round this to 5.4 years.. Always report the standard deviation when you report the mean.. When all values in the data set are the same, there is no spread and standard deviation s = 0. In all other instances, the standard deviation is a positive value. When s is large, there is a lot of spread in the data. 3. Standard deviation s has the same units as the data. This is not true of variance s, which has units squared. 4. The standard deviation is sensitive to outlier and skews. 5. When the data are Normal ( bell-shaped ) 68% of values lie within standard deviation of the mean, 95% of values lie within standard deviations of the mean, and 99.7% of values lie within 3 standard deviations of the mean. 6. For distributions that are not Normal, Chebychev s rule says at least ¾ of the values lie within standard deviations of the mean. Page 3.7 (C:\data\StatPrimer\sumstats.doc, Last printed 8//006 6:8:00 PM)

8 The Interquartile Range The first quartile (Q) is one-quarter up the ordered array. Q is also called the 5 th percentile because it is larger than 5% of the observations. The third quartile (Q3) is three-quarters up the array. Q3 is also called the 75 th percentile, because it is larger than 75% of the observations. When the data set is large (n $ 00), it is easy to find the quartiles. With small data sets, the exact location of quartiles must be interpolated. We use Tukey s hinges to interpolate quartiles. To determine hinges: (A) Put the data in rank order and locate the median of the data set. (B) Divide the data set into two groups: a low group and a high group. When n is odd, the median should be placed in both groups. (C) Find the middle value (median) of the low group. This is Q. (D) Find the middle value (median) of the high group. This is Q3. Once the quartiles have been determined, calculate the inter-quartile range (IQR). IQR = Q3 Q A good summary of locations in the distribution is provided by the points that divide the data it into four equally-sized groups. This is the 5-point summary is made of: Q0 Quartile 0 (the minimum) Q Quartile (bigger than 5% of the data points) Q Quartile (the median) Q3 Quartile 3 (bigger than 75% of the data points) Q4 Quartile 4 (the maximum) Illustrative example #. The ordered array for data in this sample is: The low group of {5,,, 4, 7} has a middle value (Q) of. The high group of {8, 30, 4, 50, 5} has a middle value (Q3) of 4. Thus, the five-point summary for these data is 5,, 7.5, 4, 5. The IQR Q3! Q = 4 =. Illustrative example #. Consider this new ordered array of n = The median is The low group consists of {.47,.06,.36, 3.43}. Notice that the median appears in both the low group and the high group Q is the average of the adjacent middle values of.06 and.36, which is ( ) =..The high group is {3.43, 3.74, 3.78, 3.94}. Q3 is the average of the adjacent middle values of 3.74 and 3.78, which is ( ) = The five point summary is.47,., 3.43, 3.76, 3.95The IQR = Q3! Q = =.55 Page 3.8 (C:\data\StatPrimer\sumstats.doc, Last printed 8//006 6:8:00 PM)

9 Boxplots Box-and-Whiskers Plot Box-and-whiskers plots display five-point summaries and potential outliers in graphical form. To constuct a boxplot. Determine the 5-point summary for the data.. Draw on graph paper a box extending from Q to Q3. 3. Inside the box, draw a line that locates the median. 4. Calculate the interquartile range (IQR = Q3 Q) 5. Calculate fences.5 hinge-spreads below and above the hinges: a. The lower fence Fence Lower = Q (.5)(IQR) b. The upper fence Fence Upper = Q3 + (.5)(IQR) c. Do not plot these fences. 6. Any value above the upper fence is an upper outside value. Any values below the lower fence is a lower outside value. Plot these values separate points on the graph. 7. The largest value still inside the upper fence is called the upper inside value. The smallest value still inside the lower fence is the lower inside value. Drawn whiskers from the upper extent of the box (upper hinge) to the upper inside value. maximum, and from the lower extent of the box ( bottom hinge ) to the minimum. Page 3.9 (C:\data\StatPrimer\sumstats.doc, Last printed 8//006 6:8:00 PM)

10 Illustrative example (sample.sav). Recall the ordered array for AGE in sample.sav The 5-point summary is 5,, 7.5, 4, 5 (see prior analysis). The box extends from to A line in the box locates the median at The IQR = 4 =. 5. Fences a. F L = (.5)() =!0.5 b. F U = 4 + (.5)() = No value is more than 73.5, so there are no upper outside values. No value is less than 0.5, so there are no lower outside values. 7. Since there are no outside values, the upper inside value is the maximum (5) and the lower inside is the minimum (5). Whiskers are drawn from the hinges to the inside values. The boxplot is: Interpretation of boxplots: Boxplots show less detail than stemplots, but still provide insight into the central location, spread, and shape of a distribution. When you look at a boxplot, consider the following elements: Central location: The line in the box locates the median. In addition, the box locates the middle 50 percent of the data. Spread: The length of the box is called the hinge-spread. This corresponds to the IQR is a good quantifier of the distribution s spread. In addition, The whiskers from tip-to-tip (the whisker-spread ) quantifies the distribution s spread. The maximum and minimum are visible as well. Shape: Shape is difficult to judge except when the sample is large, in which case symmetry or lack of symmetry will be visible. Page 3.0 (C:\data\StatPrimer\sumstats.doc, Last printed 8//006 6:8:00 PM)

11 Illustrative Example #: Let us look at a new data set with values: The five-point summary is: 3,, 5.5, 9, 5. The box extends from to 9 3. The median is marked at The IQR ( hinge-spread ) = 9 = 7 5. Fence Upper = 9 + (.5)(7) = Fence Lower = (.5)(7) = There is one value outside the upper fence (5). There is one value outside of the lower fence (value of 3). These points are plotted separately. 7. The highest value still inside the upper fences is 3. The upper whisker is drawn from the upper hinge (9) to the upper inside value (3). The lowest value still inside the lower fence is, thus demarcating the lower whisker. The lower whisker extends from the lower hinge () to this lower-inside-value (). The boxplot looks like this: Page 3. (C:\data\StatPrimer\sumstats.doc, Last printed 8//006 6:8:00 PM)

HOSP 1207 (Business Stats) Learning Centre Describing Data This worksheet focuses on describing data through measuring its central tendency and variability. These measurements will give us an idea of what

Center: Finding the Median When we think of a typical value, we usually look for the center of the distribution. For a unimodal, symmetric distribution, it s easy to find the center it s just the center

10-3 Measures of Central Tendency and Variation So far, we have discussed some graphical methods of data description. Now, we will investigate how statements of central tendency and variation can be used.

MODE The mode of the sample is the value of the variable having the greatest frequency. Example: Obtain the mode for Data Set 1 77 For a grouped frequency distribution, the modal class is the class having

Exploratory Data Analysis Exploratory Data Analysis involves both graphical displays of data and numerical summaries of data. A common situation is for a data set to be represented as a matrix. There is

13.2 Measures of Central Tendency Measures of Central Tendency For a given set of numbers, it may be desirable to have a single number to serve as a kind of representative value around which all the numbers

Individuals: The objects that are described by a set of data. They may be people, animals, things, etc. (Also referred to as Cases or Records) Variables: The characteristics recorded about each individual.

Chapter 3: Data Description Numerical Methods Learning Objectives Upon successful completion of Chapter 3, you will be able to: Summarize data using measures of central tendency, such as the mean, median,

A frequency distribution is a table used to describe a data set. A frequency table lists intervals or ranges of data values called data classes together with the number of data values from the set that

STATS8: Introduction to Biostatistics Data Exploration Babak Shahbaba Department of Statistics, UCI Introduction After clearly defining the scientific problem, selecting a set of representative members

Chapter 3 Descriptive Statistics: Numerical Measures Slide 1 Learning objectives 1. Single variable Part I (Basic) 1.1. How to calculate and use the measures of location 1.. How to calculate and use the

BNG 202 Biomechanics Lab Descriptive statistics and probability distributions I Overview The overall goal of this short course in statistics is to provide an introduction to descriptive and inferential

Section 3.1 Measures of Central Tendency: Mode, Median, and Mean One number can be used to describe the entire sample or population. Such a number is called an average. There are many ways to compute averages,

Numerical Measures of Central Tendency Often, it is useful to have special numbers which summarize characteristics of a data set These numbers are called descriptive statistics or summary statistics. A

CHINHOYI UNIVERSITY OF TECHNOLOGY SCHOOL OF NATURAL SCIENCES AND MATHEMATICS DEPARTMENT OF MATHEMATICS MEASURES OF CENTRAL TENDENCY AND DISPERSION INTRODUCTION From the previous unit, the Graphical displays

Mathematics Probability and Statistics Curriculum Guide Revised 2010 This page is intentionally left blank. Introduction The Mathematics Curriculum Guide serves as a guide for teachers when planning instruction

Exploratory Data Analysis Psychology 3256 1 Introduction If you are going to find out anything about a data set you must first understand the data Basically getting a feel for you numbers Easier to find

3.1 Measures of central tendency: mode, median, mean, midrange Dana Lee Ling (2012) Mode The mode is the value that occurs most frequently in the data. Spreadsheet programs such as Microsoft Excel or OpenOffice.org

Chapter 3: Central Tendency Central Tendency In general terms, central tendency is a statistical measure that determines a single value that accurately describes the center of the distribution and represents

Biostatistics: DESCRIPTIVE STATISTICS: 2, VARIABILITY 1. Introduction Besides arriving at an appropriate expression of an average or consensus value for observations of a population, it is important to

Module 4: Data Exploration Now that you have your data downloaded from the Streams Project database, the detective work can begin! Before computing any advanced statistics, we will first use descriptive

Statistical Geophysics WS 2008/09 7..2008 Christian Heumann und Helmut Küchenhoff Measures for location and dispersion of a sample Measures for location and dispersion of a sample In the following: Variable

Learning objectives Descriptive Statistics F. Farrokhyar, MPhil, PhD, PDoc To recognize different types of variables To learn how to appropriately explore your data How to display data using graphs How

Descriptive statistics is the discipline of quantitatively describing the main features of a collection of data. Descriptive statistics are distinguished from inferential statistics (or inductive statistics),

Session 36 Five-Number Summary and Box Plots Interpret the information given in the following box-and-whisker plot. The results from a pre-test for students for the year 2000 and the year 2010 are illustrated

Northumberland Knowledge Know Guide How to Analyse Data - November 2012 - This page has been left blank 2 About this guide The Know Guides are a suite of documents that provide useful information about

Data Exploration Data Visualization What is data exploration? A preliminary exploration of the data to better understand its characteristics. Key motivations of data exploration include Helping to select

Numerical Summarization of Data OPRE 6301 Motivation... In the previous session, we used graphical techniques to describe data. For example: While this histogram provides useful insight, other interesting

Why Taking This Course? Course Introduction, Descriptive Statistics and Data Visualization GENOME 560, Spring 2012 Data are interesting because they help us understand the world Genomics: Massive Amounts

3.2 Measures of Spread In some data sets the observations are close together, while in others they are more spread out. In addition to measures of the center, it's often important to measure the spread

Descriptive Statistics Purpose of descriptive statistics Frequency distributions Measures of central tendency Measures of dispersion Statistics as a Tool for LIS Research Importance of statistics in research

AP Statistics Topic 9 ~ Measures of Spread Activity 9 : Baseball Lineups The table to the right contains data on the ages of the two teams involved in game of the 200 National League Division Series. Is

Chapter 2 Descriptive Statistics 2.1 Descriptive Statistics 1 2.1.1 Student Learning Objectives By the end of this chapter, the student should be able to: Display data graphically and interpret graphs:

Unit 2 Part 1: Measures of Central Tendency A small video recycling business had the following daily sales over a six- day period: $305, $285, $240, $376, $198, $264 A single number that is, in some sense

Introduction to Statistics for Psychology and Quantitative Methods for Human Sciences Jonathan Marchini Course Information There is website devoted to the course at http://www.stats.ox.ac.uk/ marchini/phs.html

WHAT IT IS Return to Table of ontents Descriptive statistics include the numbers, tables, charts, and graphs used to describe, organize, summarize, and present raw data. Descriptive statistics are most

4. DESCRIPTIVE STATISTICS Descriptive Statistics is a body of techniques for summarizing and presenting the essential information in a data set. Eg: Here are daily high temperatures for Jan 6, 29 in U.S.

A Few Sources for Data Examples Used Introduction to Environmental Statistics Professor Jessica Utts University of California, Irvine jutts@uci.edu 1. Statistical Methods in Water Resources by D.R. Helsel

Statistics for Engineers 4-1 4. Introduction to Statistics Descriptive Statistics Types of data A variate or random variable is a quantity or attribute whose value may vary from one unit of investigation

Data Analysis Plan The appropriate methods of data analysis are determined by your data types and variables of interest, the actual distribution of the variables, and the number of cases. Different analyses

GCSE HIGHER Statistics Key Facts Collecting Data When writing questions for questionnaires, always ensure that: 1. the question is worded so that it will allow the recipient to give you the information

Chapter 2. DATA EXPLORATION AND SUMMARIZATION 2.1 Frequency Distributions Commonly, people refer to a population as the number of individuals in a city or county, for example, all the people in California.

STATISTICS FOR PSYCH MATH REVIEW GUIDE ORDER OF OPERATIONS Although remembering the order of operations as BEDMAS may seem simple, it is definitely worth reviewing in a new context such as statistics formulae.

32 Measures of Central Tendency and Dispersion In this section we discuss two important aspects of data which are its center and its spread. The mean, median, and the mode are measures of central tendency

Summary of Formulas and Concepts Descriptive Statistics (Ch. 1-4) Definitions Population: The complete set of numerical information on a particular quantity in which an investigator is interested. We assume

NORMAL DISTRIBTIONS MEASURES OF VARIATION In statistics, it is important to measure the spread of data. A simple way to measure spread is to find the range. But statisticians want to know if the data are

Measures of Center Section 3-1 Mean (Arithmetic Mean) AVERAGE the number obtained by adding the values and dividing the total by the number of values 1 Mean as a Balance Point 3 Mean as a Balance Point

Lecture 6: Chapter 6: Normal Probability Distributions A normal distribution is a continuous probability distribution for a random variable x. The graph of a normal distribution is called the normal curve.

Interpreting Data in Normal Distributions This curve is kind of a big deal. It shows the distribution of a set of test scores, the results of rolling a die a million times, the heights of people on Earth,

Mathematics Box-and-Whisker Plots About this Lesson This is a foundational lesson for box-and-whisker plots (boxplots), a graphical tool used throughout statistics for displaying data. During the lesson,

Lesson 13 Main Idea Describe a data distribution by its center, spread, and overall shape. Relate the choice of center and spread to the shape of the distribution. New Vocabulary distribution symmetric

Chapter 4 Exploratory Data Analysis A first look at the data. As mentioned in Chapter 1, exploratory data analysis or EDA is a critical first step in analyzing the data from an experiment. Here are the

3. Since 4. The HOMEWORK 3 Due: Feb.3 1. A set of data are put in numerical order, and a statistic is calculated that divides the data set into two equal parts with one part below it and the other part

EXPLORING SPATIAL PATTERNS IN YOUR DATA OBJECTIVES Learn how to examine your data using the Geostatistical Analysis tools in ArcMap. Learn how to use descriptive statistics in ArcMap and Geoda to analyze

Descriptive Statistics 1 Descriptive Statistics and Measurement Scales Descriptive statistics are used to describe the basic features of the data in a study. They provide simple summaries about the sample

MEASURES OF CENTRAL TENDENCY MEASURES OF CENTRAL TENDENCY The measures of central tendency are numbers that locate the center of a set of data. The three most common measures of center are mean, median

1 Central Tendency CENTRAL TENDENCY: A statistical measure that identifies a single score that is most typical or representative of the entire group Usually, a value that reflects the middle of the distribution

Chapter 1: Exploring Data Chapter 1 Review 1. As part of survey of college students a researcher is interested in the variable class standing. She records a 1 if the student is a freshman, a 2 if the student

Probability and Statistics Vocabulary List (Definitions for Middle School Teachers) B Bar graph a diagram representing the frequency distribution for nominal or discrete data. It consists of a sequence

Session 1.6 Measures of Central Tendency Measures of location (Indices of central tendency) These indices locate the center of the frequency distribution curve. The mode, median, and mean are three indices

Name Class Date 9.1 Measures of Center and Spread Essential Question: How can you describe and compare data sets? Explore Exploring Data Resource Locker Caleb and Kim have bowled three games. Their scores

Week 1 Exploratory Data Analysis Practicalities This course ST903 has students from both the MSc in Financial Mathematics and the MSc in Statistics. Two lectures and one seminar/tutorial per week. Exam

Chapter 2 Overview Name MULTIPLE CHOICE. Choose the one alternative that best completes the statement or answers the question. Classify as categorical or qualitative data. 1) A survey of autos parked in

AP * Statistics Review Descriptive Statistics Teacher Packet Advanced Placement and AP are registered trademark of the College Entrance Examination Board. The College Board was not involved in the production