FORident Software Technical Paper - HemoSpat Validation

FORident Software, Inc.
09 August 2009

Introduction

The purpose of this study was to replicate the one performed in "Further Validation of the BackTrack Computer Program for Bloodstain Pattern Analysis - Precision and Accuracy" [1] to validate the accuracy of the HemoSpat bloodstain analysis software [2] against an accepted standard and to examine the reproducibility of the results. The Royal Canadian Mounted Police (RCMP) provided FORident Software the data for the 18 bloodstain targets from the original BackTrack study. The study of HemoSpat was initiated by FORident Software in October 2006, taken over by the RCMP in December 2006, and the analyses were completed in November 2007. The results show that the average distance from the known origin across patterns is in line with the original study. The standard deviations across patterns are well within the bounds laid out by the BackTrack study and shows that the results are reproducible given different analysts working with the same data.

Methodology

The images, stain locations, and known origins for the 18 patterns were provided to FORident Software by the RCMP. FORident Software created HemoSpat projects for each pattern, added the images, named the stains and the pattern, and entered the stain locations. These projects were provided, along with a copy of the software, to the participants. Nine analysts - each trained in bloodstain analysis - carried out the analysis of each of the 18 targets using HemoSpat and submitted their results. The analysts were from at least six different labs in Australia, Canada, and the United States [we do not know how many RCMP labs were involved]. Almost all of the analysts had previous experience with BackTrack [again we are not sure about the RCMP analysts], but were using HemoSpat for the first time without any training specific to HemoSpat.

Results

We will present our results in a similar manner as the BackTrack paper. Table 1 shows the average x, y, and z for each pattern along with their standard deviations. Standard deviations were calculated using the same formula as the BackTrack study [3]. The average distances between the known and calculated values in the x, y, and z directions were 2.8 cm, 2.0 cm, and 7.4 cm. Table 2 lists the known x, y, and z of the origins, the calculated x, y, and z averages, and the differences between the two.

Target #

xav/cm

S.D. (N=9)

yav/cm

S.D. (N=9)

zav/cm

S.D. (N=9)

1

41.7

2.77

114.7

0.33

24.3

0.72

2

27.5

0.70

176.7

0.28

14.1

0.55

3

35.7

1.33

49.7

0.34

91.4

1.01

4

20.9

0.92

78.7

0.49

31.3

0.44

5

45.2

1.15

82.3

0.58

48.4

0.61

6

33.7

1.03

87.2

0.40

58.8

0.24

7

20.2

1.47

55.2

0.38

115.0

0.30

8

24.7

1.76

82.0

1.01

115.5

0.77

9

26.9

1.73

79.9

1.09

123.5

0.26

10

24.5

2.23

131.0

1.51

83.6

1.05

11

35.8

2.29

241.1

1.07

119.3

2.73

12

17.9

1.50

27.0

0.61

118.2

2.57

13

30.5

1.23

272.2

0.90

115.3

0.32

14

29.4

1.13

138.8

1.22

93.6

0.67

15

25.7

0.87

244.5

1.19

99.6

0.63

16

56.1

1.73

127.3

1.38

125.0

0.57

17

34.2

1.48

102.4

0.81

106.7

0.47

18

14.2

0.69

140.6

0.25

49.6

0.17

Table 1. Average x, y, and z values calculated using HemoSpat with their standard deviations (N = 9). The numbers with a red background and in italic are the maximums and those with a green background in bold are the minimums.

Target #

x/cm

xav/cm

Δx/cm

y/cm

yav/cm

Δy/cm

z/cm

zav/cm

Δz/cm

1

46.0

41.7

4.3

115.0

114.7

0.3

12.0

24.3

12.3

2

30.0

27.5

2.5

176.0

176.7

0.7

2.5

14.1

11.6

3

38.0

35.7

2.3

50.0

49.7

0.3

81.3

91.4

10.1

4

21.0

20.9

0.1

80.0

78.7

1.3

32.0

31.3

0.7

5

48.0

45.2

2.8

83.0

82.3

0.7

41.0

48.4

7.4

6

36.0

33.7

2.3

89.0

87.2

1.8

52.0

58.8

6.8

7

24.0

20.2

3.8

52.5

55.2

2.7

114.0

115.0

1.0

8

28.0

24.7

3.3

82.5

82.0

0.5

114.0

115.5

1.5

9

31.0

26.9

4.1

80.0

79.9

0.1

121.0

123.5

2.5

10

27.0

24.5

2.5

139.0

131.0

8.0

75.0

83.6

8.6

11

40.0

35.8

4.2

243.0

241.1

1.9

105.0

119.3

14.3

12

20.0

17.9

2.1

23.0

27.0

4.0

105.0

118.2

13.2

13

26.0

30.5

4.5

275.0

272.2

2.8

116.0

115.3

0.7

14

25.0

29.4

4.4

145.0

138.8

6.2

93.0

93.6

0.6

15

25.0

25.7

0.7

245.0

244.5

0.5

95.0

99.6

4.6

16

61.1

56.1

5.0

129.5

127.3

2.2

104.5

125.0

20.5

17

34.4

34.2

0.2

102.0

102.4

0.4

90.7

106.7

16.0

18

13.5

14.2

0.7

142.0

140.6

1.4

48.8

49.6

0.8

Table 2. Known values for x, y, and z compared to the average calculated x, y, and z values for the origins. The average differences in each direction are: Δx = 2.8 cm, Δy = 2.0 cm, Δz = 7.4 cm. (N=18) The numbers with a red background and in italic are the maximums and those with a green background in bold are the minimums.

Discussion

Accuracy

To determine the accuracy of the software, we need to look at the results produced by the analysts and compare them against the known origins for each target. This shows how close the analyst's results are to the known origins and, if they fall within the accepted limits, demonstrate the validity of the method.

Since HemoSpat uses the tangent method outlined in the literature [4], it is no surprise that the results in the x, y, and z directions of 2.8 cm, 2.0 cm, and 7.4 cm are similar to the BackTrack results of 2.5 cm, 2.3 cm, and 8.1 cm. The differences in the x and y directions could be a result of the ellipse fitting which works differently between the two software packages. The users have more direct control over the ellipse in HemoSpat using the mouse, whereas BackTrack requires the user to enter numbers to adjust the ellipse. Given that most analysts were familiar with the BackTrack method, it is possible that this affected their results. The difference in height (z) is likely because HemoSpat calculates this automatically instead of requiring user input. This removes a potential source of user error and the pixel-to-2D position rounding error that occurs when the user clicks the image in BackTrack.

The current literature lists a wide range of what an acceptable standard is for accuracy. The BackTrack paper used for this study does not quantify what range of results is acceptable, but the literature identifies various illustrations, ranging from the size of a tennis ball to the size of a volleyball to the size of a basketball [5, 6]. Bevel and Gardiner give a limit of 30.5 cm [7]. The results of this study fall well within the majority of these limits and illustrations (the smallest of these illustrations is a tennis ball which is between 6.540 cm and 6.858 cm in diameter [8], the largest is Bevel and Gardiner's 30.5 cm).

Reproducibility

To determine how reproducible the results are, it is useful to look at the standard deviations. The standard deviation measures the variability of the data, i.e. how close to each other's results the analysts were. The smaller the standard deviation, the closer the analysts results were to each other.

The standard deviations in Table 1 range from 0.17 to 2.77 cm with all of the standard deviations (54/54) falling under the 3 cm mark laid out in the BackTrack paper [9]. As Carter et al. state "A small standard deviation could be interpreted as illustrating that the precision of the method is quite good" [10]. This means that, given the same data, any trained bloodstain analyst using HemoSpat would come up with very similar results.

Conclusion

This short paper outlines the results of a validation study of HemoSpat to test its accuracy and reproducibility. We used the same data and methodology as a previous study which has been accepted by the bloodstain community. The results show that the software is accurate within the limits outlined by the literature and that these results are reproducible given several different analysts using the same data set.

Acknowledgements

FORident Software would like to acknowledge the following for their assistance in the analysis portion of this study: