Our Full Research Process

The Importance of Evaluating and Testing 3D Filament

We believe filament should not be a limiting factor when trying to print something, whether because of cost or from lack of quality, which is why we developed an evaluation process to determine the best manufacturer for our filament and make sure they remain consistent going forward. Below is a summary of the evaluation and research process we used in our quest to find the highest quality 3D filament for the most reasonable price.

Getting Started

We began by evaluating the filament provided by already established filament sellers. We saw a lot of good feedback about Diamond Age and Faberdashery on forums like RepRap, and based on our research, we agree that they produce some of the best quality filament but we knew there had to be a better solution that gave similarly great quality without costing as much. In other words, a filament that gave you the best overall value without sacrificing quality.

Diamond Age

PrintBL
(US Distributor for Diamond Age)

Faberdashery

Price Per Roll – 1.75mm PLA

300m ~0.9kg, Spool

Not Offered

300m ~0.9kg, Spool

1 Roll

$106.33

N/A

$128.73

2 Rolls

$103.09N/A$116.80

3 Rolls

$96.26

N/A

$111.76

Price Per Roll – 3.0mm PLA

~300m 1.0kg, Spool

~300m 1.0kg, Spool

~300m 1.0kg, Spool*

1 Roll

$79.25

$59.35

$83.27

2 Rolls

$65.24

$55.73

$69.71

3 Rolls

$60.54

$55.49

$63.98

*We had to adjust the price to reflect 1kg since Faberdashery sells in 100m coils

Notes:

1 USD to NZD on 5-16-14: $0.8638

1 USD to GBP on 5-16-14: $1.6806

Spool: Wrapped around a core

Roll/Coil: Not wrapped around anything

Faberdashery: Sells in max 100m coil lots and currently has no U.S. distributor so it had to be ordered directly from the UK and the price was converted.

Shipping destination ZIP code is 49002 (Portage, MI)

Used the lowest cost shipping method

Prices are based on 1kg (~300m) of Black 1.75mm PLA Filament

As you can see, both of these manufacturers are expensive, even the Diamond Age U.S. distributor still costs a lot even when purchasing 3 full rolls.

So we set out to find a better alternative. We knew we had to analyze each filament using a consistent standard that enabled us to rely on the data collected and evaluate each new shipment of filament once we had selected our manufacturer. We also wanted to approach each manufacturer with an open mind, to have no biases towards any one company, including the 2 mentioned above. After all, we were looking for filament for ourselves too!

What We Tested

We wanted to make sure that our methodology was unbiased, disciplined, and consistent, so we looked at 3 key points to develop our evaluation standards around:

The consistency of roundness (ovality) and diameter

The consistency and quality of the prints

The consistency of color

Looking at Ovality & Diameter Consistency

We began by analyzing the filament with a laser micrometer. We had it set up as a 2 person team: the first pulls the filament through the jig so that it would pass through the laser, and the second records all of the data into a computer database. The filament is measured every 1 to 3 inches, and we recorded the measurement to a 0.001 mm tolerance with 100 data points to help minimize any statistical skewing of the data. This same process was repeated for all of the filaments we measured.

We then analyzed all 100 points of data, looking at several statistical factors. The two we felt were most important were consistency of roundness and diameter.

To measure roundness, also called ovality, we took the absolute difference between the X-axis and the Y-axis to generate a positive number for all outputs and then we compared the mean and median, the maximum and minimum numbers, and the standard deviation.

To measure diameter we took the average between the X-axis and Y-axis then ran the same analysis

Printing with the Filaments

As the saying goes “The proof of the pudding is in the eating”. The next step in our analysis is to see how the filament printed, in order to test the chemical makeup of the filament and see if it can do what it was designed for. If the filament didn’t have a good design on a chemical level it wouldn’t matter how consistent the diameter appeared.

We wanted to print an object that was easy to print but with enough complexity that we could see how each filament performed. We chose a gyroscope of a carbon atom, with 2 levels and a round core.

After each object was printed, we examined how much stringing was left, how easily the gyroscope would spin after removing from the build plate, the accuracy of the edges of the print, etc. We also held the gyroscope up to a light and looked through it to measure how transparent the filament was, especially for colors other than black.

The Consistency of Color

Finally, we measured the consistency of the filament color using Pantone’s Color Matching System as our standard. Just like paint or dye, we wanted the colors to be dependable with every new roll of filament.

We compared the transparency levels of each printed object to see how much dye was mixed into the filament. More transparency meant less dye used. Unless transparency was specifically requested, we knew a solid, opaque filament was preferable.

Our Testing Tools

First Steps

We first started with a digital micrometer by Mitutoyo (model: MDC-1” PJ) which are one of the best in the industry for measurement tools. The tool worked great but unfortunately it was too slow to measure the filament and we felt it created too much human error when measuring.

The key point of error from our measuring efforts was measuring the filament on 2 axes points (the X and Y axis) and still be at the exact same spot. We just couldn’t do it with what we felt was reliable precision. This led us to choose a new micrometer.

A Better Option

Next we tried a dual-head laser micrometer from Beta LaserMike. We had to purchase two parts: the scanner and a processor for $2,000. We sent both to Beta LaserMike to be inspected, to have a new chip set installed, and finally calibrated for another $620.00. We also purchased a calibration tool to be sure it remains accurate, which cost us another $650. This cost us around $3,400 for the entire setup to accurately measure filament.

LaserMike Specs:

Measurement Range: 0.127 mm to 25.40 mm

Resolution: 0.0002 mm

Repeatability: +/- 0.0008 mm

Linearity (@100 scans per second): +/- .0013 mm

This was a lot of money for us but we felt it would give us a solid footing to measure the filament, thanks to this micrometer’s highly accurate resolution, repeatability, and linearity. We felt it was more than enough to handle our requirements.

Once we had the laser micrometer set up and ready, we had to find a way to hold the filament in place so that the measurements were as accurate as possible.

We Used 3D Printing to Help Solve the Problem

We designed a 3D image of a jig that could hold the filament in a fixed location under the laser, and then printed it on our 3D Printer. It was a fun thought that we were printing an object that would then be used to measure itself. The jig is in 2 parts and fits together around the opening of the laser, with a hole just large enough for the filament to pass through without altering it or getting stuck.

The jig kept the filament in place within the laser beam, and kept it perfectly perpendicular to the laser to create the most accurate measurement of the diameter of the filament on two axes (X and Y).

Our 3D printer is a Type A Series 1 (Serial # 234) and we are using the standard 1.75mm head along with the standard base plate that comes with the unit. We used this printer to create the jig that measured the filament in the laser micrometer, as well as to test each filament after it had been measured to print a specific 3D object.

Pantone Color Matching System (PMS)

We knew reliable colors would be just as important to our customers as it is to us. After selecting our choice of manufacturer, we used Pantone as our industry standard for matching color batches. Each shipment of filament must match the Pantone colors we selected, so that our customers can expect to receive the same red, blue, green, etc, every time they order new rolls.

Sources of Error

Before we get into the results, we want to point out some possible sources of error that could be at play in this data. We feel we have done a sufficient job in minimizing these errors and don’t significantly affect the results. None-the-less, we have to be aware of them.

Linearity and Repeatability

Linearity is the consistency of accuracy (bias) over the range of measurement. As we understand, this is how well the measuring device stays true to the number it is claiming to measure, in other words the device itself is off not just its ability to measure something.

We helped reduce this source of error by 1) having the device professionally inspected and calibrated, 2) setting the linearity accuracy to 0.0013 and only taking measurements to 0.001, and 3) since we are using the data to create a comparative analysis between the filaments and not to the true value, if one filament had the error all of them would have the same error. We did make the presumption that the error in linearity was a constant.

Repeatability

Repeatability is the consistency of a single appraiser to measure the same part multiple times with the same measurement system; it is related to the standard deviation of the measured values. The laser micrometer offers repeatability down to +/- 0.0008 mm and since we are taking data to 0.001 mm we felt that, in the worst case scenario, the number would be off by 0.001 mm. We felt comfortable with that possibility, but it is a potential error we want you to be aware of.

Human Error

The last source of error is us, humans, and our handling the filament. It is quite possible we could have input a data point wrong, altered the filament on a miniscule level with our fingers, or the jig itself could have altered the filament as it was pulled through the hole. We are making the presumption that these sources of error and negligible enough to not affect the final data and any error that does happen is consistent with all the filaments.

We set out to compare filaments with each other, and not to a true value, so if there were some minor errors in the system it would reflect in all of the filaments and therefore not favor one fitlament over others..

Our Testing Results

Below are the test results we were able to obtain from our analysis of 15 different filament companies. We compared roundness (i.e. ovality) and diameter consistency. We then took the place score for each filament and added them together to get a final result.

Standard Deviation of Diameter Variance
(i.e. Roundness or Ovality)

Ranking

Manufacturer

Variation

% More

1

Diamond Age

0.001670

2

Quantum 3D

0.004006

139.89%

3

ESUN

0.004174

149.92%

4

Filaco

0.004737

183.67%

5

Coex

0.005649

238.27%

6

Faberdashery

0.006322

278.57%

7

MakerBot

0.006614

296.04%

8

Singstek

0.006732

303.10%

9

Type A

0.007045

321.87%

10

Protoparadigm

0.008542

411.51%

11

Jet

0.008943

435.50%

12

Winbo

0.009544

471.49%

13

JustPLA

0.013785

725.44%

14

Delta Maker

0.015092

803.69%

15

Zen Tool Works

0.019963

1095.38%

Standard Deviation of Average Diameter
(i.e. Consistency of size)

Ranking

Manufacturer

Variation

% More

1

Faberdashery

0.005202

2

Quantum 3D

0.005705

9.68%

3

Zen Tool Works

0.009385

80.41%

4

Diamond Age

0.009452

81.72%

5

Jet

0.010525

102.34%

6

Protoparadigm

0.010595

103.69%

7

Coex

0.010784

107.31%

8

MakerBot

0.011775

126.38%

9

Delta Maker

0.012734

144.81%

10

ESUN

0.013175

153.29%

11

Singstek

0.013685

163.08%

12

Type A

0.014454

177.87%

13

Winbo

0.017434

235.16%

14

Filaco

0.023999

361.38%

15

JustPLA

0.026105

401.85%

Combined Rankings from Both Measurements Above:

Below are the final scores for each filament by combining both scores above. For example, if filament A was in 4th place for roundness and 5th place for diameter consistency it would have a final score of 9. We then sorted all filament brands by this final score.

Combined Score

Manufacturer

4

Quantum 3D

5

Diamond Age

7

Faberdashery

12

Coex

13

ESUN

15

Makerbot

16

Jet

16

Protoparadigm

18

Filaco

18

Zen Tool Works

19

Singstek

21

Type A

23

Delta Maker

25

Winbo

28

JustPLA

Conclusion

Based upon this data we were able to select a manufacturer that we could partner with who demonstrated consistency with the roundness, diameter, and color of the filament. We then started working with them to help improve the system even more and product colors that we felt would be most attractive for 3D printing.

We stand behind our process and the filament it produces, and it is our sincere hope that you feel the same way. We understand this industry is new, especially the personal 3D printing market, and there will be many new developments that we can all learn from to make an even better filament. We welcome your feedback and input on how we might make a better 3D filament.

Here’s What You Need To Do Next:

Try a test sample of our 3D filament. Choose one of out 10 different colors and decide which is your favorite for buying a whole roll.