You can compare design specifics. For example, you test hybrid bikes with disc brakes. Or in the car world, mid-size SUVs. You figure out which company makes the best one in that category.

You can look at the intended use and figure out which design will work best. For example, you try to find the best bike for carrying its rider and briefcase in urban traffic, year-round. Or the best car to haul a family of four, a baby jogger and a dog.

With Approach 1., part of the test results is a foregone conclusion: The best hybrid bike with disc brakes will be, well, a hybrid with disc brakes. In the SUV test, you do not ask whether an SUV is a good solution to carrying a family of four, a baby jogger and a dog. These underlying assumptions are not examined. The test is “thinking inside the box.”

With Approach 2., the test result is much less predictable: Even the best SUV may be less than ideal. Something entirely different may be best for the purpose; it may even be something that does not exist yet. The reviewers are “thinking outside the box.”

The industry likes Approach 1. This gives them a clear target: “Design an SUV that beats the Ford Explorer.” Or: “Come up with a derailleur that shifts as well as Shimano Dura-Ace.”

For testers, this is easy as well. You compare a handful of products and declare a winner. There is no need to think about the “what ifs” that require additional research, thought and imagination.

For consumers, Approach 1 is less satisfactory, because it does not examine whether the underlying assumptions are valid. The best solution to the “4 people, baby jogger, dog” transportation problem may be a station wagon, a mini-minivan or something else that did not enter the discussion. For carrying a cyclist and their briefcase, a hybrid may not be ideal, and a better choice may be an urban bike with integrated fenders and lights. The briefcase might be best carried on a front rack (with suitable geometry) that allows riders to start from a light without wobbling.

Of course, Approach 2 also can be unsatisfactory. If a reader really likes the image or looks of an SUV, they might not want to read that they really should buy a different car. “Just tell me which of the five SUVs on my list is best, and leave me alone,” that reader may think. Worse, the “ideal” product may not even be easily available.

If you were car-shopping in the U.S. in 1955, you probably were thinking which tailfin shape you liked better and whether to pay a little extra for the new 427 cubic inch V8. Then you picked up Road & Track and read that unibody construction was better than Detroit’s separate chassis, and had you considered the new Alfa Romeo (below) with its efficient 1.3 litre engine and excellent handling?

Of course, that Alfa was not easily available, and getting it serviced by people who had never seen such a car was difficult at best. So there were real practical reasons to buy the tailfinned behemoth instead. Even so, Road & Track helped push American makers toward building better cars by showing what was possible, rather than what was easily available.

At Bicycle Quarterly, we try to combine both approaches. On the one hand, we consider what riders can buy today, but on the other hand, we often suggest improvements to the products we test.

Today, cyclists have more choices outside the “standard” categories than just a few years ago. I don’t think the mainstream magazines with their “Best Road Bike for under $2000” approach caused the industry to broaden their scope. Instead, many people outside the industry pushed for these designs, and small makers introduced them. Finally, the bigger manufacturers are beginning to copy them, to the benefit of every cyclist.

Each person will have a different answer. I personally hate to ride a Brompton (I am not saying it is bad, I just do not like it)… and do not need a folding bike, that I would not be allowed to store in my office, etc.
The point is that approach #2 can only be subjective because so many parameters should be considered: would a Brommie be a good choice for my 7’1″ tall colleague ? should my 5′ tall girlfriend use a 29er mountain bike ?
Combining both approaches is probably the best a magazine can do, and it can really guide someone looking for a bike for a specific use, but at the end of the day, you still have to add your input and make your own choices.

Obviously, different riders have different needs and preferences. A good test doesn’t just give a “thumbs up” or “thumbs down,” but explains the strengths and weaknesses of each product. That allows the reader to come up with their own conclusions and make their own choices.

For example, if you need to fold your bike and if you ride relatively short distances, then the Brompton may be ideal. If you ride long distances over varied terrain and on busy streets where speed is your best defense, then a more performance-oriented bike may be a better choice.

I always liked the Globe Live (the bike shown at the end of the post), and recommend it to my customers (my bikeworkshop doesn’t sell bikes) if they want to buy a new bike. It would even be better with wide 26″ tires, but who knows what future will bring.

Jan, your article describes the conundrum that I faced as a designer nearly every day. I would meet with a client who would, in turn, describe the dilemma of their product – which might be that it wasn’t selling, or that the target audience wasn’t aware of it’s existence, or something similar. The client would go to great lengths to describe the often exhaustive market research they’d conducted and just as exhaustively analyzed,. Their conclusions seemed bullet proof.

And then they’d invariably begin to describe the 8 1/2 x 11 product brochure they felt would solve all of their problems, never once considering whether the vehicle of communication was the best means of effectively conveying their message. Presumption is the bane of research in many ways. I recall an apocryphal story about famed designer Raymond Lowe, whose “re-design” of a well known brand was to return to the client and recommend they leave it alone! He realized – before they themselves did – that the value of their particular brand happened to be in the recognizability of their brand and that the presumption to “freshen” the look was a tactical mistake. Who among us – and that includes bicycle manufacturers, along with developers of every widget, doodad, and thingabob out there – is willing to look the client squarely in the eye and say, “Nope. You’re not asking the right questions.”

The prevailing fad is to treat design as the proud child of trend and fashion, rather than an offspring emerging from externalized conversations about the validity and applicability of form and functionality, purpose and application, consumer wants vs. needs. Product “design” may fail the consumer when “design” really means “fashionability.” And this is where your article resonates for me: By considering all sides, and multiple factors, the product review can be a proving ground for the end user.

It’ll be some time still before the benefit of all this really arrives at the mass market. Today for 99% of all buyers it’s still either mountain bike OR Sit up an beg OR Fitness bike OR Roadbike, each with it’s own particular shortcomings. It’s entirely a matter of fashion what people ride and not their real needs, which most bike shops and – makers don’t even begin to consider. (Cars are not really very different, that’s why cars with really economic engines don’t sell, even if every now and then one such model is actually being manufactured)

The problem being of course that it takes a lot of experience before you actually know yourself what you need, and on top of that quite a lot of research to actually know what exists, and where to get it.

The way you deal with this in BQ is of course quite right, but who reads it? At the same time the ‘which road/urban/mountain bike under 1500/over 5000 $ press’ continues to sell it’s shoddily assembled magazines that are really advertising brochures for the industry.

Today for 99% of all buyers it’s still either mountain bike OR Sit up an beg OR Fitness bike OR Roadbike.

It’s the traditional “chicken-and-egg” question: If the industry doesn’t offer anything but those categories, then how can buyers choose something more useful? My neighbor approached me a few months ago. She wanted a bike to ride around town, but also planned to ride the 200-mile, 2-day STP (Seattle-to-Portland). The mainstream bike shops had nothing that fit her requirements. I sent her to a small shop, who sold her a bike from a small brand that fits her needs better.

When did the industry turned to the dark side? Was it the appearence od the montain bike? Or was it when production went overseas?
I can just speak from an european (Austrian) perspective, but in the late 70ies you could still get a locally produced bike, that wasn’t fashionable at all, but hard to kill, and lights, fenders and fat tires were flawlessly integrated. Wheel size was actually 650B and frontend geometry was quite randonneur like.

Sure, these “waterpipe” frames weren’t performance orientated at all and braking in the rain was quite an adventure, but as a bike mechanic I have to say that most of these bikes are still going strong after all these years with little or no maintance.

Nowadays it’s almost impossible to finde a bike with this features at a reasonable price.
Locally produced? Forget it.

The third perspective common in smaller cycling publications (and sometimes BQ) is user reviews. A well written user review focuses first on describing the owner and how the product is used, and how it was selected. It also describes facts and details of the product and buying experience. The owner’s impressions have little value, being an unknown measure and having a strong bias to like the expensive new toy.

As I read through BQ from the beginning until now, I have had these impressions:
1. Each bike or piece of equipment was evaluated largely from a specific usage perspective, mainly that of Jan Heine.
2. What would be ideal from this perspective was not clear at first, but is now becoming fairly specific.
3. While it is recognized that products have different target users, these users would usually be better served by what Jan Heine prefers.

I have always accepted this because:
1. Products are measured against a known yard stick, i.e. Jan Heine’s usage.
2. I know where my own needs and experiences differ, and am equipped to make up my own mind. I expect the target audience of BQ can as well.
3. Had I followed BQ to the letter, completely ignoring my own 44 years of cycling experience, I would still be better off than where I am right now.

The easiest way to be a leader is to find a bunch of people going in a certain direction, and then get in front of them. The hardest way is to figure out the right direction to go, and then convince others to go there too.

Each bike or piece of equipment was evaluated largely from a specific usage perspective

You make a good point, there certainly is a perspective that colors our reviews. We review bikes with the idea that cycling should be fun and that a bike should be reliable. Not everybody agrees with that – some think of bikes simply as transportation, and others don’t mind tinkering with their bikes.

Beyond that, we do look at the intended use. We won’t criticize a racing bike for not accepting fenders – even though I don’t ride without fenders myself. We won’t criticize a cargo bike for not offering the ultimate in handling precision, even though I love to corner fast.

What we look at is whether the bike is as good as it can be, for the intended purpose and with the price point in mind. So we approach a Surly Long-Haul Trucker very differently from a custom-built Peter Weigle randonneur bike.

We do apply the science of our and others’ testing, so we prefer front loads that are easier to balance, supple tires that are more comfortable and faster, optimized geometries and a host of other things. Bikes that don’t have those usually are not “as good as they could be” – and we point that out in the reviews. Some of those things may seem controversial at first (like unibody construction was in cars), but over the years, most have proven themselves. The latest are the wide tires, which now are used even by professional racers.

Finally, we have found through a long journey of discovery that classic randonneur bikes are incredibly versatile machines that really can expand your riding horizons, and we make no excuses for championing those bikes. They have changed the way we ride, and we want others to join in the fun.

“We review bikes with the idea that cycling should be fun and that a bike should be reliable. Not everybody agrees with that – some think of bikes simply as transportation, and others don’t mind tinkering with their bikes.”

With all due respect, BQ seems to have a somewhat more particular perspective than just that bikes should be reliable and fun. Low trail is a perspective–there are other options that work just as well even for your intended use. Front load carrying is also a perspective, and likewise, it can work just as well to carry the load on a small rear rack even for your intended use–readily needed items can go in a jersey pocket or a bento. There are a whole handful of other perspectives that most of your readers would probably agree that BQ tends to favor which are more a matter of taste than a matter of clearly superior design from the standpoint of fun, reliability or even function. I’ve always appreciated that BQ seeks to quantify bicycle function and design, but it seems that there can at times be a slight blurring of function with taste in BQ’s editorial voice.

I won’t say that we are infallible – of course, our preferences will color our reviews. Everybody knows red bikes are faster… and I am not sure we are totally immune to this.

Regarding your examples, there are physical reasons why some things work better than others. Read Jim Papadopoulos’ chapter in Bicycle Science why front loads are easier to balance than rear loads. Tony Foale explains why low-trail geometries are less affected by crosswinds. Those aren’t “matters of taste,” but matters of physics.

That doesn’t mean you can’t have a blast on a sub-optimal design. I rode high-trail bikes with rear loads for many years. My most memorable PBP (my first) was ridden on a high-trail bike with a rear load and relatively narrow tires (24 mm). None of the Bicycle Quarterly editorial team grew up on the bikes we now prefer. It was a long journey of discovery, especially since most of us are skeptics by nature and training. It took years and riding many bikes until Mark was convinced of “planing,” and Hahn only recently came around to wide, supple 650B tires.

When I now ride bikes that are sub-optimal, I really do have to adjust my riding. On the Civia (moderately high trail, rear load), I almost ran off the road in the first corner I took at speed on a familiar trail. The slow response of the bike took me by complete surprise. When I ride racing bikes with 23 mm tires, I need to slow down for corners that I usually take without braking.

I think where matters of taste come in most strongly is riding style. For example, riders who grip the handlebars tightly don’t do well with low trail, which requires a light touch. I also suspect that forceful riders with a low cadence will prefer different frame flex characteristics compared to riders who spin at high cadences.

The third perspective common in smaller cycling publications is user reviews.

Bicycle Quarterly does not publish owner reviews, for the exact reasons you state. In addition, we want to provide a consistency in our evaluations, and that is hard when you have reviewers with limited experience. You really need to ride a few dozen bikes for significant periods of time before you have a body of experience to compare the bike you are testing.

We sometimes do give products to others for review, mostly because of conflicts of interest because the products under review are sold by Compass Bicycles. However, the reviewers have not paid for the product, and thus have less emotional investment in the items they review. In addition to the outside reviewers, we always test the equipment ourselves as well, to make sure our evaluations are consistent.

Our “My Favorite Bike” column does showcase readers and their bikes, but it’s not intended as a review, but as a way to balance our reviews with other perspectives.

Jan, basic physics also says:
•heavier bikes (and riders) will have a somewhat harder time climbing
•riding with a generator will reduce available power
•riding with a substantial sized handle bar bag will cause more aerodynamic drag than riding with none, especially in angled crosswinds

It’s the magnitude, and balance, of factors that matters. And the optimal mixture depends much more on preferences and context than you sometimes acknowledge. You ride in an area where rain is frequent, where old logging and gravel roads are common, and you like to go on extended rides that require substantial clothing and supplies, sometimes include all night sessions. For YOU, a bike may be suboptimal if it does not have fenders, wide tires, a rack, a bag, a generator, and lights. In contrast, I live in an area with great roads, sunny year round weather, and 99% of my rides are 4 hours or less (during the day, not night). In addition, the prettiest rides in my area all require sustained climbing of 3.5 to 10 miles, at gradients of 7 to 12%. For ME, also operating under the laws of physics, adding 10 to 15 pounds of lights, generators, fenders, fat tires, racks, bags, and extra clothing does little to solve any problems I usually face, but does makes the bike “sub-optimal” for climbing. I readily admit my own preferences are based on where and how I ride. Your preferences also depend on your own area and riding style, which I think differ substantially from that of many other riders.

David, you make good points. You are lucky that there are many bikes available off the shelf that meet your needs. Among the more than 50 bikes we have tested, there are a number that are designed for the riding you describe: the Terraferma Race, the Pegoretti Love 3, the Trek Madone, the Calfee Adventure, the Lyon 650B racer, and even the classic 1957 Cinelli Supercorsa. I think you would find those tests instructive, because they considered exactly your riding conditions. While you may not be interested in our tests of fenders, even you would benefit from considering wider tires, because they improve cornering and comfort even on ultra-smooth roads. (That is why pro racers are going to wider tires even on the smooth roads of Europe.)

For the rest of us who ride in places where it occasionally rains, where the sun eventually sets, and where we like to include errands in our rides, the weight of fenders (1 lb), lights (1 lb), wide tires (1 lb) and a rack with handlebar bag (1.5-2 lb) does not add up to 10-15 pounds – more like 4-5. As you point out, the extra weight does slow you down incrementally, but unless you race, taking a few seconds longer to climb a mountain pass doesn’t change your riding experience, if the bike responds to your pedal strokes the same way. Nor will it make you late for dinner.

The difference between front- and rear-loading, or between an optimized geometry and one that tends to run straight and then fall into corners, is not measured in split-seconds, but in smiles. Those qualities greatly affect the riding experience, and the differences are huge, not incremental.

Jan, “While you may not be interested in our tests of fenders, even you would benefit from considering wider tires, because they improve cornering and comfort even on ultra-smooth roads. (That is why pro racers are going to wider tires even on the smooth roads of Europe.)” I returned from two weeks riding in southern Europe yesterday. The region’s provincial and local roads, used in the host nation’s grand tour recently, had surfaces in horrible condition, pot holes and chip seal which were much worse than Clackamas County Oregon where I usually ride. I brought 32mm tires, expecting decent road surfaces as described by the trip promoter. Soon, I wished for my 42mm 650B tires. Despite this, I could descend and corner faster than my companions on 23mm racing tires. The predicted sunny weather included daily thunderstorms and 3 days of constant rain. A set of fenders would have been most welcome.

Comments such as David’s and MRSW’s above which focus on Jan’s recreational cycling ignore the valuable influence BQ has on bike as regular transportation design.

Unless a cyclist using a bike for transit rides in a location with perpetual sun light and no weather, integrated lighting and fenders are critically important. Low trail bikes with wide tires have a very stable ride, absorb city and suburban street bumps better, and are less prone to flats than skinny road race tires. When running errands to multiple locations in they city, I find it much easier to store my things and purchases in a bag custom fit to a porteur rack than pulling panniers off a rear rack.

The road bike solution – wearing ones cargo over the shoulder is uncomfortable, leaves clothes and rider rumpled, and severely limits the amount one can carry. Even if you only put a dinner’s worth of groceries in a courier bag, delicate fixings tend to get mushed.

The main point is that there is room in this world for both approaches. Bicycle Quarterly tests both. We look for optimized solutions to each. In some cases that means low trail (porteurs, randonneurs), in other cases, high-trail can work fine (race bikes). When we tested the Calfee Adventure, we liked it a lot, and didn’t worry about the fact that it couldn’t carry a load, or that it didn’t have low trail. That isn’t what it’s designed for…

Even so, the bike could have been improved slightly with a little more fork offset, as this would have made it less sensitive to sidewinds. We mentioned that in a single sentence, but didn’t focus on it. It’s not a deal-breaker on a bike like that, because the difference will be felt only once in a while on a windy day.