The first question is really more history rather than anything practical cookingwise. And the second question isn't worded in a very answerable way. If you edit it to ask perhaps about what is essential in very cookie recipe(the base ratios so you can add your own spin on the cookie with additional flavours and toppings), I will remove the -1.
–
JayMar 16 '12 at 13:00

2

I like the question much more :) Changed -1 to +1.
–
JayMar 16 '12 at 13:25

1

Hmm, my answer was worded to answer the previous version of the question. I won't change it now, even if it feels slightly off. It still contains the information you ask for now, mostly in point 2.
–
rumtscho♦Mar 16 '12 at 13:50

Terminology note. Legally, in commercial industry, a 'recipe' isn't protected, but a 'formula' is eligible to be considered a trade secret.
–
zanlokDec 8 '12 at 17:20

A question that gets reference to Ratio (a book I want to get), The Flavor Bible (my go to reference for ideas) and Alton Brown (one of my culinary heroes). It doesn't get much better.
–
ChrisDec 8 '12 at 23:57

4 Answers
4

Yes, the original recipes involved an enormous amount of trial and error. People baked, swapped recipes, and the good ones were desirable and became widespread, while the bad ones died out. The advanced knowledge of food chemistry wasn't even available at the time - Hannah Glasse published her cookbook more then 100 years before Mendeleev published the periodic table.

Luckily, you don't have to go through the same process if you want to invent a good recipe now. There are only a few basic ingredients in baking - eggs, sugar, flour, water - and you can be sure that all possible ways to combine them, as well as most secondary additions and substitutions, have been thoroughly explored. People have distilled the knowledge of this exploration in books, and you can use this basic knowledge to build new recipes which will function with a high probability.

Before you develop a baking recipe, there are four things you have to know.

Technique. If you want to create a cookie recipe, you will have to use creaming. If you want to make a new kind of eclair, you have to know how to prepare a basic pate a choux. There are cookbooks teaching these techniques, some of them are explained in questions here on SA.

Base ratio. For most baked goods, there are ratios which give you the best results. For example, for a crepe you want 1 part flour, two parts egg and two parts liquid. As long as you keep that, you can let your imagination run free by using different liquids, or adding spices, or even plopping pieces of fruit in the pan and pouring the batter over them. You can learn about this from Ruhlman's book Ratio, or use a known-good recipe for a plain variant of the good you are trying to make as your starting point.

Flavor combinations. You can add any ingredients you want, but there is no guarantee they will work well together. Choosing the right ones is a combination of talent (being able to imagine what a combination will taste like before you have had it) and experience. You should try to be more analytical about the flavors of things you eat - which tastes can you distinguish? Which aromas? What makes them go well together? Is it their similarity, or the contrast? - and when you have done that long enough, you will be able to predict the goodness of a combination. A book to help you along is The Flavor Bible, which describes good combinations. You can also take popular combinations and transfer them across types of food. For example, if you like apple-cinnamon pie, you can try to make cinnamon cookies glued in pairs with apple butter.

Ingredients' structure and their role in the baked good. This knowledge is maybe the least usual among home cooks. When you create new recipes, you will always dilute the base formula, or use substitutions. If you don't have this knowledge, it will be hit-or-miss whether your new recipe will work or not. It is a bit like rebuilding a house's interior without knowing which walls are load-bearing and which aren't. If you know what each ingredient does in a recipe, you will know when a substitution is possible and when it isn't. For example, many people will tell you that applesauce is a substitute for eggs. And you can indeed bake a cake with applesauce instead of eggs, but don't try to whip a mayonnaise with it. This is because eggs have a different role in cake and mayonnaise. Learning about these roles requires a lot of effort, and a curiosity
about these things. Most people will find it easier to just try
whatever they feel like, and live with the occasional failed
new-recipe attempt. If you would rather learn about them, I can
recommend two sources. One is Cooking for geeks by Jeff Potter, which is short
and doesn't go into the depth of things, but offers lots of useful
information and is easy to read, or the really exhaustive book
called On Food and Cooking by Harold McGee, which runs to about 800
pages and goes into detail most people don't want to know about, but
is a must-read if you are fascinated by food science.

And, of course you can always just run into the kitchen and experiment. Don't forget to document along the way so you can reproduce your results! Basically, you can always have luck and hit a good new recipe, the knowledge I listed above (combined with some baking experience) helps you increase your first-try success rate from about 2-3% as a full amateur to somewhere about 80% or more. It is your decision how to divide your time between reading and baking, any combination can function, depending on your learning style.

@CosCallis I meant to write Cooking for Geeks (the book), not Cooking for engineers (the site) - slip of the keyboard. While the site is nice too, it doesn't offer the info I was referring to. I'll change the answer. I haven't seen the other book, so I don't know where it comes handy for making new recipes.
–
rumtscho♦Mar 16 '12 at 20:41

1

@rumtscho, The New Best Recipe Book (my own typo above, the link is to the right place) contains not only great recipes but extensive topic specific narratives explaining experiments, findings and information from the good folks at Cooks Illustrated.
–
Cos CallisMar 16 '12 at 23:17

Now take your favorite chocolate chip cookie recipes, add a little extra soda, replace some or all of eggs with milk, up the ratio of
white to brown sugar and use butter rather than shortening and thin,
crisp cookies will result.

If your tastes run to the soft and cakey, use cake flour, baking powder rather than soda, and shortening instead of butter. Chilling
the batter and scooping on the small side will add to the puff factor,
too.

Now finally, chewiness calls for melting the butter, holding back on the egg whites and using more, if not all, brown sugar.

These things need lots of experience, and trial and error of the recipes.
A recipe might not taste good the first time around, but with time you can improve the recipe.

As for inventing your own recipes, try mixing things that match with each other (for example: instead of putting water, try adding pineapple juice with some pineapples on the top to get a pineapple cookie). Take into consideration the amounts of water and flour (to get a perfect dough) used in the recipe.