10 Answers
10

Metaprogramming is a pretty exotic topic. It's interesting to learn about, it's a powerful tool, and occasionally you might find it useful. But it'll never be the most often used tool in your toolbox. Sometimes, you might want your code to act on a range of unrelated types with different properties, and that's where metaprogramming comes in. With a bit of trickery, you can write an overload of a function that is only available if the argument type is integral, or if it is a pointer, or if it is either type X, Y, or Z (perhaps ignoring constness on Z).

It is essentially programming with types. Normally, your program can do things like take two numbers and produce a third number, or tell you whether a number satisfies some requirement. Metaprogramming can take two types and produce a third type, or tell you whether a type satisfies some requirement. And yes, it is probably mostly useful in library development. But then again, most code could be considered library code. You could say that everything outside your main() function is library code.

Usually, if you want to solve a problem through metaprogramming, you'll probably want to use the relevant boost libraries to do the heavy lifting. Boost.TypeTraits and of course Boost.Mpl can really simplify things for you. But it's not something you need to know, and it's not something you're likely to need very often.

Generic programming is related (and may in some cases use metaprogramming under the hood to become really generic, for example the standard library uses a touch of metaprogramming to turn raw pointers into valid iterators which is required for the "iterator" concept to be generic), but not entirely the same. And it is much more widely used.

Every time you instantiate a std::vector, you use generic programming. Every time you use a pair of iterators to process a sequence of values, you use generic programming. Generic programming is just the idea that your code should be as generic as possible, and should work regardless of what types are put into it. A std::vector doesn't require the contained type to implement a "ICanBeContained" interface (remember how Java requires everything to be derived from Object in order for it to be stored in a container class? Which means primitive types get boxed, and that we lose type safety. That's not generic, and it's a pointless restriction.)

The code to iterate over a sequence using iterators is generic, and works with any type of iterators, or even with plain pointers.

Generic programming is very widely useful, and can often to a large extent replace OOP. (see the above example. Why would I write a container that required the contained types to implement an interface, if I can avoid that limitation?)

Often, when you use interfaces in OOP, it is not to allow the type to change during runtime (although of course that happens from time to time too), but to allow you to swap in another type at compile-time (perhaps injecting a mock object during tests, rather than using the full-fledged implementation), or just to decouple two classes. Generic programming can do that, without having you do the tedious work of defining and maintaining the interface. In those cases, generic programming means you have to write and maintain less code, and you get better performance and better type-safety. So yes, you should definitely feel at home with generic programming. C++ isn't a very good OOP language. If you want to stick strictly with OOP, you should switch to Java or another more more OOP-fixated language. C++ allows you to write OO code, but it's often not the best solution. There's a reason why almost the entire standard library relies on generic programming, rather than OOP. There is very little inheritance or polymorphism in the standard library. They didn't need it, and the code became simpler to use and more powerful without it.

And to answer your other questions, yes, Generic programming is pretty much a separate paradigm. Template metaprogramming is not. It is a fairly specific technique for manipulating the type system, and is very good at solving a small number of problems. To be considered a paradigm, I think it'd have to be much more generally useful, and approach you can use for basically everything, like functional, OO or generic programming.

I think xtofl really nailed it: Generic programming is about making your code type-unaware. (A std::vector doesn't care, or need to know what type is stored in it. It just works.)

Metaprogramming on the other hand, is about type computations. Given type T0 and T1, we can define a type T2, just like how we can, given integers N0 and N1, we can define a N2 that is the sum of N0 and N1.

The Boost.Mpl library has an obvious example of this.
In your normal code, if you have the integers N0, N1 and N2, you can create a std::vector containing these three values. I can then use some other algorithm to compute an index, and then extract the value stored at that location in the vector.

Given types T0, T1 and T2, we can create a mpl::vector containing these three types.
I can now use some other algorithm to compute an index at compile-time, and extract the type stored at that location in the vector.

thanks for an objective opinion showing the diff between meta and generic!
–
xtoflJun 11 '09 at 10:50

Well, it's how I understand the two terms. Others might disagree. (But if so, I'm sure we'll hear about it in the comments ;))
–
jalfJun 11 '09 at 11:00

The post raises several points i should google/explore. Thanks for the reply.
–
fizzbuzzJun 11 '09 at 11:01

Nice post, one small note is that doing generic programming with templates still typically requires you to define an "interface", (ie the T in std::vector must be copyable, the input to your template function might require an argument have a foo() method, etc...) it just doesn't require you to inherit from an abstract class like OOP.
–
Greg RogersJun 11 '09 at 13:36

But you don't need to specify anywhere in code that the type is copyable. You just get a compiler error if it isn't. And if you add in a bit of metaprogramming, you can even make exceptions to this (iterators are required to expose certain typedefs for example, but plain pointers can be retrofitted to work as iterators too). It is generally less intrusive and can be made to work with existing types that weren't designed for it. (ints weren't designed to be stored in vectors. That's a bit awkward due to the limitations of OOP in Java, but no problem with generic programming in C++)
–
jalfJun 11 '09 at 13:49

You really have to differentiate between generic programming (which is kinda type-unaware) and metaprogramming, which is a perfectly legal way to do calculations within the type system.

Generic programming is very useful when you find a generalizable pattern in a lot of code. If the difference in the pattern lies not only in variable values, but also in different types, generic programming is useful to refactor the code.

Metaprogramming is applicable in a totally different domain. That domain is, indeed, quite new, and needs some exploring of your own. It is fun, too!

One very useful, and common, metaprogramming pattern is the Traits/Policy concept.

C++ Template metaprogramming is a powerful code obfuscating technique that's applicable to a range of applications:

When you want to write code that nobody else on your team can understand

When you want code that you won't be able to understand 7 days after you wrote it

When code performance is more important to you than maintainability

When you want to be able to list "Template Metaprogramming" as a skill on your resumé.

When you need to write code that's unlikely to work on many real-world compilers

If you'd rather eat razor blades than use preprocessor macros

One other case:

If you want to you know how the Boost libraries work under the hood, or you want to contribute to them.

The distinction between "generic programming" (think STL containers, auto_ptrs etc), which is what C++ templates were designed to accomplish and "template metaprogramming" (use of the template system to get the compiler to effectively "run algorithms" for you) is important.

I'm all in favour of the first, but find it hard to see real-world benefits from the latter.

That sounds so perfect. So i do not really need to read/understand Modern C++. I hope somebody convinces me otherwise
–
fizzbuzzJun 11 '09 at 10:01

Are you saying that macros are not a good code obfuscating technique ?? Seriously I think templates are great @hotadvice you should definitely learn about them.
–
BenJun 11 '09 at 10:55

2

You could say the same about OOP though. When people go overboard with class hierarchies and hide everything behind 8 interfaces and every function is virtual, you achieve pretty much all of the above as well. Anyway, take a look at tydok's example, and tell me that it would become more readable and more maintainable if the argumnt type was ISwappable instead of a template type T. Of course it wouldn't, and that is why generic programming is useful.
–
jalfJun 11 '09 at 10:55

@ben: I'm not anti-templates, and I use them extensively in my own code. It's metaprogramming I have concerns about. Do you use that?
–
RoddyJun 11 '09 at 11:12

For advanced templates and techniques, I recommend: Modern C++ Design, by Andrei Alexandrescu

C++ is a rigid language, with practically no runtime introspection capabilities. A lot of the troubles you'll encounter can be dealt with templates. Also, the templating language is turing-complete, making it possible to generate complex data-structures and pre-calculate values at compile-time, among other things.
For many scenarios, it may be more trouble than it's worth it though.

Template meta programming is a way of writing programs which are interpreted by the compiler. It is one level of abstraction higher. You can do all kinds of weird and funky stuff with it. The boost libraries contain many examples:

One answer which has not yet been given: although generic programming and meta-programming are indeed quite different techniques from each other, they were both designed to solve (essentially) the same problem: how to get rid of boilerplate. These are also embodied into not just one but two acronyms of software engineering principles: DRY (don't repeat yourself) and DIE (duplication is evil). This is a modern-day rephrasing of Occam's Razor, ie that "entities must not be multiplied beyond necessity" (entia non sunt multiplicanda praeter necessitatem). [Who knew that 14th century logicians could come up with something useful for 21rst century software design?].

Anyways, the point of both techniques is to find ways to unify codes which are 'almost the same' into a single piece of parametrized code. In the meta-programming case, you are using code generation so the technique involves specializing a general piece of code (aka a template, or more generally a code-generating-function) to a specific case. In the generic programming case, you try to use the type system to recognize that various pieces of code are 'the same' and then use type-based dispatch for the specialization.

Yeah that has nothing to do with template metaprogramming.
–
Simon H.Jun 11 '09 at 10:28

2

Basic, yes, but still it's metaprogramming. The arguments can also be template types.
–
Nick DandoulakisJun 11 '09 at 10:30

2

That's a good point imo. Metaprogramming isn't entirely separate from "normal" programming. Technically, this example is metaprogramming. It's just so a simple and commonly used that we don't usually think of it as such. But it's a good point that metaprogramming doesn't have to be some obscure guru-level thing that is only ever used to show off your C++ skills and prevent others from reading your code.
–
jalfJun 11 '09 at 10:51

I think, if you are not library developer, then it is better to forget about template metaprogramming. I beleive, that it is useless in production code, giving more problems, then benefits: "Once you started to use it, then you loose flexibility being tied to this solution. It is very hard to come away from template-boiled code after you started to use it. And this is a problem, as templates are not flexible enough".

I don't see what you mean. How do you lose flexibility, and how is it hard to get away from template-heavy code? I think I disagree, but more in a "I can't think of a single example where that'd be true" kind of way. Can you elaborate on it a bit?
–
jalfJun 11 '09 at 10:59

@xtofl I agree with you in the part, that any dogmatic statement is wrong. For example, we also use explicit instantiation. Probably type selectors can be useful. Question is: "Where is frontier to stop?" After some time, it seems for me, that it is much closer to trivial uses, then it seemed before.
–
Konstantin TenzinJun 11 '09 at 11:04

I use it when i cannot use type overloading to do the job. Most of the time you can use type overloading and ATM i cannot think of when inheriting a class wont solve the problem (i inherit maybe empty class). However in a case you need to know T::value_type (or another property) and IGNORE what T is (meaning its not allowed to inherit a dummy class) then you would need meta programming to get the type and change the code path based on it.

Consider passing a vector to your class, your class does not care its a vector but wants to convert it to another container or an array. IIRC you cant use normal means to extract int from the type so you use typename T::Value_Type to extract int and do whatever you like.