I returned home yesterday from attending the ISO C++ 2016 standard committee meeting held in Jacksonville, Florida 02/29-03/05 and decided to share my observations regarding why the Concepts TS wasn’t adopted for C++17. This decision by the committee may come as a surprise to many who are eager for Concepts and have been expecting it to be included in C++17.

The short explanation is: the committee failed to achieve consensus that Concepts, as specified in the TS, has attained sufficient implementation and usage experience to be confident in the current design. Basically, the committee did not say “no” to concepts, it said “not yet.”

The long explanation is, well, longer…

The most significant opposition was not due to technical concerns. The primary concerns raised included:

The Concepts TS[Concepts] was published 2015-11-15 following approval by the committee in between the Lenexa and Kona meetings. The TS has therefore only existed in a published form for less than four months.

The only known publicly available implementation is in an unreleased version of the gcc compiler.

The implementation in the gcc compiler was developed by the same (very talented) individual that wrote the specification. An implementation is therefore available for testing, but no known attempt has been made to produce an implementation based on the specification and the specification is therefore untested. Several core working group (CWG) members indicated that having an implementation produced from specification is critical for identifying specification issues.

The most significant known usage of Concepts is in the Ranges TS[Ranges] and in its only known implementation in Casey Carter and Eric Niebler’s cmcstl2[cmcstl2]. There are a few other projects experimenting with Concepts (including my own text_view[Text_view] library), but none that approach the scale that would be expected when developers really start making use of the feature. Performance and error handling issues with the current gcc implementation provide further evidence that no such large scale attempts at using Concepts exists.

The Concepts TS does not specify any concept definitions. Some committee members question the usefulness of concepts without the availability of a concept definition library such as that in the Ranges TS. Adopting the Concepts TS into C++17 without a corresponding concept definition library risks locking down the language without proof that it provides the features needed to implement a library as might be designed to conceptify the standard library.

If more implementation and usage experience had been available, would it have affected the decision to adopt Concepts into C++17? I’m not sure. A number of technical concerns were raised and I suspect that at least one nation body was prepared to vote no on a final C++17 publication if it included Concepts in its current form. Technical concerns raised throughout the week included:

The Concepts TS includes new syntax to define function templates. An abbreviated function template declaration looks similar to a non-template function declaration except that at least one of its parameters is declared with a placeholder type specifier; either ‘auto’ or the name of a concept. The concern is that a declaration like this: void f(X x) {}
defines a non-template function if ‘X’ is a type, but defines a function template if ‘X’ is a concept. This has subtle ramifications for whether the function can be defined in a header file, whether the typename keyword is needed to reference member types of ‘X’, whether parameters declared with rvalue reference qualifiers are forwarding references or parameters that only bind to rvalue arguments, whether there is exactly one variable or potentially none or many for each declared static local variable, etc…

The Concepts TS also includes a template-introduction syntax that allows omitting the verbose template declaration syntax that we’re all used to while simultaneously stating type constraints. For example, the following declares function template ‘f’ taking two parameters ‘A’ and ‘B’ that satisfy concept C:C{A,B} void f(A a, B b);
This syntax is not loved by all. It was mentioned that a version of the Ranges TS used it at one point and the library evolution working group (LEWG) requested that it be changed and never used again.

There are two forms of concept definitions; function and variable. The function form exists to support overloading of concept definitions based on template parameter arity. The variable form exists to support slightly shorter definitions. // function form: template<typename T> concept bool C() { return ...; } // variable form: template<typename T> concept bool C = ...;
All concepts that can be defined using the variable form can be defined using the function form. The form that is used impacts the syntax required to evaluate a concept, thus usage of a concept requires knowing the form used to define the concept. An early version of the Ranges TS used both the variable and function forms to define concepts and the inconsistency produced many errors in specification. The current Ranges TS uses only the function form to define specified concepts. Some committee members feel that a single concept definition form would simplify the language and avoid usage and teaching difficulties. Providing a distinct syntax for defining concepts rather than defining them in terms of functions or variables would also avoid the awkward ‘concept bool’ syntax.

A revision of P0127R0[P0127R0] was approved by the evolution working group (EWG) in Jacksonville for C++17. This proposal adds the ability to use ‘auto’ as a type specifier for a non-type template parameter: template<auto V> constexpr auto v = V*2;
With Concepts, one might want to constrain the above template such that the type of ‘V’ must satisfy the Integral concept: template<Integral V> constexpr auto v = V*2;
However, this is the same syntax currently used by the Concepts TS to declare a constrained template type parameter. If the Concepts TS were to be adopted, then some other syntax would be needed to declare a constrained non-type template parameter. Arguably, the syntax used by the Concepts TS would be more suitable for declaring template non-type parameters as shown above since this matches the syntax used for other variable declarations. This implies that a new syntax for declaring constrained type parameters would be desirable for language consistency reasons.

Concepts have been widely expected to produce better error messages than are currently produced when template instantiation fails. The theory goes, since Concepts enables rejecting code based on a constraint at the point of usage of a template, the compiler can simply report the constraint failure rather than an error in some expression in a potentially deeply nested template instantiation stack. Unfortunately, it turns out not to be so simple and use of concepts sometimes results in worse error messages. Constraint failures frequently manifest as overload resolution failures resulting in a potentially long list of candidates, each with its own list of reasons for rejection. Identifying the candidate that was intended for a given use and then figuring out why the constraint failure occurred, can be a worse experience than navigating a template instantiation stack.

A number of committee members are concerned about whether the current Concepts design suffices as a foundation on which full template definition checking can be implemented in the future. Though assertions were made by Concepts advocates that such checking will be possible, many questions remain unanswered, and these committee members remain unconvinced. It seems unlikely that these concerns will be addressed other than through an implementation of definition checking.

I’m confident that Concepts, in some form, will be added to C++19/20. I expect all of gcc, Clang, and Visual C++ to be shipping implementations well before the next standard is complete, hopefully within the next year. While we await the growth of confidence in the Concepts design, I hope to see changes made to address at least some of the technical concerns listed above as I think addressing these will help to build consensus and avoid surprises when we next consider adopting Concepts into the C++ standard.

In the next few posts, I’ll explore the technical design concerns above in more depth and discuss possible solutions.

This entry was posted
on Sunday, March 6th, 2016 at 11:45 pm and is filed under C++ Concepts.
You can follow any responses to this entry through the RSS 2.0 feed.
Both comments and pings are currently closed.

8 Responses to “Why Concepts didn’t make C++17”

Thanks, I find this kind of explanation really useful for understanding the issues. It also helps stave off impatience when these new features aren’t delivered as quickly as we would all like! In the end, having broken features delivered early is far more troubling than good features delivered later.

“It was mentioned that a version of the Ranges TS used it at one point and the library evolution working group (LEWG) requested that it be changed and never used again.”

I’m rather shocked by that. But mainly that the LEWG took such a harsh stand against it.

I have *never* liked that alternative template syntax that Concepts TS. While I’m a bit iffy about the fully terse syntax (funcName(ConceptName x)), I’ve always felt that the short-but-not-that-short template syntax was worse than just using a regular template. Sometimes it can save keystrokes. But I never thought it was a good idea.

I do hope that the committee will take some time to massage Concepts into a more palatable form. We have good behavior; it’s the definitions that seem most oddball (and potentially non-viable for declaration checking).

Some have decried the committee’s decision as a bad thing. I see it as a good one. We all want concepts of some form, and we all hate having to use std::enable_if. But we don’t need a *bad* form of concepts. Take the time to get it right, because if we get it wrong, we’ll never be able to fix it.

Different from Mike’s (understandable reaction, I don’t actually think this event is bad news. In fact, I think it can be considered great news. That’s not because I don’t like the Concepts concept, and it’s also not because it will be some time before Concepts are fully integrated into C++ (which according to your estimate will be at C++19/20), but it illustrates one of the great and really important characteristics of C++: it’s not a toy language, written by a single author or by a specifc company, but it’s a language that was (and is) actively being developed by an international group of experienced C++ programmers, designers and –can I say this?– philosophers, all aiming at improving the language, who, as a group, are able to avoid ideas that may appear to be attractive (fashionable?) right now, but in the end could very well work against us. It’s actually great (and a case in point) to read about the concerns that were raised during the meeting in Jacksonville, and I think the decision not to adopt Concepts in C++17 has been, considering the voiced concerns, a courageous, wise, and good one: if we’re not ready for Concepts, so be it, and it’s way better to sleep on it for a few more nights and to have it discussed within the standards committee, in order to distill a final definition that is well-designed, than to accept a half-hearted Concept concept that could otherwise have haunted us for many, many years. Just think about it: I think concepts started to appear around C++11. Now their estimated time of arrival is C++19/20. That means that it nearly takes a decade to get it right, and *that’s* simply the amount of time it takes to design new language features that are able to withstand any rigorous use. Thumbs up for the C++ committee: this is WAY better than prematurely publishing immaturely developed concepts. And please don’t get me wrong: I love the ideas behind Concepts, and since g++-6 allowed me to fool around with Concepts a little bit I’ve really started to like them. But in the end I’m perfectly happy to wait a little longer. What’s that saying again? Rome wasn’t built in a day either

Thank you!
Honestly ALL point are valid reasons to hold concepts up. Hell, No5 is good enough!
As for concept definitions syntax - please, use new syntax! Right now it looks like (yet another) abuse and “overloading” of meaning - something C++ is rich on.
Same for template-introduction - it is overloading of meaning. How can be expected C{A,B} to be parsed by humans to anything different other then “construct C with A and B”? I hate the old template syntax, but stealing a new one from other parts of the language is not a good idea.
I any case, I am happy the committee is taking its time with that one. Templates are powerful, but horrible, horrible, but powerful - Concepts should only bring improvement and not make matters worse for this already hard and messy section of the language.

I think the committee has completely missed the idea of concepts which the original proposal actually tried to capture and is well proven in other languages (such as Haskell and Felix), although that proposal had some issues.

The core problem is that template definitions cannot be type checked.

The concepts-lite proposal current is badly flawed because it fails to address the important issue and tries to do something, which, in general and C++ in particular is provably impossible. The problem is very simple and easy to understand: in general constraints don’t propagate. The proof is along the lines of: constraints are general programs and propagation is equivalent to solving the halting problem.

However in this case it is even worse. Consider a template with a constraint C1 that calls ANOTHER template in its definition, with constraint C2. How is it possible for the compiler to PROVE that the constraint C2 is met .. when it cannot even type check the definition??? Even if it COULD type check the template (which the original proposal would have allowed with some reworking, by injecting suitable overloads into the scope), second order logical implications are a little tricky to prove (yes, that’s an understatement).

Consider for example, C1 says that T is copy constructable and copy assignable, Now take the address and pass it to a template which has a C2 that says U is a pointer to some type which is copy constructable. It is not even enough to prove that Assign AND Copy implies Copy, you would ALSO have to decode the specification that the parameter of the second template U was required to be a pointer. And this is a trivial case. How about constraints like: T is a sequential container of pointers to associative containers whose value type is copy assignable and which has bidirectional iterators but not random ones? Or would you propose that a function with random iterators would be selected over a mere bidirectional iterator constraint?? How would you even state that??

Sorry but this whole concept is totally broken. Even when some cases can be done in theory in practice the solutions are exponential time.

As a case study examine Hongwei Xi’s ATS2 programming language which has dependent typing. This requires solving constraints on integer ranges. There is a known fast algorithm for that. The language then got generalised to solve harder constraints by plugging in Microsofts Z3 solver and all hell broke loose. The simple fact is even solving BOOLEAN constraints is known to be a very hard problem, and a solution was only found recently (by Xavier Leroy if I recall).

Heck, there is a whole *family* of programming languages dedicated to hand tailoring constraint problems: Prolog and its successor Mercury, for example. Unless C++ constrains constraints to a few simple properties, it cannot be done, and a few simple properties simply aren’t useful enough to warrant a language extension.

SO you want to get rid of messy long winded template errors?
Go back to the original idea. It solves the problem. Here’s how:

1. The template definition is TYPE CHECKED. If you mess up you get a type
error immediately. One ordinary (polymorphic) type error.
Typically overload resolution failure. Exactly the same as you would get in a non-template.

2. The instances are independently TYPE CHECKED. Same. Ordinary error.
There is also a possible check that the instance function match the concept requirements, although this is not necessary (in Felix I do not do that check).

3. Calls of the template cannot fail. End of story. All the type checking is already done.

4. The linker can find an instance is required that is not available.
This is conceptually the same as not providing any other entity required at link time.
However in practice, its a mangled name, so there is a real quality of implementation issue deciphering the name conveniently.

There is also another issue: the programmer may ask: why do I need to provide that function? Where is it used??

This one is harder. Its a genuine downside. Conceptually, however, it is no different to the situation with monomorphic code, except that with monomorphic code it is a bit easier to find a use with a tool like grep. This is because a call to a template function does not have to name type arguments, they can be deduced.

I have real experience with this, Felix has had Haskell like typeclasses for 10 years. It does whole program analysis where C++ would use external linkage, but the missing instance problem, when it arises is a bit tricky but usually not a show stopper. The fully type checked generics are a major advantage.

Ok, so implementation? Constraints are impossible, the original proposal can be done right now by hand by following a simple protocol with one caveat: the overloads a concept injects can’t be injected into a function, so you have to put them before the function which causes them to leak. A compiler can easily inject them in the right place.

Here’s how:

1. write your concept as an ordinary template class with ONLY static member functions. Its really just a namespace with type parameters. Put it in a header file.

2. To use a concept, #include it. Yes, the stuff will leak to the rest of the program, but this is only an emulation and that problem was mentioned.

3. To invoke the functions, qualify them with the concept class name.

4. To write an instance, just define an ordinary template specialisation. You can compile it anywhere, in any translation unit, you do have to link that unit into the program of course.

5. The compiler will generate calls to the instance via the usual mangled name protocol. The definition does not have to be visible. IF the definition is visible either in the concept class OR a visible specialisation, THAT function will be called and might be inlined. Exactly the same as can already do.

6. Partial specialisations are allowed but MUST be declared (same as now in C++).

There are two issues:

Q1) what about ordinary non-static member functions?
A) They’re NOT allowed, without loss of generality.
Just use a forwarding function in the instance.

Q2) what about constructors/assignment etc?
A) Assume for this proposal that all types are first class.
For non-copy constructors etc, just use a factory function. No loss of generality.

The special functions (copy, move, assign, destroy etc) will require special handling.
Since this is a pure extension it can be ignored until later. The compiler will just
assume it can move or copy or assign values of type T and we will use ordinary
phase 2 lookup for these. Therefore this is NOT a complete solution.

however unlike a generalised constraints proposal, the number of “properties”
like copy assignable we’re dealing with are quite finite and we can probably just build in
to the language or library the required rules (though I’m not sure). Felix solves this
problem by demanding ALL types are first class so I don’t have much experience
dealing with second class types (i.e. ones lacking properties like copy constuctable).

As I do not profess to have expertise in the fine points of C++ syntax, the above may be a great oversimplification. However, it may be useful in considering a great deal of potential simplification. Simplification addressed here is from language user perspective, not implementation perspective, but I suspect the latter follows from the former.

In the above, Tom Honermann suggests a considerable simplification in reviewing the concept of “concept”, i.e. a concept is a set of constraint definitions for template parameters. Well, the classic definition of a “type” is a set of values with operations and constraints on those values.

From a naïve language user’s perspective, a declaration form such as “vector” would seem to allow for vectors of almost any type. Clarification of “almost any” then requires considerable investigation of fine print and curious syntax, or more usually, reliance on error messages, and trial and error. However, there is indeed a well-defined set of constraints for the vector element type, except they are not called a type, they are called a concept.

Both types and concepts support polymorphism, but what is the essential difference? Generally, it is described as template evaluation through code generation at compile time, rather than run-time evaluation through “vtables”. What is common though is the simple concept of defined access to a set of attributes, functions or operators. For a type this occurs through “slots” indexed with a “vtable”. A concept would be the same, if there were a simple (well simple from a user perspective anyway) mapping of user type slots to parameter type slots.

So what is suggested here is a new kind of type definition which is essentially a mapping of specifications from one type to another. With this come all the benefits of polymorphism, inheritance, compile time checking, etc. One impact is in the preference rules for function name resolution, but this should be considerably simpler anyway - perhaps only that the final tie break is to use a basic type rather than converted type, where applicable. Function resolution for non-type parameters could still occur with “const expressions” (or even at run time – heresy?).

Such mappings can include standard generation of related constructs, such as the strict weak ordering operator “less” mapped to all the other comparison operators. An explicit definition for equality could override that for equivalence. (In these terms, what to many are confusing concepts can be much more easily understood.)

Of course, all this could lead to more general concepts for type expressions and type generators, but I leave that only as a hint to intrigue language experts.

Related to this is another of my irritations with deriving language concepts through implementation concepts rather than from user perspectives. That is (at least until the current “range” proposal) iteration is expressed through a pair of iterators, rather than through a simpler concept of a “sequence”, from which iterator and iterator value constructs can be derived. Sequence is a far more useful concept; for instance, it can be relatively easy to define sequence paths through hierarchies, networks, “joins”, queries etc. Similarly, it is simper (for the user again) to specify particular characteristics for containers, such as for stacks, queues, streams, etc. through concepts for the behavior of generic insert and remove operations, rather than in terms of a set of particular implementation related function calls for these similar operations (although both can be made available for use, depending on what is clearer to the user and reader.)

I got a feeling that again happened the same thing as with C++11 - some group of improper enthusiasts started to go further and further with simplifications, extending to special cases etc. - while there’s still needed some BASIC FRAMEWORK.

Seems like lots of committee members don’t understand that concepts in C++ is something that was considered initially in the beginning (see the first Bjarne Stroustrup’s book about C++), which means that they are late by at least 20 years already. And if they continue this way, I doubt I’ll see concepts in C++ when I retire.

Here’s some of my concept for concepts in C++. It’s quite old article, so I’m referring as “base proposal” to the proposal for C++11. Many things seem to be even similar to the proposal in C++17, just replace “concept SomeName” with “template concept SomeName”. I was hoping someone would see it on time and get at least a bit of inspiration… Of course, this proposal is also quite large, but some of the parts are quite optional (like e.g. partial matching).