2021 C++ Standardization Highlights

Introduction

The ISO C++ Standards Committee (also known as WG21) has not met in person since its February 2020 meeting in Prague, which I wrote about here.

However, the committee and its subgroups have continued their work through remote collaboration, and a number of notable proposals have been adopted into C++23, the next language version, in this way, with many others in the pipeline.

In this post, I will outline some of the highlights of the committee’s work in 2021. (The post will also cover some material from the latter part of 2020, a period when remote collaboration was already underway but which I have not covered in any previous post.) I’ve been less involved in the committee than before, so this post will not be as comprehensive as my previous trip reports, but I hope to share the proposals I’ve found most notable.

Collaborating Remotely

The committee consists of four Working Groups and 22 Study Groups (a subset of which are active at any given time). The in-person meeting schedule was three 6-day meetings per year; other than a plenary session at the beginning at the end, most of the week would be spent with the subgroups meeting in parallel (not all of them necessarily for the whole week).

To continue evolving C++ remotely, subgroups began to meet remotely via Zoom teleconferences. (For some subgroups, this was more a continuation of a practice they had already adopted to hold telecons in between meetings.) Subgroups would meet independently, at a frequency determined by their respective chairs. In addition, virtual plenary telecons to hold committee-wide votes continued to be held three times per year.

Remote collaboration has a number of challenges, such as timezone differences, and the lower fidelity communication signal of video conferences over physically being in the same room. It can also be more challenging to make time for meetings that recur regularly over the course of a year (for example, 90 minutes every week, as is the case for the Evolution Working Group telecons) than to set a week aside for committe work. At the same time, remote collaboration has the upside of making the committee’s work more accessible, allowing experts to contribute from around the world without making the time or cost investment of physically travelling to a meeting location that may be on a different continent for them.

One key to successful remote collaboration is being well-organized, including documenting your processes in writing. I think the committee has done pretty well in that regard, making a lot of implicit process knowledge explicit in recent years. Some key process documents include:

• The C++ International Standard schedule, which lays out the committee’s intention to put out a new standard version every 3 years, and the key deadlines that need to be met for various milestones leading up to a standard release. The committee has retained this 3-year train model notwithstanding the transition to remote collaboration; while the amount of material that makes it into C++23 may be reduced as a result of pandemic disruptions, the C++23 train will leave on time.
• An outline of the committee’s top-level priorities for the C++23 cycle
• An outline of the committee’s remote collaboration model
• Additional documentation of how proposals move between subgroups

Some key resources for keeping track of the committee’s work:

• A list of proposals submitted to the committee can be found here. They are organized into “mailings”, which are now posted monthly to facilitate faster collaboration.
• The status of each proposal can be found in the committee’s github issue tracker. Every paper has an issue (which can be found by searching for the paper number, or by using the forwarding link https://wg21.link/pXXXX/github), which documents which subgroups have seen the paper and the outcomes of any polls that were taken.

In the rest of this post, when referring to a proposal, I’m going to link to its github issue, so that it’s easy to check on its precise status. The proposal itself is linked from the issue (scroll down for later revisions).

C++23

Here is a selection of proposals that have been voted into C++23 working draft:

Language

• Deducing this, which allows making the object (this) parameter of member functions explicit, and making its type templated and deduced. This versatile language change enables a number of interesting use cases:
  • Avoiding code duplication when the object parameter has different constness or reference qualification. Previously, separate overloads would be needed to cover these cases.
  • A design pattern for static polymorphism that’s considerably simplified compared to the Curiously Recurring Template Pattern (CRTP).
  • Member functions that take their object parameter by value. (Compare Rust methods that take self by value, consuming the object when called.)
  • Recursive lambdas, made possible by the fact that the explicit object parameter of the lambda’s call operator gives you a way to refer to the previously-unnamed lambda for recursive calls.

  See the examples section of the paper for illustrations of these and more.

• Multidimensional subscript operator. This allows the [] operator to take multiple arguments (e.g. matrix[i, j]), which improves the interfaces of multidimensional data structures like matrices. Previously, you’d have to use chained calls like matrix[i][j] (which requires a temporary proxy object to represent the intermediate result matrix[i]), or use the () operator instead.
• if consteval, which gives you a reliable way to check if a function is being called during constant evaluation or during runtime, and do something different in each case.
• Decay-copy in the language. This adds a new expression, auto(x), which makes a copy of x following decay rules (for example, array-to-pointer decay) much as if x were assigned to a new variable (e.g. auto y = x;), but without introducing a new lvalue to refer to the copy.
• A literal suffix for size_t. The suffix is uz, to be consistent with the existing convention that suffixes for unsigned integer types include u. The suffix z is also introduced (again for consistency) to represent the signed integer type corresponding to size_t, though in practice this is not likely to be needed often.
• Narrowing contextual conversions to bool. This allows narrowing conversions to bool in some cases where it’s clear that a boolean result is desired, such as in if (flags & Flag).
• #elifdef and #elifndef. This fixes a small consistency gap in the preprocessor which had #ifdef as a shorthand for #if defined(...) but no corresponding shorthand for #elif. The new directives were added to C as well.
• Mandating declaration-order class layout. This mandates that data members of a class must be laid out in memory in order of their declaration. Previously, compilers had the freedom to reorder sections of the class with different access control, though no implementation was known to make use of this, so this paper is standardizing existing practice.

Library

• stack_trace, a facility for representing stack traces and querying the current stack trace
• Size feedback in the allocator interface. This provides a mechanism for allocators to communicate back to their caller the actual size of the performed allocation, which may be larger than the requested size (if e.g. the allocator is rounding up for alignment purposes, or allocating from a set of pools of fixed sizes). The caller may then make use of the full allocation.
• A monadic interface for optional. This adds the new methods and_then(), map(), and or_else(), allowing working wih optional in a more functional style (compare similarly-named methods in Rust’s Option).
• out_ptr, a facility to make it easier to use smart pointers with C APIs that expect raw pointers
• move_only_function, a move-only version of std::function
• Various additions to the std::string intrface, such as starts_with() and ends_with(), contains(), and resize_and_overwrite().
• Various improvements to Ranges, such as better string splitting, and zip() and related range views.

Technical Specifications (TS)

In addition to the C++ International Standard, the following Technical Specifications (sets of features specified independently to gather implementation and use experience and consider for inclusion into a future International Standard) are under development or ballot:

• A Minimal Transactional Memory TS (formerly called “Transactional Memory Lite”). This is a simplification of the Transactional Memory TS published in 2015, with the goal of making conforming implementations easier to produce, and thus increasing the chances of collecting implementation and use experience and proceeding with standardization. This has been approved to be sent out for a DTS (Draft Technical Specification) ballot, where national standards bodies can provide formal comments prior to final publication.
• Version 2 of the Concurrency TS, containing facilities for implementing the hazard pointer and read-copy-update techniques.

Evolution Working Group (EWG)

In these sections, I’ll highlight some proposals that have not yet made it into the working draft but are still cooking in the language and library Evolution Working Groups.

These proposals have been approved in EWG telecons. Their next stop per the new collaboration process is an EWG electronic poll, followed by wording review and finally a plenary poll; assuming these steps go well, they have a good chance of making C++23:

• Portable assumptions, which allows expressing instructions for the compiler to assume for optimization purposes that certain expressions are true (currently accomplished using compiler-specific builtins such as __builtin_assume) in a portable way. In the C++20 cycle this use case was going to be subsumed by contracts, but since contracts are not on track to appear in C++23, EWG has now gone back to addressing this use case independently.
• static operator(), which allows for a more efficient representation of stateless callables
• constexpr class, a shorthand for declaring a class whose member functions are all constexpr
• A type trait to detect reference binding to a temporary

These proposals are still under discussion, in most cases waiting for a revision from the proposal authors. Given where we are in the cycle, these are less likely to make C++23:

• Pattern matching remains a proposal under active development and a high priority for EWG, with recent discussions addressing topics such as exhaustiveness checking and the syntax for identifier patterns.
  • A set of enhancements to pattern matching, including new keywords is and as, has also been proposed.
• Efforts continue to hash out a way to allow constexpr allocations to survive to runtime.
• #embed, a preprocessor-based mechanism for embedding the contents of external resources into a program
• A language mechanism for customization points
• A pipeline rewrite operator, which would allow writing nested function calls using cleaner syntax (inspired by the way Ranges overloads | for range adaptors)
• Argument type deduction for non-trailing parameter packs
• Guaranteed copy elision for return variables
• Postfix fold expressions

Fixing the Range-Based For Loop

The range-based for loop suffers from a tricky lifetime issue, where the way it’s specified using rewrite rules means that the result of the range expression itself gets its lifetime extended (if it’s a temporary) for the duration of the loop, but this does not apply to sub-expressions of the range expression.

This makes loops like for (auto& item : foo(temp())), where temp() returns a temporary object and foo() returns a reference into that object (rather than, say, taking ownership of it), exhibit undefined behaviour, because the temporary does not remain alive for the loop body. This contrasts from the same construct written using an algorithm like std::for_each(), where the endpoint for temporary lifetimes is the entire function call which includes the body of the loop in the implementation.

This has been a persistent gotcha, and it has limited the design of Range adaptors (where e.g. get_vector() | transformed(f) is currently not allowed because otherwise it would be susceptible to this issue).

It was encouraging, therefore, to see a proposal to fix this long-standing issue, by adjusting the lifetime extension rules to apply to all subexpressions of the range expression in this case.

Unfortunately, the proposal narrowly failed to garner consensus in EWG, with some participants feeling that this was too specialized a solution that would not address similar cases of dangling references in contexts other than the range-based for loop, such as auto&& range = foo(temp()); /* work with range */. I think the counterargument here is that in these cases, the dangling reference is explicitly visible, whereas in the range-based for loop it’s hidden inside the implicit rewrite of the loop.

Named Arguments Making a Comeback?

While it has not yet been officially submitted as a P-numbered paper, nor has it been reviewed by EWG, I’ve come across a draft proposal for named arguments (in this formulation called designated arguments) which was circulated on the committee mailing lists (including the public std-discussion list); there’s also an accompanying video explainer.

This looks to me like a thorough, well-researched proposal which addresses the concerns raised during discussions of previous proposals for named arguments (including one Ehsan Akhgari and I co-authored), most notably by making the declaration syntax opt-in and the argument names encoded in the function type.

I look forward to seeing this presented to EWG in the future.

Library Evolution Working Group (LEWG)

LEWG has identified the following areas as priorities to focus on during the C++23 cycle. Note that not all of these may ultimately make it into C++23; executors, networking, and coroutine support are particularly at risk.

• Executors. The approach here continues to evolve; an earlier proposal involving properties has been abandoned in favour of std::execution.
• Networking. This continues to be tied up in questions surrounding its interaction with executors, and whether networking support in the C++ standard needs to come with support for TLS out of the box. It’s unlikely to make C++23.
• Library support for coroutines, including std::generator and std::lazy
• Modules in the standard library. There’s a minimal proposal (single module), as well as a more forward-looking one.
• Improvements to formatting, including formatted output and formatting ranges.
• Improvements to Ranges. There are various proposals, collected in this overview paper.
• mdspan, a multi-dimensional array view
• Text and unicode (this area has its own study group, and its overall direction is outlined here)
• Extended floating-point types

Additionally, std::expected, a type representing a value or an error (similar to Rust’s Result) has been approved by LEWG for C++23 and is currently awaiting wording review.

Here are some other proposals that LEWG has reviewed but which are not headed for C++23:

• Linear algebra, with significant proposals including a set of fundamental linear algebra types and functions, and a complementary proposal for a free function interface based on BLAS.
• std::hive, a “bucket array” data structure
• An extension to stack_trace to allow getting a stack trace from an exception (representing the point where the exception was thrown)
• Extending the type system to provide API and ABI flexibility. This is an ambitious core language proposal, but it’s motivated by library needs (how to evolve library interfaces while respecting ABI compatibility concerns), hence its initial review by LEWG.

The Future of Library Technical Specifications

One of the recent LEWG reports contained a section discussing the future of library Technical Specifications, which describes a shift away from TSes in favour of putting library facilities directly into the next International Standard. A key part of the justification is:

Technical Specifications provide implementation experience, but they do not deliver the levels of usage and deployment experience, or user feedback, that we had wished for.

This matches a similar observation I made about language Technical Specifications in a previous trip report.

Study Groups

The committee has at least 18 active study groups (see list here). I haven’t been following all of their work in detail, but I’ll call out a couple:

Reflection Study Group

Progress on Reflection continues to be slow, with no proposal ready for standardization in the C++23 time frame.

Interestingly, while recent proposals (like this one) have focused on constexpr, value-based interfaces, there has also been recent implementation progress on the older, template metaprogramming based Reflection TS. Matúš Chochlík, the author of the original reflection proposal that substantially shaped the Reflection TS, has implemented the Reflection TS in a fork of clang (also available on Compiler Explorer).

I view this as an exciting development because it opens the door to gathering concrete usage experience with reflection facilities; even if that experience is using a different syntax than what we’d ultimately like to standardize, it’s likely to produce valuable and actionable feedback that can inform the standardization process going forward.

Tooling Study Group

One of the areas of focus of the Tooling Study Group remains promoting interoperability in the ecosystem surrounding C++20 Modules usage and distrbution. Notable proposals in this area include a proposal for a format for describing the dependencies of source files, and a proposal for a format for describing information to facilitate the consumption of pre-built module artifacts.

Future Plans

The committee is starting to wrap up work on new C++23 features, with the deadline for design approval of new features being February 2022, and the deadline for the C++23 draft wording to be feature complete (and sent out for its first, Committee Draft ballot) being July 2022.

Collaboration for the time being continues to be remote. As of this writing, the earliest in-person meeting not to be definitively cancelled is the one in July 2022; it remains to be seen whether we will in fact be able to hold this meeting in person.

Whenever the committee does resume in person meetings, they’re likely to (at least initially) be hybrid, meaning there will be A/V equipment to allow continued remote participation for those who prefer it.

I know there are many subgroups and topics I haven’t covered; if you’re interested in one in particular, please feel free to ask about it in a comment.