An Update on Writing Memory Safety Bugs

How to Build Abstractions in Rust Applications: The Missing Rung on the Rust Education Ladder

2023-12-27T00:00:00+00:00

I read yet another reddit post where an OO developer struggles with building abstractions in Rust. These things feel familiar to me, I have had the same struggles moving from being a C++ developer to building a non-trivial Rust application and trying to use abstractions throughout to reduce coupling. And I see colleagues at work ask the same things as they start to pick up Rust.

https://www.reddit.com/r/rust/comments/18rme0v/interface_abstractions_in_rust/

To me, the most helpful reply in here was a reframing or changing of perspective. Instead of building a storage system, which is behaviour and state wrapped up in a typed object, build a function that needs to use a storage system and provide the functions it needs through a generic trait.

Another way of saying this could be to move the abstraction out of the incoming object (a virtual object) and into the function signature (a generic function). But I feel like it does not really explain why that is better, though it is more idiomatic. In fact, you could reword what I just said as “use impl T instead of Box<dyn T>” but that seems to miss the actual point that is hiding inside this advice.

Another way to think about it could be in terms of design direction. Write the code that needs the trait first, then define the trait in terms of what was needed, and provide the impl. But what if you have two teams, and you want one to go build the storage system for the other team? You don’t want to block on them building their application on top of some no-op storage system which you will then later use to design an API with. You want the storage system team to go write an API and provide it as a service. How it’s consumed, by generic or trait object, is not ineherently tied to the order in which they are developed, but it has big implications for how the abstraction is used and built. And since working with concrete types is very different from working with trait objects, starting from concrete types can lead to a better outcome without explicitly stating the underlying intentions.

There’s an intense need for OO developers to build a typed object that provides functionality. That is a basic part of the recipe for abstraction C++ and Java. So it’s extremely natural for devs to end up at Box<dyn T>, which provides for this need but does so very poorly and in a way that ends up feeling like you’re fighting the Rust language.

Trait objects can be useful for small independent implementations with a narrow scope (like an iterator, or a closure), but as a tool for building application abstractions they impose a ton of pain on both the implementor of the abstraction and the consumer. For the implementor, they end up building everything into this one trait. Even Clone, as we see mentioned in the above Reddit discussion. Downcasting is another example which you have to roll yourself through your trait API. Then the consumer of your trait can’t make use of standard vocabulary and traits, and everything becomes quite full of friction and complexity.

Since trait objects APIs become monolithic, they lead developers toward composing traits together through “inheriting”. This is a totally normal thing to do in C++, and the syntax even looks the same, but combining traits in this way to build godlike trait objects does not lead to happy times. There is lots of documentation around explaining that trait inheritance is different in Rust and that you should not use it in this way, but less on how to build abstractions for your application correctly in spite of this.

Restating the given advice in another way, it says to use “generic objects which provide a trait impl” instead of “trait objects”. Then there is no need to make the traits object-safe at all, which is great because requiring it could get in the way of providing the ideal interface, such as preventing the use of generics within the interface. Not that you’d put generics in a C++ abstraction, since you can’t combine generics and virtual in C++, so this doesn’t seem like a problem worth worrying about to the new Rust developer.

It is helpful advice to Rust devs that functions should receive abstractions as generics instead of as trait objects. But it’s missing the underlying intention of why a trait object seemed like the better fit. There’s two main reasons that come to mind:

Testing

It is standard practice in large C++ codebases, take Chromium and Blink for example, to build virtual abstractions around pieces of production code and then replace the implementation with something else when running tests. This is especially true for integration tests, but it is a mechanism used exhaustively (almost certainly over-used) for unit tests as well. I don’t think advice that handwaves at how generics make mocking harder are doing service to this intent.

Blink’s WebTest harness is a good example of this type of system. The production code provides all kind of functionality to Blink, but in tests various things are stubbed out and replaced in order to control inputs that would normally be coming from devices, or the user, or the network. This is foundational for being able to reliably test the Blink implementation of the web platform.

With generics, it is still possible to write tests for functions which make use of an abstraction in isolation, but you can no longer write a factory that returns an implementation of some abstract behaviour, and replace the whole implementation in tests. So writing integration tests of a large system appears impossible without trait objects, unless… the entire codebase is parameterized over a set of generic traits. There may be a single production implementation of all those traits, maybe even a single type/object which implements all those traits. Teams can freely write traits and implementations for those traits in their own modules. And when building for tests, a different type may be selected at compile time, which comes with different implementations.

If this should be a blessed design pattern, Rust education needs to lean into it when teaching developers about abstractions in Rust. Current education seems heavily skewed toward authoring libraries. And while you can think of an application as a collection of libraries, it is also meaningfully different in how it all comes together as one huge unit, and how concepts may have to span across many parts of a codebase, and be implemented by many teams. It should be expected that a large application comes with a need to provide abstractions that differ in testing. And thus it would be expected up front that most of your application, at least the core glue of the application, is parameterized over generics.

This is not at all how you would write C++ or Java, so it will not be obvious to new Rust application developers.

Heterogenous collections

Applications are full of collections of heterogenous objects. We use dependency injection and type erasure to build collections of these objects. For instance, lists of callback/task objects or collections of modules that are written by many teams, and are built on top of your core system but are decoupled from it and which you cannot name. Abstractions are used to provide this decoupling, and in Rust you quickly land on collections of trait objects to make things compile, since Box<dyn T> is a consistent size. This feels familiar and consistent with a std::vector<std::unique_ptr<VirtualClass>> in C++. Except the latter works well in the language (while causing security bugs, sorry I had to say it) while the former causes a lot of friction for the developer in Rust.

The “write your code with a generic parameter” solution will not suffice here. As soon as a Rust application developer goes from a single generic object to multiple heterogenous ones, the strategy for abstraction breaks down and creates confusion.

There are better (I think? I haven’t used these myself) ways to do this heterogeous-type-erasure-with-ownership than trait objects, though they appear to be incredibly complicated to implement. The Bevy game engine does this in order to build an ECS model.

Here’s some articles about doing type erasure for dependency injection:

In this case, I think there’s an opportunity in both Rust education and in the language to make this less difficult.

Integer overflow and arithmetic safety in C++

2023-11-29T00:00:00+00:00

Another 0-day security bug in Chrome has been found being used “in the wild”. Once again, it was built on the bug class of integer overflow. This is not the first integer overflow used in an 0-day against Chrome, even within this year.

When a security bug is used in the wild, it means that it is being used to attack users of that software and that the vendor found out about it. These attacks are going on all the time, with no real way to actually know what bugs exist and are being used like that until you find one being used. The ones that go undetected can remain in active use indefinitely, until the software vendor makes systemic changes to make those types of bugs go away.

Software vendors never talk about how a 0-day security bug is being used, so news media doesn’t have much to go on and could only really speculate. However these bugs are worth millions of dollars. They can be used for things that hurt people and society, like espionage and intelligence, war, assassination, theft and ransomware, oppression, surveillance, and supporting human rights abuses. And when a piece of software becomes ubiquitous, the reach of attacks in that software grow, as should the responsibility to stop them from happening.

Integer overflow in C++

Chrome is written in C and C++ (with a tiny touch of Rust now), and thus is highly vulnerable to the classes of security bugs that affect these languages. Integers in C and C++ follow some very problematic rules that cause these types of bugs:

They will implicitly convert to smaller types, truncating values and changing the valid range.
They will implicitly change sign, changing the meaning of their value and their valid range.
Unsigned integers will implicitly wrap around on overflow.
Signed integers will cause Undefined Behaviour (essentially, mis-compilation) on overflow.
Additionally, converting from floating point values to integers can also cause Undefined Behaviour.

Integer overflow is a common 0-day target in C and C++ codebases. Developers don’t write software expecting integers to overflow. So when they do, the program gets into an unexpected state, and can become a useful weird machine for an attacker.

Every major C or C++ codebase that is ubiquitous across user devices, and interacts with remote data like images or websites will be similarly valuable to attackers, and vulnerable to the same classes of bugs. And we see active 0-day bugs being reported in such software products regularly (for example, iOS and Android, Safari and Firefox ).

The safety of C++ is a hot topic right now, with CISA calling out languages that lack the ability to prevent these types of security bugs, specifically C and C++, and advocating for the use of memory-safe languages.

This has led to a bit of a reckoning in the C++ language design community. The CppNow 2023 conference had multiple talks on C++ Successor Languages, which positions moving off C++ as the strategy to stop putting customers and users into harm’s way.

Carbon: Carbon Language Successor Strategy: From C++ Interop to Memory Safety - Chandler Carruth
Swift: Introducing a Memory-Safe Successor Language in Large C++ Code Bases - John McCall
Cpp2: The Evolution of C++ - A Typescript for C++ - Herb Sutter
- This is the least “moving off C++” of the three, but still entails working in a new syntax/language even if it shares a compiler.

There’s also been many talks on making C++ itself safer at CppNow, CppCon, and elsewhere. For example:

Most recently, Dr. Stroustrup gave the Delivering Safe C++ talk at CppCon 2023.

The first challenge for the C++ language design community has seemed to be the task of figuring out what “safety” means. The vast majority of software exploits are written against memory safety bugs, and CISA has been clear that memory safety is the issue they are concerned with. I think the C++ community is slowly coming to focus on this as well, though its leaders do still want to consider a broader scope of safety for better or for worse.

In “Delivering Safe C++”, Dr. Stroustrup talks about profiles to address some forms of safety, and one of those is arithmetic. This to the class of bugs with integer and floating point types, like integer overflow, for which C++ currently lacks the guardrails to protect developers and their customers alike.

Integer overflow elsewhere

There’s two other important languages that C++ developers should be thinking about for production code right now: Swift and Rust.

John McCall talks about Swift’s position on integer overflow in this video from CppNow. The language does not allow integer overflow. The designers consider an overflowing value to be a bug in the program which leads to the program misbehaving, and they consider it better to stop the program than to let it continue running in a state it was not designed for. This is completely in line with what you’d want to protect your software and your device from being exploited.

Rust is known for being memory-safe yet only traps on integer overflow in debug builds, by default. Chrome enables overflow trapping in Rust for release builds too, for now, but how does Rust get away without catching overflow in release builds by default? I believe it is primarily because Rust makes pretty much all pointer access go through slices, which include bounds checks. There are no memcpy, memset, malloc, or alloca functions that take a pointer without a bound. You use Vec for memory allocation and slices provide safe APIs for copying contiguous ranges of data around. It is still sketchy to allow overflow to go unchecked however, and definitely leaves your program in an unexpected state which has the potential to be abused. However it’s critical to note that Rust defines the behaviour of overflow in release builds even if it’s allowed; there is no Undefined Behaviour with signed integers as we see in C++.

What about upcoming potential languages?

So far, Carbon takes the same position as C++ for release builds, with Undefined Behaviour on signed overflow, and no trapping on overflow in general. It’s worth noting the plan includes trapping in debug builds, like with Rust. Though the high level of interop with C/C++ may leave the language with a similar level of vulnerability as C++, depending on how code ends up being ported to or written in Carbon.

Zig makes overflow into Undefined Behaviour in release builds and panic/trap in debug builds. They provide wrapping arithmetic operators to make it explicit and lightweight, but the risk of Undefined Behaviour in release builds would leave the language vulnerable to the same class of bugs.

C has all the same problems as C++ but fewer ways to address them, since it does not allow creating abstractions with the same power as in C++. C has recently gained library functions for performing checked arithmetic. But opting in explicitly to safety means the defaults will still do the dangerous thing and bugs will persist.

Solving integer overflow (and the class of arithmetic bugs) in C++

C++ gives us the opportunity to build abstractions that change or redefine behaviour of the code we write. A great example of such an abstraction in the memory-safety space is raw_ptr<T>, aka MiraclePtr. This is an abstraction around a native pointer that works with the allocator to keep the memory from being reclaimed and reused while there’s a dangling pointer to it.

Over 2022 and 2023 I have been building an abstraction for numeric types that drop in as a replacement for primitive types (like int, float, uint32_t, etc.) and which eliminate the class of arithmetic unsafety bugs in C++.

These new types are a drop-in replacement as they implicitly convert to and from primitive integer types, when there is no truncation or sign change required. This allows them to be used while working with APIs built on primitive types. And it allows the migration of APIs to use them even while callers continue to use primitive types. However, they will catch at compile time any callers that were previously losing potential data in primitive types.

These types protect against integer overflow by panicking in the default operations (+, -, *, /, %). But since they are a user-defined type, they can provide a rich API for handling overflow dynamically, saturating, or wrapping. And they can give you Undefined Behaviour on signed overflow if you really want it, but you have to ask for it in an explicit way that both you and your code reviewer can see.

Read more about the types in the API documentation here: https://suslib.cc/sus-namespace.num.html.

For basic stuff, they work just like primitive types. i32 takes the place of int in most cases. And they work with streams or fmtlib out of the box.

void fib(i32 n) { 
    if (n < 3) return; 

    i32 fn = prev1 + prev2; 
    prev2 = prev1; 
    prev1 = fn; 
    std::cerr << fn << " ";
    fmt::print("{} ", fn);
    return fib(n - 1); 
} 

Designing is expensive and risky, and creating new APIs for folks to learn is a problem. We have enough APIs! So rather than make something completely new, the numeric APIs are modeled after Rust’s numeric APIs.

std::optional<usize> calculate_size(usize elem_size, usize length, usize offset) {
    auto [acc1, oflo1] = elem_size.overflowing_mul(length);
    auto [acc, oflo2] = acc1.overflowing_add(offset);
    if (oflo1 | oflo2) {
        return std::nullopt;
    } else {
        return acc;
    }
}

If you are familiar with Rust, then you already know the provided APIs, there’s nothing more to learn. If you’re working in a mixed-language codebase, you can use the same APIs and expect the same behaviour regardless of where you happen to be writing code that day.

Making overflow checks optional

I really want to stop seeing integer overflow bugs in the news, and for me that means catching overflow as it happens in production, as integers in Swift will do.

However applying these checks to existing code can be risky:

There are unknown stability risks. What if there are load-bearing overflows that you were not aware of? Maybe they are a bug that you would fix, once you know about it, but crashing the program would be catastrophic. Or maybe you want to preserve the overflows and apply wrapping behaviour.
There are unknown performance risks. Using safe numeric types with overflow checks does imply some performance costs, and without the ability to turn them on and off, it’s impossible to verify what the cost is and do the performance work needed to drive those costs down through different algorithms (such as iterators instead of indexing) or with explicit unchecked arithmetic.

With the Make overflow checks optional at compile time PR that landed today, the overflow checks in these types are now able to be turned on and off at compile time. Crucially, like Rust, when disabled the overflow behaviour of signed integers is still well-defined, the conversions between integers and floating point types are all well-defined, uninitialized memory is always avoided, and invalid type conversions are still prevented at compile time.

These safe numeric types correspond to the overflows and unanticipated conversions safety profile proposed by Dr. Stroustrup, where different types of codebases can opt into different behaviour. While a piece of software that exposes you to attackers on the internet (like a browser, or a phone OS) should do everything possible to keep you safe, there are classes of software that do not have to deal with security threats. Yet even there, when debugging, the ability to turn on runtime checks can save many hours.

By using a new set of types, instead of a compiler warning, it is possible to apply safer arithmetic and numeric type safety across an existing codebase in an incremental manner. By making overflow checks optional, it is possible to look for overflow in fuzzers and tests and work toward preventing overflow in production in an incremental manner.

There’s still some work to do on these types and their associated types in the library, and a sharp edge in the language to work through, but they are already proving useful in my experience. The type safety prevents doing the wrong thing by accident, and the overflow checks make debugging any surprises almost trivial, instead of the gnarly debugging sessions that C++ normally makes me think of. I am intending to reach a public release of them very soon.

Complexity of reference containers

2023-09-24T00:00:00+00:00

While we can mourn the lack of holding references in std::optional, I want yall to know that holding references in a C++ type (or, appearing to anyway because it’s got to be stored as a pointer) is incredibly tricky and subtle.

Subspace can represent references in Option<T&>, Result<T&, E>, Choice<...>, and Tuple<T&, U&, ...>.

I think this is a very powerful and useful thing, especially as Option<T&> is represented internally as just a pointer, no extra bool. This makes programming mistakes into very clear and actionable errors instead of UB and failures that require debugging and backtracking to even know what state was wrong.

But I think this set of vocab types is it, the value/cost drops off quickly, so I don’t think I will want to support references in any other types.

What cost? Implementing support in a template for T being a value or a reference adds a lot of complexity to the implementation - which to be fair would be intractable before C++20 in my opinion. It adds a ton of testing complexity to make sure you’re actually handling the references correctly and that you’re never testing a concept against a reference which then answers for the pointee type, or acting on a reference in a way that reaches through to the pointee.

But also, receiving and storing a reference can lead to implicit and completely hidden memory safety bugs.

Memory safety bugs

Let’s take for example an Option<const int64_t&>. You can construct this Option by giving it a reference to an int64_t lvalue, or to an rvalue.

This is already a problem but one that is visible in the code at least. If you pass an rvalue to something that captures it as a reference, it will dangle. With [[clang::lifetimebound]] annotations, you get a warning/error too. Which I put every effort into making exhaustive, though it’s a challenge since it’s not possible to test for errors like these inside the language.

There’s a worse situation though. A const T& will also apply implicit conversions!

So if you construct an Option<int64_t&> but pass it an int32_t, C++ will helpfully construct a temporary int64_t and use the reference to that.

Now you have Undefined Behaviour from a dangling reference, but one you didn’t even write!

I wrote a concept that catches this scenario so that any methods that receive and hold a reference can reject this in a clear way. And it can be rejected in every compiler, not just through [[clang::lifetimebound]]. But this is yet another thing to test for and think about while building the APIs.

The concept is SafelyConstructibleFromReference.

I wrote down all the rules I had in my head today about building reference types in a markdown doc so that I can come back to them and can use them in code reviews in the future.

https://github.com/chromium/subspace/blob/main/STYLE.md#containers-that-hold-references

Seeing it in Compiler Explorer

Here’s a little example of these cases: https://godbolt.org/z/q6q7GWaEY

You can see Clang can make a warning for each of the UB cases, but the other two compilers don’t at all.

With the SafelyConstructibleFromReference concept, the last case which is very sneaky UB will be rejected on all compilers, though not the others.

If you remove some of the abstractions inside the Option there, GCC ends up seeing one of the three UB cases, but as we can see it’s easy for it to lose that.

On push_back_unchecked: Performance with FromIterator and Collect

2023-08-29T00:00:00+00:00

A very nice blog post went by yesterday titled “The Little Things: The Missing Performance in std::vector”.

You should read it, but in case you don’t, the premise is that std::vector::push_back is wasteful when you’re appending a bunch of things to a vector, since you can reserve space for them all but it keeps checking if it needs to allocate more space on every loop iteration. The author calls for a std::vector::push_back_unchecked method that appends under the assumption that space is already allocated. This exposes Undefined Behaviour if you do it wrong, of course, and the author argues that this is both necessary and good. And I agree.

This blog, and my efforts in the C++ language space, are all around reducing the negative impact of Undefined Behaviour and eliminating memory-safety bugs. So the above may seem counterintuitive at first glance.

I recently argued that the nature of the C++ standard library is basically to expose Undefined Behaviour:

This is the nature of bare C++; it is as close to the hardware as possible for historical and good reasons. The C++ standard intentionally leaves no room for a lower level language (except hardware assembly), and it exposes all the complexity of the compiler through the std lib (e.g. type_traits). This is an “assembly language for systems programming”.

And in that same vein, adding std::vector::push_back_unchecked would expose the nature of the conceptual C++ machine, and allow authors to control that machine more effectively.

Undefined Behaviour is Good.

Chandler Carruth argues for Undefined Behaviour from a slightly different perspective in his CppCon 2016 talk, Garbage In, Garbage Out: Arguing about Undefined Behavior With Nasal Demons. Undefined Behaviour gives us performance, as we can see clearly from the push_back_unchecked proposal.

But that sentence above is not complete without a qualification which usually gets missed in the C++ world. Completed, it reads:

Undefined Behaviour is Good when it is Encapsulated.

I believe std::vector should expose that Undefined Behaviour along with, and because it exposes, many other Undefined Behaviours. However that also implies that std::vector is not suitable for use in most application code. It is a building block on which performant APIs can be built which encapsulate the inherent unsafety of its own API that leaks Undefined Behaviour all throughout.

FromIterator and Collect

Subspace provides the FromIterator concept and then implements this concept for sus::Vec and std::vector as well as all the types in the standard containers library.

Like most of the Subspace library, this is a reimplementation of the Rust FromIterator trait. If not already familiar with the trait and its many uses, the key points for us here are:

You can construct a container from another container, or an iterator.
The construction happens in a single atomic step, from the perspective of application logic, which is typically the iterator collect method.
- See also the Rust collect for more code examples that will look similar in C++ but aren’t in the Subspace docs (yet).

By providing the ability to construct a container like a vector in an atomic operation from an arbitrary set of inputs, we provide the ability for the library to give a safe and simple API abstraction around a powerful operation which can be implemented entirely on top of APIs that expose Undefined Behaviour. Because of the atomic nature, the use of those unsafe APIs is fully encapsulated from the application layer, allowing the use of them in a controlled manner. Much like in how “safe Rust” applications are built on top of a stdlib full of unsafe Rust, yet retain memory safety for themselves, this creates the opportunity to build a safer security-critical C++ application without sacrificing performance. In fact, as we’ll see, we can gain performance.

Performance Optimizing FromIterator and Collect

The benchmarks provided in the aforementioned “Missing Performance in std::vector” article showed on Clang 15:

A 5x improvement when building a small to medium vector from a simple integer mapping.
A 1.3x improvement when building a large vector of the same.
A nearly 3x improvement when building a small to medium vector from a simple mapping function that unwraps a struct.
A 1.1x improvement when building a large vector of the same.

I was curious how FromIterator would compare with sus::Vec. If push_back_unchecked existed, the implementation of FromIterator for std::vector could absolutely make use of the method, with no risk of introducing Undefined Behaviour and memory safety bugs into the application as a result. For sus::Vec we have full access to the underlying data structure so we can do all the unsafe things internally that we want, and we should be able to see the same impact as with push_back_unchecked. Given that, how does FromIterator compare to the naive v.reserve(..); for (..) { v.push_back(..) } approach?

I copied the benchmarks from the above article and added them to the Subspace nanobench test suite.

At first, it was slow. Really slow. Vec does not define that it is empty after a move, instead the moved-from Vec is left invalid and Vec checks for use-after-move. Additionally, Vec tracks outstanding iterators and terminates if mutated instead of invalidating the iterator and proceeding with Undefined Behaviour. But the inner loop of FromIterator was checking these things over and over again. So I added some internal methods to break apart checks and implementation, and to split up methods for better inlining. Then I used local variables to avoid indirections through this-pointers.

The PR containing the changes and the benchmarks is https://github.com/chromium/subspace/pull/337.

Here’s the results of building sus::Vec via FromIterator compared to building a std::vector, on my M1 Mac with Clang 18:

A nearly 4.5x improvement when building a small to medium vector from a simple integer mapping.
A 1.7x improvement when building a large vector of the same.
A 1.5-2x improvement when building a small to medium vector from a simple mapping function that unwraps a struct.
A 1.03x improvement when building a large vector of the same.

Comparing FromIterator on std::vector is done to verify the iterator approach isn’t changing the nature of the operation, and indeed we see that putting the push_back loop into the collect() operation has no visible impact. But the structure enables something more.

benchmark	n	std::vector	std::vector + FromIterator	sus::Vec + FromIterator	std::vector / sus::Vec
copy + mult	1,000	426.00 +- 0.85 ns	452.67 +- 2.7 ns	98.45 +- 0.1 ns	432.7%
copy + mult	100,000	39.144 +- 0.04 us	37.95 +- 0.04 us	8.75 +- 0.41 us	447.4%
copy + mult	10,000,000	6.14 ms +- 0.04 ms	6.33 +- 0.03 ms	3.67 +- 0.03 ms	167.3%
to_index	1,000	351.07 +- 0.35 ns	350.30 +- 0.35 ns	176.97 +- 1.24 ns	198.4%
to_index	100,000	31.20 +- 0.031 us	31.20 +- 0.031 us	20.56 +- 0.88 us	151.8%
to_index	10,000,000	7.83 +- 0.031 ms	8.19 +- 0.057 ms	7.65 +- 0.038 ms	102.4%

The last column shows the relative speedup of the sus::Vec + FromIterator approach is compared to std::vector.

We have a similar speed improvement to what we saw by using push_back_unchecked, and without any Undefined Behaviour exposed to application developers.

A tale of C/C++ development in three parts

2023-08-20T00:00:00+00:00

Here’s what it’s like doing C++ development. I am writing a C++ application called Subdoc. It has to be in C++ because it has an important C++ dependency: Clang.

I would write it in Rust if it were easy to use Clang from Rust, but alas interop is not at that point.

Part One: Finding Greatness

To generate html comments from markdown needs a markdown parser. So I hunted the internets and found md4c.

What does the README have to say about it:

It is very fast.
It is easy to integrate into your application.
It is used by large projects who you’d expect to vet their dependencies, like Qt.

So I integrate md4c into Subdoc, and yes it’s quite easy to do so and it works great!

Part Two: Trouble

I want to add some extensions to md4c for Subdoc, so that I can generate headers as links to themselves, like you’d see on GitHub. To do that, md4c needs to provide additional signal to md4c-html, and md4c-html needs some additional callbacks.

So I go looking through the issue tracker to see what others are doing.

And I find an issue near the top: “Project Dead?”. Folks talk about trying to reach the developer on Twitter and that he has disappeared from development.

There are many forks of the project but none appear to be active. So the codebase is essentially dead in the water, abandonware. You get what you get.

So I fork and add some extensions to the project for my needs, and it’s working great.

Part Three: The C/C++ Security Wow Factor

I look a little more in the issue tracker, and I find:

And a pull request titled “Update md4c.c” which is fixing at least three more memory safety bugs.

So there’s at least 4 memory safety bugs in this piece of software. It’s kinda widely used. And now malicious markdown could compromise machines through Subdoc or other consumers of this library, if those bugs can be triggered with the flags they use. I was able to reproduce a timeout but not the OOB bugs so far.

md4c is written in C not in C++. But the reason I am using it is because I am working in C++. Otherwise I would be looking at the markdown crate library. Even though it is in an alpha state, it would still be more trustworthy for security than this well-used C library.

The bugs

What kinds of bugs are they?

One is using an error code as an index into an array. Oops.
The rest appear to be many occurances of integer overflow when subtracting two signed values and then using them as an unsigned value. Classic stuff.

So I have added a whole lot of asserts into the library to try prevent this. To actually feel safe I would need to turn it into C++ and apply Subspace numerics which would properly catch bad values at runtime from malicious inputs.

But no other users of md4c will get these asserts so… This is how C++ is going right now.

Comparing floats in Subspace and stdlib

2023-08-06T00:00:00+00:00

I am continuing to go through the sus::iter::Iterator methods and turn them all into constexpr based on the performance results from doing so. With constexpr an iterator of chunks_exact(), and take_while() is able to outperform a “bare-metal” loop over pointers. More on that over at Mastodon.

As I am doing so, I am writing tests, and this test felt really good to write!

static_assert(sus::Vec<f32>::with(2.f, 3.f, 2.f)
    .into_iter()
    .max_by(&f32::total_cmp) == sus::some(3.f));

So I interrogated myself why it did. First off, I was happy that you can’t write this wrong. You can’t call max() on floats because they are not strongly ordered. In other words, operator<=>() on floats returns std::partial_ordering, not std::strong_ordering. That means that floats satisfy the PartialOrd concept but not the Ord concept. And max() requires Ord.

This is true for float as well, but f32 is nicer to work with for reasons like the NAN constant and that is_nan() is constexpr unlike std::isnan() until C++23.

// Doesn't compile, as floats are not strongly ordered. Constraint does not match.
// error C7500: 'max': no function satisfied its constraints
sus::Vec<f32>::with(2.f, 3.f, 2.f).into_iter().max();

The f32::total_cmp() method provides a total ordering over floats and does return std::strong_ordering. It does the same thing as f32::total_cmp in Rust std. So we can use it as a callback in max_by().

What does the standard ranges library do when you try to max floats, I wondered. At least on MSVC this is what I got.

// Ok sure, looks good.
static_assert(std::ranges::max(
  std::vector<f32>({2.f, 3.f, 2.f})) == 3.f);
// At compile time, NAN is largest.
static_assert(std::ranges::max(
  std::vector<f32>({2.f, 3.f, f32::NAN})).is_nan());
// Unless 3 comes after.
static_assert(std::ranges::max(
  std::vector<f32>({f32::NAN, 3.f, 2.f})) == 3.f);
// At run time, different answers. 3 is largest if it comes first.
EXPECT_EQ(std::ranges::max(
  std::vector<f32>({2.f, 3.f, f32::NAN})), 3.f);
// At run time, NAN is larger if it comes first.
EXPECT_EQ(std::ranges::max(
  std::vector<f32>({f32::NAN, 3.f, 2.f})).is_nan(), true);
// Undefined behaviour!!!
EXPECT_EQ(
  std::ranges::max(std::vector<f32>()), 0.f);

Those results are really not okay! They are the kind of thing that doesn’t show up in tests and then completely takes down production. Maybe introduces a security vuln.

Here’s what I got with Subspace iterators.

// Reproducible ordering.
static_assert(sus::Vec<f32>::with(2.f, 3.f, 2.f)
  .into_iter()
  .max_by(&f32::total_cmp) == sus::some(3.f));
static_assert(sus::Vec<f32>::with(2.f, 3.f, f32::NAN)
  .into_iter()
  .max_by(&f32::total_cmp).unwrap().is_nan());
static_assert(sus::Vec<f32>::with(f32::NAN, 3.f, 2.f)
  .into_iter()
  .max_by(&f32::total_cmp).unwrap().is_nan());
// Same thing at runtime as at compile time.
EXPECT_EQ(sus::Vec<f32>::with(2.f, 3.f, f32::NAN)
  .into_iter()
  .max_by(&f32::total_cmp).unwrap().is_nan(), true);
EXPECT_EQ(sus::Vec<f32>::with(f32::NAN, 3.f, 2.f)
  .into_iter()
  .max_by(&f32::total_cmp).unwrap().is_nan(), true);
// Defined behaviour. ^_^b
static_assert(sus::Vec<f32>::with()
  .into_iter()
  .max_by(&f32::total_cmp) == sus::none());

Sure you can just write a total_cmp() method yourself and use that as the comparator with the standard ranges library. But you don’t have to, it compiles anyway so there’s nothing suggesting something is wrong. And then it returns garbage or corrupts your compilation with Undefined Behaviour.

With Subspace total_cmp() is already there for you, and non-sense inputs don’t compile.

This stuff is why Rust is pleasant to work in. C++ can be too.

Implementing Iterator::flatten() in C++ and in Rust

2023-07-01T00:00:00+00:00

You may have heard that C++ has concepts in version 20. You may have heard these compared to traits in Rust. Indeed you can do many of the same things with them. Today I found them to be a good demonstration of the complexity of writing C++ vs Rust.

Here’s the Iterator::flatten() method in Rust:

The Items in the Iterator need to each be convertible to an Iterator as well. Then the resulting Iterator will return all the items from those Iterators. As in “Iterator[Iterator[i32]] => Iterator[i32]”.

IntoIterator

IntoIterator is the trait that flatten() used, if it’s satisfied, the type can be converted to an Iterator via calling into_iter():

The trait has an associated type called Item, which is what the created Iterator will return.

It requires that the type returned by into_iter() actually implements Iterator by constraining it with the IntoIter rule.

The Iterator::flatten() implementation

The actual implementation of flatten() is a single function call:

Which creates a Flatten type that is the resulting iterator, the “IntoIter” type mentioned in the IntoIterator trait above.

The Flatten iterator type

The Flatten type is also pretty simple. The implementation relies on a FlattenCompat abstraction to share implementation with flat_map():

The where clause on Flatten points out once again that the Items in the outer Iterator can be converted to (or are) Iterators themselves.

The Flatten implementation

The Flatten::next() method, which is used to implement the Iterator trait returns a U::Item type:

The bound here requires that the Iterator we’re flattening has Items that satisfy IntoIterator, and we assign names to the associated types in IntoIterator.

The U is a name we assign to the return type of the into_iter() method in the IntoIterator trait, as we see by the assignment to IntoIter. The Items from those inner iterators are U::Item.

So now we’re going to return from the Flatten type’s Iterator implementation the inner types. This is not the most simplest of APIs but the type relationships are pretty straightforward.

What does this look like in C++ Subspace?

Iterator::flatten() in C++

Here’s the C++ version of Iterator::flatten(). It’s in the IteratorBase class, not the Iterator concept. In Rust this goes directly in the trait, but you can’t add default methods to a C++ concept as they only provide boolean yes/no matching against types. So instead this lives on IteratorBase, which all Iterator types are required to inherit from (as final) in order to satisfy the Iterator concept.

The method has a concept requirement, much like the Rust trait bound on IntoIterator. But it’s called IntoIteratorAny. Why the “Any” part?

The IntoIterator concept in C++

We have an IntoIterator concept in C++ too. It has a type T which is the type that can be converted to an Iterator, and something like Rust’s associated type, Item, as a second template parameter.

If a type satisfies IntoIterator, then we know it can convert to an Iterator that returns Items through the into_iter() method. And the return type of into_iter() is constrained to satisfy the Iterator concept, like the Rust trait.

The C++ complexity cracks start to show here. In the Rust generics and type system when you have a type you.. Well you have a type. But in C++ you have to worry about whether that type is const, or volatile, or an lvalue reference, or a const reference, or an rvalue reference! You do some 6D chess in your head to figure out just what type of thing you want in all of those situations, which you want to accept or reject.

Here we don’t care about reference input types, we want to know that when we have an rvalue of the type, and we call into_iter() on it, we get an Iterator, so we need to use std::remove_cvref_t. Hopefully that was the right choice, and not std::remove_const_t<std::remove_reference_t> or some other configuration - it’s hard to ever be completely confident.

Getting the IntoIterator Item type

But why didn’t we use IntoIterator on the flatten() method? Originally, I did that, as I’ve done with other methods, however for flatten() we don’t know apriori what the types are inside the inner Iterators.

That was not a problem in the Rust flatten() method at all, it constrained Self::Item to IntoIterator and moved on. But to use the IntoIterator concept in C++ we need to pass two types, the type-to-be-converted as well as the Item. But the Item is not known here. We can figure it out though, by seeing what Iterator type gets returned from into_iter() and getting the Item type off of it:

typename std::decay_t<decltype(std::declval<std::remove_cvref_t&&>().into_iter())>::Item

Lol. Since that’s not decipherable, here it is as a picture.

But putting that in random code is horrifying.

Adding the IntoIteratorAny concept in C++

So to avoid writing inscrutable type traits on our flatten() method, we shove it behind another concept, IntoIteratorAny. It’s a concept that is satisfied for a type that can convert to an Iterator through the into_iter() method, without placing a bound on the Item type of the returned Iterator:

We don’t need two traits to express this same thing in Rust, but I have not found a way to avoid it and keep the flatten() method concise in C++.

The body of Iterator::flatten() in C++

The body of the Rust flatten() method was simply Flatten::new(self). The C++ method has a bit more going on, all of which is hiding a ton of other machinery.

  using Sized = SizedIteratorType<Iter>::type;

The Sized alias is a template instantiation of SizedIterator, which allows us to type-erase Iterator types, and then nest Iterators inside each other.

  using Flatten = Flatten<typename IntoIteratorOutputType<Item>::Item, Sized>;

The sus::iter::IntoIteratorOutputType type alias is a helper used to figure out what Iterator type will be constructed from the call to into_iter() on a type that is IntoIterator. The Flatten type’s bounds in Rust allowed us to name that type U, as the trait has an associated type called IntoIter. Concepts in C++ don’t come with associated types, so you must use independent type inspection to get to it instead.

Here’s the implementation of IntoIteratorOutputType:

Note that the IntoIteratorAny concept and the IntoIteratorOutputType alias were handled entirely by the generic bounds of the IntoIterator trait in Rust in a concise and simple way. C++ required a lot more typing, a lot more complexity, a lot more room for mistakes. You can express more too, but at what cost.

The C++ Flatten Iterator type

Last, the method call to Flatten constructs the Flatten type which is the returned Iterator. We move “this” into the Flatten type by converting it into the SizedIterator.

  return Flatten::with(make_sized_iterator(static_cast<Iter&&>(*this)));

The Flatten type is a class that subclasses InteratorBase, like we mentioned before. It’s [[nodiscard]], which Rust achieves by putting the same in a single place, on the Iterator trait. The “InnerSizedIter” is like the “I” type on the Rust version of Flatten. The [[sus_trivial_abi]] attribute marks the class as “clang::trivial_abi” if your compiler is Clang, which generates a warning on other compilers so has to go behind a macro.

Conclusion and Iterator concept

And with that we have the tools to write Iterator::flatten() in Rust or in C++ with Subspace.

For the interested, here’s the Iterator C++ concept, which has its fair share of worrying about const, references, and lvalue/rvalue-state embedded in the types:

Bounds Safety: Avoiding Death by a Thousand Constructors

2023-06-05T00:00:00+00:00

tl;dr We are going to see how Subspace types allow a zero-cost transition from native pointers to bounded view types, which enable spatial memory safety, and that can’t be achieved with the standard library types.

Long ago, C and C++ programmers passed around pointers to arrays, hopefully with a size to indicate how large the array was in order to avoid reading/writing out of bounds, though frequently not. This is still a common pattern, though “modern C++” has provided tools to eliminate this class of memory safety bug, through the use of “view” types. These are types like std::string_view or std::span in the standard library. Unfortunately, these types don’t actually check for using memory out of bounds in the C++ standard.

Much has been written about the memory unsafety of lack of bounds checking, and we (as in the software industry) have known ways to solve this simple issue for decades now.

Here’s an article from 1996 (almost 30 years ago) proposing a bounds-checked structure for interacting with OLE.
Here’s a NIST paper titled “C++ in Safety Critical Systems” from 1995 which states “Class libraries that provide smart (safe) pointers and array bounds checking should be used.”

In 2022, The Clang/Libc++ project accepted the C++ Buffer Hardening proposal that makes it possible to enable hardening checks inside these standard library types with bounds - though it requires you to compile the standard library yourself with the option enabled. The compiled copy of the standard library that shipped with your operating system likely doesn’t have this enabled. At least your web browser may have this enabled (Chrome has since Q1 2023, but I don’t know about any other browser, Chromium-derived or not).

The same proposal introduces the -Wunsafe-buffer-usage warning in Clang which you can enable to ban pointer arithmetic. This is very useful to help a project convert all their native array pointers into view types, which pair a size with the pointer through the type system. See, the problem is that even if you enable bounds checking in std::span and std::string_view, the majority of C++ code really doesn’t use those types. The large C++ codebases that power modern technology are still full of pointers to arrays. So this warning gives a mechanism to start migrating away from pointers, and then prevent backsliding, working toward banning pointer arithmetic entirely outside of a few exceptional types.

Performance implications

Introducing memory safety into C++ codebases always comes with some performance degradation, because there’s little you can check at compile time. So it means checking for bad program states when the program is running. These performance changes are not the kind you as a user will see or feel, but the kind that shows up on benchmarks and that software teams then have many discussions about. And these often end up blocking meaningful improvements to the safety of users and their data.

In bounds checking, the compiler can see the values involved in the comparisons, as they come from constants or other values related to the array. For instance when iterating through a span, stopping before the span’s length, the compiler can see that every index into the span is less than its length fairly trivially. This usually means the compiler removes the bounds checks. So no runtime overhead, yay!

But there’s another type of overhead that comes not from the bounds checks in view types like std::span or std::string_view, but from the change to using those types at all.

Here’s a commit that converts a bunch of functions to use base::StringPiece, the Chromium string view type which is much like std::string_view: Make histogram functions take base::StringPiece.

The functions were previously overloaded to take const std::string& or const char*. The const char* overload presumes that:

the string being passed in is null-terminated, and
the function should be consuming the full string up to the null character.

These assumptions are easily wrong when working with view types that encode a bounded view of an array. Calling these functions requires converting that bounded string view into a std::string which involves generating a constructor, and quite possibly a heap allocation. But many callers instead convert their bounded string view into a const char* through .data() instead, which is a bug. Props to @davidben for noticing this common anti-pattern in Chromium and working to fix the many places it occurs.

So it appears at first glance to be a win to change these functions to take a string view type, in this case base::StringPiece. For callers with a std::string, it will copy the pointer and length into the base::StringPiece. For callers with a bounded string view type, they don’t need to throw away the bounds or allocate on the heap. Yet this created a pretty large binary size regression for doing almost nothing, and trying to eliminate heap allocations.

Chromium is an interesting test bed because it is an enormous codebase. So when you make a small change, it can easily be amplified immensely. In this case, we discovered that all those calls to a base::StringPiece constructor grow the binary by 36KB. For unknown reasons, the base::StringPiece constructor gets inlined, while the std::string_view constructor does not, so you get less binary size growth with that type (and we’ll look at marking base::StringPiece with a noinline attribute).

In Chromium, when a change impacts performance, we can often just look for where binary size grew to track down where the regression lies. So we’ve seen binary size growth correlate well with performance loss. In this case we see strictly more codegen at each call site, which can certainly become visible at scale in performance testing. So this also suggests a performance degradation by moving from pointers to a bounded view type. And herein lies the problem. A project may successfully flip libc++ hardening on for their std::span types without seeing much of a performance impact, because the majority of bounds checks are elided. But then as they convert more pointers to std::span to increase their coverage of bounds safety, the use of std::span may introduce a different kind of performance (and binary size) regression.

Here’s a code example (in compiler explorer):

#include <stddef.h>
#include <vector>
#include <span>
#include <fmt/core.h>

void take_vector(const std::vector<int>& v) {
    fmt::println("{}", v[0]);
}

void take_pointer(const int* p, size_t s) {
    if (0 >= s) abort();
    fmt::println("{}", p[0]);
}

void take_span(std::span<const int> s) {
    fmt::println("{}", s[0]);
}

int main() {
    std::vector<int> a = {1, 2, 3, 4, 5, 6, 7};
    take_vector(a);
    take_pointer(a.data(), a.size());
    take_span(a);
}

We will look at the resulting code gen unoptimized, to keep things simple. But the above commit demonstrated the same principle under optimizations.

Here’s the call to take_vector():

lea     rax, [rbp-80]
mov     rdi, rax
call    take_vector(std::vector<int, std::allocator<int> > const&)

This is three instructions. Grab the pointer, store it, and call the function. Nice.

Here’s the call to take_pointer():

lea     rax, [rbp-80]
mov     rdi, rax
call    std::vector<int, std::allocator<int> >::size() const
mov     rbx, rax
lea     rax, [rbp-80]
mov     rdi, rax
call    std::vector<int, std::allocator<int> >::data()
mov     rsi, rbx
mov     rdi, rax
call    take_pointer(int const*, unsigned long)

That’s a lot more instructions, as there’s two function calls first. Then store the two values and make the call. This is why we had a const std::string& overload for those functions. In the case the caller already had a decomposed pointer/length, this would result in just something like the last three instructions, which is as good as take_vector().

And here’s the call to take_span():

lea     rdx, [rbp-80]
lea     rax, [rbp-48]
mov     rsi, rdx
mov     rdi, rax
call    std::span<int const, 18446744073709551615ul>::span<std::vector<int, std::allocator<int> >&>(std::vector<int, std::allocator<int> >&)
mov     rdx, QWORD PTR [rbp-48]
mov     rax, QWORD PTR [rbp-40]
mov     rdi, rdx
mov     rsi, rax
call    take_span(std::span<int const, 18446744073709551615ul>)

Oof, that’s a lot more instructions than take_vector(). Calling this function with a vector has changed three instructions into ten, and that’s without inlining the std::span constructor.

I guess it’s easy to see why the binary size changes when we start to receive std::span in function parameters instead of a reference to the owning container.

What if we had a pointer and length already, but we wanted to call take_span() instead of take_pointer(), is that any better? Something like this:

auto p = a.data();
auto s = a.size();
take_span({p, s});

That turns into the following when calling take_span():

mov     rdx, QWORD PTR [rbp-32]
mov     rcx, QWORD PTR [rbp-24]
lea     rax, [rbp-64]
mov     rsi, rcx
mov     rdi, rax
call    std::span<int const, 18446744073709551615ul>::span<int*>(int*, unsigned long)
mov     rdx, QWORD PTR [rbp-64]
mov     rax, QWORD PTR [rbp-56]
mov     rdi, rdx
mov     rsi, rax
call    take_span(std::span<int const, 18446744073709551615ul>)

It’s one more instruction to call take_span() when you have a pointer/length pair than to call take_pointer() when you have a std::span!

Conversions can be costly for codegen in C++, which is why they should pretty much always be explicit, even for (if not especially for) these small vocabulary types. I go further to take the point of view that conversion constructors are harmful to understanding what code you are generating, even if explicit, and static constructor methods are better.

But anyway, what do we do? I want to stop out-of-bounds bugs in C++ code from constantly being exploited. I want to stop the people who want to steal all of your private data. Silent and persistent compromise, which allows theft of your past, present and future data, is not as rare or as difficult as it should be, and that is due in part to memory unsafety bugs like out of bounds memory accesses.

Making the conversion free

While walking my puppy today, it occurred to me that it is possible to have that conversion to std::vectorfrom std::span be free; to generate the same code when calling take_span() as when calling take_vector(). Well, no okay it’s not possible with the standard library. But it is possible with the Subspace library.

Let’s expand our example a bit (in compiler explorer):

void take_vector(const std::vector<int>& v) {
    fmt::println("{}", v[0]);
}

void take_pointer(const int* p, size_t s) {
    if (0 >= s) abort();
    fmt::println("{}", p[0]);
}

void take_span(std::span<const int> s) {
    fmt::println("{}", s[0]);
}

void take_span_ref(const std::span<const int>& s) {
    fmt::println("{}", s[0]);
}

void take_vec(const sus::Vec<int>& v) {
    fmt::println("{}", v[0]);
}

void take_slice(sus::Slice<int> s) {
    fmt::println("{}", s[0]);
}

void take_slice_ref(const sus::Slice<int>& s) {
    fmt::println("{}", s[0]);
}

int main() {
    std::vector<int> a = {1, 2, 3, 4, 5, 6, 7};
    take_vector(a);
    take_pointer(a.data(), a.size());
    take_span(a);
    take_span_ref(a);

    auto b = sus::Vec<int>::with_values(1, 2, 3, 4, 5, 6, 7);
    take_vec(b);
    take_slice(b);
    take_slice_ref(b);
}

We’ve added a call to take_span_ref() which receives a const std::span&. Unsurprisingly this does not improve anything, here is the codegen for that call:

lea    rdx,[rbp-0x60]
lea    rax,[rbp-0x30]
mov    rsi,rdx
mov    rdi,rax
call   404b2e <std::span<int const, 18446744073709551615ul>::span<std::vector<int, std::allocator<int> >&>(std::vector<int, std::allocator<int> >&)>
lea    rax,[rbp-0x30]
mov    rdi,rax
call   403acf <take_span_ref(std::span<int const, 18446744073709551615ul> const&)>

We have to construct a std::span from the std::vector in order to give a reference to it in the function call, so this comes out at eight instructions, without the std::span constructor being inlined.

Now let’s look at the Subspace types.

First, receiving a const sus::Vec& looks the same as const std::vector&, which isn’t surprising:

lea    rax,[rbp-0x70]
mov    rdi,rax
call   403c75 <take_vec(sus::Vec<int> const&)>

We grab the pointer, store it and call the function, for three instructions.

And receiving a sus::Slice looks much like receiving a std::span:

lea    rax,[rbp-0x70]
mov    rdi,rax
call   404bf0 <sus::Vec<int>::operator sus::Slice<int>&() &>
mov    rdx,QWORD PTR [rax]
mov    rax,QWORD PTR [rax+0x8]
mov    rdi,rdx
mov    rsi,rax
call   403e1b <take_slice(sus::Slice<int>)>

We construct a sus::Slice and then pass that by value to take_slice(). This actually comes in two instructions shorter than the call to take_span(), but it’s still way more codegen than passing the const sus::Vec& directly.

But lastly, we have the call to take_slice_ref(), which now differs significantly from the equivalent take_span_ref() with the standard library types:

lea    rax,[rbp-0x70]
mov    rdi,rax
call   404bf0 <sus::Vec<int>::operator sus::Slice<int>&() &>
mov    rdi,rax
call   403fd1 <take_slice_ref(sus::Slice<int> const&)>

Now we’re down to five instructions. We call the conversion operator to sus::Slice&, which returns a pointer that we store, and then call take_slice_ref() with it.

But why is this operator returning a pointer, instead of a sus::Slice by value? The trick here is that sus::Vec is implemented by holding a sus::SliceMut inside it, which holds a sus::Slice inside it¹, which is where the pointer/length pair is finally found. This means that Vec can convert to a SliceMut and Slice without any constructor being invoked.

With optimizations

Ok let’s turn some optimizations back on and see what we get.

Calling take_vector(const std::vector&) grabs the pointer and invokes call:

lea    rdi,[rsp+0x10]
call   4035e6 <take_vector(std::vector<int, std::allocator<int> > const&)>

Calling take_span(std::span) inlined the std::span constructor for us, which sets its two data members and then invokes call:

mov    rdi,rbx
mov    esi,0x7
call   403719 <take_span(std::span<int const, 18446744073709551615ul>)>

Calling take_span_ref(const std::span&) is worse, as it constructs a std::span inline, then has to take its address. No doubt that’s where we get the guidance from to pass it by value:

mov    QWORD PTR [rsp+0x30],rbx
mov    QWORD PTR [rsp+0x38],0x7
lea    rdi,[rsp+0x30]
call   4037ab <take_span_ref(std::span<int const, 18446744073709551615ul> const&)>

Calling take_vec(const sus::Vec&) looks similar to receiving const std::vector&, as it grabs the pointer and invokes call:

mov    rdi,rsp
call   403840 <take_vec(sus::Vec<int> const&)>

Calling take_slice(sus::Slice) looks similar to receiving std::span, inlining the construction of a sus::Slice, then passing it by value:

mov    rdi,QWORD PTR [rsp]
mov    rsi,QWORD PTR [rsp+0x8]
call   4038e3 <take_slice(sus::Slice<int>)>

Calling take_slice_ref(const sus::Slice&) is where things differ in a meaningful way:

mov    rdi,rsp
call   403981 <take_slice_ref(sus::Slice<int> const&)>

The call to take_slice_ref() uses the exact same instructions as the call to take_vec(). This means we are able to use a bounded view type (sus::Slice) as a function parameter without taking a binary size and performance cost at every function call that moves from an owning type to a view type. And that’s something that we can’t do with the standard library types.

Benchmarks

Here’s all of the options in Quickbench, compiled with GCC. We call the function once per iteration, and the function does a single indexing operation into each slice. The const sus::Slice<T>& option comes out on top, exactly equal with const sus::Vec<T>& which aligns with what we see in the assembly code above.

Note: We don’t call the function a bunch of times, like we do with Clang, because GCC ends up setting up the stack for the function call once and then just doing call successively, which is not representative of the whole function call operation.

And here’s the same thing in Quickbench, but compiled with Clang 15. We call the function 21 times per iteration to avoid Clang mostly measuring the benchmark harness itself, and the function again does a single indexing operation into each slice.

Benchmarks Result

Across both compilers, we can see that receiving a const sus::Slice& parameter from an owning vector argument is as efficient as const sus::Vec& or const std::vector&, while all three are more efficient than receiving std::span.

Library choices that got us here

There’s a couple differences in Subspace that work together to get us to this point.

First, the nesting strategy of Vec{SliceMut{Slice{T*}}}. This is not done with inheritance as you might first expect. That would allow a const Vec& to be used as a const Slice&, but it would also allow a const Vec& to be used as a const SliceMut&. A SliceMut provides a mutable view of the array, meaning this would be a backdoor through const Vec& to trivially mutate the data inside it. So instead Vec and SliceMut provide operators to convert to references of their nested types which provide the library control over when the types should be allowed to convert. This ended up very similar in implementation to the Deref trait in Rust, which is what inspired me to try this at all.

Secondly, the splitting of Slice and SliceMut into different types. Originally I had designed Slice to specify the const-ness of its view through the template parameter, like you would with std::span. That is Slice<const T> was a const view and Slice<T> was mutable. This made hiding methods that should not exist on a const view much more annoying, as they would need to each grow a requires(!std::is_const_v<T>). For programmers reading the API these are purely noise when you have a Slice<const T>, making the API harder to read and use and hack on. I think the standard library type gets away with this by putting almost no methods on std::span, as it has a methodology of providing free functions (like std::find()) instead. This type-splitting means that it’s much simpler to have a const SliceMut<T>& still implement a mutable view of the array. A const sus::Slice<T> would have been more deceptive.

Since the standard library differs on both of these, I don’t think the same strategy would work as well in that environment, even if standard was willing to break backward compatibility to get this performance win.

Thanks

Thank you @noncombatant for the excellent proofreading and suggestions for this post.

For Vec to convert to SliceMut in this way, it must hold a SliceMut inside for Vec to return a reference to the SliceMut. Likewise, SliceMut can convert to a Slice since gaining const is a valid operation, though not the inverse. Thus SliceMut contains a Slice and can convert to Slice by returning a reference to it. ↩

Subspace Update: Slices and Vec, Generators, More Iterators, Std Ranges, Callable concepts

2023-04-30T00:00:00+00:00

I’ve spent a few months mostly working on an enormous branch, so Subspace hasn’t seen a lot of changes on main. Today I merged a lot of stuff, which makes it a good time to mention some of the new things.

Slices and Vec

Micro post: https://sunny.garden/@blinkygal/110289198753682615

Today in Subspace I finished and merged all the methods for Slice and SliceMut.

Slice<T> is a const reference to a contiguous range of const T (like a std::span<const T>), while SliceMut<T> is a mutable reference to a contiguous range of T (like a std::span<T>). Why are these different types? First, this allows the default, shorter wording to be the safer and preferred one: const is default. Secondly, it’s not possible to sort() on a Slice, but if they were one type, it would require all methods that require mutable access to the pointed-to range to be qualified with requires (!std::is_const_v<T>). The API gets harder to understand and to use as a result. This way, the SliceMut type has methods that allow mutation, and Slice simply does not.

There are 95 methods in all! I started working on these two months ago on March 3. The Pull Request is here: https://github.com/chromium/subspace/pull/218/.

I’m kinda relieved to finally get through them all. There’s lots of useful stuff in there though, including:

searching
sorting
splitting
sub-slicing with ranges (including range literals)
iterating (forward and reverse) on splits, chunks, and sliding windows
joining
filling or repeating
working with prefixes and suffixes
swapping
reordering

These methods all appear on Vec as well, which is an owning Slice. In addition, Vec is usable in all places that a Slice or SliceMut (if the Vec is not const) would be usable (except as an rvalue, which would allow references to escape from a temporary object). And similarly SliceMut is usuable anywhere a Slice would be.

I am sure There’s lots of room for performance tuning, and some TODOs left in this regard. As always everything comes with unit tests, so making future changes can be done with confidence.

Range literals

In the C++ standard library, the std::span type allows subslicing through the subspan(offset) and subspan(offset, count) overloads (implemented as a default argument). Then, s.subspan(offset) in C++ is equivalent to &s[offset..] in Rust and s.subspan(offset, count) in C++ is equivalent to &s[offset..(offset + count)] in Rust.

But Rust is more expressive here. Its RangeBounds can describe any possible range with or without a front edge or back edge. .. is everything, 2.. starts from 2 and goes to the end, ..5 starts at the beginning and stops at 5 (excluding 5), and 2..5 starts at 2 and ends at 5 (excluding 5). It even allows 2..=5 to include 5 in the bounds.

Instead of providing a bunch of overloads for this type of expressivity, Subspace slices (and Vec) can be subsliced using the operator[](sus::ops::RangeBounds<usize> auto) operator, which aborts if given a range out of bounds, or get_range(sus::ops::RangeBounds<usize> auto) and get_range_mut(sus::ops::RangeBounds<usize> auto) which return a sus::Option holding nothing if out of bounds or a Slice (or SliceMut respectively) otherwise. The RangeBounds concept is satisfied by a set of types provided in sus::ops that specify a start, an end, both, or neither:

sus::ops::Range has a start and end (exclusive).
sus::ops::RangeFrom has a start, with an unbounded end.
sus::ops::RangeTo has an end, with an unbounded start.
sus::ops::RangeFull represented an unbounded start and end.

For dynamic values, these can be constructed from numeric values as they are aggegrate types. For constant values, a literal syntax is provided. C++ doesn’t let us have all the nice things, so we can’t use the exact literal syntax from Rust, as C++ requires non-quoted literals to parse as numbers even if you’re going to do the parsing yourself. But we get this:

"3..7"_r is a Range<usize> from 3 to 7 (exclusive).
"3..=7"_r is a Range<usize> from 3 to 7 (inclusive).
"3.."_r is a RangeFrom<usize> that starts at 3.
"..7"_r is a RangeTo<usize> that ends at 7 (exclusive).
".."_r is a RangeFull<usize> that represents an unbounded range.

Range types can be modified to produce new ranges with the .start_at() and .end_at() methods. These change, or add, a start or end bound, changing the type of the object when a new bound is added. For instance ::sus::ops::RangeFull().start_at(3) makes a ::sus::ops::RangeFrom(3).

Mixing literal values with dynamic values is possible as well with these methods, such as a range that starts at 5, but ends at x: "5.."_r.end_at(x).

Certainly this is nowhere as nice as Rust’s 5..x syntax. Maybe C++ can give us that one day. But I want to provide the tools to write literal ranges when code authors would benefit from doing so, and the range type aggregate constructors are always available too.

Bringing this back to slices, a Vec can produce a slice for any range using its operator[], get_range() or get_range_mut():

auto vec = sus::Vec<i32>::with_values(1, 2, 3, 4, 5);

auto s1 = vec[".."_r];  // A SliceMut over [1, 2, 3, 4, 5].
auto s2 = vec["3.."_r];  // A SliceMut over [4, 5].
auto s3 = vec["..3"_r];  // A SliceMut over [1, 2, 3].
auto s4 = vec["2..3"_r];  // A SliceMut over [3].

Lastly, for signed ranges, there’s also a _rs literal suffix that produces ranges over isize instead of usize, such as "3..12"_rs to make sus::ops::Range<isize>(3, 12). But slices always work with ranges over usize.

Iterators

While I was slogging through the slice methods, I also added some exciting iterator features.

Generator functions

Micro post: https://sunny.garden/@blinkygal/110044818048764779

It’s now possible to write an Iterator that will compose nicely with all the existing Iterator methods, but as a single function, instead of having to write a whole type. This is thanks to C++ coroutines, as they allow writing a generator function to modify iteration.

This provides a means to write “control flow” into an iteration, in the spirit of https://without.boats/blog/the-registers-of-rust/.

Here’s the generator unit tests to demonstrate what I mean.

Here the x() function produces an iterator over the set 1, 2, 3, 4. This is composed with the filter() method that keeps only things in the range (1, 4) (exclusive). The result is 2, 3.

TEST(IterGenerator, ComposeFromGenerator) {
  auto x = []() -> Generator<i32> {
    co_yield 1;
    co_yield 2;
    co_yield 3;
    co_yield 4;
  };

  // Generator is trivially relocatable, so no need to box() it.
  sus::iter::Iterator<i32> auto it =
      x().filter([](const i32& a) { return a > 1 && a < 4; });
  EXPECT_EQ(it.next().unwrap(), 2);
  EXPECT_EQ(it.next().unwrap(), 3);
  EXPECT_EQ(it.next(), sus::None);
}

Here the x() function composes with an input iterator, and it does the same as the filter() call above, discarding anything outside the range (1, 4) (exclusive). It is chained with an iterator from a Vec<i32> that contains 1, 2, 3, 4 and again the result is 2, 3.

TEST(IterGenerator, ComposeIntoGenerator) {
  auto x = [](sus::iter::Iterator<i32> auto it) -> Generator<i32> {
    for (i32 i : it) {
      if (i > 1 && i < 4) co_yield i;
    }
  };

  sus::iter::Iterator<i32> auto it =
      sus::vec(1, 2, 3, 4).construct<i32>().into_iter().generate(x);
  EXPECT_EQ(it.next().unwrap(), 2);
  EXPECT_EQ(it.next().unwrap(), 3);
  EXPECT_EQ(it.next(), sus::None);
}

Enumerate

Micro post: https://sunny.garden/@blinkygal/110223458907555381

I also have added Iterator::enumerate. Adding the sus::iter::Enumerate type was fairly trivial, but in order to iterate backwards as a sus::iter::DoubleEndedIterator, it needs to know the total length of the iteration sequence. That is, the input iterator needs to also be a sus::iter::ExactSizeIterator. While these concepts were already present in the library, I had not plumbed them through composing iterator types. So that is now done.

This means, for example, calling filter() or map() on a DoubleEndedIterator will produce another DoubleEndedIterator. And when possible, such as when calling rev() on an ExactSizeIterator, the resulting reversed iterator will also be an ExactSizeIterator. And in the case of enumerate(), if the input iterator was a DoubleEndedIerator and ExactSizeIterator, the output iterator will also be.

Notably, iterators over a Vec, Array or slice are double-ended and have an exact known size and thus satisfy these traits. The same will be true for most

Here’s an example where we have an iterator over 1, 2, 3, 4, 5. It is double-ended and its exact size is known. The iterator is reversed with rev(), so it will output 5, 4, 3, 2, 1. Then it is composed with enumerate() which means each step includes the position in the iteration sequence.

auto chars = Vec<char>::with_values('a', 'b', 'c', 'd', 'e');

{
  auto it = sus::move(chars).into_iter().rev().enumerate();

  // enumerate() makes an iterator over a Tuple of position and value.
  using E = sus::Tuple<usize, char>;
  static_assert(sus::iter::Iterator<decltype(it), E>);
  // The output iterator of enumerate() is a DoubleSidedIterator and
  // ExactSizeIterator.
  static_assert(sus::iter::DoubleEndedIterator<decltype(it), E>);
  static_assert(sus::iter::ExactSizeIterator<decltype(it), E>);

  // The output the position paired with the reversed values.
  for (auto [pos, val]: it) {
    // Prints:
    // (0, e)
    // (1, d)
    // (2, c)
    // (3, b)
    // (4, a)
    std::cerr << "(" << size_t{pos} << ", " << val << ")\n";
  }
}

Side note: In this example, the usize position is casted to a primitive to print, as I did not yet decide on a path for integrating with streams or to otherwise print things.

Stdlib Ranges

Subspace’s first major compatability hook with the C++ standard library was added, with conversions from sus::iter::Iterator types to C++ ranges.

Calling .range() on any iterator will produce a std::ranges::range output, which satisfies the std::ranges::input_range concept.

Here are some unit tests that demonstrate the behaviour, showing that the output from .range() is usable as a std::ranges::viewable_range and a std::ranges_input_range. In these examples, the iterator owns the values as vec is consumed to produce the iterator through into_iter().

TEST(CompatRanges, ViewableRange) {
  sus::Vec<i32> vec = sus::vec(1, 2, 3, 4, 5, 6);

  // all() requires a `std::ranges::viewable_range`.
  auto view = std::ranges::views::all(sus::move(vec).into_iter().range());

  i32 e = 1;
  for (i32 i : view) {
    EXPECT_EQ(e, i);
    e += 1;
  }
  EXPECT_EQ(e, 7);
}

TEST(CompatRanges, InputRange) {
  sus::Vec<i32> vec = sus::vec(1, 2, 3, 4, 5, 6);

  // filter() requires a `std::ranges::input_range`.
  auto filter = std::ranges::views::filter(sus::move(vec).into_iter().range(),
                                           [](const i32& i) { return i > 3; });

  i32 e = 4;
  for (i32 i : filter) {
    EXPECT_EQ(e, i);
    e += 1;
  }
  EXPECT_EQ(e, 7);
}

Closures/Callable concepts

Fn, FnMut and FnOnce are now concepts, not concrete types, just as the same are traits in Rust. Most notably they manage to take a function signature syntax in their template arguments while still producing useful errors when given a non-matching type. This was rather non-trivial as C++ concepts do not allow partial specialization, and referring to structs in a concept stops good error propogation.

Also the template arguments allow some simple pattern matching. sus::fn::Fn<sus::fn::Anything(i32)> auto will match any const-callable thing that takes an i32, regardless of its return type. And sus:fn::Fn<sus::fn::NonVoid(i32)> auto will match any const-callable thing that takes an i32 input and produces some non-void output.

I spent a week on this rewrite, which also introduced lightwight FnRef, FnMutRef and FnOnceRef that are concrete types satifying the above concepts and which refer to a function pointer or callable object like a lambda without owning any state. These types are super useful to receive a callable that will be used during a function but not stored beyond the lifetime of the function. And they don’t require your function to become a template, which will help keep down compile times.

Most Subspace methods that receive a callable now use these “Ref” implementation types instead of the concepts. Though limitations of type deduction still mean we need to use concept types in some cases, where the callable’s type signature is templated. For example with Option::map() where the return type of the callable determines the output of the map() function, using FnRef<R(T&&)> means the compiler can not deduce the R type:

note: 'Option<R> Option<int>::map(sus::fn::FnRef<R(T &&),>) noexcept &&':
  could not deduce template argument for 'sus::fn::FnRef<R(T &&),>'

The result is some pretty unweildly C++ concept bounds, and there’s a ton of space for C++ to improve here in the future:

  template <::sus::fn::FnOnce<::sus::fn::NonVoid(T&&)> MapFn, int&...,
            class R = std::invoke_result_t<MapFn&&, T&&>>
  constexpr Option<R> map(MapFn&& m) && noexcept {...}

The “Ref” types also can’t be used in a constexpr context (they rely on reinterpret_cast), but the concepts can be.

The old stateful, heap-allocated capturing types have been renamed to FnBox, FnMutBox, and FnOnceBox and they still exist for storing a callable past the lifetime of a function. For instance, Iterator::map() stores its callable as a FnMutBox on its output iterator. These types reason for being is to try provide guardrails for binding values instead of pointers, but these APIs definitely need some love and/or reworking.

Hopefully I will write a longer post about how the implementation of these concepts works, and the evolution of all these types.

Me versus C++ Mutable References.

2023-01-26T00:00:00+00:00

Me: Let me make a recursive operation. It will recurse through a heterogeneous data structure and call a function for each Thing it finds.

struct Q {
    Thing t;
    void for_all_things(sus::fn::FnMut<void(Thing&)>& fn) { fn(t); }
}

struct R {
    Thing t;
    void for_all_things(sus::fn::FnMut<void(Thing&)>& fn) { fn(t); }
}

struct S {
    sus::Vec<Q> qs;
    sus::Vec<R> rs;
    void for_all_things(sus::fn::FnMut<void(Thing&)>& fn) {
        for (Q& r: qs) q.for_all_things(fn);
        for (R& r: rs) r.for_all_things(fn);
    }
};

Me: Cool, let’s call it!

void blast_off(S& s) {
    s.for_all_things([count = 0_i32] (Thing& t) mutable {
        count += 1;
    });
}

C++: No. You can’t pass an rvalue as a mutable reference.

Me: Right, okay, I guess.

struct S {
    sus::Vec<Q> qs;
    sus::Vec<R> rs;
    void for_all_things(sus::fn::FnMut<void(Thing&)>&& fn) {
        for (Q& r: qs) q.for_all_things(fn);
        for (R& r: rs) r.for_all_things(fn);
    }
};

C++: No. You have a mutable reference to an rvalue now, so you have to move the reference to continue using it as such.

Me: But I’m not trying to pass ownership, I just want a mutable reference. Move would be misleading to read! Gr.

C++: Tough. It’s that or write a template.

Me: Fine. I will make an lvalue for you, for no good reason at all.

struct S {
    sus::Vec<Q> qs;
    sus::Vec<R> rs;
    void for_all_things(sus::fn::FnMut<void(Thing&)>& fn) {
        for (Q& r: qs) q.for_all_things(fn);
        for (R& r: rs) r.for_all_things(fn);
    }
};

void blast_off(S& s) {
    auto fn = sus::fn::FnMut<void(Thing&)>([count = 0_i32] (Thing& t) mutable {
        count += 1;
    });
    s.for_all_things(fn);
}

Me: But I am also going to blog about it.