Type Safety, Custom Iterators, and Runtime Structures in the Real World

I develop indie games live and decided to build my own prefab system, similar to that in Unity. This was the most difficult and technical challenge I’ve encountered in my career of building games. But the system saves time for designers with flexible and reusable objects, like an extremely powerful copy and paste mechanism. Today I present an overview of the data structures and objects used to create the prefab system. The article will explore creating custom iterators to make range-based loops, how type safety prevents mix-ups between different Keys (UUIDs), and how I approach json-like objects in C++. Finally I’ll share a decision I was unhappy with, and would approach differently with the hindsight I have today. Grab a barrel of your favorite beverage and get ready to learn something new! Clusters of Nodes I didn’t want Track Builder to be a generic editor, you know the typical scene-graph / object hierarchy. I thought I could make a better workflow specific to designing a racetrack without it, but as features were added the flow became clunky and was working about as well as my first car. Using Unity for the Eggcelerate! series convinced me to build a prefab system. Read the introduction article to discover more about the system and why I built it. The game engine has no entity-component system (ECS) built in, or at least didn’t when I started working on the prefab system. I dive more into this at the end of the article but it led to design decisions that I later regretted. To get started my “entities” are called “Nodes” and they basically have a name, a position and maybe a parent node. Any other behaviors or attributes are defined by the components that are attached to the node. TrackBuilder holds the nodes in a NodeCluster which contains; class NodeCluster; //Forward-declaration for PrefabTable using ComponentContainer = std::vector; using NodeHierarchyTable = std::unordered_map; using NodeComponentsTable = std::unordered_map; using PrefabTable = std::unordered_map; class NodeCluster { public: NodeHierarchyTable mNodeHierarchy; NodeComponentsTable mNodeComponents; DataResourceTable mResourceTable; PrefabTable mPrefabTable; ResourceKey mResourceKey; NodeKey mRootNodeKey; }; This is a terrible time to mention the C++ standard does not require all containers to work with forward declared types. GCC 11 did not like the PrefabTable declaration above as I shared on reddit. There is actually a lot of complexity to unpack in the above sample, it’s basically trees of trees of trees all the way down. A prefab is simply a NodeCluster as you can see from the PrefabTable declaration above. A NodeCluster also represents the full racetrack or level structure and when a Node inside (any) cluster has a valid PrefabKey it copies the properties from the prefab it points to unless that property was overridden. “If you don’t understand, don’t worry - I wrote this entire system with a barely functioning understanding of it.” An instance of a prefab is the node that actually exists that points towards the prefab to copy from. This instance can exist inside another prefab, like I can have a grandstand.pfab that contains two nodes; a flag and a spectator, where that spectator is an instance of the spectator.pfab. The grandstand of course could be an instance in another prefab itself and quickly you begin to see the complexity explode. When the designer changes the shirt color in the spectator.pfab, everything else has to update unless it was overridden elsewhere in the chain. If you didn’t understand that, don’t worry - I wrote this whole system and I barely have a functioning understanding of it. Explaining it is near impossible, too many edge cases, expanding problem area and the limited terminology for recursive depth does not help. Creating Type-Safe UUIDs to Prevent Accidents From the overview you can see the role ‘Keys’ are playing in this system to uniquely identify different elements. All keys are UUIDs but they shouldn’t intermingle or mix. We don’t want to confuse a NodeKey for a ResourceKey or such. By leveraging type-safety the compiler can protect us! The term ‘Key’ is used for aesthetics and my coding-standards, ComponentKey feels much better than ComponentID. sole::uuid was used to avoid implementing UUID specific details. using NodeKey = sole::uuid; using ComponentKey = sole::uuid; using ResourceKey = sole::uuid; // continued for other Key types Anyway, the above doesn’t offer any protection from writing code that mixes the different key types. Forgive this simplistic example, but you might see how it could accidentally occur in a more complex codebase. void FindNode(NodeKey& nodeKey) { /* do stuff */ } // Later, somewhere in code: { ComponentKey key = ComponentKey::uuid4(); FindNode(key); } Since building a wrapper around external code is a reasonable idea, we can combine that with a template to create additional ty

Mar 26, 2025 - 13:41

Type Safety, Custom Iterators, and Runtime Structures in the Real World

I develop indie games live and decided to build my own prefab system, similar to that in Unity. This was the most difficult and technical challenge I’ve encountered in my career of building games. But the system saves time for designers with flexible and reusable objects, like an extremely powerful copy and paste mechanism.

Today I present an overview of the data structures and objects used to create the prefab system. The article will explore creating custom iterators to make range-based loops, how type safety prevents mix-ups between different Keys (UUIDs), and how I approach json-like objects in C++. Finally I’ll share a decision I was unhappy with, and would approach differently with the hindsight I have today. Grab a barrel of your favorite beverage and get ready to learn something new!

Clusters of Nodes

I didn’t want Track Builder to be a generic editor, you know the typical scene-graph / object hierarchy. I thought I could make a better workflow specific to designing a racetrack without it, but as features were added the flow became clunky and was working about as well as my first car. Using Unity for the Eggcelerate! series convinced me to build a prefab system. Read the introduction article to discover more about the system and why I built it.

The game engine has no entity-component system (ECS) built in, or at least didn’t when I started working on the prefab system. I dive more into this at the end of the article but it led to design decisions that I later regretted. To get started my “entities” are called “Nodes” and they basically have a name, a position and maybe a parent node. Any other behaviors or attributes are defined by the components that are attached to the node.

TrackBuilder holds the nodes in a NodeCluster which contains;

class NodeCluster; //Forward-declaration for PrefabTable
using ComponentContainer = std::vector;
using NodeHierarchyTable = std::unordered_map;
using NodeComponentsTable = std::unordered_map;
using PrefabTable = std::unordered_map;

class NodeCluster {
public:
    NodeHierarchyTable mNodeHierarchy;
    NodeComponentsTable mNodeComponents;
    DataResourceTable mResourceTable;
    PrefabTable mPrefabTable;
    ResourceKey mResourceKey;
    NodeKey mRootNodeKey;
};

This is a terrible time to mention the C++ standard does not require all containers to work with forward declared types. GCC 11 did not like the PrefabTable declaration above as I shared on reddit.

There is actually a lot of complexity to unpack in the above sample, it’s basically trees of trees of trees all the way down. A prefab is simply a NodeCluster as you can see from the PrefabTable declaration above. A NodeCluster also represents the full racetrack or level structure and when a Node inside (any) cluster has a valid PrefabKey it copies the properties from the prefab it points to unless that property was overridden.

“If you don’t understand, don’t worry - I wrote this entire system with a barely functioning understanding of it.”

An instance of a prefab is the node that actually exists that points towards the prefab to copy from. This instance can exist inside another prefab, like I can have a grandstand.pfab that contains two nodes; a flag and a spectator, where that spectator is an instance of the spectator.pfab. The grandstand of course could be an instance in another prefab itself and quickly you begin to see the complexity explode. When the designer changes the shirt color in the spectator.pfab, everything else has to update unless it was overridden elsewhere in the chain.

If you didn’t understand that, don’t worry - I wrote this whole system and I barely have a functioning understanding of it. Explaining it is near impossible, too many edge cases, expanding problem area and the limited terminology for recursive depth does not help.

Creating Type-Safe UUIDs to Prevent Accidents

From the overview you can see the role ‘Keys’ are playing in this system to uniquely identify different elements. All keys are UUIDs but they shouldn’t intermingle or mix. We don’t want to confuse a NodeKey for a ResourceKey or such. By leveraging type-safety the compiler can protect us! The term ‘Key’ is used for aesthetics and my coding-standards, ComponentKey feels much better than ComponentID. sole::uuid was used to avoid implementing UUID specific details.

using NodeKey = sole::uuid;
using ComponentKey = sole::uuid;
using ResourceKey = sole::uuid;
// continued for other Key types

Anyway, the above doesn’t offer any protection from writing code that mixes the different key types. Forgive this simplistic example, but you might see how it could accidentally occur in a more complex codebase.

void FindNode(NodeKey& nodeKey) { /* do stuff */ }

// Later, somewhere in code:
{
    ComponentKey key = ComponentKey::uuid4();
    FindNode(key);
}

Since building a wrapper around external code is a reasonable idea, we can combine that with a template to create additional type-safety protection that ensures a NodeKey is different from a ComponentKey even if both are UUIDs under the hood like so;

template class MyUUID {
public:
    //wrapper API here
private:
    sole::uuid id
};

And change our type aliases to;

enum class NodeKeyType { };
using NodeKey = MyUUID;

enum class ComponentKeyType { };
using ComponentKey = MyUUID;

enum class ResourceKeyType { };
using ResourceKey = MyUUID;

// continued for other Key types

This will make the prior example fail to compile, since the NodeKey and ComponentKey types are different. I enjoy the compiler telling me when I’m doing unexpected things.

Custom Iterators for Maintainable Tree Recursion

Speaking of compiler assistance, while I am not a fan of syntactic-sugar for sugars sake, I am all for it improving maintainability! As a reader of a pretty technical article you probably already know what recursion is. Just in-case you played games during that lecture, like I definitely was, recursion is where a function calls itself, possibly many times over. A quick example;

int CalculateSum(int number) {
    if (0 == number) { return 0; } //The end case
    return number + CalculateSum(number - 1); // Recursive case
}

It is very useful for recursing trees, like the NodeHierarchy. A naive and quick way to RescurseTree() in a generic manner is to use a function callback that gets called for each node. The following differs from my Node contents but fits the common tree better.

void RecurseTree(Node& node, std::function callback) {
    callback(node);
    for (Node& child : node.mChildren) {
        RecurseTree(child, callback);
    }
}

The following shows how it would be used, although isolated from any other code like this example makes it a little easier to digest. Try placing it in a particularly busy area and you might see how it can mislead a future maintainer.

RecurseTree(rootNode, [](Node& node) {
    if (true == node.IsPrefabInstance()) {
        return;
    }

    node.DoSomething();
});

This code works, but it is hard to read. For instance what happens at the return? It acts just like a continue would in a typical loop, it does not actually return since it is contained within the callback lambda. Maybe just my personal preference, using continue like you can in a typical loop would be much cleaner. Worse, this can’t even break out of the loop early without requiring the callback to return a value for RecurseTree() to check.

For this reason I spent some time untangling iterators to make a much nicer recursion that actually supports continue and break properly. This was somewhat syntactic sugar, and I delayed the process longer than I should have, but it actually significantly helps writing and reading of the code. Even a slight improvement in the maintainability is a powerful lever in a system as complex as the prefabs.

for (Node& node : RecurseTree(rootNode)) {
    if (true == node.IsPrefabInstance()) {
        continue;
    }

    node.DoSomething();
});

The difference is small, especially when singled out like these examples. I didn’t have to write a lambda function or worry about how continue or break might work; it just does. The downside, if there is any, was the complexity moved from code managing the nodes into RecurseTree(). The following is the approach I took;

struct RangedBasedTree {
    typedef std::vector> ContainerType;
    ContainerType mRecursedNodes;

    RangedBasedTree(const Node& node) {
        RecurseTree(node, [this](Node& node) {
            mRecursedNodes.emplace_back(node);
        });
    }

    typename ContainerType::iterator begin(void) {
        return mRecursedNodes.begin();
    }

    typename ContainerType::iterator end(void) {
        return mRecursedNodes.end();
    }
};

RangedBasedTree RecurseTree(Node& node) {
    return RangedBasedTree(node);
}

I’ll leave supporting const correctness as an exercise for you, as a hint it may involve templating the RangedBasedTree while using std::conditional and std::is_const… Instead let’s dive into how the custom properties for components are stored when the types are unknown at compile time.

Structures Defined at Runtime

A very long time ago I read an article or perhaps one of Steve Meyers C++ books, I wish I remembered, and was inspired to create a semi type-safe but dynamically defined structure type, the DynamicStructure. The best way I have of describing this is using ‘json’ like objects in C++ code. Sure you don’t know the exact types at compile time, hence semi type-safe, but my implementation can throw exceptions when types mismatch or try implicit conversion.

Over the years I typically use the DynamicStructure for loading files like json, yaml etc, handling data from network requests and most recently holding the properties for a Component in the prefab system. Components have a type and definition but they are unknown to Track Builder, since components can be customized to a game.

class DynamicStructure {
public:
    using String = std::string;
    using Array = std::vector;
    using Struct = std::unordered_map;

private:
    enum DynamicStructureValueType : uint8_t {
        kNil, kInteger, kFloat, kBoolean, kString, kArray, kStructure
    };

    DynamicStructureValueType mValueType;
    union { //unnamed anonymous union for access like this->mFloat;
        char mRawBytes[8];

        int64_t mInteger;
        bool mBoolean;
        float mFloat;

        String* mString;
        Array* mArray;
        Struct* mStructure;
    };
};

Obviously the above sample is missing a ton of details like the whole public API, how constructors work and everything else, but I am fairly certain it shows enough to implement your own version from here. The real magic is in the union, although you must remember only to access the active member, which mValueType helps with.

Bringing this back to the Node / Component system, as stated earlier Track Builder can’t know the properties of a Component at compile time because those types can be defined by the game. These definitions are a json file. Someone more ambitious could probably add metadata in game-code with pre-build steps to generate the ComponentDefinitions file, but creating and maintaining a json file is much less effort.

{
    "version":2,
    "components":[
        {
            "display_name": "Hammer Obstacle",
            "type_key":"510fa5af-b454-41a3-9780-2c578a3cf645",
            "properties":[
                { "name":"timeout", "type":"float", "default":5.0 },
                { "name":"shockwave", "type":"boolean", "default":false },
                { "name":"shockwave_radius", "type":"float", "default":15.0 },
                { "name":"damage", "type":"integer", "default":25 }
            ]
        }
    ]
}

This component gets attached to hammers that crash down on cars, causing damage and perhaps a shockwave that does partial damage. Since TrackBuilder did not know any of this, at compile time, the properties are stored in a DynamicStructure and “just works”.

How Not to Store a Node Hierarchy

Not everything just works when we tackle big projects, and one of the major pain-points when working through the prefab system was how I organized the data. Specifically the duplication of a Node/Component (editor data) vs the Node/Component (in-engine). This entire series only explores the data side. While building the system, past-me built around an assumption that the engine would not be or have an ECS.

To be fair at the time the engine really didn’t have that support. Past-me figured we could get the benefits of working with ‘nodes’ and ‘components’ in the editor without further requirements. Past-me also hoped to dump the memory block to disk to save/load without any processing or dynamic resizing. As these decisions were made, the NodeHierarchy was a flattened structure, simply a std::vector, to which all benefits later dissolved.

Considering Components had custom properties this was all a very very silly idea, and portions of this have been refactored, but the legacy of cruft is thick and created functions like;

NodeKeyContainer GetChildren(const NodeKey& nodeKey,
    const NodeHierarchyTable& nodes)
{
    NodeKeyContainer children;
    for (const auto& nodeIterator : nodeHierarchy)
    {
        if (nodeIterator.second.mParentNodeKey == nodeKey)
        {
            children.push_back(nodeIterator.second.mNodeKey);
        }
    }
    return children;
}

This is just one of the many examples that leave room for improvement but requires a substantial refactor, all stemming back to the idea of this flattened hierarchy for saving/loading. The code should not be iterating the entire node hierarchy to get the children, and yet, it does. Perhaps I’m just leaving room for future optimizations.

This article was a journey through data organization, custom iterators to simplify recursion, type safe UUIDs to prevent mix-ups and design choices I would make differently with hindsight. Especially regarding the flattened structure of the NodeHierarchy. Writing complex systems is much easier with test-driven development which will be the next article in this series. Until then you can watch me make games live where I build racing games, tackle technical challenges, and share my experience. To catch the next article Follow HERE.