CompoConf - the lengthy version

June 19, 2025 compoconf composition configuration machine learning

CompoConf: Embracing Composition Over Inheritance in Deep Learning Configurations

(see also the self-written version)

Library Code: https://github.com/kpoeppel/compoconf

Deep learning projects often juggle complex model architectures and their configurations. Two prevalent approaches have emerged for managing these configurations: inheritance-based configs (as seen in libraries like Hugging Face Transformers) and composition-based design (as encouraged by PyTorch’s modular nn.Module patterns). In this post, we introduce CompoConf, a new Python library that brings the “composition over inheritance” principle to configuration management. We’ll explore why this principle matters for machine learning configs, how Hugging Face and PyTorch differ in their philosophies, and how CompoConf bridges a gap by enabling clean, type-safe, and composable configurations in Python.

The Case for Composition Over Inheritance

In software design, “favor composition over inheritance” is a well-known principle. Instead of building complex hierarchies of classes through subclassing, composition suggests building systems by combining smaller, interchangeable components. This leads to more flexible and maintainable code. The concept isn’t just academic – it can greatly simplify how we configure and extend machine learning models.

Why does this matter for configurations? In an inheritance-heavy approach, a single configuration class often attempts to cover many scenarios via subclassing. Every new model might require a new subclass with added fields, leading to deep inheritance trees or clunky conditional logic. In a composition-friendly approach, you’d instead assemble configurations from reusable pieces – for example, a top-level config containing sub-configs for each component of your system (model, dataset, training loop, etc.), each defined independently. This aligns with how we often build models from components and enables switching out parts of a system easily without rewriting large config classes.

Configurations in Today’s ML Libraries

Let’s briefly look at how two popular frameworks handle model configuration and see where they stand on the inheritance vs. composition spectrum.

Hugging Face Transformers: Inheritance-Based Configs

Hugging Face’s Transformers library uses an inheritance-centric configuration design. All models come with a PretrainedConfig base class, and each specific model defines its own config subclass (e.g. BertConfig, GPT2Config) that extends that base. These config classes hold all the hyperparameters and architecture settings for the model, and are tightly coupled to the model class itself. For instance, when adding a brand new model, one is expected to create a corresponding Config class (say BrandNewLlamaConfig) inheriting from PretrainedConfig, and attach it to the model class. Many common attributes (like hidden_size, num_attention_heads, etc.) are repeated across these classes.

While this design ensures that each model has a self-contained “blueprint”, it isn’t very compositional. Config classes don’t naturally nest or combine – if you wanted to, say, plug a BERT encoder into another model, you’d likely have to manually merge or translate configurations. The inheritance approach can also lead to large monolithic config objects that grow as models become more complex.

Downside: Limited modularity. You can’t easily mix and match pieces of configs – each config is essentially one big object tied to one model. This can make it harder to reuse components or to configure composite models that consist of multiple sub-models, because there’s no built-in notion of a config “containing” another config.

PyTorch: Composable Modules (But No Unified Config System)

On the other hand, PyTorch takes an extremely compositional approach to building models. Every neural network component in PyTorch is an nn.Module, which you can combine inside larger nn.Module classes. Composing layers or sub-networks is straightforward – you simply instantiate them and assign as attributes in your model’s __init__, then use them in forward. In fact, this is the recommended way to build complex models: “Compose layers into more complex structures by assigning them as attributes within a custom nn.Module subclass.”. PyTorch’s design encourages a HAS-A relationship (a model has submodules) rather than deep IS-A class hierarchies.

However, PyTorch by itself doesn’t prescribe any standard for how to configure these modules from external inputs. In practice, researchers and developers often end up using ad-hoc solutions: passing dictionaries of parameters, using command-line arguments, or adopting third-party configuration libraries. There’s no built-in equivalent to Hugging Face’s PretrainedConfig in vanilla PyTorch. Tools like Hydra and OmegaConf have become popular to fill this gap – they allow you to define structured configs (often as dataclasses) and compose them for different parts of an application. Hydra, for example, lets you organize YAML config files into groups and switch out components by changing a single line in a config file, which is a form of composition at the config-file level. But keeping things type-safe in hydra is quite complex. Even with such external tools, PyTorch users are left to implement their own configuration handling if they what to have interface compatibility and type-safety.

Downside: No native config integration. While you can freely compose modules, PyTorch doesn’t help you organize the configuration of those modules. The burden is on the developer to ensure consistency, type-checking, and ease of switching components. This often leads to either rigid code or a lot of boilerplate to parse configs and instantiate objects.

The Gap

In summary, Hugging Face provides a structured config system but leans on inheritance and monolithic design, whereas PyTorch gives ultimate compositional flexibility but no structure for configs. Ideally, we want the best of both worlds: composable architectures with equally composable, type-safe configurations. This is exactly the gap that CompoConf aims to fill.

Introducing CompoConf: Composition-Centric Configuration for Python

CompoConf is a Python library built to bring compositional design to configuration management. It provides a simple, declarative way to define configuration classes as dataclasses and wire them up with the actual implementation classes, supporting a flexible registry system. In a nutshell:

Type-safe configs: CompoConf uses Python’s dataclass types to define configuration schemas. This means your config fields are strongly typed (e.g. an int is an int), and the library can validate types for you.
Composition through nesting: Config classes can be nested within each other to mirror the composition of your actual objects. For example, a TrainerConfig can contain a model field which is itself a configuration for a model component, enabling hierarchical config structures.
Interfaces and implementations: You define abstract interfaces for components (e.g. “Model” interface) and then register concrete implementations (e.g. MLP, CNN, Transformer models) to those interfaces. This allows easy switching of components – the config just needs to specify which implementation to use, and CompoConf handles instantiating the right class. It’s a “pluggable” design achieved by a registry and a class_name field in the config.
Registry-based instantiation: Under the hood, CompoConf maintains a registry of all classes that have been registered for a given interface. When you parse a config, it can look up the specified implementation and create the object on the fly. No need for lengthy if/elif blocks or factory patterns in your own code – it’s handled generically.
Strict validation: The library will catch common errors like unknown config keys or missing fields (in strict mode) and ensure that the class you’re instantiating actually matches the expected interface. This reduces runtime surprises.
OmegaConf/Dataclass integration: CompoConf can integrate with OmegaConf (which Hydra is built on) if you want to load YAML or use Hydra’s compositional config files. You can directly parse an OmegaConf dictionary into CompoConf’s typed configs. In other words, it plays nicely with the broader ecosystem – you could use Hydra’s powerful config composition and then let CompoConf check type consistency and instantiate the objects, combining the strengths of both approaches.
Simplicity: It’s meant to be lightweight and unobtrusive. Your config classes are just dataclasses, and your model classes only need a couple of decorators to register. You don’t have to inherit special base classes except a few provided interfaces (ConfigInterface base, etc.), and there’s no huge framework to learn – just a few functions and decorators to get started.

According to the library’s documentation, “CompoConf provides a type-safe way to define, parse, and instantiate configurations for complex, modular systems.” In other words, it treats configuration as a first-class part of your modular design. Let’s see what using CompoConf actually looks like in practice.

Using CompoConf: Basic Example

To understand how CompoConf encourages composition, consider a simple scenario: you want to define a generic Model interface and two implementations (say a multilayer perceptron and a CNN). With CompoConf, you would do the following:

Define an interface for the component type. This is an abstract placeholder that represents, for example, “any kind of Model”. In code, an interface is a class that inherits from RegistrableConfigInterface and is marked with @register_interface. It doesn’t need to have any content – it’s just a name and type that we will register implementations to.
Define configuration classes for each implementation. These are simple dataclasses (subclassing ConfigInterface) that specify the parameters for that implementation. For instance, MLPConfig might have fields like hidden_size: int = 128 and num_layers: int = 2. Each such config class corresponds one-to-one with a concrete class that uses those parameters.
Register the implementation classes to the interface. We create the actual classes MLPModel and CNNModel (subclassing the interface, e.g. ModelInterface). Each class is decorated with @register and has a class attribute linking it to its config class (e.g. config_class = MLPConfig). By doing this, we tell CompoConf “this model class can be constructed via the MLPConfig, and it fulfills the ModelInterface contract.”
Instantiate via config. Now, instead of manually instantiating MLPModel(hidden_size=256, num_layers=2), we let CompoConf do it. We can create a config object MLPConfig(hidden_size=256, num_layers=2) and then call config.instantiate(ModelInterface). CompoConf will find that ModelInterface has a registered implementation MLPModel (since our config is of that type) and will call the MLPModel constructor, passing in the config object. The result is an MLPModel instance with the configuration applied. Alternatively, we could parse from a generic dictionary: if we have a dict like {"class_name": "MLPModel", "hidden_size": 256, "num_layers": 2}, CompoConf can parse it into an MLPConfig and again instantiate the model.

Let’s illustrate this with a bit of code (a toy example adapted from CompoConf’s documentation):

from dataclasses import dataclass
from compoconf import RegistrableConfigInterface, ConfigInterface, register_interface, register

# 1. Define an interface for models
@register_interface
class ModelInterface(RegistrableConfigInterface):
    pass

# 2. Define configuration dataclasses for each model type
@dataclass
class MLPConfig(ConfigInterface):
    hidden_size: int = 128
    num_layers: int = 2

@dataclass
class CNNConfig(ConfigInterface):
    kernel_size: int = 3

# 3. Define and register model implementation classes
@register
class MLPModel(ModelInterface):
    config_class = MLPConfig
    def __init__(self, config: MLPConfig):
        self.hidden_size = config.hidden_size
        self.num_layers = config.num_layers
        print(f"Initialized MLPModel with hidden_size={self.hidden_size}, layers={self.num_layers}")

@register
class CNNModel(ModelInterface):
    config_class = CNNConfig
    def __init__(self, config: CNNConfig):
        self.kernel_size = config.kernel_size
        print(f"Initialized CNNModel with kernel_size={self.kernel_size}")

# 4. Instantiate models via their configs
mlp_conf = MLPConfig(hidden_size=256, num_layers=4)
model_a = mlp_conf.instantiate(ModelInterface)   # creates an MLPModel instance
# Output: Initialized MLPModel with hidden_size=256, layers=4

# Alternatively, parse a config from a dict (could come from YAML/JSON)
raw_config = {"class_name": "CNNModel", "kernel_size": 5}
from compoconf import parse_config
cnn_conf = parse_config(CNNConfig, raw_config)   # parse_config returns a CNNConfig object
model_b = cnn_conf.instantiate(ModelInterface)   # creates a CNNModel instance
# Output: Initialized CNNModel with kernel_size=5

Even in this simple example, several powerful things happened:

We declared what our configuration looks like in a clean way (using dataclass syntax) and tied it to the code that uses it. This means if we add a new parameter to MLPConfig, it will be automatically available in the MLPModel constructor (via the config object) and validated.
We can easily switch the implementation by changing class_name in a config. If TrainerConfig references a ModelInterface, we could supply either an MLP or a CNN config (or any other registered model) without changing the training code. This is akin to dependency injection: the trainer just asks for a ModelInterface and doesn’t need to know which one it gets.
The instantiation is done centrally by CompoConf’s registry. This decouples our high-level code from the specifics of which class is being constructed. In fact, you could imagine having dozens of model implementations registered, and selecting them purely through config entries (which could even come from a file or command-line). This is very much in line with the idea of composing systems by plugging components together.

Nested Configurations and Composable Components

CompoConf’s design truly shines when you have configurations that themselves contain sub-configurations – reflecting a composed object graph. For example, consider a Trainer that has its own settings (like learning rate, number of epochs) and also needs a Model as a sub-component. Traditionally, you might hardcode a specific model class or pass in a model instance to the trainer. With CompoConf, you can make the trainer’s config include a field for the model config, using the interface type we defined above. For instance:

@dataclass
class TrainerConfig(ConfigInterface):
    model: ModelInterface.cfgtype  # this field expects a config of any ModelInterface implementation
    learning_rate: float = 0.001

# Example configuration data for the trainer:
trainer_cfg_dict = {
    "model": {
        "class_name": "MLPModel",   # specify which model implementation to use
        "hidden_size": 256
    },
    "learning_rate": 0.01
}
trainer_conf = parse_config(TrainerConfig, trainer_cfg_dict)

In the snippet above, ModelInterface.cfgtype is a special attribute provided by CompoConf that represents “the configuration type associated with any class implementing ModelInterface”. By using that as the type of the model field, we declare that TrainerConfig contains a sub-config for a Model. We then provided a dictionary where the model entry had a class_name and some parameters. When we call parse_config, CompoConf will:

Look at the class_name inside model (which is “MLPModel”).
Find the registered class for “MLPModel” (which we registered to ModelInterface).
Know that MLPModel expects an MLPConfig, so it creates an MLPConfig(hidden_size=256) from the remaining fields.
Set that as the model field of our TrainerConfig instance, and fill in learning_rate as well.
We end up with a TrainerConfig object where trainer_conf.model is actually an MLPConfig instance, and trainer_conf.learning_rate == 0.01. At this point, we could easily instantiate the actual model by doing trainer_conf.model.instantiate(ModelInterface) (getting an MLPModel), and then use it in a Trainer class.

This nested composition of configs means you can build very complex configurations in a hierarchical yet declarative way. Each piece of the configuration corresponds to a piece of the system. Need to swap the model? Just change the model.class_name in a config file or CLI override and you can go from an MLP to a Transformer – all while maintaining type checking. CompoConf will ensure that the config you provided for the model is valid for the chosen class (and will error out if you, say, put a num_heads field for an MLPModel which doesn’t expect it). This approach is both flexible (thanks to composition) and safe (thanks to type validations).

Comparing Code: With and Without CompoConf

It’s insightful to compare how your code might look managing configurations manually versus using CompoConf:

Without CompoConf (manual approach): You might define various classes and handle their configs through either inheritance or dictionaries. For example, suppose you want to allow two types of models in your trainer. You might end up writing something like:
```
# Manual approach (pseudo-code)
if config["model_type"] == "mlp":
    model = MLPModel(hidden_size=config["hidden_size"], num_layers=config.get("num_layers", 2))
elif config["model_type"] == "cnn":
    model = CNNModel(kernel_size=config["kernel_size"])
else:
    raise ValueError("Unknown model_type")
```
Here, config might be a dictionary (from JSON/YAML), and you manually branch logic to instantiate the right model. There’s no type safety on the config (typos in keys or wrong types of values would only crash at runtime). If the model classes have complex initialization, you must be careful to pass all needed arguments. Adding a new model type means adding another elif branch and ensuring all necessary parameters are handled. If one model has a nested sub-component, you’d need nested if-else logic to configure that too.
With CompoConf: Most of that logic disappears into a declarative schema. The above becomes: define your config dataclasses and register classes once. Then simply do:
```
trainer_conf = parse_config(TrainerConfig, yaml_load("trainer_config.yaml"))
model = trainer_conf.model.instantiate(ModelInterface)
```
CompoConf figures out which class to construct based on trainer_config.yaml (no hardcoded if needed) and ensures the parameters match that class’s config schema. The code is shorter and focused only on high-level steps (parse config, instantiate). All the conditional instantiation logic is handled by the library’s registry. This not only reduces boilerplate, but also makes it easier to extend – to add a new model type, you can create a new config+class and register it, without touching the trainer logic at all.

The difference grows with complexity. Imagine a scenario with multiple configurable components (model, tokenizer, optimizer, etc.). Without a system like CompoConf or Hydra, you would have to write extensive imperative code to read a config and dispatch to the correct classes. With CompoConf, you define each component’s config and class once, and then your experiment configurations can simply declare which implementations to use. This leads to a clean separation of concerns: configuration definition is separate from execution logic. The code that trains a model doesn’t need to know the details of each possible model; it just asks for the interface implementation specified by the config.

How CompoConf Fits into the Ecosystem

It’s worth noting that CompoConf isn’t trying to reinvent the entire configuration management wheel. In fact, it is designed to work in harmony with existing tools like OmegaConf/Hydra. Hydra is a powerful framework for composing configuration files – you can have a directory of YAMLs for different models and easily switch between them. CompoConf can complement this by adding strong typing and direct instantiation to that process. For example, you could use Hydra to load a configuration (yielding an OmegaConf DictConfig), then pass it to CompoConf’s parse_config to get dataclass instances and actual Python objects. This gives you the best of both: Hydra’s flexibility in assembling configs, and CompoConf’s rigor in enforcing types and linking to code.

To illustrate, OmegaConf (which Hydra uses under the hood) already supports the notion of structured configs (dataclasses) and will do runtime type checking. CompoConf builds on similar ideas but provides the additional concept of interfaces and registries. Hydra has a concept of config groups and the _target_ field for instantiation, which is somewhat analogous – you could argue Hydra’s _target_ (pointing to a class path) plus hydra.utils.instantiate is another way to instantiate objects from configs. The difference is that CompoConf ties the class registration to the config classes directly in code, rather than through string class paths in the config. This can make refactoring easier (since class references are resolved via the registry, not hard-coded strings) and encourages a more schema-driven approach (the dataclass defines what’s allowed). In practice, you might choose one approach or combine them. CompoConf is not a one-size-fits-all solution, but rather a focused library that plays nicely with others. If you have a strong existing config setup with Hydra, you can still incorporate CompoConf’s type-safe classes for particular modules. Conversely, if your project doesn’t need the full generality of Hydra, CompoConf might be a lightweight alternative to get structured configs without a complex framework.

When to use CompoConf? If you are building research code or a library where you foresee lots of interchangeable components (different model architectures, various encoder/decoder combinations, etc.), CompoConf provides a neat way to manage that. It allows you to declare the supported options and parameters in a clear manner and lets you instantiate components dynamically. This can make your code more maintainable and self-documenting – the config classes act as a form of documentation for what parameters are available. On the other hand, for very simple scripts or when you don’t need much flexibility, plain old argument parsing or a single YAML might suffice. CompoConf shines as things get more modular.

Finally, because CompoConf is young (as of writing, version 0.1.0), it’s a good idea to try it out on smaller problems or alongside established tools. It’s released under the MIT license and welcomes contributions, so it may evolve with feedback. It also means you won’t find a huge ecosystem of plugins yet, but given it’s built on straightforward Python dataclasses, it should interoperate with many standard practices.

Conclusion

“Composition over inheritance” is more than a slogan – it’s a design philosophy that can lead to clearer and more modular code. Hugging Face’s Transformers showed the drawbacks of an inheritance-heavy config system: it works, but it can be inflexible when models get complex or when you want to mix components. PyTorch demonstrated the power of composition in building models, but left a gap in configuring those models. CompoConf steps in as a pragmatic solution: it doesn’t throw away the benefits of structured configs (we still have classes and dataclasses for clarity and safety), but it steers the design toward composition by making configs nestable and implementations switchable.

In using CompoConf, you define what can be configured and let the library handle how those pieces are plugged together. This leads to code that is both declarative and flexible. Deep learning researchers can appreciate the ability to swap out parts of a model or experiment simply by editing a config file or command-line option, without touching the code that runs the experiment. General Python developers can appreciate the type safety and reduction in boilerplate when dealing with configuration-heavy applications.

In summary, CompoConf offers a fresh take on ML configurations by bringing composition front and center. It aligns with the way we build complex systems (as assemblies of components) and provides a clean, typed layer to manage those pieces. While it’s not a silver bullet for all configuration needs, it fills an important niche. If you’ve felt the pain of contorting your configs or scattering if-else statements to handle various options, CompoConf is definitely worth a look. It embodies a balanced, modern approach: use composition for flexibility, use types for safety, and keep the whole process enjoyable for the developer. Happy configuring!

Sources:

Hugging Face Transformers documentation (configuration system)
PyTorch documentation and community notes on composing nn.Module layers
CompoConf library documentation (Korbinian Pöppel, 2025)
OmegaConf/Hydra documentation on structured configs and composition
Wikimedia Commons – UML diagram of composition over inheritance (CC BY-SA 3.0)