Hyperbench & Hyperboost: Rethinking Hyperparameter Optimization

Finding the optimal configuration for a machine learning model is frequently treated as an afterthought, yet it can be the difference between a mediocre model and a state-of-the-art one. The challenge isn't just finding the best parameters, but doing so in a way that is reproducible, rigorous, and efficient.

My recent work focuses on two core components of this ecosystem: Hyperbench, a platform for standardized HPO evaluation, and Hyperboost, a novel Empirical Performance Model (EPM) designed to enhance Sequential Model-based Algorithm Configuration (SMAC).

The Hyperbench Framework

Hyperbench was born out of the need for a reproducible benchmarking environment. While many HPO algorithms exist (Random Search, TPE, BOHB, SMAC), comparing them fairly across different datasets and models is notoriously difficult.

Hyperboost: Boosting the EPM with CatBoost

At the heart of Bayesian Optimization (like SMAC) lies the Empirical Performance Model (EPM). The EPM predicts how a configuration will perform and, crucially, how uncertain that prediction is. Traditionally, Random Forests or Gaussian Processes are used.

Hyperboost replaces these traditional models with CatBoost. By utilizing CatBoost's RMSEWithUncertainty loss function, Hyperboost treats regression as a probabilistic task, estimating both the mean and the variance of a normal distribution.

class HyperboostEPM(AbstractEPM):
    def _train(self, X: np.ndarray, y: np.ndarray):
        # Using CatBoost with built-in uncertainty estimation
        self.model = CatBoostRegressor(
            iterations=100,
            learning_rate=1.0,
            loss_function="RMSEWithUncertainty",
            posterior_sampling=True
        )
        self.model.fit(X, y)

    def _predict(self, X: np.ndarray):
        # Leverage virtual ensembles for Total Uncertainty
        mean, total_unc = self.model.virtual_ensembles_predict(
            X, prediction_type="TotalUncertainty"
        )
        # Extract knowledge uncertainty for the predictive variance
        knowledge_unc = total_unc[:, 1]
        return mean, knowledge_unc ** 0.3

Quantifying the Unknown: Data vs. Knowledge Uncertainty

One of the key innovations in Hyperboost is how it handles different sources of uncertainty. By leveraging CatBoost's ability to differentiate between these types, we can make more informed decisions during the optimization process.

By leveraging TotalUncertainty, SMAC can distinguish between a configuration that is likely bad due to noise (Data Uncertainty) and a configuration that we simply haven't explored yet (Knowledge Uncertainty). This allows for much more efficient exploration of high-dimensional hyperparameter spaces.

Why CatBoost for HPO?

While Gaussian Processes are the "gold standard" for Bayesian Optimization, they scale poorly ($O(n^3)$) with the number of observations. CatBoost offers a middle ground:

1. Superior Scaling: Handles thousands of configurations easily, making it suitable for long-running HPO benchmarks.
2. Structured Data Mastery: HPO logs are tabular data, where gradient boosting typically outperforms other methods.
3. Efficient Uncertainty: Virtual Ensembles provide ensemble-like benefits without the computational cost of training multiple models.

"Uncertainty is not an error; it's a signal. By quantifying what our model doesn't know, we can navigate the high-dimensional search space of hyperparameters with surgical precision."

Benchmarks & Observations

Testing Hyperboost across different target algorithms revealed that the choice of configuration space significantly impacts performance. Here’s what we observed:

• Random Forest & XGBoost (High-Dim Success): Hyperboost consistently outperformed standard SMAC. These models have 4+ parameters with complex non-linearities (like RF's power-scaled max_features). CatBoost's tree-based structure is naturally suited for capturing these high-dimensional, tabular interactions.
• SGD (The Low-Dim Baseline): Performance was strong, but struggled to beat ROAR. In simple 2-parameter spaces (alpha, eta0), the overhead of a complex surrogate model often yields diminishing returns compared to the raw efficiency of random search.
• SVC (The Log-Scale Challenge): Performance was less than ideal. The SVC search space uses extremely wide log-ranges ($2^{-10}$ to $2^{10}$). Tree-based surrogates approximate space using axis-aligned "boxes," which makes them struggle with the smooth, continuous response surfaces of an SVM compared to the inherent smoothness of Gaussian Processes.

Next Steps: Beyond the "Magical" 0.3

In the implementation above, the knowledge_unc ** 0.3 transformation serves as a form of variance scaling. Bayesian Optimization relies on the scale of uncertainty to balance exploration and exploitation. The 0.3 exponent (which is more aggressive than a standard square root) works well because it "squashes" extreme uncertainty values, preventing the optimizer from being distracted by model artifacts in distant regions of the search space.

However, a hardcoded constant is a heuristic. More robust approaches include:

• NLL Calibration: Finding a scaling factor that minimizes the Negative Log-Likelihood on a validation set.
• Isotonic Regression: Using a secondary model to map raw uncertainty to actual observed Mean Squared Error.
• Exploration Weight Tuning: Instead of warping the uncertainty, we can return the standard deviation and tune the acquisition function's $\beta$ parameter (e.g., in UCB).

Summary & Takeaways

The combination of Hyperbench and Hyperboost provides a powerful toolkit for both researchers and practitioners. Whether you are trying to benchmark a new HPO algorithm or optimize a production model with SMAC, these tools offer the rigor and performance needed for modern ML workflows.