Bayesian Optimization¶

The fr5.bayes_opt module provides a comprehensive framework for Bayesian optimization (BO) tailored to chemical experimentation. It enables efficient exploration of chemical parameter spaces to find optimal experimental conditions with minimal experimental effort.

What is Bayesian Optimization?¶

Bayesian optimization is a sequential design strategy for optimizing expensive-to-evaluate functions. In chemistry, this means finding the best experimental conditions (temperature, pressure, catalyst loading, etc.) with the fewest possible experiments.

Key Advantages for Chemistry¶

Efficiency: Find optimal conditions with 5-10x fewer experiments
Uncertainty awareness: Models provide confidence estimates
Multi-objective: Optimize multiple properties simultaneously
Constraint handling: Respect experimental limitations
Batch optimization: Design multiple experiments at once
High-dimensional optimization: Specialized kernels for complex chemical spaces
Adaptive strategies: Automatically balance exploration and exploitation

Core Concepts¶

1. Surrogate Models¶

The BO framework builds statistical models (Gaussian Processes) to approximate the relationship between experimental variables and outcomes. The framework uses high-dimensional kernels optimized for chemical optimization problems.

2. Acquisition Functions¶

These functions determine which experiments to perform next by balancing: - Exploitation: Areas likely to have good performance - Exploration: Areas with high uncertainty - Multi-objective trade-offs: Pareto front advancement for competing objectives

3. Optimization Loop¶

Design initial experiments using space-filling methods
Train surrogate model on existing data with specialized chemical kernels
Optimize acquisition function to suggest next experiments
Perform experiments and update data
Repeat until converged or budget exhausted

Quick Start Example¶

import pandas as pd
from fr5.bayes_opt import BOExperimentManager

# Define your experimental variables
input_details = [
    {
        "name": "temperature",
        "type": "continuous",
        "bounds": (20, 100),  # °C
        "unit": "°C"
    },
    {
        "name": "pressure", 
        "type": "continuous",
        "bounds": (1, 10),    # bar
        "unit": "bar"
    },
    {
        "name": "catalyst",
        "type": "categorical", 
        "categories": ["Pd", "Pt", "Ni"]
    }
]

# Your initial experimental data
initial_data = pd.DataFrame({
    'temperature': [25, 50, 75],
    'pressure': [1, 5, 10], 
    'catalyst': ['Pd', 'Pt', 'Ni'],
    'yield': [0.45, 0.72, 0.68],      # Target to maximize
    'selectivity': [0.85, 0.91, 0.78] # Target to maximize
})

# Create BO manager for multi-objective optimization  
manager = BOExperimentManager(
    input_details=input_details,
    initial_experiments=initial_data,
    db_path="optimization.db",
    objective_names=["yield", "selectivity"],
    strategy_type="mobo_pareto",      # Multi-objective Pareto
    acquisition_function="qNEHVI",    # Noisy Expected Hypervolume Improvement
    minimize=[False, False],          # Maximize both objectives
    kernel="rbf",                     # High-dimensional RBF kernel
    adaptive_acquisition=False        # Can enable for single-objective
)

# Suggest next 3 experiments
suggestions = manager.suggest_experiments(batch_size=3)
print(suggestions)

Variable Types¶

Continuous Variables¶

For numerical parameters with bounds:

{
    "name": "temperature",
    "type": "continuous", 
    "bounds": (273.15, 373.15),  # Min, max values
    "step": 0.1,                 # Optional: discretization step
    "unit": "K"                  # Optional: for documentation
}

Categorical Variables¶

For discrete choices:

{
    "name": "solvent",
    "type": "categorical",
    "categories": ["water", "ethanol", "acetone", "dmso"]
}

Categorical with Descriptors¶

For categorical variables with numerical descriptors (e.g., molecular descriptors):

{
    "name": "solvent",
    "type": "categorical_descr",
    "data": (
        ["water", "ethanol", "acetone"],           # Category names
        ["polarity", "boiling_point", "viscosity"], # Descriptor names
        [[1.0, 100.0, 0.89],                      # water descriptors
         [0.7, 78.3, 1.07],                       # ethanol descriptors  
         [0.4, 56.1, 0.31]]                       # acetone descriptors
    )
}

VAE Latent Variables¶

For continuous optimization in latent spaces (e.g., molecular VAE):

{
    "name": "molecule_latent",
    "type": "categorical_descr_continuous",
    "bounds": (-3.0, 3.0),                    # Latent space bounds
    "vae_decoder": decode_molecule_function   # Optional decoder
}

Optimization Strategies¶

Single-Objective Optimization¶

For optimizing a single target property:

manager = BOExperimentManager(
    input_details=input_details,
    initial_experiments=data,
    db_path="single_obj.db",
    objective_names=["yield"],
    strategy_type="sobo",
    acquisition_function="qUCB",    # Upper confidence bound
    minimize=False,                 # Maximize yield
    adaptive_acquisition=True       # Enable adaptive strategy
)

Available Single-Objective Acquisition Functions:

qUCB: Upper Confidence Bound (best for exploration-exploitation balance)
qEI: Expected Improvement
qLogEI: Log Expected Improvement (numerically stable)

Multi-Objective Optimization¶

Pareto-Based Optimization¶

For true multi-objective optimization finding the Pareto front:

manager = BOExperimentManager(
    input_details=input_details,
    initial_experiments=data, 
    db_path="multi_obj.db",
    objective_names=["yield", "selectivity"],
    strategy_type="mobo_pareto",
    acquisition_function="qNEHVI",  # Noisy Expected Hypervolume Improvement
    minimize=[False, False],        # Maximize both objectives
    ref_point=[0.0, 0.0]           # Reference point for hypervolume
)

Available Multi-Objective Acquisition Functions:

qNEHVI: Noisy Expected Hypervolume Improvement (recommended default)
qLogNEHVI: Log Noisy Expected Hypervolume Improvement (numerically stable)
qEHVI: Expected Hypervolume Improvement (for noise-free objectives)
qLogEHVI: Log Expected Hypervolume Improvement (numerically stable)

Scalarization Strategies¶

For single-objective treatment of multi-objective problems:

# Additive scalarization (weighted sum)
manager = BOExperimentManager(
    input_details=input_details,
    initial_experiments=data,
    db_path="additive.db",
    objective_names=["yield", "selectivity"],
    strategy_type="mobo_additive",
    acquisition_function="qLogEI"
)

# Multiplicative scalarization (weighted product)
manager = BOExperimentManager(
    input_details=input_details,
    initial_experiments=data,
    db_path="multiplicative.db",
    objective_names=["yield", "selectivity"],
    strategy_type="mobo_multiplicative",
    acquisition_function="qUCB",
    adaptive_acquisition=True  # Can use adaptive for scalarized objectives
)

Advanced Features¶

Adaptive Acquisition Functions¶

Automatically adjust exploration-exploitation balance over time:

manager = BOExperimentManager(
    input_details=input_details,
    initial_experiments=data,
    db_path="adaptive.db", 
    objective_names=["yield"],
    strategy_type="sobo",
    adaptive_acquisition=True,     # Enable adaptive behavior
    acquisition_function="qUCB"    # Starting function (will adapt)
)

Adaptive Strategy: - Early iterations: High exploration with qUCB (β=5.0 → 0.1) - Later iterations: Focused exploitation with qLogEI - Transition: Gradual over specified number of iterations

High-Dimensional Optimization¶

Specialized kernels for high-dimensional problems:

manager = BOExperimentManager(
    input_details=input_details,
    initial_experiments=data,
    db_path="high_dim.db",
    objective_names=["yield"],
    kernel="rbf",                 # RBF kernel (recommended)
    strategy_type="sobo",
    n_raw_samples=128,           # Increased for high-dim
    n_restarts=15                # More restarts for better optimization
)

Available Kernels:

rbf: RBF kernel with high-dimensional initialization (default, recommended) based on Hvarfner et al. 2024
rbf_standard: Standard RBF kernel with ARD
matern: High-dimensional Matérn 5/2 kernel based on Xu et al. 2025
matern_standard: Standard Matérn 5/2 kernel with ARD

Batch Optimization with Constant Variables¶

Optimize experiments where some variables remain constant within a batch:

# Suggest 5 experiments where temperature is constant per batch
suggestions = manager.suggest_experiments(
    batch_size=5,
    const_vars=["temperature"]
)

Constant Variable Features: - Two-step optimization: First select optimal constant values, then optimize other variables - Aggregation strategies: "mean", "max", or "sum" for acquisition value aggregation - Configurable evaluation: Control number of evaluations per constant variable

This is useful when:

Changing certain variables is expensive (e.g., reactor setup)
You want to explore multiple compositions at fixed conditions
Experimental constraints require constant parameters

Custom Bounds and Data Handling¶

Control optimization bounds and data validation:

manager = BOExperimentManager(
    input_details=input_details,
    initial_experiments=data,
    db_path="constrained.db",
    objective_names=["yield"],
    use_data_bounds=True,         # Expand bounds to fit all training data
    validate_training_bounds=False, # Allow extrapolation
    n_constant_evaluations=1000,  # Evaluations per constant variable
    constant_aggregation_strategy="mean"  # How to aggregate acquisition values
)

Variable Types and Descriptors¶

Categorical Variables with Descriptors¶

For categorical variables with numerical descriptors:

{
    "name": "solvent",
    "type": "categorical_descr",
    "data": (
        ["water", "ethanol", "acetone"],           # Category names
        ["polarity", "boiling_point", "viscosity"], # Descriptor names
        [[1.0, 100.0, 0.89],                      # water descriptors
         [0.7, 78.3, 1.07],                       # ethanol descriptors  
         [0.4, 56.1, 0.31]]                       # acetone descriptors
    ),
    "data_kw_include": ["water", "ethanol"]      # Optional: filter categories
}

Descriptor Handling Methods¶

manager = BOExperimentManager(
    # ... other parameters ...
    descriptor_method="exhaustive",  # or "continuous"
)

"exhaustive": True mixed variable optimization with all categorical combinations
"continuous": Fast continuous optimization in descriptor space with post-processing

Experiment Management & Logging¶

Automatic Database Integration¶

All experiments are automatically tracked in SQLite database:

# Load experiment history
from fr5.bayes_opt.data_loader import load_dataframe_from_db

experiments = load_dataframe_from_db(
    db_path="optimization.db",
    table_name="experiments"
)

# View optimization progress
import matplotlib.pyplot as plt
plt.plot(experiments.index, experiments['yield'])
plt.xlabel('Experiment Number')
plt.ylabel('Yield')
plt.title('Optimization Progress')
plt.show()

Custom Logging Configuration¶

The framework provides comprehensive logging that can be customized:

from fr5.bayes_opt import setup_logging

# Default: writes to 'bo_experiment.log'
manager = BOExperimentManager(
    # ... parameters ...
    log_file="my_experiment.log"  # Custom log file
)

# Advanced logging setup
setup_logging(
    log_file="detailed_experiment.log",
    console_output=True,    # Also print to console
    log_level=logging.DEBUG # Detailed debug information
)

Excel Integration¶

Import/export experiments from Excel:

# Load new experiments from Excel file
manager = BOExperimentManager(
    input_details=input_details,
    initial_experiments=initial_data,
    db_path="optimization.db",
    excel_path="new_results.xlsx",  # Automatically import on initialization
    sheet_name="Results"
)

# Or update database from Excel later
from fr5.bayes_opt.data_loader import update_db_from_excel

update_db_from_excel(
    excel_path="additional_results.xlsx",
    db_path="optimization.db",
    table_name="experiments"
)

Data Validation¶

Ensure data quality before optimization:

from fr5.bayes_opt.data_loader import validate_model_data

is_valid, issues = validate_model_data(
    data=experiments,
    input_details=input_details,
    objective_names=["yield", "selectivity"]
)

if not is_valid:
    print("Data issues found:")
    for issue in issues:
        print(f"- {issue}")

Performance Optimization¶

Parameter Tuning¶

Optimize acquisition function performance:

manager = BOExperimentManager(
    # ... other parameters ...
    n_raw_samples=128,    # More initial samples for better coverage
    n_restarts=15,        # More optimization restarts for global optimum
    kernel="rbf",         # High-dimensional kernel for complex spaces
    descriptor_method="continuous"  # Faster optimization for categorical descriptors
)

Memory and Computational Efficiency¶

For large-scale problems:

manager = BOExperimentManager(
    # ... other parameters ...
    n_constant_evaluations=500,  # Reduce for faster constant variable selection
    n_eval_const_samples=5000,   # Control evaluation samples
    validate_training_bounds=False  # Skip validation for speed
)

Troubleshooting¶

Common Issues and Solutions¶

Poor exploration/exploitation balance: - Try adaptive_acquisition=True for single-objective problems - Adjust beta parameter for qUCB acquisition function - Use different acquisition functions (qLogEI vs qUCB)

Slow optimization: - Reduce n_restarts and n_raw_samples - Use descriptor_method="continuous" for categorical variables - Consider simpler kernels (rbf_standard vs rbf)

Numerical instability: - Use log-based acquisition functions (qLogEI, qLogNEHVI) - Check for duplicate experiments in training data - Validate data ranges and normalize objectives

Memory issues: - Reduce n_constant_evaluations for constant variable optimization - Use smaller batch_size for suggestion generation - Consider data filtering with data_kw_include for large categorical spaces

Debugging Tools¶

# Get detailed model information
print("Optimization summary:")
print(f"Strategy: {manager.strategy_type.value}")
print(f"Kernel: {manager.kernel}")
print(f"Input dimension: {manager.input_dim}")
print(f"Training samples: {manager.X_train.shape[0]}")

# Check bounds and normalization
print(f"Original bounds: {manager.original_bounds}")
print(f"Training bounds: {manager.training_bounds}")

# Get Pareto front for multi-objective
if manager.strategy_type.value == "mobo_pareto":
    pareto_front = manager.get_pareto_front()
    print(f"Pareto front size: {len(pareto_front)}")

Best Practices¶

1. Initial Experimental Design¶

Start with 5-10 initial experiments using space-filling designs
Include corner cases and known good conditions
Ensure coverage of the full variable space

2. Objective Function Design¶

Normalize objectives to similar scales (0-1 or standardized)
Consider log-transformation for highly skewed properties
Use minimize parameter to specify optimization direction correctly

3. Convergence Criteria¶

Monitor acquisition function values for plateau detection
Stop when improvement plateaus or budget is exhausted
Consider multi-criteria stopping (time, budget, performance)

4. Model Validation¶

Regularly check model predictions vs. experiments
Use built-in logging to track optimization progress
Validate data quality before starting optimization

Example Applications¶

Catalyst Screening¶

# Optimize reaction conditions for maximum yield and selectivity
input_details = [
    {"name": "temperature", "type": "continuous", "bounds": (50, 150)},
    {"name": "catalyst_loading", "type": "continuous", "bounds": (0.1, 5.0)},
    {"name": "catalyst", "type": "categorical", "categories": ["Pd", "Pt", "Ni"]},
    {"name": "solvent", "type": "categorical", "categories": ["toluene", "dmf", "thf"]}
]

manager = BOExperimentManager(
    input_details=input_details,
    initial_experiments=initial_data,
    db_path="catalyst_optimization.db",
    objective_names=["yield", "selectivity"],
    strategy_type="mobo_pareto",
    acquisition_function="qNEHVI"
)

High-Dimensional Materials Discovery¶

# Optimize complex formulation with many components
input_details = [
    {"name": "component_A", "type": "continuous", "bounds": (0, 100)},
    {"name": "component_B", "type": "continuous", "bounds": (0, 100)}, 
    {"name": "component_C", "type": "continuous", "bounds": (0, 100)},
    {"name": "additive", "type": "categorical_descr", "data": additive_data},
    {"name": "process_temp", "type": "continuous", "bounds": (25, 200)},
    {"name": "process_time", "type": "continuous", "bounds": (1, 24)}
]

manager = BOExperimentManager(
    input_details=input_details,
    initial_experiments=initial_data,
    db_path="materials_discovery.db",
    objective_names=["performance"],
    strategy_type="sobo",
    kernel="rbf",  # High-dimensional kernel
    adaptive_acquisition=True,
    n_raw_samples=128,
    n_restarts=15
)

This documentation covers the comprehensive Bayesian optimization capabilities in fr5. For more examples and advanced usage, see the examples directory in the repository.