Automated Algorithm Configuration of $\alpha\beta$-CROWN

Code appendix for:

Automated Algorithm Configuration of $\alpha\beta$-CROWN
Konstantin Kaulen and Holger H. Hoos

This appendix contains the experiment code, VNN-COMP configuration files, selected result artifacts, and plotting scripts used to configure alpha-beta-CROWN automatically with SMAC3.

In one sentence: we let SMAC search over a carefully designed alpha-beta-CROWN parameter space, evaluate the resulting configurations on VNN-COMP benchmarks, and regenerate the plots and score tables from the paper.

Paper Summary

Neural network verifiers such as alpha-beta-CROWN expose many parameters, and their performance can depend strongly on how those parameters are set. This work applies automated algorithm configuration to alpha-beta-CROWN using SMAC3, with the goal of reducing manual tuning effort while retaining competitive verification performance.

The experiments use the regular-track VNN-COMP 2025 benchmarks and compare:

• Default alpha-beta-CROWN settings
• Author-provided benchmark configurations
• SMAC-discovered configurations

The included artifacts let you inspect the tuned configurations, rerun HPO and evaluation jobs, and regenerate the summary plots.

---

⚙️ Setup

Python Environment

Create or activate a Python 3.11 environment, then install the dependencies:

pip install torch==2.8.0 torchvision==0.23.0 torchaudio==2.8.0 \
  --index-url https://download.pytorch.org/whl/cu128
pip install -r requirements.txt
pip install git+https://github.com/Verified-Intelligence/auto_LiRPA.git@9d100ec070868440b48d34e2f1dd21b97aab9172

For meaningful HPO/evaluation runs, make sure that:

• a CUDA-capable GPU is available
• a valid Gurobi license is active when MIP-based components are enabled
• PYTHONPATH includes this directory and the bundled verifier code

export PYTHONPATH="$PWD:$PWD/abCROWN/complete_verifier"

Benchmarks

The experiments use the VNN-COMP 2025 benchmark repository.

For test instances:

git clone https://github.com/VNN-COMP/vnncomp2025_benchmarks.git

For the training instances used for tool-author tuning:

git clone https://github.com/VNN-COMP/vnncomp2025_benchmarks.git ./vnncomp2025_benchmarks_train
cd ./vnncomp2025_benchmarks_train
git checkout 6d481f7324403c4eb3c5490e5c4cb35327bcba22
cd ..

Then adjust the benchmark locations in:

experiments/vnncomp_25/configs/*.json
experiments/vnncomp_25/eval/*.json

---

🚀 Quick Start

Run all commands from this directory:

export PYTHONPATH="$PWD:$PWD/abCROWN/complete_verifier"

1. Tune a Benchmark

python hpo_vnncomp.py \
  --config experiments/vnncomp_25/configs/hpo_vnncomp_acas_test.json

New HPO outputs are written below:

experiments/vnncomp_25/results/

2. Evaluate Configurations

Make sure that all paths in the evaluation config are valid on your machine, then run:

python eval_abcrown_vnncomp.py \
  --config experiments/vnncomp_25/eval/eval_vnncomp_acasxu.json \
  --config-types std,author,hpo

Evaluation outputs are written below:

experiments/vnncomp_25/eval_results/

3. Submit Jobs with SLURM

The VNN-COMP experiment launcher uses submitit:

python experiments/vnncomp_25/submit_experiments.py --type hpo --dry-run
python experiments/vnncomp_25/submit_experiments.py --type hpo

Use the launcher flags to adjust account, partition, memory, wall-clock time, and GPU count for your cluster.

4. Recreate Plots and Tables

python analyze_eval_results.py \
  --results-dir experiments/vnncomp_25/eval_results \
  --output-dir eval_plots

This regenerates:

• eval_plots/benchmark_stats.csv
• eval_plots/vnncomp_scores.csv
• eval_plots/vnncomp_scores_aggregated.csv
• all PDF/PNG plots in eval_plots/

📚 Citation

If you use this code appendix or build on the paper, please cite:

@inproceedings{KauHoo26,
  author    = {Konstantin Kaulen and Holger H. Hoos},
  title     = {Automated Algorithm Configuration of alpha-beta-CROWN},
  booktitle = {Proceedings of the 9th International Symposium on AI Verification (SAIV 2026)},
  year      = {2026},
  note      = {To appear}
}