Publications

2021

Benjamins, Carolin; Eimer, Theresa; Schubert, Frederik; Biedenkapp, André; Rosenhan, Bodo; Hutter, Frank; Lindauer, Marius

CARL: A Benchmark for Contextual and Adaptive Reinforcement Learning Inproceedings

In: Workshop on Ecological Theory of Reinforcement Learning (EcoRL@NeurIPS'21), 2021.

Lindauer, Marius; Eggensperger, Katharina; Feurer, Matthias; Biedenkapp, André; Deng, Difan; Benjamins, Carolin; Sass, René; Hutter, Frank

SMAC3: A Versatile Bayesian Optimization Package for Hyperparameter Optimization Journal Article

In: arXiv:2109.09831, 2021.

Eggensperger, Katharina; Müller, Philipp; Mallik, Neeratyoy; Feurer, Matthias; Sass, René; Klein, Aaron; Awad, Noor; Lindauer, Marius; Hutter, Frank

HPOBench: A Collection of Reproducible Multi-Fidelity Benchmark Problems for HPO Journal Article

In: arXiv:2109.06716, 2021.

Feurer, Matthias; Eggensperger, Katharina; Falkner, Stefan; Lindauer, Marius; Hutter, Frank

Auto-Sklearn 2.0: Hands-free AutoML via Meta-Learning Journal Article

In: arXiv:2007.04074, 2021.

Awad, Noor; Mallik, Neeratyoy; Hutter, Frank

DEHB: Evolutionary Hyberband for Scalable, Robust and Efficient Hyperparameter Optimization Inproceedings

In: Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence (IJCAI'21), ijcai.org, 2021.

Eimer, Theresa; Biedenkapp, André; Reimer, Maximilian; Adriaensen, Steven; Hutter, Frank; Lindauer, Marius

DACBench: A Benchmark Library for Dynamic Algorithm Configuration Inproceedings

In: Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence (IJCAI'21), ijcai.org, 2021.

Speck, David; Biedenkapp, André; Hutter, Frank; Mattmüller, Robert; Lindauer, Marius

Learning Heuristic Selection with Dynamic Algorithm Configuration Inproceedings

In: Proceedings of the 31st International Conference on Automated Planning and Scheduling (ICAPS'21), 2021.

Narayanan, Ashwin Raaghav; Zela, Arber; Saikia, Tonmoy; Brox, Thomas; Hutter, Frank

Multi-headed Neural Ensemble Search Inproceedings

In: Workshop on Uncertainty and Robustness in Deep Learning (UDL@ICML`21), 2021.

Biedenkapp, André; Rajan, Raghu; Hutter, Frank; Lindauer, Marius

TempoRL: Learning When to Act Inproceedings

In: Proceedings of the 38th International Conference on Machine Learning (ICML 2021), 2021.

Eimer, Theresa; Biedenkapp, André; Hutter, Frank; Lindauer, Marius

Self-Paced Context Evaluations for Contextual Reinforcement Learning Inproceedings

In: Proceedings of the 38th International Conference on Machine Learning (ICML 2021), 2021.

Izquierdo, Sergio; Guerrero-Viu, Julia; Hauns, Sven; Miotto, Guilherme; Schrodi, Simon; Biedenkapp, André; Elsken, Thomas; Deng, Difan; Lindauer, Marius; Hutter, Frank

Bag of Baselines for Multi-objective Joint Neural Architecture Search and Hyperparameter Optimization Inproceedings

In: Workshop on Automated Machine Learning (AutoML@ICML'21), 2021.

Colin White Shen Yan, Yash Savani; Hutter, Frank

NAS-Bench-x11 and the Power of Learning Curves Inproceedings

In: Proceedings of the CVPR 2021 Workshop on Neural Architecture Search (CVPR-NAS '21), 2021.

Elsken, Thomas; Staffler, Benedikt; Zela, Arber; Metzen, Jan Hendrik; Hutter, Frank

Bag of Tricks for Neural Architecture Search Journal Article

In: Proceedings of the CVPR 2021 Workshop on Neural Architecture Search (CVPR-NAS '21), 2021.

Chatzimichailidis, Avraam; Zela, Arber; Shalini, Shalini; Labus, Peter; Keuper, Janis; Hutter, Frank; Yang, Yang

Group Sparsity: A Unified Framework for Network Pruning and Neural Architecture Search Journal Article

In: Proceedings of the CVPR 2021 Workshop on Neural Architecture Search (CVPR-NAS '21), 2021.

Franke, Jörg K H; Köhler, Gregor; Biedenkapp, André; Hutter, Frank

Sample-Efficient Automated Deep Reinforcement Learning Journal Article

In: International Conference on Learning Representations (ICLR) 2021, 2021.

Zhang, Baohe; Rajan, Raghu; Pineda, Luis; Lambert, Nathan; Biedenkapp, André; Chua, Kurtland; Hutter, Frank; Calandra, Roberto

On the Importance of Hyperparameter Optimization for Model-based Reinforcement Learning Inproceedings

In: Proceedings of the 24th International Conference on Artificial Intelligence and Statistics (AISTATS)'21, 2021.

Müller, Samuel; Hutter, Frank

TrivialAugment: Tuning-free Yet State-of-the-Art Data Augmentation Journal Article

In: arXiv:2103.10158 [cs.CV], 2021, (ICCV 2021 (Oral Presentation (Top 3%))).

Ferreira, Fabio; Nierhoff, Thomas; Hutter, Frank

Learning Synthetic Environments for Reinforcement Learning with Evolution Strategies Journal Article

In: AAAI workshop on Meta-Learning Challenges, 2021.

Müller, Samuel; Biedenkapp, André; Hutter, Frank

In-Loop Meta-Learning with Gradient-Alignment Reward Journal Article

In: AAAI workshop on Meta-Learning Challenges, 2021.

Feurer, Matthias; van Rijn, Jan N; Kadra, Arlind; Gijsbers, Pieter; Mallik, Neeratyoy; Ravi, Sahithya; Müller, Andreas; Vanschoren, Joaquin; Hutter, Frank

OpenML-Python: an extensible Python API for OpenML Journal Article

In: Journal of Machine Learning Research, 22 (100), pp. 1-5, 2021.

Zimmer, Lucas; Lindauer, Marius; Hutter, Frank

Auto-Pytorch: Multi-Fidelity MetaLearning for Efficient and Robust AutoDL Journal Article

In: IEEE Transactions on Pattern Analysis and Machine Intelligence, pp. 1-1, 2021.

Souza, Artur; Nardi, Luigi; Oliveira, Leonardo; Olukotun, Kunle; Lindauer, Marius; Hutter, Frank

Bayesian Optimization with a Prior for the Optimum Inproceedings

In: Proceedings of the European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML-PKDD), 2021.

2020

Awad, Noor; Shala, Gresa; Deng, Difan; Mallik, Neeratyoy; Feurer, Matthias; Eggensperger, Katharina; Biedenkapp, André; Vermetten, Diederick; Wang, Hao; Doerr, Carola; Lindauer, Marius; Hutter, Frank

Squirrel: A Switching Hyperparameter Optimizer Description of the entry by AutoML.org & IOHprofiler to the NeurIPS 2020 BBO challenge Journal Article

In: arXiv:2012.08180 [cs.LG], 2020, (Optimizer description for the NeurIPS 2020 BBO competition. Squirrel won the competition´s warm-starting friendly leaderboard.).

Lindauer, Marius; Hutter, Frank

Best Practices for Scientific Research on Neural Architecture Search Journal Article

In: Journal of Machine Learning Research, 21 (243), pp. 1-18, 2020.

Liu, Zhengying; Pavao, Adrien; Xu, Zhen; Escalera, Sergio; Ferreira, Fabio; Guyon, Isabelle; Hong, Sirui; Hutter, Frank; Ji, Rongrong; Junior, Julio C S Jacques; Li, Ge; Lindauer, Marius; Luo, Zhipeng; Madadi, Meysam; Nierhoff, Thomas; Niu, Kangning; Pan, Chunguang; Stoll, Danny; Treguer, Sebastien; Wang, Jin; Wang, Peng; Wu, Chenglin; Xiong, Youcheng; Zela, Arber; Zhang, Yang

Winning Solutions and Post-Challenge Analyses of the ChaLearn AutoDL Challenge 2019 Journal Article

In: IEEE Transactions on Pattern Analysis and Machine Intelligence, 43 (9), pp. 3108-3125, 2020.

Lukasik, Jovita; Friede, David; Zela, Arber; Stuckenschmidt, Heiner; Hutter, Frank; Keuper, Margret

Smooth Variational Graph Embeddings for Efficient Neural Architecture Search Journal Article

In: arXiv:2010.04683 [cs.LG], 2020.

Rajan, Raghu; Diaz, Jessica Lizeth Borja; Guttikonda, Suresh; Ferreira, Fabio; Biedenkapp, André; Hutter, Frank

MDP Playground: Controlling Dimensions of Hardness in Reinforcement Learning Inproceedings

In: arXiv:1909.07750, 2020.

Siems, Julien; Zimmer, Lucas; Zela, Arber; Lukasik, Jovita; Keuper, Margret; Hutter, Frank

NAS-Bench-301 and the Case for Surrogate Benchmarks for Neural Architecture Search Journal Article

In: NeurIPS 4th Workshop on Meta-Learning, 2020.

Souza, Artur; Nardi, Luigi; Oliveira, Leonardo B; Olukotun, Kunle; Lindauer, Marius; Hutter, Frank

Prior-guided Bayesian Optimization Journal Article

In: NeurIPS 4th Workshop on Meta-Learning, 2020.

Speck, David; Biedenkapp, André; Hutter, Frank; Mattmüller, Robert; Lindauer, Marius

Learning Heuristic Selection with Dynamic Algorithm Configuration Inproceedings

In: Workshop on Bridging the Gap Between AI Planning and Reinforcement Learning (PRL@ICAPS'20), 2020.

Stoll, Danny; Franke, Jörg K H; Wagner, Diane; Selg, Simon; Hutter, Frank

Hyperparameter Transfer Across Developer Adjustments Journal Article

In: NeurIPS 4th Workshop on Meta-Learning, 2020.

Eggensperger, Katharina; Haase, Kai; Müller, Philipp; Lindauer, Marius; Hutter, Frank

Neural Model-based Optimization with Right-Censored Observations Journal Article

In: arXiv:2009:13828 [cs.AI], 2020.

Shala, Gresa; Biedenkapp, André; Awad, Noor; Adriaensen, Steven; Lindauer, Marius; Hutter, Frank

Learning Step-Size Adaptation in CMA-ES Inproceedings

In: Proceedings of the Sixteenth International Conference on Parallel Problem Solving from Nature (PPSN'20), 2020.

Biedenkapp, André; Rajan, Raghu; Hutter, Frank; Lindauer, Marius

Towards TempoRL: Learning When to Act Inproceedings

In: Workshop on Inductive Biases, Invariances and Generalization in RL (BIG@ICML'20), 2020.

Eimer, Theresa; Biedenkapp, André; Hutter, Frank; Lindauer, Marius

Towards Self-Paced Context Evaluations for Contextual Reinforcement Learning Inproceedings

In: Workshop on Inductive Biases, Invariances and Generalization in RL (BIG@ICML'20), 2020.

Biedenkapp, André; Bozkurt, Furkan H; Eimer, Theresa; Hutter, Frank; Lindauer, Marius

Dynamic Algorithm Configuration: Foundation of a New Meta-Algorithmic Framework Inproceedings

In: Proceedings of the Twenty-fourth European Conference on Artificial Intelligence (ECAI'20), 2020.

Elsken, Thomas; Staffler, Benedikt; Metzen, Jan Hendrik; Hutter, Frank

Meta-Learning of Neural Architectures for Few-Shot Learning Inproceedings

In: The IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020, (Oral Presentation (Top 6%)).

Gargiani, Matilde; Zanelli, Andrea; Diehl, Moritz; Hutter, Frank

On the Promise of the Stochastic Generalized Gauss-Newton Method for Training DNNs Journal Article

In: arXiv:2006.02409 [cs.LG], 2020.

Zaidi, Sheheryar; Zela, Arber; Elsken, Thomas; Holmes, Chris; Hutter, Frank; Teh, Yee Whye

Neural Ensemble Search for Performant and Calibrated Predictions Journal Article

In: Workshop on Uncertainty and Robustness in Deep Learning (UDL@ICML`20), 2020, (Oral Presentation).

Zimmer, Lucas; Lindauer, Marius; Hutter, Frank

Auto-PyTorch Tabular: Multi-Fidelity MetaLearning for Efficient and Robust AutoDL Journal Article

In: arXiv:2006.13799 [cs.LG], 2020.

Lehman, J; Clune, J; Misevic, D; Adami, C; Beaulieu, J; Bentley, P J; Bernard, S; Beslon, G; Bryson, D M; Chrabaszcz, P; Cheney, N; Cully, A; Doncieux, S; Dyer, F C; Ellefsen, K O; Feldt, R; Fischer, S; Forrest, S; Frénoy, A; Gagné, C; Goff, Le L K; Grabowski, L M; Hodjat, B; Hutter, F; Keller, L; Knibbe, C; Krcah, P; Lenski, R E; Lipson, H; MacCurdy, R; Maestre, C; Miikkulainen, R; Mitri, S; Moriarty, D E; Mouret, J -B; Nguyen, A; Ofria, C; Parizeau, M; Parsons, D P; Pennock, R T; Punch, W F; Ray, T S; Schoenauer, M; Shulte, E; Sims, K; Stanley, K O; Taddei, F; Tarapore, D; Thibault, S; Weimer, W; Watson, R; Yosinksi, J

The Surprising Creativity of Digital Evolution: A Collection of Anecdotes from the Evolutionary Computation and Artificial Life Research Communities Journal Article

In: Artificial Life, 26 (2), pp. 274-306, 2020.

Awad, N; Mallik, N; Hutter, F

Differential Evolution for Neural Architecture Search Inproceedings

In: Proceedings of the 1st workshop on neural architecture search(@ICLR'20), 2020.

Gargiani, Matilde; Zanelli, Andrea; Tran-Dinh, Quoc; Diehl, Moritz; Hutter, Frank

Transferring Optimally Across Data Distrutions via Homotopy Methods Inproceedings

In: International Conference on Learning Representations, 2020.

Schirrmeister, Robin; Zhou, Yuxuan; Ball, Tonio; Zhang, Dan

Understanding Anomaly Detection with Deep Invertible Networks through Hierarchies of Distributions and Features Inproceedings

In: Larochelle, H; Ranzato, M; Hadsell, R; Balcan, M F; Lin, H (Ed.): Advances in Neural Information Processing Systems, pp. 21038–21049, Curran Associates, Inc., 2020.

Schorn, Christoph; Elsken, Thomas; Vogel, Sebastian; Runge, Armin; Guntoro, Andre; Ascheid, Gerd

Automated design of error-resilient and hardware-efficient deep neural networks Journal Article

In: Neural Computing and Applications, pp. 1 - 19, 2020.

Tomašev, Nenad; Cornebise, Julien; Hutter, Frank; Mohamed, Shakir; Khan, Mohammad Emtiyaz; Winne, Ruben De; Schaul, Tom; Clopath, Claudia

AI for social good: unlocking the opportunity for positive impact Journal Article

In: Nature Communications, 11 (1), 2020.

Volpp, Michael; Fröhlich, Lukas P; Fischer, Kirsten; Doerr, Andreas; Falkner, Stefan; Hutter, Frank; Daniel, Christian

Meta-Learning Acquisition Functions for Transfer Learning in Bayesian Optimization Inproceedings

In: International Conference on Learning Representations, 2020.

Zela, Arber; Siems, Julien; Hutter, Frank

NAS-Bench-1Shot1: Benchmarking and Dissecting One-shot Neural Architecture Search Inproceedings

In: International Conference on Learning Representations, 2020.

Zela, Arber; Elsken, Thomas; Saikia, Tonmoy; Marrakchi, Yassine; Brox, Thomas; Hutter, Frank

Understanding and Robustifying Differentiable Architecture Search Inproceedings

In: International Conference on Learning Representations, 2020, (Oral Presentation (Top 7%)).

2019

Rajan, Raghu; Hutter, Frank

MDP Playground: Meta-Features in Reinforcement Learning Inproceedings

In: NeurIPS 2019 Deep RL Workshop, 2019.

Bischl, Bernd; Casalicchio, Giuseppe; Feurer, Matthias; Hutter, Frank; Lang, Michel; Mantovani, Rafael G; van Rijn, Jan N; Vanschoren, Joaquin

OpenML Benchmarking Suites Journal Article

In: arXiv, 1708.0373v2 , pp. 1-6, 2019.

Biedenkapp, André; Bozkurt, Furkan H; Hutter, Frank; Lindauer, Marius

Towards White-box Benchmarks for Algorithm Control Inproceedings

In: IJCAI 2019 DSO Workshop, 2019.

Fuks, L; Awad, N; Hutter, F; Lindauer, M

An Evolution Strategy with Progressive Episode Lengths for Playing Games Inproceedings

In: Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI’19), 2019.

Lindauer, Marius; Eggensperger, Katharina; Feurer, Matthias; Biedenkapp, André; Marben, Joshua; Müller, Philipp; Hutter, Frank

BOAH: A Tool Suite for Multi-Fidelity Bayesian Optimization & Analysis of Hyperparameters Journal Article

In: arXiv:1908.06756 [cs.LG], 2019.

Lindauer, Marius; Feurer, Matthias; Eggensperger, Katharina; Biedenkapp, André; Hutter, Frank

Towards Assessing the Impact of Bayesian Optimization's Own Hyperparameters Inproceedings

In: IJCAI 2019 DSO Workshop, 2019.

Gargiani, M; Klein, A; Falkner, S; Hutter, F

Probabilistic Rollouts for Learning Curve Extrapolation Across Hyperparameter Settings Inproceedings

In: 6th ICML Workshop on Automated Machine Learning, 2019.

Feurer, Matthias; Hutter, Frank

Hyperparameter Optimization Incollection

In: Hutter, Frank; Kotthoff, Lars; Vanschoren, Joaquin (Ed.): AutoML: Methods, Sytems, Challenges, pp. 3–33, Springer, 2019.

Feurer, Matthias; Klein, Aaron; Eggensperger, Katharina; Springenberg, Jost; Blum, Manuel; Hutter, Frank

Auto-sklearn: Efficient and Robust Automated Machine Learning Incollection

In: Hutter, Frank; Kotthoff, Lars; Vanschoren, Joaquin (Ed.): AutoML: Methods, Systems, Challenges, pp. 113–134, Springer, 2019.

Mendoza, Hector; Klein, Aaron; Feurer, Matthias; Springenberg, Jost Tobias; Urban, Matthias; Burkart, Michael; Dippel, Max; Lindauer, Marius; Hutter, Frank

Towards Automatically-Tuned Deep Neural Networks Incollection

In: Hutter, Frank; Kotthoff, Lars; Vanschoren, Joaquin (Ed.): AutoML: Methods, Sytems, Challenges, pp. 135–149, Springer, 2019.

Elsken, Thomas; Metzen, Jan Hendrik; Hutter, Frank

Neural Architecture Search: A Survey Journal Article

In: Journal of Machine Learning Research, 20 (55), pp. 1-21, 2019.

Eggensperger, Katharina; Lindauer, Marius; Hutter, Frank

Pitfalls and Best Practices in Algorithm Configuration Journal Article

In: Journal of Artificial Intelligence Research (JAIR), 64 , pp. 861–893, 2019.

Elsken, Thomas; Metzen, Jan Hendrik; Hutter, Frank

Efficient Multi-Objective Neural Architecture Search via Lamarckian Evolution Inproceedings

In: International Conference on Learning Representations, 2019.

Franke, Jörg KH; Köhler, Gregor; Awad, Noor; Hutter, Frank

Neural Architecture Evolution in Deep Reinforcement Learning for Continuous Control Journal Article

In: NeurIPS 2019 Workshop on Meta-Learning, 2019.

Hutter, Frank; Kotthoff, Lars; Vanschoren, Joaquin (Ed.)

Automated Machine Learning - Methods, Systems, Challenges Book

Springer, 2019.

Loshchilov, Ilya; Hutter, Frank

Decoupled Weight Decay Regularization Inproceedings

In: International Conference on Learning Representations, 2019.

Klein, Aaron; Dai, Zhenwen; Hutter, Frank; Lawrence, Neil; Gonzalez, Javier

Meta-Surrogate Benchmarking for Hyperparameter Optimization Incollection

In: Wallach, H; Larochelle, H; Beygelzimer, A; d' Alché-Buc, F; Fox, E; Garnett, R (Ed.): Advances in Neural Information Processing Systems 32, pp. 6270–6280, Curran Associates, Inc., 2019.

Runge, Frederic; Stoll, Danny; Falkner, Stefan; Hutter, Frank

Learning to Design RNA Inproceedings

In: International Conference on Learning Representations, 2019.

Saikia, T; Marrakchi, Y; Zela, A; Hutter, F; Brox, T

AutoDispNet: Improving Disparity Estimation With AutoML Inproceedings

In: IEEE International Conference on Computer Vision (ICCV), 2019.

Ying, Chris; Klein, Aaron; Real, Esteban; Christiansen, Eric; Murphy, Kevin; Hutter, Frank

Nas-bench-101: Towards reproducible neural architecture search Inproceedings

In: Thirty-sixth International Conference on Machine Learning, 2019.

2018

Lindauer, M; van Rijn, J N; Kotthoff, L

The Algorithm Selection Competitions 2015 and 2017 Journal Article

In: Artificial Intelligence, pp. 1-35, 2018.

Chrabąszcz, Patryk; Loshchilov, Ilya; Hutter, Frank

Back to Basics: Benchmarking Canonical Evolution Strategies for Playing Atari Inproceedings

In: Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, IJCAI-18, pp. 1419–1426, International Joint Conferences on Artificial Intelligence Organization, 2018.

Eggensperger, Katharina; Lindauer, Marius; Hutter, Frank

Neural Networks for Predicting Algorithm Runtime Distributions Inproceedings

In: Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI’18), pp. 1442-1448, 2018.

Falkner, Stefan; Klein, Aaron; Hutter, Frank

BOHB: Robust and Efficient Hyperparameter Optimization at Scale Inproceedings

In: Proceedings of the 35th International Conference on Machine Learning (ICML 2018), pp. 1436–1445, 2018.

Feurer, M; Hutter, F

Towards Further Automation in AutoML Inproceedings

In: ICML 2018 AutoML Workshop, 2018.

Feurer, Matthias; Eggensperger, Katharina; Falkner, Stefan; Lindauer, Marius; Hutter, Frank

Practical Automated Machine Learning for the AutoML Challenge 2018 Inproceedings

In: ICML 2018 AutoML Workshop, 2018.

Feurer, Matthias; Letham, Benjamin; Bakshy, Eytan

Scalable Meta-Learning for Bayesian Optimization using Ranking-Weighted Gaussian Process Ensembles Inproceedings

In: ICML 2018 AutoML Workshop, 2018.

Schirrmeister, R; Chrabąszcz, P; Hutter, F; Ball, T

Training Generative Reversible Networks Inproceedings

In: ICML 2018 workshop on Theoretical Foundations and Applications of Deep Generative Models, 2018.

Zela, Arber; Klein, Aaron; Falkner, Stefan; Hutter, Frank

Towards Automated Deep Learning: Efficient Joint Neural Architecture and Hyperparameter Search Inproceedings

In: ICML 2018 AutoML Workshop, 2018.

Biedenkapp, André; Marben, Joshua; Lindauer, Marius; Hutter, Frank

CAVE: Configuration Assessment, Visualization and Evaluation Inproceedings

In: Proceedings of the International Conference on Learning and Intelligent Optimization (LION'18), 2018.

Elsken, Thomas; Metzen, Jan Hendrik; Hutter, Frank

Efficient Multi-objective Neural Architecture Search via Lamarckian Evolution Journal Article

In: ArXiv e-prints, 1804.09081 , 2018.

Ilg, Eddy; Cicek, Oezguen; Galesso, Silvio; Klein, Aaron; Makansi, Osama; Hutter, Frank; Brox, Thomas

Uncertainty Estimates for Optical Flow with Multi-Hypotheses Networks Journal Article

In: Proceedings of ECCV 2018, 2018.

Lindauer, M; Hutter, F

Warmstarting of Model-based Algorithm Configuration Inproceedings

In: Proceedings of the AAAI conference, pp. 1355–1362, 2018.

Abdulrahman, S M; Brazdil, P; van Rijn, J N; Vanschoren, J

Speeding up algorithm selection using average ranking and active testing by introducing runtime Inproceedings

In: Machine Learning, pp. 79–108, 2018.

Bandi, Peter; Geessink, Oscar; Manson, Quirine; Dijk, Marcory Van; Balkenhol, Maschenka; Hermsen, Meyke; Bejnordi, Babak Ehteshami; Lee, Byungjae; Paeng, Kyunghyun; Zhong, Aoxiao; Franke, Jörg; Both, Fabian; others,

From detection of individual metastases to classification of lymph node status at the patient level: the CAMELYON17 challenge Journal Article

In: IEEE transactions on medical imaging, 38 (2), pp. 550–560, 2018.

Eggensperger, Katharina; Lindauer, Marius; Hoos, Holger H; Hutter, Frank; Leyton-Brown, Kevin

Efficient Benchmarking of Algorithm Configurators via Model-Based Surrogates Journal Article

In: Machine Learning, 107 , pp. 15-41, 2018.

Franke, Jörg; Niehues, Jan; Waibel, Alex

Robust and Scalable Differentiable Neural Computer for Question Answering Inproceedings

In: Proceedings of the Workshop on Machine Reading for Question Answering, pp. 47–59, 2018.

van Rijn, J N; Hutter, F

Hyperparameter Importance Across Datasets Journal Article

In: SIGKDD Conference on Knowledge Discovery and Data Mining (KDD 2018), 2018.

van Rijn, J N; Holmes, G; Pfahringer, B; Vanschoren, J

The online performance estimation framework: heterogeneous ensemble learning for data streams Inproceedings

In: Machine Learning, pp. 149–176, 2018.

Wilson, James; Hutter, Frank; Deisenroth, Marc

Maximizing acquisition functions for Bayesian optimization Inproceedings

In: Bengio, S; Wallach, H; Larochelle, H; Grauman, K; Cesa-Bianchi, N; Garnett, R (Ed.): Advances in Neural Information Processing Systems 31, pp. 9906–9917, Curran Associates, Inc., 2018.

Wilson, Dennis; Rodrigues, Silvio; Segura, Carlos; Loshchilov, Ilya; Hutter, Frank; Buenfil, Guillermo López; Kheiri, Ahmed; Keedwell, Ed; Ocampo-Pineda, Mario; Özcan, Ender; Peña, Sergio Ivvan Valdez; Goldman, Brian; Rionda, Salvador Botello; Hernández-Aguirre, Arturo; Veeramachaneni, Kalyan; Cussat-Blanc, Sylvain

Evolutionary computation for wind farm layout optimization Journal Article

In: Renewable Energy, 126 , pp. 681 - 691, 2018, ISSN: 0960-1481.

2017

Martinez-Cantin, Ruben; Tee, Kevin; McCourt, Mike; Eggensperger, Katharina

Filtering Outliers in Bayesian Optimization Inproceedings

In: NeuriPS workshop on Bayesian Optimization (BayesOpt'17), 2017.

Falkner, S; Klein, A; Hutter, F

Combining Hyperband and Bayesian Optimization Inproceedings

In: NIPS 2017 Bayesian Optimization Workshop, 2017.

Klein, A; Falkner, S; Mansur, N; Hutter, F

RoBO: A Flexible and Robust Bayesian Optimization Framework in Python Inproceedings

In: NIPS 2017 Bayesian Optimization Workshop, 2017.

Lindauer, Marius; van Rijn, Jan N; Kotthoff, Lars

Open Algorithm Selection Challenge 2017: Setup and Scenarios Inproceedings

In: Lindauer, Marius; van Rijn, Jan N; Kotthoff, Lars (Ed.): Proceedings of the Open Algorithm Selection Challenge, pp. 1–7, PMLR, Brussels, Belgium, 2017.

Elsken, Thomas; Metzen, Jan Hendrik; Hutter, Frank

Simple And Efficient Architecture Search for Convolutional Neural Networks Inproceedings

In: NIPS Workshop on Meta-Learning, 2017.

Bischl, Bernd; Casalicchio, Giuseppe; Feurer, Matthias; Hutter, Frank; Lang, Michel; Mantovani, Rafael G; van Rijn, Jan N; Vanschoren, Joaquin

OpenML Benchmarking Suites and the OpenML100 Journal Article

In: arXiv, 1708.0373v1 , pp. 1-6, 2017.

Greff, K; Klein, A; Chovanec, M; Hutter, F; Schmidhuber, J

The Sacred Infrastructure for Computational Research Inproceedings

In: Proceedings of the 15th Python in Science Conference (SciPy 2017), 2017.

Lindauer, M; Hoos, H; Hutter, F; Schaub, T

AutoFolio: An Automatically Configured Algorithm Selector (Extended Abstract) Inproceedings

In: Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI'17), 2017.

Klein, A; Falkner, S; Springenberg, J T; Hutter, F

Learning Curve Prediction with Bayesian Neural Networks Inproceedings

In: International Conference on Learning Representations (ICLR) 2017 Conference Track, 2017.

Loshchilov, I; Hutter, F

SGDR: Stochastic Gradient Descent with Warm Restarts Inproceedings

In: International Conference on Learning Representations (ICLR) 2017 Conference Track, 2017.

Lindauer, M; Hoos, H; Leyton-Brown, K; Schaub, T

Automatic Construction of Parallel Portfolios via Algorithm Configuration Journal Article

In: Artificial Intelligence Journal (AIJ), 244 , pp. 272-290, 2017.

Wagner, M; Friedrich, T; Lindauer, M

Improving local search in a minimum vertex cover solver for classes of networks Inproceedings

In: Proceedings of the IEEE Congress on Evolutionary Computation (CEC), 2017.

Wagner, M; Lindauer, M; Misir, M; Nallaperuma, S; Hutter, F

A case study of algorithm selection for the traveling thief problem Journal Article

In: Journal of Heuristics, pp. 1-26, 2017.

Biedenkapp, André; Lindauer, Marius; Eggensperger, Katharina; Fawcett, Chris; Hoos, Holger H; Hutter, Frank

Efficient Parameter Importance Analysis via Ablation with Surrogates Inproceedings

In: Proceedings of the Thirty-First Conference on Artificial Intelligence (AAAI'17), pp. 773–779, 2017.

Hutter, F; Lindauer, M; Balint, A; Bayless, S; Hoos, H; Leyton-Brown, K

The Configurable SAT Solver Challenge (CSSC) Journal Article

In: Artificial Intelligence Journal (AIJ), 243 , pp. 1-25, 2017.

Lindauer, M; Hutter, F

Pitfalls and Best Practices for Algorithm Configuration (Breakout Session Report) Journal Article

In: Dagstuhl Reports, 6 , pp. 70-72, 2017.

Lindauer, M; Kotthoff, L

What can we learn from algorithm selection data? (Breakout Session Report) Journal Article

In: Dagstuhl Reports, 6 , pp. 64-65, 2017.

Klein, A; Falkner, S; Bartels, S; Hennig, P; Hutter, F

Fast Bayesian Optimization of Machine Learning Hyperparameters on Large Datasets Inproceedings

In: Proceedings of the AISTATS conference, 2017.

Klein, A; Falkner, S; Bartels, S; Hennig, P; Hutter, F

Fast Bayesian hyperparameter optimization on large datasets Inproceedings

In: Electronic Journal of Statistics, 2017.

Lindauer, M; Hoos, H; Hutter, F; Leyton-Brown, K

Selection and Configuration of Parallel Portfolios Incollection

In: Hamadi, Y; Sais, L (Ed.): Handbook of Parallel Constraint Reasoning, Springer, 2017.

Müller, Markus; Franke, Jörg; Stüker, Sebastian; Waibe, Alex

Improving phoneme set discovery for documenting unwritten languages Journal Article

In: Studientexte zur Sprachkommunikation: Elektronische Sprachsignalverarbeitung 2017, pp. 202–209, 2017.

Müller, Markus; Franke, Jörg; Waibel, Alex; Stüker, Sebastian

Towards phoneme inventory discovery for documentation of unwritten languages Inproceedings

In: 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 5200–5204, IEEE 2017.

van Rijn, J N; Hutter, F

An Empirical Study of Hyperparameter Importance Across Datasets Inproceedings

In: Proceedings of the International Workshop on Automatic Selection, Configuration and Composition of Machine Learning Algorithms (AutoML 2017), pp. 97–104, 2017.

Schirrmeister, Robin; Springenberg, Jost Tobias; Fiederer, Lukas; Glasstetter, Martin; Eggensperger, Katharina; Tangermann, Michael; Hutter, Frank; Burgard, Wolfram; Ball, Tonio

Deep learning with convolutional neural networks for EEG decoding and visualization Journal Article

In: Human Brain Mapping, 38 , pp. 5391–5420, 2017.

2016

Springenberg, J T; Klein, A; Falkner, S; Hutter, F

Bayesian optimization with robust Bayesian neural networks Inproceedings

In: Advances in Neural Information Processing Systems 29, 2016.

Bischl, B; Kerschke, P; Kotthoff, L; Lindauer, M; Malitsky, Y; Frechétte, A; Hoos, H; Hutter, F; Leyton-Brown, K; Tierney, K; Vanschoren, J

ASlib: A Benchmark Library for Algorithm Selection Journal Article

In: Artificial Intelligence Journal (AIJ), 237 , pp. 41-58, 2016.

Mendoza, H; Klein, A; Feurer, M; Springenberg, J; Hutter, F

Towards Automatically-Tuned Neural Networks Inproceedings

In: ICML 2016 AutoML Workshop, 2016.

Loshchilov, I; Hutter, F

Online Batch Selection for Faster Training of Neural Networks Inproceedings

In: International Conference on Learning Representations (ICLR) 2016 Workshop Track, 2016.

Loshchilov, I; Hutter, F

CMA-ES for Hyperparameter Optimization of Deep Neural Networks Inproceedings

In: International Conference on Learning Representations (ICLR) 2016 Workshop Track, 2016.

Manthey, N; Lindauer, M

SpyBug: Automated Bug Detection in the Configuration Space of SAT Solvers Inproceedings

In: Proceedings of the International Conference on Satisfiability Solving (SAT'16), 2016.

Wang, Ziyu; Hutter, Frank; Zoghi, Masrour; Matheson, David; de Freitas, Nando

Bayesian Optimization in a Billion Dimensions via Random Embeddings Journal Article

In: Journal of Artificial Intelligence Research (JAIR), 55 , pp. 361-387, 2016.

Lindauer, M.; Bergdoll, D.; Hutter, F.

An Empirical Study of Per-Instance Algorithm Scheduling Inproceedings

In: Proceedings of the International Conference on Learning and Intelligent Optimization (LION'16), 2016.

Franke, Joerg; Mueller, Markus; Hamlaoui, Fatima; Stueker, Sebastian; Waibel, Alex

Phoneme boundary detection using deep bidirectional lstms Inproceedings

In: Speech Communication; 12. ITG Symposium, pp. 1–5, VDE 2016.

Meinel, Andreas; Eggensperger, Katharina; Tangermann, Michael; Hutter, Frank

Hyperparameter Optimization for Machine Learning Problems in BCI (Abstract) Inproceedings

In: Proceedings of the International Brain Computer Interface Meeting 2016, 2016.

Post, Martijn J; van der Putten, Peter; van Rijn, J N

Does Feature Selection Improve Classification? A Large Scale Experiment in OpenML Inproceedings

In: Advances in Intelligent Data Analysis XV, pp. 158–170, Springer 2016.

van Rijn, J N

Massively Collaborative Machine Learning PhD Thesis

Leiden University, 2016.

Schubert, Tobias; Eggensperger, Katharina; Gkogkidis, Alexis; Hutter, Frank; Ball, Tonio; Burgard, Wolfram

Automatic Bone Parameter Estimation for Skeleton Tracking in Optical Motion Capture Inproceedings

In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA'16), 2016, (Video showing the results of the optimization procedure).

2015

Feurer, Matthias; Klein, Aaron; Eggensperger, Katharina; Springenberg, Jost Tobias; Blum, Manuel; Hutter, Frank

Efficient and Robust Automated Machine Learning Inproceedings

In: Advances in Neural Information Processing Systems 28 (NeurIPS'15), pp. 2962–2970, 2015.

Klein, A; Bartels, S; Falkner, S; Hennig, P; Hutter, F

Towards efficient Bayesian Optimization for Big Data Inproceedings

In: NIPS 2015 Bayesian Optimization Workshop, 2015.

Falkner, S; Lindauer, M; Hutter, F

SpySMAC: Automated Configuration and Performance Analysis of SAT Solvers Inproceedings

In: Proceedings of the International Conference on Satisfiability Solving (SAT'15), pp. 1-8, 2015.

Lindauer, M; Hoos, H; Hutter, F; Schaub, T

AutoFolio: An automatically configured Algorithm Selector Journal Article

In: Journal of Artificial Intelligence, 53 , pp. 745-778, 2015.

Domhan, T; Springenberg, J T; Hutter, F

Speeding up Automatic Hyperparameter Optimization of Deep Neural Networks by Extrapolation of Learning Curves Inproceedings

In: Proceedings of the 24th International Joint Conference on Artificial Intelligence (IJCAI), 2015.

Feurer, Matthias; Klein, Aaron; Eggensperger, Katharina; Springenberg, Jost Tobias; Blum, Manuel; Hutter, Frank

Methods for Improving Bayesian Optimization for AutoML Inproceedings

In: ICML 2015 AutoML Workshop, 2015.

Hutter, F; Lücke, J; Schmidt-Thieme, L

Beyond Manual Tuning of Hyperparameters Journal Article

In: Künstliche Intelligenz, 0 , pp. 1-9, 2015.

Hutter, F; Xu, L; Hoos, H H; Leyton-Brown, K

Algorithm runtime prediction: Methods & evaluation (extended abstract) Inproceedings

In: Proceedings of the 24th International Joint Conference on Artificial Intelligence (IJCAI), 2015.

Vallati, Mauro; Hutter, Frank; Chrpa, Lukáš; McCluskey, T L

On the Effective Configuration of Planning Domain Models Inproceedings

In: Proceedings of the 24th International Joint Conference on Artificial Intelligence (IJCAI), AAAI press, 2015.

Vanschoren, J; van Rijn, J; Bischl, B; Casalicchio, G; Lang, M; Feurer, M

OpenML: a Networked Science Platform for Machine Learning (Abstract) Inproceedings

In: ICML 2015 MLOSS Workshop, 2015.

Eggensperger, K; Hutter, F; Hoos, H H; Leyton-Brown, K

Efficient Benchmarking of Hyperparameter Optimizers via Surrogates Inproceedings

In: Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence, 2015.

Feurer, M; Springenberg, T; Hutter, F

Initializing Bayesian Hyperparameter Optimization via Meta-Learning Inproceedings

In: Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence, 2015.

Hoos, H; Kaminski, R; Lindauer, M; Schaub, T

aspeed: Solver Scheduling via Answer Set Programming Journal Article

In: Theory and Practice of Logic Programming, 15 , pp. 117-142, 2015.

Lindauer, M; Hoos, H; Hutter, F; Schaub, T

AutoFolio: Algorithm Configuration for Algorithm Selection Inproceedings

In: Proceedings of the Twenty-Ninth AAAI Workshops on Artificial Intelligence, 2015.

Lindauer, M; Hoos, H; F,; Hutter,

From Sequential Algorithm Selection to Parallel Portfolio Selection Inproceedings

In: Proceedings of the International Conference on Learning and Intelligent Optimization (LION'15), 2015.

van Rijn, J N; Abdulrahman, S M; Brazdil, P; Vanschoren, J

Fast algorithm selection using learning curves Inproceedings

In: Advances in Intelligent Data Analysis XIV, pp. 298–309, Springer 2015.

van Rijn, J N; Holmes, G; Pfahringer, B; Vanschoren, J

Having a Blast: Meta-Learning and Heterogeneous Ensembles for Data Streams Inproceedings

In: Data Mining (ICDM), 2015 IEEE International Conference on, pp. 1003–1008, IEEE 2015.

van Rijn, J N; Vanschoren, J

Sharing RapidMiner Workflows and Experiments with OpenML Inproceedings

In: Vanschoren, Joaquin; Brazdil, Pavel; Giraud-Carrier, Christophe; Kotthoff, Lars (Ed.): Proceedings of the 2015 International Workshop on Meta-Learning and Algorithm Selection (MetaSel), pp. 93–103, Aachen, 2015.

van Rijn, J N; Holmes, G; Pfahringer, B; Vanschoren, J

Case study on bagging stable classifiers for data streams Inproceedings

In: BENELEARN 2015, 2015.

Seipp, J; Sievers, S; Helmert, M; Hutter, F

Automatic Configuration of Sequential Planning Portfolios Inproceedings

In: Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence, 2015.

van Rijn, J N; Takes, F W; Vis, J K

The Complexity of Rummikub Problems Inproceedings

In: BNAIC 2015: Proceedings of the 27th Benelux Conference on Artificial Intelligence, 2015.

Vanschoren, J; Bischl, B; Hutter, F; Sebag, M; Kegl, B; Schmid, M; Napolitano, G; Wolstencroft, K; Williams, A R; Lawrence, N

Towards a Data Science Collaboratory Inproceedings

In: Advances in Intelligent Data Analysis XIV (IDA 2015), 2015.

Vanschoren, J; van Rijn, J N; Bischl, B

Taking machine learning research online with OpenML Inproceedings

In: Proceedings of the 4th International Workshop on Big Data, Streams and Heterogeneous Source Mining: Algorithms, Systems, Programming Models and Applications, pp. 1–4, 2015.

2014

Seipp, Jendrick; Sievers, Silvan; Hutter, Frank

Fast Downward Cedalion Miscellaneous

2014, (Planner abstract, IPC 2014 Planning and Learning TrackBest learner award, and second place in the category overall best quality at the IPC 2014 Planning and Learning Track. Also achieved the highest coverage in the IPC 2014 sequential agile planning track.).

Seipp, Jendrick; Sievers, Silvan; Hutter, Frank

Fast Downward SMAC Miscellaneous

2014, (Planner abstract, IPC 2014 Planning and Learning TrackBest basic solver award, and third place in the categories overall best quality and best learner.).

Eggensperger, Katharina; Hutter, Frank; Hoos, Holger H; Leyton-Brown, Kevin

Surrogate Benchmarks for Hyperparameter Optimization Inproceedings

In: ECAI workshop on Metalearning and Algorithm Selection (MetaSel), pp. 24-31, 2014, (Superseeded by the AAAI15 paper _Efficient Benchmarking of Hyperparameter Optimizers via Surrogates_).

Feurer, M; Springenberg, T; Hutter, F

Using Meta-Learning to Initialize Bayesian Optimization of Hyperparameters Inproceedings

In: ECAI workshop on Metalearning and Algorithm Selection (MetaSel), pp. 3–10, 2014, (Superseeded by the AAAI15 paper _Initializing Bayesian Hyperparameter Optimization via Meta-Learning_).

Domhan, Tobias; Springenberg, Tobias; Hutter, Frank

Extrapolating Learning Curves of Deep Neural Networks Inproceedings

In: ICML 2014 AutoML Workshop, 2014.

Fawcett, Chris; Vallati, Mauro; Hutter, Frank; Hoffmann, Jörg; Hoos, Holger; Leyton-Brown, Kevin

Improved Features for Runtime Prediction of Domain-Independent Planners Inproceedings

In: Proceedings of the International Conference on Automated Planning and Scheduling (ICAPS 2014), 2014.

Hutter, F; Hoos, H; Leyton-Brown, K

An Efficient Approach for Assessing Hyperparameter Importance Inproceedings

In: Proceedings of International Conference on Machine Learning 2014 (ICML 2014), pp. 754–762, 2014.

Leyton-Brown, Kevin; Hoos, Holger; Hutter, Frank; Xu, Lin

Understanding the Empirical Hardness of NP-complete Problems Journal Article

In: Communications of the Association for Computing Machinery (CACM), 57 (5), pp. 98–107, 2014.

Geschwender, Daniel; Hutter, Frank; Kotthoff, Lars; Malitsky, Yuri; Hoos, Holger; Leyton-Brown, Kevin

Algorithm Configuration in the Cloud: A Feasibility Study Inproceedings

In: Proceedings of the Learning and Intelligent OptimizatioN Conference (LION 8), 2014.

Hoos, H; Lindauer, M; Schaub, T

claspfolio 2: Advances in Algorithm Selection for Answer Set Programming Journal Article

In: Theory and Practice of Logic Programming, 14 , pp. 569-585, 2014.

Hoogeboom, H J; Kosters, W A; van Rijn, J N; Vis, J K

Acyclic Constraint Logic and Games Journal Article

In: ICGA Journal, 37 (1), pp. 3–16, 2014.

Hutter, Frank; López-Ibáñez, Manuel; Fawcett, Chris; Lindauer, Marius; Hoos, Holger; Leyton-Brown, Kevin; Stützle, Thomas

AClib: a Benchmark Library for Algorithm Configuration Inproceedings

In: Proceedings of the Learning and Intelligent OptimizatioN Conference (LION 8), 2014.

Hutter, F; Xu, L; Hoos, H H; Leyton-Brown, K

Algorithm runtime prediction: Methods & evaluation Journal Article

In: Artificial Intelligence, 206 (0), pp. 79–111, 2014, (The data and source code for this paper are available from our Empirical Performance Models project page).

Lindauer, M

Algorithm Selection, Scheduling and Configuration of Boolean Constraint Solvers PhD Thesis

University of Potsdam, 2014, (Preliminary Version).

van Rijn, J N; Holmes, G; Pfahringer, B; Vanschoren, J

Algorithm Selection on Data Streams Incollection

In: Discovery Science, 8777 , pp. 325–336, Springer, 2014.

van Rijn, J N; Vis, J K

Endgame Analysis of Dou Shou Qi Journal Article

In: ICGA Journal, 37 (2), pp. 120–124, 2014.

van Rijn, J N; Holmes, G; Pfahringer, B; Vanschoren, J

Towards meta-learning over data streams Inproceedings

In: MetaSel 2014, pp. 37–38, CEUR-WS 2014.

Vanschoren, J; van Rijn, J N; Bischl, B; Torgo, L

OpenML: networked science in machine learning Journal Article

In: ACM SIGKDD Explorations Newsletter, 15 (2), pp. 49–60, 2014.

2013

Eggensperger, Katharina; Feurer, Matthias; Hutter, Frank; Bergstra, James; Snoek, Jasper; Hoos, Holger H; Leyton-Brown, Kevin

Towards an Empirical Foundation for Assessing Bayesian Optimization of Hyperparameters Inproceedings

In: NeurIPS workshop on Bayesian Optimization in Theory and Practice, 2013, (Software and benchmarks are available from our HPOlib website.).

Hutter, Frank; Hoos, Holger H; Leyton-Brown, Kevin

An Efficient Approach for Assessing Parameter Importance in Bayesian Optimization Inproceedings

In: NIPS workshop on Bayesian Optimization in Theory and Practice, 2013.

Swersky, Kevin; Duvenaud, David; Snoek, Jasper; Hutter, Frank; Osborne, Michael

Raiders of the Lost Architecture: Kernels for Bayesian Optimization in Conditional Parameter Spaces Inproceedings

In: NIPS workshop on Bayesian Optimization in Theory and Practice, 2013.

Gebser, M; Jost, H; Kaminski, R; Obermeier, P; Sabuncu, O; Schaub, T; Schneider, M

Ricochet Robots: A transverse ASP benchmark Inproceedings

In: pp. 348-360, 2013.

Thornton, C; Hutter, F; Hoos, H H; Leyton-Brown, K

Auto-WEKA: Combined Selection and Hyperparameter Optimization of Classification Algorithms Inproceedings

In: Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD'13), 2013, (The software is available from our Auto-WEKA page.).

Wang, Z; Zoghi, M; Hutter, F; Matheson, D; de Freitas, N

Bayesian Optimization in High Dimensions via Random Embeddings Inproceedings

In: Proceedings of the 23rd international joint conference on Artificial Intelligence (IJCAI), pp. 1778-1784, AAAI Press 2013, (Distinguished paper award. ).

Hutter, F; Hoos, H H; Leyton-Brown, K

An Evaluation of Sequential Model-Based Optimization for Expensive Blackbox Functions Inproceedings

In: Proceedings of GECCO-13 Workshop on Blackbox Optimization Benchmarking (BBOB'13), 2013, (Software and data are available from the SMAC page.).

Hoos, H; Kaufmann, B; Schaub, T; Schneider, M

Robust Benchmark Set Selection for Boolean Constraint Solvers Inproceedings

In: pp. 138-152, 2013.

Hutter, F; Hoos, H H; Leyton-Brown, K

Identifying Key Algorithm Parameters and Instance Features using Forward Selection Incollection

In: Nicosia, Giuseppe; Pardalos, Panos (Ed.): Proceedings of the 7th International Conference on Learning and Optimization (LION-7), Springer Berlin Heidelberg, 2013, (The data and source code for this paper are available from our Empirical Performance Models project page.).

Pardalos, P; Nicosia, G (Ed.)

Proceedings of the Seventh International Conference on Learning and Intelligent Optimization (LION'13) Proceeding

Springer-Verlag, 7997 , 2013.

Cabalar, P; Son, T (Ed.)

Proceedings of the Twelfth International Conference on Logic Programming and Nonmonotonic Reasoning (LPNMR'13) Proceeding

Springer-Verlag, 8148 , 2013.

van Rijn, J N; Bischl, B; Torgo, L; Gao, B; Umaashankar, V; Fischer, S; Winter, P; Wiswedel, B; Berthold, M R; Vanschoren, J

OpenML: A Collaborative Science Platform Incollection

In: Machine Learning and Knowledge Discovery in Databases, pp. 645–649, Springer, 2013.

van Rijn, J N; Umaashankar, V; Fischer, S; Bischl, B; Torgo, L; Gao, B; Winter, P; Wiswedel, B; Berthold, M R; Vanschoren, J

A RapidMiner extension for open machine learning Inproceedings

In: RapidMiner Community Meeting and Conference, pp. 59–70, 2013.

van Rijn, J N; Vis, J K

Complexity and retrograde analysis of the game Dou Shou Qi Inproceedings

In: BNAIC 2013: Proceedings of the 25th Benelux Conference on Artificial Intelligence, Delft University of Technology (TU Delft); under the auspices of the Benelux Association for Artificial Intelligence (BNVKI) and the Dutch Research School for Information and Knowledge Systems (SIKS) 2013.

2012

Kaufmann, B; Schaub, T; Schneider, M

clasp, claspfolio, aspeed: Three Solvers from the Answer Set Solving Collection Potassco Inproceedings

In: pp. 17-19, 2012.

Hoos, H; Kaminski, R; Schaub, T; Schneider, M

aspeed: ASP-based Solver Scheduling Inproceedings

In: pp. 176-187, 2012.

Silverthorn, B; Lierler, Y; Schneider, M

Surviving Solver Sensitivity: An ASP Practitioner's Guide Inproceedings

In: pp. 164-175, 2012.

Xu, L; Hutter, F; Shen, J; Hoos, H; Leyton-Brown, K

SATzilla2012: Improved Algorithm Selection Based on Cost-sensitive Classification Models Unpublished

2012, (Published online. Solver description for the 2012 SAT challenge. SATzilla2012 won 3 out of the 4 categories for which it was eligible, and placed 2nd in the remaining one. Details: it won the sequential portfolio track, was the best solver for 2 of the 3 main sequential categories (Application and Hard Combinatorial), and 2nd in the sequential Random Category (beaten only by a new non-portfolio solver, CCASAT). See the SATzilla project page for details on SATzilla and source code.).

Xu, Lin; Hutter, Frank; Hoos, Holger H; Leyton-Brown, Kevin

Evaluating Component Solver Contributions to Portfolio-Based Algorithm Selectors Inproceedings

In: International Conference on Theory and Applications of Satisfiability Testing (SAT'12), 2012.

Schneider, M; Hoos, H

Quantifying Homogeneity of Instance Sets for Algorithm Configuration Inproceedings

In: Learning and Intelligent Optimization (LION'12), 2012.

Hoos, Holger; Leyton-Brown, Kevin; Schaub, Torsten; Schneider, Marius

Algorithm Configuration for Portfolio-based Parallel SAT-Solving Journal Article

In: Proceedings of the First Workshop on Combining Constraint Solving with Mining and Learning (CoCoMile'12), 2012.

Hutter, F; Hoos, H H; Leyton-Brown, K

Parallel Algorithm Configuration Inproceedings

In: Proceedings of the Learning and Intelligent OptimizatioN Conference LION 6, pp. 55-70, 2012.

Dovier, A; Costa, Santos V (Ed.)

Technical Communications of the Twenty-eighth International Conference on Logic Programming (ICLP'12) Proceeding

Leibniz International Proceedings in Informatics (LIPIcs), 17 , 2012.

Dovier, A; Costa, Santos V (Ed.)

Technical Communications of the Twenty-eighth International Conference on Logic Programming (ICLP'12) Proceeding

Leibniz International Proceedings in Informatics (LIPIcs), 17 , 2012.

Hamadi, Y; Schoenauer, M (Ed.)

Proceedings of the Sixth International Conference on Learning and Intelligent Optimization (LION'12) Proceeding

Springer-Verlag, 2012.

Balint, A; Belov, A; Diepold, D; Gerber, S; Järvisalo, M; Sinz, C (Ed.)

Proceedings of SAT Challenge 2012: Solver and Benchmark Descriptions Proceeding

University of Helsinki, B-2012-2 , 2012, (Available at r̆lhttps://helda.helsinki.fi/handle/10138/34218).

2011

Hutter, Frank; Hoos, Holger H; Leyton-Brown, Kevin

Bayesian Optimization With Censored Response Data Inproceedings

In: NIPS workshop on Bayesian Optimization, Sequential Experimental Design, and Bandits, 2011, (Published online. There is also a new, extended arXiv version.).

Xu, Lin; Hutter, Frank; Hoos, Holger H; Leyton-Brown, Kevin

Hydra-MIP: Automated Algorithm Configuration and Selection for Mixed Integer Programming Inproceedings

In: RCRA workshop on Experimental Evaluation of Algorithms for Solving Problems with Combinatorial Explosion at the International Joint Conference on Artificial Intelligence (IJCAI), 2011.

Xu, Lin; Hutter, Frank; Hoos, Holger; Leyton-Brown, Kevin

Detailed SATzilla Results from the Data Analysis Track of the 2011 SAT Competition Miscellaneous

2011.

Gebser, M; Kaminski, R; Kaufmann, B; Schaub, T; Schneider, M; Ziller, S

A Portfolio Solver for Answer Set Programming: Preliminary Report Inproceedings

In: pp. 352-357, 2011.

Gebser, M; Kaminski, R; Kaufmann, B; M, Ostrowski; Schaub, T; Schneider, M

Potassco: The Potsdam Answer Set Solving Collection Journal Article

In: AI Communications, 24 (2), pp. 107-124, 2011.

Hutter, F; Hoos, H H; Leyton-Brown, K

Sequential Model-Based Optimization for General Algorithm Configuration Inproceedings

In: Proceedings of the conference on Learning and Intelligent OptimizatioN (LION 5), pp. 507-523, 2011, (Best paper award (second prize). SMAC, ROAR, and the instances used are available from the Automated Algorithm Configuration project page. An extended version with additional details is available as UBC tech report TR-2010-10. (pdf) (bib)).

Delgrande, J; Faber, W (Ed.)

Proceedings of the Eleventh International Conference on Logic Programming and Nonmonotonic Reasoning (LPNMR'11) Proceeding

Springer-Verlag, 6645 , 2011.

Möller, M; Schneider, M; Wegner, M; Schaub, T

Centurio, a General Game Player: Parallel, Java- and ASP-based Journal Article

In: Künstliche Intelligenz, 25 (1), pp. 17-24, 2011.

2010

Hutter, F; Bartz-Beielstein, T; Hoos, H H; Leyton-Brown, K; Murphy, K P

Sequential Model-Based Parameter Optimisation: an Experimental Investigation of Automated and Interactive Approaches Incollection

In: Bartz-Beielstein, T; Chiarandini, M; Paquete, L; Preuss, M (Ed.): Empirical Methods for the Analysis of Optimization Algorithms, pp. 361–411, Springer, 2010.

Hutter, Frank; Hoos, Holger H; Leyton-Brown, Kevin

Tradeoffs in the Empirical Evaluation of Competing Algorithm Designs Journal Article

In: Annals of Mathematics and Artificial Intelligenc (AMAI), Special Issue on Learning and Intelligent Optimization, 60 (1), pp. 65–89, 2010, (The data from this paper, as well as the empirical analysis tools we introduced are available from the Automated Algorithm Configuration project page.).

Hutter, F; Hoos, H H; Leyton-Brown, K

Sequential Model-Based Optimization for General Algorithm Configuration (extended version) Technical Report

University of British Columbia, Department of Computer Science (TR-2010-10), 2010.

Hutter, F; Hoos, H H; Leyton-Brown, K

Automated Configuration of Mixed Integer Programming Solvers Inproceedings

In: Proceedings of the Conference on Integration of Artificial Intelligence and Operations Research techniques in Constraint Programming (CPAIOR), pp. 186-202, 2010, (Our webpage on Automated Configuration of MIP solvers also gives the parameter files for CPLEX, Gurobi, and lpsolve.).

Hutter, F; Hoos, H H; Leyton-Brown, K; Murphy, K P

Time-Bounded Sequential Parameter Optimization Inproceedings

In: Proceedings of the conference on Learning and Intelligent OptimizatioN (LION 4), 2010, (Runner-up for the best paper award).

2009

Hutter, Frank; Hoos, Holger H; Leyton-Brown, Kevin; Stützle, Thomas

ParamILS: An Automatic Algorithm Configuration Framework Journal Article

In: Journal of Artificial Intelligence Research, 36 , pp. 267–306, 2009, (See the ParamILS project page for a lot of experimental data for this paper (target algorithms, parameters, resulting parameter configurations). There's also a quick start guide available to help you apply ParamILS for tuning your own algorithms. There's also an older tech report about ParamILS, including additional material (pdf) (bib).).

Hutter, F

Automated Configuration of Algorithms for Solving Hard Computational Problems PhD Thesis

University of British Columbia, Department of Computer Science, 2009, (There are also slides from invited presentation at Canadian AI grad student symposium. 2010 CAIAC Doctoral Dissertation Award for the best thesis in Artificial Intelligence at a Canadian University completed in 2009. See the Automated Algorithm Configuration project page for a lot of experimental data (target algorithms, parameters, benchmark instances, and configuration proceduers).).

Hutter, F; de Oca, M A Montes (Ed.)

SLS-DS 2009: Doctoral Symposium on Engineering Stochastic Local Search Algorithms Proceeding

IRIDIA, Université Libre de Bruxelles, Brussels, Belgium 2009.

Hutter, F; Hoos, H H; Leyton-Brown, K; Murphy, K P

An Experimental Investigation of Model-Based Parameter Optimisation: SPO and Beyond Inproceedings

In: Proceedings of the 11th annual conference on Genetic and evolutionary computation (GECCO '09), pp. 271–278, 2009.

Hutter, F; Hoos, H H; Leyton-Brown, K; Stützle, T

ParamILS: An Automatic Algorithm Configuration Framework Technical Report

University of British Columbia (TR-2009-01), 2009.

Xu, L; Hutter, F; Hoos, H; Leyton-Brown, K

SATzilla2009: an Automatic Algorithm Portfolio for SAT Unpublished

2009, (Solver description, SAT competition 2009Solver description for the 2009 SAT competition. SATzilla2009 won 3 gold and 2 silver medals in that competition. See the SATzilla project page for details and source code.).

2008

Xu, Lin; Hutter, Frank; Hoos, Holger H; Leyton-Brown, Kevin

SATzilla: Portfolio-based Algorithm Selection for SAT Journal Article

In: Journal of Artificial Intelligence Research, 32 , pp. 565–606, 2008, (2010 IJCAI/JAIR Best Paper Prize for the period 2005-2009. See the SATzilla project page for details and source code.).

2007

Hutter, Frank; Babic, Domagoj; Hoos, Holger H; Hu, Alan J

Boosting Verification by Automatic Tuning of Decision Proceedingsdures Inproceedings

In: Proceedings of Formal Methods in Computer Aided Design (FMCAD'07), pp. 27–34, IEEE Computer Society, Washington, DC, USA, 2007, (With the tuning discussed in this paper Domagoj's solver Spear won the QF_BV (Quantifier-Free Bit Vector) category of the 2007 Satisfiability Modulo Theories Competition.).

Hutter, Frank

On the Potential of Automatic Algorithm Configuration Inproceedings

In: Proceedings of the Doctoral Symposium on Engineering Stochastic Local Search Algorithms (SLS-DS)., 2007, (Best poster award (voted by the attendees of SLS 07).).

Xu, L; Hutter, F; Hoos, H H; Leyton-Brown, K

SATzilla-07: The Design and Analysis of an Algorithm Portfolio for SAT Inproceedings

In: Principles and Practice of Constraint Programming (CP'07), 2007, (SATzilla won 3 gold medals, 1 silver and 1 bronze in the 2007 SAT competition! It is available for download from the SATzilla website.).

Hutter, F; Hoos, H; Stützle, T

Automatic Algorithm Configuration based on Local Search Inproceedings

In: Proceedings of the Twenty-Second Conference on Artifical Intelligence (AAAI '07), pp. 1152–1157, 2007, (The ParamILS algorithm introduced in this paper is available for download from the ParamILS website. There's also a quick start guide available to help you apply it for tuning your own algorithms.).

Babić, D; Hutter, F

SPEAR Theorem Prover Miscellaneous

2007, (Solver description, SAT competitionSPEAR is a new tree search algorithm with 25 free parameters. I tuned it (using the techniques from the AAAI-07 paper on ParamILS), getting a 30% speedup; for software verification, my parameter settings beat the default by a factor of 50!).

Tompkins, Dave; Hutter, Frank; H, Holger; Hoos,

Scaling and Probabilistic Smoothing (SAPS) Miscellaneous

2007, (SAPS is unchanged from last year, but I got a tenfold speedup by automated parameter tuning (using the techniques from the AAAI-07 ParamILS paper)).

Xu, Lin; Hutter, Frank; Hoos, Holger H; Leyton-brown, Kevin

SATzilla2007: a new & improved algorithm portfolio for SAT Miscellaneous

2007, (In a nutshell, SATzilla predicts the runtime of each solver in the portfolio and picks the most promising one. SATzilla2007 won 3 gold medals, 1 silver and 1 bronze! See the SAT competition webpage for details.).

2006

Hutter, F; Hamadi, Y; Hoos, H H; Leyton-Brown, K

Performance Prediction and Automated Tuning of Randomized and Parametric Algorithms Inproceedings

In: Principles and Practice of Constraint Programming (CP'06), pp. 213–228, 2006, (All our experimental data for this paper, as well as our Matlab code, is available on the Empirical Hardness Models project page.).

Hutter, Frank

Automated Algorithm Configuration Based on Machine Learning Miscellaneous

2006.

2005

Hutter, Frank; Hamadi, Youssef

Parameter Adjustment Based on Performance Prediction: Towards an Instance-Aware Problem Solver Technical Report

Microsoft Research Cambridge, UK, (MSR-TR-2005-125), 2005, (Slides from a talk I gave at the Cork Constraint Computation Centre (4C) Slides from a talk I gave in the Lab for Computational Intelligence at UBC).

Hutter, Frank; Hoos, Holger H; Stützle, Thomas

Efficient Stochastic Local Search for MPE Solving Inproceedings

In: Proceedings of the Nineteenth International Joint Conference on Artificial Intelligence (IJCAI-05), pp. 169–174, 2005, (My solver GLS+ and the test instances we used are available on our MPE page. The solver can read general factor graphs, i.e. Bayes nets (in BNT format), MRFs, CRFs, etc. There's also a nice Matlab interface.).

2004

Hutter, Frank

Stochastic Local Search for Solving the Most Probable Explanation Problem in Bayesian Networks Masters Thesis

Darmstadt University of Technoloy, 2004, (Supervisor: Thomas Stützle, Cosupervisor: Holger Hoos; My solver GLS+ and most of the test instances I used are available on our MPE page.).

Hutter, Frank; Ng, Brenda; Dearden, Richard

Incremental Thin Junction Trees for Dynamic Bayesian Networks Technical Report

Intellectics Group, Darmstadt University of Technology (TR-AIDA-04-01), 2004.

Dearden, Richard; Willeke, Thomas; Hutter, Frank; Simmons, Reid; Verma, Vandi; Thrun, Sebastian

Real-time Fault Detection and Situational Awareness for Rovers: Report on the Mars Technology Program Task Inproceedings

In: In Proceedings of IEEE Aerospace Conference, 2004, pp. 826–840, IEEE Press, 2004, (Check out my GPF webpage for the particle filtering code.).

de Freitas, Nando; Dearden, Richard; Hutter, Frank; Morales-Menendez, Ruben; Mutch, Jim; Poole, David

Diagnosis by a Waiter and a Mars Explorer Journal Article

In: Proceedings of the IEEE, 92 (4), pp. 139-144, 2004, (Check out my GPF webpage for the particle filtering code used for the rover examples.).

Andronescu, M; Fejes, A P; Hutter, F; Hoos, H H; Condon, A

A new algorithm for RNA secondary structure design Journal Article

In: Journal of Molecular Biology, 336 (3), pp. 607–624, 2004, (Check out the free RNA Designer Software at ).

2003

Hutter, Frank; Dearden, Richard

The Gaussian Particle Filter for Diagnosis of Non-Linear Systems Inproceedings

In: Proceedings of the 14th International Conference on Principles of Diagnosis(DX03), pp. 5–70, 2003, (Check out my GPF webpage for the Gaussian particle filtering code.).

Hutter, Frank; Dearden, Richard

Efficient On-line Fault Diagnosis for Non-Linear Systems Inproceedings

In: Seventh International Symposium on Artificial Intelligence and Robotics in Space (i-SAIRAS-03), 2003, (Check out my GPF webpage for the Gaussian particle filtering code.).

2002

Hutter, F; Tompkins, D A D; Hoos, H H

Scaling and Probabilistic Smoothing: Efficient Dynamic Local Search for SAT Inproceedings

In: Hentenryck, Pascal (Ed.): Principles and Practice of Constraint Programming - CP 2002, pp. 233-248, Springer Berlin Heidelberg, 2002, (Check out the DLS for SAT webpage, maintained by Dave.).

Andronescu, M; Fejes, A P; Hutter, F; Hoos, H H; Condon, A

A New SLS Algorithm for RNA Secondary Structure Design Technical Report

Department of Computer Science, University of British Columbia (TR-2002-10), 2002, (Available as a postscript file. Check out the free RNA Designer Software at http://www.rnasoft.ca/).