CLSquare: Closed Loop Simulation System
CLS2 (pronounced: clsquare) provides a standardized framework for testing (not only) Reinforcement Learning controllers on a (growing) number of different plants.
- Easy combination of different plants and controllers by definition in a configuration file.
- Lean interfaces for plants and controllers for a quick migration to and from CLS2 from and to other software environments.
- Optional graphical interface and visualization.
- Many demos for easy starting.
- Cart Pole (single, double and parallel), underactuated pole
- Mountain Car
- Hardware Interfaces
- Various static standard controllers (linear, PID…)
- Table-based Q Reinforcement Learning
- Neural Q and policy controllers
- Kernel-based batch controller (Ormoneit and Sen, 2002)
- Fitted Q Iteration with Extra Trees (Ernst, Geurts and Wehenkel, 2005)
The software is developed and tested under:
- Ubuntu Linux 12.04
- Mac OSX 10.8
- a C++ compiler (build-essential for Ubuntu, xcode for OSX)
- CMake 2.6.3 (2.8.2 is required for libgp)
Each module that has special requirements will report them during installation if they are missing.
The latest software release can be downloaded from here.
- Get a fresh copy of CLS2
- Unpack the source code
tar xzf clsquare.tgz
- Compile the program
The most recent manual is included in the source package. For convenience reasons, you can find the manual here, too: CLSquare Manual v4.0
In addition, documentation for the available modules can be generated using Doxygen by typing
make doc in the program's build folder.
Researchers contributing to this project:
- Prof. Dr. Martin Riedmiller
- Manuel Blum
- Thomas Lampe
- Dr. Roland Hafner
- Dr. Sascha Lange
- Dr. Stephan Timmer
For more information on this research project, please contact Martin Riedmiller.