Duration: 4 hours


Computers that can program themselves is an old dream of Artificial Intelligence, but only nowadays there is some progress of remark. In relation to Machine Learning, a computer program is the most powerful structure that can be learned, pushing the final goal well beyond neural networks or decision trees. There are currently many separate areas, working independently, related to automatic programming, both deductive and inductive. The first goal of this tutorial is to give to the attendants a comprehensive view of the main areas related to the automatic induction of programs, a view which is not currently available to the community. ML researchers which do not know about Automatic Programming or researchers which work in just one of the areas would benefit from this tutorial.

The expressivity of most Machine Learning languages (attribute-value) is basically equivalent to propositional logic, excluding work on ILP. The second goal of the tutorial is to show how we can go beyond these techniques by extending the expression power of the representation language. This can be done by adding elements programmers typically use, like variables, subroutines, loops, and recursion. This way, more complex problems can be addressed.

The tutorial will start with a short overview of the different areas related to Automatic Programming. Most of the tutorial will focus on evolutionary / search-based techniques for generating programs. Slides are available at full size as pdf files. Please, take them from here and print them with several slides per page. Most likely, I will not be able to print them when I get to Pittsburgh on Sunday. In case you find any problem with the files, please contact me at


  1. Overview: ( Intro-Survey.pdf )
    • Deductive and inductive automatic programming
    • Synthesis of LISP programs and Learning by Example
    • Inductive Logic Programming (ILP) for program synthesis
  2. Genetic Algorithms and AIP: Genetic Programming ( Genetic-Programming-short.pdf )
  3. Estimation of Distribution Approaches and AIP: Probabilistic Incremental Program Evolution ( PIPE-short.pdf )
  4. Automatic Functional Programming: Automatic Design of Algorithms Through Evolution ( ADATE.pdf )
  5. Near-Bias-Optimal Search: The Optimal Ordered Problem Solver (OOPS) ( OOPS.pdf )
  6. Conclusions
For Genetic Programming and PIPE, some longer versions are available, although it will not be possible to cover all that ground in the tutorial due to time constraints. Here they are in case you want to peruse them;


Ricardo Aler Mur is associate professor at Universidad Carlos III de Madrid. He received a PhD from Universidad Politecnica de Madrid (1999). The research applied Genetic Programming to Learning Planning Heuristics. He also received a MSc from the University of Sunderland (1993). His main research interests are Machine Learning and Genetic Programming. In the remote case you need to cite this tutorial, please do so by quoting:

R. Aler, Automatic Inductive Programming , Tutorial at the International Conference on Machine Learning, 2006. Carnegie Mellon, Pittsburgh, Pennsylvania, U.S.A.

Interesting links:


  • This tutorial was partially supported by the OPLINK project (TIN2005-08818-C04-02) [Ministerio de Educación y Ciencia] and CIBMIN (UC3M-TEC-05-029)) [Comunidad Autónoma de Madrid y Universidad Carlos III de MAdrid]

Back to ICML'06 tutorials here.