Perceptrovement

Introduction

This framework called Perceptrovement implements all the algorithms from the bachelor thesis Improving the Perceptron-Algorithm by Tobias Krönke in 2011. Get your hands dirty and find out how the simple, yet famous perceptron-algorithm (Rosenblatt 1962) can help you solve your machine learning problem by incrementally learning a linear binary classification model. It was upgraded with new means to update the current model, voting and surviving model approaches, ideas to reduce the number of required iterations (epochs) over your data and different options to grasp the idea of SVMs' (soft) margins to increase robustness against noisy data and labels.

Included algorithms and functions

• Pegasos, Primal Estimated sub-GrAdient SOlver for SVM
  Shai S. Shwartz, Yoram Singer und Nathan Srebro: Pegasos: Primal Estimated sub-GrAdient SOlver for SVM. In: ICML ’07: Proceedings of the 24th international conference on Machine learning, pp. 807–814, 2007. ACM.
  
• MICRA, mistake-controlled rule algorithm and reduced epochs
  Petroula Tsampouka und John Shawe-Taylor: Approximate maximum margin algorithms with rules controlled by the number of mistakes. In: Proceedings of the 24th international conference on Machine learning, ICML ’07,pp. 903–910, 2007. ACM.
  
• PA, passive-aggressive family of online binary classification algorithms
  Koby Crammer, Ofer Dekel, Joseph Keshet, Shai Shalev-Shwartz und Yoram Singer: Online Passive-Aggressive Algorithms. Journal of Machine Learning Research, 7:551–585, 2006.
  
• PAUM, perceptron with uneven margins
  Yaoyong Li, Hugo Zaragoza, Ralf Herbrich, John Shawe-Taylor und Jaz S. Kandola: The Perceptron Algorithm with Uneven Margins. In: Proceedings of the Nineteenth International Conference on Machine Learning, ICML ’02, pp. 379–386.
  
• support vector budgeting through removal and projection
  Zhuang Wang, Koby Crammer und Slobodan Vucetic: Multi-Class Pegasos on a Budget. In: ICML, 2010.
• lamda-trick and alpha-bound
  Roni Khardon und Gabriel Wachman: Noise Tolerant Variants of the Perceptron Algorithm. J. Mach. Learn. Res., 8:227–248, 2007.
• longest surviving and voting perceptrons
  Yoav Freund und Robert E. Schapire: Large Margin Classification Using the Perceptron Algorithm. Machine Learning, 37(3):277–296, 1999.
• perceptrons in dual and primal (linear) form
• kernels
  

Downloads

Framework (Java 1.6 required, WEKA included): perceptrovement1106.zip

Thesis (German): Kroenke_Tobias.pdf

Setup

For Linux, Mac, Windows:

• Extract the .zip file

• Execute run.sh or run.bat (Windows) with Perceptrovement as your working directory

  • /usr/share/java/postgresql.jar may be removed from the class path

Quick usage instructions

Perceptrovement is made available as a classifier in the WEKA machine learning framework. With the explorer for example you can easily test it on appropriate data sets and easily adjust its options with the generic option editor. Mouse-over hints give you a quick idea of what does what. For more details please refer to the thesis.

Terms of Use

Feel free to modify and further extend this framework for your needs. Nevertheless, we would be glad if you cite this site or the thesis if you use this framework. For external software or data that may be included in the distributables like libraries or datasets, please refer to the information given therein or contact the original authors for their terms of use.

Contact

Tobias Krönke, Eneldo Loza Mencía