Reconstruction of Boolean Functions from Deep Neural Networks
| Type of publication: | Mastersthesis |
| Citation: | ba:gonzalez |
| Type: | Bachelor Thesis |
| Year: | 2017 |
| Month: | May |
| School: | TU Darmstadt, Knowledge Engineering Group |
| URL: | /lehre/arbeiten/bachelor/2017/Gonzalez_Camila.pdf |
| Abstract: | Due to improvements in the training practices, the popularity of deep neural networks and their applicability to a wider set of problems have increased considerably in the last few years. This has brought about a renewed interest in increasing their interpretability, an aspect where they fall behind other prediction models. It is unclear what provides a sufficiently expressive explanation of the inner workings of a deep model, but symbolic representations in the form of rule sets have been proven capable of illustrating their behaviour as a whole, as well as the hidden concepts they model in the intermediate layers. Diverse methods have been developed for this purpose. However, most approaches are not scalable to deep architec- tures with a high number of hidden units and which were trained with huge amounts of data. In order to enable rule extraction in such situations, the search space of possible concepts has to be appropriately reduced. Within this thesis, the possibility is explored of extending techniques which have been shown to facilitate the extraction of rule representations from shallow neural networks to deep architectures. These methods, which go from altering the internal structure of the network to choosing appropriate values to partition the activation ranges, are tested in combination with an existing algorithm for extracting rules from deep models. Instead of using datasets which combine the attributes in an unclear manner, the networks are trained to reproduce predefined boolean concepts so it can later be assessed to what degree the patterns were captured in the rule sets. The evaluation shows that reducing the connectivity of the neural networks significantly assists latter rule extraction, as does encouraging minimal or maximal activation states. It is also demonstrated that these can be discretised without causing a significant decrease in accuracy. |
| Userfields: | betreuer={ELM} |
| Keywords: | |
| Authors | |
|
Topics
|
|
|
|
