This is a past event
Deep Learning and Ensemble Learning
Chrisina Jayne and Eyad Elyan
There has been a significant research interest in recent years related to deep architectures, which refer to neural network architectures with a larger number of hidden layers. The breakthrough paper published in 2006 by Hinton et al. introduced unsupervised fast, greedy learning algorithm that finds a fairly good set of parameters quickly in deep networks with millions of parameters and many hidden layers. The unsupervised learning in the deep architectures helps also with extracting salient information about the input distribution captured in the hidden layers. In addition to this we now have sufficient computing power and data to train and tune deep neural network models.
In this presentation we will consider two contributions related to deep learning architectures. The first one relates to the application of a novel Deep Neural Network (DNN) architecture to the problem of matching data in different modes. There are a number of situations in which we wish to identify matching objects, which have been represented in different ways. Common examples are faces (or images in general) viewed in different frequency bands or lighting conditions, or represented as photos or sketches. While the representations differ, there are similarities that may be exploited.
In the second contribution we will present an alternative way for preforming the dropout algorithm, which has proven to be an effective method for reducing overfitting in deep artificial neural networks. Dropout refers to the way of training neural networks by randomly dropping neurons in order to avoid the co-adaptation of feature detectors. The dropout can be viewed as form of ensemble learning.
Ensemble Learning
Ensemble learning methods proved to be amongst the most accurate supervised machine learning techniques. According to the winning solutions in prestigious competitions such as Kaggle , the state-of-the-art ensemble methods are Random Forest Forests and Gradient Boosting trees. Random Forest, originally introduced by Leo Breiman, proved to be amongst the most accurate ensembles. We compared more than 179 different classifiers using 121 datasets from the UCI repository in a recent large-scale experiment. Random Forest (RF) came top, followed by SVM with Gaussian Kernel.
In this presentation we will discuss two main contributions in relation to Random Forest. In the first part we will show how a Random Forest with 5 trees (classifiers) outperforms or at least perform as good as an RF with 500 tree. We will discuss our method, which resulted in extreme pruning to the Ensemble without compromising the performance of the original RF. In the second contribution, we will show how re-engineering a dataset by means of optimized clustering algorithm contributes to improvement of the Random Forest performance across Medical and Life Science Datasets.
Short Bio – Chrisina Jayne
Chrisina received her Ph.D. degree in applied mathematics from Sofia University, Sofia, Bulgaria in 1998. She is currently head of the School of Computing Science and Digital Media at Robert Gordon University. Her research includes developing and applying neural network methods to a number of applications. Professor Jayne is a Fellow of the British Computer Society and Senior Fellow of the UK Higher Education Academy. She is the coordinator of the International Neural Network Society's Special Interest Group on Engineering Applications of Neural Networks. She chaired the Engineering Applications of Neural Network Conference in 2012, 2014, and 2016. Chrisina is the program chair of the 2017 International Joint Conference on Neural Networks (IJCNN 2017) and a member of the Board of Governors of the International Neural Network Society.
Short Bio – Eyad Elyan
Eyad obtained his PhD from the University of Bradford in 2008. He is currently a Senior Lecturer at the School of Computing Science and Digital Media, Robert Gordon University (RGU) and the course leader for the MSc Data Science. His main research activities are focused on machine learning, machine vision and data analytics. He attracted funds to support his research from different resources including Innovate UK, the Data Lab Innovation Centre and Oil and Gas Innovation Centre (OGIC), Historic Environment Scotland and others. He is leading several projects with industrial partners across different domains including Construction, Sports Science, Oil and Gas and Health. Eyad is a Fellow member of the British Higher Education Academy. He served as a Program Committee member for several international conferences and as a reviewer for several international.
Speaker: Dr. Jayne and Dr. Elyan
Venue: Meston 311
Date: 29th March
Time: 14:00 to 15:00
- Speaker
- Dr. Chrisina Jayne and Dr. Eyad Elyan
- Venue
- Meston 311