What is Deep Learning?

Diving into Machine Learning

At first glance, Deep Learning appears to be quite a convoluted part of the AI field. It’s typically defined as a field inside of Machine Learning, which is essentially the study and practice of using algorithms to help a computer learn from data that it takes in. Keep in mind that this includes AI systems. In fact, AI is the field in which Machine Learning is mainly used.

Since Machine Learning arose from the development of algorithms that have been helping machines begin to learn and think like humans, one might ask: what is the difference between this and Deep Learning? Is Deep Learning simply a form of Machine Learning that just goes into more detail? Andrew NG, a leader in the field of Machine Learning as well as its subfield of Deep Learning has stated that deep learning is basically an umbrella term for the research behind improving machine learning as a whole. In addition, he’s also said that a large part of Deep Learning involves making the algorithms in the Machine Learning field clearer and easier to use.

Therefore, it seems that a good definition for deep learning would be: the field that seeks to significantly improve how we look at and use Machine Learning.

Deep Learning and Artificial Neural Networks

Given this definition, how exactly does deep learning relate to the development of and the usage of Artificial Neural networks? This answer, in its entirety, would call for a lengthy series of posts. For now, we’ll stick to the basic advantage of deep learning that comes from creating deep neural networks.

Before jumping into this further, it’s important that you understand the difference between deep and shallow neural networks. First of all, according to jungleML, a blog that specializes in Machine Learning and related topics, both types of neural networks can consistently “approximate any function.” Therefore, we take this to be true, then we also need to keep in mind the possibility that neither type of neural network would have constraints related to the basics of the math behind it.

Shallow neural networks first differ from deep neural networks in that they have only one hidden layer of nodes to help process data as well as only one set of weights to help run the function that drives this processing. It’s consequently reasonable to assume at this point that due to such constraints, shallow neural networks could be much slower and much more prone to error than deep neural networks. Another problem that can easily arise from the limitation of input data is that the system will then lack enough of an output data range to make informed decisions. Just like our brains, an AI system needs to quickly process a wide range of information to make the ideal decision in a specific situation.

Judging by all of this, as jungleML states, why not then make the most layered neural network that you can, to take in the most data that you can, at once? In addition to this, you could have said network run multiple algorithms at once and take in as much data as possible due to its many layers. The simple answer is, as they say in the business world, if you take in or input too much data in a short period of time, then you will have too much output data. If this is the case, then you’ll also have what is often called too much noise in the data. In clearer terms, this means that you’ll have a surplus of insignificant data. This insignificant data, just like in business analysis, takes away from the reliability of your AI system, in this case, related to decision making.

Cross-Validation and Better Data Analysis

It seems that those who work in the AI space have been wary in making networks too deep as this reportedly takes away from what is termed the cross-validation accuracy. Essentially, cross-validation is the process of applying a Monte-Carlo analysis to a prediction system in order to land on a percentage range that can be used to state how accurately said the system will perform, in the real world. It seems that those who work in the AI space have been wary in making networks too deep as this reportedly takes away from what is termed the cross-validation accuracy. Essentially, cross-validation is the process of applying a Monte-Carlo analysis to a prediction system in order to land on a percentage range that can be used to state how accurately said system will perform.

The math behind this gets quite complicated. We will delve into some of the specifics of this process in future pieces, but at this point, think about it all in this way: an AI system is given data that it has already processed and effectively, understood. It is then given one or more sets of unknown data and it analyzes these sets. While it does this, the AI team behind the system watches the process and adjusts the data sets accordingly to try to more accurately get the results that they want. For an even more specific example of this, think about an AI system working to identify certain animals, by their pictures. Perhaps it has already identified cats with a successful level of accuracy and the team then wants it to move on to dogs. To do this, cats might be the known data or what is often called “the training group,” and dogs would then be the unknown data. In the end, the team would use the findings to change the model to try to help it work more efficiently with the data.

Delving Further into Deep Learning

In our next piece, we’ll delve further into what deep learning can do, in an attempt to make all of this easier to understand. At the same time, we’ll look at AI and speech recognition to introduce advances that have recently been made in the space as well as roadblocks that still exist. As always, if you have questions or thoughts on future pieces, drop us a line.