CNN vs. RNN vs. ANN – Analyzing 3 Types of Neural Networks in Deep Learning

The different types of neural networks in deep learning, such as convolutional neural networks (CNN), recurrent neural networks (RNN), artificial neural networks (ANN), etc. are changing the way we interact with the world. These different types of neural networks are at the core of the deep learning revolution, powering applications like unmanned aerial vehicles, self-driving cars, speech recognition, etc.

It’s natural to wonder – can’t machine learning algorithms do the same? Well, here are two key reasons why researchers and experts tend to prefer Deep Learning over Machine Learning:

  • Decision Boundary

  • Feature Engineering

Curious? Good – let me explain.

1. Machine Learning vs. Deep Learning: Decision Boundary

Every Machine Learning algorithm learns the mapping from an input to output. In case of parametric models, the algorithm learns a function with a few sets of weights:

Input -> f(w1,w2…..wn) -> Output

In the case of classification problems, the algorithm learns the function that separates 2 classes – this is known as a Decision boundary. A decision boundary helps us in determining whether a given data point belongs to a positive class or a negative class.

For example, in the case of logistic regression, the learning function is a Sigmoid function that tries to separate the 2 classes:

Decision boundary of logistic regression

As you can see here, the logistic regression algorithm learns the linear decision boundary. It cannot learn decision boundaries for nonlinear data like this one: