Recurrent Neural Networks(RNN) - What is it? | Intellipaat

They are frequently employed for ordinal or temporal issues, such as translating, natural language processing (nlp), voice recognition, and image captioning. Recurrent neural networks (RNNs) use training data to learn, just like backpropagation and convolutional neural networks (CNNs) do. They stand out due to their "memory," which allows them to affect the current output and input by using data from previous inputs. Recurrent neural networks' outputs are dependent on the previous parts in the sequence, unlike typical deep neural networks, which presume that inputs and results are independent of one another. Uni - directional recurrent neural networks are unable to take into account future events in their forecasts, despite the fact that they would be useful in deciding the output of a particular sequence.

Let's use an expression that is frequently used to describe someone who is ill—"feeling under the weather"—to help us understand RNNs. The idiom must be stated in that particular order for it to make sense. Recurrent networks must therefore take into account the order in which each word appears in the idiom. They then utilise this knowledge to predict the following word in the series.

The "rolled" graphic of the RNN in the visual below depicts the complete neural network, instead of the entire predicted sentence, such as "feeling under the weather." The constituent levels, or time steps, of a neural network are represented by the "unrolled" image. Every layer corresponds to a single term in that sentence, like "weather." In order to forecast the output inside the sequence, "the," the previous inputs "feeling" and "under" would be presented as a hidden layer in the third timestep.

Recurrent networks are distinguished by the fact that each layer of the network uses the same parameters. Recurrent neural networks share the very same weight parameter inside each layer of the network, in contrast to feedforward networks, which have distinct weights across each node. However, to support reinforcement learning, these weights are still modified using the techniques of backpropagation and gradient descent.

Recurrent neural networks use the backpropagation through time (BPTT) algorithm, which differs slightly from conventional backpropagation because it is tailored to sequence data, to find the gradients. In classical backpropagation, the model trains itself through computing mistakes out of its output layer towards its input layer. This is how BPTT works. These computations enable us to accurately alter and fit the model's parameters. In contrast to feedforward networks that do not communicate parameters between layers, BPTT adds mistakes at each time step, which is how it varies from the conventional technique.

RNNs frequently experience the two issues of "exploding gradients" and "vanishing gradients" throughout this process. The gradient's size, or the slope of an error function all along the error curve, is what categorises these problems. When the gradient is insufficient, it keeps becoming smaller, updating the feature weights until they become negligible, or 0. The algorithm stops learning when something happens. Whenever the gradient is just too great, exploding gradients happen, which makes the model unstable. In this scenario, the model weights would eventually become too enormous and be rendered as NaN. Reducing the number of hidden layers in the neural network and hence some of the intricacy in the RNN model is one way to address these problems.