RNNs have **difficulty handling long sequences of text.** They use a type of memory called a **"hidden state"** which can only retain information for **a limited amount of time.** Enter [[Transformers]]—saving the day.