Day 34 encoder only transformers (bert) & decoder only transformers (chatgpt)

Check out our live web application for this program - https://newdaynewlearning.netlify.app/

[!NOTE] There is a game waiting for you today, the best/first answer can win an exciting gift🎁

• Click on the link : https://www.nytimes.com/games/wordle/index.html
• You will get a 6 chance to guess the 5-letter word correctly

More about me:

I am just a Colleague of your’ s, Learning and exploring how Math, Business, and Technology can help us to make better decisions in the field of data science.

• Check out my Second brain:  https://medium.com/@ravikumar10593/
• Check out my Portfolio : Link

Topic : Encoder only transformers (BERT) & Decoder only transformers (ChatGPT)

Encoder only transformers (BERT)

• Word Embedding : Converts the words into vectors (numbers)
• Positional Encoding : Helps to keep track of word order
• Self-Attention : Helps establish relationship among words
• Combining all the 3 above we get context Aware embeddings that helps cluster similar sentences and documents

Decoder only transformers (ChatGPT)

• Word Embedding : Converts the words into vectors (numbers)
• Positional Encoding : Helps to keep track of word order
• Masked Self-Attention : Helps establish relationship among words and the prompts

Article Source :

• BERT Model - NLP
• Encoder-Only Transformers (like BERT) for RAG, Clearly Explained!!!
• Decoder-Only Transformers, ChatGPTs specific Transformer, Clearly Explained!!!

TL;DR : In the world of transformer models, the “encoder” and “decoder” represent two distinct architectural approaches, primarily used in sequence-to-sequence tasks like machine translation. Encoders process an input sequence and compress it into a context vector, while decoders use this context vector to generate an output sequence. The key difference lies in how they handle input and output sequences and the nature of their attention mechanisms.