Note 1
Building Language Models
From the third note, we discussed foundations and intuitions on language models. In our previous note, we discussed about in-depth about neural networks. Before going into different topics including AI safety, I wanted to build language models because there are a lot of concepts to discuss there. First, using our knowledge, let’s build a bigram. Then, let’s build a more sophisticated language model that actually sounds like a chatbot with the introduction of transformers.