Note 1
Foundations and Intuitions

This note covers basic concepts as well as intuitions through the introduction of Large Language Models.

1.1 Fundamental Terms and Concepts

This section is an introduction to basic terms and concepts that will be used throughout my notes. That being said, more advanced concepts will be introduced in latter notes. Please feel free to skip over this section if you already know the bold words.

The primary purpose of ML is to make AI “smarter”. For instance, calculators are “dumb” in a sense that it requires an input and can only do certain tasks such as outputting calculated input. However, we can make it “smarter” by teaching data. We can clearly see that WolframAlpha is smarter than a normal scientific calculator. (Please note that this is just an analogy not an example because WolframAlpha is a knowledge engine, not a machine learning model. But I mean you get the point.)

On the high level, you can think of an AI model as a giant function with billions of array of numbers, called parameters.

Indeed, models are prediction machines based on the trained parameters. One fun fact here is that it generally costs way less to run inference than to train the model. More in-depth explanation will be available in latter notes where we discuss neural nets.

1.1.1 ML Paradigms

In ML, there are three primary learning paradigms: supervised learning, unsupervised learning, and reinforcement learning. First, what does it mean to “learn” and “train”?

There are multiple ways in which models are trained. One of the most used method out of the three paradigms is supervised learning.

With supervised learning, there are two primary predictions that models make.

Classic examples of classification would be categorizing dog/cat pictures and email spam filtering. An example of regression would be the house price over years.

The second most common learning paradigm is unsupervised learning.

How is it possible for a model to learn without labeled data? One primary example algorithm is \(k\)-means clustering, where the model categorizes the data with relevance and similarity. To cluster high dimensional data, we often perform dimensionality reduction to reduce the dimension for clustering with the trust in manifold hypothesis. I won’t go to deep into unsupervised learning and \(k\)-means clustering in this note, but you can check out this awesome visualization of \(k\)-means clustering.

The third most used learning paradigm is reinforcement learning.

Personally, reinforcement learning is more complicated compared to the two paradigms introduced above. On the high level, an agent will take an action in an environment, where it returns rewards based on the action. The agent will then update its policies with the provided reward. The process is continued until the “adequate” amount is reached.

Training a model can be understood as writing a best fit mathematical function to training datasets that captures the holistic trend. There are few terms that describe the function.

On the high level, our goal is to generalize and capture the general pattern and prevent overfitting or underfitting. We normally would stop training a model when it does not appear to improve, or is degrading, but how do we assess it?

We have discussed the fundamental terms and concepts that would repeatedly appear in the following notes. Now let’s build our high level intuitions on ML with the introduction to LLMs!