BizPlatform → AI Glossary: Key Terms Explained

A

Algorithm: A defined sequence of steps a computer follows to complete a task. Algorithms are not unique to AI; they are the basis of all computing. In machine learning, algorithms describe how a model adjusts its internal values during training.
Attention Mechanism: A component of transformer models that allows the system to weigh the relevance of different parts of an input when generating a response. Rather than processing words in sequence, attention mechanisms can consider relationships across an entire input at once.
Artificial General Intelligence (AGI): A hypothetical form of AI that could perform any intellectual task a human can, without being specifically trained for each one. No system with these properties exists today. The term is used frequently in discussions about the long-term direction of AI research.
Autoregressive Model: A model that generates output one unit at a time, using each preceding unit to inform the next. Language models that produce text word-by-word are typically autoregressive. The approach is well-suited for sequential data like text and audio.

B

Backpropagation: The process by which a neural network adjusts its internal weights after making a prediction. The model calculates how far off its prediction was, then works backward through its layers to update each weight proportionally. This process repeats across many examples during training.
Bias (in AI models): Systematic patterns in a model's outputs that reflect skewed or unrepresentative training data. Bias is distinct from statistical bias used in mathematical contexts. In public discourse, the term often refers to outputs that disadvantage or misrepresent particular groups of people.
Benchmark: A standardized test used to evaluate and compare AI model performance. Benchmarks measure specific capabilities such as language understanding, reasoning, or image recognition. Performance on benchmarks is widely reported but does not always reflect real-world usefulness.

C

Classification: A machine learning task in which a model assigns inputs to one of a fixed set of categories. Spam detection, image labeling, and medical diagnosis tools are all classification systems. The model learns which features are associated with each category during training.
Convolutional Neural Network (CNN): A type of neural network designed for processing grid-structured data, particularly images. CNNs apply filters across sections of an image to detect patterns such as edges, textures, and shapes. They became the basis of modern image recognition systems during the 2010s.
Context Window: The amount of text a language model can consider at one time when generating a response. Information outside the context window is not available to the model during that interaction. Context window size varies across different models and affects their ability to handle long documents.

D

Deep Learning: A branch of machine learning that uses neural networks with many layers to model complex patterns in data. The term "deep" refers to the number of layers in the network. Deep learning underlies most current AI capabilities in image recognition, language processing, and audio generation.
Dataset: A structured collection of data used to train, validate, or test a machine learning model. The composition and quality of a dataset significantly affects what a model learns. Datasets can include text, images, audio, structured records, or any combination of data types.

E

Embedding: A numerical representation of a word, phrase, or concept that captures its meaning in relation to other concepts. Embeddings allow AI models to work with language mathematically. Words with similar meanings tend to have similar embeddings, which is how models recognize semantic relationships.
Epoch: One complete pass through an entire training dataset during model training. Models typically train over many epochs, adjusting their weights with each pass. Too few epochs can leave a model undertrained; too many can cause it to memorize training data rather than learn general patterns.

F

Fine-Tuning: The process of continuing to train a pre-trained model on a smaller, more specific dataset to adapt its behavior for a particular task or domain. Fine-tuning preserves the general knowledge acquired during initial training while adjusting the model for a narrower application.
Foundation Model: A large model trained on broad data that can be adapted for a wide range of tasks. The term was coined in 2021 to describe models like GPT and BERT that serve as starting points for more specific applications. Foundation models represent a shift from training separate models for each task.

G

Generative AI: AI systems that produce new content rather than simply classifying or predicting. Generative models can create text, images, audio, video, and code. They learn the patterns and structure of their training data well enough to produce novel outputs that resemble it.
GPU (Graphics Processing Unit): A processor originally designed for rendering graphics that became central to AI training due to its ability to perform many calculations simultaneously. Training large neural networks requires enormous computational resources, and GPUs made this practically feasible at scale during the 2010s.
Gradient Descent: An optimization method used to train machine learning models by iteratively adjusting parameters to minimize prediction errors. The "gradient" indicates the direction of steepest increase in error; descending means moving in the opposite direction to reduce it. Most neural network training relies on variants of this approach.

H

Hallucination: The production of factually incorrect or fabricated information by an AI model, presented with the same confidence as accurate information. Hallucination is a known limitation of current language models, resulting from the way they generate text based on patterns rather than verified facts.
Hyperparameter: Settings that control how a model is trained, as distinct from the parameters the model learns during training. Learning rate, batch size, and number of layers are examples. Hyperparameters are set before training begins and influence the speed and quality of learning.

I

Inference: The process of using a trained model to generate outputs from new inputs. Inference is what happens when you interact with an AI system; training is what happened before it was made available. Inference requires significantly less computation than training.

L

Large Language Model (LLM): A type of neural network trained on large quantities of text to understand and generate human language. LLMs are trained to predict the next token in a sequence, a task that requires learning grammar, facts, reasoning patterns, and style. Most current conversational AI systems are built on LLMs.
Loss Function: A mathematical measure of how far a model's predictions are from the correct answers during training. The loss function quantifies the error, which the training process then works to minimize. Different tasks use different loss functions suited to the type of prediction being made.

M

Machine Learning: A field of computer science in which systems learn from data rather than following explicitly programmed rules. Instead of a programmer specifying every decision, a machine learning model develops its own internal rules by identifying patterns across many examples.
Multimodal: Refers to AI systems that can process and generate more than one type of data, such as both text and images. A multimodal model might accept a photograph as input and produce a text description, or accept a text prompt and generate an image.

N

Neural Network: A computational system loosely inspired by the structure of biological brains, composed of interconnected nodes organized in layers. Information passes through the network, with each layer transforming it in some way. The connections between nodes have numerical values called weights that change during training.
Natural Language Processing (NLP): The field of AI concerned with enabling computers to understand, interpret, and generate human language. NLP encompasses tasks ranging from spell-checking and translation to summarization and question answering. Most language AI applications fall under this umbrella.

O

Overfitting: A condition in which a model performs well on training data but poorly on new, unseen data. Overfitting occurs when a model learns the specific details of its training examples rather than the general patterns they represent. It is one of the central challenges of machine learning practice.

P

Parameter: A numerical value within a model that is learned during training and shapes how the model processes inputs. Large language models are described by their parameter count, which can reach into the hundreds of billions. More parameters allow a model to represent more complex patterns, though size alone does not determine capability.
Prompt: The input provided to an AI model to elicit a response. In language model contexts, a prompt is typically text. The wording and structure of a prompt can significantly influence the quality and nature of the model's output.
Pre-Training: The initial training phase in which a model is trained on a large, general dataset before being adapted for specific tasks. Pre-training builds broad capabilities that can be refined through fine-tuning. Foundation models are defined by their pre-training on large, diverse data.

R

Reinforcement Learning: A learning approach in which a model learns by interacting with an environment and receiving feedback in the form of rewards or penalties. The model adjusts its behavior to maximize cumulative reward over time. Reinforcement learning is used in game-playing AI and has been applied to language model training.
RLHF (Reinforcement Learning from Human Feedback): A training technique in which human evaluators rate model outputs, and those ratings are used to train a reward model that guides further learning. RLHF has been widely used to align language model behavior with human preferences and reduce harmful outputs.

S

Supervised Learning: A machine learning approach in which a model is trained on labeled data, meaning each input is paired with a known correct output. The model learns to associate inputs with outputs by minimizing prediction errors across the training set. Most practical machine learning applications use supervised learning.

T

Token: The basic unit of text that language models process. Tokens are not always whole words; they can be word segments, punctuation, or spaces. A sentence of 20 words might contain 25 to 30 tokens depending on the model's tokenization approach. Models have limits on how many tokens they can process at once.
Transformer: A neural network architecture introduced in 2017 that uses attention mechanisms to process sequential data without relying on recurrence. Transformers process entire sequences simultaneously rather than step-by-step, enabling more efficient training at scale. The transformer is the architecture underlying most current large language models.
Training: The process of exposing a model to data and adjusting its parameters to improve its predictions. Training requires significant computational resources and can take days or weeks for large models. A trained model is a snapshot of the parameter values arrived at after training is complete.

U

Unsupervised Learning: A machine learning approach in which a model is trained on data without labeled outputs. The model identifies patterns, groupings, or structures within the data on its own. Clustering, dimensionality reduction, and some generative models use unsupervised learning approaches.

W

Weight: A numerical value assigned to the connection between two nodes in a neural network. Weights determine how much influence one node has on another. During training, weights are adjusted through backpropagation to reduce prediction errors. A trained model's weights encode everything it has learned.

The AI Glossary

A

B

C

D

E

F

G

H

I

L

M

N

O

P

R

S

T

U

W

Missing a term?