Glossary

A nonlinear function applied to a neuron's weighted sum that lets neural networks learn complex patterns. Common examples include ReLU, sigmoid, and tanh.

Deliberate, often small, perturbations to input data designed to fool a machine-learning model into making incorrect predictions. They expose vulnerabilities in models and are a key concern for AI robustness and security.

Functions of the form f(x) = Wx + b that combine a linear transformation with a translation. They are the mathematical building blocks of linear layers in neural networks.

An AI system that can perceive its environment, reason, and take actions autonomously to achieve a goal. Modern agents often combine language models with tools and memory.

The field of ensuring an AI system's goals and behavior reliably match human values and intentions. It is central to building safe and beneficial advanced AI.

The discipline of preventing AI systems from causing harm, whether through accidents, misuse, or unintended behavior. It spans technical robustness, alignment, and governance.

The study and practice of designing algorithms that avoid unjust discrimination across groups defined by attributes like race, gender, or age.

The field of computer science devoted to building machines that can perform tasks requiring human-like intelligence, such as reasoning, perception, and learning.

Neural networks that learn to compress input data into a compact representation and reconstruct it, useful for dimensionality reduction, denoising, and anomaly detection.

The core algorithm for training neural networks that computes gradients of the loss with respect to weights via the chain rule and propagates them backward.

A technique that normalizes the inputs of each layer within a mini-batch to stabilize and accelerate neural-network training.

A transformer-based language model that reads text in both directions to build deep contextual word representations, revolutionizing NLP tasks.

Methods applied to data, models, or decisions to reduce unfair bias and improve fairness across demographic groups.

The balance between a model's tendency to oversimplify (bias) and its sensitivity to noise in training data (variance); good models minimize total error from both.

A family of 1-bit large language models (such as BitNet b1.58) that use ternary or binary weights to drastically cut memory and compute while preserving accuracy.

A prompting technique that encourages a model to generate intermediate reasoning steps, improving performance on complex math and logic problems.

Saved snapshots of a model's weights and training state at a point in time, allowing training to resume or the model to be reused and deployed.

The study and development of AI systems that exhibit behavior deemed creative in domains such as art, music, literature, and design.

Approaches, such as sparse models and quantization, that achieve strong performance while reducing the computational cost of training and inference.

An approximation algorithm used to train energy-based models like Restricted Boltzmann Machines by comparing data and generated samples.

A self-supervised method that learns representations by pulling similar examples together and pushing dissimilar ones apart.

Neural networks using convolution operations to efficiently process grid-structured data like images, excelling at vision tasks.

Techniques that artificially expand a training set by transforming inputs (e.g., cropping, rotating, or paraphrasing) to improve generalization.

A branch of machine learning using multi-layer neural networks to learn hierarchical representations from large amounts of data.

Generative models that learn to reverse a gradual noising process, producing high-quality images, audio, and other data from random noise.

A transfer-learning technique that adapts a model trained on one distribution (source) to perform well on a related but different distribution (target).

Small, often battery-powered computing devices (phones, sensors, IoT hardware) that run AI models locally rather than in the cloud.

A list of numbers representing a piece of text's meaning, so similar texts sit close together in vector space.

The geometric space in which embeddings live, where proximity reflects similarity in meaning or function.

The practice of designing, deploying, and governing AI in ways that respect human rights, fairness, accountability, and societal well-being.

The degree to which a human can understand the reasons a model produced a particular output.

Techniques and tools that make AI decisions transparent and interpretable to users, regulators, and developers.

The process of selecting, transforming, or creating input variables to improve a model's predictive performance.

AI systems that coordinate learning across many decentralized devices or organizations while keeping raw data local.

A training approach where models are updated on-device and only the updates are shared centrally, preserving data privacy.

A network architecture where information flows in one direction from input to output with no cycles or feedback loops.

A model's ability to learn a new task from only a handful of examples, often by leveraging prior knowledge or in-context examples.

Providing a language model with a few illustrative examples inside the prompt so it can infer the desired task without weight updates.

Further training a pretrained model on a narrow dataset so it adopts a style or specialty.

A framework where a generator and discriminator compete, enabling the synthesis of realistic images, audio, and other data.

AI systems that create new content such as text, images, audio, or video, typically by learning the underlying distribution of training data.

A family of decoder-only transformer models pre-trained on vast text corpora and fine-tuned for language generation and instruction following.

An optimization method that iteratively adjusts parameters in the direction opposite their gradient to minimize a loss function.

Confidently wrong output — the model fills gaps with plausible text instead of saying "I don't know."

A configuration setting chosen before training (such as learning rate or batch size) that controls how a model learns.

Training a smaller "student" model to mimic a larger "teacher" model, transferring capability while reducing size and cost.

The compressed, abstract representation space a model learns, where each point corresponds to a possible input or concept.

A parameter-efficient fine-tuning method that injects small trainable rank-decomposition matrices, avoiding updates to all original weights.

A function that quantifies how far a model's predictions are from the true targets, guiding the learning process.

A type of recurrent neural network with gating mechanisms that captures long-range dependencies in sequential data.

A subset of AI where systems improve at tasks by learning patterns from data rather than through explicit programming.

Learning algorithms that improve their own learning process, sometimes called "learning to learn," across many tasks.

The learned function that turns input (text, image) into output. Training fixes its weights; inference just runs it.

Techniques such as pruning, quantization, and distillation that shrink models for faster, cheaper deployment.

A form of knowledge distillation in which a compact model is trained to reproduce the behavior of a larger, more capable model.

AI systems that process and relate information across multiple modalities such as text, images, audio, and video.

An NLP task that identifies and classifies entities such as names, organizations, and locations within text.

The field of enabling computers to understand, generate, and manipulate human language.

Computational models inspired by the brain, composed of layers of interconnected nodes that learn mappings from inputs to outputs.

The algorithm (such as Adam or SGD) that updates model weights using gradients to reduce the loss during training.

When a model learns training-data noise rather than the underlying pattern, performing well on training data but poorly on new data.

The simplest type of neural unit that computes a weighted sum and applies a step function, foundational to neural-network theory.

Converting a trained model's weights to lower-precision formats after training to reduce size and speed up inference.

A classification metric measuring the share of positive predictions that were actually correct (true positives divided by all positive predictions).

The craft of designing input prompts to reliably steer a language model toward desired outputs.

An attack that embeds malicious instructions inside text a model processes, hijacking its behavior or bypassing safeguards.

A parameter-efficient method that prepends trainable soft-prompt vectors to a frozen model instead of retraining its weights.

Removing unimportant weights or neurons from a model to make it smaller and faster with minimal accuracy loss.

Representing model weights and activations with fewer bits (e.g., 8-bit integers) to reduce memory and computation.

Retrieval-Augmented Generation: the model fetches relevant documents, then answers from them — reducing hallucination.

A technique that grounds a language model's answers in retrieved external documents, improving factual accuracy and citation.

Neural networks with loops that maintain a hidden state, suited to sequential data like text and time series.

Methods such as dropout and weight decay that constrain a model to reduce overfitting and improve generalization.

A learning paradigm where an agent improves by receiving rewards or penalties from interacting with an environment.

Fine-tuning models using human preference signals to make their outputs more helpful, harmless, and aligned.

Learning from unlabeled data by generating supervisory signals from the data itself, such as predicting masked parts.

Assigning a class label to every pixel in an image so that objects are understood at the pixel level.

Training with a small amount of labeled data combined with a large amount of unlabeled data to improve performance.

Representing data or models with mostly zero values, improving efficiency and interpretability.

A trick that passes gradients unchanged through non-differentiable operations like rounding, enabling quantization-aware training.

Artificially generated data used to train or test models, especially when real data is scarce, sensitive, or costly.

Audio, images, video, or text generated or manipulated by AI, including deepfakes and other generative content.

A chunk of text (roughly 4 characters in English) that models read and write. Pricing and context limits are counted in tokens.

Splitting raw text into tokens (words, subwords, or characters) that a model can process numerically.

The process of adjusting a model's parameters using data and an optimization algorithm to minimize a loss function.

Reusing a model trained on one task as the starting point for a related task, speeding up and improving learning.

Models built on the transformer architecture, which uses self-attention to weigh relationships across an entire sequence.

A neural architecture introduced in 2017 that relies on self-attention, now the backbone of most modern language and vision models.

When a model is too simple to capture the underlying pattern, performing poorly even on training data.

Learning from unlabeled data to discover structure, such as clusters or low-dimensional representations.

A casual, AI-assisted coding style where a developer describes intent to a language model and accepts its generated code largely by feel, with minimal manual review.

A model's ability to perform a task or recognize a class it was never explicitly trained on, using general knowledge and instructions.