Model Monster's AI Glossary

A general term for permitted use of a service. In many settings it refers to an Acceptable Use Policy (AUP) or similar contractual restrictions.

The proportion of predictions a model gets correct. Accuracy is context-dependent and often misleading without additional metrics: a model that predicts "no fraud" for every transaction achieves 99% accuracy if fraud occurs in only 1% of cases. Marketing claims citing accuracy should specify the dataset, task, and conditions; high accuracy on benchmarks may mask poor performance on important subgroups.

A machine learning approach where the model identifies which unlabeled examples would be most valuable to label, reducing annotation costs. Active learning strategies may involve human labelers viewing sensitive content, raising labor and content moderation considerations.

A small set of parameters added to a frozen base model to customize behavior without full fine-tuning; includes techniques like LoRA. Adapters are often the deliverable in custom AI projects, so ownership, portability, and confidentiality terms are often stated explicitly. Frequently used as a way to provide organization-specific capabilities without creating full separate base models.

An attempt to cause a model to produce incorrect or harmful outputs through crafted inputs. Adversarial robustness is relevant to security representations, product liability, possibly-infringing intellectual property outputs, and contractual performance standards. This can be an attack against the AI system as a whole or even just a prompt designed to elicit an unwanted result.

A data instance purposefully perturbed to induce misclassification by a deployed model. Relevant to security testing, autonomous vehicle litigation, and content moderation failures.

An AI system failure due to adversarial attack, where unwanted model output results in adverse effects like leakage of privileged data, violation of guardrails, expansion of privilege, or unwanted output. This defines the failure condition for many types of testing and is relevant to breach notification and incident response obligations.

Software infrastructure for building and deploying AI agents, often including tool integration, memory, and orchestration capabilities. Framework choice affects portability, vendor lock-in, and security posture; open source frameworks have different risk profiles than proprietary ones.

A security framework developed by Meta stating that AI agents should satisfy no more than two of the following three properties within a session: (A) processing untrustworthy inputs, (B) accessing sensitive systems or private data, and (C) changing state or communicating externally. Building on Simon Willison's Lethal Trifecta, the Rule of Two extends protection beyond data exfiltration to cover any state-changing action an agent might take, including examples like issuing refunds, modifying files, sending messages, or executing code. If a task requires all three properties, the agent should not operate autonomously and must include human-in-the-loop approval or equivalent supervision. The Rule of Two reflects the current consensus that prompt injection cannot be reliably detected or filtered, making architectural constraints the most practical defense for agentic AI systems.

Research and practices aimed at reducing harmful or unintended behavior of AI systems. In enterprise, procurement, and policy contexts, “AI safety” may refer to model training and post-training methods (e.g., RLHF), system-level guardrails, abuse monitoring, and evaluation programs tied to a stated threat model.

The process of labeling data for training or evaluation; also called "tagging" or "labeling." Annotation involves human labor (often outsourced), content exposure, and quality control issues; IP questions arise for labeled datasets.

A retrieval-augmented generation (RAG) evaluation measure assessing how well an answer addresses the user’s question, often compared to a reference or judged by humans/LLMs. It is frequently used alongside context relevance and groundedness.

The high-level structure of a model or AI system and how components interact (e.g., model + retrieval + tools + guardrails + monitoring). Architecture choices influence performance, security, privacy, and auditability. One way to identify the parts of an architecture is by using the mnemonic CORE - Components, Operations, Resources, and Execution.

A non-standard term used to describe hypothetical AI with broad, human-level capability across many domains. The term is used inconsistently in technical and marketing contexts; many current systems are better described as foundation models or general-purpose AI rather than “general intelligence.” In diligence and policy discussions, “AGI” sometimes signals discussion of frontier capability thresholds and risk controls.

A record of system events that supports tracing actions, changes, and access (e.g., who accessed data, what tools were called, what model version ran). Audit logs are commonly used for security investigations, compliance, and dispute resolution.

Verification of user or system identity before granting access to AI services or data. Authentication controls are baseline security requirements; failures can create breach liability and confidentiality exposure.

The process of determining what an authenticated user, service, or agent is allowed to do (permissions), often implemented via roles, policies, or scopes. Authorization differs from authentication (identity verification).

A hidden behavior in a model or system that activates under specific triggers (e.g., particular phrases, patterns, or inputs), causing unintended outputs or actions. Backdoors are discussed in model security, supply chain risk, and red teaming contexts.

The algorithm used to train neural networks by computing how much each weight contributed to prediction errors, then adjusting weights to reduce those errors. Backpropagation is how models "learn" from training data; errors propagate backward through the network, and weights are updated accordingly.

The number of training examples processed together before updating model weights; a hyperparameter affecting training dynamics. Affects training resource requirements and costs.

A system whose internal decision process is difficult to interpret or explain in a human-understandable way. The term is used in technical, governance, and legal contexts when evaluating transparency, accountability, and auditability.

The tasks and performance characteristics a model or system can reliably support (e.g., summarization, coding, extraction, tool use). “Capabilities” is often tied to evaluation results and can be discussed in procurement, marketing, and policy settings.

A saved state of model weights during training or fine-tuning, used to resume training or to preserve intermediate versions. In training agreements, checkpoints may be deliverables; agreements often specify ownership, retention, access controls, and permitted reuse.

Categories of weapons or hazardous materials; AI systems' potential to assist with CBRN threats is a key safety concern. CBRN-related content is typically prohibited in AUPs; frontier AI safety evaluations specifically test for CBRN assistance capabilities.

Dividing documents into smaller segments for processing within context window limits or for retrieval purposes. Chunking strategies affect retrieval accuracy and completeness; relevant when assessing whether AI systems properly considered full documents.

A reference to a source used to support a statement or output (e.g., a retrieved document chunk in RAG, or a legal citation to authority). In AI systems, “citations” may be generated automatically and can be incorrect or incomplete unless the system is designed to capture provenance.

The task of assigning inputs to predefined categories (e.g., spam detection, sentiment analysis, content moderation). Classification errors have different consequences depending on the application; false positives and negatives have different risk profiles.

Grouping similar items together based on their features without predefined labels. Clustering can produce de facto sensitive inferences (grouping by health, demographics) even without explicit attributes.

A technical standard for embedding provenance information in digital content, including AI-generated content. C2PA and similar standards support content authenticity verification; increasingly relevant for deepfake detection and evidence authentication.

In the CORE framework, the functional elements that comprise an AI system: models, adapters, guardrails, databases, APIs, connectors, human review steps, and other nodes through which data flows. Each component has properties relevant to governance: its provider or origin, the operations it performs, the resources it accesses, and how it transforms the data flowing through it. A list of components is the minimum information needed for an AIBOM.

Hardware-supported techniques protecting data while in use (in memory) using secure enclaves. Confidential computing may support stronger security representations for processing sensitive data in cloud environments.

Protection of non-public information from unauthorized disclosure. In AI contracting, confidentiality terms often address prompts, outputs, logs, and resources available to the AI system such as connected enterprise data.

An integration that pulls content from enterprise systems (e.g., SharePoint, Google Drive, Slack) into an AI system for retrieval or context. Connectors expand the data-access surface area; permissions, logging, and retention practices are commonly evaluated to reduce privilege, confidentiality, and privacy risk.

An alignment approach using a set of principles ("constitution") to guide model behavior, often using AI-generated feedback. Constitutional policies can be relevant to content and safety representations; request documentation for high-stakes use cases.

An adversarial technique overloading the prompt with excessive tokens to predispose models to a vulnerable state. A specific prompt injection variant; relevant to security testing and incident analysis.

A RAG quality metric measuring whether retrieved context contains information pertinent to the user's query. Poor context relevance can cause unreliable outputs; relevant when evaluating RAG system performance claims.

A learning paradigm where AI systems incrementally learn from new data while preserving prior knowledge (avoiding "catastrophic forgetting"). Continual learning systems may evolve in ways that affect prior representations about behavior; governance often addresses ongoing changes.

Technical or organizational measures used to achieve defined objectives (e.g., security controls, privacy controls, safety controls). In governance and audits, “controls” are often documented, tested, and monitored. In contrast to regular GRC systems, AI controls need to be implemented with technical measures, frequently as components external to the model.

A metric comparing embeddings by measuring the angle between vectors; higher similarity implies closer semantic meaning. Relevant when understanding RAG retrieval mechanics; similarity thresholds affect retrieval defensibility.

Techniques that expand or vary training data to improve model generalization (e.g., transformations, paraphrases, synthetic examples). Augmentation can affect performance and bias characteristics depending on how it is applied.

Changes over time in the distribution or characteristics of input data (or user behavior) that can degrade model performance. Data drift is often monitored alongside model drift (changes in the model itself).

Technologies and policies preventing unauthorized data transmission or exposure. DLP tools can help prevent sensitive data from being sent to AI systems; relevant for shadow AI governance.

An adversarial attack inserting or modifying training data to compromise model behavior. Data poisoning attacks can undermine model integrity; supply chain security and data provenance controls are relevant defenses.

Attacks designed to gain access to sensitive information in training data, including membership inference and model inversion. Privacy attacks demonstrate that training data can sometimes be extracted; relevant to privacy representations and security controls.

Documentation of data origin, collection methods, transformations, and chain of custody. Provenance is essential for IP compliance, bias assessment, and regulatory documentation; request provenance information for training data.

A privacy attack designed to reconstruct sensitive information from training data through model queries. Reconstruction attacks demonstrate privacy risks from training; relevant to data sensitivity assessments.

A facility housing computing infrastructure (servers, storage, networking) used for training and operating AI systems. Datacenter location and subcontracting can matter for data residency, security, and resilience.

Machine learning using neural networks with multiple layers, enabling detection of complex patterns in data. The term is often used interchangeably with "AI" in business contexts, though technically it refers to a specific architectural approach. The "deep" in deep learning refers to the number of layers between input and output, not to any quality of understanding.

Removal of data from systems and backups according to defined policies and technical processes. In privacy and AI contexts, deletion can relate to data subject rights, retention policies, and requests to remove training data or derived artifacts.

A system property where the same input always produces the same output. Traditional software is deterministic by design. AI systems can in theory be configured for deterministic behavior, though hardware and infrastructure variations may still introduce variability. Deterministic operation supports reproducibility, testing, audit, and regulatory compliance, but may reduce output quality or diversity compared to default settings.

The processes and standards used to build and maintain software and AI systems (e.g., secure coding, testing, review, documentation, incident response). These practices are often described in security questionnaires, audits, or contractual representations.

Practices combining software development and operations to improve deployment frequency, reliability, and monitoring (e.g., CI/CD, infrastructure as code). In AI settings, DevOps is often paired with MLOps for model lifecycle management.

Communication of information to a user, counterparty, regulator, or the public (e.g., about AI use, limitations, data practices, or incidents). Disclosure duties can arise from contracts, consumer protection rules, sector regulations, or internal governance.

A model that classifies inputs or distinguishes between categories rather than generating new content. Discriminative models (classifiers, detectors) have different risk profiles than generative models; errors are often binary.

A model's ability to perform equitably across the range of possible inputs, including rare or "long-tail" cases. Distributional robustness affects fairness and reliability; models may fail on underrepresented populations or edge cases.

Written materials describing a model, system, dataset, or process (e.g., model cards, system cards, technical specs, policies, runbooks). Documentation is commonly used for governance, audits, contracting, and incident response.

Running AI models on local devices rather than cloud servers. Edge deployment affects data residency, latency, security, and update mechanisms; may be preferred for privacy-sensitive applications.

Unexpected abilities that appear in larger models without explicit training for those tasks. Emergent capabilities complicate capability assessment and safety testing; models may have abilities not anticipated at launch.

The point at which a vendor or developer stops supporting a model, API, or product version (e.g., no updates, limited availability, retirement). Deprecation affects continuity, change control, and portability planning. Despite broad improvement in model capabilities, the deprecation of a model can change risks in a particular AI system for the worse.

One complete pass through the entire training dataset during model training. Affects training cost and model behavior.

A broad term describing efforts to develop and use AI in ways aligned with stated values (e.g., fairness, transparency, accountability, privacy, safety). The term is used in governance frameworks and policy statements and is not a single technical standard.

An adversarial attack crafting inputs to cause misclassification of malicious content as benign. Evasion attacks can bypass content moderation and security filters; relevant to security representations.

In the CORE framework, the dataflow connecting components from input to output, documenting how data travels through the system. Multiple execution paths may exist based on routing logic, conditional branches, or error handling. Documenting execution paths is needed for compliance with safety-by-design and privacy-by-design regulations as well as explainable AI.

Unauthorized extraction of sensitive data from a system, including via model outputs or tool calls. Exfiltration risk is central to confidentiality and privacy; address via least privilege, prompt injection defenses, and logging.

Techniques and methods designed to make AI model behavior more understandable to humans. XAI encompasses various approaches with different fidelity-complexity tradeoffs; claims often specify which methods are used and their limitations. Be careful not to confuse with X.ai, the AI provider associated with the microblogging platform X.

An adversarial attack seeking to extract training data, model weights, or system prompts through queries. Extraction attacks can expose confidential information and IP; relevant to security testing and rate limiting design.

A metric combining precision and recall, useful when classes are imbalanced. If a vendor promises "performance," clarify which metric matters (accuracy vs. F1 vs. recall) and on what dataset.

An instance incorrectly classified as negative when the true label is positive. False negative rates matter for safety-critical applications (missed fraud, missed medical conditions).

An instance incorrectly classified as positive when the true label is negative. False positive rates affect user experience and can create liability (wrongful denials, false accusations).

An individual measurable property of data used as input to a model (e.g., age, location, purchase history, medical codes). Feature choice affects model behavior; in regulated contexts, using protected characteristics (or close proxies) as features may trigger heightened review or restrictions depending on the use case and jurisdiction. In older models, features were frequently chosen by data scientists; in modern large models, features are usually discovered as part of the training process and may not be explicit.

The process of selecting, transforming, and creating features for model training. Feature engineering choices can introduce bias or encode protected characteristics indirectly.

A training approach where model updates happen on distributed devices without centralizing raw data. Federated learning may be used as a privacy-enhancing technique.

A measure of computational work; used to quantify training and inference costs. FLOP thresholds appear in regulatory definitions (e.g. the EU AI Act and AI Executive Orders); relevant to compute cost negotiations.

A model's ability to perform well on new, unseen data beyond its training set. Generalization relates to reliability; poor generalization can undermine product claims and cause failure on edge cases.

Content produced by a generative model (text, images, audio, video, code). “Generated content” is often used interchangeably with “Output,” though contracts may define these differently.

An optimization method that adjusts model weights in small steps toward lower prediction error. Think of it as rolling a ball downhill: the algorithm repeatedly moves weights in whatever direction reduces the loss function. Gradient descent determines both training speed and whether the model converges on useful patterns.

The best available correct labels or answers used to train or evaluate a system. Ground truth can be disputed in subjective domains; define labeling standards and dispute processes.

A RAG quality metric measuring whether outputs are supported by the provided context rather than fabricated. Groundedness is key to RAG reliability claims; ungrounded outputs may involve fabrication.

A training technique for reasoning models that optimizes based on relative performance within groups of responses. Introduced in late 2024 and used in many frontier models.

Retrieval that combines traditional keyword search with vector/semantic search to improve recall and precision. Hybrid search is common in enterprise RAG implementations.

AI creation of images from text prompts or other inputs. Image generation raises copyright questions regarding training data and outputs, deepfake and NCII concerns, and trademark issues when generated images depict recognizable brands or logos.

A model's ability to adapt behavior within a prompt using provided examples, without weight changes. Because in-context learning allows prompts and retrieved documents to materially change behavior, prompt content is often treated as part of the controlled system.

A security risk where AI outputs are used in downstream systems without adequate validation or escaping (e.g., injecting generated content into code, HTML, SQL, or commands). This can convert hallucinations or malicious outputs into system actions.

Fine-tuning a model to follow instructions and engage in dialogue. Instruction-tuned models (often called "chat" models) behave differently than base models, which is relevant when assessing capabilities and limitations.

The degree to which a human can understand how a model produces its outputs. Interpretability is stronger than explainability and implies genuine understanding of internal mechanisms; truly interpretable models are often less capable than black-box alternatives.

A family/name used for architectures combining elements of transformers with state space models (SSMs) to improve efficiency for long sequences. The term is used in technical discussions about model architecture and inference cost.

A setting constraining model outputs to valid JSON format. Structured output modes like JSON mode improve reliability and parsing for automated workflows, reducing integration errors and enabling programmatic processing of AI outputs.

Memory storing prior context during inference to enable efficient generation. KV cache size affects context window limits and inference cost, which is relevant when understanding capacity constraints and pricing models.

The process of moving know-how, documentation, and operational understanding from one team or vendor to another (e.g., during vendor transitions, M\&A, or outsourcing). In AI deployments, knowledge transfer may include model documentation, evals, runbooks, and data pipelines.

An LLM specifically trained for complex reasoning, often using extended chain-of-thought processing. Reasoning models like o1 and DeepSeek-R1 represent a capability shift and may be more reliable for complex analysis, though they have different cost and latency profiles.

The abstract multi-dimensional space encoding learned representations of data. Embeddings exist in latent space, and understanding this concept helps explain how AI systems represent and compare semantic meaning.

A security vulnerability pattern identified by Simon Willison occurring when an AI agent simultaneously possesses three capabilities: (1) access to private or sensitive data, (2) exposure to untrusted content, and (3) the ability to communicate externally. When all three capabilities are present, prompt injection attacks can cause the agent to access private data and transmit it to an attacker. The Lethal Trifecta has been demonstrated against major products including Microsoft 365 Copilot, ChatGPT, Google Gemini, Slack, and GitHub Copilot. Because prompt injection remains an unsolved problem, the primary defense is to ensure AI systems never combine all three capabilities simultaneously.

Whether multiple licenses (e.g., open source licenses, model licenses, and proprietary licenses) can be complied with simultaneously when components are combined or distributed. Incompatibilities can arise from obligations such as copyleft, attribution, field-of-use limits, or downstream restrictions.

A mathematical formula that measures how wrong the model's predictions are. The choice of loss function determines what the model optimizes for; a model trained to minimize one type of error may perform poorly by other measures. This is relevant when evaluating whether a model was designed appropriately for its intended use: a model optimizing for average accuracy may systematically fail on minority cases.

A subset of AI where systems learn patterns from data rather than following explicit rules. ML is the technical foundation of modern AI, and understanding that models learn statistical patterns (rather than "knowing" facts) helps assess capabilities and limitations.

A benchmark testing model knowledge across many academic subjects. MMLU scores are commonly cited in model marketing, but benchmarks don't guarantee real-world performance and may not reflect capabilities on domain-specific tasks.

Stored state used across interactions (e.g., conversation history, user preferences, task state). Memory can be ephemeral (within a context window) or persistent (stored and retrieved later), and it can raise retention, privacy, and confidentiality considerations.

Data describing other data, including creation dates, sources, and processing history. AI-generated content may lack authentic metadata or have synthetic metadata, which is relevant to evidence authentication and content provenance disputes.

Practices for deploying and maintaining ML systems in production, including monitoring, versioning, and updates. MLOps maturity affects reliability, reproducibility, and change control; assess vendor MLOps practices as part of due diligence.

The range and level of tasks a model can perform reliably, often evidenced through evaluations, benchmarks, and red teaming results. Capability statements can affect intended-use scoping and governance classification.

Structured documentation describing a model's intended use, performance characteristics, and limitations. Model cards support due diligence and can undermine marketing claims if disclosed limitations conflict with vendor representations.

Techniques reducing model size while maintaining performance, including quantization and distillation. Compressed models may behave differently than originals, and compression is a form of modification that may require license analysis.

An open source standard protocol for connecting AI models to external data sources and tools. MCP addresses the execution layer: how an agent calls tools and retrieves data. MCP enables interoperability between different AI systems and data sources, with implications for data access control, logging, and vendor lock-in. The use of MCP (usually described in terms of an “MCP server”) almost always implies that an AI system will be given access to some Resource.

Changes in model behavior over time due to updates, data changes, or environmental shifts. Drift drives change control clauses in contracts and can create unexpected compliance failures in validated workflows.

Attempting to recreate a model or approximate its behavior through repeated API queries. Model extraction is often prohibited in AUPs and relevant to trade secret protection; rate limiting and query monitoring serve as partial defenses.

Attacks inferring training data characteristics from model outputs. Model inversion is relevant to privacy assurances and security controls when sensitive data may have influenced training, even indirectly.

A system tracking model versions, metadata, approvals, and deployment status. Registries support governance and auditability by providing a single source of truth for what models exist, where they're deployed, and who approved them.

The set of upstream components and processes used to build and operate a model or AI system (datasets, code, weights, third-party models, tools, connectors, hosting, and subprocessors). Supply chain analysis is used for security, IP provenance, and compliance.

An architecture where multiple AI agents collaborate or compete to accomplish tasks. Multi-agent systems create complex liability and attribution challenges because harm may result from emergent interactions rather than any single agent's action.

A service architecture in which multiple customers share underlying infrastructure and software while remaining logically separated. Multi-tenancy affects data segregation, logging, and security controls.

AI designed for specific tasks rather than general intelligence. All current AI systems are narrow AI, regardless of marketing claims; this is relevant when assessing vendor capability representations.

AI techniques for understanding, analyzing, and generating human language. NLP encompasses many AI applications from classification to generation, with legal issues varying significantly by task type.

A system property where the same input may produce different outputs across runs. Most generative AI systems are non-deterministic by default due to sampling strategies, floating-point computation variations, and infrastructure differences. Non-determinism affects auditability, testing reproducibility, and user expectations; controls like temperature settings can enforce more deterministic behavior when needed.

Using an AI system for purposes beyond its intended or permitted use. Off-label use can affect contractual rights (including warranties and indemnities), compliance posture, and safety assumptions because the system may not have been evaluated or controlled for the new context.

In the CORE framework, the actions that components perform on external Resources or on data flowing through an AI system. Operations include data transformations (summarization, classification, generation), resource interactions (reading from databases, calling external APIs, writing outputs), and control functions (filtering, routing, logging).

Methods to improve a model or system’s performance, cost, latency, or resource use (e.g., quantization, caching, batching, prompt compression). Optimization choices can affect accuracy, safety behavior, and reproducibility.

Coordinating multiple AI components, tools, and workflows to accomplish complex tasks. Orchestration layers make consequential decisions about tool selection and sequencing, requiring oversight, logging, and clear accountability.

When a model performs well on training data but poorly on new data because it memorized specific examples rather than learning generalizable patterns. Overfitting explains why demo performance may not match production results, and frequently leads to memorization and subsequent regeneration of training material.

How well a model or system meets task objectives and operational requirements (accuracy, latency, robustness, cost, safety). In contracting, performance is often expressed as SLAs/SLOs, acceptance criteria, and evaluation benchmarks tied to intended use.

Information that can identify an individual, with the precise definition varying by jurisdiction. PII in training data, prompts, or outputs creates compliance obligations; detection and filtering controls are important safeguards.

A software extension enabling additional functionality in AI systems, such as web browsing, code execution, or database access. Plugins expand AI system capabilities and risks by accessing external resources, and may require separate permissions and security review.

Modifications to a model after pre-training to change behavior or improve usefulness and safety (e.g., instruction tuning, preference optimization, safety tuning, or distillation). Post-training often changes model characteristics and can affect evaluation results, safety properties, and documentation baselines.

Initial training of a model on large-scale data before any customization or alignment. Pre-training data provenance is the primary driver of copyright and privacy exposure for foundation models.

A metric measuring, of items classified as positive, the fraction that were truly positive. High precision means fewer false positives, which is important for applications where false accusations or unnecessary interventions are costly.

A fundamental characteristic of how generative AI models operate: outputs are generated by sampling from learned probability distributions over possible responses rather than by executing logical rules or retrieving stored facts. Even when configured for deterministic operation, a probabilistic model is selecting the statistically most likely output based on training patterns, not computing a provably correct answer. This distinction explains why models can be confidently wrong (hallucination), why explanations of "reasoning" may be post-hoc rationalizations, and why traditional software warranties and performance guarantees require adaptation for AI systems.

Making information publicly available (e.g., papers, open source repositories, model releases, technical reports). Publications can be relevant to prior art, marketing claims, and transparency commitments.

Processes ensuring AI systems meet quality standards before and during deployment. QA processes support reliability claims and reasonable care arguments; document testing methodologies and acceptance criteria.

A retrieval step (often in RAG) that reorders candidate results using a second model (e.g., a cross-encoder or an LLM) to improve relevance. Re-ranking can affect what content is presented to the generation model and therefore affects grounding and auditability.

An agent prompting pattern combining reasoning traces with action execution in an interleaved manner. ReAct is a common pattern in agentic AI systems that makes agent decision-making somewhat more transparent.

A metric measuring, of truly positive items, the fraction that were correctly identified. High recall is critical for safety applications where missing positives is costly, such as fraud detection or medical screening.

A training approach where the model learns by receiving rewards or penalties for its outputs rather than by studying labeled examples. A primary benefit of RL is that in some circumstances it allows synthetic data or self-play to be used in the place of human-labeled data. RL is the foundation for RLHF and is used in game-playing AI and robotics; understanding RL helps explain how models learn to follow instructions.

A variant of RLHF using AI-generated feedback rather than human feedback to reduce cost and scale. RLAIF may have different alignment properties than human feedback and is often combined with Constitutional AI approaches.

Training using automatically verifiable outcomes such as correct code execution or valid mathematical proofs. RLVR is used for training reasoning models on tasks with objectively checkable answers.

Learning useful internal representations of data that can transfer to multiple downstream tasks. Foundation models learn representations that encode semantic meaning, enabling capabilities like semantic search and few-shot learning.

In the CORE framework, external assets that an AI system accesses but does not control, such as data, third-party APIs, file systems, knowledge bases, and external services. Resources exist outside the system boundary but are invoked during execution. Resource mapping is essential for data sovereignty compliance, confidentiality protection, and contractual obligation tracking.

Policies and technical practices governing how long data and logs are kept and when they are deleted. In AI systems, retention can apply to prompts, outputs, embeddings, retrieval indexes, and tool call records.

In reinforcement learning, the function that assigns a numeric score (“reward”) to behaviors, guiding the model toward preferred outcomes. Reward function design influences aligned behavior and can encode tradeoffs.

A model trained to predict human preferences, used to guide RL training by scoring candidate outputs. Reward model quality directly affects alignment effectiveness and the behaviors the final model learns to exhibit.

Selecting outputs probabilistically from the model's predicted distribution rather than always choosing the highest-probability option. Sampling contributes to output variability; deterministic modes may be needed for audit and reproducibility requirements.

Empirical relationships showing that model performance improves predictably with increased size, data, and compute. Scaling laws drive investment in larger models and help explain capability improvements, though they don't guarantee specific abilities.

Unauthorized disclosure of secrets (API keys, credentials, tokens, confidential prompts) through prompts, logs, tool outputs, or model behavior. This is discussed in AI security, incident response, and vendor controls.

Unauthorized use of AI tools by employees outside official channels and governance processes. Shadow AI increases confidentiality breach risk, privilege waiver concerns, and compliance exposure; it requires both policy controls and technical measures to address.

Pejorative term for low-quality, mass-produced AI-generated content, typically created with minimal human oversight or input.

Linking generated outputs to the documents or data used to produce them. Attribution supports defensibility and user trust but can be incorrect or fabricated; validate citation mechanisms through testing.

Condensing longer content into shorter form while preserving key information. AI summarization may miss important details, introduce errors, or change emphasis; users often verify accuracy for legal and business-critical matters.

Machine learning from labeled examples that map inputs to correct outputs. Supervised learning requires labeled training data, raising data rights, annotation labor, and quality control considerations.

Environmental and resource considerations of AI development and deployment (energy use, water use, hardware lifecycle). Sustainability may be discussed in procurement, ESG reporting, and policy debates about compute-intensive systems.

Data generated by a computer process (possibly an AI system) rather than collected from real-world sources. Synthetic data can reduce privacy concerns and augment training sets but may not accurately represent real-world distributions or edge cases.

AI-generated content including images, audio, video, and text created to resemble authentic content. Synthetic media raises authenticity, deepfake, evidence authentication, and misinformation concerns; provenance and detection tools are evolving. A number of laws require the disclosure of synthetic media.

Documentation describing a complete AI system's design, intended use, capabilities, limitations, and safety measures. System cards provide more complete context than model cards by covering the full deployed system, not just the underlying model.

Methods for assessing whether a model or system meets requirements and behaves safely and reliably (e.g., unit tests, red teaming, evals, regression tests). Testing results are often used in acceptance, governance, and incident response.

A common inference task where a model assigns one or more labels to text (e.g., spam/not spam, topic tagging, sentiment). Classification models may be built with traditional ML, deep learning, or LLM prompting.

AI generation of images from text descriptions or prompts. Text-to-image systems raise copyright questions about training data and outputs, trademark concerns, and deepfake risks.

An analysis identifying potential attackers, attack vectors, assets at risk, and security boundaries for a system. Threat models inform security requirements, testing priorities, and control design; they should be documented and updated as systems evolve.

A risk where untrusted tool outputs (e.g., web content, search results, emails) introduce malicious instructions or data that affect subsequent model behavior. This is closely related to indirect prompt injection in tool-enabled systems.

Adapting a pre-trained model to new tasks rather than training from scratch, leveraging learned representations. Transfer learning underlies most commercial AI applications and drives questions about base model rights versus adaptation rights.

Dropping content when inputs exceed context window limits or other constraints, often without explicit notification. Truncation can undermine reliability because users may not know the system ignored portions of their input; it creates risk for legal document review.

A hardware- or enclave-based environment designed to protect code and data during execution (e.g., isolating workloads from a host OS). TEEs are discussed in confidential computing and can be used to reduce exposure when processing sensitive data.

Learning patterns from unlabeled data without explicit correct answers. Unsupervised techniques like clustering can create sensitive inferences about individuals or groups even without labeled protected attributes.

Testing that a system meets its specified requirements and performs acceptably for intended uses. Validation is key to acceptance criteria and regulatory compliance; document validation methodology and results.

Ensuring AI systems pursue goals and exhibit behaviors consistent with human values and intentions. Value alignment is a central AI safety concept; misalignment between system objectives and human values can cause harmful behavior even without adversarial attack.

Confirming that outputs, claims, or system behaviors are accurate and meet requirements. Human verification is a key control for managing hallucination risk in high-stakes applications; define verification requirements and responsibilities.