FINOS-AIGF/AIR-VEC Vector Catalog

Usage patterns inadvertently lead to excessive costs, throttling, or service disruptions. Overly long prompts from large document chunking, multimedia content, or token-expensive adversarial queries can exhaust token limits or drive up charges. Poorly throttled scripts or agentic systems may generate excessive API calls, overwhelming resources and bypassing capacity planning.

External technology service providers may lack operational maturity to maintain stable service levels, leading to unexpected outages or performance degradation under load. Tight coupling to a specific proprietary provider limits failover capability, violating business continuity expectations.

Video RAM exhaustion on serving infrastructure compromises model responsiveness or triggers crashes. Causes include configuration changes that exceed available resources, caching strategies that trade VRAM for throughput, and memory leaks in model-serving libraries that prevent proper resource release.

AI systems may be used beyond their originally intended and validated scope, leading to unreliable outputs in contexts the model was not designed or tested for. Scope creep can occur gradually as users discover new applications, or suddenly when systems are repurposed without adequate re-evaluation of risks and performance characteristics.

AI systems may generate outputs that are offensive, inappropriate, misleading, or otherwise damaging to the organization's reputation. This risk is amplified when attackers deliberately manipulate models into producing harmful content that is then attributed to the organization.

Attackers interact directly with the LLM to override its intended behaviour. Crafted inputs attempt to bypass system prompts, ignore safety guardrails, or coerce the model into disclosing sensitive information. Requires no special privileges and can be executed through simple input manipulation.

Malicious instructions are embedded in third-party content such as websites, emails, or uploaded documents. When the LLM processes this contaminated data, the injected prompts can hijack decision-making, escalate privileges, trigger unauthorized actions, or exfiltrate data being processed. Especially dangerous in automated workflows or multi-agent architectures.

Sophisticated prompt injection techniques probe the internal structure of an LLM to extract model biases, proprietary system prompts, configurations, or training data used in fine-tuning or RAG corpora. Enables intellectual property theft, facilitates future attacks, or supports creation of clone models.

Adversaries alter training datasets by changing labels or injecting crafted data points with hidden patterns. In financial services, this includes marking fraudulent transactions as legitimate to corrupt fraud detection models, or embedding backdoor triggers exploitable after deployment.

Systems that continuously learn from new data are vulnerable when validation mechanisms are inadequate. Adversaries systematically feed misleading information over time to gradually skew decision-making in credit scoring, trading, or risk models.

Financial institutions rely on external data feeds such as market data, credit references, and KYC/AML watchlists. Compromise of these sources introduces poisoned data that can unknowingly embed biases or vulnerabilities into downstream models.

Deliberate data poisoning amplifies biases in credit scoring or loan approval models, leading to discriminatory outcomes and regulatory non-compliance. Effects are subtle and may remain hidden until major failures or regulatory interventions occur.

LLMs can memorize sensitive data from training or user interactions, later disclosing customer details, loan terms, or trading strategies in unrelated sessions. This includes cross-user leakage, where one user's sensitive data is disclosed to another.

Adversaries craft prompts to extract memorized sensitive information from hosted models. Targeted prompt sequences can cause the model to reproduce confidential training data, PII, or proprietary algorithms that were not intended to be accessible.

Insufficient sanitization, encryption, or access controls by hosted model providers increases disclosure risk. Providers may lack transparent mechanisms for how input data is processed, retained, or sanitized, leading to persistent exposure of proprietary data.

Without clear contracts ensuring encryption, retention limits, and secure deletion, institutions lose control over sensitive data sent to hosted models. Providers may lack transparency about data processing and retention practices.

Using proprietary data for fine-tuning embeds sensitive information directly into model weights, potentially making it accessible to unauthorized users if access controls are inadequate.

Although embeddings are not human-readable, inversion attacks can reconstruct the original text from stored vectors, exposing proprietary or personally identifiable information held in a RAG vector store.

An adversary probes the vector store to determine whether specific information is present, for example generating embeddings for a confidential transaction and inferring from similarity whether such a deal is being discussed internally.

An attacker with access injects malicious or misleading embeddings into the vector store, degrading the accuracy of retrieved context; dense numerical representations make such tampering difficult to detect.

Missing role-based access control or overly permissive settings on the vector store allow unauthorized users to retrieve embeddings of sensitive internal data.

Vector stores lacking encryption at rest expose embeddings to anyone with storage access, while absent audit logging prevents detection of unauthorized access, modification, or exfiltration.

The model cannot distinguish accurate from inaccurate information in its training corpus, so it may generate plausible but fabricated financial facts, figures, or citations.

When prompts lack clarity or precision, the model is more likely to fabricate plausible-sounding but incorrect details to fill the gaps.

Hallucinated content is delivered with high fluency and syntactic confidence, making inaccuracies difficult for users to recognise and increasing the chance they act on false information.

Instructions or fine-tuning intended to improve helpfulness or creativity can inadvertently increase the model's tendency to produce unsupported statements.

Because models sample from a probability distribution over next tokens rather than always selecting the most likely token, identical inputs can yield different outputs across runs.

Random seeds, GPU computation variations, and floating-point precision differences cause non-reproducible outputs even with fixed inputs and parameters.

Output varies with the position of content in the token window or with slight rephrasing, producing inconsistent results for semantically equivalent prompts.

Sampling parameters such as temperature and top-p amplify or dampen variability, trading consistency against creativity.

The model generates confident responses that contradict or misinterpret the retrieved financial documents, for example omitting critical regulatory exceptions documented in policy.

Important regulatory caveats, disclaimers, or conditional statements are truncated or deprioritised when documents exceed the context window, yielding authoritative-looking but incomplete guidance.

When retrieved documents do not fully address a query, the model fills gaps with plausible but incorrect general knowledge, blending accurate institutional content with inaccurate information.

The model provides advice or recommendations beyond its authorised scope, such as offering investment advice from a system licensed only for general account information.

The model adopts an inappropriate tone or level of certainty for financial communications, such as being overly definitive about complex regulatory matters.

Providers retrain, fine-tune, or re-architect foundation models without explicit notification or version pinning, causing behaviour to shift even when inputs are unchanged and breaking testing and reproducibility.

Changes to a model's hidden or implicit system prompt, for example for safety or compliance, alter outputs subtly or significantly even when user inputs remain identical.

Changes to deployment infrastructure such as hardware, quantization, or tokenization, or to API defaults, affect model behaviour, particularly for latency- or performance-sensitive applications.

Minor variations in phrasing significantly change outputs and can be exploited to attack model grounding or circumvent safeguards, introducing further unpredictability.

Adversaries tamper with training data, fine-tuning datasets, or pretrained model weights in the provider's pipeline, embedding subtle manipulations that are difficult to detect downstream.

Compromise of GPU firmware, operating systems, cloud orchestration, or ML libraries such as TensorFlow, PyTorch, and CUDA enables tampering with the model or its runtime behaviour without detection.

Where model weights are accessible, attackers craft subtle adversarial modifications during fine-tuning that cause unsafe responses or bypass content filters under specific conditions.

A model is engineered to behave maliciously when presented with a specific trigger phrase or input pattern, activating offensive outputs, bypassing constraints, or revealing sensitive information.

Tampering disables alignment or content-moderation systems, neutralising the safeguards intended to enforce responsible model behaviour.

Inaccurate, incomplete, or biased training or fine-tuning data leads the model to produce unreliable, misleading, or irrelevant outputs, especially in decision-making and risk analysis.

Models become stale as the statistical properties of input data change over time, eroding predictive power and causing failure to recognise emerging market shifts or new regulatory requirements.

Errors or embedded biases in historical training data propagate into the model and are magnified at scale, undermining performance and introducing legal and reputational risk.

Training datasets reflect historical societal biases or under-represent populations, leading the model to learn and perpetuate discriminatory patterns such as lower loan-approval rates for certain groups.

Model architecture, feature selection, or optimization choices unintentionally introduce or amplify bias, for example by over-weighting a feature correlated with a protected characteristic.

Seemingly neutral data points such as postal codes or transaction history act as proxies for protected characteristics, producing discriminatory decisions.

A biased system's outputs are fed back into its learning cycle without correction, making the bias self-reinforcing and amplified over time.

Complex foundation models operate as black boxes, producing outputs without a clear, traceable rationale. Firms cannot adequately justify AI-driven decisions to regulators, stakeholders, or customers, and underlying errors or biases may go undetected, complicating model-soundness assessment and risk management.

AI-generated financial advice, marketing, or communications must meet the same standards as human-produced outputs, including KYC, suitability, fair and accurate disclosure, and record-keeping; failing to do so breaches regimes such as MiFID II, SEC rules, and FINRA guidelines.

AI models informing critical decisions fall under divergent model-risk-management expectations, such as the UK PRA's SS1/23, with recently shifting US scope; inadequate validation, monitoring, documentation, and oversight create compliance exposure.

Firms remain accountable for supervising AI systems; failure to define clear lines of accountability and ensure staff understand system capabilities and limitations leads directly to non-compliance.

New and diverging legislation, such as the EU AI Act's high-risk classification and Fundamental Rights Impact Assessments alongside US fair-lending, FCRA, and state AI laws, imposes additional transparency, fairness, and oversight obligations that firms must anticipate.

AI outputs may replicate copyrighted material from training data, creating legal liability when used in marketing, code generation, or research reports.

Employees inputting proprietary algorithms, M&A strategies, or confidential data into public AI tools risk irretrievable loss of valuable intellectual property.

Improper licensing of AI platforms or failure to comply with terms of service results in contractual breaches.

Agents discover and use API endpoints not intended for their use case, for example a balance-inquiry agent invoking payment-transfer APIs, because endpoint restrictions are insufficient.

By chaining individually authorized API calls, an agent achieves outcomes that no single authorized action should permit, such as aggregating data to enable unauthorized decisions.

Agents bypass intended workflows, approval processes, or segregation-of-duties requirements on which regulatory compliance depends.

An agent's interpretation of its granted permissions expands during operation, producing permission creep and broader access than originally intended without explicit reconfiguration.

Crafted inputs cause the agent to select inappropriate tools for the task, for example choosing payment-transfer tools when only a balance check was requested.

Malicious inputs influence the parameters an agent passes to legitimate API calls, such as injecting attacker-controlled account numbers, amounts, or authorization codes.

Adversaries manipulate the order in which an agent executes tools, creating dangerous combinations of otherwise safe individual operations.

Attacks corrupt the agent's understanding of tool states, capabilities, or relationships, leading to inappropriate or dangerous tool usage.

Outputs from one tool are used to inject malicious data into subsequent tool calls, creating a chain of compromised operations.

External MCP servers operated by vendors or partners are compromised, injecting malicious data or logic into services that agents consume.

Legitimate MCP servers receive malicious updates or patches that introduce backdoors, data corruption, or logic manipulation without operator knowledge.

Malicious insiders with access to MCP infrastructure deliberately corrupt data, introduce backdoors, or modify business logic to benefit attackers.

Attacks target the MCP communication protocol itself, including man-in-the-middle, protocol-downgrade, or exploitation of protocol vulnerabilities.

Agent MCP connections are redirected to attacker-controlled servers through DNS poisoning, BGP hijacking, or other network-level attacks.

Prompt injection or similar techniques cause agents to store malicious instructions or compromised reasoning patterns in their persistent memory.

Through repeated exposure to malicious inputs, agents learn inappropriate patterns or exceptions to business rules that persist across sessions.

Direct attacks on the databases, files, or cloud storage holding agent state allow attackers to modify agent memory without interacting with the agent.

Malicious instructions embedded in one session persist and influence agent behaviour in subsequent sessions with different users or contexts.

Corrupting agent preferences, configuration, or learned user patterns biases the agent toward specific outcomes or bypasses security controls.

Malicious agents inject harmful data, instructions, or corrupted state into communication channels, causing receiving agents to adopt compromised behaviours.

Compromised agents corrupt shared databases, APIs, or state storage relied upon by other agents, causing systematic errors across multiple agent types.

Compromised agents impersonate higher-privilege agents or use stolen credentials to access resources or influence decisions outside their intended scope.

Design flaws allow agents to inherit or assume privileges from agents they interact with, escalating privilege across the multi-agent system.

Failures or compromises in one agent cascade to dependent agents, potentially bringing down entire business processes or decision chains.

Agents are manipulated to use legitimate file, database, or API tools to systematically search for credentials in configuration files, environment variables, logs, and source repositories.

Compromised agents use system access to extract credentials from process memory, swap files, core dumps, or temporary storage where they may be cached.

Agents exploit database access to search for credentials stored in user tables, configuration tables, or other locations holding passwords, API keys, or tokens.

Agents leverage cloud-management APIs and infrastructure tools to discover credentials in key vaults, secret stores, instance metadata, or infrastructure-as-code.

Agents correlate partial credential information across systems, reconstruct full credentials from fragments, or identify credential-reuse patterns.