Reading List
π Centralized Reading List & Field Advancements
Select a research vector below to isolate the literature and view its trend.
π Test-Time Adaptation
The field is converging on a paradigm shift from static prompt-based inference to dynamic, inference-time weight compression and online optimization, where models continuously map streaming contexts to low-rank fast weights or compact state matrices via single-pass hypernetworks and gradient-driven updates In-Place Test-Time Training Context-to-LoRA Hypernetwork Dynamic Layer-Wise TTA Large Chunk Test-Time Training. This trajectory directly circumvents the quadratic compute and memory bottlenecks of long-context expansion by reformulating attention as linear online training ViT3 Linear Test-Time Vision and stabilizing unsupervised parameter updates through meta-initialized initialization and adaptive learning rate multipliers End-to-End Test-Time Training Titans Test-Time Memory, ultimately enabling constant-latency scaling and persistent plasticity without catastrophic forgetting Fast-Slow LLM Training Latent Thought Models.
ββ Ξ΄-mem enables frozen LLMs to dynamically accumulate and reuse long-term context via a compact 8Γ8 state matrix updated at test-time, bypassing costly context expansion.
ββ Interleaving prompt optimization with reinforcement learning enables rapid task adaptation while preserving model plasticity and preventing catastrophic forgetting.
ββ Retrofitting frozen LLMs with inference-time layer looping and numerical integration yields consistent accuracy gains without training or architectural changes.
ββ Repurposes standard MLP projection matrices as dynamic fast weights to enable scalable, drop-in test-time adaptation for LLMs without costly retraining.
ββ A lightweight hypernetwork dynamically predicts per-layer, per-step learning rate multipliers to stabilize unsupervised inference-time parameter updates in LLMs.
ββ Instantly converts diverse text contexts into high-quality LoRA adapters via a single-pass hypernetwork, bypassing iterative fine-tuning and long-context latency.
ββ Redirecting inference compute from generating thinking tokens to lightweight query-gradient updates dramatically improves long-context retrieval and reasoning.
ββ Compressing long contexts into model weights via test-time gradient updates and meta-initialized initialization enables constant-latency scaling that rivals full-attention Transformers.
ββ Dynamically learns sample-specific augmentations via dual-loop meta-optimization to robustly adapt vision-language models to unseen domains during inference.
ββ A 7B RL-trained model dynamically orchestrates and scales worker LLMs at inference time, achieving state-of-the-art reasoning performance through adaptive test-time coordination.
ββ ViT3 reformulates vision attention as online test-time training, achieving linear O(N) complexity while matching or surpassing state-of-the-art efficient and quadratic models across classification, detection, segmentation, and generation.
ββ SEAL trains LLMs via reinforcement learning to autonomously generate synthetic data and optimization directives, enabling persistent, self-directed weight updates for novel tasks without external supervision.
ββ A hypernetwork generates task-specific LoRA adapters on-the-fly from natural language prompts, enabling zero-shot model adaptation without retraining.
ββ LaCT scales fast-weight updates to millions of tokens, boosting GPU utilization by orders of magnitude and enabling scalable long-context modeling across diverse modalities.
ββ Minimizing input perplexity on unlabeled test data via LoRA and high-perplexity sampling enables efficient, label-free adaptation of LLMs to new domains.
ββ LTMs leverage inference-time optimization of explicit latent thought vectors to achieve superior sample efficiency and emergent reasoning with drastically fewer parameters than standard LLMs.
ββ Replaces computationally heavy test-time prompt tuning with lightweight low-rank image encoder updates, enabling fast, memory-efficient domain adaptation for vision-language models.
ββ A novel architecture that dynamically memorizes surprising context at inference time, enabling efficient scaling beyond 2M tokens while outperforming Transformers and linear RNNs.
ββ Dynamically converts streaming text contexts into low-rank parameter updates via a single forward pass, enabling efficient test-time adaptation without fine-tuning or inference overhead.
ββ Dynamically updating model weights during inference using few-shot examples drastically improves reasoning and generalization on novel tasks compared to static prompting.
π§ In-Context Learning
The field has converged on a paradigm where in-context learning is mathematically formalized as token-dependent rank-1 implicit weight updates to MLP layers Context-to-Weight Equivalence Implicit Dynamics of ICL, transforming prompt composition from a static heuristic into a gradient-optimized, dynamically routed mechanism for parameter-free adaptation Context Tuning for LLMs Active Example Selection. Consequently, the collective frontier addresses the architectural and modality bottlenecks of fixed context windows by externalizing long-horizon processing through recursive decomposition Recursive Long Context Scaling, adapter-based internalization Instant Context Internalization, and non-parametric memory routing HippoRAG 2 Memory Framework, while bridging cross-modal alignment gaps via lightweight projection and intent-aligned retrieval Continuous Vector In-Context Learning Visual In-Context Learning.
ββ Reframes demonstration selection as a reinforcement learning problem to dynamically balance visual relevance and diversity, outperforming kNN on objective visual regression tasks.
ββ A meta-learned hypernetwork instantly converts long document contexts into lightweight LoRA adapters, enabling fast, memory-efficient in-context learning without iterative fine-tuning.
ββ A model-agnostic inference strategy that treats prompts as an external environment, enabling LLMs to recursively decompose and process arbitrarily long contexts without architectural changes or context rot.
ββ Proves that the computational effect of context in modern LLMs can be perfectly mapped to token-dependent rank-1 patches on MLP weights and normalization layers.
ββ Gradient-optimized context initialization from few-shot examples significantly boosts LLM few-shot performance without parameter updates.
ββ Mathematically proves that transformer prompts act as exact rank-1 implicit weight updates to MLP layers, unifying in-context learning with gradient descent dynamics.
ββ A lightweight transformer with task-aware attention dynamically optimizes in-context demonstration sequences to significantly boost multimodal ICL performance.
ββ HippoRAG 2 integrates Personalized PageRank with dynamic passage weighting and online LLM filtering to deliver robust, human-like non-parametric memory across factual, associative, and sense-making tasks without catastrophic forgetting.
ββ LLMs can effectively perform in-context learning on diverse continuous data modalities by aligning them to their embedding space via lightweight projectors.
ββ Scaling in-context examples to hundreds or thousands dramatically boosts LLM reasoning and generation, often matching fine-tuning while bypassing the need for human-labeled rationales.
ββ Replaces visual demonstrations with intent-oriented text summaries to bridge cross-modal gaps and significantly boost ICL in large vision-language models.
ββ RetICL leverages reinforcement learning to sequentially retrieve and order in-context examples, dynamically optimizing prompt composition for superior LLM reasoning.
ββ Reinforcement learning optimizes demonstration example selection to stabilize and boost in-context learning performance, particularly for smaller language models.
β‘ Efficient Architectures
The field is undergoing a structural paradigm shift from quadratic autoregressive attention to linear-recurrent and continuous latent frameworks that explicitly decouple parameter count from compute depth, directly resolving the KV cache bloat and sequential decoding bottlenecks that constrain long-context scaling Dynamic Linear Attention Functional Attention Architecture Kimi Linear Architecture CALM: Next-Vector Language Models. This mathematical trajectory is unified by curvature-aware preconditioning Curvature-Aware Sequence Modeling, manifold-constrained hyper-connections Manifold-Constrained Hyper-Connections, and dynamic sparse routing protocols Hierarchical Sparse Attention query-oriented-sparse-attention that enforce resolution-invariant operator learning while preserving representation fidelity across agentic reasoning and generative workloads.
ββ DLA replaces fixed memory merging in linear attention with adaptive, information-aware state boundaries, enabling efficient long-context modeling without sacrificing representation capacity.
ββ A systematic data-management framework decomposes and evaluates 12 LLM agent memory architectures, revealing workload-dependent trade-offs between retrieval fidelity, update robustness, and operational cost.
ββ Replaces quadratic token-wise attention with linear functional operators to enable resolution-invariant, geometry-aware operator learning.
ββ Decomposing agentic reasoning into a dynamically gated configurator and simulative planner slashes token consumption by up to 95% while matching trillion-parameter model accuracy.
ββ MEMO replaces brittle retrieval with a plug-and-play modular memory model, enabling efficient, noise-robust knowledge integration without retraining or modifying the base LLM.
ββ GRKV eliminates over-merging in long-context LLMs by using training-free global ridge regression to align compressed and full KV caches, boosting performance with minimal memory overhead.
ββ Internalizing agentic routing logic directly into small model weights delivers near-frontier procedural quality at two orders of magnitude lower inference cost than external orchestration.
ββ Diagonal preconditioning of linear recurrences captures second-order optimization curvature, enabling efficient chunkwise parallel architectures that consistently improve long-context retrieval and language modeling.
ββ Parcae stabilizes looped transformers via spectral norm constraints, enabling predictable training and test-time compute scaling that matches larger Transformer baselines.
ββ LCA natively condenses compressed latent representations to slash KV cache and compute by up to 90% without adding parameters or sacrificing accuracy.
ββ ELT decouples parameter count from compute depth via recurrent weight-sharing and intra-loop distillation, enabling any-time visual generation with a 4Γ efficiency gain.
ββ A learnable, hierarchical gist-compression framework that routes sparse attention to relevant context chunks, achieving log-linear complexity without architectural modifications.
ββ Native diffusion LLMs exhibit hierarchical representational redundancy that enables aggressive, static inference-time layer skipping for significant FLOPs reduction without performance degradation.
ββ Novel CUDA kernels and sparse packing formats unlock >99% unstructured sparsity in LLMs, yielding major throughput and memory gains with minimal accuracy trade-offs.
ββ ConFu accelerates LLM inference by equipping draft models with dynamic, future-oriented contemplate tokens that mitigate error accumulation and boost token acceptance rates over SOTA baselines.
ββ FastForward slashes prefill latency via block-wise predictive FFN sparsity and error compensation, delivering up to 1.45Γ speedup with minimal accuracy loss on constrained hardware.
ββ DualPath eliminates KV-cache I/O bottlenecks in agentic LLM inference by dynamically routing cache loads across prefill and decode engines, boosting throughput nearly 2Γ.
ββ A novel training-time distribution evolution framework that enables high-quality, single-step image generation without iterative inference or distillation.
ββ Introduces a confidence-driven latent reasoning framework that fuses contextual hidden states with predictive vocabulary embeddings to eliminate feature collapse and reduce inference latency in LLMs.
ββ IHA overcomes multi-head attentionβs linear scaling bottleneck by enabling cross-head mixing via pseudo-projections, drastically improving parameter efficiency and multi-step reasoning performance.
ββ Adaptive hierarchical top-p sparse attention eliminates fixed-budget constraints to deliver near-zero accuracy loss with up to 2.23Γ decoding speedup for long-context LLMs.
ββ Compressing evicted KV cache tokens into LoRA adapters enables memory-efficient RL training and long-context reasoning under strict hardware constraints.
ββ QUOKA accelerates LLM prefill latency by up to 7Γ through a training-free, hardware-agnostic sparse attention mechanism that dynamically selects representative queries and aligned KV pairs.
ββ Caching intermediate RNN memory states enables linear-complexity models to scale context length without quadratic overhead.
ββ ComprExIT bypasses trainable compressor LLMs by using frozen hidden states and optimal transport to compress context efficiently, outperforming existing methods with only 1% extra parameters.
ββ Projects unconstrained hyper-connections onto a constrained manifold to restore identity mapping, enabling stable and efficient large-scale LLM training.
ββ A hybrid linear attention design that slashes KV cache by 75% and boosts decoding speed up to 6Γ while surpassing full-attention baselines.
ββ CALM replaces discrete token prediction with continuous vector generation, cutting autoregressive steps by K and significantly improving the performance-compute trade-off for LLMs.
ββ LoopLM replaces explicit chain-of-thought with iterative latent computation and adaptive early-exit gating, delivering 2β3Γ parameter efficiency at frontier scale.
ββ Spiffy introduces lossless speculative decoding with directed draft graphs to accelerate diffusion LLM inference by up to 7.9Γ without training auxiliary models.
ββ NSNs enable post-hoc, continuous compute-performance trade-offs in pre-trained LLMs via nested low-rank factorization without retraining.
ββ Pruning the drafterβs language modeling head to frequent tokens eliminates memory-bound drafting overhead, boosting speculative decoding speed without training.
ββ FLoE slashes PEFT memory and compute by using Fisher information to sparsely activate only critical transformer layers for MoE-based LoRA adaptation.
ββ Transforms residual networks into independently trainable blocks via diffusion theory, slashing training memory by a factor of B while matching end-to-end performance.
ββ KEYDIFF enables memory-constrained long-context LLM inference by evicting KV cache tokens based on key diversity rather than attention scores, cutting latency up to 30% with near-baseline accuracy.
ββ KEYDIFF enables memory-constrained long-context LLM inference by evicting KV cache entries based on key diversity rather than attention scores, cutting latency up to 30% with negligible accuracy loss.
ββ CAOTE minimizes long-context inference bottlenecks by introducing a closed-form token eviction metric that directly optimizes attention output error through key-value integration, boosting accuracy without retraining.
ββ TurboQuant achieves near-optimal online vector quantization via random rotation and residual transforms, enabling lossless LLM KV caching and instant nearest-neighbor search.
ββ Weight-sharing looped transformers achieve deep-network reasoning performance with a fraction of parameters by implicitly simulating chain-of-thought steps.
ββ Coconut replaces discrete chain-of-thought with continuous latent feedback, enabling efficient breadth-first search reasoning and significantly reducing token costs while boosting accuracy.
ββ Unifies data-dependent gating and delta rules into a hardware-efficient linear attention mechanism that enables precise memory control and outperforms prior SOTA across long-context benchmarks.
ββ AdaEDL dynamically optimizes speculative decoding draft lengths using entropy-based bounds, boosting inference speed by up to 57% without training.
ββ Hybrid opto-electronic hardware enables energy-efficient, massively parallel training of billion-parameter models by replacing backpropagation with optical direct feedback alignment.
ββ Modifying just 1β2% of base weights via sparse masking enables instant adapter switching and eliminates concept interference during multi-adapter fusion, outperforming LoRA.
ββ EAGLE-2 dynamically adjusts speculative draft trees using confidence scores to achieve lossless, up to 4.26x inference speedups without extra training.
ββ A dynamic, hierarchical gist-token compression method that drastically reduces LLM inference compute and memory while preserving zero-shot API calling accuracy.
ββ MoLA dynamically allocates more LoRA experts to higher Transformer layers, boosting PEFT performance while eliminating redundant parameters.
ββ EAGLE accelerates LLM inference up to 3.5x by predicting second-to-top-layer features and resolving sampling ambiguity via shifted token inputs.
ββ AutoCompressors recursively compress long texts into reusable soft prompts, enabling efficient long-window reasoning and accelerated inference without architectural overhauls.
ββ Compresses arbitrary prompts into cached gist tokens via modified attention masks, cutting inference FLOPs by up to 40% with minimal quality loss.
ββ GRAM leverages graph propagation to efficiently search flexible neural architectures, yielding SwiftNet models with state-of-the-art accuracy-density and latency for edge deployment.
ποΈ Multimodal & Vision
The field is converging on a predictive latent-space paradigm that replaces flat reconstruction and contrastive objectives with hierarchical feature fusion and dynamic cross-modal alignment to enable structured semantic reasoning without heavy tuning Hierarchical Vision LLM Pretraining Next-Embedding Predictive Autoregression Multi-Modal Latent CoT. This trajectory directly addresses the latency-quality and cross-image aggregation bottlenecks in real-time multimodal pipelines by injecting asynchronous retrieval Async RAG for Full-Duplex Speech and orthogonal attention mechanisms Zero-Shot Subject Style Composition into frozen latent predictors VLM-Guided Latent World Models LatentLens Interpretable Visual Tokens Multi-Image VLM Analysis Tandem S2S-LLM Architecture.
ββ Introduces an asynchronous retrieval mechanism that boosts factuality in real-time full-duplex speech models without compromising conversational latency.
ββ HIVE replaces flattened vision embeddings with multi-layer cross-attention, enabling structured hierarchical feature fusion that boosts multimodal alignment and cuts training costs by half.
ββ Bridges dense visual dynamics and long-horizon semantic reasoning by injecting hierarchical VLM features into JEPA latent predictors for robust trajectory forecasting.
ββ LATENTLENS demonstrates that visual tokens in frozen LLMs are highly interpretable by matching them to contextualized text embeddings, fundamentally challenging prior methods that underestimated cross-modal semantic alignment.
ββ Resolves critical cross-image aggregation failures in VLMs through a controlled benchmark, synthetic data generation, and targeted attention masking.
ββ A minimalist causal transformer trained to predict future patch embeddings achieves state-of-the-art self-supervised vision learning without reconstruction or contrastive losses.
ββ A tandem architecture injects real-time LLM reasoning into a speech-to-speech model, bridging the latency-quality gap in conversational AI.
ββ SubZero enables tuning-free subject and style composition in diffusion models via orthogonal attention and latent optimization, eliminating content leakage for efficient edge deployment.
ββ Introduces a diffusion-based latent space learning framework that dynamically aligns visual and textual features to significantly boost multi-modal chain-of-thought reasoning.
π€ Embodied AI & Robotics
The field is converging on unified differentiable mappings from raw pixel observations to continuous motor commands, where cross-embodiment generalization is achieved by scaling vision-language priors through efficient fine-tuning OpenVLA Generalist Robot Policy and stabilizing latent dynamics with minimal loss formulations that eliminate heuristic collapse prevention End-to-End Latent World Models. This trajectory replaces modular control pipelines with end-to-end trainable architectures that couple flow-matched action generation Generalist Robot Control Model with predictive world modeling, establishing a scalable mathematical framework where high-frequency dexterous manipulation emerges directly from large-scale multimodal demonstrations without task-specific architectural inductive biases.
ββ LeWorldModel enables stable, end-to-end training of latent world models from raw pixels using only two loss terms, eliminating heuristic collapse-prevention tricks while enabling fast planning and physical understanding.
ββ Merges pre-trained vision-language models with flow matching to enable high-frequency, dexterous robot control across diverse embodiments using a massive 10k-hour dataset.
ββ OpenVLA sets a new SOTA for generalist robot control by efficiently fine-tuning a 7B vision-language model on diverse real-world demonstrations.
π Theory & Optimization
The field is converging on a paradigm shift from weight-transport-dependent backpropagation toward biologically plausible, layer-independent optimization via direct feedback alignment Direct Feedback Alignment Training Feedback Alignment Dynamics, which fundamentally circumvents the severe gradient compression bottleneck inherent in standard linear projection heads LM Head Gradient Bottleneck. This mathematical trajectory replaces homogeneous autoregressive weight updates with self-supervised latent dynamics and intrinsic adaptation mechanisms that leverage continuous predictive objectives Next Implicit Token Prediction and unweighted external state optimization SkillOpt Text-Space Optimizer, establishing a new frontier where generalization and specialization are governed by compact belief states and topology-aware forward training rather than sparse external supervision or validation-dependent risk minimization Compact Latent Dynamics in Transformers Superhuman Adaptable Intelligence.
ββ Proposes temporal derivative predictive learning as the biologically plausible foundation for neocortical intelligence, bridging error backpropagation with synaptic plasticity mechanisms.
ββ SMT replaces unstable BPTT with time-parallel supervised learning on Transformer-generated memory states, enabling stable O(1) gradient paths for long-range RNN training.
ββ SkillOpt treats agent skills as trainable external states, using a structured text-space optimizer with learning rates, validation gates, and momentum to reliably improve procedural performance without weight updates.
ββ Augmenting discrete token prediction with continuous latent-space supervision prevents representation collapse and consistently boosts LLM generalization.
ββ Derives a practical population-risk objective from training dynamics that explains generalization phenomena and accelerates optimization without validation data.
ββ Backpropagating through a standard linear LM head compresses 95β99% of gradients, fundamentally degrading training efficiency and convergence.
ββ Leverages conditional Lagrangian optimal transport to interpolate neural network outputs across unobserved hyperparameter settings, enabling dynamic inference-time adaptation without retraining.
ββ Shifts the AI research paradigm from anthropocentric AGI to measurable adaptation speed and specialization, advocating for self-supervised world models over homogeneous autoregressive architectures.
ββ NextLat injects a recurrent inductive bias into transformers via self-supervised latent prediction, provably yielding compact belief states that generalize better and accelerate inference.
ββ GEPA replaces sparse RL gradients with dense natural language reflection and evolutionary search, optimizing LLM prompts with up to 35x fewer rollouts while surpassing prior SOTA.
ββ INTUITOR replaces external rewards with self-certainty scores, enabling fully unsupervised LLM reasoning that matches supervised RL while generalizing better across domains.
ββ Models can iteratively bootstrap their own data efficiency by inferring and training on synthetic latent reasoning thoughts via an EM algorithm.
ββ DFA-GNN replaces biologically implausible backpropagation with a topology-aware direct feedback alignment mechanism, enabling efficient, parallel, and robust training for graph neural networks.
ββ Direct Feedback Alignment succeeds by sequentially aligning gradients before memorizing data, with performance dictated by the conditioning of alignment matrices.
ββ DFA successfully trains state-of-the-art architectures like Transformers and GNNs without backpropagationβs weight transport requirement, matching BP performance with targeted optimizer tuning.
ββ Replaces backpropagation with fixed random feedback weights, enabling biologically plausible, layer-independent training while maintaining competitive benchmark performance.