Papers and projects from Huzama Ahmad — Ph.D. candidate at KAIST AI working on efficient language modeling.https://huzama.comen-ushttps://huzama.com/work/attention-spanhttps://huzama.com/work/attention-spanLLMs spend most of their attention on a small fraction of context, yet they must read the entire KV cache at every decoding step to find it. In a 1M-token conversation where the user asks about a detail mentioned briefly in the middle, the model still scans the full history to generate each word of the reply. Existing methods for reducing this cost mostly approximate an attention mask from internal activations, but the approximation still costs O(N) per step. In this paper, we introduce Dynamic Attention Control (DAC), a prompting protocol that elicits the model to declare where it will attend as part of its chain of thought, partitioning generation into three modes: global (full context), focus (a specific region), and local (recent context only). The attention mask is read directly from the model's reasoning, with no auxiliary scorer. This unique design allows the model to dynamically modify its internal attention function at inference time via its own generated text on-the-fly. Across 15 long-context tasks on off-the-shelf models, DAC reduces decoding attention cost by 15.1% on Qwen-3.6-27B and 53.1% on Gemma-4-31B with accuracy drops of only 1.41pp and 0.08pp, zero-shot. We extend vLLM to support in-place KV cache masking with block-aligned masks compatible with state-of-the-art kernels including FlashAttention.Wed, 01 Jul 2026 00:00:00 GMTNamgyu Ho, Huzama Ahmad, Woosung Koh, Se-Young Yun, Tal Schuster, Cicero Nogueira dos Santospublicationhttps://huzama.com/work/spotattentionhttps://huzama.com/work/spotattentionLong contexts have become standard in pretrained LLMs, yet they remain expensive to run: prefill compute grows quadratically with sequence length, and every decode step re-reads a key-value cache that grows linearly with it. Sparse attention cuts these costs by attending only to a relevant subset of past tokens, but selecting that subset is itself expensive. We present SpotAttention, a lightweight selector that attaches to a frozen pretrained transformer and learns by KL distillation to estimate its attention distribution. The selector picks the top-K keys each query attends to, and because its estimate is a calibrated distribution, a dual top-p rule reads the per-query, per-layer budget directly from it. Across Qwen3 (dense, 4B–32B) and Qwen3.5 (hybrid linear/full attention, 4B–9B), SpotAttention matches dense accuracy at contexts up to 128K tokens, eight times the training length. Decode at L=128K runs 3.9× faster than FlashAttention and 1.8× faster than Twilight, the strongest training-free baseline. Quantizing the selector's K-cache to INT4 or FP4 microscale shrinks it 3.5× at no accuracy cost.Mon, 22 Jun 2026 00:00:00 GMTHuzama Ahmad, Se-Young Yunpublicationhttps://huzama.com/work/bastionhttps://huzama.com/work/bastionBlock-diffusion drafters have recently emerged as a powerful alternative for speculative decoding by predicting multiple future-token distributions in a single parallel step. However, since these parallel predictions are sampled from position-wise marginals rather than fully conditioned sequences, committing to a single greedy path often fails to capture the target model's preferred trajectory. To address this, we propose BASTION, a budget-aware speculative decoding framework with tree-based diffusion drafting. Unlike existing methods that rely on static tree topologies, BASTION dynamically constructs query-dependent trees by balancing draft quality against hardware constraints. Our framework integrates three synergistic components: (1) an acceptance surrogate that estimates expected accepted length via path confidence, (2) an online latency estimator that calibrates a hardware-aware roofline model, and (3) an adaptive best-first expansion that grows the tree until marginal gains no longer justify incremental verification costs. BASTION is training-free, preserves the target model's distribution, and requires no per-setting tuning. Across diverse benchmarks and GPU architectures, BASTION achieves up to a 6.61× speedup over standard autoregressive decoding, outperforming state-of-the-art block-diffusion baselines by 39%.Thu, 28 May 2026 00:00:00 GMTSoowon Oh, Nam Cao, Yujin Kim, Hojung Jung, Huzama Ahmad, Sangmin Bae, Se-Young Yunpublicationhttps://huzama.com/work/cascadeformerhttps://huzama.com/work/cascadeformerDeep Transformers are composed of uniformly stacked residual blocks, yet their deepest layers often add little value. We present two efficiency methods that exploit this asymmetry. CascadeFormer tapers width with depth to match the uneven information flow across layers, achieving comparable perplexity to a uniform baseline at the same training budget while reducing latency by 8.6% and increasing throughput by 9.4%. CascadeFlow Pruning removes layers using accumulated training gradients and outperforms standard heuristics on perplexity and rank-stability (while remaining competitive on downstream accuracy), without expensive post hoc analysis. To motivate these methods, we propose Gradient Fan-in Asymmetry (GFA) as a structural account of why deeper layers contribute less. In Pre-LayerNorm residual stacks, the gradient at a layer is the sum of an identity path and all downstream functional paths, producing a gradient fan-in that decays linearly with depth (and quadratically under deep supervision), yielding rich signals early and sparse for later layers. We provide correlational and interventional evidence for GFA on models trained from scratch up to 1.2B parameters: across Transformers and ResNets, accumulated training gradients follow the theoretical fan-in and are associated with post hoc layer importance; and two interventional experiments are consistent with structure (not magnitude) as the bottleneck, since equalizing per-layer gradient norms does not restore late-layer value, whereas increasing downstream path counts via parameter-shared repetition restores and elevates their impact. Whether gradient magnitude proxies fan-in beyond high-rank regimes, and how these dynamics behave at the 100B+ scale, remain open questions.Wed, 24 Sep 2025 00:00:00 GMTHuzama Ahmad, Cao Viet Hai Nam, Se-Young Yunpublicationhttps://huzama.com/work/diffusion-cultural-lenshttps://huzama.com/work/diffusion-cultural-lensText-to-image diffusion models have recently enabled the creation of visually compelling, detailed images from textual prompts. However, their ability to accurately represent various cultural nuances remains an open question. In our work, we introduce CULTDIFF benchmark, evaluating whether state-of-the-art diffusion models can generate culturally specific images spanning ten countries. We show that these models often fail to generate cultural artifacts in architecture, clothing, and food, especially for underrepresented country regions, by conducting a fine-grained analysis of different similarity aspects, revealing significant disparities in cultural relevance, description fidelity, and realism compared to real-world reference images. With the collected human evaluations, we develop a neural-based image-image similarity metric, namely, CULTDIFF-S, to predict human judgment on real and generated images with cultural artifacts. Our work highlights the need for more inclusive generative AI systems and equitable dataset representation over a wide range of cultures.Sun, 01 Jun 2025 00:00:00 GMTZahra Bayramli, Ayhan Suleymanzade, Na Min An, Huzama Ahmad, Eunsu Kim, Junyeong Park, James Thorne, Alice Ohpublicationhttps://huzama.com/work/tom-eats-kimchihttps://huzama.com/work/tom-eats-kimchiIn a highly globalized world, it is important for multi-modal large language models (MLLMs) to recognize and respond correctly to mixed-cultural inputs. For example, a model should correctly identify kimchi (Korean food) in an image both when an Asian woman is eating it, as well as an African man is eating it. However, current MLLMs show an over-reliance on the visual features of the person, leading to misclassification of the entities. To examine the robustness of MLLMs to different ethnicity, we introduce MIXCUBE, a cross-cultural bias benchmark, and study elements from five countries and four ethnicities. Our findings reveal that MLLMs achieve both higher accuracy and lower sensitivity to such perturbation for high-resource cultures, but not for low-resource cultures. GPT-4o, the best-performing model overall, shows up to 58% difference in accuracy between the original and perturbed cultural settings in low-resource cultures.Thu, 01 May 2025 00:00:00 GMTJun Seong Kim, Kyaw Ye Thu, Javad Ismayilzada, Junyeong Park, Eunsu Kim, Huzama Ahmad, Na Min An, James Thorne, Alice Ohpublicationhttps://huzama.com/work/efficient-llm-foundationshttps://huzama.com/work/efficient-llm-foundationsTwo years spent on a single question: how much of a large model's work is actually necessary? Answering it meant building the machinery first: running language models from one to eight billion parameters on TPU pods, training the smaller ones from scratch, and writing the GSPMD sharding and the static-graph, recompilation-free generation needed to run any of them. What followed were three lines of research, and one habit worth keeping: test a result until it either survives or breaks.Sun, 01 Oct 2023 00:00:00 GMTHuzama Ahmadprojecthttps://huzama.com/work/steperhttps://huzama.com/work/steperCan a language model get better at math by checking its own work? This project starts by measuring how reliably models can evaluate themselves, then turns that signal into training: a reinforcement-learning loop that rewards the model's greedy answer when it holds up against its own diverse samples, so the reasoning sharpens without a human grader in the loop. Across models from one to twenty billion parameters, it lifts accuracy by up to 5% on four math-reasoning benchmarks while also improving commonsense and holding onto general language ability, and both human and automated review found the explanations grew more detailed and logical.Mon, 01 May 2023 00:00:00 GMTJiwoo Hong, Huzama Ahmad, Minsu Kim, James Thorneprojecthttps://huzama.com/work/low-resource-languageshttps://huzama.com/work/low-resource-languagesA semester-long undergraduate research program testing how well large language models actually work once you leave English. Each of the eight teams took a different low-resource or non-English language, from Urdu and Bengali to Korean, Russian, and Indonesian, and a different capability: chain-of-thought reasoning, sentiment analysis, question answering, named-entity recognition, or standardized-exam performance. The recurring finding, language after language, was that today's models grow sharply weaker, and less culturally grounded, the further a language sits from their English-heavy training data. Two of the teams took their work all the way to published benchmarks.Mon, 27 Mar 2023 00:00:00 GMTHuzama Ahmadproject