Under development — I'm actively building this site.

huzama

Summary

Ph.D. candidate building efficient long-context language models, from algorithm to GPU kernel. Recent work makes sparse attention practical for pretrained models and lets a model control its own attention span, implemented in custom Triton kernels, FlexAttention, and vLLM, with contributions upstreamed to PyTorch/XLA and Hugging Face Transformers. Builds and runs a shared GPU cluster.

Education

  • Integrated M.S./Ph.D. in Artificial Intelligence
    Korea Advanced Institute of Science & Technology (KAIST) Seoul, South Korea
    Advisor
    Se-Young Yun (Optimization & Statistical Inference Lab)
    Thesis
    Efficient Long-Context Large Language Models
    Teaching Assistant
    Deep Learning for Natural Language Processing (Fall 2023)
  • B.E. in Computer Science and Engineering
    National University of Sciences & Technology (NUST) Islamabad, Pakistan
    Thesis
    IoT-Based Intelligent Manufacturing Execution System with Predictive Analysis

Research Experience

  • Graduate Researcher
    Korea Advanced Institute of Science & Technology (KAIST) Seoul, South Korea
    • Ph.D. research on efficient long-context LLMs in the OSI Lab (advisor Se-Young Yun): sparse attention, KV-cache optimization, and speculative decoding, carried from algorithm to Triton and CUDA kernel.
    • Built and operate the lab's GPU training cluster: Slurm scheduling, FreeIPA identity, NUMA-aware per-job GPU isolation, and WireGuard networking.
    • Led a cross-cultural and multilingual LLM evaluation program, mentoring undergraduate teams to build benchmarks (BEnQA, ACL'24; CLIcK, COLING'24) and surface cultural biases across models.
  • Research Intern
    RiseTech Islamabad, Pakistan
    • Built a U-Net for spinal segmentation and scoliosis classification, and a vision-language model generating diagnostic text from chest X-rays.

Selected Projects

see all
Efficient Long-Context Inference
2025 – Present
  • SpotAttention : a plug-in block-sparse selector that matches dense accuracy at long context while decoding 3.9× faster than FlashAttention.
  • Attention-Span : a prompting protocol that lets a model declare where it will attend, cutting decode attention cost up to 53% at near-zero accuracy loss.
  • Content-Aware Sparsity : prunes distractor tokens by content, not position, cutting noise as well as compute in long-context models.
Efficient-LLM Architecture & Training Systems
2023–2025
  • CascadeFormer : depth-tapered Transformers and gradient-based layer pruning motivated by Gradient Fan-in Asymmetry, cutting latency 8.6% and raising throughput 9.4% at equal perplexity.
  • Foundations of Efficient LLMs : GSPMD training on TPU pods, context compression, and uncertainty-aware prediction, with contributions upstreamed to PyTorch/XLA and Transformers.

Publications

Equal contribution.

Honors & Awards

Awards

  • Oral Presentation · AdaptFM @ International Conference on Machine Learning (ICML)
  • Outstanding Paper Award · C3NLP @ North American Chapter of the Association for Computational Linguistics (NAACL)
  • Oral Presentation · Association for Computational Linguistics (ACL)
  • 3x Medal of Excellence · National University of Sciences & Technology
    2017–2018

Scholarships

  • Fully-Funded Ph.D. Scholarship · Korea Advanced Institute of Science & Technology
    2021–2026
  • Merit-Based Scholarship · National University of Sciences & Technology
    2018–2020

Skills

Efficient-LLM Methods
Long-context inference, Sparse attention, KV-cache optimization, Speculative & parallel decoding, Knowledge distillation, Reinforcement learning (PPO, GRPO), PEFT
ML Systems & Performance
Large-scale distributed training (FSDP, GSPMD, RDMA), TPU/XLA static graphs, Custom Triton kernels, Quantization, HPC cluster management (Slurm, FreeIPA)
Frameworks & Tooling
PyTorch, Hugging Face, vLLM, FlexAttention, Weights & Biases, Docker, ZFS / TrueNAS, uv

BibTeX