Efficient AI & Model Compression
Making language models more accessible through compact architectures and efficient inference, without losing the representations that matter.
Knowledge distillation · Pruning · 4-bit / 8-bit PTQ
Sukkur IBA University · Department of Computer Systems Engineering · Sukkur, Sindh, Pakistan
Research & engineering
Efficient AI. Grounded in systems.
“Investigating efficient, quantized language models under limited data and compute — and how machine learning and pretrained foundation models can reason over scientific data.”
3.48/4.0
Cumulative CGPA
50+
Production AI deployments
2
National fellowships
STHP / STFS
01 / About
An engineering lens on AI.
I am an undergraduate in Computer Systems Engineering at Sukkur IBA University , with an expected graduation in May 2027 and a cumulative GPA of 3.48/4.00.
My work centers on a practical question: how can language models remain capable, efficient, and reliable when data and compute are limited? I study parameter-efficient adaptation, model compression, and out-of-distribution robustness through the lens of real systems constraints.
I value reproducible experiments, clear evaluation protocols, and hardware-aware deep learning. Alongside independent research, I build applied AI systems—from document-grounded retrieval to agentic automation—connecting scientific questions with production experience.
Academic chronicle
2026
50+ client solutions delivered
Production AI, retrieval, and automation systems
May 2025
KPMG Australia internship
Data analysis · Remote
Aug 2023
Began B.E. at Sukkur IBA
Computer Systems Engineering · STHP & STFS Scholar
02 / Research program
Efficient learning. Broader discovery.
My research interests span resource-efficient models, AI-driven scientific discovery, and foundation models that support scientific reasoning.
AI for Science
Exploring how machine learning can support scientific discovery, model complex physical systems, and connect data-driven predictions with domain knowledge.
Scientific machine learning · Physics-informed learning · Surrogate modeling
Foundation Models for Science
Exploring how pretrained models can support scientific reasoning, learn from heterogeneous scientific data, and assist with complex research tasks.
Scientific foundation models · Multimodal learning · Scientific reasoning
From parameter counts to practical compute constraints.
The cost of adaptation
Illustrative LoRA parameter and memory estimates.
Trainable parameters
8.39 M
Parameter reduction
99.90 %
Estimated PEFT VRAM
6.6 GiB
Full fine-tuning · FP16 / Adam
121.2 GiB
Multi-GPU memory budget
LoRA · INT4 backbone
6.6 GiB
Within a 24 GiB consumer GPU budget
Estimation assumptions & limitations
Representative dense, non-GQA transformer: 32 layers, hidden width 4096, feed-forward width 14336. Not a named checkpoint. Q/V: 4Ldr; attention: 8Ldr; all linear: Lr[8d + 3(d + f)]. Embeddings and output head are excluded.
LoRA adapters remain mixed precision. Memory uses frozen weights × 1.2 for overhead, 16 bytes per trainable parameter for Adam-style training, and a fixed 2 GiB activation allowance. Full fine-tuning uses 16 bytes per backbone parameter + 2 GiB. 1 GiB = 2³⁰ bytes.
These are planning estimates, not measured performance. Sequence length, batch size, checkpointing, grouped-query attention, kernels, and quantization metadata change actual requirements. Parameter reduction is not VRAM reduction; 99.8% is not guaranteed.
03 / Publications
Published research.
Machine Learning / NLP Systems · Peer-Reviewed Journal Publication
Energy-Efficient NLP Through Tiny-Model Distillation, Pruning, and Quantized Inference
Hamadullah Bijarani · Kashf Journal of Multidisciplinary Research (KJMR), 2026
Model Compression / Quantization
Abstract
Transformer models achieve state-of-the-art NLP performance but face deployment constraints due to high inference energy costs. Numerous techniques have been proposed to optimize model operations. Due to limitations of hardware and time, we analyzed a few of them, which are knowledge distillation, unstructured pruning, and quantization within a unified pipeline for BERT and TinyBERT. Results demonstrate that the Distilled Student serves as the dominant baseline, offering the lowest energy per sample among all variants. Conversely, while post-training quantization and pruning reduce model size, they introduce significant kernel overheads on GPUs. Notably, combining INT8 quantization with aggressive pruning results in severe latency penalties, causing up to a 4× increase in energy consumption compared to the standard Student. These findings underscore the critical disparity between theoretical compression metrics and hardware-realized energy efficiency.
04 / Selected AI systems
Research ideas, working systems.
All repositories
01
RAG & NLP
Multi-Document Retrieval-Augmented Generation Architecture
A document-grounded question-answering pipeline spanning PDF parsing, chunking, embeddings, vector similarity search, and language-model generation.
Python · LangChain · FAISS · OpenAI API · Streamlit
GitHub source
02
RAG & NLP
Adaptive AI Study Coach & Semantic Memory Engine
An AI-assisted learning system with semantic retrieval, conversational question answering, quizzes, and flashcards. Developed as team lead at the PEC Generative AI Hackathon.
Python · Vector retrieval · SQLite · Streamlit
GitHub source
03
Deep Learning
High-Throughput Sentiment Classifier
Sentiment classification across 100,000+ customer reviews, bringing together pretrained embeddings, recurrent modeling, preprocessing, and model evaluation.
Python · LSTM · Pretrained embeddings · NLP
GitHub source
04
Agentic Orchestration
Autonomous Agent Orchestration & Tool-Calling Systems
Production AI workflows delivered for 50+ clients, covering tool-calling agents, document processing, API integrations, and automated business processes.
Python · LLM APIs · FastAPI · n8n · REST APIs
GitHub repositories05 / Education & experience
Academic foundations. Applied practice.
-
Aug 2023 — May 2027
Sukkur IBA University
B.E. Computer Systems Engineering
CGPA 3.48/4.00 · STHP & STFS Scholar · Expected graduation May 2027
-
2026 — Present
Independent AI Researcher
Natural language processing & efficient AI
Self-directed research in language models, model compression, parameter-efficient fine-tuning, and low-resource learning.
-
2026 — Present
Freelance AI Automation & Agent Engineer
Applied AI systems & orchestration
Delivered AI, automation, and software solutions for 50+ clients across RAG, LLM applications, and tool-calling workflows.
-
May — Jun 2025
KPMG Australia
Data Analyst Intern · Remote
Developed Tableau dashboards, performed statistical analysis for reporting and decision support, and automated routine data processing.
06 / Technical arsenal & honors
Tools of the work.
Programming
- Python (Advanced)
- C / C++
- CUDA · SQL · MATLAB
- Bash / Linux · LaTeX
ML & Deep Learning
- PyTorch
- Hugging Face Transformers
- TensorFlow / Keras · Scikit-Learn
- NumPy · OpenCV
NLP & Infrastructure
- PEFT · LoRA / DoRA
- Distillation · Quantization · RAG
- Docker · FastAPI
- Git · Linux
Scholarships & honors
- 01 Sukkur IBA STHP Merit Scholarship
- 02 PSF Science Talent Farming Scheme (STFS) National Scholarship
- 03 Team Lead, PEC Generative AI Hackathon
- 04 DeepLearning.AI Machine Learning & Deep Learning Specializations
- 05 Harvard CS50 / CS50’s Introduction to Artificial Intelligence