AI/ML engineer · Mumbai, India

Kartik Soni

I work on making machine learning models cheaper to run: 4-bit quantization, GPU kernels and retrieval systems. Final-year B.Tech in AI & ML at NMIMS, Mumbai.

Open to research internships, Dec 2026 – May 2027

Email me Resume (PDF)
  • 1st placeANRF AISEHack 2026, pollution forecasting
  • −68% memory4-bit Mistral-7B vs FP16, at about 5% more energy
  • 73.87%Recall@10 for offline search across text, images and audio
  • Days → hoursOcean-data reports with FloatChat
01

About

I like the parts of machine learning that sit inside the model and close to the hardware: what an LLM actually computes, where inference time, memory and energy go, and how to run a good model on modest infrastructure.

My B.Tech thesis measures the energy cost of 4-bit quantization, and my capstone is a Triton matmul kernel aimed at lowering GPU power draw. Before that I built a physics-informed PM2.5 forecaster that won ANRF AISEHack 2026, a satellite flood-damage mapper for a Devpost hackathon, an offline multimodal RAG system, and a natural-language interface for ocean data. In summer 2026 I interned as a data analyst at Bhilwara Halchal.

Inference efficiency
Quantization formats, throughput, and measuring energy as well as accuracy.
GPU kernels
Triton and CUDA C++, starting from matrix multiplication.
Retrieval systems
Vector search, multimodal embeddings and RAG pipelines that run offline.
Scientific ML
Physics-informed and satellite-based models for air quality and floods.
02

Research

B.Tech thesis · In progress · With two teammates, advised by Prof. Rajesh Verma

Is a smaller model greener? The energy cost of 4-bit LLMs

We compare the two 4-bit post-training quantization formats, FP4 and NF4, on energy use as well as accuracy. The early results cut against the assumption that smaller means greener.

Model
Mistral-7B in FP16, FP4 and NF4
Hardware
NVIDIA RTX PRO 6000
Quality
MMLU, WikiText-2 perplexity, hidden-state similarity to FP16
Cost
Throughput, and GPU power from NVML (idle-subtracted, PUE-scaled)
Protocol
Five trials in randomized order

Findings so far

  1. 4-bit cuts peak GPU memory from 14.5 GB to about 4.6 GB and raises throughput from 71.1 to about 90 tokens/s.
  2. It doesn't save energy. A full MMLU run uses about 5% more than FP16: 0.104 kWh (FP4) and 0.105 kWh (NF4) against 0.0994.
  3. NF4 beats FP4 on accuracy (0.491 vs 0.487), perplexity and hidden-state similarity, and uses slightly more energy.

Mistral-7B at three precisions

Preliminary results. Hover or focus a bar for details.

  • FP16
  • FP4
  • NF4
Peak GPU memoryGB, lower is better
FP1614.5
FP4≈4.6
NF4≈4.6

4-bit: −68% vs FP16

Throughputtokens/s, higher is better
FP1671.1
FP4≈90
NF4≈90

4-bit: +27% vs FP16

Energy for a full MMLU runkWh, lower is better
FP160.0994
FP40.104
NF40.105

FP4 +4.6% · NF4 +5.6% vs FP16

MMLU accuracyhigher is better
FP16n/a
FP40.487
NF40.491

NF4 leads FP4 by 0.004

View as table
MetricFP16FP4NF4
Peak GPU memory (GB)14.5≈4.6≈4.6
Throughput (tokens/s)71.1≈90≈90
Energy, full MMLU (kWh)0.09940.1040.105
MMLU accuracyn/a0.4870.491

4-bit memory and throughput are approximate and shared by FP4 and NF4. FP16 MMLU accuracy isn't reported here yet.

NextPhi-3, Gemma 3 and Qwen 2.5 7B, then a paper.

Capstone project · In progress

A matmul kernel that draws less power

Matrix multiplication dominates LLM inference cost, so any saving there carries into every model in the thesis. I'm writing my own matmul kernel in Triton aimed at lower GPU power draw, and I'll measure it with the same NVML harness so its numbers compare directly with the thesis results.

Research statement (PDF)

03

Projects

Physics-informed ML · Feb – Apr 2026

PM2.5 Forecaster

1st place, ANRF AISEHack 2026

Forecasts PM2.5 concentration fields over India up to 16 hours ahead. A ConvLSTM encoder reads 10 hours of weather and emissions data. The decoder rolls forward one hour at a time, and a WindWarp module moves pollution along the wind field with differentiable grid sampling, so transport follows the physics instead of being learned from scratch.

  1. 10 h of history15 weather & emission variables
  2. ConvLSTM encoderwith dilated convolutions
  3. WindWarp decoderadvection + spatial attention
  4. 16 h forecast140 × 124 grid over India
  • PyTorch
  • ConvLSTM
  • Physics-informed ML
  • Mixed precision
Code
Hackathon score (SMAPE-based)
89.25
Forecast horizon
16 h
Grid
140 × 124
Inputs
15 variables
Training
~10 h on one T4

Geospatial ML · Multimodal AI Hackathon 2026, Devpost

Baadhi Watch

Track B: Mapping flood damage from space

Give it an area and a flood date, and it maps where the water went, which buildings, roads and bridges were hit, and which settlements are cut off from hospitals and towns. It fuses Sentinel-1 radar, Sentinel-2 optical imagery and elevation data with pre-flood OpenStreetMap, then writes a situation report in English and Nepali.

  1. Before/after imagerySentinel-1 radar, Sentinel-2 optical, 30 m DEM
  2. Flood evidenceU-Net, radar change and terrain checks vote
  3. Damage overlaybuildings, roads and bridges from OSM
  4. Cut-off analysisroad-graph search from hospitals and towns
  • PyTorch
  • U-Net
  • ONNX
  • Sentinel-1/2
  • OpenStreetMap
Code
Flood-extent IoU vs Copernicus EMS
0.71
F1 score
0.83
Damaged buildings found
96%
Runtime from raw data
526 s
Case study
Nepal, Aug 2026

Database systems · 2026

MediCare HMS

A hospital management system for six roles, from receptionist to pharmacist. Each signed-in user's queries run as their own PostgreSQL role, so grants, row-level security and triggers enforce the rules inside the database.

  • Row-level security limits doctors to their own records
  • Row locks on payments and dispensing prevent double charges and overselling stock
  • bcrypt password hashing, with lockout after 5 failed logins
  • PostgreSQL
  • PL/pgSQL
  • Streamlit
  • pytest
Code

Retrieval · Aug – Oct 2025

Offline Multimodal RAG

Searches text, images and audio with one query, entirely on your own machine with no cloud APIs. Images and audio share a CLIP embedding space, so a spoken phrase can find a photo.

  • 73.87% Recall@10 across 2,000+ documents and 500 audio clips
  • Whisper transcription, CLIP and sentence-transformer embeddings, FAISS index
  • Separate FastAPI services for ingestion and retrieval, with a Dash dashboard
  • FAISS
  • CLIP
  • Whisper
  • FastAPI
Code

LLM agents · Dec 2025 – Feb 2026

FloatChat

A natural-language interface for Argo ocean-float data from the Indian Ocean, so researchers can explore it without writing scripts.

  • Report prep cut from 2–3 days to a few hours, about 4× less working time
  • A LangGraph agent queries PostgreSQL through a custom MCP server
  • Maps float tracks and plots T-S diagrams, BGC parameters and depth profiles
  • LangGraph
  • MCP
  • PostgreSQL
  • Plotly
Live demo Code

GPU systems · Aug 2026 – present

Koala

In development

A vector search library I'm writing from scratch. A flat in-memory index runs on Triton GPU kernels and falls back to PyTorch on CPU.

  • Exact nearest-neighbour search with custom Triton kernels
  • Falls back to PyTorch when no CUDA device is present
  • Micro-benchmark harness and pytest suite
  • Triton
  • PyTorch
  • pytest
Code

More on GitHub

  • Support ticket automation

    Fine-tuned DistilBERT on Banking77 to sort support queries into 77 intents, set a priority, and send low-confidence cases to a human.

  • Document query system

    Question answering over PDF, Word, Excel, PowerPoint and images, with FAISS retrieval, query routing and a FastAPI backend.

  • Market anomaly detection

    A three-stage detector for NIFTY indices using rolling z-scores and Mahalanobis distance, aimed at spotting speculative bubbles.

  • GPT-2 from scratch

    A GPT-2-style decoder written by hand in PyTorch: causal multi-head attention, the training loop and greedy decoding.

  • CUDA practice

    CUDA C++ fundamentals: host and device memory, kernel launches and the runtime API.

04

Experience

  1. May – Aug 2026

    Data Analysis Intern

    Bhilwara Halchal

    • Analysed app-based news consumption data for engagement trends, content performance and readership patterns
    • Cleaned and processed datasets with SQL, Python and Excel
    • Built reports and visualizations that tracked key metrics for the team

  2. Oct 2025 – present

    Member & Volunteer

    ACM Student Chapter, NMIMS Mumbai

Education

  1. Jul 2023 – Sep 2027

    B.Tech, Artificial Intelligence and Machine Learning

    SVKM's NMIMS MPSTME, Mumbai

    Thesis on the energy cost of 4-bit LLM quantization. Capstone on a low-power Triton matmul kernel.

05

Skills

Languages

  • Python
  • C++
  • CUDA C++
  • SQL
  • PL/pgSQL

Machine learning

  • PyTorch
  • TensorFlow
  • scikit-learn
  • Hugging Face Transformers
  • ONNX
  • AutoML

GPU & efficiency

  • Triton
  • 4-bit quantization (FP4, NF4)
  • NVML power measurement
  • Mixed precision

LLMs & retrieval

  • RAG pipelines
  • FAISS
  • sentence-transformers
  • CLIP
  • Whisper
  • LangChain & LangGraph
  • MCP
  • OpenAI & Claude APIs

Data

  • NumPy
  • Pandas
  • SciPy
  • xarray
  • NetCDF
  • Sentinel-1/2 imagery
  • OpenStreetMap
  • Excel
  • Power BI
  • Tableau

Backend & tools

  • PostgreSQL
  • MySQL
  • SQL Server
  • FastAPI
  • Flask
  • Streamlit
  • pytest
  • Git
  • Linux
  • AWS (SageMaker, S3)

Awards

  • Apr 20261st place, ANRF AISEHack 2026Theme 2: Pollution forecasting
  • Oct 20252nd place, Yugantar 2025Smart India Hackathon internal round

Certifications

  • NPTELMachine Learning with Python Programming
  • NPTELDeep LearningIIT Ropar
  • GoogleData Analytics
  • AWSAWS Builder Essentials
06

Contact

I'm looking for a six-month research internship from December 2026 to May 2027, ideally on inference efficiency. Email is the fastest way to reach me.

Mumbai, India · IST (UTC+5:30)