Abhirama Subramanyam

Email  /  LinkedIn  /  Google Scholar  /  GitHub  /  Twitter /  CV

I'm a PMRF Ph.D. scholar in the Vision, Language and Learning Group (VL2G) led by Dr.Anand Mishra at IIT Jodhpur. My research focuses on developing open-source retrieval augmented generation (RAG) enabled large multimodal models (LMMs), specifically for knowledge-intensive question- answering tasks over multimodal data. These tasks include variants of visual question answering (VQA) and audio question answering (AQA), which require external knowledge reasoning.

Research

We introduced the Model Parity Aligner (MPA), a label-free framework that improves Small Vision-Language Models (S-VLMs) by aligning them with larger models. Unlike traditional distillation, MPA relies only on unlabeled images and parity-based supervision to target knowledge gaps. Across four VQA benchmarks (TextVQA, ST-VQA, ChartQA, OKVQA), MPA consistently boosts S-VLM performance while retaining efficiency, and even leverages closed-source models like GPT-4o to help compact models rival or surpass much larger ones.

Introduced MMCricBench-3K, a VQA benchmark that evaluates structure-aware, cross-lingual, multi-image, and mathematical reasoning of large vision-language models on cricket scorecards. The dataset includes 3,000 QA pairs across 1,463 English and Hindi scorecards, spanning tasks from simple retrieval to complex numerical analysis.

Audiopedia is introduced (with 3 subtasks, s-AQA, m-AQA and r-AQA), a novel Audio QA task, requiring audio comprehension and external knowledge reasoning. Additionally, a framework that combines Audio Entity Linking (AEL) and a Knowledge-Augmented Audio Multimodal Model (KA²LM) is proposed to enhance large audio language models for knowledge-intensive tasks.

Text-KVQA is revisited with advancements in large multimodal models, introducing VisTEL, a method for visual text entity linking that leverages visual and textual cues. Additionally, KaLMA, a knowledge-aware assistant, is proposed to enhance LMMs by incorporating knowledge related to the visual text entity for improved accuracy.

RetVQA is introduced as a more challenging extension of traditional VQA, where a model retrieves relevant images from a pool to answer questions. The proposed MI-BART model, along with the new RETVQA dataset, achieves significant improvements in both accuracy and fluency over existing methods.

The COFAR dataset is introduced to evaluate image search involving commonsense and factual reasoning. To address this, KRAMT is proposed, integrating visual entities with encyclopedic knowledge and natural language queries for more accurate image retrieval.

Business logo identification in natural scenes using an open-set one-shot framework with multi-view textual-visual encoding, outperforming state-of-the-art techniques. The Wikidata Reference Logo Dataset (WiRLD) of 100,000 brand logos is introduced to study one-shot identification at scale.

The invention presents an intelligent recruitment management system that automates the recruitment process through a recruitment intelligence platform, utilizing modules for requisition parsing, resume analysis, candidate submissions, and job matching. This platform allows recruiters to track all steps of the recruitment process efficiently.

Template Source: John Barron