publications

2027

1. ICWSM

   

   Echoes of Human Malice in Agents: Benchmarking LLMs for Multi-Turn Online Harassment Attacks

   Trilok Padhi, Pinxian Lu, Abdulkadir Erol, and 5 more authors

   In Proceedings of the International AAAI Conference on Web and Social Media (ICWSM) (equal contribution with P. Lu), 2027

   Abs arXiv Bib HTML

   Large Language Model (LLM) agents are powering a growing share of interactive web applications, yet remain vulnerable to misuse and harm. Prior jailbreak research has largely focused on single-turn prompts, whereas real harassment often unfolds over multi-turn interactions. In this work, we present the Online Harassment Agentic Benchmark consisting of: (i) a synthetic multi-turn harassment conversation dataset, (ii) a multi-agent (e.g., harasser, victim) simulation informed by repeated game theory, (iii) three jailbreak methods attacking agents across memory, planning, and fine-tuning, and (iv) a mixed-methods evaluation framework. We utilize two prominent LLMs, LLaMA-3.1-8B-Instruct (open-source) and Gemini-2.0-flash (closed-source). Our results show that jailbreak tuning makes harassment nearly guaranteed with an attack success rate of 95.78–96.89% vs. 57.25–64.19% without tuning in Llama, and 99.33% vs. 98.46% without tuning in Gemini, while sharply reducing refusal rate to 1-2% in both models. The most prevalent toxic behaviors are Insult with 84.9–87.8% vs. 44.2–50.8% without tuning, and Flaming with 81.2–85.1% vs. 31.5–38.8% without tuning, indicating weaker guardrails compared to sensitive categories such as sexual or racial harassment. Qualitative evaluation further reveals that attacked agents reproduce human-like aggression profiles, such as Machiavellian/psychopathic patterns under planning, and narcissistic tendencies with memory. Counterintuitively, closed-source and open-source models exhibit distinct escalation trajectories across turns, with closed-source models showing significant vulnerability. Overall, our findings show that multi-turn and theory-grounded attacks not only succeed at high rates but also mimic human-like harassment dynamics, motivating the development of robust safety guardrails to ultimately keep online platforms safe and responsible.

   @inproceedings{padhi2027echoes,
     title = {Echoes of Human Malice in Agents: Benchmarking LLMs for Multi-Turn Online Harassment Attacks},
     author = {Padhi, Trilok and Lu, Pinxian and Erol, Abdulkadir and Sutar, Tanmay and Sharma, Gauri and Sonmez, Mina and De Choudhury, Munmun and Kursuncu, Ugur},
     booktitle = {Proceedings of the International AAAI Conference on Web and Social Media (ICWSM)},
     publisher = {Association for the Advancement of Artificial Intelligence},
     year = {2027},
   }

2026

1. ICML WS

   

   From Actions to Understanding: Conformal Interpretability of Temporal Concepts in LLM Agents

   Trilok Padhi, Ramneet Kaur, Krishiv Agarwal, and 9 more authors

   In Mechanistic Interpretability Workshop, 43rd International Conference on Machine Learning (ICML), Seoul, South Korea, 2026

   Abs arXiv Bib HTML

   Large Language Models (LLMs) are increasingly deployed as autonomous agents capable of reasoning, planning, and acting within interactive environments. Despite their growing capability to perform multi-step reasoning and decision-making tasks, internal mechanisms guiding their sequential behavior remain opaque. This paper presents a framework for interpreting the temporal evolution of concepts in LLM agents through a step-wise conformal lens. We introduce the conformal interpretability framework for temporal tasks, which combines step-wise reward modeling with conformal prediction to statistically label model’s internal representation at each step as successful or failing. Linear probes are then trained on these representations to identify directions of temporal concepts - latent directions in the model’s activation space that correspond to consistent notions of success, failure or reasoning drift. Experimental results on two simulated interactive environments, namely ScienceWorld and AlfWorld, demonstrate that these temporal concepts are linearly separable, revealing interpretable structures aligned with task success. We further show preliminary results on improving an LLM agent’s performance by leveraging the proposed framework for steering the identified successful directions inside the model. The proposed approach, thus, offers a principled method for early failure detection as well as intervention in LLM-based agents, paving the path towards trustworthy autonomous language models in complex interactive settings.

   @inproceedings{padhi2026actions,
     title = {From Actions to Understanding: Conformal Interpretability of Temporal Concepts in LLM Agents},
     author = {Padhi, Trilok and Kaur, Ramneet and Agarwal, Krishiv and Cobb, Adam D. and Elenius, Daniel and Acharya, Manoj and Samplawski, Colin and Berenbeim, Alexander M. and Bastian, Nathaniel D. and Jha, Susmit and Kursuncu, Ugur and Roy, Anirban},
     booktitle = {Mechanistic Interpretability Workshop, 43rd International Conference on Machine Learning (ICML)},
     location = {Seoul, South Korea},
     year = {2026},
   }

2. ACM TIST

   

   Playing Devil’s Advocate: Unmasking Toxicity and Vulnerabilities in Large Vision-Language Models

   Abdulkadir Erol, Trilok Padhi, Agnik Saha, and 2 more authors

   ACM Transactions on Intelligent Systems and Technology, 2026

   Abs arXiv Bib HTML

   The rapid advancement of Large Vision-Language Models (LVLMs) has enhanced capabilities offering potential applications from content creation to productivity enhancement. Despite their innovative potential, LVLMs exhibit vulnerabilities, especially in generating potentially toxic or unsafe responses. Malicious actors can exploit these vulnerabilities to propagate toxic content in an automated (or semi-) manner, leveraging the susceptibility of LVLMs to deception via strategically crafted prompts without fine-tuning or compute-intensive procedures. Despite the red-teaming efforts and inherent potential risks associated with the LVLMs, exploring vulnerabilities of LVLMs remains nascent and yet to be fully addressed in a systematic manner. This study systematically examines the vulnerabilities of open-source LVLMs, including LLaVA, InstructBLIP, Fuyu, and Qwen, using adversarial prompt strategies that simulate real-world social manipulation tactics informed by social theories. Our findings show that (i) toxicity and insulting are the most prevalent behaviors, with the mean rates of 16.13% and 9.75%, respectively; (ii) Qwen-VL-Chat, LLaVA-v1.6-Vicuna-7b, and InstructBLIP-Vicuna-7b are the most vulnerable models, exhibiting toxic response rates of 21.50%, 18.30% and 17.90%, and insulting responses of 13.40%, 11.70% and 10.10%, respectively; (iii) prompting strategies incorporating dark humor and multimodal toxic prompt completion significantly elevated these vulnerabilities. Despite being fine-tuned for safety, these models still generate content with varying degrees of toxicity when prompted with adversarial inputs, highlighting the urgent need for enhanced safety mechanisms and robust guardrails in LVLM development.

   @article{erol2026devils,
     title = {Playing Devil's Advocate: Unmasking Toxicity and Vulnerabilities in Large Vision-Language Models},
     author = {Erol, Abdulkadir and Padhi, Trilok and Saha, Agnik and Kursuncu, Ugur and Aktas, Mehmet Emin},
     journal = {ACM Transactions on Intelligent Systems and Technology},
     publisher = {Association for Computing Machinery},
     year = {2026},
   }

2025

1. Preprint

   

   Co-Evolving Agents: Learning from Failures as Hard Negatives

   Yeonsung Jung, Trilok Padhi, Sina Shaham, and 4 more authors

   arXiv preprint arXiv:2511.22254 (under review), 2025

   Abs arXiv Bib HTML

   The rapid progress of large foundation models has accelerated the development of task-specialized agents across diverse domains. However, the effectiveness of agents remains tightly coupled with the quality of training data, while curating task-specific datasets remains costly and often infeasible in real-world scenarios. Recent work has explored self-improving agents that autonomously generate, refine, and re-train on their own trajectories. A prominent line of approaches further leverages preference optimization by pairing predicted trajectories with scarce ground-truth trajectories, enabling agents to learn directly from their own failures. While these methods outperform supervised fine-tuning, their heavy reliance on predicted trajectories under limited ground-truth supervision leaves them prone to overfitting. To address this, we propose a co-evolving agents framework in which a target agent improves jointly with an auxiliary failure agent. The failure agent learns through preference optimization over failure trajectories from both the target and itself, thereby generating hard negatives that are close to success yet remain failures. Incorporating these informative hard negatives into the target agent’s optimization sharpens decision boundaries and enhances generalization. Our comprehensive analysis and experiments across benchmark datasets show that our method not only shows improved performance but also demonstrates that failures, instead of being used as-is, can be systematically transformed into structured and valuable learning signals in self-improving agents.

   @article{jung2025coevolving,
     title = {Co-Evolving Agents: Learning from Failures as Hard Negatives},
     author = {Jung, Yeonsung and Padhi, Trilok and Shaham, Sina and Khullar, Dipika and Jeong, Joonhyun and Mehrabi, Ninareh and Yang, Eunho},
     journal = {arXiv preprint arXiv:2511.22254},
     year = {2025},
   }

2. ACL Findings

   

   Just KIDDIN’: Knowledge Infusion and Distillation for Detection of INdecent Memes

   Rahul Garg, Trilok Padhi, Hemang Jain, and 2 more authors

   In Findings of the Association for Computational Linguistics: ACL 2025 (equal contribution with R. Garg; acceptance rate 19.1%), 2025

   Abs arXiv Bib HTML

   Toxicity identification in online multimodal environments remains a challenging task due to the complexity of contextual connections across modalities (e.g., textual and visual). In this paper, we propose a novel framework that integrates Knowledge Distillation (KD) from Large Visual Language Models (LVLMs) and knowledge infusion to enhance the performance of toxicity detection in hateful memes. Our approach extracts sub-knowledge graphs from ConceptNet, a large-scale commonsense Knowledge Graph (KG) to be infused within a compact VLM framework. The relational context between toxic phrases in captions and memes, as well as visual concepts in memes enhance the model’s reasoning capabilities. Experimental results from our study on two hate speech benchmark datasets demonstrate superior performance over the state-of-the-art baselines across AU-ROC, F1, and Recall with improvements of 1.1%, 7%, and 35%, respectively. Given the contextual complexity of the toxicity detection task, our approach showcases the significance of learning from both explicit (i.e. KG) as well as implicit (i.e. LVLMs) contextual cues incorporated through a hybrid neurosymbolic approach. This is crucial for real-world applications where accurate and scalable recognition of toxic content is critical for creating safer online environments.

   @inproceedings{garg2025kiddin,
     title = {Just KIDDIN': Knowledge Infusion and Distillation for Detection of INdecent Memes},
     author = {Garg, Rahul and Padhi, Trilok and Jain, Hemang and Kursuncu, Ugur and Kumaraguru, Ponnurangam},
     booktitle = {Findings of the Association for Computational Linguistics: ACL 2025},
     publisher = {Association for Computational Linguistics},
     year = {2025},
   }

3. Preprint

   

   From Reddit to Generative AI: Evaluating Large Language Models for Anxiety Support Fine-tuned on Social Media Data

   Ugur Kursuncu, Trilok Padhi, Gaurav Sinha, and 3 more authors

   arXiv preprint arXiv:2505.18464 (under review), 2025

   Abs arXiv Bib HTML

   The growing demand for accessible mental health support, compounded by workforce shortages and logistical barriers, has led to increased interest in utilizing Large Language Models (LLMs) for scalable and real-time assistance. However, their use in sensitive domains such as anxiety support remains underexamined. This study presents a systematic evaluation of LLMs (GPT and Llama) for their potential utility in anxiety support by using real user-generated posts from the r/Anxiety subreddit for both prompting and fine-tuning. Our approach utilizes a mixed-method evaluation framework incorporating three main categories of criteria: (i) linguistic quality, (ii) safety and trustworthiness, and (iii) supportiveness. Results show that fine-tuning LLMs with naturalistic anxiety-related data enhanced linguistic quality but increased toxicity and bias, and diminished emotional responsiveness. While LLMs exhibited limited empathy, GPT was evaluated as more supportive overall. Our findings highlight the risks of fine-tuning LLMs on unprocessed social media content without mitigation strategies.

   @article{kursuncu2025reddit,
     title = {From Reddit to Generative AI: Evaluating Large Language Models for Anxiety Support Fine-tuned on Social Media Data},
     author = {Kursuncu, Ugur and Padhi, Trilok and Sinha, Gaurav and Erol, Abdulkadir and Mandivarapu, Jaya Krishna and Larrison, Christopher R.},
     journal = {arXiv preprint arXiv:2505.18464},
     year = {2025},
   }

4. Preprint

   

   Seeing is Believing: Calibrating Uncertainty Quantification of Multi-Modal LLMs using Grounding

   Trilok Padhi, Ramneet Kaur, Adam D. Cobb, and 7 more authors

   arXiv preprint arXiv:2505.03788 (under review), 2025

   Abs arXiv Bib HTML

   We introduce a novel approach for calibrating uncertainty quantification (UQ) tailored for multi-modal large language models (LLMs). Existing state-of-the-art UQ methods rely on consistency among multiple responses generated by the LLM on an input query under diverse settings. However, these approaches often report higher confidence in scenarios where the LLM is consistently incorrect. This leads to a poorly calibrated confidence with respect to accuracy. To address this, we leverage cross-modal consistency in addition to self-consistency to improve the calibration of the multi-modal models. Specifically, we ground the textual responses to the visual inputs. The confidence from the grounding model is used to calibrate the overall confidence. Given that using a grounding model adds its own uncertainty in the pipeline, we apply temperature scaling - a widely accepted parametric calibration technique - to calibrate the grounding model’s confidence in the accuracy of generated responses. We evaluate the proposed approach across multiple multi-modal tasks, such as medical question answering (Slake) and visual question answering (VQAv2), considering multi-modal models such as LLaVA-Med and LLaVA. The experiments demonstrate that the proposed framework achieves significantly improved calibration on both tasks.

   @article{padhi2025seeing,
     title = {Seeing is Believing: Calibrating Uncertainty Quantification of Multi-Modal LLMs using Grounding},
     author = {Padhi, Trilok and Kaur, Ramneet and Cobb, Adam D. and Acharya, Manoj and Roy, Anirban and Samplawski, Colin and Matejek, Brian and Berenbeim, Alexander M. and Bastian, Nathaniel D. and Jha, Susmit},
     journal = {arXiv preprint arXiv:2505.03788},
     year = {2025},
   }

2024

1. IEEE BigData

   

   Enhancing Cross-Modal Contextual Congruence for Crowdfunding Success using Knowledge-infused Learning

   Trilok Padhi, Ugur Kursuncu, Yaman Kumar, and 2 more authors

   In 2024 IEEE International Conference on Big Data (BigData) (acceptance rate 18.7%), 2024

   Abs arXiv Bib HTML

   The digital landscape continually evolves with multimodality, enriching the online experience for users. Creators and marketers aim to weave subtle contextual cues from various modalities into congruent content to engage users with a harmonious message. This interplay of multimodal cues is often a crucial factor in attracting users’ attention. However, this richness of multimodality presents a challenge to computational modeling, as the semantic contextual cues spanning across modalities need to be unified to capture the true holistic meaning of the multimodal content. This contextual meaning is critical in attracting user engagement as it conveys the intended message of the brand or the organization. In this work, we incorporate external commonsense knowledge from knowledge graphs to enhance the representation of multimodal data using compact Visual Language Models (VLMs) and predict the success of multi-modal crowdfunding campaigns. Our results show that external knowledge commonsense bridges the semantic gap between text and image modalities, and the enhanced knowledge-infused representations improve the predictive performance of models for campaign success upon the baselines without knowledge. Our findings highlight the significance of contextual congruence in online multimodal content for engaging and successful crowdfunding campaigns.

   @inproceedings{padhi2024congruence,
     title = {Enhancing Cross-Modal Contextual Congruence for Crowdfunding Success using Knowledge-infused Learning},
     author = {Padhi, Trilok and Kursuncu, Ugur and Kumar, Yaman and Shalin, Valerie L. and Fronczek, Lane Peterson},
     booktitle = {2024 IEEE International Conference on Big Data (BigData)},
     publisher = {IEEE},
     year = {2024},
   }

2020

1. Evolving Sys.

   MFOFLANN: Moth Flame Optimized Functional Link Artificial Neural Network for Prediction of Earthquake Magnitude

   Santosh Kumar Majhi, Sk Sajeed Hossain, and Trilok Padhi

   Evolving Systems, 2020

   DOI Bib HTML

   @article{majhi2020mfoflann,
     title = {MFOFLANN: Moth Flame Optimized Functional Link Artificial Neural Network for Prediction of Earthquake Magnitude},
     author = {Majhi, Santosh Kumar and Hossain, Sk Sajeed and Padhi, Trilok},
     journal = {Evolving Systems},
     volume = {11},
     number = {1},
     pages = {45--63},
     year = {2020},
     publisher = {Springer},
     doi = {10.1007/s12530-019-09293-6},
   }