KDD'25 · Benchmark & Dataset Track
HiBench: Benchmarking LLMs Capability on Hierarchical Structure Reasoning
Zhuohang Jiang, Pangjing Wu, Ziran Liang, Peter Q. Chen, Xu Yuan, Ye Jia, Jiancheng
Tu, Chen Li, Peter H.F. Ng, Qing Li
Hierarchical ReasoningLLM Benchmark30
Tasks39,519 Queries
As first author and project lead, I led benchmark design and code development for evaluating hierarchical structure reasoning in LLMs. I built tree, JSON and formula generators with controlled depth, width and node counts, algorithmic ground-truth answers and unified prompts, and incorporated code and paper tasks. The benchmark contains 30 tasks and 39,519 queries.
We evaluated 20 LLMs from 10 families, comparing structural representations, complexity and chain-of-thought prompting. I constructed instruction-tuning data that yielded average relative gains of 88.84% for Llama-3.1-8B and 31.38% for Qwen2.5-7B. The dataset and modular evaluation toolkit are open-source, supporting reproducible evaluation and further research on structure-aware reasoning.
作为第一作者与项目负责人,负责层次结构推理基准设计与代码统筹。控制树深度、宽度与节点数,实现树、JSON、公式生成、算法标答与统一提示,并纳入代码和论文任务;共构建 30 项任务、39,519 条样例。
对 10 个家族的 20 个 LLM 开展结构表示、复杂度与 CoT 对比评测;构造指令微调数据,使 Llama-3.1-8B、Qwen2.5-7B 分别获得 88.84%、31.38% 的平均相对提升。开源数据与模块化评测工具库,支持结构化推理的复现、评测与后续研究。
CVPR'26 Findings
SUPERGLASSES: Benchmarking Vision Language Models as Intelligent Agents for AI
Smart Glasses
Zhuohang Jiang, Xu Yuan, Haohao Qu, Shanru Lin, Kanglong Liu, Wenqi Fan, Qing Li
AI Smart GlassesVQA BenchmarkVision-Language
ModelsMultimodal RAG
As first author, I built SUPERGLASSES, a benchmark for evaluating vision-language models as smart-glasses agents, and SUPERLENS, the accompanying multimodal agent. The real-device dataset contains 2,422 QA pairs across 14 domains and 8 question types, with tool-use traces. Domain routing selects direct answering or RAG, while open-vocabulary detection localizes small objects and multi-hop questions are decomposed into single-hop subqueries.
The agent searches image and text evidence, cleans HTML and reranks evidence chunks. A two-level Redis cache reuses query-to-URL and URL-to-content results. We evaluated 26 model/RAG variants; SUPERLENS improved the Qwen2.5-VL-7B score from 32.82 to 44.10, a gain of 11.28 points.
作为第一作者,构建智能眼镜视觉语言模型评测基准 SUPERGLASSES 与多模态智能体 SUPERLENS。基于真实设备采集 2,422 个问答对,覆盖 14 个领域、8 类问题,并保留工具调用轨迹。通过领域路由选择直接回答或 RAG,以开放词汇检测定位小目标,并将多跳问题拆为单跳子查询。
智能体执行图像与文本搜索、HTML 清洗及证据分块重排;采用两级 Redis 缓存复用“查询到 URL、URL 到正文”的结果。评测 26 个模型/RAG 变体,基于 Qwen2.5-VL-7B 的 SUPERLENS 得分由 32.82 提至 44.10,提升 11.28 分。
arXiv Preprint
A/B Agent: A Self-Evolving Agent for Strategy Iteration in Industrial A/B Testing
Zhuohang Jiang, Yuxin Chen, Yongsen Pan, Zheng Hu, Wenqi Fan, Qing Li, Hongyang
Wang, Jun Wang, Wenwu Ou
Self-Evolving AgentsTree-RAGA/B
TestingStrategy Optimization
As first author, I developed a self-evolving LLM agent for strategy iteration in industrial A/B testing. It organizes experience by business domain, scenario, stage and optimization objective. Hybrid sparse/dense retrieval, tree-path proximity weighting and Qwen-Reranker retrieve relevant evidence; generated strategy configurations are checked for engineering feasibility and guardrail risks.
The experiment tree records strategies, parameters, metrics and context, enabling parent-child and sibling-branch comparisons. Confidence-weighted gains and guardrail-loss penalties guide local search within valid parameter ranges, or a switch to another strategy branch when gains stall. Recording successful and failed trials provides a reusable feedback loop for subsequent iterations. This work is a 2026 arXiv preprint.
作为第一作者,研发面向工业 A/B 测试策略迭代的自进化大模型智能体。按业务域、场景、阶段与优化目标组织经验树;融合稀疏与稠密检索,结合树路径邻近度加权与 Qwen-Reranker 精排,为策略配置生成检索历史证据,并检查工程可行性与护栏风险。
以实验树节点记录策略、参数、指标与上下文,进行父子版本和兄弟分支对比;结合置信度加权收益与护栏损失惩罚,决定在有效参数区间局部搜索,或在收益停滞时切换策略分支。成功与失败实验持续沉淀为后续迭代可复用的经验。该工作为 2026 年 arXiv 预印本。
KDD'25 Workshop
QA‑Dragon: Query‑Aware Dynamic RAG System for Knowledge‑Intensive Visual Question
Answering
Zhuohang Jiang, Pangjing Wu, Xu Yuan, Wenqi Fan, Qing Li
Query-Aware RAGKnowledge-Intensive VQAMulti-Hop
Reasoning
I am the first author of QA-Dragon, a multimodal retrieval-augmented generation system for complex visual questions. The system drafts a candidate answer first, then decides in a query-aware manner whether that draft should be returned directly, verified through search, or handed to the retrieval-augmented pipeline.
Tool selection follows the detected knowledge gap of the draft, and the retrieval pipeline combines target cropping, image and text retrieval, evidence reranking, and answer verification to ground the final answer in retrieved evidence. Under single-source, multi-source, and multi-turn question settings, the system improves over the baseline by 5.06%, 6.35%, and 5.03% respectively.
我是 QA-Dragon 的第一作者,这是一个面向复杂视觉问题的多模态检索增强生成系统。系统先生成一个候选答案,再以问题感知的方式判断该草稿应当直接输出、交由搜索验证,还是进入检索增强流程。
工具选择由草稿所暴露的知识缺口决定,检索侧结合目标裁剪、图像与文本检索、证据重排与答案校验,利用检索证据支撑最终回答。在单源、多源与多轮问题设定下,较基线分别提升 5.06%、6.35% 与 5.03%。
Atomic Intent Reasoning: Bringing LLM Semantics to Industrial Cross-Domain
Recommendations
Zhuohang Jiang, Yuxin Chen, Shijie Wang, Haohao Qu, Zhou Jindong, Wenqi Fan, Li
Qing, Dongxu Liang, Jun Wang
LLM ReasoningAtomic IntentsOffline-to-Online RetrievalIndustrial Deployment
As first author, I developed AIR to decouple LLM intent reasoning from online serving. Offline, an LLM maps behavior records, user attributes and object descriptions into hierarchical intent paths, stored as reusable atomic knowledge. This moves semantic generation out of the online request path and makes the resulting knowledge cacheable and composable.
Online, AIR builds intent trees with behavior weights and time decay, retrieves target-relevant intent chains and fuses evidence with multi-head attention. Retrieval and composition approximate sequence-level semantic reasoning without live LLM calls. The system achieves approximately 400× throughput improvement, demonstrating an efficient path from offline LLM-generated knowledge to online evidence retrieval and composition.
作为第一作者,研发 AIR,实现大模型意图推理与在线服务解耦。离线将行为记录、用户属性与对象描述输入 LLM,生成分层语义意图路径并存储为可复用的原子知识,将语义生成移出在线请求链路,使知识能够缓存与组合。
在线按行为权重与时间衰减构建意图树,检索目标相关意图链,并以多头注意力融合证据;通过检索与组合近似序列级语义推理,无需实时调用 LLM,实现约 400× 吞吐提升。重点在于将离线生成的语义知识转化为在线可检索、可组合的证据。
KDD'25
A Survey of WebAgents: Towards Next-Generation AI Agents for Web Automation with
Large Foundation Models
Liangbo Ning, Ziran Liang, Zhuohang Jiang, Haohao Qu, Yujuan Ding, Wenqi Fan,
Xiao-yong Wei, Shanru Lin, Hui Liu, Philip S. Yu, Qing Li
Web AgentsFoundation ModelsWeb
AutomationTrustworthiness
A survey of web automation agents built on large foundation models. This work complements my implementation research on retrieval and tool-using agents.
围绕大基础模型驱动的网页自动化智能体开展综述研究,与我的检索增强、工具使用和智能体系统实践形成互补。