arxiv:2609.31847

Omni-IO Skills: Harnessing Your Agent Omni-Native

Published on Sep 25

· Submitted by

Yanlin Li on Sep 30

National University of Singapore

Upvote

268

Authors:

Yanlin Li ,

Hao Fei ,

Abstract

General-purpose agents can plan, reason, and act over long horizons, yet their production capabilities remain fragmented across text, images, audio, video, documents, 3D assets, and code. Extending a foundation model to additional modalities ties capability growth to costly model updates, while assembling specialist models and tools leaves unresolved how procedures, dependencies, intermediate assets, and cross-turn revisions should be coordinated. We present Omni-IO Skills, a plug-and-play Agent Harness that makes existing agents omni-native through hierarchical Skills, a standardized multimodal execution interface, dependency-aware orchestration, and a persistent Asset Registry. Multi-asset workflows are represented as Declare Execution Graphs, which schedule independent operations concurrently and register successful outputs for downstream and cross-turn reuse across replaceable execution backends. Its 27 Skills cover 38 representative tasks spanning seven artifact modalities and four capability families: understanding, generation, reasoning, and retrieval. On UniM-90, the harness raises the input-support rates of GPT-5.6 Sol and Claude Sonnet 5 from 40.00% and 38.89% to 100%, while increasing relative Semantic--Quality Coupled Score from 26.99 to 74.94 and from 27.82 to 77.78, respectively; Strict Structure Score reaches 100.00 and 99.78. These results establish harness-level capability composition as a practical route to broad, evolvable Omni systems without changing the host agent's reasoning core.

View arXiv page View PDF Project page GitHub 33 Add to collection

Community

yanlinli

Paper author Paper submitter 1 day ago

Hi everyone! Sharing our new work: Omni-IO Skills: Harnessing Your Agent Omni-Native 🚀

Your agent is only one harness away from being omni-native!

General-purpose agents like Claude and Codex are strong at reasoning and planning, but audio, video, 3D, and coordinated media workflows still depend on external tools. Extending a foundation model to new modalities ties capability growth to costly retraining. Real tasks also never stay within one modality. For example, "watch this video, compose a BGM, and turn it into a slide deck with a cover" chains understanding, generation, and conversion, and reuses intermediate assets across turns.

Research question: Can a plug-and-play harness make an existing general-purpose agent omni-native while preserving its reasoning and planning core?

Our answer: Omni-IO Skills, a multimodal harness that wraps around the agent instead of replacing it:

🧩 Hierarchical Skills: 27 Skills covering 38 representative tasks across 7 artifact modalities and 4 capability families (understanding, generation, reasoning, retrieval)
🔀 Declarative execution graphs: multi-step workflows are planned as dependency graphs, and independent tasks run in parallel
🗂️ Asset Registry: intermediate outputs persist and are reused across turns
🔌 Pluggable backends: switching providers only requires a config change

On UniM, with two host agents (GPT-5.6 Sol and Claude Sonnet 5), support rates rise from under 40% to 100%, and the SQCS rises from about 27% to 75–78%.

Code, along with 20 example workflows, is open-sourced. Feel free to read, share, and star! 🙏

📝 Paper: https://arxiv.org/abs/2609.31847
💻 Code: https://github.com/any2any-mllm/Omni-IO-Skill