Exclusive | JD.com to Open Source Real-Time Visual Language Interaction Model

According to Mars Finance, a reporter from the Science and Technology Innovation Board Daily has exclusively learned that JD.com's team is about to open-source its real-time visual language interaction model, JoyAI-VL-Interaction. A technical report that has garnered attention in the open-source community and overseas AI technology circles reveals that JoyAI-VL-Interaction advances multimodal large-scale models from "question-and-answer" to "real-time streaming interaction," making it suitable for scenarios requiring continuous AI presence. The report conducted manual evaluations of JoyAI-VL-Interaction against the in-app video call assistants of Doubao and Gemini, covering six scenarios: monitoring and alerting, real-time counting, real-time translation, time awareness, live commentary and guidance, and long-term memory. In 58 cases, JoyAI-VL-Interaction achieved an overall win rate of 77.6% against Doubao and 87.9% against Gemini. Notably, in the monitoring and alerting scenario, it achieved a 100% win rate against both baselines. (Reporter: Huang Xinyi, Science and Technology Innovation Board Daily)