Core Viewpoint - The article introduces the launch of 可灵's O1, a unified video and image generation and editing tool that integrates multiple tasks into a single interface, allowing for seamless video and image editing and generation. Group 1: Features of O1 - O1 integrates multi-modal video models, combining reference videos, text-to-video, frame manipulation, content addition/removal, and style redrawing into a one-stop solution for generation and modification [2]. - It supports multi-modal inputs including images, videos, subjects, and text, enabling precise editing through natural language without the need for masks or keyframes [2][4]. - The tool maintains consistency in character, props, and scene features across shots through multi-angle subjects and reference materials, ensuring coherent visuals [2]. Group 2: Editing Capabilities - Users can generate narrative shots lasting approximately 3 to 10 seconds, allowing for flexible control over pacing and shot length [2]. - The editing process allows for direct modifications through text prompts, where users can upload videos and specify changes using references [4][6]. - O1 supports the use of single or multiple reference images for background or character modifications, enhancing the realism of the final output [7]. Group 3: Subject Creation and Consistency - O1 introduces a new element called "subject," which allows users to create and select characters for easier integration into videos without frequent uploads [10][13]. - Users can upload multiple images from different angles to improve consistency in character and scene representation during video generation [13][17]. - The tool is particularly beneficial for e-commerce, as it ensures that products remain consistent in appearance during various camera movements [17]. Group 4: Style and Frame Generation - O1 allows users to convert video styles easily, supporting various artistic styles such as felt, anime, and 8-bit pixel [19]. - The tool also supports frame generation, enabling users to create complex effects by combining image references with frame inputs [20][21]. - The overall capabilities of O1 in video editing are seen as a significant advancement, with the potential for creating impressive effects with minimal effort [29].

Core Viewpoint - The article discusses the advancements in video editing through the introduction of the VIRES method, which combines sketch and text guidance for video instance repainting, significantly improving editing efficiency and accuracy in complex scenes [2][10][31]. Group 1: VIRES Methodology - VIRES supports various editing operations such as repainting, replacing, generating, and removing video subjects, ensuring temporal consistency through the use of prior knowledge from text-to-video models [2][16]. - The method incorporates a Sequential ControlNet with a standardized adaptive scaling mechanism to effectively extract structural layouts and capture high-contrast sketch details [2][11]. - The research team introduced a sketch attention mechanism within the DiT backbone to interpret and inject fine-grained sketch semantics into the video editing process [2][14]. Group 2: Performance and Comparisons - VIRES has been shown to outperform existing state-of-the-art (SOTA) models across multiple metrics, including visual quality (PSNR), spatial structure consistency (SSIM), frame motion accuracy (WE), inter-frame consistency (FC), and text description consistency (TC) [22][24]. - The research team conducted comparisons with five advanced methods, demonstrating that VIRES achieved the best results in both objective evaluations and user studies [23][24]. Group 3: Data Set and Training - A large-scale video instance dataset named VireSet was created, containing 86,000 video segments, continuous video masks, detailed sketch sequences, and high-quality text descriptions to facilitate precise video instance repainting [6][8]. - The team improved the mask consistency of existing datasets by utilizing pre-trained models to annotate intermediate frames, increasing the mask's frame rate from 6 FPS to 24 FPS [8][12]. Group 4: Future Directions - The research team is also exploring panoramic video generation with a new method called PanoWan, which aims to extend pre-trained text-to-video models to panoramic contexts while maintaining high-quality outputs [31].