多模态大模型
Search documents
自动驾驶中有“纯血VLA"吗?盘点自动驾驶VLM到底能起到哪些作用~
自动驾驶之心· 2025-09-06 16:05
Core Viewpoint - The article discusses the challenges and methodologies involved in developing datasets for autonomous driving, particularly focusing on the VLA (Visual Language Action) model and its applications in trajectory prediction and scene understanding [1]. Dataset Handling - Different datasets have varying numbers of cameras, and the VLM model can handle this by automatically processing different image token inputs without needing explicit camera counts [2] - The output trajectories are based on the vehicle's current coordinate system, with predictions given as relative (x, y) values rather than image coordinates, requiring additional camera parameters for mapping to images [6] - The VLA model's output format is generally adhered to, but occasional discrepancies occur, which are corrected through Python programming for format normalization [8][9] Trajectory Prediction - VLA trajectory prediction differs from traditional methods by incorporating scene understanding capabilities through QA training, enhancing the model's ability to predict trajectories of dynamic objects like vehicles and pedestrians [11] - The dataset construction faced challenges such as data quality issues and inconsistencies in coordinate formats, which were addressed through rigorous data cleaning and standardization processes [14][15] Data Alignment and Structure - Data alignment is achieved by converting various dataset formats into a unified relative displacement in the vehicle's coordinate system, organized in a QA format that includes trajectory prediction and dynamic object forecasting [18] - The input data format consists of images and trajectory points from the previous 1.5 seconds to predict future trajectory points over 5 seconds, adhering to the SANA standard [20] Community and Resources - The "Autonomous Driving Heart Knowledge Planet" community focuses on cutting-edge technologies in autonomous driving, covering nearly 40 technical directions and fostering collaboration between industry and academia [22][24] - The community offers a comprehensive platform for learning, including video tutorials, Q&A sessions, and job opportunities in the autonomous driving sector [28][29]
自动驾驶之心开学季火热进行中,所有课程七折优惠!
自动驾驶之心· 2025-09-06 16:05
Group 1 - The article introduces a significant learning package for the new academic season, including a 299 yuan discount card that offers a 30% discount on all platform courses for one year [3][5]. - Various course benefits are highlighted, such as a 1000 yuan purchase giving access to two selected courses, and discounts on specific classes and hardware [3][6]. - The focus is on cutting-edge autonomous driving technologies for 2025, particularly end-to-end (E2E) and VLA (Vision-Language Alignment) autonomous driving systems [5][6]. Group 2 - End-to-end autonomous driving is emphasized as a core algorithm for mass production, with a notable mention of the competition sparked by the UniAD paper winning the CVPR Best Paper award [6][7]. - The article discusses the challenges faced by beginners in mastering multi-modal large models and the fragmented nature of knowledge in the field, which can lead to discouragement [7][8]. - A course on automated 4D annotation algorithms is introduced, addressing the increasing complexity of training data requirements for autonomous driving systems [11][12]. Group 3 - The article outlines a course on multi-modal large models and practical applications in autonomous driving, reflecting the rapid growth and demand for expertise in this area [15][16]. - It mentions the increasing job opportunities in the field, with companies actively seeking talent and offering competitive salaries [15][16]. - The course aims to provide a systematic learning platform, covering topics from general multi-modal large models to fine-tuning for end-to-end autonomous driving applications [16][18]. Group 4 - The article emphasizes the importance of community and communication in the learning process, with dedicated VIP groups for course participants to discuss challenges and share insights [29]. - It highlights the need for practical guidance in transitioning from theory to practice, particularly in the context of real-world applications and job readiness [29][31]. - The article also mentions the availability of specialized small group courses to address specific industry needs and enhance practical skills [23][24].
筹备了很久,下周和大家线上聊一聊~
自动驾驶之心· 2025-09-05 07:50
Core Viewpoint - The article emphasizes the establishment of an online community focused on autonomous driving technology, aiming to facilitate knowledge sharing and networking among industry professionals and enthusiasts [5][12]. Group 1: Community and Activities - The community has over 4,000 members and aims to grow to nearly 10,000 in the next two years, providing a platform for technical exchange and sharing [5][11]. - An online event is planned to engage community members, allowing them to ask questions and interact with industry experts [1][3]. - The community includes members from leading autonomous driving companies and top academic institutions, fostering a collaborative environment [12][20]. Group 2: Technical Focus Areas - The community covers nearly 40 technical directions in autonomous driving, including multi-modal large models, closed-loop simulation, and sensor fusion, suitable for both beginners and advanced learners [3][5]. - A comprehensive learning path is provided for various topics, such as end-to-end autonomous driving, multi-sensor fusion, and world models, to assist members in their studies [12][26]. - The community has compiled resources on open-source projects, datasets, and industry trends, making it easier for members to access relevant information [24][25]. Group 3: Job Opportunities and Networking - The community has established a job referral mechanism with several autonomous driving companies, facilitating connections between job seekers and potential employers [8][54]. - Members can freely ask questions regarding career choices and research directions, receiving guidance from experienced professionals [54][57]. - Regular discussions with industry leaders are held to share insights on the development trends and challenges in autonomous driving [57][59].
开学了,需要一个报团取暖的自驾学习社区...
自动驾驶之心· 2025-09-04 23:33
Group 1 - The article discusses the importance of the autumn recruitment season, highlighting a student's experience of receiving an offer from a tier 1 company but feeling unfulfilled due to a desire to transition to a more advanced algorithm position [1] - The article encourages perseverance and self-challenge, emphasizing that pushing oneself can reveal personal limits and potential [2] Group 2 - A significant learning package is introduced, including a 299 yuan discount card for a year of courses at a 30% discount, various course benefits, and hardware discounts [4][6] - The focus is on cutting-edge autonomous driving technologies for 2025, particularly end-to-end (E2E) and VLA autonomous driving systems, which are becoming central to the industry [7][8] Group 3 - The article outlines the development of end-to-end autonomous driving algorithms, emphasizing the need for knowledge in multimodal large models, BEV perception, reinforcement learning, and more [8] - It highlights the challenges faced by beginners in synthesizing knowledge from fragmented research papers and the lack of practical guidance in transitioning from theory to practice [8] Group 4 - The introduction of a new course on automated 4D annotation algorithms is aimed at addressing the increasing complexity of training data requirements for autonomous driving systems [11][12] - The course is designed to help students navigate the challenges of data annotation and improve the efficiency of data loops in autonomous driving [12] Group 5 - The article discusses the emergence of multimodal large models in autonomous driving, noting the rapid growth of job opportunities in this area and the need for a structured learning platform [14] - It emphasizes the importance of practical experience and project involvement for job seekers in the autonomous driving sector [21] Group 6 - The article mentions various specialized courses available, including those focused on perception, model deployment, planning control, and simulation in autonomous driving [16][18][20] - It highlights the importance of community engagement and support through dedicated VIP groups for course participants [26]
国投智能(300188.SZ):已将多模态能力应用到了视觉理解和增强上
Ge Long Hui· 2025-09-04 07:26
Core Viewpoint - The company has made significant progress in the field of multimodal large models, applying them across various business lines for enhanced operational capabilities [1] Group 1: Application of Multimodal Large Models - The company utilizes dynamic rules and instructions to implement multimodal large models in behavior recognition, scene analysis, risk warning, and emergency command [1] - Each video is equipped with an intelligent brain through the application of these models, enhancing the understanding of video content [1] - The company has achieved comprehensive perception in video streaming by extracting target event information, creating a complete information cognitive landscape [1] Group 2: Integration with Smart Wearable Devices - The multimodal capabilities have been applied to visual understanding and enhancement in smart wearable devices [1] - The integration of data and service resources has led to a synergy between business scenarios and data capabilities [1]
开放几个大模型技术交流群(RAG/Agent/通用大模型等)
自动驾驶之心· 2025-09-04 03:35
Group 1 - The establishment of a Tech communication group focused on large models, inviting participants to discuss topics such as RAG, AI Agents, multimodal large models, and deployment of large models [1] - Interested individuals can join the group by adding a designated WeChat assistant and providing their nickname along with a request to join the large model discussion group [2]
自动驾驶之心开学季活动来了(超级折扣卡/课程/硬件/论文辅导福利放送)
自动驾驶之心· 2025-09-02 09:57
Core Viewpoint - The article reflects on the evolution of autonomous driving over the past decade, highlighting significant technological advancements and the ongoing need for innovation and talent in the industry [2][3][4]. Group 1: Evolution of Autonomous Driving - Autonomous driving has progressed from basic image classification to advanced perception systems, including 3D detection and end-to-end models [3]. - The industry has witnessed both failures and successes, with companies like Tesla, Huawei, and NIO establishing strong technological foundations [3]. - The journey of autonomous driving is characterized by continuous efforts rather than sudden breakthroughs, emphasizing the importance of sustained innovation [3]. Group 2: Importance of Talent and Innovation - The future of autonomous driving relies on a steady influx of talent dedicated to enhancing safety and performance [4]. - Innovation is identified as the core of sustainable business growth, with a focus on practical applications and real-world problem-solving [6]. - The article encourages a mindset of continuous learning and adaptation to keep pace with rapid technological changes [6]. Group 3: Educational Initiatives and Resources - The company has developed a series of educational resources, including video tutorials and courses covering nearly 40 subfields of autonomous driving [8][9]. - Collaborations with industry leaders and academic institutions are emphasized to bridge the gap between theory and practice [8]. - The article outlines various courses aimed at equipping learners with the necessary skills for careers in leading autonomous driving companies [9][10]. Group 4: Future Directions in Technology - Key technological directions for 2025 include end-to-end autonomous driving and the integration of large models [12][20]. - The article discusses the significance of multi-modal large models in enhancing the capabilities of autonomous systems [20]. - The need for advanced data annotation techniques, such as automated 4D labeling, is highlighted as crucial for improving training data quality [16].
业务合伙人招募来啦!模型部署/VLA/端到端方向~
自动驾驶之心· 2025-09-02 03:14
Group 1 - The article announces the recruitment of 10 partners for the autonomous driving sector, focusing on course development, research guidance, and hardware development [2][5] - The recruitment targets individuals with expertise in various advanced models and technologies related to autonomous driving, such as large models, multimodal models, and 3D target detection [3] - Candidates are preferred from QS top 200 universities with a master's degree or higher, especially those with significant conference contributions [4] Group 2 - The company offers benefits including resource sharing for job seeking, PhD recommendations, and study abroad opportunities, along with substantial cash incentives [5] - There are opportunities for collaboration on entrepreneurial projects [5] - Interested parties are encouraged to contact the company via WeChat for further inquiries [6]
4000人的自动驾驶社区,开学季招生了!!!
自动驾驶之心· 2025-09-02 03:14
Core Viewpoint - The article emphasizes the establishment of a comprehensive community focused on autonomous driving technology, aiming to provide valuable resources and networking opportunities for both beginners and advanced learners in the field [1][3][12]. Group 1: Community Structure and Offerings - The community has been focusing on nearly 40 cutting-edge technology directions in autonomous driving, including multimodal large models, VLM, VLA, closed-loop simulation, world models, and sensor fusion [1][3]. - The community consists of members from leading autonomous driving companies, top academic laboratories, and traditional robotics firms, creating a complementary dynamic between industry and academia [1][12]. - The community has over 4,000 members and aims to grow to nearly 10,000 within two years, serving as a hub for technical sharing and communication [3][12]. Group 2: Learning and Development Resources - The community provides a variety of resources, including video content, articles, learning paths, and Q&A sessions, to assist members in their learning journey [3][12]. - It has organized nearly 40 technical routes for members, covering various aspects of autonomous driving, from entry-level to advanced topics [3][12]. - Members can access practical solutions to common questions, such as how to start with end-to-end autonomous driving and the learning paths for multimodal large models [3][12]. Group 3: Networking and Career Opportunities - The community facilitates job referrals and connections with various autonomous driving companies, enhancing members' employment opportunities [8][12]. - Regular discussions with industry leaders and experts are held to explore trends, technological directions, and challenges in mass production [4][12]. - Members are encouraged to engage with each other to discuss academic and engineering-related questions, fostering a collaborative environment [12][54]. Group 4: Technical Focus Areas - The community has compiled extensive resources on various technical areas, including 3DGS, NeRF, world models, and VLA, providing insights into the latest research and applications [12][27][31]. - Specific learning paths are available for different aspects of autonomous driving, such as perception, simulation, and planning control [12][13]. - The community also offers a detailed overview of open-source projects and datasets relevant to autonomous driving, aiding members in practical applications [24][25].
事关AI芯片,阿里发声
财联社· 2025-09-02 00:34
Core Viewpoint - Alibaba Cloud is facing a potential computing power shortage for its Tongyi Qianwen large model, prompting the company to reportedly increase its order of Cambricon's Siyuan 370 chips to 150,000 units, although this claim has been denied by Alibaba Cloud representatives [2][3]. Group 1: AI Chip Market Dynamics - Major players in the domestic AI chip market include Cambricon, Huawei, Haiguang Information, Birun, Muxi, Suyuan, and Moore Threads [4]. - According to IDC, the market scale for accelerated chips in China is expected to exceed 2.7 million units by 2024, with GPU cards holding a 70% market share [4]. - Domestic AI chip brands have shipped over 820,000 units, with Huawei's Ascend series capturing a significant portion of the market [4]. Group 2: Alibaba's Chip Development Strategy - Alibaba is actively developing its own chips through its independent semiconductor company, Pingtouge, established in 2018 [6]. - Pingtouge has launched several chip series, including the "Xuantie" RISC-V processors and "Hanguang" AI chips, with some chips already deployed at scale on Alibaba Cloud [6]. - Reports suggest that Alibaba's new AI chip, which is compatible with Nvidia, is currently in testing and will be manufactured by a domestic company instead of TSMC [6][7]. Group 3: Competitive Landscape in AI Chips - Other internet companies like Baidu, ByteDance, and Tencent are also exploring chip development [8]. - Baidu's Kunlun chip supernode has been fully operational since August, supporting extensive AI model training [8]. - Tencent has introduced several self-developed chips, including AI inference and video transcoding chips, and has collaborated with AMD on GPU cards [8]. Group 4: Strategic Importance of Chip Supply Chains - Establishing a self-sufficient supply chain that includes domestic chips is crucial for the future development of the AI ecosystem [8]. - Alibaba's "One Cloud, Multiple Chips" strategy aims to ensure compatibility with various chip architectures, including X86, ARM, and RISC-V [8]. - Experts emphasize the need for domestic chips to enhance performance and build ecosystems to meet global computing power challenges [9].