L4 is really the pinnacle of vehicle autonomy. L4 autonomy refers to a self-driving system that can handle all driving tasks within a limited geographical area within specific operating conditions without any human intervention. Immediata's approach to L4 autonomy builds on the concept of foundation models.most specifically builds upon Cosmos, which is Nvidia's family of foundation models that have been developed specifically for physical AI applications such as autonomous vehicles. The beauty of co-develop ...

My name is Marco Pavon and I lead autonomous vehicle research at Nvidia. I'm also a professor at Stanford where I lead the autonomous systems lab. I was used to tell my students maybe 3 four years ago that as a researcher I felt a bit stuck.Of course we were making progress but that was very incremental. But in the past three years, there's been a massive influx of new technologies primarily from the field of artificial intelligence that have revolutionized essentially all the building blocks through which ...

Core Insights - NVIDIA is partnering with Uber to scale the world's largest level 4-ready mobility network, targeting a global autonomous fleet of 100,000 vehicles by 2027 [2][3][16] - The NVIDIA DRIVE AGX Hyperion 10 platform serves as a foundational architecture for automakers to develop level 4-ready vehicles, integrating advanced AI technologies [4][12][29] - Major automakers like Stellantis, Lucid, and Mercedes-Benz are collaborating with NVIDIA to enhance their autonomous vehicle capabilities, leveraging the DRIVE platform [8][9][16] Company Collaborations - Uber is integrating human drivers and autonomous vehicles into a unified ride-hailing network, utilizing NVIDIA's technology to bridge current mobility with future autonomous fleets [5][16] - Stellantis is developing AV-Ready Platforms optimized for level 4 capabilities, collaborating with Foxconn for hardware integration [8] - Lucid is advancing its next-generation passenger vehicles with level 4 capabilities using NVIDIA's full-stack AV software [9] Technological Advancements - The NVIDIA DRIVE AGX Hyperion 10 platform features a comprehensive sensor suite, including 14 high-definition cameras, nine radars, one lidar, and 12 ultrasonics, enabling safe and scalable autonomous driving [12][14] - NVIDIA's approach incorporates generative AI and foundation models, trained on extensive driving data, to enhance the decision-making capabilities of autonomous vehicles [18][20] - The introduction of the NVIDIA Halos Certified Program aims to set new standards in vehicle safety and certification for autonomous systems [21][23] Industry Impact - The collaboration between NVIDIA and Uber is expected to transform urban mobility, making transportation safer, cleaner, and more efficient [6][16] - The growing level 4 ecosystem includes partnerships with various companies such as Avride, May Mobility, and Pony.ai, indicating a robust industry movement towards autonomous driving [10][16] - The development of level 4 autonomous trucks by companies like Aurora and Volvo Autonomous Solutions extends the application of NVIDIA's technology from passenger mobility to freight [11]

robotics dev costs collapsed 100x? nvidia hands out foundation models? got it. everyone else still training from scratch. ...

AI发展与应用 - DeepMind 的使命是负责任地构建 AI，以造福人类，深度学习已成为解决图像分类、语音识别和机器翻译等问题的最佳方法 [5][6] - Transformer 架构促成了大型语言模型的构建，这些模型在大量公开数据上进行训练，能够解决广泛的问题 [8] - 现代基础模型（LLM）已超越文本，成为多模态模型，能够处理文本、手写文本和图像，为个性化辅导等学习方式带来可能性 [11][12] - Gemini 1.5 Pro 能够处理高达 1 million 多模态 tokens 的上下文窗口，可以处理大量信息作为输入 [15] - AI Agents 不仅限于简单的聊天机器人，还可以进行语音交互，甚至在 3D 世界中进行实时交互 [16] AI的包容性与可及性 - 行业致力于弥合英语和其他语言（特别是印度语言）之间 AI 能力的差距，目标是开发能够理解 125 种以上印度语言的模型 [19][20][21][22] - Vani 项目与印度科学研究所合作，旨在收集印度各个角落的语音数据，目标是从印度每个地区收集数据，以覆盖更多零语料库语言 [24][25] AI在特定领域的应用 - 行业正在构建数字农业堆栈的基础层，利用卫星图像识别农田边界、作物类型和水源，为农民提供个性化服务，如作物保险 [26][27][28] - AlphaFold 通过预测蛋白质结构，将原本需要 5 年的研究缩短到几秒钟，并在不到一年的时间内完成了 200 million 个蛋白质结构的预测，并免费提供数据，极大地加速了科学发现 [29][30][31][32] 未来展望 - 行业期望 AI 能够帮助更多人，使他们能够做出诺贝尔奖级别的贡献 [35]

Autonomous Driving History and Challenges - Autonomous driving research started in the 1980s with simple neural networks and evolved to end-to-end driving models by 2020 [2] - Scaling autonomous driving presents challenges, requiring solutions for long-tail events and rare scenarios [5][7] - Foundation models, like Gemini, show promise in generalizing to rare driving events and providing appropriate responses [8][9][10][11] Emma: A Multimodal Large Language Model for Autonomous Driving - The company is exploring Emma, a driving system leveraging Gemini, which uses routing text and camera input to predict future waypoints [11][12][13][14] - Emma is self-supervised, camera-only, and high-dimension map-free, achieving state-of-the-art quality on the nuScenes benchmark [15][16][17] - Channel reasoning is incorporated into Emma, allowing the model to explain its driving decisions and improve performance on a 100k dataset [17] Evaluation and Validation - Evaluation is crucial for the success of autonomous driving models, including open loop evaluation, simulations, and real-world testing [25] - Generative models are being explored for sensor simulation to evaluate the planner under various conditions like rain and different times of day [26][27][28] Future Directions - The company aims to improve generalization and scale autonomous driving by leveraging foundation models [30] - Training on larger datasets improves the quality of the planner [19][20] - The company is exploring training on various tasks, such as 3D detection and rograph estimation, to create a more generalizable model [21][22][23][24]

Robotics hasn't had its Jad GPT moment yet, but we think it is almost here. Everyone has known that robots are the future, but that proved elusive because previous generations of robots were expensive, brittle, and only worked in control conditions. With the rapid improvements in foundation models, it is finally possible to make robots that have human level perception and judgment.That has been the missing piece. While consumer use case feature heavily in science fiction, some of the overlooked and most imm ...