Workflow
Data Intelligence Platform
icon
Search documents
Data Intelligence Platform for Nation Scale AI Factories (Presented by DDN)
DDN· 2025-11-25 20:54
As you probably know, AI is already redefining the world economy from financial services to healthcare to automotive, energy, manufacturing, public sector. We are really starting to see great new AI applications come and this is all in partnerships with Nvidia, our great partner who has been pushing us on the envelope of innovation. So what we are talking about here is yes we've been here for many years 27 years in fact but the last 10 years has been amazing in 2015 almost 10 years ago we were supercharging ...
What is the DDN Data Intelligence Platform? (with voiceover)
DDN· 2025-10-06 20:24
Core Functionality - DDN data intelligence platform unifies and harmonizes data, extracting its value and efficiently orchestrating every stage of the AI pipeline [1] - DDN accelerates performance securely and reliably [1] Integration and Compatibility - DDN integrates simply and seamlessly with NVIDIA Enterprise AI factory [1] - DDN accelerates advanced workflows like life science with RAG [1] Performance Enhancement - DDN accelerates performance to reliably meet modern AI demands and scale into the future [1] - DDN enables lightning fast search and low latency object access [1]
AI Infrastructure for Pharma: Powering Complex Drug Simulations
DDN· 2025-09-22 18:42
Pharmaceutical Industry Challenges - Bringing a new drug to market is a complex process requiring significant data processing, primarily write-intensive simulations [1] - The pharmaceutical industry needs an AI infrastructure and data intelligence platform to run complex simulations at massive scale [2] DDN's Solution - DDN provides a fast and efficient data intelligence platform for complex simulations [2] - DDN has over a decade of experience in deploying and optimizing these types of constructs, powering hundreds of sites [2]
A Deep Dive into the Next-Generation Data Intelligence Platform for AI - Sven Oehme, DDN
DDN· 2025-09-18 15:10
Infinia Platform Overview - Infinia is a data intelligence platform, not just a traditional storage product, offering S3 object interface, CSI, Cinder, and file system interfaces [1] - The platform is designed for large-scale deployments, already tested at almost an exabyte in size across approximately 1,000 nodes [1][2] - Infinia is a pure software product that can be integrated into the cloud, with a system already running at GCP for testing [2] Key Features and Capabilities - Infinia supports extensive metadata tagging, allowing tens of thousands of metadata attributes per object for enhanced data discovery and enrichment [1][3] - The system is highly multi-tenant, enabling service providers to manage large-scale systems efficiently while providing SLAs for individual end-users [1][2] - Infinia offers quality of service (QoS) at the application level, allowing prioritization of performance for critical tasks [2] - The platform supports online upgrades, capacity expansion, and reduction without downtime [2] Data Intelligence and AI Workloads - Infinia can serve as a Lakehouse on-premise, providing object, block, and parallel file system access to the same data [2][3] - Remote bucket support allows Infinia to pull metadata from existing data sets, enabling querying and caching of data from external sources [2] - Native library support and SDK can provide up to 10x performance improvement for data ingestion with frameworks like Spark [3] Performance and Scalability - The system is designed for very large scale, with deployments reaching almost an exabyte of capacity [1][2] - Infinia can deliver millions of object operations per second with single-digit millisecond latency [4] - Client-side data reduction and erasure coding eliminate east-west traffic, improving overall performance [3] Resilience and Availability - The system demonstrated high resilience by maintaining operation with only a short IO delay (10-15 seconds) after an entire rack of 480 drives went offline [4]
Building Scalable Foundations for Large Language Models
DDN· 2025-05-27 22:00
AI Infrastructure & Market Trends - Modern AI applications are expanding across various sectors like finance, energy, healthcare, and research [3] - The industry is evolving from initial LLM training to Retrieval Augmented Generation (RAG) pipelines and agentic AI [3] - Vulture is positioned as an alternative hyperscaler, offering cloud infrastructure with 50-90% cost savings compared to traditional providers [4] - A new 10-year cycle requires rethinking infrastructure to support global AI model deployment, necessitating AI-native architectures [4] Vulture & DDN Partnership - Vulture and DDN share a vision for radically rethinking the infrastructure landscape to support global AI deployment [4] - The partnership aims to build a data pipeline to bring data to GPU clusters for training, tuning, and deploying models [4] - Vulture provides the compute infrastructure pipeline, while DDN offers the data intelligence platform to move data [4] Scalability & Flexibility - Enterprises need composable infrastructure for cost-efficient AI model delivery at scale, including automated provisioning of GPUs, models, networking, and storage [2] - Elasticity is crucial to scale GPU and storage resources up and down based on demand, avoiding over-provisioning [3] - Vulture's worldwide serverless inference infrastructure scales GPU resources to meet peak demand in different regions, optimizing costs [3] Performance & Customer Experience - Improving customer experience requires lightning-fast and relevant responses, making time to first token and tokens per second critical metrics [4] - Consistency in response times is essential, even with thousands of concurrent users [4] - The fastest response for a customer is the ultimate measure of customer satisfaction [4] Data Intelligence Platform - DDN's Exascaler offers high throughput for training, with up to 16x faster data loading and checkpointing compared to other parallel file systems [5] - DDN's Infinia provides low latency for tokenization, vector search, and RAG lookups, with up to 30% lower latency [5] - The DDN data intelligence platform helps speed up data response times, enabling saturated GPUs to respond quickly [6]
Introducing the Data Intelligence Platform: DDN Infinia 2.0
DDN· 2025-05-15 19:49
Product Overview - Infinia is a software-defined, next-generation data intelligence platform designed to support both structured and unstructured data, integrating diverse data sources into a unified platform [1] - Infinia aims to reduce complexity in AI deployments and can be deployed on various hardware configurations, including cloud, data centers, reference architectures, and OEM solutions [1] - Infinia operates on the premise of SLAs for capacity, performance, and resilience, abstracting away traditional storage management complexities like volumes and IOPS capping [1] Key Features and Capabilities - Infinia focuses on simplifying data ingestion with low-latency access, emphasizing data tagging for categorization and retrieval in complex environments with billions of objects [1] - Infinia addresses limitations in existing storage solutions by offering unlimited metadata scalability, allowing for thousands of tags per object to refine data definition [1] - Infinia includes capabilities for filtering data at the edge, enabling efficient transfer of relevant data to the core data center for analytics [2] - Infinia offers a software development kit (SDK) to enable customers and service providers to accelerate their applications by using a high-performance interface into Infinia [2] Performance and Efficiency - Infinia achieved a 22x speedup in an indexing and ingest workload on AWS by integrating Nvidia GPUs and utilizing Infinia's S3 interface, resulting in a 70% reduction in GPU resource consumption [1][2] - The company demonstrated a factor of 600 speedup in certain workloads related to identifying data for training AI models [1] Architecture and Integration - Infinia implements data services on top of a key-value store, providing different views onto the data through protocols like S3, file interfaces, and SQL interfaces [2] - Infinia includes a massively distributed SQL server, allowing native access to data and metadata stored in the key-value store through SQL queries [2]
Nebius and DDN: Partnering to Power the Future of Enterprise AI
DDN· 2025-05-01 21:27
Strategic Partnerships & Market Positioning - The AI market is highly competitive, necessitating strong partnerships with GPU providers who advocate for customers and offer enterprise-grade infrastructure [1] - Nebius aims to position itself as a global AI company by partnering with leading players like DDN in the data intelligence platform space [3] - Synergy between Nebius and DDN is seen as a recipe for success, providing access to enterprise and sovereign AI customers [2] Customer Needs & Solutions - Many customers are new to AI and seek guidance on ROI, facing challenges with expensive and confusing data center infrastructure investments [1] - Customers require assistance in navigating the complexities of AI deployment, seeking solutions for large-scale training that are not limited to hyperscalers or on-premise deployments due to high capital expenditure (capex) and megawatt requirements [2] - Nebius addresses these challenges by providing preemptive data center solutions, eliminating on-demand deployment constraints [2] Product Development & Adaptation - Nebius is continuously adapting its product based on feedback from partners like DDN and their enterprise clients to meet industry needs [4] - Data management platforms are crucial for customers to ensure reliability as they scale their cloud infrastructure [3]