So we've always set the bar very very high. Uh and that combined with the fact that we started out at DDN solving massive scale data set challenges gave us the foundation to pivot our technology into AI and DDN is the only company today which is solving the right at scale challenge which applies to LLM and Gen AI. power consumption, the movement of data, the ability to support hundreds and hundreds of software suites of AI frameworks and make sure that each and every one of them operates at the highest leve ...

Welcome to this Infinia demo. Today we'll be showing how Infinia can handle all the IO needs in an AI data pipeline. The Infinia architecture can be broken down into sections.Storage services providing enhanced resilience and elastic scale capabilities of the storage itself. The data plane comprised of a key value store as well as the presentation of data to clients via IO protocols. SQL queries of the KV store data and metadata and an SDK to integrate directly with applications and frameworks.And finally, ...

Welcome to this demonstration. Today we'll be showing how DDN enables a one-click high-performance rag pipeline for enterprise use. Our rag pipeline solution is enterprise class and easy to deploy and use in any cloud environment whether AWS, GCP, Azure, any NCP cloud and of course on prem.Let's take a closer look at the architecture. This rag pipeline solution is made of several NVIDIA Nemo NIMS or NVIDIA inference microservices which host embedding reranking LLM models a milild vector database a front-end ...

AI Workflow & Data Preparation - Infinia plays a crucial role in AI workflows, particularly in data preparation stages, by handling diverse data ingestion, providing low-latency KV store access at scale, and integrating with various AI platforms [2] - The AI pipeline involves data collection, pre-processing, tagging, and indexing as key data preparation steps [1] - DDN's Infinia, combined with Spark integrations, facilitates a smooth and scalable workflow using familiar tools for AI developers [6][7] Data Management & Security - Infinia addresses the challenge of providing secure data buckets for multiple developers through multi-tenancy controls, enabling dynamic addition or removal of secure tenants and subtenants [6] - DDN has developed Spark integrations to efficiently move data into developer tenant buckets [6] - Infinia's multi-tenancy can create secure locations for hosting data used in each inference pipeline [9] Mortgage Default Modeling Demo - The demonstration uses 10 years of quarterly mortgage finance data to model delinquency rates and probabilities on mortgage defaults [4] - Apache Spark is used to prepare the data and pipe it into a model training process that could be run on top of Infinia [3] - The workflow includes extracting recent data subsets, copying them into new Infinia buckets using Spark, and transforming the data into parquet files for model training [4][8] - The model training utilizes the XGBoost machine learning library to create a predictive model for mortgage defaults [9]

Performance Overview - DDN Infinia demonstrates excellent performance in Oracle Cloud Infrastructure (OCI) with a small six-node cluster [7] - Achieved a consistent 5 milliseconds Time To First Byte (TTFB), which is excellent for S3 object IO [6] Throughput Metrics - Achieved approximately 30 GB/s of put throughput during object population [5] - Each client and Infinia node processed puts at roughly 5 GB/s [5] - Sustained approximately 37.5 GB/s of get throughput during the get benchmark [6] - Load was evenly distributed across all clients and Infinia nodes at around 6.5 GB/s of throughput during get operations [6] Infrastructure and Configuration - The test used six BM dense ioe5 compute instances as hosts for the Infinia cluster [2] - Six BM standard E5.192 instances with single 100 GB connections were used for the clients to avoid networking bottlenecks [2] - Only 32 out of the 128 cores available in the dense ioe5 instances were utilized for the Infinia software [2] - DDN is investigating other OCI instances to prevent overallocation of hardware [3] Technology and Architecture - Infinia architecture provides capabilities for data management, including data IO paths, object file querying, scale-out KV store, always-on encryption, and data reduction [2] - Infinia is fully software-defined and containerized, enabling it to run on physical or virtualized hardware with Intel, AMD, or ARM processors [2] - Implemented high-performance eraser coding, custom fall domains, and the ability to use both TLC and QLC flash [2] Testing Methodology - IO generation was performed using warp in distributed benchmarking mode to ensure a full mesh of IO across all clients and Infinia cluster nodes [3] - Parallel warp was used across all six clients and six Infinia nodes during the put and get tests [4][5][6] Disclaimer - The information presented is for potential future integrations and is a tech preview [1] - The overall capabilities, including the performance of this feature, can and will change [1] - No timelines for delivering this capability should be inferred from this demo [1]

Performance Comparison - DDN Infinia writes chunks at 0041 seconds (4 milliseconds) per chunk, significantly faster than AWS [6] - AWS object store writes each chunk at 01169 seconds (112 milliseconds) per chunk [7] - DDN Infinia uploads a 628-chunk document in approximately 25 seconds, while AWS takes around 74 seconds [7] - DDN Infinia is approximately 285 times faster than AWS in document upload [7] - DDN Infinia retrieves chunks in 01600 seconds (160 milliseconds) total, averaging 32 milliseconds per chunk [13] - AWS retrieves chunks in 165 seconds, with each chunk taking 331 milliseconds [14] - DDN Infinia is 103 times faster than AWS in total query retrieval time [14] AI Pipeline Impact - With DDN Infinia, an analyst can upload and query an annual report in just 2 seconds [8] - A 30x performance advantage transforms the entire AI pipeline, making documents readily available for AI consumption [9] - Reduced latency with DDN Infinia can save significant time, potentially turning a 5-minute research task into 3 seconds [15] - Latency compounds across multiple users and sessions, impacting GPU economics and overall productivity [15]

RAG Acceleration with DDN Infinia - DDN Infinia accelerates retrieval-augmented generation (RAG) by removing I/O and object storage delays [1] - DDN Infinia delivers sub-second retrieval [1] - DDN Infinia achieves 96% GPU utilization [1] - DDN Infinia enables seamless scaling for hybrid vector and keyword search workloads [1] Key Benefits - DDN Infinia accelerates hybrid RAG retrieval [1] - DDN Infinia reduces latency and maximizes throughput [1] - DDN Infinia streamlines vector search and context retrieval [1] - DDN Infinia improves LLM performance in enterprise AI environments [1] DDN Overview - DDN is a pioneer in high-performance data storage and management [1] - DDN delivers innovative solutions that empower organizations across the globe [1]

[Music] The countdown is on. SC25 is almost here and DDN is heading to St. Louis this November 17th through the 20th.Here's a sneak peek at what's in store. [Music] [Applause] DDN is the exclusive IO sponsor at Supercomputing 25. Find us at booth 1527, that's 1527, for live demos, our booth theater, and book a one-on-one meeting with our experts building tomorrow's AI and HPC infrastructure.[Music] And don't miss BDN's Beyond Artificial Data Summit, supported by NVIDIA and sponsored by Google Cloud and Supe ...

AI Inference Challenges & KV Caching Solution - AI inference faces challenges with large context windows, impacting tokenization and latency [1][2] - Caching context tokens speeds up responsiveness, lowers latency, and allows storing larger context amounts [4] - Effective caching requires storage systems with low latency and large capacity at scale [5] DDN's Solution & Performance - DDN's Exoscaler platform enables high-performance KV caching for AI inference, improving user concurrency, responsiveness, and user experience [7] - DDN leverages GPU direct storage (GDS) for cached engine [9] - Caching demonstrates a 10x improvement in performance with larger context [14] - DDN's Exoscaler performance can improve time to first token during inference by 10-25x [16] - DDN improves response times, provides larger cache repository space, and delivers cost-effective performance and capacity density [17] Capacity Implications - KV caching accelerates the end-user experience, putting a premium on high-performance shared storage [16] - Approximately 200,000 input characters resulted in a cache of 796 files, totaling almost 13 gigabytes [15]

AI ROI and Business Outcomes - Achieving real AI ROI requires focusing on specific business outcomes and problem-solving [4][5] - Infrastructure planning is crucial for optimizing AI investments and achieving a greater return on invested capital [6] - Enterprises should clearly define measurable metrics to gauge the success of AI projects [21] Infrastructure as a Strategic Asset - Data infrastructure is a strategic asset that drives efficiency and optimization for AI projects [8][9] - Integrating infrastructure tightly into the ecosystem maximizes investments and drives ROI [9] - Early AI deployments sometimes overlook infrastructure efficiencies, leading to underutilization and wasted resources [10] Scaling AI Factories - DDN's new enterprise hyperpod, built with Super Micro and powered by NVIDIA, helps enterprises scale AI from pilot to exascale [11] - The Hyper Pod is a pre-engineered platform that simplifies AI inference tuning for various industries, sovereign clouds, and AI factories [11][12] - This platform enables scalable deployment and is optimized for high-performance, high-scale inference or tuning [12] Industry Impact of AI Infrastructure - Healthcare and life sciences benefit from AI in drug discovery, precision medicine, and genomics, improving physician efficiency and patient care [14] - Financial services leverage AI for algorithmic trading, fraud analytics, and risk management [14] - Other industries benefiting from AI include oil and gas, automotive (self-driving cars), and next-generation hyperscalers [15][16] Advice for Enterprise Leaders - Enterprise leaders should clearly define the outcomes they want to drive and the problems they aim to solve with AI [17][18] - Maximizing return on investment in infrastructure assets is essential, considering speed, performance, and utilization [18] - Enterprises should be mindful of their unique goals when deploying AI systems [20]