Object Store
Search documents
A Deep Dive into the Next-Generation Data Intelligence Platform for AI - Sven Oehme, DDN
DDN· 2025-09-18 15:10
Infinia Platform Overview - Infinia is a data intelligence platform, not just a traditional storage product, offering S3 object interface, CSI, Cinder, and file system interfaces [1] - The platform is designed for large-scale deployments, already tested at almost an exabyte in size across approximately 1,000 nodes [1][2] - Infinia is a pure software product that can be integrated into the cloud, with a system already running at GCP for testing [2] Key Features and Capabilities - Infinia supports extensive metadata tagging, allowing tens of thousands of metadata attributes per object for enhanced data discovery and enrichment [1][3] - The system is highly multi-tenant, enabling service providers to manage large-scale systems efficiently while providing SLAs for individual end-users [1][2] - Infinia offers quality of service (QoS) at the application level, allowing prioritization of performance for critical tasks [2] - The platform supports online upgrades, capacity expansion, and reduction without downtime [2] Data Intelligence and AI Workloads - Infinia can serve as a Lakehouse on-premise, providing object, block, and parallel file system access to the same data [2][3] - Remote bucket support allows Infinia to pull metadata from existing data sets, enabling querying and caching of data from external sources [2] - Native library support and SDK can provide up to 10x performance improvement for data ingestion with frameworks like Spark [3] Performance and Scalability - The system is designed for very large scale, with deployments reaching almost an exabyte of capacity [1][2] - Infinia can deliver millions of object operations per second with single-digit millisecond latency [4] - Client-side data reduction and erasure coding eliminate east-west traffic, improving overall performance [3] Resilience and Availability - The system demonstrated high resilience by maintaining operation with only a short IO delay (10-15 seconds) after an entire rack of 480 drives went offline [4]
DDN Infinia: Next-Gen Object Store for AI, Multi-Tenancy & Data Workflows
DDN· 2025-08-04 18:00
Product Overview - Infinia is a ground-up developed product intended as more than just an object store, offering multiple data interfaces [1] - The system provides database access through an SQL interface and can be used as a notification bus, critical for AI workflows [2] - It is a pure software product designed to solve complicated problems, making it attractive for service providers [3] Key Features & Capabilities - Built-in architectural capabilities around multi-tenancy provide technology as a service [4] - Deep integration capabilities through an SDK enable customers to accelerate applications and AI workflows [4] Ecosystem & Partnerships - DDN has ongoing partner development with ecosystem partners to accelerate technology adoption through the Infinia SDK [4]