Workflow
DDNX Scaler
icon
Search documents
Accelerate Your HPC Workloads with Google Cloud Managed Lustre | Kirill Tropin
DDNยท 2025-12-08 23:41
Google Cloud Managed Luster Overview - Google Cloud Managed Luster is a fully managed service running on top of DDNX Scaler, launched 4 and a half months ago [1][7] - It addresses the need for high throughput and low latency storage in HPC environments to keep GPUs and CPUs efficiently fed with data [4][5] - The service is integrated with Google Cloud services like DCS and GKE, offering easy data import/export from/to Google Cloud Storage [7][8] Performance and Scalability - Google Cloud Managed Luster offers up to 1 TB (Terabyte) per second of throughput with sub-millisecond latency and millions of IOPS [9] - It scales from a starting size of 9 TB (Terabyte) up to 8 PB (Petabyte) [9] - Performance tiers range from 125 MB (Megabyte) per second per TB (Terabyte) to 1,000 MB (Megabyte) per second per TB (Terabyte), catering to different throughput needs [15] Customer Benefits and Use Cases - Customers have experienced significant performance improvements, with one customer, Resemble AI, achieving full GPU saturation and 6x faster performance compared to other storage solutions [10] - Sony Honda Mobility's department, Fila, saw a 3x performance improvement compared to their previous storage solution [17] - Key use cases include KV cache, multimodal training, and checkpointing, all requiring low latency and high throughput [11][13][14] Partnership with DDN - Google partnered with DDN (DataDirect Networks) due to their mature, reliable Exoscaler product with a rich feature set [6] - The partnership aims to provide a fully managed solution, relieving customers of storage management burdens [6]