MIT system boosts data centre storage efficiency
A method improves storage efficiency by reducing performance differences across shared storage systems and enhancing overall resource use.
Researchers from MIT have developed a software-based system designed to improve the efficiency of data centre storage by addressing performance variability across pooled solid-state drives (SSDs). The approach targets inefficiencies that persist when multiple devices are shared across applications in large-scale environments.
The Sandook system identifies and manages three key sources of SSD variability, including hardware differences, read-write interference, and unpredictable garbage collection. Instead of treating these issues separately, the method addresses them simultaneously to improve overall throughput.
Sandook uses a two-tier architecture combining a global controller, which assigns workloads across the storage pool, and local controllers on each device that respond in real time to performance slowdowns. The design enables the system to redistribute tasks and reduce strain on underperforming drives dynamically.
Testing on workloads such as machine learning training, databases, and image compression showed performance gains of up to 94 percent compared with traditional methods, along with higher overall storage utilisation.
The approach could extend the lifespan of existing infrastructure and reduce the need for additional hardware investment. The shift may improve efficiency and cost-effectiveness while making more sustainable use of existing computing resources and limiting the pace of hardware replacement.
Would you like to learn more about AI, tech and digital diplomacy? If so, ask our Diplo chatbot!