Taipei, Friday, Mar 29, 2024, 19:28

News

Fujitsu Develops In-Memory Deduplication Technology to Accelerate Response for Large-Scale Storage

Published: Dec 05,2016

All-Flash Array Storage System and Deduplication

Fujitsu Laboratories Ltd. today announced the development of a high-speed in-memory data deduplication technology for all-flash arrays, which are large-scale, high-speed storage systems and use multiple flash devices such as solid-state drives. This technology enables the production of storage systems with up to twice the response speed when writing data, compared to previous methods.

More on This

Fujitsu Launches 6 New Enterprise Notebooks Optimized for Remote Work

Fujitsu announced the launch of six models of Notebook LIFEBOOK with Windows 10 and the latest 10th generation Intel Core Processor...

Fujitsu Launches 7 New Enterprise Notebooks Optimized for Remote Work

Fujitsu announced the launch of seven models of Notebook LIFEBOOK with Windows 10 and the latest 10th generation Intel Core Processor...

In recent years, all-flash arrays have incorporated deduplication technology that consolidates duplicate data into one to write to a flash device, in order to utilize the limited capacity of flash devices.

However, as the system connects to multiple flash devices through a network in order to search for duplicate data each time it writes data, and storage devices grow in capacity and increase in speed, a problem of lowered response speed during write operations arises.

Now Fujitsu Laboratories has developed a new method that can accelerate response speeds by executing deduplication after writing data. In addition, as data may be written to memory twice in some cases when processing is continued with the new method, thereby increasing communications volume and lowering overall processing performance, Fujitsu Laboratories has developed technology to automatically switch between the new method and the previous method, as operational conditions require.

With the newly developed deduplication technology, the storage devices that make up the all-flash array cooperate to complete a write operation by temporarily storing the write data in open memory (the cache), improving response times.

Then, while the server is preparing for the next write operation depending on the response from the memory, the technology searches for duplicate data, communicating between the storage devices in parallel, and the data is finally written to the solid-state drive after duplicates are eliminated. With this technology, response times for write operations can improve by up to two times when the storage system's network load is low.

Deduplicating after responding improves response speed compared to previous methods when the storage system's network load is low, but when the load is high the network becomes congested. This reduces response times more than previous methods due to data writes across multiple devices taking longer.

With the newly developed response time optimization technology, of the two deduplication methods, the original method that writes after deduplication, for which the response time does not greatly vary, and the newly developed method, for which response times vary widely, the system automatically chooses the method with the shortest response time by calculating the expected value for the average response time, based on records measuring the actual response time.

CTIMES loves to interact with the global technology related companies and individuals, you can deliver your products information or share industrial intelligence. Please email us to en@ctimes.com.tw

822 viewed

Most Popular

comments powered by Disqus