This Storage Intelligence (StorInt) dispatch covers SPECsfs2008 benchmark results[1]. There have been seven new SPECsfs2008 results since our last discussion in March of 2014. Five of the new submissions are from Avere using a 3 node FXT 3800 cluster with five different backends: direct connect Amazon SSS FlashCloud storage; Amplidata AmpliStor storage; Cleversafe dsNet storage; local (generic) ZFS storage; and non-local (generic) ZFS storage systems and the other two submissions were from Hitachi one with their new VSP G1000 and an 8-node 4100 File module and the other an HUS 150 with a 2-node 4040 File module. Both our top 10 Overall Response Time (ORT) results and top 10 NFS throughput results charts have changed. There have been no new CIFS/SMB results.

NFSv3 results

We begin our discussion with our Top 10 Overall Response Time (ORT) in Figure 1.

Figure 1 SPECsfs2008 Top 10 NFSv3 ORT results

Recall that ORT is an average response time calculated across the entire SPECsfs2008 benchmark run. In Figure 1 one can see four of the new Avere submissions and the new Hitachi VSP G1000. The VSP G1000 used enough FlashModule storage to qualify as an all-flash array from our perspective. The Avere FXT3800 front-ends all had 2.4TB of flash and some of the backend storage had flash as well.

The Avere run with Amazon FlashCloud storage presents an interesting conundrum. In the SPECsfs2008 report there was no real description of the FlashCloud storage other than an indication of the useable capacity and the network connection between the frontend and the FlashCloud storage. This is very unusual for SPECsfs2008 reports and we mentioned it to SPEC support, Avere management and Amazon marketing. But as this went to press, we had still not heard any answers to questions on this result. Nonetheless, regardless of the backend storage used the Avere-FlashCloud storage benchmark still came in at 6^th place for ORT.

The other item of note here is that two out of the three object storage system backends/Avere frontend submissions were fast enough to reach top 10 in response time. To our knowledge object storage hasn’t been known for lightening fast response time ever before. To have these two show up in our top 10 ORT is a significant milestone. And we would congratulate Avere, Amazon and Cleversafe for reaching this new milestone.

In Figure 2 we show the top 10 NFS throughput operations/second results.

Figure 2 SPECsfs2008 Top 10 NFSv3 throughput results 

In Figure 2 we see the Hitachi VSP G1000/8-node 4100 result came in at 8^th place with ~1.2 million NFS ops/second. The 8-node, Hitachi solution with its 1.6TB FlashModules did well against the other, scale out NAS solutions with 20-140 nodes each.

All of the systems on our Top 10 NFS throughput chart now have at least some SSDs or flash in their solutions. However, none of the other systems in Figure 2 had enough NAND storage to constitute an all-flash solution. So, the Hitachi VSP G1000/4100 storage is the first, all-flash system to make it into our Top 10 NFS throughput that we can recall.

Significance

It’s great to see object storage solutions finally submitting SPECsfs2008 benchmark results. Avere should get some kudos from the industry for showing that with a proper backend configuration and a powerful enough, front-end acceleration, object storage can compete in ORT with any storage, as a NAS solution.

On the other hand, the lack of information from the Avere-Amazon FlashCloud submission is a quandary. Yes, it’s cloud storage and one shouldn’t really need to know what’s behind the (cloud) curtain. But if a benchmark like SPECsfs2008 is to be independently reproducible we need to have all the information about the storage configuration that would allow us to do just that. We realize that Avere, Amazon and SPECsfs probably didn’t have enough time to respond to our queries but the lack of information should be rectified, ASAP.

As always, suggestions on how to improve any of our performance analyses are welcomed. Additionally, if you are interested in more file performance details, we now provide a fuller version (Top 20 results) of some of these charts and a set of new NFSv3 and CIFS/SMB ChampionsCharts in our recently updated (April 2019), NAS Buying Guide available from our website. Also Top 20 versions of some of these charts are displayed in our recently updated (May 2019) SAN-NAS Buying guide also purchasable from our website.

[This performance dispatch was originally sent out to our newsletter subscribers in June of 2014.  If you would like to receive this information via email please consider signing up for our free monthly newsletter (see subscription request, above right) and we will send our current issue along with download instructions for this and other reports. Dispatches are posted to our website at least a quarter or more after they are sent to our subscribers. ]  

Silverton Consulting, Inc. is a Storage, Strategy & Systems consulting services company, based in the USA offering products and services to the data storage community.