Understanding the NVMe Market Segments

NVMe is a relatively new standard, but it’s popping up in multiple different markets. While SATA & SAS drives are still popular, NVMe offers superior performance. However, what capabilities a NVMe drive has depends on the specific market segment it’s tuned to.

There are four distinct market segments for NVMe drives.

• Client Side Computing
• Enterprise Computing
• Data Center / Cloud Computing
• Storage Class Memory

Each of these segments tends to have different requriements from endurance, form factor, performance and capabilities.

Client Side Computing

Client computing is generally for PC’s or laptops. They might be for use in someone’s home, in a work laptop, or a SFF box laying around. These drives generally have limited capabilities, offer moderate performance and limited endurance. However many of these drives do support OPAL encryption.

Almost always these drives are in the M.2 format. In the past, many of these drives were SATA based 2.5 inch drives. That form factor is becoming less common, as the SATA bus limits to 6 Gb/s (about 550 MB/s). While this may remain popular for NAS computers, or retrofitting older client side devices - it does not represent the future of drives.

The endurance of client side M.2 drives is the most striking. The drives are generally rated for 0.3 Drive Writes per Day (DPWD) over 5 years. That’s pretty limited, but it’s not something the standard user needs to be aware of. The amount of writing to a disk that occurs on the client side is generally pretty limited. There’s a static operating system and most of the I/O that is done is much smaller than the drive itself.

The performance of these drives is generally “read optimized”. Meaning fantastic read performance and then somewhat limited write performance. Now mind you, the write performance is still really good. Just not where an enterprise drive will be. It used to be 500+ MB/s was common place, but now many M.2 drives are going to 1-2 GB/s.

These drives don’t have much in the way of capabilities. Often times they’re used as a boot device and as such don’t need fancy capabilities like NVMe namespaces, dual port, and the like. This is generally OK, but given that many motherboards are now offering multiple M.2 slots, I would have liked to see more in the way of capabilities.

Lastly, this market segment is where you’ll see OPAL encryption the most. OPAL encryption enables the user to lock their harddrive at the physical drive level, requiring a unique key at boot time. While OPAL is not on every M.2 drive, this is decidedly the segment that has the most OPAL enablement. For good reason, if your laptop gets stolen you want to make sure that they can’t access the data on that hard drive.

Enterprise Computing

This space is distinctly different from client side computing. The primary driving attributes are performance, endurance, reliability and capability. These drives are often used for applications that require multiple nines of uptime.

The primary drive form factor in this space is U.2. At one point the adapter cards were popular, but the U.2 form factor has taken over. This is likely because it’s a direct fit replacement for traditional 2.5 inch drives - just a different connector on the back plane.

One of the benefits of the U.2 form factor is that the increased space allows for more memory to be put on the drive. It can increase it’s over provisioning space to increase endurance. You often see manufacturers offer a reaad optimized versionon of a drive at around 1 DWPD. Then there is often a write optimized version at 3 DWPD that has slightly less space available for the user.

Performance on these drives are fantastic. This is where PCIe Gen 4 really shines. New drives can hit up to 8 GB/s of read speed. The write speed isn’t yet approaching the limits of PCIe Gen 3, but I suspect it will shortly. It should be noted that write optimized drives generally offer higher performance as well.

These drives often have the most capabilities as well. Multiple NVMe namespaces, secure instant erase, sanitize and many offer dual port controllers. Initially, many of these drives did not support encryption though. That’s changing and the TCG Ruby standard is coming to more drives.

Given all of these capabilities, it is very important to do your drive research. All of these capabilities are optional. As such, it’s really hit or miss which vendors support which capabilities. Vendors also often offer multiple versions of the model - one that’s the base, one with secure instant erase, another for Ruby, and maybe another that’s FIPS certified.

You really have to do your homework in this space to make sure you’re getting drives that support your needs.

Data Center / Cloud Computing

The data center space is where the most innovation is occurring. This space is generally focused on cost and performance optimization. Willing to be on the bleeding edge, they’ll try new form factors.

The drives in this space are generally one of the following formats:

• U.2: The most common format, as it is a widely accepted standard.
• E1.S: A new, small form factor that will likely replace M.2 going forward.
• E1.L: A new, long ruler form factor meant for dense storage rich systems.
• M.2: A persist entrant in the space. But losing steam due to E1.S.

Data Center’s have a diverse set of requirements. It’s common to see drives from both the client space and the enterprise space show up in this segment. That’s because cloud computing has to build systems with both sets of requirements.

If you were building a Ceph cluster, you may want to opt for the U.2 drives with increased endurance and performance. But if you’re building a CDN, you may instead want read/cost optimized M.2 drives instead. It really depends on the use case.

Storage Class Memory

Big data workloads require more memory than I can generally wrap my head around. When you evaluate systems today, we can easily build servers with 16+ TB of memory. It’s just not inexpensive, and sometimes not even enough.

Storage class memory is an emerging market filled with entrants like Intel Optane. The basic idea is that storage can be fast enough now to partially act as memory to a system. It provides an expanded space for the servers to use for memory (though many times applications have to be SCM enabled).

The capabilities of these drives are the most unique of the bunch. Since memory is written to very often, they must have very high endurance. It’s not uncommon to see 20+ DWPD’s of endurance on these drives. The form factor is generally different too - mostly still PCIe adapter cards. Though U.2 and E1.S are starting to emerge.

These drives offer phenominal performance. While an enterprise SSD can saturate the PCIe bus on sequential read/writes, they can not when it comes to 4k IO operations. Since these drives are meant to compliment the main system memory, they tend to be very good at small I/O operations as well.

This is an emerging space, and the one with the most innovation. It will be interesting to see how it changes over time.

Wrapping it Up

Due to the form factor overlap with HDD’s, it seems that the U.2 format will be the leader in the enterprise for a while. Both the HDD and SSD markets are growing, and having a common form factor simplifies complexity.

For client side computing, M.2 is clearly the leader. It providers thinner, simpler builds. In a market where form factor is critical, it’s the right option. However I am not comfortable with how low some vendors are taking endurance on these drives.

When it comes to picking a drive, there are hundreds of options. It’s overwhelming, and once you commit to purchasing one it can be expensive. Make sure you do your research.