Performance Benchmarks

Benchmarks such as ATTO and CrystalDiskMark help provide a quick look at the performance of the direct-attached storage device. The results translate to the instantaneous performance numbers that consumers can expect for specific workloads, but do not account for changes in behavior when the unit is subject to long-term conditioning and/or thermal throttling. Yet another use of these synthetic benchmarks is the ability to gather information regarding support for specific storage device features that affect performance.

Synthetic Benchmarks - ATTO and CrystalDiskMark

OWC claims speeds of up to 946 MBps for the Envoy Pro Mini. While these are not hit easily under the typical 128KB transfer sizes, the ATTO benchmarks below show that the claimed numbers can indeed be hit under specific conditions. It must be noted that ATTO benchmarking is restricted to a single configuration in terms of queue depth, and is only representative of a small sub-set of real-world workloads. It does allow the visualization of change in transfer rates as the I/O size changes, with optimal performance being reached around 512 KB for a queue depth of 4.

ATTO Benchmarks
TOP: BOTTOM:

CrystalDiskMark. for example, uses four different access traces for reads and writes over a configurable region size. Two of the traces are sequential accesses, while two are 4K random accesses. Internally, CrystalDiskMark uses the Microsoft DiskSpd storage testing tool. The 'Seq128K Q32T1' sequential traces use 128K block size with a queue depth of 32 from a single thread, while the '4K Q32T16' one does random 4K accesses with the same queue configuration, but from multiple threads. The 'Seq1M' traces use a 1MiB block size. The plain 'Rnd4K' one uses only a single queue and single thread . Comparing the '4K Q32T16' and '4K Q1T1' numbers can quickly tell us whether the storage device supports NCQ (native command queuing) / UASP (USB-attached SCSI protocol). If the numbers for the two access traces are in the same ballpark, NCQ / UASP is not supported. This assumes that the host port / drivers on the PC support UASP.

CrystalDiskMark Benchmarks
TOP: BOTTOM:

The numbers here match OWC's claims on the read side, but the write claims narrowly misses out - while we did see the 32GB span in ATTO deliver 900 MBps+ writes, ATTO and CrystalDiskMark treat speeds differently for reporting purposes. While ATTO records instantaneous transfer rates within that span, CrystalDiskMark reports average numbers across the full span. Our targeted benchmarks detailed further down bring out the SLC cache sizes which justify the 700 MBps numbers where 900 MBps should have made an appearance.

AnandTech DAS Suite - Benchmarking for Performance Consistency

Our testing methodology for storage bridges / direct-attached storage units takes into consideration the usual use-case for such devices. The most common usage scenario is transfer of large amounts of photos and videos to and from the unit. Other usage scenarios include the use of the unit as a download or install location for games and importing files directly from it into a multimedia editing program such as Adobe Photoshop. Some users may even opt to boot an OS off an external storage device.

The AnandTech DAS Suite tackles the first use-case. The evaluation involves processing five different workloads:

  • AV: Multimedia content with audio and video files totalling 24.03 GB over 1263 files in 109 sub-folders
  • Home: Photos and document files totalling 18.86 GB over 7627 files in 382 sub-folders
  • BR: Blu-ray folder structure totalling 23.09 GB over 111 files in 10 sub-folders
  • ISOs: OS installation files (ISOs) totalling 28.61 GB over 4 files in one folder
  • Disk-to-Disk: Addition of 223.32 GB spread over 171 files in 29 sub-folders to the above four workloads (total of 317.91 GB over 9176 files in 535 sub-folders)

Except for the 'Disk-to-Disk' workload, each data set is first placed in a 29GB RAM drive, and a robocopy command is issue to transfer it to the external storage unit (formatted in exFAT for flash-based units, and NTFS for HDD-based units).

robocopy /NP /MIR /NFL /J /NDL /MT:32 $SRC_PATH $DEST_PATH

Upon completion of the transfer (write test), the contents from the unit are read back into the RAM drive (read test) after a 10 second idling interval. This process is repeated three times for each workload. Read and write speeds, as well as the time taken to complete each pass are recorded. Whenever possible, the temperature of the external storage device is recorded during the idling intervals. Bandwidth for each data set is computed as the average of all three passes.

The 'Disk-to-Disk' workload involves a similar process, but with one iteration only. The data is copied to the external unit from the CPU-attached NVMe drive, and then copied back to the internal drive. It does include more amount of continuous data transfer in a single direction, as data that doesn't fit in the RAM drive is also part of the workload set.

Audio and Video Read

Based on the ordering of the test components, the performance in the disk-to-disk segment represents what end users can see under the worst-case conditions for DAS use. The numbers in the graphs above show that for short bursts, the performance of the Envoy Pro Mini is not impressive. In the disk-to-disk segment, we see it shine through and getting ahead of the two DataTraveler Max 1 TB SKUs. In addition to these numbers, power users may want to dig deeper to understand the limits of each device. To address this concern, we also instrumented our evaluation scheme for determining performance consistency.

Performance Consistency

Aspects influencing the performance consistency include SLC caching and thermal throttling / firmware caps on access rates to avoid overheating. This is important for power users, as the last thing that they want to see when copying over 100s of GB of data is the transfer rate going down to USB 2.0 speeds.

In addition to tracking the instantaneous read and write speeds of the DAS when processing the AnandTech DAS Suite, the temperature of the drive was also recorded. In earlier reviews, we used to track the temperature all through. However, we have observed that SMART read-outs for the temperature in NVMe SSDs using USB 3.2 Gen 2 bridge chips end up negatively affecting the actual transfer rates. To avoid this problem, we have restricted ourselves to recording the temperature only during the idling intervals. The graphs below present the recorded data.

AnandTech DAS Suite - Performance Consistency
TOP: BOTTOM:

The first three sets of writes and reads correspond to the AV suite. A small gap (for the transfer of the video suite from the internal SSD to the RAM drive) is followed by three sets for the Home suite. Another small RAM-drive transfer gap is followed by three sets for the Blu-ray folder. This is followed up with the large-sized ISO files set. Finally, we have the single disk-to-disk transfer set. These graphs bring out the temporal aspect associated with the numbers in the preceding section's graphs. It is evident that the OWC Envoy Pro Mini, despite its middling peak performance numbers, shows remarkable performance consistency over a much longer duration compared to the Kingston DataTraveler Max units. This must be kept in the context of the Envoy Pro Mini's use of a native UFD controller.

PCMark 10 Storage Bench - Real-World Access Traces

There are a number of storage benchmarks that can subject a device to artificial access traces by varying the mix of reads and writes, the access block sizes, and the queue depth / number of outstanding data requests. We saw results from two popular ones - ATTO, and CrystalDiskMark - in a previous sub-section. More serious benchmarks, however, actually replicate access traces from real-world workloads to determine the suitability of a particular device for a particular workload. Real-world access traces may be used for simulating the behavior of computing activities that are limited by storage performance. Examples include booting an operating system or loading a particular game from the disk.

PCMark 10's storage bench (introduced in v2.1.2153) includes four storage benchmarks that use relevant real-world traces from popular applications and common tasks to fully test the performance of the latest modern drives:

  • The Full System Drive Benchmark uses a wide-ranging set of real-world traces from popular applications and common tasks to fully test the performance of the fastest modern drives. It involves a total of 204 GB of write traffic.
  • The Quick System Drive Benchmark is a shorter test with a smaller set of less demanding real-world traces. It subjects the device to 23 GB of writes.
  • The Data Drive Benchmark is designed to test drives that are used for storing files rather than applications. These typically include NAS drives, USB sticks, memory cards, and other external storage devices. The device is subjected to 15 GB of writes.
  • The Drive Performance Consistency Test is a long-running and extremely demanding test with a heavy, continuous load for expert users. In-depth reporting shows how the performance of the drive varies under different conditions. This writes more than 23 TB of data to the drive.

Despite the data drive benchmark appearing most suitable for testing direct-attached storage, we opt to run the full system drive benchmark as part of our evaluation flow. Many of us use portable flash drives as boot drives and storage for Steam games. These types of use-cases are addressed only in the full system drive benchmark.

The Full System Drive Benchmark comprises of 23 different traces. For the purpose of presenting results, we classify them under five different categories:

  • Boot: Replay of storage access trace recorded while booting Windows 10
  • Creative: Replay of storage access traces recorded during the start up and usage of Adobe applications such as Acrobat, After Effects, Illustrator, Premiere Pro, Lightroom, and Photoshop.
  • Office: Replay of storage access traces recorded during the usage of Microsoft Office applications such as Excel and Powerpoint.
  • Gaming: Replay of storage access traces recorded during the start up of games such as Battlefield V, Call of Duty Black Ops 4, and Overwatch.
  • File Transfers: Replay of storage access traces (Write-Only, Read-Write, and Read-Only) recorded during the transfer of data such as ISOs and photographs.

PCMark 10 also generates an overall score, bandwidth, and average latency number for quick comparison of different drives. The sub-sections in the rest of the page reference the access traces specified in the PCMark 10 Technical Guide.

Booting Windows 10

The read-write bandwidth recorded for each drive in the boo access trace is presented below.

Windows 10 Boot

The Envoy Pro Mini can make for a decent boot drive, but other options appear better suited as the focus in this workload is more on random access performance over a narrow drive span for a limited duration. The Envoy Pro Mini's firmware is not optimized for this use-case.

Creative Workloads

The read-write bandwidth recorded for each drive in the sacr, saft, sill, spre, slig, sps, aft, exc, ill, ind, psh, and psl access traces are presented below.

Startup - Adobe Acrobat

Consumers looking to edit multimedia files directly off a USB drive are better off with OWC's Envoy Pro Elektron rather than the Envoy Pro Mini. Even within the thumb drive form-factor, the Kingston DataTraveler Max can deliver better performance for these use-cases.

Office Workloads

The read-write bandwidth recorded for each drive in the exc and pow access traces are presented below.

Usage - Microsoft Excel

Similar to the creative workloads, the office workloads also see the Envoy Pro Mini making up the rear of the pack - and possibly for similar reasons.

Gaming Workloads

The read-write bandwidth recorded for each drive in the bf, cod, and ow access traces are presented below.

Startup - Battlefield V

The absolute bandwidth numbers are possibly acceptable for game consoles and transporting of casual games, but other storage drives deliver much better performance in terms of game loading times on PCs.

Files Transfer Workloads

The read-write bandwidth recorded for each drive in the cp1, cp2, cp3, cps1, cps2, and cps3 access traces are presented below.

Duplicating ISOs (Read-Write)

The Envoy Pro Mini comes out as the better thumb drive in some of the test components. As we saw in the performance consistency section, periods of long use tend to show the drive in better light compared to its competitors.

Overall Scores

PCMark 10 reports an overall score based on the observed bandwidth and access times for the full workload set. The score, bandwidth, and average access latency for each of the drives are presented below.

Full System Drive Benchmark Bandwidth (MBps)

In terms of the overall PCMark score, the Envoy Pro Mini comes with a passable score, but its firmware is not optimized for these types of use-cases. Only the Files Transfers workload component bears relevance to OWC's focus with this product, and the numbers in that don't contribute enough to prevent the Envoy Pro Mini from dropping to the bottom of the pack.

Introduction and Product Impressions Miscellaneous Aspects and Concluding Remarks
Comments Locked

18 Comments

View All Comments

  • ikjadoon - Thursday, March 2, 2023 - link

    I almost bought this a few weeks ago, thinking it had some water resistance. Turns out, none.

    Unfortunately to me, drives that fit onto a keychain will one day end get exposed to the rain and I'd hate to lose a $90 or $120 SSD that way.

    Otherwise, it's perfect. They've nailed the sneakernet re: performance, compatibility, and form factor.
  • ballsystemlord - Thursday, March 2, 2023 - link

    If you actually read the review, you'll find that this drive has poor performance compared to its rivals.
    Also, I'm uncertain how "USB thumb drive" would ever be classified as a "form factor". They literally come in many shapes and sizes.
  • Wrs - Thursday, March 2, 2023 - link

    It did fine. The rivals are either 2-2.5" connected by cable (T7, Envoy Pro) or a thin proper thumb drive that forces you to choose USB A/C and stutters to 50MB/s transferring large loads (DataTraveler).
    Thumb is an important form factor for some - keychain portability and not blocking adjacent USB ports. The compromise is thermal dissipation, and review shows Envoy Mini is more efficient than DTMax.
  • ikjadoon - Friday, March 3, 2023 - link

    If you actually read my comment (jeez, such hostility from someone that's wrong…), you notice I wrote for sneakernet.

    For that, the form factor is much better: no flailing cables, not too large even for small pockets, not too large to block adjacent ports, light enough to keep on you 24x7, durable caps, etc.

    How do you not understand that a USB flash drive can have multiple form factors? How do you misread a comment so badly? See, I wouldn't post a comment if I was as confused as you.

    Sigh….
  • ballsystemlord - Friday, March 3, 2023 - link

    I did not actually have pictures of the other items to compare so I had no idea this USB thumb drive was being compared to non-USB thumb drive devices. I did check that they were all USB connected devices before posting.

    Regarding sneakernet, I did not cache the reference. I thought that practice was actually dead.

    Now that I understand I have to say that USB thumb drives, at least several that I have owned, do block the other USB ports. To the point where I bought a 6in extension cable. It doesn't take much to block the next port over.
  • PeachNCream - Friday, March 3, 2023 - link

    Absence of waterproofing may not be a significant problem. As someone with a history of running USB storage through laundry (4 different drives so far - wash and high temp dry - two of them have been neglected in that manner multiple times), nothing I've done has caused a failure. One even survived as a bare PCB after the cheap plastic casing came apart. I just reassembled the plastic parts and its currently held together with masking tape. You'd be surprised how durable solid state storage can be even when neglected and abused.
  • fazalmajid - Thursday, March 2, 2023 - link

    So you should be able to ditch the bulky A-to-C adapter by unscrewing the Torx screws. I've been looking a long time for a good replacement for my 2015 vintage Lexar Jumpdrive P20. The plastic construction of the Kingston or SanDisk Extreme rule them out. The dual A and C swiveling SanDisk Ultra Drive Luxe is convenient and it's available in capacities up to 1TB but it's also very slow.
  • Samus - Thursday, March 2, 2023 - link

    I've had one of these for almost a year and am thrilled you reviewed it. After seeing it at CES last year (where I believe it was announced) I got one as soon as it came out and it's basically my entire IT portfolio (images, ISO's, tools, software) in a highly accessible portable fast enclosure.

    It's worth pointing out it is also bootable to the point you could run an entire system off it if need be. I have mine setup to boot WinPE.
  • ganeshts - Thursday, March 2, 2023 - link

    For IT folks, that is the perfect use-case for these dual-interface high-performance thumb drives. I have been using an old 128GB NVMe SSD in a dual-interface 'thumb form-factor' enclosure (Akasa) for a couple of years now (with Medicat USB installed).

    Since the use-case is mostly read-heavy after the initial loading, something like the Envoy Pro Mini is a good enough alternative. Despite not emerging the outright victor in any of the short-term benchmark numbers, I think OWC has a winner here. Some additional SLC cache would be nice to have, but that is nit-picking at this stage.

    Hopefully, OWC keeps improving this product with newer NAND generations and controllers (Gen 2x2 / USB4 etc.)
  • ViRGE - Thursday, March 2, 2023 - link

    I like the performance, but the fact that this isn't natively USB-C is a downer. It's 2023, and OWC is a company that has its roots in providing Mac accessories. It should be a USB-C drive with an adapter for A, not the other way around. After all, it's not like USB-A adoption is increasing over time.

Log in

Don't have an account? Sign up now