EMC VNX Storage Processor (SP) Cache Configuration

One of the most commonly mis-configured parameters on the VNX is the Storage Processor (SP) Cache Configuration.  The allocation of read and write cache memory on the VNX is typically mis-configured when a Storage Administrator adheres to antiquated application recommendations for cache configuration.  For example, a table in the Microsoft Technet Exchange Server 2010 TechCenter Library (see bottom) provides the following “Best Practice” regarding Storage array cache settings: “Best practice: 75 percent write cache and 25 percent read cache (battery-backed cache).“  Let’s give Microsoft credit they did add the sentence,  ”Follow storage vendor best practices.” to the end of that recommendation. Unfortunately most Administrators do not consult with their storage vendor for best practices recommendations, 25 percent read cache configuration for a VNX would be considered grossly oversized.  The EMC Unified Storage Best Practices for Performance and Availability – Common Platform for Block Storage 31.0 – Applied Best Practices document makes the following recommendations:

• It is always more important to maximize the storage system’s write cache size.
• Always allocate all the remaining memory after system memory is allocated to features and applications to the read/write cache.
• Note that write cache allocation applies to both SPs, it is mirrored.  Read cache is not mirrored, to use the available SP memory most efficiently ensure the same amount of read cache is allocated to both storage processors.
• It is advisable to have at least 100 MB of read cache for the block-only VNX5100, and at least 256 MB of read cache for File-enabled systems.
• Only rarely and with serious consideration should more than 10-percent (10%), or more than 1024 MB, of available memory be allocated to the read cache.
• Very few workloads benefit from very large read caches.  A 1024 MB read cache is considered a very large read cache.
• Recommended initial Read Cache (MB) – VNX5100 (100 MB), VNX5300 (400 MB), VNX5500 (700 MB), VNX5700 (1024 MB), VNX7500 (1024 MB)

IMPORTANT: Although the document also states, “The amount of memory used by read or write cache can be changed at any time without disruption.”, it is EXTREMELY IMPORTANT to note that in order to make changes to the cache, cache will be disabled.  LUNs are still accessible however performance is degraded SIGNIFICANTLY when caching is disabled.  DO NOT make changes to the cache configuration casually, schedule a period of time when there will be lower I/O activity to limit the impact of the change.

The following table provides guidance about storage array configurations for Exchange 2010.

Supported RAID types for the Exchange 2010 Mailbox server role