Mapping Technology Trends to Enterprise Product Innovation

Scope: Focusses on enterprise platform software: Big Data, Cloud platforms, software-defined, micro-services, DevOps.
Why: We are living in an era of continuous change, and a low barrier to entry. Net result: Lot of noise!
What: Sharing my expertise gained over nearly two decades in the skill of extracting the signal from the noise! More precisely, identifying shifts in ground realities before they become cited trends and pain-points.
How: NOT based on reading tea leaves! Instead synthesizing technical and business understanding of the domain at 500 ft. 5000 ft., and 50K ft.

(Disclaimer: Personal views not representing my employer)

Sunday, December 11, 2011

Storage Array Clustering: The Hypervisor of the Storage Layer?

Enterprises are experiencing an exponential growth in data -- Exabytes is now the new Petabytes.  As administrators incrementally add capacity by rolling in new storage arrays, they soon realize the management nightmare of silo'ed storage islands. Manually mapping servers and applications to islands of storage soon becomes sub-optimal -- the inverse of an elastic dynamic storage framework that allocates storage capacity, bandwidth and performance in an on-demand fashion.

Storage companies are seeing the writing on the wall. New breed of storage array offerings (block and file) will increasingly focus on the resource federation platform that essentially is a layer of indirection between the data namespace and the physical storage resources (storage ports, controllers and disk resources). This layer is also dubbed as the "hypervisor for the storage layer" --  the host accessing storage continues to connect to a vNIC/vHBA on the storage array side, while the array is potentially optimizing the allocation of physical disks, ports, CPUs, under the covers, and completely transparent of the host.

This is arguably a disruptive storage trend.  NetApp's recent ONTAP 8.1 C-mode announcement is indicative of this trend -- it has two offerings,  one for FAS (upto 24 node) & the other for block (upto 4 nodes). Resource federation is essentially similar to the goals of a scale-out file-systems (Panasas, EMC Isilon, IBM SoNAS), with an emphasis of making the process of continuos storage optimization completely transparent to the host.

In summary, resource federation within the storage layer hides/virtualizes the physical storage resources and physical data placement from the host/hypervisor. As such, QoS optimization operations need to coordinated much more tightly between the hypervisor and storage layers. This also gets us closer in making policy-based communication between the hypervisor and storage layer indispensable.   

No comments:

Post a Comment