The Future Of Storage Management: When The Going Gets Complex, The Smart Enable Self-Service

Drowning in data? Traditional storage can’t keep up. AI and automation are enabling a new era of self-service storage provisioning and automation. Learn how to streamline storage management and empower your IT team for innovation in this article by Fred Lherault, Field CTO, EMEA Emerging, Pure Storage.

Storage management is quickly evolving. Driven by AI and digital transformation, data centres must handle massive data growth, unprecedented application agility requirements, spiralling complexity, plus skills shortages and soaring energy costs.

It’s not so much a sea change as a perfect storm. But, that brings opportunities for improvement. Organisations can modernise their IT infrastructure and operations to take advantage of a new era in storage management built around policy-based management, self-service provisioning and automated storage-as-code approaches.

The data challenge

Of the fundamentals that impact storage management, our relationship with data is crucial. Since 2010 the average annual rate of data growth has been around 38%, according to Statista. That includes some peak years — such as during the pandemic — when data creation spiked, with a total predicted volume of 181 zettabytes globally by 2025. A staggering amount, with no signs of slowing down. Deployed storage has tracked increased data volume, with 6.7 zettabytes installed in 2020 — bearing in mind a lot of data created is not retained — and an annual growth rate of 19% predicted to 2025.

Planning for such rates of data growth is a headache for enterprises, especially with the traditional investment and depreciation cycle. In the past, being able to accommodate unplanned requirements meant having to hold extra hardware in reserve.

At the same time, soaring electricity costs have driven the need for better use of energy in the data centre. IT organisations now also need to focus on efficiency and sustainability of their infrastructure.

The act of storage provisioning was also a time (and headcount) consuming process, particularly in organisations vertically-segmented between business departments and IT’s sub-units of software, servers, storage and networking. Requests to deploy and provision storage necessitated a tortuous route from requesting units to storage admins. Often the organisation needed many admins with multiple skills to manage numerous storage environments across several storage vendors and technology. All of this could easily take weeks.

When the going gets complex . . .

Dealing with data growth and increased application complexity previously meant throwing extra resources at the problem in the form of additional storage administrators. However, to simply increase storage admin headcount is not only inefficient, but practically impossible in today’s IT landscape. The skills to manage multiple legacy storage arrays often don’t exist or are very costly to hire. Not only this, but modern organisations want their employees to be focused on innovation, increasing productivity and customer experience, rather than day-to-day provisioning tasks.

Today, labour-intensive approaches to storage management no longer make sense. The old vertically-segmented paradigm of IT management has melted away in the face of new IT architectures. We are in transition — via digital transformation — to new norms that include cloud operating models and infrastructure-as-code with storage specified as part of the application development and deployment process. Agility and speed of delivery are now more important than ever and organisations are expected to be able to adapt to new requirements swiftly.

. . . the smart enable self-service

Where dedicated storage resources are employed, organisations want to significantly increase the volume of storage managed per FTE. But, crucially to the new paradigms, much storage provisioning passes over to development phases of application deployment, triggerable by self-service selection of storage performance profiles or coded in via APIs. With new application deployment models such as containerisation, storage resources are defined as part of the application configuration file, designed to be deployed automatically by the orchestration platform.

Here, applications spin up and scale rapidly with storage requirements built into those lifecycles, and at a rate often beyond what humans can manage manually. At such a speed of operations, storage management has to work via pre-set policy, auto-provisioning and auto-scaling.

Storage management in the digital transformation era

The future of storage management centres on customer-set storage performance profiles selected via self-service or entirely automated in the application environment via storage-as-code and orchestrators.

The smart IT organisations will transform the role of the storage administrator to look like a product manager. This means defining storage “products” through abstraction concepts such as storage classes and policies. This also means enabling self-service by the organisations’ internal customers and orchestration platforms rather than being the person that provisions storage objects, leaving the new storage product manager free to manage fleets of devices rather than individual storage arrays, monitor the use of the existing services and create new offerings.

To enable this cloud operating model, some storage vendors can offer platforms with which storage admins can manage and add capacity to their fleet of storage devices non-disruptively, as well as define classes of storage, policies and availability zones to be made available to their consumers. Meanwhile, developers that use infrastructure-as-code platforms can specify the class of storage they want for an application, code it into templates, test it and deploy it.

For fully-automated containerised environments, businesses should look for the leading enterprise-grade Kubernetes data platform for modern applications. This will provide storage management for Kubernetes clusters with automated deployment and scaling of persistent storage, plus data protection. Some solutions even include data services on top of this, such as a curated set of databases for use in Kubernetes environments, which deploy with just one click of a button, saving a huge amount of time for developers.

Note however that enabling self-service while still being dependent on long purchasing and delivery cycles leaves the organisation open to risk. Fully embracing these concepts means consuming the underlying storage platforms as-a-Service, to enable rapid scale where needed, enable flexibility, but also avoid investing in hardware ahead of when it is actually needed. Businesses should look not only at vendor’s technology but also their ability to provide their platforms through flexible, on-demand consumption models.

How can you achieve seamless, modern storage management?

Some storage vendors can indeed offer all of the above; the benefit of working with a single storage supplier is that integrated product sets create efficiencies all through the stack. All flash storage arrays linked across a single operating environment means workloads can be managed at scale across the business from a single view, seamlessly alongside a Kubernetes data platform and all the self service storage functionality a modern business needs.

There are also huge benefits in utilisation and energy efficiency. Some all-flash vendors can offer up to 85% increased energy efficiency compared to competing all-flash products, and a much smaller footprint to maximise data centre efficiency. Finally, working with a vendor that can offer all of this through a flexible, SLA-backed storage as-a-Service offering that features guarantees around energy usage, zero data migrations and non disruptive upgrades is the key to achieving seamless, modern storage management.