Storage Wars – Techniques and Technologies to Manage Rising Data Volumes

Share on linkedin
Share on facebook
Share on twitter
Share on email
Share on print

Some very interesting technologies are emerging in the area of data storage. In this blog post, we’ll explore the topic of storage, looking at the following:

  • Why the need for storage is rising in large enterprises,
  • Various types of storage methods and models,
  • Why recovery is equally if not more important than storage, and finally,
  • Emerging trends in storage that we’re observing in our client base.

Why Storage is More Critical than Ever Before

The term “big data” is on everyone’s lips these days and it is an accurate indicator of what is happening in today’s large enterprise.  We are living in the age of mushrooming data volumes. Whether it is online trading, online shopping, manufacturing automation, the use of mobile devices, or participation in social media large enterprises and their workers are generating data at an exponential rate.

For those enterprises using business intelligence (BI) tools, there is also a larger data requirement. Companies use BI to better understand their business, often looking at business performance over multiple years, which requires the retention of many years worth of data.  Many companies also want to slice and dice the data at hand, or view data through different “lenses” to gain perspective and insight into their business. This approach may require companies to keep multiple copies of data, or many years of data on hand.

Regulatory compliance is yet a third reason for companies to up their data storage requirements. Many public enterprises, including those in industries such as healthcare or financial services, are subject to stringent compliance requirements that legally require companies to keep multiple years of data – in some cases up to seven years.

Also, new applications the larger data storage requirements that come with them can create a ‘perfect storm’ of data exploding through the enterprise. Combine any two, or all of these factors together and it is clear that IT Directors must continually revisit their storage requirements, their storage strategy & methods, and its enabling technologies.

Techniques and Technologies for Data Storage

If your goal is to invest in a storage strategy that minimizes the cost and complexity of storing data, and wherever possible minimizes the time required for backup while accelerating recovery time, there are a number of approaches now available to IT departments.

This strategy begins with determining the storage requirements of your business applications. It is necessary to consider the actual nature of the data you need to store, as different types of data demand different storage methods.  If capacity is the primary requirement, then a company may opt for slower speed, larger drives.  In the “old” days ten terabytes of storage was considered enormous. Today, with data volumes mushrooming and with companies leveraging BI tools, it is not at all uncommon to see companies investing in hundreds of terabytes of storage.

If performance is the main requirement, then a company may look to SAS, FibreChannel, or solid-state drives and possibly memory caching of data.  Solid-state disks and memory caching have become very popular of late in companies with virtual desktop environments, where there are large numbers of desktops starting up at various times of the day. Data caching also helps reduce the time to load frequently accessed files whether it be virtual machine disks or heavily used database or application files.

It is critical to consider readiness for data loss and Disaster Recovery. These will drive back up and data restoration requirements, which in turn will determine technology requirements.  Very often, back up problems are related to a failure to back up the necessary data. Many companies will address backups by taking a snapshot of data at a particular point in time, but may not actually capture and back up data in real time.  Yet despite this concern, an IT Director must weigh the cost/returns of storing absolutely everything against backing up high value business data.


To minimize storage required and improve efficiency, companies also are turning to data deduplication.  Companies such as NetApp are catering to VMware virtual desktop environments, with tens or hundreds of near-identical Windows servers and desktops that deduplicate very well.  By identifying redundant data and saving only the changes, deduplication can dramatically reduce the amount of storage required.  For example, an email system might contain 100 instances of the same one-megabyte (MB) file attachment. If the email platform is backed up or archived, all 100 instances are saved, requiring 100 MB storage space. With data deduplication, only one instance of the attachment is actually stored; each subsequent instance is just referenced back to the one saved copy. In this example, a 100 MB storage demand could be reduced to only 1MB.

For organizations where backup speed and efficiency are key, HP offers appliance-based backup technology that provides deduplication on the fly that not only reduces disk requirements, but also restores very fast. This allows companies to more effectively utilize backup to disk versus backup to tape – saving costs and time.

Migrating and Balancing Data

With such rapid growth a lot of customers may find themselves with several different storage devices.  Migrating and balancing data between these devices can be a difficult task.  Foxnet looks to the F5 ARX solution set to assist in these areas.  The ARX will act as a data proxy and enable storage tiering between storage devices.  Ensure your most utilized data is running on the expensive fast disk and the relatively untouched data is running on slower more inexpensive disk.  Similarly this product can help with large data migration tasks allowing customers to move from an older storage system to a newer system with ease.

Foxnet Can Solve Your Storage Dilemma

If your company is reviewing its storage strategy, or is in need of a partner to devise and implement storage technology, FoxNet has significant expertise in this area. Our solution consultants have crafted creative storage solutions for many of our clients.  Drop us an email at info@ or give us a call at 519-886-8895 to discuss your storage needs.

Foxnet Storage Partners