Have a strategy for archiving? If not.. get one!

You need a plan for archiving

You have static data that needs to be archived in a secure way. This has to happen because the data volumes are exploding. EMC has a way to make this happen, within one offering, with one plan.
Not having a plan costs you real money, real customer satisfaction and stops your future development in its tracks!

Challenges ahead

Today, data protection is so much more than just backup. Customer demands drive ever tighter protection needs, all the way through continuous availability. This says that we cannot keep all the data at that level, it’s just not financially viable. It’s not getting any easier, EMC and IDC in cooperation just announced the 7th Digital Universe study, the findings give cause for thought. We will double the size of all data every 24 months, but there is reason to believe that there will never be double the amount of IT professionals. Also, hands up everyone who thinks their IT budget in two years is twice what it is today.

This calls for a strategy!

Static data just has to move out of the way and stay protected without having to get copied away over and over again. Backup certainly isn’t the answer here..       

In order to strategize, we need information, tactics and visibility.

Information

What is getting stored, what is it worth,
who needs to have access to it,
how long should we keep it for?

Tactics

How do we extract the data, where do we put it, how will it be accessed, how will it get protected?

Visibility

Is the data compliant with rules, regulations and laws? How do we search for it? How to track it, retain it, discard it?

Let’s plan!

You will need three components to make the strategy a reality:

Protection storage

Put the archived content somewhere where it can stay for as long as it needs to, in a pool of storage that reduces costs, heals itself, and guarantees the integrity.

Application integration

No matter where your content comes from, it needs to be analysed, tiered and moved. All the while making sure it is as seemless as possible for your end users.

Data Management

Offer the services out, make sure they are easily consumed and user driven, no matter what the compliance and economic demands are. Easily add value by offering discovery, search and hold to the data that is now in one central archive.

Components of the solution

EMC offers a wide range of systems that can assist in being the persistant and central storage for your archive. More and more, we see customers converging archive and backup storage on to DataDomain systems because of a few reasons:

• Deduplication because frequently many copies exist, even in an archive, content is also often redundant within different documents and files. Deduplication allows more content to be stored within a smaller footprint, more economically and with less administration.
• Data Domain Data Invulnerability Architecture assures the data is safe from the initial write, then over time by checking the integrity after reduction, write and periodically after that. Finally content is verified again at the retrieval request.
• Data Domain has been designed to support both backup and archive workloads. These multiple use cases are vastly different, but DataDomain can support 100’s of millions of small files, protect them and replicate them. Other demands for archival protection is retention enforcement and possibly encryption, all ticked off by an impressive list of archive application integration, to date over 20 different, a likewise impressive of software options that handle these demands.

The EMC Data Protection Suite for Archive will support your needs for file, email and SharePoint archiving. You can easily add in functions for supervision and discovery as needed. The foundation here is EMC SourceOne which scales from small, single server deployments for teams and small business needs, all the way through vast enterprises with hundreds of thousands of mailboxes, petabyte scale file storage and ginormous Sharepoint farms, EMC SourceOne provides you with the application integration and the visibility required.

Let’s go!

The time to strike is now, data growth isn’t waiting, you need to get situational awareness sooner rather than later. Reach out to an EMC partner of choice or your EMC representative, get an assessment of your situation. There are readily available tools to look in to your email, file servers and Sharepoint data that assesses volumes, age, utilization. This builds a picture for you to build and start executing on a strategy.

Announcing EMC DataDomain DD Boost for Enterprise Applications

EMC has announced availability of DD Boost for Enterprise Applications.

 

What is this then?

New ways of protecting data?

Something new to learn, again?

DD Boost for Enterprise Applications consists of two parts:

• Modules for using EMC DataDomain DD Boost together with native data protection functionality from:
  • SAP HANA
  • SAP with Oracle (BR*Tools)
  • Oracle (RMAN)
  • Microsoft SQL
  • IBM DB2
• EMC Data Protection Advisor (DPA) for monitoring and reporting on the various sources.

The modules allows application administrators (DBAs) to work with their own interfaces and workflows, yet still move data to protection storage in an efficient manner.

This means that instead of having to install third party backup software and giving another team the control over scheduling and workflow, the DBAs now control, when and if they protect their data.

In some applications they even stay in control of moving this data offsite, simply specifying that the DataDomain target should perform what is known as a Managed File Replication to one (or more) DataDomain system(s). Recovery of the data can take place from any of these systems.

For some of the systems, replication can be controlled by writing to different folders in the DataDomain system, if they write to one that is enabled for DataDomain mtree replication, the data is moved offsite as soon as the backup is done, write to one where it isn’t and data stays onsite.

The whole point of developing this functionality is that in many organisations, there is a rift opening up between what application owners demand and require and what the IT operations department can deliver. Eventually, you see everyone rolling their own solutions and creating the dreaded silos that breed cost, complexity and poor interoperability.

DD Boost for Enterprise Applications addresses one of these rift creating scenarios, namely the classic problem that DBA’s does not want someone else to interfere with their often very complex environments, in this case through backup and recovery via a backup application.

The “classic” way, dump and sweep or backup agents:

EMC began addressing this with DD Boost for Oracle RMAN which allowed Oracle DBA’s to continue using their own RMAN scripts, schedulers and so on.

They could write directly to highly resilient data protection storage in an efficient manner.

‘The success has been great with this, thousands of EMC customers has adopted it and carved out complexity and cost while doing so.

For the rest, it was still dump and sweep or backup agents.

Now, the same opportunity comes to administrators of SAP, SAP HANA, Microsoft SQL Server, and IBM DB2.

Nothing new to learn for the backup admins, minute long deployment and automatic monitoring for the data protection teams. That’s what this announcement means!

Backup to the future: Datadomain DDOS 5.4

In this post I will try to pick apart some of the new functionality that appeared in the latest and greatest DataDomain DDOS release.

In one of my earlier posts , I have an overview of the four new DataDomain systems and what they are, you can read that here:

Backup to the future: New DataDomain systems

DDOS is the denominator for the operating environment (OE) of EMC Datadomain systems and these have now reached version 5.4.

Functionality in DataDomain systems come in play both in the Protection Storage part as well as in the Data Source Integration parts of the overall EMC data protection architecture.

 

The object of DataDomain, a Purpose Built Backup Appliance (PBBA) is of course backup, but also to protect archival data, DataDomain systems are designed to take care of both use cases.

 

A big part of the DDOS 5.4 release was supporting the expanded integration into Avamar 7, which I wrote about here:

Backup to the future: Avamar 7

We also launched support for what is now a very extensive list of Archive partners:

 

SAP HANA backup

Something we can all be sure of is that all databases will get bigger and have higher and higher demands on availability and performance.

This usually translates to database administrators will demand control over the protection of their data, if the backup teams doesn’t deliver the protection architecture, they will take matters in their own hands, this has been discussed by people much smarter than me, such as the CTO for EMC Backup & Recovery, Steve Manley.

What EMC presented today is a way to protect SAP HANA in-memory databases, all within the database administrators control.

Backups will flow directly to the DataDomain system, without any need to stage it in between. This means significant savings on infrastructure and storage.

At the same time, the efficient replication that DataDomain systems provide will make sure that protected data is sent off-site in the shortest time possible.

Oracle Exadata Backup and DR

Another really exiting industry first is that EMC presented support for direct integration to Oracle Exadata environments through EMC DataDomain DDBoost.

Again, this means that administrators or Exadata systems can take direct control of their data protection and if need be, disaster recovery.

Data Domain systems are the first deduplication storage to be Oracle Backup Solutions Program validated for Exadata.

Thank you for taking your time to read my musings on this and keep an eye open for upcoming posts where I dig deeper in the subjects.

Backup to the future: Networker 8.1

In a few other posts, I have written about other components of EMC’s Backup to the future launch.

The turn has now come to another member of the data Management and data source integration components of the EMC data protection stack:

Today, EMC announced Networker 8.1, building on the foundations of Networker 8.0 that was launched in 2012.
Networker 8.0 was the biggest release since EMC acquired Legato ten years ago.
The foundation that was put in then was dubbed “Networker for the next ten years” and certainly is fundamental for the barrage of groundbreaking news that was announced.

In this post, I will try my best to shine a light at some of these, there will be many opportunities going forward to go deeper and discuss more.

Virtualization backup and recovery:

With a sibling like Avamar, with its leading VMware functionality, Networker has been seen as behind in many virtualization use cases.
This all changes now, EMC Networker is getting backup and recovery powered by Avamar.

This is a theme you might recognize from VMware vSphere Data Protection, which is also powered by EMC Avamar technology.

So the goodies delivered here are some of those seen in Avamar, like Changed Block Tracking for backup and restore, load balancing, multi streaming and more.

VMware administrators can work with image level backup and recovery straight from the vSphere Web Client through an integration plugin and set up backup policies, kick off both restores and backups and assign policies to VMs.

New VM’s can inherit backup policies automagically when such have been established for the environment.

End users can make their restores without relying on the IT team through a new web interface.

All these treats are unlocked through deploying a Virtual Backup Applicance, which is then managed through the normal Networker Administration Console.

Integration with EMC primary storage systems:

NetWorker 8.1 has brand new snapshot management for EMC primary storage including VMAX, VNX, and RecoverPoint systems.

The integration is dead simple as one would rightfully expect when the components all come from one vendor.
Everything is done through a wizard that discovers the primary storage systems and queries the snapshot schedules there.

When all this is done, snapshots can be controlled from Networker and rolled off to either a DataDomain system or any other disk.
If need be, these can then be cloned off to tape or other devices as needed.

Block based backups:

This is a feature with an innocent sounding name, but wields great power.

It is an integrated feature in the Windows client, and works from Windows Server 2008 R2 and up.
Block based backup works in conjunction with Microsoft’s VSS framework.

What it does is take an image-based backup at the volume level, rather than walking the entire file system, in the backup process.

It’s as simple as ticking a check-box to get working and when you tick that box… hang on tight.

As said, there is no need to walk the file system, so backups start flowing immediately and no index is created.
Only the changed blocks are backed up, enabling fast backup with minimal impact.

Recovery is achieved by mounting the resulting Virtual Hard Drive (VHD) to browse and extract the necessary files, single file, directories or the whole shebang.

In internal testing, we have seen file system backups complete up to 5x faster than a traditional file backup and recovery operations are 2-4x faster.

Backup and recovery control for system owners:

For DBA’s, just like with the example of VMware administrators, there are great news too.

Since most database administrators like full control over their backups and recoveries, many organizations still work with a classic “dump and sweep” workflow.
While it does provide the desired control, it also creates multiple handoffs, synchronization issues and generally a lot of growling between IT roles.

Oracle collaborative protection

For Oracle, Networker 8.1 introduces “collaborative protection”.
Fundamentally, DBAs are able to easily manage their own backups with their native tools and RMAN scripts, but Networker provides backup catalog synchronization as soon as these RMAN scripts are kicked off.

When credentials are exchanged between the DBA and backup crowds, they can each go back and continue on, no need for Networker admins to learn Oracle, no need for Oracle admins to learn Networker.

Suddenly, database dumps are automatically backed up to protection storage as needed, when needed and at the same time they are indexed as such.
Restores from RMAN scripts, without any changes, are now in a single, automated step, directly from the backup location.

Other system owner integration

Microsoft SQL owners get an SQL Management Studio plug-in that enables them to take backups, perform restores and more, all without having to involve the backup team.

For IBM DB2, there is integration into the new’ish IBM Data Studio.

There’s more

Much more…

Networker Module for databases and applications (NMDA) in its new version 1.5 provides support for.. drumroll please, MySQL.

There has been debate and thoughts on if such a plug-in would actually be used, but with the growth of MySQL, EMC decided to put forth the engineering resources needed  to develop this support.
This isn’t your typical release 1.0, straight out of the doors you see a very comprehensive feature set.
You have hot and cold backups, full and granular backup and restore as well as backup and restore of logs. Also supported is restore to any point-in-time of a backup with no third party tool dependency. Both MyISAM and InnoDB, the most popular underlying storage engines for MySQL, are supported.

In Microsoft land, you see support for optimized backup of deduplicated volumes, support for SQL Server 2012, Exchange 2013 and SharePoint 2013 as well currency with all new updates and service packs.

I could go on and on about all the cool stuff in Networker, but for now I will relent and let any surviving readers (thank you) actually go out and be productive again.

Backup to the future: Avamar 7

In the big launch today, EMC launched a whole array of new and groundbreaking things.
I am trying to break these up in a series of posts and explain them in a high-level fashion.

They all have roles to play in the EMC BRS stack:

The first post was a bit of explanation about the new DataDomain systems.

The turn has now come to Avamar, which was announced in it’s latest generation, Avamar 7.0

This is a major release and my objective here is to give a brief introduction to some of the many enhancements.

Avamar 7 brings forth a bunch of things, the first one I am going to touch on is the further integration that is happening with the Protection storage part of the stack, namely the DataDomain systems.

All data center workloads can now be targeted to a Data Domain System using EMC DataDomain DD Boost (except one, I’ll get back to that).

Previously file systems and NDMP workloads were exclusively sent to the Avamar DataStore, or as it is popularly called, “the grid”.
This means even more scalability for Avamar, unstructured content such as files certainly are high growth workloads and many are struggling with containing these.
An excellent strategy for limiting the growth of file backups is archiving, but that is a subject for a later post. 😉

New functionality for virtual environments

You will see a bunch of new and cool functionality for both VMware and Microsoft virtual environments.

On the VMware side, Avamar, when backups are sent to a DataDomain system now offers what is called Instant Access.
Instant Access means that the VM is presented from an NFS mount on the DataDomain and then is accessed there and is up and

running, without any “restore” having been performed.
The performance characteristics of a purpose built backup appliance is radically different than a production storage system, so expect that the VM will run with much fewer IOPS. The good news is that you can immediately start a  Storage vMotion, which will move the running machine back to production storage.

Now, on this matter, I will go on a tangent, and describe a bit about how Avamar normally does restores, to show the different tools that a backup admin has at his or her disposal.

Avamar can perform an image backups with VMware Change Block Tracking (CBT) like most other backup software

variants out there.What that means is that at the time of backup, only the blocks changed since the last backup will be scanned, these blocks are then deduplicated against all other blocks Avamar tracks and those that are globally unique are then compressed, encrypted and sent to the protection storage.

Nothing unique here, most backup brands have this functionality today, some only from full backups, etc, Avamar does it every day, every backup is a full backup, but only globally unique data is sent.

Now, on to the really unique stuff. When it comes to restores, Avamar uses the CBT functionality again, querying the vSphere again about what has changed between the point of backup and now. Only the blocks that is needed to rebuild the point in time is then sent back.
Most changes that require a full image restore are trivial, so this means that restores are amazingly fast in most scenarios.

Comparison between a standard image restore and a Change block tracking restore

So what we see here is that there are a multitude of possibilities for making restores, from a single file to a whole VM.

Now, on the VMware backup/restore management side, Avamar 7 also has a whole host of new functionality, some of it really redefining stuff.

The first new functionality I will mention here is the vCenter interface integration.

The VMware admin usually feels most at home in the vSphere Client.
Avamar 7 provides a plugin here, so that there is no need for her or him to go learn another interface.

From here, the VMware admin just clicks EMC Backup and Recovery and can immediately configure backup policies, initiate restores or kick off a backup job.

Another new very useful functionality is what is referred to as “Dynamic backup policies”.
What that means is that backup policies can be applied to various different entities within the vSphere hierarchy,  such as a cluster, a whole datacenter or what I guess will be most common, a group (folder) of VMs. Any VM that gets added to these entities will inherit the backup policy automatically to ensure new VMs are always protected.

For those out there that are using Microsoft Hyper-V, there are also really good news!
EMC is adding support for Hyper-V 2012 to the already existing Hyper-V 2008 support.
Support for using backup proxies in clustered shared volumes (CSV) has been added, meaning that one host can be pointed out to take care of the I/O generated by backups.
Further, in a Hyper-V 2008 R2 implementation, when a backup was started for one server in a Hyper-V over CSV cluster, all of the other Hyper-V servers in the cluster went into what is called I/O redirection mode. This meant that when the backup was running, all other Hyper-V servers had to send their I/O write requests to the Hyper-V server being backed up, slowing their write I/O performance. This re-direction now eliminated in Hyper-V 2012 backups with Avamar. Thanks for fixing that one Microsoft :).
Hyper-V 2012 got a lot better, and Avamar is right there with it.

NAS system / NDMP backups

Since many many years, Avamar has had really leading functionality for NDMP backups through the innovative use of an “NDMP accelerator”.
This means that after an initial full  (level 0) backup, Avamar can continue to only request incremental (level 1) dumps from the filer daily, but still providing a full backup.

Avamar supports the backup of EMC Isilon, VNX and VNXe, Celerra and NetApp systems with this method, and Isilon is the new kid in here.
For now, Isilon has some restrictions, primarily because Isilon can get tremendously big, we want to make sure that the backup solutions are positioned correctly here.

Management of Avamar itself has also gotten an overhaul. A *ahem* less than loved part of Avamar management has been that there has been what is a tad non-poetically named “Blackout Window”, where no backups could be performed.
In Avamar 7, this restriction is removed, only leaving what is called a “maintenance window”, with fewer restrictions.
All deduplication systems need some time to take care of cleaning, expiration, integrity checks, etc. The good news is that the parts which required a read-only system previously now run in the background.

Wrap-up:

Still there?
Thanks! 😀

There are tons more to mention about this release, especially on the application integration side, with Microsoft Windows, Microsoft applications, Oracle, IBM DB2 and many many more, all getting major enhancements. I will save this for yet another blog post.

Watch this space!

Backup to the future: New DataDomain systems

This is a big launch, EMC is presenting four new DataDomain systems, joining the others.

You can rewatch the whole “Backup to the future” launch event:
http://bit.ly/134gk7U

These are:

• DataDomain DD2500
• DataDomain DD4200
• DataDomain DD4500
• DataDomain DD7200

All these new systems are based on Intel Sandy Bridge processors and in some cases provide 4x more performance than the systems they are replacing.

More performance is important, even if the systems they replace certainly weren’t slouches.
For some users of the systems, even more important is the fact that these systems can take 2-3x more streams (write/read/replication) than the previous systems.

These systems really are a new generation, they come in new chassis types, these are fully EMC designed and those that use more than one type of system from EMC will appreciate the similarities.

With the new design comes the usage of the nice EMC designed I/O modules which will come in all known varieties known from before, such as copper and optical 1/10 GbE, Fiberchannel. They are smaller and easier to service than the legacy form factor.

These so called “SLIC” modules will also be used for things like the NVRAM and battery backup.

All in all, these can be installed and swapped out without opening the chassis or removing the unit from the rack, making it much more serviceable.

So, some specifics about these systems.

DataDomain DD2500

The DD2500 replaces it’s predecessor, the DD640.
It offers 4x faster performance and 4x more scalability.

This is a 2U system that is powered by an Intel Sandy Bridge CPU.

The system can be expanded with one DataDomain ES30-30TB shelf and can scale to four of those or three ES30-45TB shelves.

Internally, it has seven or 12 3 TB HDD’s.

The maximum of this system is 133 TB of usable capacity, that is, before the data gets reduced through compression and deduplication.

DataDomain DD4200

The DD4200 replaces it’s predecessor, the DD670 and offers 4x faster performance and 10x more scalability.

This system can back up 175 TB in under eight hours
since it can do 22 TB/hr reads with the use of EMC DataDomain DD Boost.

It can be expanded with 8 ES30-30TB shelves or 5 ES30-45TB shelves with up to 189TB of usable space.

This system can also use legacy shelves from older systems if a customer chooses to upgrade his older DataDomain.
All it takes is to swap the head out with all the data in place.

Remember, all DataDomain systems have a single namespace and everything deduplicates against everything else.

DataDomain DD4500

The DD4500 is the replacement for the DD860, which certainly wasn’t a slouch, but the DD4500 offers 2x faster performance and  2x more scalability.

This is a very scalable system that allows up to 12 ES30-30TB expansion shelves or 8 ES30-45TB shelves. This makes for an impressive 285 TB usable space.

Just like the DD4200, the DD4500 can use legacy shelves from older DataDomain systems in an upgrade scenario.

The goodies doesn’t end there through. The DataDomain DD4500 can also be supercharged with the Data Domain Extended Retention software option. With that, customers can use this system for a massive 1.1 PB usable capacity for long-term backup retention.

DataDomain DD7200

Now we are talking large scale, but not largest 🙂

The previous second largest system in the DataDomain family was the DD890, the DD7200 replaces it and provides twice the performance and 6x the scalability, impressive numbers.

In it’s base configuration this system allows for 18 ES30-30TB shelves or 12 ES30-45TB shelves. That means 428 TB usable capacity in a single namespace and single deduplication domain.

Just as the previous two systems listed this system supports legacy ES20 shelves and can be upgraded to with a simple headswap.

Install the Extended Retention software option and you can protect 1.7 PB usable capacity for long-term backup retention.

Summary

All in all, this post tells only a small portion of what the Backup To The Future launch presented. I will be back with more cool news about our DataDomain DDOS 5.4 functionalities, as well as the massive updates that took place in EMC Networker 8.1 and EMC Avamar 7.

So, this really is only the high level news with regards to the protection storage portion of the EMC Data protection stack:

Are we having fun yet? 🙂

Backup to the future is here

Today we hit 88 miles per hour and took backup another step towards the future.

You can watch the whole event replay here: http://bit.ly/134gk7U

There is a whole host of releases that EMC presented today:

• Four new DataDomain systems
• DataDomain DDOS 5.4
• Avamar 7
• Networker 8.1
• New sweet Mozy features

There are days and days worth of geeking if you want to get into every nook and cranny, but I will try to pick it apart in a number of posts for you, it’s good exercise for me too 🙂

Who’s going to back the cloud up?

The most vehement of the cloud evangelists seem to think that any day now anything and everything will live in the cloud.

Whatever pace and end state you foresee, we can all agree that there certainly will be steps and that the journey will be different for all organizations.

A big enabler for “all in” cloud consumption is Software-as-a-service with a large number of very mature suppliers of various types of applications “on tap”.
Perhaps the most well known of these is Salesforce.com, which is part of the CRM market, which continues to be the largest market for SaaS.

However, trust is the ultimate gateway to cloud adoption, and in trust, backup is a very important component.

I am a sceptic guy, but to get anywhere in life, you need to trust.
So.. trust, but verify.

All credible SaaS suppliers provide backup functionality, of course, they do it with varying success and granularity though.

With the advent of CRM SaaS, and the success of Salesforce.com, customers began to see challenges with the rather rigid policies Salesforce.com commits to.
The backups are rather infrequent and tough to export and restore.

This, however, is only one example which I selected because of the popularity and general knowledge about Salesforce.com.

The bigger issue on the road to trust and maturity is a standard for these matters.
Standards don’t come easy, it’s a classic design by committee, and the results are bland, if there even are results.

There certainly must be a huge market for whomever grabs the opportunity to provide a middle layer that SaaS providers can implement and SaaS buyers can hook into.
Think of it as RSS for backup.

I would imagine the control and data paths to take place over REST to be flexible, web enabled and programmable.
There would be a great opportunity for all makers of backup solutions to easily expand their capabilities to control and move data between public and private cloud solutions.

In the end, this could take care of a huge trust issue, maintain compliance and accelerate cloud adoption.

First post

I will try to use this blog to get some of my thoughts out of my head.

It is my hope that some of my musings might be of use to my peers in the IT business, meaning both colleagues, customers and competitors.