Wednesday, January 8, 2020

Ransomware attack - how can we recover?

Whatever we do in our life, it is always good to have Plan B.

What if my data are encrypted by ransomware?
Do I have Plan B?
How can I access my data?

What if I have backups that have no chance to be ransomwared/ removed?
What if I have a cyber bunker?
What if I have Cyber Recovery plan?
And... can it be with very attractive cost?

What backup/recovery solution can offer me in case of ransomware/hacker attack?
Why Plan B is important?

How can I lock against ransmoware/hacker?
How can I be sure that none can remove /change my backups.
Can I restore in case of ransomware/hacker attack.

Bunker in IT? - Why, When, How?
Can we do everything automatically?
Can we control our IT over ransomware?
Can we recover immediately?

All methods to protect against ransomware/hacker

Hardening - let's remember about this simple approach

Data Domain Snapshots 
No cost, no additional space, no performance degradation method to have protection of our backups

Why source de-duplication increases my security?

We can protect ourselves against ransomware/hacker
We can have plan B
We can make it easy and automated...

Let's consider it...

Presentation about Cyber Recovery (the one used in video):

Only successful recoveries!

Monday, December 9, 2019

Why new Data Domain models? - DD6900 / DD9400 / DD9900 backup/recovery appliances

In September 2019 Dell announced 3 new Data Domain models (DD6900 / DD9400 / DD9900)
Shall we take under considerations new Data Domain models?

If so why?
What is the architecture?

Below is the summary of changes in new Data Domain models:
  • DD6900
  • DD9400
  • DD9900

Automated protection against ransomware / hacker.
Any backup performed on new Data Domain models is automatically protected for defined period.
Neither ransomware nor hacker can delete it.
How is it possible?

Reducing disk space required for backups
New models require 10-30% less space for storing backups comparing to previous ones.
Wow! Can it be?

Starting even 64 Virtual Machines from new Data Domains with 60 000 IOps performance!
Hugh, can backup be faster than production?

Faster restores from new Data Domain

Capacity on demand
In new Data Domain models we can get more space but pay for the one that we use.

Faster internal components
Does it matter?

Scalability from 48 TB to 1 250 TB

100Gb Ethernet card is possible!

Very little space required Can you store 1.25PB in single rack?

Monitoring and visualizing online all its components

How can it move to new models?

New Data Domain models have a number new & exciting things.
It makes sense to have a closer look on them...

Presentation about new Data Domain models (the one used in video):

Only immediate recoveries!

Wednesday, August 21, 2019

How to keep long term backups? - Tiering or... something simple!

Recently, customer asked me the question:
      I’m looking for a discussion document which outlines best practice for tiering backups.
This question comes for ages.

Idea of tiering in backup comes from our internal dream:
Let’s have fast, secure and good media (storage) for recent backups/data
but at the same time
Let’s keep old backups as cheap as possible

Recent backups we usually keep on Data Domain (
Data Domain gives us huge performance, security, and restore.
This is what we require from protection production: no load, fast backup and easy, fast restore.

For older backups copies (archive backups) we really do not know what to do?
We just need old backups to keep cheap...

We have couple of options for storing old backups:
Object storage (local/cloud)
Or maybe…

Why not just keep 5 years backup on Data Domain?
Local Data Domain / Data Domain in cloud – as one wishes.

During one of the workshops I asked customer:
       How long do you keep backups on Data Domain, how long on tape?
He said:
       Daniel, I have no time and money to play with backup tiering.
       I have bought 2 Data Domains (little bigger) and I just replicate between 2 cities.
       I have 90 copies on each Data Domain (30 daily for last month and 60 monthly for last 5 years)
       No touch, same cost, dream life.

My experience (I would choose it for myself) is that little bigger DD - for both production backups and archive backups - in many cases is better than tiering.
Money are similar, simplicity is incomparable.

Of course, every case is different, but…
Most simple solutions are the best ones!

And let me quote my great friend from UK:

When I started dealing with backup I was said that the more we retain on Data Domain, the better de-duplication is.
And that is so true…

The current push for tiering backups in reality makes more problems rather than solving any.
Tiering backup is:
  • Expensive
  • Complicated because of restore 
  • And many other compatibility issue
Why keeping last backups on great media (like Data Domain) and moving older backups to tape/object provides complication, less flexibility, additional costs?

With tiering in backup, we have 2 different media, software, policies to manage those 2 different backup media?

Why do we do that all complication?
To make backup cheaper.

But… Even hard costs of tiering like additional storage (second tier) + potential licenses for tiering in many cases eat that whole difference in price between 2 different media.

Not mentioning soft costs like management, integration, know how, …
This is what my customer just said from his experience.

So having fantastic media like Data Domain, we can just increase a little bit its space – not much due to de-duplication – and we can enjoy f.e. 5 years retention!
With no problems, no management, easy.
And with similar or even less money…

And... Data Domain can be in the cloud using Object Storage as the space for backups.
But this something for another article...

Most simple solutions are the best ones!

Only successful recoveries….

Friday, August 17, 2018

DP4400 backup/recovery appliance and… everything is ready!

Video about DP4400 (below there are links to each part of discussion):

Presentation about DP4400:   

Let’s imagine we have chosen our dreamed backup solution. It was not easy, we had to compare and check plenty of parameters. But it is over. We know what data protection is best for us.
Is it really over? Can we really just wait for delivery of our preferable solution?

Unfortunately not.  We still need to take care about:
  • Design
  • Implementation

Design is important. If we do not take care about proper design and we provide too few resources for:
  • Backup server
  • Media servers
  • De-duplication pool
  • Network

we can have poor backup/recovery performance best. More probably we will expirience instability of our backup solution or in the worst case data loss (f.e. because of currupted de-dup database).

Also implementation of the above take us weeks if not months. This is pain for OS guys, Virtualization guys, Network guys.

Do we really have to take risk (as company, personally) for design / implementation / maintenance?
Is there another approach?

DP4400 is 2U backup/recovery appliance that is ready to backup our environment just after power on. The ease is not compromised with capabilities. We get top market features ready to run.
Just after power on and providing our IP addresses.

What functionalities does DP4400 provide?
DP4400 includes:
  • Software to backup/recovery of VMware, HyperV, OpenStack, databases, file systems, NAS devices, remote offices, laptops, … - any systemsBackup engine is based on the Avamar system
  • Media to keep our backups
    It is based on top market Data Domain technology and offers plenty unique features:
    • Best de-duplication (least storage required, least bandwidth required for replication)
    • No network usage during backup (source de-duplication)
    • Ransomware protection (locking backups)
    • ...
  • Finding any phrase in backups (GDPR)We get huge efficient Elastic Search engine
  • Cloud as Disaster RecoveryWe can replicate our Virtual Machines to Cloud and start them for Disaster Recovery / tests
  • Export old backups to cheap S3 storageDD technology within DP4400 has CloudTier functionality that moves old backups to cheap and secure S3 storage
  • ReportsOver 300 reports, analysis of problems, predictions of future provided by DP Advisor engine
  • Single console to mange the whole backup environment - our DP4400

DP4400 provides all the below beauty of backup solutions:

What about desing / implementation / maintance of DP4400?
DP4400 is 2U appliance that comes to us:
  • Installed
  • Configured
  • All components are properly tuned
  • All components have resources required to work with full power
  • Ready to backup / recover our data just after power on

We turn on our 2U device and we can enjoy the backup/recovery:

DP4400 is installed in rack - what do I need to do perform backup/recovery?
After power on DP4400, to perform backup/recovery we need to provide things that factory does not know:
  • our IP addresses
  • define what we want to backup (SQL, VMware, Hyper-V, Windows, laptops, remote offices, …)

What environments can DP4400 protect?

DP 4400 can backup:
  • Virtual Machines (VMware, Hyper-V, OpenStack, ….)
  • Databases and applications (SQL, Oracle, Sybase, Exchange, Lotus, …)
  • Files (Windows, Linux, Unix, FreeBSD)
  • Remote offices (protected environment can be even far 100ms from DP4400)
  • NAS (Unity, Isilon, Netapp, ZFS)
  • Standalone servers
  • Laptops
And we can use Cloud as Disaster Recovery!

What features we can enjoy with DP4400?
Let me share with you some of the capabilities that we get within DP4400:
  • Backup from 100 to 1000 VMware Virtual Machines per hour
    Extremely important for ransomware attack – we almost not lose any data!
  • Every backup is full backup!
  • Restore of single file, table, mail from image level backups
  • Immediate repairing of damaged VM (2 minutes to repair 1TB VM)
  • Starting Virtual Machines directly from DP4400 with SSD disks
    Zero time recovery with SSD disks speed!
  • Excluding any files (pagefile.sys) / paths  from VMware image level backup
  • Built in global de-duplication, source de-duplication, smallest block size on the market
  • Very small pipe for replication between DP4400
  • Sending about 1% of data to another DP4400 to replicate backups
  • Finding phrases in backed up documents in less than 1 second!

All the above we always get - no matter which DP4400 we will order! All the features are ready to work, configured in every DP4400.

What are DP4400 options?
Let’s say I take under consideration DP4400. What options do I have?
DP4400 2U device that we can buy in the below capacities:
  • 8TB
  • 12TB
  • 16TB
  • 20TB
  • 24TB
  • 36TB
  • 48TB
  • 60TB
  • 72TB
  • 84TB
  • 96TB

We can buy any capacity and upgrade as many times as we want to higher capacities. For example:
  • We can purchase DP4400 12TB and upgrade to 16TB
  • We can purchase  DP4400 24TB net capacity
  • Upgrade to 48TB net capacity
  • And then upgrade to 72TB net capacity
  • And then upgrade to 96TB net capacity

The upgrade is easy because DP4440 always come with full capacity:
  • 24TB net for Entry Model
  • 96TB net for Full Model
Upgrade with the model is just by entering the license
So for example upgrading from 24TB to 60TB is just entering the license.

If we want to upgrade from entry level model (24TB) to full model (96TB) we get couple of disks that we or Dell put in and that is all!

How big DP4400 do I need to protect my environment?
You shall contact Dell EMC or Dell EMC partner for sizing DP4400 for your environment.
Anyhow, the rule of thumb tells us the DP4400 24TB net allows us to protect 25TB environment with 30 days retention and 10TB environment with 5 years retention. The same proportion we can use for bigger capacities.

What is the price?
For price you shall contact Dell EMC. Vendor claims that price is very attractive – less than buying components separately.
But we shall take under consideration that within the price we get:
  • No risk approach (full power, made by factory)
  • Implementation / configuration
  • All the hardware resources for backup/recovery solution

DP4400 presentation
DP4400 video

And below there links to topics covered in the video.
DP4400 – what it is:
DP4400 – one slide summary:
DP4400 – capacities, upgrades:
De-duplication within DP4400:
DP4400 – licensing, environment we can backup:
Cloud as Disaster Recovery
NAS backup by DP4400:
Management, VMware backup, search and other features of DP4400:
Initial configuration

Only successful recoveries!

Thursday, July 12, 2018

Backup / recovery materials

Articles about NetWorker
Self NetWorker tutorial
Second link from the top
Articles about Avamar
Self Avamar tutorial
First link from the top
Avamar treats Cloud as Disaster Recovery
Avamar automatically backups new Virtual Machines
Articles about Data Domain
Self Data Domain tutorial (about 100MB)
Step by step guides leading from installation through configuration and integration with aplications.
Full explanations of features
Data Domain DD3300
Articles about RecoverPoint for Virtual Machines
Self RP4VM tutorial
4th link from the top
RecoverPoint for Virtual Machines - Video describing idea of RP4VM
Articles about DPS DD VE license package
Article describing DPS DD VE license package:
DPS DD VE video:

Thursday, June 21, 2018

DPS DD VE - a great license package for start!

DPS DD VE - video:

What if we need a backup solution for start:
• for couple of servers
• best of breed technology
• easy
And of course at the great price!

It makes sense to have a look at DPS DD VE package, which includes top backup solutions on the market:
Data Domain Virtual Edition (DDVE) - backup medium
Avamar - backup software
RecoverPoint for Virtual Machine (RP4VM) - backup/DR solution recovering VMs to any point from the past
DP Search (Search) - finding any info in backups, great for GDPR
DP Central / DP Advisor (Central/Advisor) - monitoring / reporting

DPS DD VE is licensed per number of physical processors (occupied sockets).
So to get the price for DPS DD VE for our environment we need to perform 2 simple steps:
  • Count all the physical processors in protected environment 
          We should know it easily
  • Multiply “single processor price” by “number of physical processors”
          We can get single processor price from our integrator / distributor

And that is all. We have the price for DDVE / Avamar / RP4VM / Search / Central / Advisor for our environment. Those solutions have included all possible features & modules!

What is the price for a single processor in DPS DD VE package?
I checked and… I was shocked! Really makes sense to ask IT integrators / distributors for DPS DD VE single physical processor price.
For those who know the super price-attractive DPS4VM license package (, I can tell you that DPS DD VE is lots of cheaper!

How does the typical backup architecture look like when using DPS DD VE?
Let's consider the below environment:

In the above example, we have 2 sites and totally we have in both locations:
• 5 physical servers with Virtual Machines (number of virtual machines does not matter)
• 3 standalone servers
16 physical processors totally (16 occupied sockets) - This is the only important thing from licensing perspective
• 14TB data to backup

DPS DD VE for 16 physical processors in our environment
• storage space for Data Domain Virtual Edition (DDVE)
• providing 2 virtual machines per site (for Avamar / DDVE)
we have a full backup solution with best features on the market:

Avamar & Data Domain Virtual Edition – which are included within DPS DD VE – they have all possible features and functions. Dell EMC did not limit anything that Avamar / DDVE provides. For example, we can enjoy:
• Automatic backup of newly created virtual machines
• Disaster Recovery with minimal data transfer (1%)
• 100 - 1000 virtual machines FULL backup per hour
• Restore a single file, restore of single email, restore of single SQL table image level backups
• Simultaneous image backup / guest backup at no additional cost (global de-duplication)
• Backup of physical servers with all available Avamar agents
• The highest performance on the market
• Incredible NAS backup/recovery
• Finding any phrase in backed up documents (Search module – GDPR ready)
• Over 300 ready reports in DPA
• ...
Additionally, apart from the above, the package contains all functionalities of RecoverPoint for Virtual Machines (RP4VM) - another top market solution.

What exactly do we get within DPS DD VE package?
DPS DD VE is license package calculated per physical processor. We need to purchase DPS DD VE for number of processors that we have in protected environment.
When purchasing DPS DD VE license we get the following solutions:
DDVE (Data Domain Virtual Edition) - 2TB DDVE for each physical processor
For example if we buy DPS DD VE for 10 processors, we get 20 TBs of  Data Domain Virtual Edition
All functionalities, all modules, all possibilities for a protected environment
2 Avamar backup servers: one for Primary Site and one for Disaster Recovery site
RP4VM (RecoverPoint for Virtual Machines)
15 VMs per each physical processor
For example if we buy DPS DD VE for 10 processors, we can protect 150 VMs with RP4VM.
• Search
Find phrases in backuped documents (GDPR)
• DPA (Data Protection Advisor) / DPC (Data Protection Central)
Monitoring, reporting, warning, single pane of control

What if my environment grow?
If we grow from 15 servers to 40 servers - no problem. It is just enough to purchase missing physical processor.
Having bigger environment, we have bigger expectations towards backup. We can attach Data Domain appliance instead of using DDVE. And we can enjoy:
  • ransomware protection
  • starting VMs from Data Domain with SSD speed (no recovery required)
  • separate backup medium from production
  • ...
So in future, while growing, no change to our environment  - just replacing media. Backup software, interface, all features - it stays with us. We already have the enterprise ones!

Who shall be interested within DPS DD VE?
In my opinion, the package is great for:
  • Environments having 1-20 physical servers with any number of Virtual Machines
  • New companies, just creating their infrastructure
  • Remote offices
  • F.e. we do not want to invest a lot in remote office or we do not want to put any additional backup hardware to remote office
  • Separated environments from main Data Centers required their own infrastructure
  • Environments required protection with minimal investments
  • VxRail, Nutanix, any hyper converged infrastructures …

Infrastructure as a Service
I think that especially Cloud Providers shall have a close look at this package. Why? DPS DD VE is
  • Extremely cost effective
  • Powerful
  • Easy to divide for tenants
I have talked with my friend from Cloud Provider. He compared DPS DD VE against another solution. DPS DD VE was 25K Euro cheaper for 5 years assuming 20 physical processor customer. And he said that DPS DD VE functionalities are absolutly #1.

Again. Have a close look at DPS DD VE - a very strong package.
And the price? Check, you can be surprised…

Video describing DPS DD VE backup licenses:
Presentation about DPS DD VE package

Only successful recoveries!

Monday, April 9, 2018

SAP backup / recovery - easy, but...

Recently, my partner called me and asked if NetWorker can backup SAP:
-        Sure!
-        But I am talking about SAP WITH ORACLE  –  Oracle platform was emphasized
-        Still, this is standard for NetWorker…
-        Hm… Can you make Proof of Concept (PoC) at customer site?

What occurred?
Another solution could NOT perform a single backup during 6 months! During another 2 months, this vendor could NOT perform restore!!!
And it was just 300GB SAP/Oracle.
This made customer wondering:
   Is it standard on the market? Or maybe I can still expect easy backup & restore of my SAP/Oracle.

Even more interesting was the next part of the conversation:
-        Let’s meet with the customer. I explain how backup works with SAP/Oracle, what is required, what can we expect  –  I proposed the fair approach
-        No  –  the answer was very strict
-        It cannot be NO... Before anyone (customer, you, me) invest in tests, I want to explain what we will achieve after PoC. If fine, I have no problem to prove the architecture in tests.

Customer was desperate to see that its SAP/Oracle can be easily backup&restore. Functionalities, performance did not matter. 

There was no choice. Starting meeting did not happen...
Our guy (very good one, greetings!) made tests with results as below:

·        FULL backup of SAP/Oracle 300GB took 5 minutes
YES! 5 minutes to make FULL backup!

·        De-duplication after just 4 backups: 1:15

·        Implementation of backup environment: 6 hours!
6h, it was time for the whole backup solution installation/configuration: Backup media (Data Domain) + Backup software (NetWorker), Backup agents, Defining backup policy, …
For previous solution, implementation was 8 months and not working well…

·        In the above environment, it was used:
o   Data Domain Virtual Edition as backup media
(, 3rd link)
o   NetWorker as backup software

The customer was shocked!
I was shocked that he was shocked!!!
Anyhow, the buying process started...

100% of successful backups & restore!