Top Accessories For Managing Newborns!

Taking a break from blogging about virtualization, storage and disaster recovery, I thought I’d blog about the other big part of my life these days, baby management. My Wife recently gave birth to twin girls back in October and the road traveled so far as been quite challenging, but rewarding.

I have discovered along the way many cool “baby accessories” that are out there to assist in your daily baby activities. Now I will be the first to admit that most of these accessories are luxuries and not requirements, but I’m all about cutting some corners in order to make a happy, healthy and sleepy baby.

As I have grown to rely on these new toys in my life, It’s hard to imagine my Parents having to survive taking care of yours truly without these cool toys.

How did they manage?

So here it is folks, the first list, in no particular order, of some accessories that have made both my Wife and my life much easier:

• Munchkin Deluxe Bottle/Food Warmer
  Man it’s nothing like throwing a bottle already topped off with formula into this warmer at 3am and having a warm bottle ready to go in a matter seconds, love it!

• Wipes Warmer
  We use the Prince Lionheart model. Maybe a luxury item for the babies, but it keeps the nasty off my hands and let’s face it, who wants to shock their baby in the middle of the night wiping their bum with a cold wet wipe? I sure as heck don’t. By the way, I told my Wife once they out grow the baby wipes I’m hijacking this for myself so I can keep my adult wipes warm.

• Diaper Genie II “Elite”
  My girls are cute, but they can be the source of some foul and disgusting things, aka the smell of dirty diapers. I don’t know about you, but the last thing I need is competition in improving the aroma of the house, that’s my job.
  Best part about it, hands free baby!

• Paci Wipes

Nothing remedies the cacophony of irritated or tired babies like a pacifier, but what happens when he or she decides to rebel and spit that paci out on the floor? No fear, first year pacifier wipes to the rescue. Sure the 3 second rule may apply when the Wife isn’t looking, but nothing like being able to safely clean those pacifiers, teething toys and bottle nipples in a hurry. I also recommend taking these on the go, nothing like being prepared.

• Arm & Hammer Diaper Bag dispenser
  Must admit I thought this was going to far until we took the girls for their 1st doctors visit and we had to take off a stinky diaper. Where do you throw the dirty diaper? Of course the answer is obvious, in the closest trash can. But what if you are at family or a friend’s house and want to avoid the embarrassment of adding a new aroma to the ambience (yes, I now it smells like roses); quickly pull out one of these handy disposable bags and presto, dirty diaper gone and no bad smell left lingering in the air. Not only will you make avoid blushing with this accessory, but you can also check off your environment friendly deed of the day. Best part about it, the dispenser attaches to almost anything including strollers, diaper bags, even your laptop bag.

Avamar Tape Out using Data Transport

One hot topic when discussing Avamar always seems to be tape out for weekly and monthly backups. In the past it has been a functionality that was missing from Avamar. All that has changed with the release of Avamar Data Transport.
Avamar Data Transport is a server that runs virtually inside ESX 3.5 U4 or ESX 4.0. The Data Transport is a virtual 1TB node that acts as a replication partner to a primary Avamar server, once the replication job is complete, the data is then moved to tape.
It is currently EMC best practices to run at least 2 transport nodes in your environment, currently Avamar supports up to 4 transport nodes.
r

Avamar Data Transport system requirements:

• Supports 1 to 22 TB configurations
• Avamar versions 4.1.0, 4.1.1 or 4.1.2
• A dedicated ESX server only running Avamar Data Transport Node and Control Node
• ESX Host must have at least 4 gigabit Ethernet NICs with teaming capabilities for the Avamar Data Transport VM Network in addition to the dedicated NIC for the ESX VMKernel and Service Console
• Tape backup server with either EMC Networker (versions 7.4 SP4 or 7.5SP1) or Symantec NetBackup
• Memory requirements for the ESX server are as follows: Control Node 2 GB, Transport Node 4 GB each
• Disk requirements are as follows: Control Node 60-90GB, Transport Node 1.56 TB of disk space for each node

Avamar Data Transport require two Avamar systems, a primary that replicates to the secondary Avamar that then exports out to tape, here is a diagram of all the components needed:

Dealing with an IT Outage or Crisis

So the day you’ve been dreading as an IT Consultant has finally arrived. An outage has occurred with you or your client’s IT infrastructure. In a time of crisis it’s natural for your stress level to rapidly rise and you may even find that your life has just skyrocketed to Defcon 1.

Step Back, Calm Yourself

In situations like this, it’s real easy to hurt you or your companies reputation. In these times of high level stress, it’s important to focus on solving the issues at hand, but you also want to keep in mind that your emotions are on display for everyone to see and your reactions not only can be contagious but can also make matters worse, not better.

So what are some steps you can take to help you make it out of this world alive?

Here is a list I personalized based on my experiences and Steve Tobak’s How to manage a crisis, any crisis | Train Wreck – CNET News.

1. Be focused
Assess the situation, don’t give in to the urge to make quick and careless reactions. Have a goal in mind of what you want to accomplish.

2. Look around
Investigate, what events led up to the outage? What is the most likely cause of the outage? These questions can assist you in developing a clear picture of the situation.

3. Who is affected?
Notify all key stakeholders of the situation and be honest with them. If working with a client as a consultant, involve those parties that may be able to provide some assistance with assessing the situation and arriving at a positive outcome. Continue to communicate constantly with all through E-mails, online conferences, voice mail, etc. Make sure all parties that need to be updated are included.

4. Don’t be afraid
Call for help, being part of a strong team goes a long way. At Varrow, the company I work for, success is always derived from the right people and teamwork. There have been several occasions where I have had to rely on co-workers to help me through a situation, and let me tell ya’, they were always there for me.

5.Be like Batman
Once you have gathered all information and consulted with all parties required to move forward, construct a plan on how to tackle best, worst and typical scenarios. Be prepared for any obstacle you may face.

6. Execute
You have a plan, now put it in motion. you should feel more confident about what actions need to take place and how to react to any hurdles that might present themselves.

7. Remain confident.
Know what you need to do, but be flexible. Remember that your client or colleagues’ emotions maybe predicated by how you present yourself during this time of crisis.

8. Regroup
Now that the smoke has cleared and everything is hopefully back to normal, have a meeting with all parties involved to review the outage and what steps were taken to solve the issue. Also review any next course actions that need to take place for clean up.

9. Find the Root
Re-investigate what events happened leading up to the outage, what steps were taken to resolve the issues at hand. Review any logs that are helpful and interview any parties that had interaction that could have caused the outage. Come to a conclusion and create a root cause analysis report and review it with your colleagues and/or clients.

10. Learn
Do what we Humans do best, learn from mistakes. Were there any steps that could have been taken that would have prevented this crisis from occurring? Document them, adjust any internal procedures, so forth. Don’t forget what Mr. Benjamin Franklin said, “an ounce of prevention is worth a pound of cure…”.

Hopefully after following these steps, you will find yourself in a situation where you, your colleague, your company, or all of the above is admired for their honesty, determination and calm manner during this time of a crisis. It’s these traits that will be remembered in the end!

Avamar 5.0 New Features and Enhancements

 

In case you haven’t read up on it, Avamar 5 was released recently, there have been several new features introduced and some annoying factors remedied. 

The biggest new feature in my opinion would have to be the vSphere integration, I recently implemented this at a client site and in our lab, it’s pretty impressive.  Here is a list of the new features and much need changes:

• Avamar GEN3 storage nodes with Avamar 5.0 software installed offer 3.3 TB of licensable storage capacity per node. 

*Note: Storage Nodes can not be mixed inside an Avamar Data Store, all storage nodes must have the same storage capacity.

• NAT Support for Multi-Node Avamar Servers is now available allowing for interoperability with NAT firewalls and routers.

• The Avamar Data Store system can optionally be configured with two switches to support high availability in the event that a NIC or a switch fails.  This was an often topic discussed with clients, it’s nice to know that EMC has eliminated these single points of failure.

• Avamar 5.0 provides the ability to perform server-level backup and bare metal recovery of Windows Server 2003 32-bit and 64-bit systems.

• The big one in my opinion, very impressive integration with vSphere 4.0 by introducing:

• Image-level backup and restore

• File-level recovery from above image backups

• The ability to schedule both types of backups using the policy-based features

• Enhancements to Windows and Mac Desktop/laptop backup and restores

• Windows 2008 SQL and Oracle 11g support 

• The annoying problem with the Daylight Savings notification with Java Runtime 1.5 has been resolved with the option to install JRE 1.6.

That’s it for now, check back as I discuss vSphere integration in more detail at a later time.

Avamar Version 5 patch

In case you are not aware, there is a newly patched version of Avamar, release 5.0.0-409, it’s recommended to upgrade to this.

Also, if you happen to be dealing with a 1×2 non-RAIN configuration, there is a known problem reported under escalation 845:

"there is a SIGNIFICANT bug in v5.0 with 1×2 non-RAIN grids.

Basically, ASCD will NOT redirect clients to the data nodes.  Therefore, you have to type the following command (as user admin on the utility node) every time you run dpnctl start on 1×2 non-RAIN grids:

ascd –forcestatus restart

You can download it off the FTP

It is under Software/5.0.0

VMware vSphere 4 Guided Consolidation

I recently passed the VCP 410 exam and was surprised to find several questions on Guided Consolidation, a very basic tool that I had really no experience with.  To get more acquainted with it, I decided to write a blog about it.

If you are preparing for the VCP 4 exam, and have little or no knowledge about this tool, then maybe this blog may help you a little.

Guided Consolidation is a tool that will allow you to monitor a physical computer and determine it’s potential for adding to your virtual environment. I would highly recommend it for a small to mid sized business looking for some assistance with their P2V process. It has an easy to use interface with a more simplified approach than using the full VMware Capacity Planner utility.

It is free and can be installed from the vCenter installation CD.  Once installed, it can be accessed from the vCenter Home Menu.  In order to scan a physical computer, you will need to make sure the credentials provided during installation must have administrative permissions on any remote Windows systems selected for analysis.  This user account supplied during installation is used to run the “VMware vCenter Collector Service”, a Windows service used to connect to the remote systems. 

Once the tool begins to scan the system, it will gather data and display CPU information and utilization, memory information and utilization as well as the computer name.  Sometimes it may take up to 1hr before this process begins.  Be prepared to allow 24 to 48hrs for this process to complete as Guided Consolidation builds a confidence metric level.  Once a high enough confidence level is reached, the status of the system changes to “Ready for consolidation”. 

At this point, a consolidation plan is available by selecting the analyzed computer and clicking the “Plan Consolidation” button.

This plan will include a star rating which will indentify how likely the physical computer is for virtualization and will even make a recommendation for the target host.  The rating system ranges from 1 to 5 stars, with 5 stars indicating the system is a high candidate for the proposed host.  From the Consolidation Wizard, you can change several things, the name of the VM, the host being assigned too or even remove a VM altogether from the list.

Multiple systems can be converted to VM’s using this wizard, it’s a very simple process and by the click of one button you are off to the races converting your physical systems to virtual machines.

***Keep in mind that before using Guided Consolidation that a datacenter exists and a host is added to the vCenter Server inventory.  ***

I would highly recommend looking at this pdf: http://www.vmware.com/pdf/vsphere4/r40/vsp_40_admin_guide.pdf or picking up a copy of Mastering VMware vSphere 4 by Scott Lowe for more in-depth information on Guided Consolidation.

I hope this gives you a basic understanding of how Guided Consolidation can help you plan a smooth P2V process.

Thanks for stopping by!

Finding Disk Space Hogs

Have you ever experienced a situation where your workstation or server is completely out of space?  Trying to hunt down where all that valuable storage went can be frustrating.  Not to worry, there is a free tool available that will not only search and display all folders listed in order by space used, but it will also allow you to explore each folder.  Like I said before, it’s FREE!  I have included a screenshot of an example of my very own VM of XP Pro that I work on from day to day, as you can see, Documents and Settings leads the pack.

The download is available at Jam Software, link below:

http://www.jam-software.com/freeware/index.shtml

 

 

 

How to change Avamar Node Type

When running Avamar in a lab environment, I have come across situations where I want to use a node that was part of a previous Avamar configuration and make it a spare, single node server, a data node  or possibly even an accelerator node for another Avamar Data Store.

In the past I was not sure how to change the node type. 

Fortunately, I recently came across the command to make this happen.

As root, run change_nodetype in the /usr/local/avamar/src directory.

(Verify that you are logged in as root) whoami command

# /usr/local/avamar/src/change_nodetype –[type]

–[type] = the type of node to be configured.

The accepted types include:
data,axion-e (single node server),accelerator,spare,utility

As an example, in a recent situation, an Avamar storage node was mistakenly setup as an Avamar Accelerator node.  The desired setup was a RAIN configuration with a Utility Node, 4 1TB Storage Nodes and 1 Spare Node.  In our setup, we had the Utility Node, 3 1TB Storage nodes, 1 Spare node and the 1TB Accelerator Node that was configured improperly. 

To correct the issue, I performed the following tasks on the accelerator node to make it a data node:

Opened a Putty session to the Accelerator Node and logged in as root

typed the following command:

 /usr/local/avamar/src/change_nodetype –data

Afterwards, I had to update the usersettings.cfg file using vi or nano and confirm that the server name is correct by typing the following command:

nano /usr/local/avamar/etc/usersettings.cfg (as seen below):

***If you need to change the hostname or IP address as well, use the following command as root:

/usr/sbin/netconfig –ip=ipaddress –netmask=subnet mask hostname=hostname –domain=domain.name –device=eth0 –gateway=default gateway

You also want to make sure that the following configuration files are correct:

/etc/hosts
/etc/resolv.conf

/etc/sysconfig/network

You may now reboot, once the node boots back up, log in from a Putty session and verify that the notification screen states the correct node type.

From here I was able to continue with the procedures to add an additional storage node…

Reaching Avamar Zen or as EMC likes to call it “Steady State”

Now that you have implemented your Avamar System, you may become easily restless by the rapidly upsurge in storage used during the 1st to 2nd week.  Do not worry, this is common for an Avamar System as it initially goes through the process of backing up new clients during each client’s “initialization” phase where nearly every client contains high amounts of unique data.  Once clients have been backed up at least one time, you will begin to leverage the commonality feature that Avamar has to offer. 

You will begin to notice that your Avamar Server will back up less and and less unique data as the weeks pass by and the new data written to your Avamar Server will begin to plateau.  EMC refers to this as reaching a “steady state” of capacity utilization which I refer to as Avamar Zen.  You can expect for this to occur sometime after the longest retention period of your first initial backups are met.  The goal of successfully managing your Avamar Server is to achieve this Avamar Zen (steady-state) where the rate at which new data chunks added to your server are equal to or less than the rate of data chunks removed from the server during the Garbage Collection cron job that occurs for up to 2hrs during the daily maintenance schedule.

So you may be wondering, how do I tell if I have reached Avamar Zen, well not to worry, I have included two different methods for receiving this information.

• You can issue the following command /usr/local/avamar/bin/capacity.sh to ascertain whether the system is running in steady-state.  The information reported back includes amount of new data added to your Avamar System, amount of data removed, net change and clients with the highest change rate.

Example below:

 

*As you can see, the Microsoft Exchange Server listed above has the highest change rate at 84%

 

• The other option is running the DPN Summary Report from inside the Avamar Administrator.  The report will provide information about data stored in the Avamar server as well as statistical data on the individual Avamar clients. 

As you can see, the DPN Summary Report will include statistics such as daily change rate, high change rate clients, amount of data/# of clients being protected, etc.  This information can be useful in determining abnormal client behavior as well as comparisons to previous backup strategies such as tape backup.

Come back soon and we will discuss factors affecting capacity utilization and monitoring Avamar Checkpoint Overhead.

Thanks for stopping by!!!

Solution for Avamar Windows 2008 Backup and Trend Micro

I recently posted steps on how to restore a Windows 2008 Server and System State with EMC’s Avamar.  I was also made aware of another issue where the Trend Micro anti-virus client conflicted with the VSS and System State backup with the same server running Windows 2008.  I’d like to thank Stephen Saunders for passing this work around provided by Trend Micro to me, here are the steps to to alleviate this issue:

 

Problem:	After installing the Client/Server Security Agent on Windows 2008 32-bit and 64-bit versions, you can no longer backup the Windows System State.   When you run the command "WBADMIN START SYSTEMSTATEBACKUP -backuptarget:D:", the operation fails with the message Enumeration of the files failed when identifying system state files to back up.   You stopped the Trend Micro services, but this did not resolve the issue. When you uninstalled the Security Agent, the backup works just fine.
Solution:	Just disabling all the Trend Micro services does not work because the Windows System State Backup does not parse the file path of the WFBS Agent: C:\Program Files\Trend Micro\Client Server Security Agent\..\BM. This is because the backup does not recognize the (..) characters.   Before using Windows System State Backup from the command line, please do the following:

1.	Stop the Trend Micro Unauthorized Change Prevention Service.
2.	Open the Registry Editor (regedit.exe).
	Important: Always create a backup before modifying the registry.  Incorrect registry changes may cause serious issues.  Should this occur, restore it by referring to the "Restoring the Registry" Help topic in Regedit.exe or the "Restoring a Registry Key" Help topic in Regedt32.exe.
3.	Go to the HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\TMBMServer registry hive.
4.	Look for the registry key "HomeDir" and change its value to "C:\Program Files\Trend Micro\BM".
5.	Look for the registry key "ImagePath" and change its value to "C:\Program Files\Trend Micro\BM\TMBMSRV.exe" /service.
6.	Start the Trend Micro Unauthorized Change Prevention Service.
7.	Open a DOS command window and run this command: "wbadmin systemstatebackup"
	The backup process will now work.