Right here. Right now.

It has been a while since my last blog post, but there is a reason or in this case for me LOTS of reasons.

My intention to blog has always and still is to give back to the community. Take what you can find useful and share what you can to give back. I will post ideas and technical challenges I have come across in the field that I can share.

Well… My professional role has changed. I am less customer facing but still face the same challenges! I am putting together some cool – or what I think are cool relevant posts for the hard working techs out there doing the needful.

Stay tuned….

Change Control Station IP for VNX

The installation of a VNX2

The first day is the toughest and easiest. Like baking a cake you need to make sure all the ingredients are there and ready to go. You verify what  was ordered is installed and is in working order. Sounds simple enough?

As it turns out the initial VNX “Rack and Stack” was correct.

  • All cables were plugged in correctly
  • No amber lights on the system.
  • The DAE (Disk Assembly Enclosures) were recognized.
  • No faults in Unisphere

Ok. Good to go? No? Why. Mr. Customer provided a duplicate IP for one of the control stations. The correct IP is the same subnet and mask but needs to be changed.

Here are the steps

  1. Log onto Unisphere confirm IP information for Control Station
  2. Log onto Control station via ssh

Use root/nasadmin (default)

3. Verify status of Control station. You can make changes to the primary not standby.

In this case the primary Control station was IP’d correctly. I need to change the secondary Control station, the standby.

/nasmcd/getreason  <<– run this command as root

Here is what you will see:

10 – slot_0 primary control station
11 – slot_1 secondary control station
 5 – slot_2 contacted
 5 – slot_3 contacted
4. Failover the Control station with this command:
    /nasmcd/sbin/cs_standby -failover
About 5 minutes later you will reboot the primary and failover to the secondary. 
5. Run this: /nasmcd/getreason
11 – slot_0 secondary control station
10 – slot_1 primary control station
 5 – slot_2 contacted
 5 – slot_3 contacted

6. Log into Unisphere

Normally you login to the VNX with using the primary Control Station IP. However since it is in failover mode use the secondary IP. In this case I am using the duplicate IP and will change it to the correct IP by using the Unisphere GUI

Screen Shot 2016-02-19 at 5.10.39 PM

Log into the Unisphere GUI as root. Make sure you choose scope “local” not Global

Navigate to the “Control station Properties Tab”

Screen Shot 2016-02-19 at 5.11.43 PM

Change the IP of the Control Station

Screen Shot 2016-02-19 at 5.12.20 PM


Click Apply.

In Confirm Action, click OK.


Modifying the Control Station hostname, IP address, subnet mask, or gateway may disrupt the Unisphere software connection and any other client access to the Control Station. It may be necessary to reconnect to continue administrative activities. If you make a mistake changing the network, the Control Station may no longer be reachable remotely.

7. ssh to the primary Control station and failback. Verify that the primary control station is in slot 0

#  /nasmcd/getreason
10 – slot_0 primary control station
11 – slot_1 secondary control station
 5 – slot_2 contacted
 5 – slot_3 contacted

That is it.


Remember when you login to Unisphere you connect via the Primary Control Station IP, if you are failed over you will connect to the secondary Control Station.


Thanks for reading! If you have a more efficient way please share. Yes, you can ssh in and do all of this via cli.😉




EMC Elect 2016.. by the numbers

The EMC Elect 2016 members have been announced.


Wow what a privileged honor to be a part of this list. This year will be filled with many new challenges as EMC will have a flurry of changes internal to help the customer make their goals a success! I refer to the Dell + EMC adventure. I look forward to helping my customers and fellow IT professionals in 2016. Don’t forget that is just the business side. The never stopping change of technology is still in full force! The information to be consumed is non-stop, and the challenges for the customer to take advantage of this is incredible.  I know I am excited for 2016. Sharing the information and empower each other!

EMC Elect 2016 — Official list


EMC Elect Summary

  • 2015 – 102 members chosen out of 450 nominations
  • 2016 – 71 members chosen out of 700 nominations

In 2015 – There were 450 nominations leading to 200 finalists, the 102 official directory of its members for 2015
In 2016 – There were 700 nominations, leading to 150 finalists. Of those finalists, under the toughest selection process for the EMC Elect here are the 71 members for 2016

In my opinion it seems that for 2016.. the selection process was roughly 10% of the nominated persons were chosen. There was an increase in nominations, but a decrease in the number of selected. I can only imagine how much a difficult task this was. Thanks for all the hard work Mr. Mark Brown @DathBrun

Here are some interesting facts by the numbers:


Number crunching for 2015  Great job Mr. Henry!

My attempt inspired by Mr. Henry @davemhenry




I didn’t have a chance to calculate repeat EMC Elect members from former years. Metrics tell all sort of stories, but more over. Take a look at the content provided by the EMC Elect. There is A LOT!

Follow an EMC Elect member become informed and part of the discussion. Search for the tag #EMCElect.  Read the blogs (like this one) read up on the ECN (EMC Community Network). If you have a question reach out to a member via twitter or post on ECN. You will find very helpful individuals; who generally are very open to objective discussion on the technical merits of X, Y or Z!

Congrats to ALL the EMC ELECT 2016! And to all past EMC Elect… A famous quote to consider.

                   “If I have seen further than others, it is by standing upon the shoulders of giants.”                                                    -Sir Isaac Newton




Snapshots are not backups! or VDP and YOU

Ominous words would be echoed in the meeting… “You do have a BACKUP right?”


Working in production environments the constant challenge of maintaining uptime aka ‘steady-state’ but at the same slowly or as quick as feasible move forward with changing demands of the business.

Change can came in many forms. It is a driver for your organization.

A simple response to a vulnerability; patching is a necessity.

New features are required. Upgrades will be needed.

And more importantly disaster avoidance. The idea is to prepare in advance avoid disaster. It is akin to shift and dodge BEFORE some bump comes in the road. There are many approaches to this like having a stretched geo-location metro cluster.

Whatever the driver you have to have a fallback plan. If the post-change activity fails, if there is an unforeseen after-effect.. Things do not always work 100% as planned. What is your fallback plan? What? You have a VMware environment. You did click the snapshot button.. Well, that does work but it isn’t a full backup

From KB 1025279

  • Snapshots are not backups. A snapshot file is only a change log of the original virtual disk.
  • Snapshots are not complete copies of the original vmdk disk files….it only copies the delta disks. The change log in the snapshot file combines with the original disk files to make up the current state of the virtual machine. If the base disks are deleted, the snapshot files are useless.
  • Delta files can grow to the same size as the original base disk file, which is why the provisioned storage size of a virtual machine increases by an amount up to the original size of the virtual machine multiplied by the number of snapshots on the virtual machine.
  • The maximum supported amount of snapshots in a chain is 32. However, VMware recommends that you use only 2-3 snapshots in a chain. — [ed The reason is there is a performance hit]

In fact VMware recommendation is to setup an alarm in vcenter if the VM is running from a snapshot to avoid this condition

See KB 1018029 “Configuring VMware vCenter Server to send alarms when virtual machines are running from snapshots”

Now the question still remains.. What options do you have?

Well there is good news!! VMware as of March 1, 2015. “VMware vSphere Data Protection Advanced will be consolidated into VMware vSphere Data Protection (available through vSphere Essentials Plus Kit or higher vSphere editions, all vSphere with Operations Management editions and all vCloud Suite editions) and will no longer require purchase of a separate license. All functionality available with vSphere Data Protection Advanced, previously available as a standalone product, is now included in VMware vSphere Data Protection 6.0 – See more at: Announcement


Why is this cool? There are many reasons but to sum things up.

VMware Data Protection Advanced (VDP) is very cool. It is based on modern backup solutions.

  • There are no tapes
  • There is deduplication – Variable length up
  • There is replication
  • File recovery
  • VM recovery
  • Application aware backups
  • Efficient, bandwidth throttling
  • Changed Block Tracking (CBT) Restore
    vSphere Data Protection uses Changed Block Tracking (CBT) during image-level backups. CBT is also utilized with image-level restores in some cases to improve speed and efficiency
  • It can plug into something really big (Data Domain and Avamar)
    • Data Domain allows for “Consolidate backup, archive, and disaster recovery with high-speed deduplication”
    • Avamar  DEDUPLICATION BACKUP SOFTWARE AND SYSTEM — VDP is based on Avamar. See the announcement.

It is super easy to install and use. I did say easy and it is, because you can even configure VDP to allow for.

  • Linux-based virtual appliance: Easily install and configure backups.
  • Self-Service File Level Recovery: Enable guest OS administrators to restore individual files and folders.
  • Wizard-driven backup policies: Assign backup jobs to individual virtual machines or larger containers such as a cluster or resource pool, with specific schedules and retention policies.
  • There is no need for agents in the VM for normal backups.
  • Application aware backups. Backup agents for Microsoft SQL Server, Exchange, and SharePoint. The agents enable application consistent backup and recovery of these applications on virtual and physical machines

– See more at: http://www.vmware.com/products/vsphere/features/data-protection.html

Some tips about VDP deployment.

Do not put the all your eggs in the same basket

— Don’t setup your backup volumes in the same datastore your VMs reside in. The option to use Data Domain is great option! Data Domain can be data backup target.

DNS. Have it working!

It is fast enough? Avoid problems and run a performance test before your backups. Make sure your Data backup targets are validated for performance.

Initial configuration is deployment via OVF. 

Log on via the https://ip-address-assigned/:8543/vdp-configure/

Here you log in as root/changeme


BUT if you need to ssh in later via IP or hostname. You cannot use the root account. You must use the admin account, which has the same password then you su to root.

“Currently, users can access the VDP appliance command line using the vSphere Client console, SSH, or Putty sessions. With the VDP 5.8 and later releases, the ability to use SSH or Putty to log on to the VDP appliance with the root user has been removed.” — Administration Guide

and lastly

VAMI is your friend and so is the log.

The VAMI is: Virtual Appliance Management Infrastructure (VAMI). VAMI provides end‐users of virtual appliances with a Web console and command line interface that can:

  • „Configure network settings
  • „Check for updates and install them, manually or automatically„
  • Review basic system information for the virtual appliance
  • Stop or restart the virtual appliance

Where is the magical vami?

From the command line you can find it here: /opt/vmware/share/vami/


and if you run into problems..

Log in via ssh to the vdp appliance. Run the following while you attempt the action where you see the error.

root@vdp01:~/#: tail -f /usr/local/avamar/var/vdr/server_logs/vdr-server.log

Then watch the log and try to reproduce the error.

Additional Resources:

Here is a great overview from the VMware HOL team!

VDP overview install and backups! VDP DEMO

and more VDP feature walk through DEMO

and learn how to:

  1. Creating a Virtual Machine Backup Job
  2. Creating a Replication Job
  3. Creating an Application Backup Job
  4. File Level Restore
  5. Restoring a Virtual Machine
  6. Restoring an Application

HP C7000 iLo

What happened? Why doesn’t my iLo have an IP Address? The blade inserted has the correct profile but the iLo cannot obtain an IP address. I cannot start a remote session to the iLo….

What? What about configuration of the iLo IP Address?

HP iLo uses DHCP to obtain an IP address.

The Onboard Administrator configures this parameter.  The iLos have DHCP enabled and OA distributes addresses. Feature is called Enclosure Bay IP Addressing or EBIPA.

I was scratching my head, as it has been a while since I had to configure a new blade with the iLo.

I forget to run the full configuration for the iLo in the Onboard Administrator.

Verify the blade is functional powered on and recognized in the and the chassis isn’t throwing any errors.


Go into the OA.

Locate the Menu “First Time Setup Wizard >>> EBIPA section”

  1. Make sure the enable box is checked for the new blade
  2. Notice how the blade has not obtained an IP address yet, pending set of feature and reboot


Once the blade has rebooted or ILo has rebooted it will have an IP address and you will be able to remotely console into the blade.

Remember to patch the latest firmware for the blade. Download the full ISO image of the SPP from HP’s site and boot from that ISO.

vmworld 2015 Highlights Day 3 VSAN 6.1

Well it is almost the end of vmworld2015. I have been incredibly busy attending so many sessions it is numbing both for the mind and my feet! The content this year has a different perspective for me. In years past it has been on the tip of everyone’s tongue a new release of x, y or z but this year it is more than that. For me it is Day 3 as I have been at vmworld 2015 since Sunday.

VSAN 6.1

This is the third major release for this product. VSAN has isn’t the new kid on the block. It is shown its ability to deliver Enterprise Class performance with the flexibility of the Software Defined Data Center. It is delivery flexibility and ease of operations as a foundation of SDS. Performance is just one of it’s attributes. The beta for a long awaited feature is announced. DEDUPLICATION


“VSAN 6.next beta – A glimpse of the future”

VSAN stretched clusters.

If you have a chance watch the session delivered by Rawlinson Rivera @Punchingclouds and Duncan Epping @duncanYB. It is a deep dive of the design of using VSAN as a stretched cluster option. Key points to remember are L2 L3 requirements and the newest feature the Witness VM. While VSAN requires L2 Multicast; the witness vm only requires L3 communication. The impact is a stretched cluster Site A and Site B while a lower latency site can host your witness VM with L3 requirements. As the Fault Domains are dependent on the use of a Witness VM and this re-design of the witness lends itself to use for VSAN as A ROBO solution.

Look for it STO5333 “Building a Stretched Cluster with Virtual SAN”

The impact of VSAN on design and deployment of a stretched / Metro cluster combined with the versatility and flexibility of vsphere 6 (Think PSC HA Think the flexibility of vmotion across VDS, VSS and different Datacenters!)



What is it? In nutshell VSAN can now be deployed at remote sites (Remote Office Branch Office) with only 2 nodes! Technically the third node is a witness vm.


I am very excited to leverage this ROBO configuration for many customers. There is a lot of flexibility in how to implement this design. It decrease the complexity and cost of shared storage, but that is the crux of VSAN of vmware’s SDS.

Once last note about VSAN 6.1

I have had many conversations with customers and as well as the vmware VSAN team about the requirements and benefits of such architecture. This focus of this conversation was address directly in the session STO5336 “VMware Virtual SAN – Architecture Deep Dive” delivered by Rawlinson Rivera @Punchingclouds and Christos Karamanolis

Note the Hybrid Design vs the AFA VSAN are NOT the same. It that flame bait? No.

I will try to summarize it here. Both are hyper-converged SDS solutions employing the benefits of clustered storage and SSD. Both have simplified operations and lower TCO. Both deliver fantastic consistent predictable performance.

Cache is where it is at. Read Write

SSD is leverage in both the hybrid model and the AFA model but the key is the are leverage DIFFERENTLY.

70/30 Read Write for Hybrid moreover… this is the crux. The algorithm used to deliver performance for a AFA is different than the Hybrid approach.

  1.  LRU vs ARC.   That is why you cannot. I repeat CANNOT bolt on All SSD in a licensed Hybrid VSAN design and get the same performance of a  AFA solution. I have tried to explain this to a customer but one persons results skew their reasoning.  I would tend believe results that you and review that are from countless man-hours of R&D vetted against multiple manufacturers / vendors and various RA bench marks
  2.  Read Write characteristics of SSD / HDD   Not all SSD are created equal. SSD and HDD are different. vmware knows this. Hardware manufactures know this. Storage engineers know this. But Consumers sometimes know this. You have the flexibility of choosing your own hardware to fulfill your requirements for VSAN ref: vmware compatibility guide

“A candle that burns twice as bright burns half as long” 

VSAN 6.1 AFA design understands this at a much deeper level. It is designed and tested for not only performance but for longevity. VSAN 6.1 helps you sort out which SSD is appropriate.  HDD aka spinning magnetic disks can handle long term data and dense capacity at a price: IOPS. Conversely SSD can handle 10x increase in IOPS but at the cost of density and longevity.

Pick carefully your components. To borrow from the culinary world. The best ingredients make for the best food. You will consume something awesome if you cook something awesome. “You can’t make chicken salad out of chicken sh*t” lol

Reference Duncan’s explanation of the VSAN Ready Nodes http://www.yellow-bricks.com/2015/08/25/virtual-san-ready-nodes-taking-charge/

Well there is more to VSAN 6.1 and vmworld2015… more when I can find some time to share. Thanks for reading.

Is your IT infrastructure an Oil Tanker?

I had the opportunity to attend an Avnet/EMC/VMware/Brocade sponsored for channel partners EVO-RAIL VSPEX Blue BootCamp.

In a nutshell it was all you can drink information from a firehose— about EVO-RAIL specifically the EMC VSPEX BLUE.

EVO RAIL is a new beast of an animal. It is a different breed. No, not in and single dimension you measure. The combination of technology presented is a synergy. Definition: Synergy is the creation of a whole that is greater than the simple sum of its parts.

Yes; you can get the the form factor for compute separately. You can also do the same for VSAN and ESXi vSphere 5.5 and networking. but you cannot get what the entire VSPEX BLUE offering of EVO RAIL provides TOGETHER.

I get ahead of myself.

You have to have perspective to understand where we are today. To me that means if you don’t know where you come from you  cannot know where are you today and where you will be tomorrow. It is all relative.

EVO RAIL is a clustered system. It is a Datacenter in a 2U form factor. Not just compute but a modern hyper converged solution.


Sure. Another IT buzzword. Is it just talk? I would say no.

Everyone is talking about it but only a few are “doing it”… more often than not the IT industry is a buzz with the new technology of the day.
In this case it is Hyper-converged.
To put it simply storage is local to compute. What a minute how is that different than 15 years ago when Client-Server model was the norm and storage was already local to the compute. Compute meaning the processing of the server CPU. Well lots has changed.
How is it better. A snapshot of what is the current available technology:
  • Compute is way, way faster and more dense.
  • Networks are 10 gigabit vs Fast Ethernet 100 Base-T or FDDI Optical rings are no longer the only viable choice.
  • Storage is IOPs crazy.
  • And add to that the agility of VMware Virtualization!
So look at the speed of compute, network and storage. Technology will continue to get faster and better (lower cost for the return on investment).
But what really hasn’t changed much is the complexity of the solution. There are many moving parts but how do you delivery your solution today to support legacy applications and have the agility to respond to changing business objectives.
The old phrase “turning an oil tanker on a dime”. Is IT today an Oil tanker? Does your private cloud have the agility your business requires? How will the current toolset respond? How will your staff? Oh what was that “IT staffing has been reduced and that is a trend that hasn’t gone away”
IT organizations are forced to do more with little… queue in viable alternatives…
Do you outsource.. the simplest short term gain, but not always the best long term investment.
I view EVO rail as a datacenter in a box. EVO-RAIL VPSEX Blue is Not the Cluster in a box solution but much much more.
This reminds me of the forerunning of MCSC cluster in a box. Like I said.. perspective. Where has the IT industry been before relative to where it is today. I recall back in the day when cluster in a box was a viable (and the best solution at the time) era 2000. That was only a high-availablity MSCS cluster with one node active at a time. 
Yes, I deployed and supported a few of these solutions. It was cutting edge back then.
Adjusted for inflation:
$1000.00 USD in 2000 is $1395.20
so the CL1850 was $27,864.00 (2000 $)
or $38,569 in 2015 $.
But what did you get for your IT dollar?
Each “node” in the CL1850
  • 1 GB RAM (128-MB 100-MHz registered ECC SDRAM memory)
  • 2 Pentium III processors @550MHz
  • 3 NICS (one dedicate for internode communication) 100 BaseT
  • 2 RAID controllers
Shared Storage System:
  • 218.4 GB (6 x 36.4-GB 1′′ Ultra3 10,000 drives)
Form Factor 10U
Fast forward to 2015….What does EVO-RAIL VSPEX BLUE PROVIDE:
Each EMC VSPEX BLUE appliance includes:
  • 4 nodes of integrated compute and storage, including flash (SSD) and HDD — 2U
  • VMware EVO:RAIL software including VMware Virtual SAN (VSAN), Log Insight
        12 cores @ 2.1GHz per node
  • 128 or 192 GB memory per node
  • Choice of 10 Gigabit Ethernet network connectivity: SFP+ or RJ45
  • Drives: up to 16 (four per node)
  • Drives per node: 1 x 2.5” SSD, 3 x 2.5” HDD
  • Drive capacities
    • HDD: 1.2TB (max total 14.4TB)
    • SSD for caching: 400GB (max total 1.6TB)
               14.4TB capacity RAW
Additional Software Solutions – Exclusive to VSPEX-BLUE
  • EMC VSPEX BLUE Manager providing a system health dashboard and support portal
  • EMC CloudArray to expand storage capacity into the cloud (license for 1 TB cache and 10 TB cloud storage included)
  • VMware vSphere Data Protection Advanced (VDPA) for centralized backup and recovery
  • EMC RecoverPoint for Virtual Machines for continuous data protection of VMs. Includes licenses for 15 VMs.

The question still remains. Is your IT infrastructure an Oil Tanker? OR Can you turn on a dime??

How does your IT respond the the ever changing business demands?

Is EVO RAIL for everyone? There are a lot of use cases that EVO RAIL VSPEX BLUE will work perfect for. But,  No it isn’t for everyone.  BUT what it does is usher in a new consumption of IT that is different manner. You will not have to be provision your datacenter in the same piece meal function. You can commoditize that to a pre-validated solution that is supported by a single vendor.


This graphic has a lot more details than this single blog post can explain! I will try to explain each section that helps to make VSPEX BLUE a different redefined EVO RAIL solution.


– VSPEX BLUE Manager users can conveniently access electronic services, such as the EMC knowledge base articles, access to the VSPEX Community for online and real-time information and EMC VSPEX BLUE best practices.


– ESRS is a two-way, secure remote connection between your EMC environment and EMC Customer Service that enables remote monitoring, diagnosis, and repair – assuring availability and optimization of your EMC products






– Built in Deduplicated Backups, powered by EMC Avamar


– Block and File upto 10 TB FREE. “EMC CloudArray software provided scalable cloud based storage with your choice of many leading cloud providers enabling limitless Network Attached Storage, offsite backup and disaster recovery and the ability to support both Block and File simply”


– Built into the VSPEX MANAGER dashboard. This unique feature enables customers to browse complementary EMC and 3rd party products that easily extend the capabilities of the appliance.

Again there are a lot of take aways for the VSPEX BLUE – EVO RAIL solution. Contact me if you more information.



What is it: 

“The vVNX Community Edition (also referred to as vVNX) provides a flexible storage alternative for test and development environments.”

SDS = Software Defined Storage!!
Download a full-featured version of vVNX available for non-production use without any time limits.”
File type: *.ova
File Size: 2.1 GB
Release date: 5/4/2015
Version: 3.1.2

Download here! https://www.emc.com/products-solutions/trial-software-download/vvnx.htm

Architecture and functionality:

White Paper details: vVNX Community Edition https://community.emc.com/docs/DOC-44552

Installation Guide: https://community.emc.com/docs/DOC-42029


Before you can obtain and activate the trial vVNX license, you must have completed the following tasks:

  1. Registered to create a product support account.
  2. Downloaded the vVNX software.
  3. Installed vVNX.
  4. Launched Unisphere.
The Configuration  runs when you log in to Unisphere for the first time.
  1. Note the vVNX system UUID provided on the License dialog in the Configuration Wizard.
  2. Go to the Electronic License Management System (ELMS) download page at www.emc.com/auth/elmeval.htm
  3. Click Obtain evaluation license for vVNX.
  4. Enter the vVNX system UUID and select vVNX as the product type. 
  5. Click Download to save the license to your local system.
    Note: An email confirming that you have successfully obtained the evaluation license is sent to the email address you provided when you registered.
  6. Return to the License dialog in the Configuration Wizard and click Install License File
  7. Locate the license file, select it, and click Upload to install and activate it. 

Note: Do not repeat this procedure once you have saved the license and received the confirmation email. If you try to enter the vVNX system UUID again, you will receive a “Duplicate UUID” error message.

EMCWorld 2015 – Redefine Next … two big items to consider



One “little” event, but this one event is in truth packed with so much activity;  you can actually feel the energy in the air.  For those who don’t know what it is about: EMC World is about: cloud computing, virtualization, Big Data storage and analytics, and IT security and protection. Each individual item alone is enough for a 4 day conference but all together under one roof. It is a lot to take in.

There is a wide audience in attendance: Vendors, Customers, Partners. So basically a little (or a lot) of something for everyone. There are multiple tracks in each topic: beginner, Intermediate and advanced. And there are different focus areas to learn about.

EMC isn’t just about storage, sure it is at its core DNA, but there is so much more to EMC than storage. If you don’t think so .. look at this:

EMC {code}


There are currently 33 Public Repos in development, not counting any forks.

Storage isn’t just about LUNS, RAID Groups and IOPS. No sir/ madam. DevOps is alive and well and pushing the envelop, really maximizing the technology under the hood. Making things happen you never thought possible, but more importantly redefining Next.. the next use case, the next different way to do things.

And if you are ready.. look at this current offering that well, is more mature than you might think. Don’t assume All Flash Arrays are equal. Today lots of different vendors are offering a flash array of some kind, hybrird or 100% All Flash array options.

I had a conversation recently and told my colleague I am storage agnostic, as I have worked with just about every storage vendor out there. BUT that doesn’t every storage is for everyone. You can’t force one-size fit all for storage.

There are requirements and understanding those requirements, well that is my job.




Today I had the opportunity to meet some very incredible individuals. It may be over-used by “passionate” is what this guy is about. The details shared, I cannot discuss publicly at this time, but WOW. xtremIO 4.0 release isn’t a maintenance release it is HUGE.. to paraphrase “4.0 release is a big deal, if not as big a deal as the 1.0 release.”

Read more here:



Here is one thing to really dig deep into:

Think about how you do business today. What do you have in production? What is working? What could work better?

How could it work better?

Do you expect your Application development team to turn on a dime to adopt something new and shiny?

Agility may be a buzz word, but the cost to re-tool is a big deal. I mean, cap-ex is only one dimension. OpEX and adoption rate is not a small deal. If you have invested effort, evaluate how to make your TCO maximize the ROI. Buzz words or reality? You can’t expect to drop everything and jump to the latest and greatest just “because”. Nope. That is irresponsible. The luxury of a greenfield deployment isn’t something everyone has at their disposal.


Hint: dig deeper into “Snapshots” find about how to really leverage this for your organization.

More thoughts to consider later. Sure there are fun and games, but EMCWorld is a lot of game changing information to absorb. The theme this year is “Redefine Next” what is your next step? what is your next technology investment? Who will be on your side to put your organization in pro-active quadrant? How will you empower your team/company/self for the next 90, 180 days? How about the next fiscal year?

more later.

part 2/2 Troubleshooting VSAN errors. VSAN misconfiguration??

VSAN cluster isn’t 100% there is some problems with see all the storage. 
1. Cannot write to the VSAN Datastore
2. Correct amount of capacity isn’t present
How to resolve with ESXCLI
Log into each host via ssh. You do remember the root password right?? This example is a three node VSAN cluster.
node 1::::::
~ # esxcli vsan cluster get
Cluster Information
Enabled: true
Current Local Time: 2015-03-31T01:11:33Z
Local Node UUID: 551374b5-03f9-7bd6-6257-a0369f58b8e8
Local Node State: MASTER
Local Node Health State: HEALTHY
Sub-Cluster Master UUID: 551374b5-03f9-7bd6-6257-a0369f58b8e8
Sub-Cluster Backup UUID: 54f9dc6f-8674-f412-364d-a0369f58b5a8
Sub-Cluster UUID: 551374b5-03f9-7bd6-6257-a0369f58b8e8
Sub-Cluster Membership Entry Revision: 1
Sub-Cluster Member UUIDs: 551374b5-03f9-7bd6-6257-a0369f58b8e8, 54f9dc6f-8674-f412-364d-a0369f58b5a8
Sub-Cluster Membership UUID: d6da1955-e2f8-38eb-d7f0-a0369f58b8e8
NODE 2:::::
~ # esxcli vsan cluster get
Cluster Information
Enabled: true
Current Local Time: 2015-03-31T00:12:02Z
   Local Node UUID: 55197cee-f530-4966-5ea6-a0369f58b8e4
   Local Node State: MASTER  << a different master for a different UUID
   Local Node Health State: HEALTHY
   Sub-Cluster Master UUID: 55197cee-f530-4966-5ea6-a0369f58b8e4  <<< That is a different UUID!
   Sub-Cluster Backup UUID:
Sub-Cluster UUID: 54f9dc6f-8674-f412-364d-a0369f58b5a8
Sub-Cluster Membership Entry Revision: 0
Sub-Cluster Member UUIDs: 55197cee-f530-4966-5ea6-a0369f58b8e4
Sub-Cluster Membership UUID: 60df1955-b9f1-d685-13b0-a0369f58b8e4
~ # esxcli vsan cluster leave
~ # esxcli vsan cluster join -u 551374b5-03f9-7bd6-6257-a0369f58b8e8   <<<- join the correct UUID (cluster)
Validate on NODE 2
~ # esxcli vsan cluster get
Cluster Information
Enabled: true
Current Local Time: 2015-03-31T00:12:52Z
Local Node UUID: 55197cee-f530-4966-5ea6-a0369f58b8e4
Local Node State: AGENT
Local Node Health State: HEALTHY
Sub-Cluster Master UUID: 551374b5-03f9-7bd6-6257-a0369f58b8e8
Sub-Cluster Backup UUID: 54f9dc6f-8674-f412-364d-a0369f58b5a8
Sub-Cluster UUID: 551374b5-03f9-7bd6-6257-a0369f58b8e8
Sub-Cluster Membership Entry Revision: 2
Sub-Cluster Member UUIDs: 551374b5-03f9-7bd6-6257-a0369f58b8e8, 54f9dc6f-8674-f412-364d-a0369f58b5a8, 55197cee-f530-4966-5ea6-a0369f58b8e4  <<< three members
Sub-Cluster Membership UUID: d6da1955-e2f8-38eb-d7f0-a0369f58b8e8
~ # esxcli vsan network list
VmkNic Name: vmk1
IP Protocol: IPv4
Interface UUID: 17dd1955-0bdf-abac-aba6-a0369f58b8e4
Agent Group Multicast Address:
Agent Group Multicast Port: 23451
   Master Group Multicast Address:
Master Group Multicast Port: 12345
Multicast TTL: 5
Valdiate on NODE 3 and 1
NODE 3::::::
~ # esxcli vsan cluster get
Cluster Information
Enabled: true
Current Local Time: 2015-03-31T00:14:33Z
Local Node UUID: 54f9dc6f-8674-f412-364d-a0369f58b5a8
Local Node State: BACKUP
Local Node Health State: HEALTHY
Sub-Cluster Master UUID: 551374b5-03f9-7bd6-6257-a0369f58b8e8
Sub-Cluster Backup UUID: 54f9dc6f-8674-f412-364d-a0369f58b5a8
Sub-Cluster UUID: 551374b5-03f9-7bd6-6257-a0369f58b8e8
Sub-Cluster Membership Entry Revision: 2
Sub-Cluster Member UUIDs: 551374b5-03f9-7bd6-6257-a0369f58b8e8, 54f9dc6f-8674-f412-364d-a0369f58b5a8, 55197cee-f530-4966-5ea6-a0369f58b8e4
Sub-Cluster Membership UUID: d6da1955-e2f8-38eb-d7f0-a0369f58b8e8
~ # esxcli vsan network list
VmkNic Name: vmk1
IP Protocol: IPv4
Interface UUID: f16e1355-c174-11f6-2602-a0369f58b5a8
Agent Group Multicast Address:
Agent Group Multicast Port: 23451
Master Group Multicast Address:
Master Group Multicast Port: 12345
   Multicast TTL: 5
~ # esxcli vsan cluster get
Cluster Information
Enabled: true
Current Local Time: 2015-03-31T01:12:22Z
Local Node UUID: 551374b5-03f9-7bd6-6257-a0369f58b8e8
Local Node State: MASTER
Local Node Health State: HEALTHY
Sub-Cluster Master UUID: 551374b5-03f9-7bd6-6257-a0369f58b8e8
Sub-Cluster Backup UUID: 54f9dc6f-8674-f412-364d-a0369f58b5a8
Sub-Cluster UUID: 551374b5-03f9-7bd6-6257-a0369f58b8e8
   Sub-Cluster Membership Entry Revision: 2
   Sub-Cluster Member UUIDs: 551374b5-03f9-7bd6-6257-a0369f58b8e8, 54f9dc6f-8674-f412-364d-a0369f58b5a8, 55197cee-f530-4966-5ea6-a0369f58b8e4 <<— all three members!
   Sub-Cluster Membership UUID: d6da1955-e2f8-38eb-d7f0-a0369f58b8e8
~ # esxcli vsan network list
VmkNic Name: vmk1
IP Protocol: IPv4
Interface UUID: 3b5e1955-5eb6-2bbc-57bc-a0369f58b8e8
Agent Group Multicast Address:
   Agent Group Multicast Port: 23451
   Master Group Multicast Address: <<< all the same Multicast address
   Master Group Multicast Port: 12345
Multicast TTL: 5
~ #