docker
Cloud, Fixes, Tools

Docker & Vagrant & VMware Fusion bug

I was playing around with my new Mac this week since I started at Intel on Monday. I was starting to play with the Docker provider for Vagrant when I ran into some issues with it.

For those that aren’t up to speed – Vagrant now supports Docker as a Provider via plugin. If you are running on a system without native Docker support Vagrant will bootstrap a box that is native and proxy Docker commands into it. Rather slick if you ask me.

Problem is there appears to be a bug with a recent release of the vmware_fusion provider plugin which is breaking this functionality with VMware Fusion. By default the Docker provider uses a box hosted on Vagrant Cloud called mitchellh/boot2docker. This box is a lgithweight VM with an ISO mounted that is used to live boot. The bug has something to do with mucking up IDE/CDROM binding for Vmware Fusion VMX files. I didn’t dive into the bug for details.

I did search for workarounds and found some. But they were all a bit specific and fragile. Instead of using them I decided to use the built-in magic that Vagrant already provides with their Docker provider that allows you to select the box you want to use. So instead of using the currently broken mitchellh/boot2docker (on vmware_desktop) we will select a different box.

We need a box with native docker support. An Ubuntu box would serve nicely but you can use your flavor of distro. There is the option to choose a box with a built-in docker stack. I chose to use Vagrant’s Docker installer functionality to automatically install the latest Docker. This takes a little longer upfront but ensures I have the new hotness.

I uploaded my setup to Github here: https://github.com/lynxbat/boot2docker-fixie-example

The instruction from the README:

git clone git@github.com:lynxbat/boot2docker-fixie-example.git
cd boot2docker-fixie-example/docker_container
vagrant up

This will bring up the vagrant host VM (box-cutter/ubuntu1404), build a Docker image, and then run a container exposing port 80 and mapping to the host vm on port 8080. Then after the container comes up just run:

rake demo

To open up the Apache2 default page in your browser (host_vm:8080). This is apache2 running in a container on top of your host VM. Hopefully this example helps someone else in getting Vagrant and VMware Fusion working again with Docker while the bug is being fixed.

PS –  there is a weird bug where you get this:

Stderr: 2014/06/12 13:49:01 Get http:///var/run/docker.sock/v1.12/containers/json?all=1: dial unix /var/run/docker.sock: permission denied

If you run ‘vagrant up’ a second time it usually fixes itself.

Cheers,

.nick

 

Commentary

Sell, Execute, Grow, Iterate

Sell the vision well and the doors will open. Very few good ideas get to happen without someone selling the idea to someone. It is the vision of it. The story that is told. The belief in something better that greases wheels. As a recent boss loved to say:

You can do anything you can justify

One of my favorite things to say is: execution is all that actually matters. Say what you will do (sell it) then do it. An inability to move from point #1 to point #2 in my personal belief is the greatest skill gap in technology today. It is one thing to think of great ideas. It is more important to think of realistic ways to make that idea become reality. It is interesting to see how experience matters here. Sometimes those with experience in the pain of execution not happening are wise enough to be ready the next time around. Specialized skills in organizing others, communicating across disciplines, translating the vision to implementors, and the ability to persistently push to deliver are what make amazing executors.

The tail end of sell and execute is ownership. Breaking the ice on a new idea and making the impossible possible comes down to awareness that everything changes. The same skills that are awesome for selling and execution may not translate to supporting and growing. In the execution phase the pain of missing skills or missed concerns can be smoothed over with super-heroic effort. But long-term growth and stability comes from feeling pain. You cannot properly fill a hole in the armor if you do not know it exists. Growth is the addition of people, pieces, and process that make something small and shiny evolve into something large and powerful. Growth happens out of pain. In sell and execute we focus on what is possible and ignore the pain. With growth we focus on the reality of now and the steps needed to permanently solve pain and allow for so much more.

Growth can lead to stagnation. Laying down roots and becoming a tree people can depend on is important. But a tree cannot change location easily. Changes will be required and some of these changes require large effort. They will look scary to the big tree that has grown up. Which means someone needs to sell the reason why change is important and set a vision for where to go next.

Do not attempt to execute before you sell. Do not grow before you execute. The grower looks to the seller. The seller to the executor. The executor to the grower. And so we iterate towards the impossible again.

.nick

time to go
Life/Work

Goodbye Zombies

It has been amazing. That is the only way to describe it.

Project Zombie was an experience filled with amazing challenges and people. And for me this season is over.

I will be leaving VMware soon to take a position at Intel as a Principal Cloud Architect. I will be focusing on helping them develop some amazing new things in distributed automation, DevOps, and more.
This is not just a big deal for me. I will also be moving the Weaver clan from our home here in Texas. We will be settling into Portland Oregon area and starting a brand new adventure. We recently took a trip to Oregon over Spring Break and fell in love with the state. My family is excited to start exploring a new part of the United States.

The team will go on without me. I am proud to have been involved during this period of building out vCHS. And I feel lucky I was able to help assemble this mutant army of coding cloud zombies. The greatest part of this transition is knowing that the team will easily fill the small void I leave behind because of how awesome they are at what they do. The team culture of strong collaboration, ownership in what you do, and thinking out of the box was something brought by each member and it will live on.

I have been in the EMC/VMware family for over four years. And it is not without some sadness I move over to Intel. I have nothing but good memories and wicked skills thanks to the amazing opportunities I had as a part of this family.

Only thing I can promise is you will see more from Nick Weaver.

Stay tuned

.nick

Commentary

The problem with musical chairs

You walk into a room. In that room are your coworkers. Some are good friends. Most are not.

musical_chairs_sq

You look around the room and see chairs all around. And in the middle is a table with a set of speakers. A person walks in and announces to the whole group that there is a critical need for everyone to sit down in the room. ‘But first some music will play and everyone will circle the room.’ they say kindly. ‘Then when the music stops every one in the room is each personally responsible for finding themselves a seat. It is most important that everyone has a seat and through cooperation we will accomplish that.’ and with that last statement they exit the room.

Somehow this sounds vaguely familiar to you. But you shrug your shoulders and wait.

The music begins, a bright marching chorus filled with horns and stringed instruments. You and the rest of the group begin awkwardly circling the room.

About a minute after you start circling you suddenly notice something odd. There appears to be fewer chairs than people. You count in your head again. No, you are sure this time. There are 10 people but only 7 chairs.

You tap the shoulder of the person in front of you, James from the Widget Office. ‘James. James, there are not enough chairs…’ James seems to ignore or not hear you. So you tap harder, visibly making James flinch under your stiff finger. ‘JAMES’ you say with a strong voice over the music. ‘There are only 7 CHAIRS. But 10 of US.’ James quickly glances back with a fearful look on his face and and waves his hand at you dismissively.

You feel confused. James seemed irritated and maybe a little scared with your information. You look back at Selma from the Counters Team. She is looking at you with a confident smirk. She heard your interaction with James. ‘Well, I don’t know about you. But I will have no problem finding MY seat. Maybe you should focus on your seat rather than worrying about others’ she snaps at you. Then she looks off into the distance with her confident smirk.

Now you are really confused. Why would we be doing this? Why is everyone ignoring the problem in the room? If it is important that everyone finds a seat and there are not enough seats, then we should be focusing on getting more chairs rather than marching to the music. What happens to those left standing?

As you look around the room you realize what is happening. Some people look you in the eyes with concern. They see you understand and their eyes are almost a warning. Some look at you with coldness. You can almost feel them counting you as one of the unlucky 3. And the rest stare straight ahead looking at no one. They are focusing on being ready for when the music stops.

The music builds and builds. Its volume increases and you feel the climax coming. Then suddenly, it stops…

-

Failure is not always a surprise. Sometimes it is seen well ahead of time. Integrity is calling out risks even if you may not be the one left standing. The difference between a leader and a follower is the ability to ask for more chairs when the time comes. Even if it means pausing the music. The leader is concerned with the group success and not just the odds of personal failure.

And always count the chairs.

Commentary

Effort vs Credit

In life, your effort will be judged by your peers, superiors, and dependents.

Sometimes there is effort you can make which is somewhat impossible for your superiors to fully grasp. Maybe it is too technical. Perhaps they are too busy. Or it may just be your inability to communicate the value. But you know why the extra effort is valuable. You know that someday someone is not going to run into a problem because you decided to do things right. You know that you might not even be around when that time comes. Which presents every technical contributor with a problem. Do I only commit the effort that will be recognized? Or do I hold myself to a higher standard and judge myself first?

Your boss is just a single factor in what affects your life and you will have many bosses in this modern world. My opinion is you always treat your effort in life as a reflection of how you want to be seen. Even if there is no guarantee anyone will ever see it. Not because you get credit from those above you. You do it because that is the person you want to be.

The funny thing about integrity in your work is that it does not go completely unnoticed. Your peers will see it. If you have subordinates they will pick up on it. And as it bleeds into your personal life, your family will see it as well.

There are a lot of things that can rot away at our integrity. Unrecognized work, wasted effort from poor communication, random events, and much much more. Sometimes it is hard to remember why you should do the right thing when the wrong thing wins.

In life, your effort can win more than credit.

It can make a better you.

NOTE: It seems some see this post as a rant about my current position. It is not in the slightest. In fact this post is all about letting go of trying to get recognition for things you can’t and instead focusing integrity first. Just to be clear this post is about my developing philosophy and not me complaining about the wonderful role I am in currently.

Cloud, Coding, Speaking

Bring the popcorn

I have been traveling around this year speaking on automation and how it is a positive evolution to technology engineers. I love nothing more than inspiring engineers to push themselves into fun tools and patterns for making themselves more valuable.

This week and next I will be speaking at both VMworld 2013 and PuppetConf 2013 on automation and the project that has consumed my time for the last year, Project Zombie. If you are even slightly curious what the vCHS automation team has been up to and what the result of that work is, you owe it to yourself to catch one of those sessions. Here is my schedule over the next two weeks:

PuppetConf 2013

Keynote
Time: Friday 8/23, 10:25am
Title: VMware vCHS, Puppet, and Project Zombie
Linkhttp://puppetconf2013b.sched.org/event/69737e047bb25563258c1ddbc15fed2c#.UhIom2TF2nQ

This will be a 25 minute long highly compressed story of how vCHS built a custom framework alongside Puppet to manage automation (Project Zombie). I will also have special guests join me on stage.

Session
TIme: Friday 8/23, 11:00am
Title: Automation Generation: Or Why Skill is the Answer to Scale.
Link: http://puppetconf2013b.sched.org/event/e14d2b1f30886edc0a52bc3be7a5ea55#.UhInbGTF2nQ

In this session I will be covering automation and why it is a positive evolution for engineers in technology

VMworld 2013

Session
Time: Wednesday 8/28, 3:00pm
Title: The Story Behind Designing and Building a Distributed Automation Framework for vCloud Hybrid Services
Linkhttps://vmworld2013.activeevents.com/connect/sessionDetail.ww?SESSION_ID=5640

This will be the longer version of the Puppet Keynote on Project Zombie. Nan Liu will join me for this and we will cover in-depth what Project Zombie is and how we approached DevOps automation at scale. Nan will also be announcing a very cool thing for the automation community during this session

I am completely thrilled that I get to share some of the work we have been doing and I am looking forward to catching all the other amazing sessions at both of these events.
If you do attend a session, please come by the front after and say hi.

.nick

Project Zombie

Commentary

nickapedia.com 3.0

butterfly

This blog has bounced around between a lot of different things as I have adapted to what my career has placed before me. Today I start a new iteration. And in this iteration I am going to focus much more on whatever strikes my fancy as a personal blog. In the past this has been in enabling the virtualization community through comments and code. Now my focus is more open-ended. Much less working for months up to one big post and much more openness about what is on my mind.

Some will find it less useful to them. But time moves on.

Cloud, Coding, VirtualAppliances, VMware

VMware & Puppet : One More Step Forward

I wake up everyday committed to not doing things the same old way. That is why my title has that funny word ‘Automation’ in it. I am tasked with finding a way to make what is fragile, expensive, risky, and complex; easier to manage.

While most of what I am working on at VMware is not ready for the light of day. There are some things that have come out of our work that would have a greater benefit being in the public space. If you are even an infrequent reader of this blog (which given my posts lately, is everyone) you know that I am a huge fan of PuppetLabs. More than that I am a huge fan of automation using DevOps-style toolsets enabled by community development.

Right now I belong to an amazing team of crazy talented engineers. I rarely go a week without seeing something that makes me wish I was capable of doing a backflip. I am proud to announce that today some of that magic is now being put in a place for everyone.

Today VMware and PuppetLabs are releasing the first set of modules for managing VMware environments. This brings a bunch of new features for the first time:

Management for the vCenter Appliance /w

  • Complete initialization – Just point as a VM that was deployed from the OVF and it will standup vCenter.
  • Datacenter and Cluster configuration – HA, DRS, and more. The one caveat being EVC (but coming soon)
  • ESXi Host – Grab and add ESXi hosts to clusters and manage them through Puppet. Something that *might* align nicely with another tool called Razor.

Management for vShield Manager

  • Complete initialization – Just like the VCSA, point Puppet at your deployed vSM appliance and watch it complete the setup and stand it up.
  • Association to vCenter – Bind the vSM to a vCenter automagically.
  • vShield Edge Deployment – Point it at a resource pool and easily deploy vShield Edges to vCenter.
  • vShield Edge Configuration – Control the firewall, NAT, load-balancing, and more using Puppet config. – Note: We are actually adding more this now so this may be in flux.

With the features above you may now deploy out VCSA and vSM appliances and automatically configure them into vSphere clusters with firewall, NAT, load-balancing and more. You can carve up the ESXi nodes into any number of clusters or config. And this is just the beginning.

The other great part of this is that we are today releasing all of these modules under the Apache 2.0 license and submitting to the Puppet Forge. We strongly believe that VMware admins that have not played with Puppet need a way to start. And even more importantly there is a great number of brilliant people that can add to this work and make it even better for the community as a whole.

The modules can be found on github:

If you want an awesome technical walkthrough. Then read the blog on this release from one of the main authors, Nan Liu:  http://nanliu.github.com/blog/2013/02/27/puppet-managing-vcenter-and-vshield/

I want to thanks the team behind these modules for the amazing amount of work they accomplished in a crazy short amount of time:

  • Randy Brown
  • Shawn Holland
  • Nan Liu
I cannot emphasize enough how this is just the tip of the spear. There is so much more to be done in the realm of automation for cloud uses cases. And before this year is up, I am personally hoping we can deliver some amazing technology for everyone in cloud.
.nick