May 21, 2012

UBERLinkedTwit : Chambers special

So my buddy posted the following tweet today:

And since I had some calls today I decided to kick off a project to do just that.

So here is a simple github link for UBERLinkedTwit:

Basically a command line ruby script which using the LinkedIn gem to auth, pull, and grab twitter ID’s for your connections. I haven’t rigged it up to auto-follow using the Twitter gem yet. But that is pretty easy using the Twitter gem examples.

Hit the github project, get LinkedIn developer keys, read the README, and have fun. Free free to extend and I will pull into main project.

.nick

 

 

 

 

 

 

Proving a another guy’s point : MacBook Air & Node.js as a Webserver

Ok, this is a real quick example of someone else’s idea. Namely Christofer Hoff of Juniper (infamous blog & twitter account) who in response to my tweet about my MacBook and a webserver (I am a Mac-newbie). Suggested I try using Node.js.

beaker_being_beaker

At first I was a little taken back. I think @Beaker thought I didn’t get the point (and I don’t more than I do).

beaker_being_beaker2

So I *cheated* and just turned on Web Sharing with the good tips from my other twitter buddies. But the Node.js idea bugged me because technically Chris is right. It should work fine.

So I did it. I installed the Node.js package for OSX (been using the Linux version) and fired up TextEdit and wrote this simple bit of code:

var http = require('http');
var fs = require('fs');
var path = require('path');
var rootpath = "/Users/lynxbat/Webroot";
 
console.log('*Starting');
http.createServer(function (request, response) {
 console.log('Server started');
 var pathToFile = rootpath + request.url;
 console.log ('File was requested: ' + pathToFile);
 path.exists(pathToFile, function(exists) {
 if (exists) {
  fs.readFile(pathToFile, function(error, content) {
   if (error) {
    response.writeHead(500);
    response.end();
   } else {
    response.writeHead(200, { 'Content-Type': 'application/x-gzip' });
    response.end(content);
   }});
   } else {
     response.writeHead(404);
     response.end();
   }});
}).listen(8010);
console.log('Server running at http://127.0.0.1:8010/');

I fired up the server:

node webserver.js

And opened my Safari browser to request a file I had dropped in my root directory.

Screen Shot 2011-12-19 at 4.18.55 PM

Which works perfectly. Output on cli:

*Starting
Server running at http://127.0.0.1:8010/
Server started
File was requested: /Users/lynxbat/Webroot/node-v0.6.6.pkg

Now in my case I am using the Webserver to hand out packages for installing as a local mirror for some VM’s I am building (big demo for Jan). So I hardcoded the MIME type and left response encoding unset. If you were looking to properly serve you should perhaps add some logic for switching types and responses based on the file extension. Kind of like a proper web server works.

So Chris was right. This works. Though I think proving him right probably isn’t a good habit to pick up.

.nick

For Advanced Users : UBERAlign API / CLI / Powershell

From the beginning I knew some UBERAlign users would want to go into power user mode. The UBERAlign Console was designed to allow for easy use for the average Joe. But, there are people out there with the desire, guts, and ability to script and automate that want more. So this post will inform you on two other options for UBERAlign.

1. The CLI

Each vAligner is a Ubuntu Linux VM. On the VM is a set of binary files that do all work. One is a startup file for initializing, one is a daemon for accepting new jobs via the REST API, and the final is the actual magic behind the scenes. From the beginning UBERAlign was designed to be run from the command line. In fact back months ago the vSpecialist actually got a copy of this to try out and help me test. So for those that do not want to use the Console here are some reasons and instructions on how to run alignment, reclaim, and alignment+reclaim jobs manually.

Some of the reasons you may want to do this:

  1. Hate MS Windows – Since the console is a .Net WPF app some Mac users (@mcowger) have already asked how to skip using a Windows VM.
  2. VM size is too big – If the VM is more than 50% of the size of the datastore then a snapshot of it can potentially cause an out of space issue if it grows to full size (which and align can do). So if you have a good backup or are using array-based snapshots why use VMware snapshots? The CLI allows you to skip having to use a VMware snapshot which is required for using the Console.
  3. Aligns disks not attached to a VM -  The manually method just needs a valid VMDK file. Whether from a backup or anywhere. The Console requires a VM. Technically you can use tricks with snap and copy similar to the way VMware Converter works with online VM’s if you want.

So here are the steps:

  1. You must have the vAligner running and it must be on a vSphere host than can reach the datastore your VMDK is on.
  2. Make sure that any secondary disks are removed from the vAligner from a previous job.
  3. Make sure no UBERAlign Consoles are using this vAligner. Easy way is to disconnect the NIC and use the console if you are worried. If you console is showing weird FD0 garbage, grab a new copy of the vAligner OVA as this was patched.
  4. Add a Hard Disk to the vAligner (while it is running) to SCSI 0:1. Make sure and choose an existing disk and point it to the VMDK you want to run against.
  5. Login to the vAligner with root/UBERAlign.
  6. Make sure at least 15-20 seconds have passed since step 4. From the command line run: ‘fdisk –lu’ and make sure /dev/sdb is showing. Confirm the partition layout is as expected.
  7. Change directory to the UBERAlign binaries with ‘cd /opt/uberalign/bin’

Now that you have everything attached we are running to run a job. If you run ‘./uberalign’ you will see a printout of the command syntax like what follows:

./uberalign MODE  DEVICE  [OFFSET]  [ID]  [NAME]

MODE = The mode you want to run in and is formatted. Syntax:  -r:[a|r|s|c]

  • [a] = Align & Reclaim
  • [r] = Align only
  • [s] =  Simulate
  • [c] = Check alignment only
  • [z] = Reclaim only

DEVICE = The hardware device your VMDK is mounted on(normally /dev/sdb): Syntax: /dev/sdb

OFFSET = The target offset you want to align to. This defaults to 2048 sectors if not specified.

ID / NAME = Only useful for the API leave these off or use for logging purposes.

Example:

./uberalign –r:a /dev/sdb 2048

Once you run this command the uberalign program will process and log to the screen it’s progress while it works. When it completes, disconnect your VMDK from the vAligner and check your VMDK within a VM. It is as simple as that.

2. The API

The UBERAlign vAligner comes with a REST API that the UBERConsole uses for managing. While the state machine, orchestration for vSphere, and GUI are all a part of the console; you can still use the vAligner API to integrate into any existing processes you may have. Here is a short guide to the REST API. All data is return in JSON format. The first section explains the API get/put and the second explains the object structure in a pseudo format.

Methods

Root Path: http://<valigner IP>/uberalign/api/

Get Current State (GET)
Path: /uberalign/api/state
Parameter: n/a
Return: UAState
Description: Returns the current state of the vAligner

Get Current Job (GET)
Path: /uberalign/api/job/current
Parameter: n/a
Return: UAJob
Description: Returns the current job of the vAligner. On first boot this is an rather blank object.

Get All Completed(Historical) Jobs (GET)
Path: /uberalign/api/job
Parameter: n/a
Return: UAJob[]
Description: Returns an array with all completed jobs. This list will include all jobs still the vAligner was first deployed. Can be useful for historical purposes.

Get Session (GET)
Path: /uberalign/api/session
Parameter: n/a
Return: UASessionLock
Description: Returns the session lock information. This object contains information on the current lock status of the vAligner.

Lock vAligner (PUT)
Path: /uberalign/api/session/lock
Parameter: UASessionLock
Return: string (“true” | “false”)
Description: Used to lock a vAligner for 30 seconds. A locked vAligner will not accept new jobs from another source. Once a job is started it will not accept a new job until idle again. Use this method to maintain a lock on a vAligner while using it. Must specify a unique GUID that must match the GUID in the job ticket. GUID is a string and should be unique and not change for the duration of the session.

Submit New Job Ticket (PUT)
Path: /uberalign/api/job/new
Parameter: UAJobTicket
Return: string (“true” | “false”)
Description: Used to submit a new job ticket to the vAligner. A job ticket goes into a queue and is picked up by the daemon <15 seconds. The submitted UAJobTicket must have the same GUID as the lock and a lock must exist. Also the vAligner state must either be in Idle(0) or IdleWithError(9). You can use both the Get Current State and Get Current Job to watch for a job starting. The Current Job returns the UAJob object which will have a Name and ID that will match the  UAJobTicket Name and GUID fields you submitted.

 

Object Definitions:

Class UAState
{
string ip;
string errmsg;
string mac;
States state;
}

Class UAJob
{
string id;
string name;
string disk;
UADiskDetails diskdetails;
int offset;
States type;
string errmsg;
string currentstep;
string laststep;
double duration;
UAPartition[] partitions;
string msg;
string timestamp;
string completetimestamp;
}

Class UAPartition
{
string _system;
string _aligned;
string _id;
string _device;
string _offsetdiff;
string _boot;
string _start;
string _blocks;
string _end;
}

Class UADiskDetails
{
string _cyclinders;
string _size;
string _sectorsize;
string _totalsectors;
string _sectoratrack;
string _heads;
string _diskserial;
}

Class UASessionLock
{
string guid;
string ip;
int timestamp;
bool locked;
int secondsleft;
}

Class UAJobTicket
{
States type;
double offset;
string id;
string name;
string guid;
}

Enum States
{
Idle = 0,
AlignNoZero = 1,
AlignWithZero = 2,
Simulate = 3,
CheckOnly = 4,
ZeroOnly = 5,
GrowOnly = 6,
IdleWithError = 9
}

The objects above are simplified from my C# class structure.

This workflow for a job submission would look like this:

  1. Check vAligner state (is idle?)
  2. Lock vAligner (and continue to re-lock <30s until done with it)
  3. Mount Disk using vSphere API/Manually
  4. Create and submit new Job Ticket
  5. Watch vAligner State and Current Job to watch status
  6. After seeing the job completes (state=Idle or IdleWithError) un-mount disk

 

 

I know what some of you are going to say now: “But, Nick – how can I use the API now?”. The answer is provided by the vSpecialist rock-star Clint Kitson in his EMC Community release today. Clint built an awesome example of using Powershell to integrate and control UBERAlign via the REST API. This is so cool because if you already have scripting/code toolsets you use for automation, you can integrate UBERAlign right in. Check out more on this in Clint’ post here.

This is a basic overview and I am sure will probably just lead to more questions Smile. Please feel free to play around with both the CLI and API and post questions/comments below.

Thanks,

.nick

More PS Fun : Fix vCenter Startup w/ PowerShell

More to come on this, but I am working more hands-on lately in my role as a vSpecialist. As part of this I am going to make an effort to post every *shortcut* I come up with along the way. And I love shortcuts…

So here is another PowerShell trick. As many of you have experienced. If you have a local SQL server installation (express or not) of vCenter you will sometimes have vCenter not start correctly on boot. This is do to a race condition between the SQL Server services and vCenter. If SQL doesn’t beat vCenter then vCenter will not start. The correct way to fix this is to setup the Dependencies with the vCenter VPXD service to wait till SQL has done its business first. There are actually several good articles out there on how to do this manually.

I don’t like to anything manually, so here is a PowerShell script to auto-detect and configure vCenter startup for local SQL installs. It is setup to work with SQL server or SQL server express. I think I got all bases covered but you can easily customize as you like (or use on other services). [Read more...]