vSphere 6.0 REST API: A History Lesson

I’m glad to see how VMware products are becoming more and more automation-focused these days. NSX has always had rich REST API capabilities, which I can’t complain about. And vSphere is now starting to catch up. vSphere 6.5 was the first release where REST API started getting much more attention. See these official blog posts for example:

• What’s New with Developer and Automation Interfaces for vSphere 6.5?
• Getting Started with the vSphere Automation SDK for REST
• Introducing Developer and Automation Interfaces for vSphere 6.7

But not many people know that vSphere 6.5 wasn’t the first release where REST API became available. Check this forum thread on VMTN “Does vCenter 6.0 support RESTFUL api?”:

I think its only supported for 6.5 as below blogs has a customer asked the same question and reply is no..

It’s not entirely true, even though I know why the OP got a “No” answer. Let me explain.

vSphere 6.0 REST API

VMware started to make first steps towards REST API starting from 6.0 release. If you have a legacy vSphere 6.0 environment you can play with, you can easily test that by opening the following URL:

https://vcenter/rest/com/vmware/vapi/metadata/cli/command

You will get a long list of commands available in 6.0 release:

It may look impressive, but if you look closely you will quickly notice that they are all Content Library or Tagging related. Quote from the referenced blog post:

VMware vCenter has received some new extensions to its REST based API. In vSphere 6.0, this API set provides the ability to manage the Content Library and Tagging but now also includes the ability to manage and configure the vCenter Server Appliance (VCSA) based functionality and basic VM management.

That is right, in vSphere 6.0 REST API is very limited, you won’t get inventory data, backup or update API. All you can do is manage Content Library and Tagging, which, frankly, is not very practical.

Making REST API Calls

If Content Library and Tagging use cases are applicable to you or you are just feeling adventurous this is an example of how you can make a call to vSphere 6.0 REST API via Postman.

All calls are POST-based and action (get, list, create, etc.) is specified as a parameter, so pay close attention to request format.

First you will need to generate authentication token by making a POST call to https://vcenter/rest/com/vmware/cis/session, using “Basic Auth” for Authorization and you will get a token in response:

Then change Authorization to “No auth” and specify the token in “vmware-api-session-id” header in your next call. In this example I’m getting a list of all content libraries (you will obviously get an empty response if you haven’t actually created one):

Some commands require a body, to determine the body format use the following POST call to https://vcenter/rest/com/vmware/vapi/metadata/cli/command?~action=get, with the following body in JSON format:

{
	"identity": {
        "name": "get",
        "path": "com.vmware.content.library"
	}
}

Where “path” is the operation and “name” is the action from the https://vcenter/rest/com/vmware/vapi/metadata/cli/command call above.

If you’re looking for more detailed information, I found this blog post by Mitch Tulloch very useful:

• Understanding the VMware REST API Interface

Conclusion

There you have it. vSphere 6.0 does support REST API, it’s just not very useful, that’s why no one talks about it.

This blog post won’t help you if you are stuck in a stone age and need to manage vSphere 6.0 via REST API, but it at least gives you a definitive answer of whether REST API is supported in vSphere 6.0 and what you can do with it.

If you do find yourself in such situation, I recommend to fall back on PowerCLI, if possible.

Advertisement

Quick Start With Lifecycle Manager REST APIs

Just a few years ago coming across an infrastructure product (software or hardware) that supports REST APIs was a rare thing. Today it’s the opposite. Buying, say, a storage array from a major vendor, that doesn’t support some sort of an API can be seen as a potential drawback. It’s now gotten to a point where certain operations can only be done via API and are not available in the GUI. So basic programming skills become more and more important.

I have come across such situation with vRealize Suite Lifecycle Manager (vRSLCM or just LCM) product from VMware. If you have a request that got stuck, the only way to cancel it (at least at the time of writing) is to use LCM’s REST APIs. It can’t be done from the GUI.

While I was tackling this issue, I noticed that there aren’t many articles on how to make REST calls to LCM on the Internet, so I though I’d use this opportunity to show how to do it.

Authentication

First challenge you have to deal with is authentication. LCM doesn’t support basic authentication, like other products, for instance NSX. You need a token.

This is how you can get a token in Postman:

• Submit a POST call
• Use the following URL – https://lcm-hostname/lcm/api/v1/login
• Enter the following body:

{
	"username":"admin@localhost",
	"password":"vmware"
}

This is what it will look like in Postman:

When you click send you should get a token in response:

Making REST Calls

Now you need to specify the token as one of the headers, with “x-xenon-auth-token” as key and the token itself as value:

From here, you are ready to make actual REST API calls. Coming back to our example, we can go to LCM GUI and copy the ID of the stuck request from the browser window:

And then make a DELETE call with empty body to cancel the request:

As a result, traces of the request will be completely deleted from LCM.

Note: The only catch here, that you have to remove “v1” version of the API from the URL. Or it will not work.

Swagger UI

LCM supports Swagger, which lets you run REST API calls straight from the browser. So if you want to feel yourself a hacker, open the https://lcm-hostname/api URL and you can get the token and make requests by simply using the “Try It Out” button, specifying required parameters and hitting “Execute”.

Extracting vRealize Operations Data Using REST API

Scripting today is an important skill if you’re a part of IT operations team. It is common to use PowerShell or any other scripting language of your choice to automate repetitive tasks and be efficient in what you do. Another use case for scripting and automation, which is often missed, is the fact that they let you do more. Public APIs offered by many software and hardware solutions let you manipulate their data and call functions in the way you need, without being bound by the workflows provided in GUI.

Recently I was asked to extract data from vRealize Operations Manager that was not available in GUI or a report in the format I needed. At first it looked like a non-trivial task as it required scripting and using REST APIs to pull the data. But after some research it turned out to be much easier than I thought.

Using Python this can be done in a few lines of code using existing Python libraries that do most of the work for you. The goal of this blog post is to show that scripting does not have to be hard and using the right tools for the right job you can get things done in a matter of minutes, not hours or days.

Scenario

To demonstrate an example of using vRealize Operations Manager REST APIs we will retrieve the list of vROps adapters, which vROps uses to pull information from many hardware and software solutions it supports, such as Nimble Storage or Microsoft SQL Server.

vROps APIs are obviously much more powerful than that and you can use the same approach to pull other information such as: active and inactive alerts, performance statistics, recommendations. Full vROps API documentation can be found at https://your-vrops-hostname/suite-api/.

Install Python and Libraries

We will be using two Python libraries: “Requests” to make REST calls and “ElementTree” for XML parsing. ElementTree comes with Python, so we will need to install the Requests package only.

I already made a post here on how to install Python interpreter and Python libraries, so we will dive right into vROps APIs.

Retrieve the List of vROps Adapters

To get the list of all installed vROps adapters we need to make a GET REST call using the “get” method from Requests library:

import requests
from requests.auth import HTTPBasicAuth

akUrl = 'https://vrops/suite-api/api/adapterkinds'
ak = requests.get(akUrl, auth=HTTPBasicAuth('user', 'pass'))

In this code snippet using the “import” command we specify that we are using Requests library, as well as its implementation of basic HTTP authentication. Then we request the list of vROps adapters using the “get” method from Request library, and save the XML response into the “ak” variable. Add “verify=False” to the list of the get call parameters if you struggle with SSL certificate issues.

As a result you will get the full list of vROps adapters in the format similar to the following. So how do we navigate that? Using ElementTree XML library.

Parsing XML Response Sequentially

vRealize Operations Manager returns REST API responses in XML format. ElementTree lets you parse these XML responses to find the data you need, which you can output in a human-readable format, such as CSV and then import into an Excel spreadsheet.

Parsing XML tree requires traversing from top to bottom. You start from the root element:

import xml.etree.ElementTree as ET

akRoot = ET.fromstring(ak.content)

Then you can continue by iterating through child elements using nested loops:

for adapter in akRoot:
  print adapter.tag, adapter.attrib['key']
    for adapterProperty in adapter:
      print adapterProperty.name, adapterProperty.text

Childs of <ops:adapter-kinds> are <ops:adapter-kind> elements. Childs of <ops:adapter-kind> elements are <ops:name>, <ops:adapterKindType>, <ops:describeVersion> and <ops:resourceKinds>. So the output of the above code will be:

adapter-kind CITRIXNETSCALER_ADAPTER
name Citrix NetScaler Adapter
adapterKindType GENERAL
describeVersion 1
resourceKinds citrix_netscaler_adapter_instance
resourceKinds appliance
…

As you could’ve already noticed, all XML elements have tags and can additionally have attributes and associated text. From above example:

• Tags: adapter-kind, name, adapterKindType
• Attribute: key
• Text: Citrix NetScaler Adapter, GENERAL, 1

Finding Interesting Elements

Typically you are looking for specific information and don’t need to traverse the whole XML tree. So instead of walking through the tree sequentially, you can iterate trough interesting elements using the “iterfind” method. For instance if we are looking only for adapter names, the code would look as the following:

ns = {'vrops': 'http://webservice.vmware.com/vRealizeOpsMgr/1.0/'}
for akItem in akRoot.iterfind('vrops:adapter-kind', ns):
  akNameItem = akItem.find('vrops:name', ns)
  print akNameItem.text

All elements in REST API responses are usually prefixed with a namespace. To avoid using the long XML element names, such as http://webservice.vmware.com/vRealizeOpsMgr/1.0/adapter-kind, ElementTree methods support using namespaces, that can be then passed as a variable, as the “ns” variable in this code snippet.

Resulting output will be similar to:

Citrix NetScaler Adapter
Container
Dell EMC PowerEdge
Dell Storage Adapter
EP Ops Adapter
F5 BIG-IP Adapter
HP Servers Adapter

Additional Information

I intentionally tried to keep this post short to give you all information required to start using Python to parse REST API responses in XML format.

I have written two scripts that are more practical and shared them on my GitHub page here:

• vrops_object_types_1.0.py – extracts adapters, object types and number of objects. Script gives you an idea of what is actually being monitored in vROps, by providing the number of objects you have in your vROps instance for each adapter and object type.
• vrops_alert_definitions_1.0.py – extracts adapters, object types, alert names, criticality and impact. As opposed to the first script, this script provides the list of alerts for each adapter and object type, which is helpful to identify potential alerts that can be triggered in vROps.

Feel free to download these scripts from GitHub and play with them or adapt them according to your needs.

Helpful Links

• vRealize Operations Manager API Programming Guide
• The ElementTree XML API
• Requests: HTTP for Humans

Dell Compellent Enterprise Manager: SQL Server Setup

Dell Compellent Enterprise Manager is a separate piece of software, which comes with every Compellent storage array deployment and allows you to monitor, manage, and analyze one or multiple arrays from a centralized management console.

Probably the most valuable feature of Enterprise Manager for an average user is its historical performance statistics. From the Storage Center GUI you can see only real-time data. Enterprise Manager is capable of keeping statistics for up to a year. And can obviously do a multitude of other things, such as assist with capacity planning, allow you to configure replication between production and DR sites, generate reports or simply serve as a single pain of glass interface to all of your Compellent storage arrays.

Enterprise Manager installation does not include an embedded database. If you want to deploy EM database on an existing Microsoft SQL database or install a new dedicated Microsoft SQL Express instance, you need to do it manually. The following guide describes how to install Enterprise manager with Microsoft SQL Server 2012 Express.

SQL Server Authentication

During installation, Enterprise Manager will ask you for authentication credentials to connect to the SQL database. In SQL Server you can use either the Windows Authentication Mode or Mixed Mode. Mixed Mode allows both Windows authentication and SQL Server authentication via local SQL Server accounts.

For a typical SQL Server setup, Microsoft does not recommend enabling SQL Server authentication and especially with the default “sa” account, as it’s seen as insecure. So if you have a centralized Microsoft SQL Server in your network you’ll most likely be using Microsoft Authentication. But for a dedicated SQL Server Express database it’s fine to enable Mixed Mode and use SQL Server authentication.

Make sure to enable Mixed Mode during SQL Server Express installation and enter a password for the “sa” account.

SQL Server Network Configuration

Enterprise Manager uses TCP/IP to connect to the SQL database over port 1433. TCP/IP is not enabled by default in SQL Server Express, you will need to enable it manually.

Use SQL Server Configuration Manager, which is installed with the database, and browse to SQL Server Network Configuration > Protocols for SQLEXPRESS > TCP/IP. Enable TCP/IP on the Protocols tab and assign port 1433 to TCP Port field in IPAll section.

Make sure to restart the SQL server to apply the settings and you should now be able to connect Enterprise Manager to the database.

If you get stuck, refer to Dell Compellent Enterprise Manager Installation Guide and specifically the section “Prepare a Microsoft SQL Server Database”. This guide is available in Dell Compellent Knowledge Center.

Fix NetApp AutoSupport

I come across this issue too often. You need to fetch some information for the customer from the My AutoSupport web-site and can’t because the last AutoSupport message is from half a year ago.

Check AutoSupport State

When you list the AutoSupport history on the target system you see something similar to this:

# autosupport history show

Mail Server Configuration

If AutoSupport is configured to use SMTP as in this case, then the first place to check is obviously the mail server. The most common cause of the issue is blocked relay.

There are two things you need to make sure are configured: NetApp controllers management IPs are whitelisted on the mail server and authentication is disabled.

To set this up on a Exchange server go to Exchange Management Console > Server Configuration > Hub Transport, select a Receive Connector (or create it if you don’t have one for whitelisting already), go to properties and add NetApp IPs on the network tab.

Then make sure to enable Externally Secured authentication type on the Authentication tab.

Confirm AutoSupport is Working

To confirm that the issue is fixed send an AutoSupport message either from OnCommand System Manager or right from the console and make sure that status shows “sent-successfull”.

# options autosupport.doit Test

# autosupport history show

 

VNX LDAP Integration: AD Nested Groups

Have you ever stumbled upon AD authentication issues on VNX, even though it all looked configured properly? LDAP integration has always been a PITA on storage arrays and blade chassis as usually there is no way to troubleshoot what the actual error is.

If VNX cannot lookup the user or group that you’re trying to authenticate against in AD, you’ll see just this. Now go figure why it’s getting upset about it. Even though you can clearly see the group configured in “Role Mapping” and there doesn’t seem to be any typos.

Common problem is Nested Groups. By default VNX only checks if your account is under the specified AD group and doesn’t traverse the hierarchy. So for example, if your account is under the group called IT_Admins in AD, IT_Admins is added to Domain Admins and Domain Admins is in “Role Mapping” – it’s not gonna work.

To make it work change “Nested Group Level” to something appropriate for you and this’d resolve the issue and make your life happier.

PowerShell script for disk space

I’ve written simple PowerShell script for monitoring of free space on one of our important volumes. As well as inserting it as plaint text below, for the sake of convenience I uploaded the script as a text file on FileDen file hosting service. Here is the link to the file.

$path = "F:\script\dspace.txt"
Clear-Content $path
$server = "SERVERNAME"
$recps = "qwe@contoso.com", "wer@contoso.com", "ert@contoso.com"
$sender = "alerts@contoso.com"
$user = "alerts"
$passw = "123456"
$thold = 3
$drives = Get-WmiObject -ComputerName $server Win32_LogicalDisk | `
Where-Object {$_.DriveType -eq 3}
$i = 0

echo "This message is to inform you that server $server is low on disk space.
Please carry out corrective actions. Find details below. `n" >> $path

foreach($drive in $drives) {
	$size_gb = $drive.size / 1GB
	$size_gb_fmt = "{0:N2}" -f $size_gb
	$free_gb = $drive.freespace / 1GB
	$free_gb_fmt = "{0:N2}" -f $free_gb
	$ID = $drive.DeviceID
	$pct = $free_gb / $size_gb * 100
	$pct_fmt = "{0:N0}" -f $pct

	if (($ID -eq "F:") -and ($free_gb -lt $thold)) {
		echo "Server Name: $server" >> $path
		echo "Drive Letter: $ID" >> $path
		echo "Drive Size: $size_gb_fmt GB" >> $path
		echo "Free Space: $free_gb_fmt GB" >> $path
		echo "Percent Free: $pct_fmt%" >> $path
		$i++
        }
}

if ($i -gt 0) {
	$smtpServer = "mail.contoso.com"
	$smtp = New-Object Net.Mail.SmtpClient($smtpServer)
	$smtp.Credentials = New-Object System.Net.NetworkCredential($user, $passw)
	$msg = new-object Net.Mail.MailMessage
	$msg.From = $sender
	foreach ($recp in $recps) {
		$msg.To.Add($recp)
	}
	$subject = "Disk space on $server is under $thold GB"
	$msg.Subject = $subject
	foreach ($line in Get-Content $path) {
		$body += "$line `n"
	}
	$msg.Body = $body
	$smtp.Send($msg)
}

To run this script from scheduler just write another simple .bat file with following line inside:

powershell -command “& ‘.\dspace.ps1′”