MokaFive Automation

Posted: April 18, 2011 in Scripting, Virtualization

Automating tasks is a common goal for many projects, especially those targeting the implementation of a new system or the operational improvement of an existing system. The goals are typically to create a repeatable sequence of steps that can be executed on a scheduled basis with no human intervention to minimize chances of neglecting to run the sequence or introducing errors into the execution.

With most software packages, automation relies on components created by the vendor in advance (that hopefully fit the needs of the implementer) or a documented API that is sometime complex and cryptic and must be learned and understood before being used.

MokaFive takes a web services based approach to this need that makes it easier to create the automation and reduces the need for a highly skilled developer to do the work and greatly accelerates the learning curve for the task.

RESTFul Web Services

MokaFive uses RESTFul Web Services not only for customization of its enterprise product but also for the vendor supplied administration console. The large majority of actions taken within the administration console use REST calls that can be examined and then modified to execute calls needed by the automation sequence.

REST (Representational State Transfer) is a style of architecture used for web pages and services that’s characterized by a lack of state storage on the server. The result is that every client request must contain all the information needed to process the request. This facet makes examining REST calls and recreating them fairly straight forward.

There’s a lot of information about REST on the web and a good Wikipedia entry that can serve as a decent starting point for research: http://en.wikipedia.org/wiki/Representational_State_Transfer

Example

The rest of this post will go through an example of how to automate a task for MokaFive using REST. The example generates a script that will create IP ranges within MokaFive to direct clients to their nearest image store for image files. The script uses Active Directory sites and subnets as the authoritative data for the IP ranges.

The process involves the following high level steps:

  1. Obtain the necessary tools and software to work with REST
  2. Create the necessary REST requests and save them
  3. Create a script to collect sites and services data from AD
  4. Put the data and REST requests together to configure the IP ranges.

Note: as with most scripted solutions, there are multiple ways to solve this problem, this is the method I have selected and created.

Required tools

My example uses a client computer running Windows 7 with Firefox as a browser used to access the MokaFive administrator console. The code is created in PowerShell and takes advantage of the PowerShell 2.0 capabilities that are included with Windows 7.

Note: I highly recommend using the PowerShell ISE that’s included with Windows 7 if you go the PowerShell route as it makes the development experience much easier and quicker.

In addition to the core OS and the Firefox browser, the following tools were used:

  1. Firebug – this Firefox extension allows detailed examination of the interaction between the browser and web server, including REST calls. The extension can be downloaded here.
  2. rest-client – a Java based application that can be used to configure REST requests and save them for future use. There are many REST clients out there, this particular one was selected because it has a version with a full interface which is useful for creating and testing the requests, and a command line version which is very easy to include in scripts. You can find this tool here.

Creating REST requests

Once the tools are installed, the next step is to open Firefox and log into the MokaFive console in order to perform the actions that need to be scripted. For the subnet exercise, I created a new IP range using the UI which performs the three actions that I need to capture for the script: get list of IP ranges, get list of image stores (performed to populate the image store pull down) and creating the IP range. I also edited an IP range to be able to update an existing range with the script.

Now press F12 which will open the Firebug window, by default below the browsing window, make sure the console tab is selected and you can scroll through all the actions that were observed by Firebug. Finding the specific actions you need is easier if you have Firebug open when you perform the actions but is not too hard to do later since the URL path is a pretty good indicator of the action.

The next step is to reproduce each action with the REST client and save it for automated execution. Firebug allows you to right click an action URL and copy the location. After running the rest-client UI, you can paste the location into the URL field. Next, select the ‘Auth’ tab and enter the login information – in my case, Basic authentication with username and password. This example is done with http since the code will run on the server itself. For production environments, especially when running across a network, you will probably also want to configure SSL using the SSL tab.

That’s all that’s needed for a GET request. You can test the request by running it and making sure you get the expected results and then save the request. You can also use a more advanced authentication scheme by leveraging API calls directly to create an authentication cookie, but since one of my primary goals is simplicity and the script code will all reside on the MokaFive server, I didn’t go that route.

For the POST/PUT requests which include adding an IP range or editing an existing IP range, modifications to the ‘Body’ tab are required. The first modification is to set the content-type and charset to the correct setting (which will match the setting viewed in Firebug under the action/Headers\Request Headers section. The required content-type is ‘application/xml; charset=UTF-8’, which can be configured in rest-client’s ‘Body’ tab by click the leftmost icon (the one with the pencil on it) and selecting the correct value.

The second modification needed is the XML containing the data to be sent to the server. This XML can be found in the Firebug action under the ‘Put’ tab in the source section. I typically cut and paste from there into Notepad to remove any formatting and then into the rest-client ‘Body’ tab. You can also type directly into the field to avoid any hidden characters coming from Firefox/Firebug.

When going through the POST request for a new IP range and a PUT request for an update to an existing IP range, I found that the requests are almost identical except for the method and the URL. As a result, I only save and use a single request, in my case the POST version, and modify those fields on the fly as needed.

Once the requests are complete and tested, save the request files to be used by the automation script.

Collecting Active Directory site data

This is a task that PowerShell is able to handle easily. The following code sample returns and processes the required data:

$myForest =
[System.DirectoryServices.ActiveDirectory.Forest]::GetCurrentForest()
Foreach ($site in $myForest.Sites)
 {
 Foreach ($subnet in $site.Subnets)
  {
  }
 }

As the code demonstrates, the sites are contained in a collection within the forest object and subnets are a collection within each site object. The subnet itself is a string with a CIDR representation of the subnet – for example, 10.15.20.0/24.

One interesting challenge that will be discussed later is created by the fact that while AD adheres to the CIDR standard, MokaFive allows the network address part of the CIDR to be a host address – so (AD) 10.15.20.0/24 and (MokaFive) 10.15.20.1/24 can refer to the same network. Luckily, MokaFive stores the network address in a different field which I will use instead.

Putting it all together

Taking all of the tools and information presented above and using it to construct the required automated script is not very complicated but a little time consuming. While I don’t plan to include my full code here, I will go through each section of the script to demonstrate the structure and highlight potential issues.

1. Collect initial data

In addition to setting global variables and opening the connection to the AD forest, this section collects the subnet and image store data from MokaFive. The subnet and image store data will be used later in the script for quite a few purposes so I decided to collect it at the start.

This task highlights two interesting aspects of the rest-client and XML structure: first, the CLI version of rest-client allows you to specify the target directory but not the response file name. The response file name is the same as the request file name with an RCS extension (the request file uses an RCQ extension). A simple rename solves the problem of needing the file treated as an XML file.

The second issue is that the response file XML is not very useful as all the tags contain data related to the rest transaction rather than the needed data. The relevant data is all contained in a single tag called ‘body’. In order to process the ‘body’ data as XML, I extract it and create a new XML file containing only the contents of the ‘body’ tag. I could have probably done this in memory without the file, but writing the XML out makes debugging, testing and operational validation much easier. This process is done for both the subnets and image stores.

2. Primary loop – image store

The top level structure of the script loops through each image store in the MokaFive configuration. Since I am expecting that there will be AD sites and subnets that don’t participate in the MokaFive architecture, I decided to process each image store and make sure that any subnet in the site for the image store is configured as an IP range with the correct image store assigned to it.

The script needs a way to match the image stores to AD sites in order to configure subnets correctly. For this specific script, I assume that the image store server name (and therefore the URL property) will start with the site code. There are many ways to accomplish this goal but using a specific image store naming standard is one of the simplest approaches.

Once the site code has been identified, the site object is retrieved and used for the inner loop.

3. Inner loop – subnets

Processing each subnet of the identified site is the job of the inner loop. The code searches for the AD subnet name in the MokaFive IP range list. Due to the CIDR issue identified in the last topic, instead of using the CIDR field in the MokaFive subnet XML, the NetworkID field is used and concatenated with the network portion of the CIDR.

If the subnet is found, the image store data is compared, if it is correct, no action is taken. If it is incorrect a record update is initiated. If the subnet is not found a record addition is initiated.

4. Subnet record manipulation

Both the subnet update and subnet creation use the same request file since the differences can easily be changed with code. Prior to running the request, the ‘URL’, ‘method’ and ‘body’ tags are modified to create either a subnet update or a subnet creation. In the case of a subnet creation, the subnet mask (e.g. 255.255.255.0) must be determined based on the last two digits of the CIDR and used in the URL.

5. Clean up

…and that’s all there is. Removing any temporary files and deleting objects is the last section. I do leave a log file and echo some information to the screen throughout the process to simplify auditing and troubleshooting.

Last words

First, if you’re still reading, thanks for putting up with such a long post. If you have any other questions, please post in the comments or contact me directly at therdpfiles@gmail(d0t)com.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s