Pagination With Horizon And Multiple Tables In The Same View

I was making a custom dashboard for Horizon the other day. In one of my views I was using tabs and in each tab I had a table that supported pagination. There was a problem where when I would go to tab 2 page 2 I would get an error about not being able to load data for tab 1. It was pretty clear to me this was an issue with pagination. I tried various things to work around the bug I had but nothing seemed to work. I took the advice a good friend of mine always gives me when I get stuck “Dont trust the docs, they lie. Go read the code its the only source of truth.” so I did just that. I quickly found the answer to my problem. Inside the tables base class I found the class had a variable for pagination that defaulted to “marker” and in the comments for that particular variable it said “When using multiple tables in a single view this will need to be changed to differentiate between the tables.”. That was simple enough, like always I wish I had gone to read the code sooner. Now to implement the fix and get rid of my bug all I had to do was go to my table classes and add the following bits of code:

class ImageTable(tables.DataTable):
    name = tables.Column("name", verbose_name=_("Name"),
                         link="horizon:admin:images:detail")
    properties = tables.Column(images_md_to_str, verbose_name=_("Metadata"))

    class Meta(object):
        name = "images"
        verbose_name = _("Images")
        table_actions = (MetadataFilterAction, CreateImage, DeleteImage, )
        launch_actions = ()
        if getattr(settings, 'LAUNCH_INSTANCE_LEGACY_ENABLED', False):
            launch_actions = (LaunchImage,) + launch_actions
        if getattr(settings, 'LAUNCH_INSTANCE_NG_ENABLED', True):
            launch_actions = (LaunchImageNG,) + launch_actions
        row_actions = launch_actions + (CreateVolumeFromImage,
                                        EditImage, UpdateMetadataImg,
                                        DeleteImage,)
        # Define the following 2 variables to fix pagination issues
        pagination_param = 'image_marker'
        prev_pagination_param = 'prev_image_marker'

Defining the pagination_param & prev_pagination_param and using a unique name for each class is all it takes to fix that issue. I hope this info comes in handy for someone out there. I spent more time on trying to fix this than Id care to admit 😉

Happy Stacking!

Adding Swift to DevStack

If you find yourself needing to enable Swift on DevStack and you already have DevStack deployed and running and do not want to redo the whole thing I can relate. I found myself in this exact spot tonight. By default DevStack does not enable Swift, but its really trivial to enable it. The tricky part may be once its enabled how do you make it work without just redoing the whole thing?

Getting Swift enabled is pretty simple. First open your local.conf file in your /home/stack/devstack directory using your favorite editor. Next add the following lines to it:
enable_service s-proxy s-object s-container s-account
And save the file. If you havent built your DevStack yet now you can simply proceed with ./stack.sh If you have already built your DevStack and its up and running simply proceed with ./unstack.sh and when that completes then proceed with the ./stack.sh During the install it will prompt you for a hash/passphrase and I just used the same password I used for the initial DevStack install. Once this process finishes you should have Swift added to your DevStack. Happy Stacking!

How to Fix SSL issues with pyvmomi and python 2.7.9

As many of you may or may not know there were some significant changes with regards to SSL in python 2.7.9 which introduce a default verify ssl cert where in the past it was ignored. This creates one hell of a problem when using pyVmomi because 99% of us use the default self signed certs created during the install process of our vCenters or ESXi systems. This problem has been reported a few times now. I answered the question on the mailing list of how to work around the problem, but figured I would add it here as well because there is so much confusion over how to work around the problem until we are able to get upstream pyVmomi patched up.

To work around this issue:

import requests
requests.packages.urllib3.disable_warnings()

import ssl

try:
    _create_unverified_https_context = ssl._create_unverified_context
except AttributeError:
    # Legacy Python that doesn't verify HTTPS certificates by default
    pass
else:
    # Handle target environment that doesn't support HTTPS verification
    ssl._create_default_https_context = _create_unverified_https_context

Add this to your scripts or tools that need to connect. This will make your scripts continue to work like they did on 2.7.8 and older where the default was to ignore the cert and not check it. I hope to see a fix added to upstream pyVmomi soon.

Using pyvmomi to set metadata on a VirtualMachine in vSphere

Do you need to set some kind of Key, Value info up on a VirtualMachine in vCenter that you can access from the API? I thought with 5.5 and the addition of this that there was going to be some kind of actual Matadata fields added to the vSphere API, but it seems that still hasnt happened, as this is incomplete and missing any way to actually use it. The good news is that since 4.0 there has been a way to add key value pairs to the vm.config.extraConfig. The key has to be a String, and the Value can be an object. I have used this to store all kinds of stuff from Strings to JSON and XML payloads and even base64 encoded files. I wrote a sample for the pyVmomi-community-samples project that uses the vm.config.extraConfig to store some key value pairs. You can get it here until it gets accepted by the project and merged into master.

Thunderhead Bug Release — The Python SDK for VMware vCloud Usage Meter

I ran into an issue the other day when non ascii characters were in the Customer name. I tracked down the issue and put out a bug fix release for that today. You can read more about the bug here. Coming up in the next couple of weeks for version 0.5 will be support for vCloud Director. To update to the latest version of thunderhead simply run:

pip install -U thunderhead

And this will update you from 0.4 to 0.4.1 If you have any questions or need any help please feel free to reach out to me.

Ignore SSL warnings in pyVmomi caused by requests

pyVmomi made a switch to using requests a while back where it used urllib2 in the past. One of the side effects of doing that was that when you connect to a vCenter or ESX/i HostSystem that has a self signed ssl cert (which is the default) you get a warning like:

>>> from pyVim.connect import SmartConnect
>>> si = SmartConnect(host='172.16.214.129', user='administrator@vsphere.local', pwd='password')
/Users/errr/venvs/pyVmomi/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py:730: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html (This warning will only appear once by default.)
  InsecureRequestWarning)

How can you disable that message though? Well thats pretty simple. Let me show you how to do it. Since as I mentioned above pyVmomi now uses requests we can disable this at the top of our script like so:

>>> from pyVim.connect import SmartConnect
>>> import requests
>>> requests.packages.urllib3.disable_warnings()
>>> si = SmartConnect(host='172.16.214.129', user='administrator@vsphere.local', pwd='password')
>>> 

As you can see I imported requests, then I set requests.packages.urllib3.disable_warnings() and that disabled the warning message for my script.

Related post: Fix SSL issues with pyVmomi on Python 2.7.9

Using the Managed Object ID to create Objects with pyVmomi

I think this is a bad idea to do for several reasons, but I do understand the needs for doing it. Other pyVmomi developers think having to expose the Managed Object ID is bad and that having to expose it shouldn’t ever have to be done, but sometimes you just cant get away from needing it. One such instance would be if you are interacting with a vCloud Director and need to do something with a VirtualMachine, then you find you need to go to the vCenter where that VirtualMachine lives to do something else to it that is not supported from vCloud Director. vCloud Director will give you the ManagedObject Reference ID so how do you take that ID and start working with the actual object using pyVmomi? That question came up the other day in IRC but for another use case. The person who asked the question solved their own problem and left the solution. Ill show you the steps required to go from a ManagedObject Reference ID to the actual object you can work with. Please keep in mind accessing variables in python that begin with an underscore is bad, and anything bad that happens to your code because you decided to do this is your own fault. With that out of the way lets get to the code.

>>> from pyVim.connect import SmartConnect
>>> from pyVmomi import vim
>>> si = SmartConnect(host='10.12.254.137', user='administrator@vsphere.local', pwd='password')
>>> content = si.RetrieveContent()
>>> children = content.rootFolder.childEntity
>>> for child in children:
...   print child
... 
'vim.Datacenter:datacenter-33'
'vim.Datacenter:datacenter-2'
>>> children[0].name
'1000110'
>>> dc = vim.Datacenter('datacenter-33')
>>> dc._stub = si._stub
>>> dc.name
'1000110'

Lets talk about what just happened here. First I import some requirements and then I connect to the vCenter setting up a ServiceInstance (si). Next to show you what Datacenter objects I currently have in my inventory I get the content, then grab the rootFolder and the childEntity of the rootFolder which by default is the Datacenters. Next I simply loop through them printing the object it self so I can see the ManagedObject Reference ID which shows up as the ‘vim.Datacenter:datacenter-33’ where what I need is the ‘datacener-33’. The way its printed is ‘vmwareObjectType: managedObjectRefId’. Next I show you the value of children[0].name which will print the datacenter name for datacenter-33. As we see it has a value of 1000110. Now we are at the stage where I manually create a datacenter object using vim.Datacenter(‘datacecnter-33’) and assign it to dc. Next I do bad things by taking the _stub which is an instance of SoapStubAdapter from the ServiceInstance and assign it to the dc._stub This allows me to call the .name on the new object and prints its name. Thats all there is to it. Again I think doing this is really bad, and your code can break in all kinds of ways if _stub is ever changed on the back end, and it can with out notice which is why the syntax of _stub is used. Just because you can do something doesnt mean you should. Yay python and its slogan “We’re all consenting adults here”.

Using pyVmomi to set a note on a VirtualMachine

Have you ever noticed the “Notes” section when looking at the VirtualMachine view either from the web client or from the vi client? Its a handy little place to leave various bits of information. I like to use it to put in who built the VirtualMachine and when, and if it had a ticket number that I could tie it to.

vCenter Notes

Right now this VirtualMachine has no note set, and I want to set one using pyVmomi, but how do you do that? Well I made a sample that demonstrates how to do it. Lets walk though how to use this sample. First lets see what the options are:

± |master ✗| → python set_note.py -h
usage: set_note.py [-h] -s HOST [-o PORT] -u USER [-p PASSWORD] -j UUID -m
                   MESSAGE

Standard Arguments for talking to vCenter

optional arguments:
  -h, --help            show this help message and exit
  -s HOST, --host HOST  vSphere service to connect to
  -o PORT, --port PORT  Port to connect on
  -u USER, --user USER  User name to use when connecting to host
  -p PASSWORD, --password PASSWORD
                        Password to use when connecting to host
  -j UUID, --uuid UUID  UUID of the VirtualMachine you want to add a note to.
  -m MESSAGE, --message MESSAGE
                        Message to add to the notes field.

Like the last couple of samples I have covered this one takes the same basic options of Host, User, Password, and Port. This sample adds 2 new options though. The first is the -j or –uuid. What is the UUID though and why do you need it? The UUID is a unique identifier given to the VirtualMachine by the vCenter. This makes it a good choice to use for tracking the asset. If you are unsure of where to get the UUID you can use the vminfo_quick script from my previous post. The next option is the -m or –message. This option set the message that will be added to the Notes section. Lets give it a try and see how it works out.

± |master ✗| → python set_note.py -s 10.12.254.137 -u 'administrator@vsphere.local' -p password -j 421b3c9a-3aef-3662-c476-5b564d045c08 -m "Built by: Michael Rice"
Found: vCloud Director
Done.

Which leaves our vCenter with this:

updated vcenter

If you have any questions about this process please feel free to ask me and Ill try to help. Happy note setting!

pyVmomi on FreeBSD

I was looking around online and found someone has added pyVmomi to the FreeBSD ports. Here is how to use it:

root@fbsd-test01-pyvmomi:~ # pkg search pyvmomi
pyvmomi-5.5.0.2014.1.1

root@fbsd-test01-pyvmomi:~ # pkg install pyvmomi
Updating repository catalogue
New version of pkg detected; it needs to be installed first.
The following 1 packages will be installed:

        Upgrading pkg: 1.2.7_3 -> 1.3.8_2

The installation will require 1 MB more space

2 MB to be downloaded

Proceed with installing packages [y/N]: y
pkg-1.3.8_2.txz                                                                                                  100% 2264KB   2.2MB/s   2.2MB/s   00:01    
Checking integrity... done
[1/1] Upgrading pkg from 1.2.7_3 to 1.3.8_2... done
If you are upgrading from the old package format, first run:

  # pkg2ng
Updating FreeBSD repository catalogue...
pkg: Repo "FreeBSD" upgrade schema 2006 to 2007: Add conflicts and provides
pkg: Repo "FreeBSD" upgrade schema 2007 to 2008: Add FTS index
pkg: Repo "FreeBSD" upgrade schema 2008 to 2009: Optimize indicies
pkg: Repo "FreeBSD" upgrade schema 2009 to 2010: Add legacy digest field
FreeBSD repository is up-to-date.
All repositories are up-to-date.
Updating database digests format: 100%
The following 3 packages will be affected (of 0 checked):

New packages to be INSTALLED:
        pyvmomi: 5.5.0.2014.1.1
        py27-requests: 2.3.0
        py27-six: 1.8.0

The process will require 5 MB more space.
727 KB to be downloaded.

Proceed with this action? [y/N]: y
Fetching pyvmomi-5.5.0.2014.1.1.txz: 100%  264 KB 270.1k/s    00:01
Fetching py27-requests-2.3.0.txz: 100%  448 KB 459.0k/s    00:01
Fetching py27-six-1.8.0.txz: 100%   15 KB  15.3k/s    00:01
Checking integrity... done (0 conflicting)
[1/3] Installing py27-requests-2.3.0: 100%
[2/3] Installing py27-six-1.8.0: 100%
[3/3] Installing pyvmomi-5.5.0.2014.1.1: 100%

This added all the dependencies my system needed, and installed pyVmomi. Pretty simple. Now thats all done and we are ready to start running our pyVmomi code from FreeBSD.

If you arent sure where to start check out one of my other posts! Happy coding!

Using pyVmomi to create a cluster and a datacenter in vCenter

Have you ever wanted to automate creating a Datacenter or a Cluster in your vCenter using pyVmomi but didnt know where to start? I created a sample in the community samples project that does just that. It is pretty simple to use, and I will demonstrate that now.

First lets run the script with the -h argument to see what options it takes:

± |master ✗| → python make_dc_and_cluster.py -h
usage: make_dc_and_cluster.py [-h] -s HOST [-o PORT] -u USER [-p PASSWORD] -n
                              DCNAME -c CNAME

Standard Arguments for talking to vCenter

optional arguments:
  -h, --help            show this help message and exit
  -s HOST, --host HOST  vSphere service to connect to
  -o PORT, --port PORT  Port to connect on
  -u USER, --user USER  User name to use when connecting to host
  -p PASSWORD, --password PASSWORD
                        Password to use when connecting to host
  -n DCNAME, --dcname DCNAME
                        Name of the Datacenter to create.
  -c CNAME, --cname CNAME
                        Name to give the cluster to be created.

As we can see you will need to supply a HOST name (this can also be an IP), a PORT number (the default is 443), a USER to connect as, and a PASSWORD for that user. For this to work properly your user will need the following permissions in vCenter Host.Inventory.CreateCluster and Datacenter.Create. We also need to provide a DCNAME which is the name of the Datacenter we want to create, and finally a name for the new (empty) Cluster. This sample does the absolute minimum to create these objects in your inventory. If you want to customize any settings you will need to modify the code to provide your own ClusterConfigSpecEx. If one is not provided which is how the sample works it uses an empty spec which creates a Cluster using all default settings. Enough talk, lets run this and setup a new Datacenter and Cluster. Here is a screenshot of my inventory before we start:

vCenter ScreenShot

 |master ✗| → python make_dc_and_cluster.py -s 172.16.214.129 -u 'administrator@vsphere.local' -p password -n 111222333 -c '111222333 - Prod Cluster'

And here we have our new Datacenter and Cluster

vCenter after ScreenShot