using pyvmomi to get a list of all virtual machines — fast

Something that often comes up from various people new to using the vSphere API is how to get information very quickly about all the VirtualMachines in the inventory. There is a sample that comes with pyvmomi called getallvms.py which is an obvious place to start to get this info. When its run on an inventory that only has 30 VirtualMachines it seems pretty fast. It only takes about .5 seconds to complete. Try it on a larger inventory like something with 500+ VirtualMachines and it really starts to slow down going from .5 seconds all the way up to over 6 seconds. This number just keeps growing the larger the inventory gets. Once the inventory reaches over 1000 VirtualMachines it can take over 10 seconds for this info to be returned. In other words this solution just doesnt scale, but its often the only way new comers know. The good news is that VMware provides other ways to get this info. The bad news is that its not an obvious solution, and its kind of complicated to use, but thats what I am here for 🙂

Where I work we have over 45,000 vSphere powered Virtual Machines, and its my job as a Sr. Developer there to make sure our code is stream lined, efficient, and scales the way we do. This is why I use property collectors when I need to work with objects from the vSphere inventory. To help new users I provided a sample I call vminfo_quick which as its name implies get info about a VirtualMachine, quickly. To test this lets run the getallvms.py from above on a vCenter with 576 VirtualMachines and time it.


time python getallvms.py -s 10.12.254.119 -u 'administrator@vsphere.local' -p password

real 0m6.300s
user 0m2.476s
sys 0m0.123s

Almost 6 1/2 seconds. Thats not too bad right? Now lets run the vminfo_quick sample I provided against that same vCenter and see how it does. I included a counter and a timer in this sample so we dont have to run time.


python vminfo_quick.py -s 10.12.254.119 -u 'administrator@vsphere.local' -p password

Found 576 VirtualMachines.
Completion time: 0.368282 seconds.

As you can see using a property collector vastly improves performance. I have tested this on an inventory with 1500 VirtualMachines and it still finishes in just under 1 second. I plan to cover details around what the property collector is and how it works in future posts. Stay tuned!

pyvmomi now available in Fedora 19 20 and 21

Back in 2005 I got into making RPMs for Fedora, and by 2006 I became a sponsored packager. It was a hobby I really enjoyed, but back in 2009 or 2010 I just got too busy with work and had to pass the baton. About a month and a half ago I was looking for issues on the pyvmomi project that I could help close when I found this one asking for help making an RPM for Fedora. I got excited all over again about making RPMs so I made one and headed down the path to become sponsored again to package for Fedora. I created this bug report and became re-sponsored, and after a lengthy process I am happy to report that pyvmomi is now available in Fedora 19, 20, and 21. It is still in the testing phases for EPEL 6 and 7 but in another week or so should be available there as well. This puts us one step closer to being able to port the current OpenStack driver for vSphere to pyvmomi!

vCloud Usage Meter

Recently I was tasked with building a tool that would create customers and rules in vCloud Usage Meter to help enhance our monthly usage reports. The way this tool needed to work was to connect to all of our vCenter Servers, pull some data from them then connect to vCloud Usage Meter and create a customer, then make a rule that would tie their inventory to some managed object. Normally I do most of my code work in Groovy using the Grails framework, but for this project I decided to use python. The main reason for that was we needed to run this tool once a month, and they wanted it hands off. I knew our Ops Team would want to create a crontab for it on one of our Linux servers so I wanted to make something very portable. Python was an obvious choice for this based off of that requirement.

When I got started I went looking for some library that would make it easier for me to interact with vCloud Usage Meter but I didnt find anything. vCloud Usage Meter provides a REST based API so I decided to just write my own library and called it thunderhead. I thought it might be useful to others so I went ahead and open sourced the work via Apache-2.0. At the time of this writing it is not 100% feature complete, but does provide a lot of functionality, and I plan to continue development on it to get it 100% complete. I also always appreciate pull requests so if you need something I havent implemented please feel free to send me a pull request. For examples of how to use the library see the tests that are included. To install thunderhead simply

pip install thunderhead