Replace vijava with yavijava in your JVM based projects

Do you currently use vijava? Have you noticed how the forums are kind of dead (meaning lots of questions, but no one from the project answering those questions), and there have not been any more community releases since 2013 or so ending with 5.5 beta? I noticed these things, and so I did something about it. I forked the project and started maintaing it. I have done a lot of work on the project. I have added logging using log4j. I have added the ability to RESET VCENTER ALARMS FROM RED TO GREEN. I have fixed several bugs including a bug where every single exception was being wrapped in a RemoteException causing you to have to check the exception.getCause() to find the underlying exception instead of being able to catch normal vCenter exceptions. I have refactored the code allowing you the user to provide your own HTTP clients should you need to, and even added a great deal of tests to the code (this is an ongoing work in process). I also converted the project to use gradle. This makes building the project a snap.. But wait theres more!! I fixed a bug in the InventoryNavigator which made the 5.5 release not work properly with vSphere 6.0.. speaking of 6.0 I have been hard at work adding support for vSphere 6.0 and Im happy to announce the beta release is done and available right now! With the help of people from the community work is being done RIGHT NOW to add SPBM support to yavijava as well!! One of the best parts about all of this is that I have maintained compatibility with vijava so switching to yavijava is painless for you the user!

So.. How do you switch? Switching is simple. If you have a project using maven you can simply update your pom, or gradle, or what ever to point at the yavijava repo. As of this blog post the current stable release is 5.5.11. If you are just grabbing the jar file to include in your project some other way thats fine too. Please be aware that I did introduce new dependencies, so if you are manually downloading jar files (you should consider stopping that and moving to gradle its super easy to do) be sure to check the build file for the latest requirements.

If you do run into any issues please report those issues on the GitHub issue tracker. When you submit an issue please consider reading over this basic guide I created. If you do not have a bug but just have a question, you are welcome and encouraged to open an issue and ask your question there, or join the web chat and ask there, just be sure if you cant wait around for an answer to open a GitHub issue.

I really look forward to growing the community and bringing it new life. Please help me spread the word about yavijava, and welcome to the community!

yavijava 6.0 beta release ready for vsphere 6.0

I’ve been hard at work adding support for vSphere 6.0 to yavijava, the vijava fork. Im happy to announce that I am done and ready for a beta release. I am not going to push this to public maven right now. For those of you who want to test this release out please download the release from GitHub. Next follow these simple instructions to build a jar.

  • Unzip the 6.0.01b1.zip file
  • Enter the yavijava-6.0.01b1 folder
  • On Linux or Mac use ./gradlew build
  • On Windows use gradlew.bat build

A jar file will be built for you and it will be located in the build/libs directory.

If you run into any issues please please please report them to me so I can get them fixed. Simply go to https://github.com/yavijava/yavijava/issues and open a new issue.
Enjoy!

Which version of YAVIJAVA should I use

If you didn’t already know I created a fork of vijava called yavijava. It is quickly becoming a pretty popular alternative to using vijava due to the features I have implemented like fixing the InventoryNavigator in the current production release found in public maven central. That fix makes 5.5.10 and newer releases have a functional InventoryNavigator on vSphere 6.0. Ive also added other features like logging using log4j, and the ability to reset Alarms from red to green. Ive also been hard at work adding full support for vSphere 6. With all of these things going on I also decided the project was getting so popular it was time to move it to its own GitHub Org.

A lot of these things have caused a great deal of confusion for end users of the library. Questions like:

  • Which version of yavijava should I be using?
  • Im using vSphere 6. Shouldnt I be using the vsphere_6 branch of yavijava?!?
  • Where do I report issues Im having with yavijava?
  • Is there a place for chat support?

I plan to address these questions in this post. Ill start by answering the first question of Which version of yavijava should I use? The answer to that depends. Ill start by asking you a very important question:

  • Are you looking for the latest stable and supported version for use in production code?

If your answer to that question is YES! then you need to be using the latest version available in public maven.

Now for question number 2 Im using vSphere 6. Shouldnt I be using the vsphere_6 branch of yavijava?!? Let me ask you another question to help answer that for you:

  • Are you comfortable testing an ALPHA version of a library that is in no way ready for production use?
  • Are you a developer who is looking to help get vSphere 6 fully supported by submitting code to yavijava that implements new properties, DataObjects, or ManagedObjects so the new features added to vSphere 6 will work in yavijava?
  • Are you a tester who is wanting to help get vSphere 6 fully supported in yavijava by submitting detailed bug reports about things in vsphere_6 branch that have been implemented but are not working?

Unless you answered YES! to some or all of those questions DO NOT USE vsphere_6 BRANCH! Support for vSphere 6 in the vsphere_6 branch is NOT DONE. There is a lot of work left to do and pretty basic stuff will fail until the work is done.

Now for the next question: Where do I report issues Im having with yavijava? This one is pretty easy.

Finally: Is there a place for chat support?

  • There sure is! Find us on Gitter Chat here

Responses may be slow. Just join, ask your question and wait around. If you cant wait around consider opening an issue on GitHub instead.

VIJAVA InventoryNavigator not working correctly on new versions of vSphere

Are you testing new versions of vSphere with VIJAVA and finding problems with the InventoryNavigator? Missing properties, and missing inventory are common complaints when using VIJAVA and new versions of vSphere. Never fear though! YAVIJAVA is here to save the day!!

The InventoryNavigator is not a default part of the VI SOAP API and was added by the creator of VIJAVA. In that class he uses the PropertyCollectorUtil to create complex TraversalSpecs that can be used to traverse an inventory very quickly. VIJAVA has been around for a very long time and supports many versions of vSphere. To do that there are a couple of helper methods in the PropertyCollectorUtil that can be used to make traversal specs. One of them is meant for use on older vSphere systems (pre 4.0), and one of them is meant for use on version 4 and newer vSphere systems. In the InventoryNavigator there was a check being done against the string that contained the API version to try to decide which version of the TraversalSpec creator should be used. The problem with new versions of vSphere and any version of VIJAVA (as of March 8th 2015) is that VIJAVA has a conditional where it was looking specifically for version 4 or 5 in the version string to load the newer TraversalSpec method. I reworked that logic in the YAVIJAVA code base for now so if it finds any version >=4 it will load the newer TraversalSpec else it loads the old one. Give it a try on those newer vSphere versions and report any issues you may have in the GitHub issues. This rework was added to 5.5.10-DEVELOPMENT I plan to release 5.5.10 to public maven in April. To use the development version just check out the code from github and use gradle to build the jar with the following commands:

git clone https://github.com/yavijava/yavijava && cd yavijava && gradle build

This will build the jars for you. Take a look in build/libs/ for all the jar files.

yavijava now available in maven central

I began working on a fork of VIJAVA several months ago. I have reached a point where I feel it is ready for its first mainstream release. I have now pushed it to central maven. I have been using this in production now for a few months, and I am no longer giving it the beta mark and I am calling it production ready for vSphere 5.5 The versioning I am using is . so the current production release is 5.5.07 where 5.5 is the vSphere version and 07 is the release number. Please use and report any bugs to the Github issue tracker. YAVIJAVA is basically a drop in replacement for VIJAVA with the exception that new dependencies have been added, but this will NOT require a code change to use. One of the biggest benefits this release gains you from using VIJAVA is logging. Logging has been missing from VIJAVA and at times can really make troubleshooting issues nearly impossible. Now there is TONS of logging. Just configure your log4j for com.vmware Another benefit is the ability to use your own custom http clients. I have provided 1 using the Apache Http Commons library as an example, but the original http client is the default.

Reset vCenter Alarms from red to green using yavijava the vijava fork

I recently added a patch to YAVIJAVA that allows you to reset an alarm status. William Lam has posted in the past about how to hack the perl sdk to add this functionality, so I thought it would be a good idea to add this functionality to YAVIJAVA. I have manually implemented this in Groovy and in python for pyVmomi so adding it to the primary library I use seemed logical. Im still doing a bit more re-work to YAVIJAVA but if you want to use that now you can check out a copy from GitHub and use the gradle build script to build a copy for your self.

Using this new functionality in YAVIJAVA from Groovy (which is how I do it) would look something like this:

ServiceInstance si = new ServiceInstance(new URL(""),"","", true)
Folder rootFolder = si.getRootFolder()
HostSystem hostSystem = rootFolder.getChildEntity()[0].hostFolder.childEntity[0]
AlarmState[] alarmState = hostSystem.triggeredAlarmState
Alarm foo = alarmState[0].alarm as Alarm
AlarmManager am = new AlarmManager(si.getServerConnection(), foo.getMOR())
am.setAlarmStatus(foo, hostSystem, "green")

I havent tested this yet so YMMV. Before I do the next official release of YAVIJAVA I will have tested it out and include a working sample.

Simple Tomcat Hosting using Tomcat 7, Java 7, and httpd with mod_proxy_ajp

Ive been spending a lot of time in the last year and a half dealing with Apache Tomcat. The DevOps team I am on focuses on Grails for almost all development needs. We use Tomcat 6 and 7 to host a variety of applications we have written. As such I have learned a lot about Tomcat, and Im going to use this opportunity to share some of the things I have learned with you.

The server setup

I like using CentOS on my personal stuff, so for this example Ill be using CentOS 6.3 and using my Rackspace Cloud Server. Im just using the default image provided by Rackspace. From the cloud server control panel I name my server tomcat-dev1. Next I selected CentOS 6.3 for the image and 512M of RAM for the server size. Finally in the networks section I leave it defaulted to a public and private adapter, and then press the create server button to begin building my new cloud server. Keep in mind this is just a development server for me so I dont need much RAM, for a production system you will need to calculate your needs accordingly and make your server as large as you need it. Once my server is done building I will log on and install:

  • httpd
  • jdk 7 from Oracle
  • Tomcat 7
  • git (optional, but I like to keep my config files in version control)
  • jsvc (this is bundled with tomcat)

First steps

Lets begin by logging into our new cloud server as root. The first thing I do to a new cloud server is change the password. Once that is done we can begin by updating the system.
yum -y update
This is done because the base image could be several months old so we want to get the latest updates from the vendor before we begin. Once that finishes reboot if you need, if you do not know if you need a reboot or not just do it anyway to be sure.

Gathering the goods

Some of what we will be installing is either not provided by yum, or is older and we want flashy and new. We need to fetch the jdk; I will download the x64 rpm to my local desktop then upload it to my cloud server using winscp. Next we need Tomcat, and jsvc. Since these two things do not require me agreeing to some terms I will download them directly from the server.
wget http://mirror.symnds.com/software/Apache/tomcat/tomcat-7/v7.0.34/bin/apache-tomcat-7.0.34.tar.gz
Now on my server I should have the jdk rpm, and the Tomcat 7, and jsvc packages.(jsvc is bundled with tomcat)

Installing the jdk, Tomcat, and jsvc

We need to install the jdk first. To do that lets install the rpm:
rpm -Uvh jdk-7u10-linux-x64.rpm
If you see output like this:

Error: Could not open input file: /usr/java/jdk1.7.0_10/jre/lib/rt.pack
jsse.jar…
Error: Could not open input file: /usr/java/jdk1.7.0_10/jre/lib/jsse.pack
charsets.jar…
Error: Could not open input file: /usr/java/jdk1.7.0_10/jre/lib/charsets.pack
tools.jar…
Error: Could not open input file: /usr/java/jdk1.7.0_10/lib/tools.pack
localedata.jar…
Error: Could not open input file: /usr/java/jdk1.7.0_10/jre/lib/ext/localedata.pack

It is safe to ignore. I didnt dig into why this output happened but it did not seem to affect my install. Now that Java is installed we need to add the java install to the alternatives system, and make it so that the JAVA_HOME environment variable is set for all our users when they log in. Lets do that by creating a file in /etc/profile.d called java.sh Using your favorite text editor create the file /etc/profile.d/java.sh and enter the following lines into the file and then save it:

export JAVA_HOME=/usr/java/default
export JRE_HOME=/usr/java/default/jre
export JAVA_OPTS=””
export JAVA_CLASSPATH=/usr/share/java/*

PATH=$PATH:$JAVA_HOME/bin

Now the file needs to be set executable so issue the following command:

chmod +x /etc/profile.d/java.sh

Now to add java to the alternatives system

alternatives --install /usr/bin/java java /usr/java/latest/jre/bin/java 20000
alternatives --install /usr/bin/jar jar /usr/java/latest/bin/jar 20000
alternatives --install /usr/bin/javac javac /usr/java/latest/bin/javac 20000

Lets test that this works by typing:

echo $JAVA_HOME

Right now we should get no output. Now try logging off our server, and logging back in. Once you do that type the command again. We should now see:

/usr/java/default

Now we need to unpack Tomcat, and the jsvc daemon.

tar xzvf apache-tomcat-7.0.34.tar.gz -C /opt/

This will unpack Tomcat into the /opt directory into a folder named apache-tomcat-7.0.34 Once that is done unpack the commons-daemon source code. That code is bundled with tomcat:

cd /opt/apache-tomcat-7.0.34/bin/
tar xzvf commons-daemon-native.tar.gz -C /usr/local/src/

The source code for the jsvc daemon is now in /usr/local/src/commons-daemon-1.0.10-native-src and needs to be built. Before we can build it we still need to install a couple of things on our server. According to the docs we need a compiler, and make. In an effort to make it simple I will be using yum and the groupinstall option to install “Development tools” This will provide git, make, gcc, cpp, and many other tools.

yum -y groupinstall "Development tools"

Once that is complete we have what we need to build jsvc, so lets do that now. Since we are using CentOS we will want to:

cd /usr/local/src/commons-daemon-1.0.10-native-src/unix

From here we can issue the configure command:

./configure --with-java=/usr/java/default

This should render some output, the last thing it says if you were successful is this:

*** All done ***
Now you can issue “make”

Lets do that now.

make

If you got no errors you should now have a binary file in the same directory as you called jsvc. Since this Makefile does not have a target for install the next natural step of make install would fail. You have to manually place the binary somewhere on your system. I like to just put it with the tomcat binaries since thats the only thing I use jsvc for, so thats what Ill do now:

cp jsvc /opt/apache-tomcat-7.0.34/bin/

You should now have Java 7, Tomcat 7, and jsvc 1.0.10 installed on your system. We are getting close to being done. Only a few more steps left!! 🙂

Doing some Tomcat configuration, and making Tomcat start at boot

Now that we have all these things installed we need to do some configuration, and also make it so Tomcat will start when the system boots up. Lets begin with making tomcat start at boot. Im going to provide you with the init script I wrote and use on my systems. You can get the latest version from here. Download it and save the file in /etc/init.d as tomcat

wget https://raw.github.com/michaelrice/tomcat_files/master/init.d/tomcat -O /etc/init.d/tomcat

This file needs to be executable so lets issue the following command:

chmod +x /etc/init.d/tomcat

Now we need to make sure tomcat will start at boot:

chkconfig --add tomcat && chkconfig tomcat on

This will add tomcat to the chkconfig system, and enable it for start up on boot. Lets take a look at what this file does for us. On line 4 we see this:

# chkconfig: – 85 15

This is needed for chkconfig so our command above would work. For more information see man chkconfig. Scroll down in the file until you see:

TOMCAT_USER=tomcat

This is where the user that tomcat will run as is defined. By default it wants to run as the user named tomcat. We have not created that user yet, but we will shortly. The next two things to notice are right below the TOMCAT_USER line.

CATALINA_PID=/var/run/tomcat/tomcat.pid
TOMCAT_LOCK_FILE=/var/lock/subsys/tomcat/tomcat

These locations on the file system are not there by default so we need to add them, and make sure our tomcat user has access to them, but lets keep looking through this file for now and we will fix all the things at once. Lets scroll down to lines 20 and 21

[ -f /etc/profile.d/tomcat.sh ] && . /etc/profile.d/tomcat.sh
[ -f /etc/profile.d/java.sh ] && . /etc/profile.d/java.sh

This is checking to see if these files exist and if so source them, this will set up our environment variables for us. The tomcat file is missing still, but we will add it soon. I think one of the last things to note here in this file is the JSVC_BIN variable. If you put jsvc in some other location than where I put it then you need to adjust this to your location. Finally on line 62 we have the command we use to start tomcat. You should not need to adjust this, but if you need to thats where to do it. Now that we know what this file will do for us, lets fix all the things we found while going through it.

Finializing the Tomcat bits

The first problem we found above was our script is looking for a “tomcat” user and we dont have one yet, so lets add one:

useradd -r tomcat -m

This added a new system user named tomcat, and created a group also named tomcat. A home dir was created in /home/tomcat The next problem we found was that a couple of files need to be created, and our tomcat user needs access to them. Lets make it happen:

mkdir -p /var/lock/subsys/tomcat/
mkdir -p /var/run/tomcat/
chown -R tomcat. /var/run/tomcat
chown -R tomcat. /var/lock/subsys/tomcat

Finally the tomcat.sh file needs to be created and make executable. Using your favorite text editor create the file /etc/profile.d/tomcat.sh and add the following to it, then save it:

export CATALINA_HOME=/opt/tomcat
export CATALINA_TMPDIR=/opt/tomcat
export CATALINA_OPTS=”-XX:MaxPermSize=256m”

Now we need to make it executable:

chmod +x /etc/profile.d/tomcat.sh

You may have noticed that the location of CATALINA_HOME and CATALINA_TMPDIR dont exist on our server, good catch. I use symlinks to this. Lets get them created.

cd /opt
ln -s /opt/apache-tomcat-7.0.34/ /opt/tomcat

I do this so if I update tomcat I only have to update this symlink and do not have to edit my init script, or my profile.d script. Now all we have left for tomcat is to make sure we have the server.xml file created correctly, and if we want to allow access to the manager we need to make a tomcat-users.xml file as well. I have provided the server.xml and the tomcat-users.xml file I used here. Those files need to be saved in /opt/tomcat/conf/ as server.xml and tomcat-users.xml Once you have done that you should be able to start tomcat, so lets try that now.

service tomcat start

Lets verify its running:

ss -nap

We should see some output like the following:

[root@tomcat-demo unix]# ss -nap
State Recv-Q Send-Q Local Address:Port Peer Address:Port
LISTEN 0 100 ::ffff:127.0.0.1:8009 :::* users:((“jsvc”,20103,39))
LISTEN 0 100 ::ffff:127.0.0.1:8080 :::* users:((“jsvc”,20103,38))

This shows us that jsvc has a process listening on port 8009 and 8080, on the local loop back interface only, just like we setup in our config file. This is wonderful now we are done with tomcat for now. Lets move on to httpd and setting up the ajp.

Apache httpd and mod_proxy_ajp

We need to install httpd now.

yum -y install httpd && chkconfig httpd on

We do not need to do anything to get mod_proxy_ajp it is part of the base httpd install as of 2.2. Now that our web server is installed lets configure our first vhost, and serve up some tomcat content. Using your favorite text editor open the httpd.conf file located at /etc/httpd/conf/httpd.conf The vhost section is at the bottom of the file by default. In my case I added the following:

NameVirtualHost *:80
<VirtualHost *:80>
ServerName tomcat-demo.mrice.me
ProxyRequests Off
<Proxy *>
Order deny,allow
Allow from all
</Proxy>
ProxyPass / ajp://localhost:8009/
</VirtualHost>

Next I need to open port 80 on my firewall, since the default firewall will be blocking traffic on port 80. Using my text editor I will edit /etc/sysconfig/iptables so it looks like so:

# Firewall configuration written by system-config-firewall
# Manual customization of this file is not recommended.
*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
-A INPUT -m state –state ESTABLISHED,RELATED -j ACCEPT
-A INPUT -p icmp -j ACCEPT
-A INPUT -i lo -j ACCEPT
-A INPUT -i eth0 -m tcp -p tcp –dport 80 -j ACCEPT
-A INPUT -m state –state NEW -m tcp -p tcp –dport 22 -j ACCEPT
-A INPUT -j REJECT –reject-with icmp-host-prohibited
-A FORWARD -j REJECT –reject-with icmp-host-prohibited
COMMIT

Next I need to load this new rule set like so:

iptables-restore < /etc/sysconfig/iptables

Now I am done, and Im ready to fire up httpd and visit my site.

service httpd start

You can now visit your site in a browser and see the default Tomcat landing page, as well as add a /manager and log in using "tomcat" as the user, and "s3cret" as your password. In the coming weeks I will be adding more info about how to do vhosting using this method, and eventually covering how to do this all in a few mins using chef