Getting data out of JMX and into Cacti
JMX, if you don't know, is one of the things that makes Java really great. It allows you to write your Java app and instrument it as if instrumentation was free. It's not free, of course, but it's so low cost, that in Cassandra, there's a JMX "bean" that counts every single read and write to a column family. And you can have this - in production. However, it's generally somewhat difficult to get data from JMX to another programming language. Depending on how the remote side (the JMX agent) is configured, it could be expecting to exchange raw Java objects. Not great for interoperability. In our particular case, we wanted to get data from Cassandra and into Cacti. We tried several different generic JMX clients with varying levels of success, however, none of them were fast.
Most of them could only list the attributes for one bean at a time, and required spawning an entire JVM for each request. Because we knew we were going to be collecting nearly every JMX statistic in a short amount of time, this wouldn't cut it.
Our solution was to create a new client application specifically for doing this kind of monitoring. Unimaginatively, we called it 'JMXClient', and it has a few distinguishing features:
- It's fast. 993 attributes can be retrieved in 1 second. Other solutions we saw did 10 attributes in 1.5 or even 2 seconds.
- It has no dependencies other than a Java 1.5 or 1.6 install.
Some solutions required external Jar files, or even ant! - It outputs results in PHP's native serialization format.
All other solutions assumed a human would be consuming the data.
For us, all three of these items were a big win. Future plans include adding or replacing the PHP serialization output with JSON or some other cross-platform/langauge format.
Archives
- May 2013
- March 2013
- February 2013
- January 2013
- December 2012
- November 2012
- September 2012
- August 2012
- July 2012
- June 2012
- May 2012
- April 2012
- March 2012
- February 2012
- January 2012
- December 2011
- November 2011
- October 2011
- September 2011
- August 2011
- July 2011
- June 2011
- May 2011
- April 2011
- March 2011
- February 2011
- January 2011
- December 2010
- November 2010
- October 2010
- September 2010
- November 2009
- March 2008
- November 2007
- October 2007


Comments
And another option: a JMX to JSON bridge: http://www.jolokia.org/
Jerry just sent me the link to this posting.
I wanted to let you know that I've done a somewhat similar project after recently finding a big need for a solid tool to query jmx and write the data out efficiently. My tool allows you to create Queries based on easy json configuration which hit jmx for the properties that you need and OutputWriter's that can write the data however you want. One of the cool writers I created is one that shells out directly to rrdtool. So, now we have a direct JMX->rrd query transformer. Its fully multithreaded at multiple levels so it should easily support a massive server farm.
Take a look... http://jmxtrans.googlecode.com/
Reply