I suppose most of the readers of this blog know me for my Vim plugins. But unsurprisingly, my job title is neither "Vim Plugin Developer" nor "Vim Evangelist", and no, I do not make a living writing Vim plugins.
For the last few years I have been a part of the team responsible for the research and operation of non-relational database systems and we're currently focusing on HBase, which is the most used one in our shop.
During the time we've developed a wide variety of in-house tools that greatly help running HBase clusters smoothly in production, and very recently we decided to start open-sourcing the ones that are not tightly bound to our intricate running environment.
As a part of the plan, a little monitoring tool I wrote with such a lackluster name, "hbase-region-inspector", is now on GitHub.
hbase-region-inspector starts a web server that shows how the regions are distributed across region servers like follows:
So why do we need this? A "region" is a unit of data partitioning in HBase, and it's crucial for the health of a HBase cluster that we have a balanced distribution of the regions across the region servers. We also have to make sure that the partitioning scheme we employed - in other words, rowkey design - is working as expected and the cluster has no apparent "hotspot regions" which can severely limit the performance of the entire cluster. Each region server provides a web page that shows some basic statistics of the regions it serves, but there was no way to see the whole picture which is especially problematic if you have many number of region servers.
hbase-region-inspector might remind you of Hannibal if you're already familiar with it. But unlike the predecessor, this new tool not only shows the size of each region, but also visualizes the cumulative number of requests, reads and writes per second, number of storefiles, size of memstore, and data block locality (on HBase 1.0 or above), which are all essential information. As a bonus, you can even move regions around with the simple drag-and-drop interface.
The project is written in Clojure and React and distributed as executable binaries built with the handy lein-bin plugin. So given that you have JRE set up on your machine, you can simply start it like follows:
./hbase-region-inspector ZOOKEEPER_QUORUM HTTP_PORT
And most likely that's all you need to know about it. Now you can grasp the status of your HBase cluster at a glance.
If you work with HBase, please try it and let me know what you think about it. I'm absolutely confident that you'll like it. Any suggestions are deeply appreciated!