How to resolve excess disk usage by Hbase-Master

Sometimes Hbase can go into a bad state where ReplicationLogClearner continuously fails to run and the oldWALs directy grows until HBase services are restarted or the instance experiences an outage due to running out of space.

The oldWAL directories are located in /opt/serviced/var/volumes/<tenant_volume>/hbase-master/oldWALs

To reclaim the space caused by this:

1. stop the application either in CC or at the command line with serviced service stop zenoss.resmgr. you can also follow the Staged Startup and Shutdown Procedures

2. change directory on the control center master to /opt/serviced/var/volumes/<tenant_volume>/hbase-master

3. move the oldWALs folder to another directory or delete it fully with rm -rf oldWALs

4. start up the application in CC, by using the command serviced service start zenoss.resmgr, or following the previously linked staged startup procedures.