How To Troubleshoot RabbitMQ

Applies To

• Zenoss 4.x

Summary

Zenoss Resource Manager 4.x uses RabbitMQ for messaging between several application daemons. Resource Manager administrators may therefore wish to be familiar with listing RabbitMQ queues, displaying queue contents, clearing queues, and completing other activities related to troubleshooting RabbitMQ.

Procedure

The following tasks are helpful for displaying the RabbitMQ configuration and are run as the root user on the host system.

To list Zenoss exchanges (where messages are published):

# rabbitmqctl -p /zenoss list_exchanges

To list Zenoss queues (where messages are consumed):

# rabbitmqctl -p /zenoss list_queues

The following tasks are useful for troubleshooting queues and are run as the zenoss user.

To show all messages in the specified queue (the zep raw events queue is used as an example):

$ zenqdump zenoss.queues.zep.rawevents

To empty a queue, a command can be run to display and acknowledge all messages in the queue.  In the sample command below, the zep raw events queue is used as an example. This command will not empty the queue as quickly as possible (another command will be detailed below for emptying a queue more quickly):

$ zenqdump -A zenoss.queues.zep.rawevents

The following tasks are useful for troubleshooting queues and are run as the root user.

To clear queues quickly all queues may be reset by recreating the /zenoss vhost. After stopping Resource Manager, run the following:

# rabbitmqctl delete_vhost /zenoss

# rabbitmqctl add_vhost /zenoss

# rabbitmqctl set_permissions -p /zenoss zenoss '.*' '.*' '.*'

The following actions will restore RabbitMQ service after the host server is renamed. Renaming a server after initial creation can cause problems because RabbitMQ uses the server hostname in file and directory names. Additionally, changing the Zenoss server's hostname can cause RabbitMQ to stop working because RabbitMQ ties its NODENAME to the hostname when its configuration is created.  As a result, the following commands should be run whenever the server hostname is changed, after stopping Zenoss:

# service rabbitmq-server stop

# mv /var/lib/rabbitmq /var/lib/rabbitmq.old

# mkdir /var/lib/rabbitmq

# chown rabbitmq:rabbitmq /var/lib/rabbitmq

# service rabbitmq-server start

# rabbitmqctl add_user zenoss zenoss

# rabbitmqctl add_vhost /zenoss

# rabbitmqctl set_permissions -p /zenoss zenoss '.*' '.*' '.*'