How To Use zodbscan To Scan ZODB for Dangling References

Applies To

• Zenoss 5.x
• Zenoss 4.x

Summary

The zodbscan python script (available as part of the zenoss.toolbox project on Github - https://github.com/zenoss/zenoss.toolbox) scans the zodb database for dangling references. Dangling references are references that no longer point to appropriate content.

If the script discovers a dangling reference it analyzes the pickled object (see note) to determine:

• The path through the object graph
• The name of the attribute that references the missing OID
• Which class it should be

The causes for dangling references can be diverse - issues with ZenDS, issues with ZenPacks, improper zendmd scripting, etc.

Note: The terms POSKeyError, PKE, and Dangling Reference all refer to the same underlying issue - a bad or broken object in zodb.

Note: In Python, pickling implements an algorithm that serializes/de-serializes a Python object structure. Pickling is the process of converting a Python object hierarchy into a byte stream. Unpickling is the inverse operation.

Symptoms of Dangling References

The symptoms of dangling reference issues are diverse because they are related to the location of the error(s) in the Zope database.

When POSKeyErrors or dangling reference errors are present, Zenoss admins may find that:

• they see a yellow error banner in the web interface flagging a POSKeyError
• one or more Zenoss daemons have POSKeyErrors detailed in their logs
• they cannot load certain pages in the web interface
• they cannot view/delete certain devices/device components
• they cannot run certain reports

Using the zodbscan Script

The following section describes the arguments and command syntax for the script.

usage:
zodbscan [-h] [-v] [-v10] [--tmpdir TMPDIR] [-s]

zodbscan -

scans ZODB checking objects for dangling references -
https://support.zenoss.com/hc/en-us/articles/203118175

optional arguments:

Fixing Errors Found

If zodbscan finds errors, run the following tools:

• findposkeyerror
• zenrelationscan

If these tools do not resolve the error(s) displayed by zodbscan, contact Zenoss support.

Example Script Output

The following is example output from the zodbscan script:

[2016-02-17 18:15:27] Initializing zodbscan v2.0.0 (detailed log at /opt/zenoss/log/toolbox/zodbscan.log)
[2016-02-17 18:15:27] Examining 100792 items in the 'zodb' database:
[2016-02-17 18:15:43]  Verified  [==================================================] 100%

[2016-02-17 18:15:43] Execution finished in 0:00:16