Zenoss Engineering has recently identified an issue with Analytics. The
dim_date table contains incorrect settings that will cause performance ETL batches to fail after 1 January 2020. This issue affects all users of Zenoss Analytics and the following steps should be taken immediately.
To determine if you are currently experiencing this issue, you can check the contents of the
zenoss_analytics.out file on your Analytics server. Beginning after 1 January 2020, you may see messages like the following:
WARN pool-105-thread-4 org.hibernate.util.JDBCExceptionReporter - SQL Error: 1064, SQLState: 42000
ERROR pool-105-thread-4 org.hibernate.util.JDBCExceptionReporter - You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'NULL' at line 1
This failure will also generate ERROR files in the Analytics temp directory.
1. Download the attached
ANA-365-fix.tar.gz file to your Analytics server and place it in the
2. Extract the contents of
3.Switch to the zenoss user (
su - zenoss) and change to the newly created
4. Execute the fix_perf_batch_issues.sh script with
This script runs the following (included) scripts in order, and cleans up any ERROR files in the Analytics temp directory:
set_analytics_env.sh - Discovers and sets Analytics-specific environment variables required by the other scripts.
step1.sql - Drops and recreates the
dim_date table of the Analytics reporting database.
step2.sql - Adds additional columns to the
dim_date table and populates it.
retry_all_failed_perf_batches.sh - Attempts to retry perf batches that failed due to ANA-365.
4. Due to the number and size of missed batches, the final step may take some time to complete. The status of the batches can be tracked in the Analytics UI to determine when they have finished.
the "retry_all_failed_perf_batches.sh" script has typo in line3.
Copy the "set_analytics_env.sh" script into "/opt/zenoss_analytics/bin/" and then manually trigger the "retry_all_failed_perf_batches.sh" script again manually. It should confirm how many batches it has marked for retry.