65210: Acronis Cyber Infrastructure: Check for updates fails with "Cannot open Packages database in /var/lib/rpm"

use Google Translate

Applies to: 

Last update: 15-09-2022

Symptoms

You get this error message on attempt to check cluster updates or start updates via WebCP -> Settings -> Updates:

Failed to start update check
Yum repository is inaccessible on <node_name>

Yum repository is accessible from the affected node, however, the log /var/log/vstorage-ui-agent/software-updates.log contains the following messages:

DEBUG 2020-06-24 09:24:10,824 r-8bb31a50aafa4c26 agent/business/models/runner.py:31:CmdRunner._internal_checked_execute command "['/usr/bin/check-hci-updates']" failed (c: 1, o: , e: error: rpmdb: BDB0113 Thread/process 22088/140233762031424 failed: BDB1507 Thread died in Berkeley DB library
error: db5 error(-30973) from dbenv->failchk: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery
error: cannot open Packages index using db5 -  (-30973)
error: cannot open Packages database in /var/lib/rpm
error: rpmdb: BDB0113 Thread/process 22088/140233762031424 failed: BDB1507 Thread died in Berkeley DB library
error: db5 error(-30973) from dbenv->failchk: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery
error: cannot open Packages database in /var/lib/rpm
Traceback (most recent call last):
...
yum.Errors.YumBaseError: Error: rpmdb open failed

The same error is being observed upon checking packages with rpm command manually:

[root@node01 ~]# rpm -qa | head
error: rpmdb: BDB0113 Thread/process 22088/140233762031424 failed: BDB1507 Thread died in Berkeley DB library
error: db5 error(-30973) from dbenv->failchk: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery
error: cannot open Packages index using db5 -  (-30973)
error: cannot open Packages database in /var/lib/rpm
error: rpmdb: BDB0113 Thread/process 22088/140233762031424 failed: BDB1507 Thread died in Berkeley DB library
error: db5 error(-30973) from dbenv->failchk: BDB0087 DB_RUNRECOVERY: Fatal error, run database recovery
error: cannot open Packages database in /var/lib/rpm

Cause

RPM database on the node is corrupted. There are a number of factors that can lead to the RPM database corruption, such as incomplete previous transactions, installation of certain third-party software, removing specific packages.

Solution

Fix the issue by rebuilding RPM DB:

[root@node01 ~]# mkdir /tmp/bak
[root@node01 ~]# cp -r /var/lib/rpm/ /tmp/bak/
[root@node01 ~]# rm -f /var/lib/rpm/__db*
[root@node01 ~]# rpmdb --rebuilddb

Verify that no errors are being observed upon checking packages with rpm command:

[root@node01 ~]# rpm -qa | head

Re-try check for new updates from WebCP or via CLI:

[root@node01 ~]# vinfra software-updates check-for-updates
+---------+--------------------------------------+
| Field   | Value                                |
+---------+--------------------------------------+
| task_id | 69b78b39-f1f0-42d0-86b3-3b146fe8d822 |
+---------+--------------------------------------+
[root@node01 ~]# vinfra task show 69b78b39-f1f0-42d0-86b3-3b146fe8d822
+---------+------------------------------------------------------------------------+
| Field   | Value                                                                  |
+---------+------------------------------------------------------------------------+
| details |                                                                        |
| name    | backend.business.models.software_updates.tasks.CheckSoftwareUpdateTask |
| result  |                                                                        |
| state   | success                                                                |
| task_id | 69b78b39-f1f0-42d0-86b3-3b146fe8d822                                   |
+---------+------------------------------------------------------------------------+

More information

If the issue still persists, contact Acronis support for assistance.

 

Tags: