45992: Acronis Software: Troubleshooting application lockups

    Last update: 金, 2020-10-23 13:51

    Symptoms

    Product may hang under specific circumstances.

    There are 2 types of hanging:

    • Actual hanging. That is a state similar to a crash. That means that software halts and is doing nothing inside.
    • Infinite loop. This means that on the outside the software looks hanging but inside it is doing some actions over and over again.

    Usually when a software hangs its process uses either 100% of CPU (especially when it is infinite loop) or does not use CPU and does not change RAM usage at all (Actual hanging). Still it is difficult to make sure if the software hangs or not. The above means that usually 1 dump is not enough because if that is Infinite Loop – the developers will see that software is working and there is no problem inside. So in order to troubleshoot application lockups you need about 5 dumps collected one after another. A Process Monitor Log will also be useful especially if it actually turns out to be an Infinite Loop.

    Please note that an application lockup does not necessarily mean the software is completely unresponsive. A process can hang (e.g. service_process.exe hangs and a backup is stuck), but you can still browse vaults, create new plans etc.

    Define the hanging process using Windows Task Manager. In some cases multiple processes are involved in an operation. In this case the hang of 1 process can be caused by some operations in another process. In this case you need  to collect memory dumps of all processes involved in the operation. E.g. if a backup to Acronis Storage Node hangs, you need to collect a dump of service_process of the agent as well as StorageServer.exe process on ASN machine.

    Troubleshooting

    Each hanging/lockup should be investigated separately. In order for our development team to troubleshoot a lockup they need process dumps and other logs listed in this article. Select section corresponding with your operating system. With information collected please contact Acronis Customer Central.

    1. Windows

    Step-by-step description of how the problem is reproduced and detailed description of the environment.

    1.1. Process Monitor Log

    Collect a Process Monitor Log while creating a dump of the hanging process, or immediately after. See Collecting Process Monitor Log

    1.2. Dump of the hanging process

    1. Define the hanging process.
    2. Collect dumps as described in Creating Process Dumps with ProcDump using the following parameters:
      procdump -ma -n 5 <process_name>
      where <process_name> is the name of the hanging process.

      If you are collecting a dump of Acronis Storage Node (StorageServer.exe) with deduplication enabled please do not use "-ma". Use:
      procdump StorageServer.exe

    1.3. System Report

    Collect System Report from the machine: see instruction for

    If this fails for any reason, you can use this AcronisInfo Utility.

    1.4. Acronis Backup and Acronis True Image: SnapAPI Logs and VSS logs

    2. Linux

    Step-by-step description of how the problem is reproduced and detailed description of the environment.

    You will be prompted to provide root password.

    2.1. Dump of the hanging process

    1. Install a debugger.
      For example, use this command for installing GDB in Debian-based operating systems:
      apt-get install gdb ​​​​
      Or this command for installing GDB in RedHat-based systems:
      yum install gdb ​​​​​​
    2. Find the process ID of the hanging process with the ps aux command. To filter out Acronis processes, use:
      ps aux | grep Acronis
    3. Execute this command to remove any possible set limit for memory dumps:
      ulimit -c unlimited
    4. Open GDB shell with the following command:
      gdb -pid <process_id>
      where <process_id> is the identificator of the hanging process.
    5. To write a core file, use:
      gcore
    6. Detach the core file with:
      detach
    7. Collect dump from the current working directory.
    8. Repeat steps 5-7 several times to collect more dumps.
    9. Quit GDB with:
      quit
    10. Zip the collected dumps.

    2.2. Strace Output

    Collect strace output as described in Acronis Linux Products: Collecting Strace Output of Acronis Services in Linux

    2.3. System Report

    1. Naviage to /usr/lib/Acronis/BackupAndRecovery.
    2. Execute systeminfo
    3. Collect the "sysinfo" archive from /var/lib/Acronis/sysinfo

    3. Virtual Appliance

    Step-by-step description of how the problem is reproduced and detailed description of the environment.

    3.1. Dump of the hanging process

    Collect dump as described in Using GDB to Create Memory Dumps of Processes Running on Acronis Virtual Appliance or Acronis Linux-based Bootable Media

    Acronis Backup Appliance (All-in-One Appliance) is based on CentOS, follow the Linux part of this article.

    3.2. System Report

    Collect System Report from the machine: see instructions for Acronis Cyber Protect 15 or Acronis Cyber Protect Cloud. If this fails for any reason, you can use this AcronisInfo Utility.

    4. Linux-based bootable media

    Step-by-step description how the problem is reproduced and detailed description of the environment.

    4.1. Dump of the hanging process

    Collect dump as described in Using GDB to Create Memory Dumps of Processes Running on Acronis Virtual Appliance or Acronis Linux-based Bootable Media.

    4.2. System Report

    Collect System Report from the machine: see instructions for Acronis Cyber Protect 15 or Acronis Cyber Protect Cloud. If this fails for any reason, you can use this AcronisInfo Utility.

    5. Mac OS

    5.1. Sample of the hanging process

    To take a sample, open the Activity Monitor application (from the Utilities folder inside your Applications folder).

    After opening Activity Monitor, make sure the CPU tab is selected.

    Select the process from the list. To find it easily, search for it using the search field at the top right. Typically you want to sample mms and service_process.

    Click the Action pop-up menu (looks like a gear) in the toolbar, then select Sample Process.

    A new window will open to display the sample. When Activity Monitor finishes sampling, click Save… to save the result as a text file.

    5.2. System Report

    Collect System Report from the machine: see instructions for Acronis Cyber Protect 15Acronis Cyber Protect Cloud and Acronis True Image. If this fails for any reason, you can use this AcronisInfo Utility.

     

    タグ: