46360: Using GDB to Create Memory Dumps of Processes Running on Acronis Virtual Appliance or Acronis Linux-based Bootable Media

This Article applies to:

  • Acronis Software

Description

This Article describes how to collect dumps of hanging processes on a Virtual Appliance or a linux-based Bootable media. You will need a machine to remotely connect to the bootable media.

  1. Determine which version of gdb you need. If you are on Virtual Appliance, you need gdb32; if you are on Acronis Bootable Media, then you need gdb32 for 32 bit and gdb64 for 64 bit media. To determine which media you are on, proceed with below steps and once you are in shell, execute command cat /proc/version. If the output contains "x86_64", you need gdb64, otherwise gdb32.
  2. Download gdb32.zip or gdb64.zip attached to this article depending on step 1.
  3. Extract it.
  4. Download WinSCP. Download Putty.
  5. Get into shell (Shell is basically command prompt in Linux OS):
    1. On Acronis Backup and Recovery bootable media or Acronis Backup and Recovery Virtual Appliance press Ctrl+Shift+F2
    2. On vmProtect Virtual Appliance Press  Ctrl+Shift+Alt+Space+F1, then Alt+F2
  6. Start ssh daemon by issuing the command /bin/sshd (If you see any error just click "Enter") 
  7. Check the IP of the bootable media. If there is no IP - your NIC might not be detected. Try a newer version of the bootable media, for more information see Acronis Bootable Media: Troubleshooting NIC Detection
  8. Use WinSCP to connect to the IP that you've checked above. Login is "root". Password for everything except vmProtect Virtual Appliance is "root". For vmProtect Virtual Appliance the password is the same that you use to login into vmProtect web console. File protocol: "SCP". Click "yes" when it asks you about certificate.
  9. Transfer files using WinSCP: file "gdb"to the root folder (The folder name is "/"), lib* files to the /lib/ folder. If transfer fails with "not enough space left on device" then 
    1. Go back to the shell and execute command df -h. If "Available" on /dev/root is 0 or close to it, the reason of process hanging is most likely lack of space.

      2. Please contact Acronis Customer Central. You can shut down the appliance and save it to OVF Template. (File -> Export to OVF in your vSphere console) and provide it to support with reference to this article.
  10. Use Putty to connect to the above IP using the same username and password. Putty is optional in this case. It just makes all your work in Shell more convenient especially if you are connected using several remote sessions. Use the same IP, credentials as in step 7. Click "Yes" when asked about certificate.
  11. Make gdb executable and accessible issuing command in Shell chmod 0777 /gdb 
  12. Go into root folder issuing command cd /
  13. Check the PID (Process ID) of the hanging process: 
    • Bootable media: usually the process in question is "/bin/product". The command to check would be ps | grep /bin/product
    • Virtual Appliance: if the GUI and agent itself hangs the process is "/bin/product". However if the backup/replication/cleanup/recovery hangs in Acronis Backup & Recovery Virtual Appliance, the process name is "/bin/service_process". For vmProtect processes see Acronis vmProtect 9: Processes and Services. The command would be "ps | grep service_process". If you see more that 1 service process running you will have to create a dump of each.

       Note that on this picture there is only one "/bin/service_process" that you need to dump and the PID of it is 1018. Grep /bin/service_process is basically the process generated by your command. /bin/sh -c /bin/service_process is a shell instance that initiates service process. There is no need to dump it.
  14. Switch to GDB shell with the following command:
    /gdb -pid <process_id>
    where <process_id> is the identificator of the hanging process. It will start loading symbols and will eventually show you (gdb).
  15. To write a core file, issue:
    gcore <FILE_NAME_OF_THE_DUMP>
  16. Once the core file is written, issue:
    detach
  17. Repeat steps 14-16 several times to get more dumps. Make sure you specify different  FILE_NAME_OF_THE_DUMP to prevent overwriting previous dumps.
  18. Quit GDB by issuing quit

  19. Dumps will be generated in the root folder "/". They will have the names that you gave them in step 14.
  20. Use WinSCP to download them to your machine.
  21. Zip the dumps and contact Acronis Customer Central with reference to this article.

 

Tags: 

You are reporting a typo in the following text:
Simply click the "Send typo report" button to complete the report. You can also include a comment.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
3 + 1 =
Solve this simple math problem and enter the result. E.g. for 1+3, enter 4.