The rsync command hangs when "-vv" or "-vvv" flags are used

Solution Unverified - Updated -

Environment

  • Red Hat Enterprise Linux (RHEL) 7, 8
    • rsync-3.1.2-4.el7 or later
    • rsync-3.1.3-9.el8 or later

Issue

  • The rsync command hangs after some time using -vv or -vvv

    $ rsync -avv /src/ /dst/
[...]
^C
rsync error: received SIGINT, SIGTERM, or SIGHUP (code 20) at rsync.c(701) [sender=...]
rsync: [generator] write error: Broken pipe (32)

  • The same rsync command with less verbosity works fine

Resolution

The main rsync developer doesn't recommend using -vv and above.
Nonetheless, if -vv or above is necessary for monitoring purposes, add one of the following options to the actual command.

Solution 1 - Disable inc-recursive mode

Add --no-inc-recursive flag to the current options:

$ rsync -avv --no-inc-recursive /src/ /dst/

Solution 2 - Modify how messages on stderr are handled

Add --msgs2stderr and -M--msgs2stderr flag to the current options:

$ rsync -avv --msgs2stderr -M--msgs2stderr /src/ /dst/

Root Cause

There are known issues with using higher verbosity than -v.
See Issue #159 - "rsync -avv /src/ /dst/" hangs constantly at the same place for details.

Diagnostic Steps

  • rsync process is hanging.

  • Check the hanging processes.

    # ps -ef | grep rsync
root     13624  9812  6 17:35 pts/0    00:00:04 rsync -avvv /git1 /git2
root     13625 13624  0 17:35 pts/0    00:00:00 rsync -avvv /git1 /git2
root     13626 13625  7 17:35 pts/0    00:00:04 rsync -avvv /git1 /git2

  • Check the stacks for these process if they are waiting on select.

    # cat /proc/{13624,13625,13626}/stack
[<ffffffff9f634d95>] poll_schedule_timeout+0x55/0xb0
[<ffffffff9f635711>] do_select+0x6d1/0x7c0
[<ffffffff9f6359b6>] core_sys_select+0x1b6/0x2e0
[<ffffffff9f635b9a>] SyS_select+0xba/0x110
[<ffffffff9fb2579b>] system_call_fastpath+0x22/0x27
[<ffffffffffffffff>] 0xffffffffffffffff

[<ffffffff9f634d95>] poll_schedule_timeout+0x55/0xb0
[<ffffffff9f635711>] do_select+0x6d1/0x7c0
[<ffffffff9f6359b6>] core_sys_select+0x1b6/0x2e0
[<ffffffff9f635b9a>] SyS_select+0xba/0x110
[<ffffffff9fb2579b>] system_call_fastpath+0x22/0x27
[<ffffffffffffffff>] 0xffffffffffffffff

[<ffffffff9f634d95>] poll_schedule_timeout+0x55/0xb0
[<ffffffff9f635711>] do_select+0x6d1/0x7c0
[<ffffffff9f6359b6>] core_sys_select+0x1b6/0x2e0
[<ffffffff9f635b9a>] SyS_select+0xba/0x110
[<ffffffff9fb2579b>] system_call_fastpath+0x22/0x27
[<ffffffffffffffff>] 0xffffffffffffffff

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.

Comments