'ls' fails in sftp session using wildcards

Solution Unverified - Updated -

Environment

  • Red Hat Enterprise Linux 7.0
  • openssh-6.4p1-8

Issue

  • sftp is failing using wild-cards and many files
  • When running ls with wild-cards in a sftp session against a folder containing more than 128 files
    following error is shown although at least one file matching search criteria is present:
sftp>
sftp> ls /var/tmp/test/*
Can't ls: "/var/tmp/test/*" not found
sftp> ls /var/tmp/test/*.txt
Can't ls: "/var/tmp/test/*.txt" not found
sftp> 
sftp> ls /var/tmp/test/
/var/tmp/test/test-1.txt                                                                                             
/var/tmp/test/test-10.txt                                                                                                      
/var/tmp/test/test-100.txt                                                                                   
/var/tmp/test/test-1000.txt                                   
[...]
sftp>

Resolution

Possible solutions:

  • Extend limit (works, but it is still only until someone else tries to ls some larger directory)
  • Try to generate better error message/code (sftp protocol problem) [1]

Also related to current Fedora and upstream releases of openSSH.

[1] http://winscp.net/eng/docs/sftp_codes

Root Cause

This is hitting the hard limit GLOB_LIMIT_STAT which is set to 128 by default.
If a wild-card is used, ls needs to go and stat all the resulting files because star expansion is done before calling ls on remote shell, which is less efficient than the another mentioned method (ls /var/tmp/987../ ):

 /RHEL-7/openssh/6.4p1/8.el7/openssh-6.4p1/openbsd-compat/
644 glob.c  133 #define GLOB_LIMIT_STAT 128
645 limitp->glim_stat++ >= GLOB_LIMIT_STAT) {

This is an upstream feature. The result of this call is "not found" which is not self-explaining, but sftp protocol doesn't have any better error code for exceeding this limit so it ends like this.

Diagnostic Steps

[root@rhel7ga ~]# x="test"
[root@rhel7ga ~]# dir=/var/tmp/$x
[root@rhel7ga ~]# mkdir -p $dir
[root@rhel7ga ~]# for n in `seq 2000`
> do
> touch $dir/$x-$n.txt
> done
[root@rhel7ga ~]# ls -la $dir/* |wc -l 
2000
[root@rhel7ga ~]# sftp localhost
root@localhost's password: 
Connected to localhost.
sftp> 
sftp> ls /var/tmp/test/*
Can't ls: "/var/tmp/test/*" not found
sftp> ls /var/tmp/test/*.txt
Can't ls: "/var/tmp/test/*.txt" not found
sftp> 
sftp> ls /var/tmp/test/
/var/tmp/test/test-1.txt                                                                                                                         
/var/tmp/test/test-10.txt                                                                                                                        
/var/tmp/test/test-100.txt                                                                                                                       
/var/tmp/test/test-1000.txt                                                                                                                      
[...]
sftp>

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.

Close

Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.