gzip --force is not skipping existing gz files

Solution Verified - Updated -

Environment

  • Red Hat Enterprise Linux (RHEL) 6, 7, 8
    • gzip
    • bzip2
    • xz

Issue

  • Inconsistent behavior of gzip --force * between older and newer releases for existing *.gz files (for details see Diagnostic Steps).
  • A similar bzip2 --force * command for existing *.bz files has the same behavior as older versions of the gzip --force *.
  • A similar xz --force * command for existing *.xz files has the same behavior as older versions of the gzip --force *.

Resolution

A difference in the behaviour described is expected, gzip was changed in version 1.3.12 (2007-04-13).

Bug 1822939 is tracking clarifying update of bzip2 manual page.

Root Cause

Behaviour of gzip, bzip2 and xz is not identical for all command-line options.

Man page of bzip2 states:

The command-line options are deliberately very similar to those of GNU gzip, but they are not identical.

Diagnostic Steps

Newer gzip

If there are existing .gz files, it will skip them if included in the file list, for example:

$ rpm -qa gzip
gzip-1.5-10.el7.x86_64
$ touch file{1..3}
$ gzip * 
$ ls
file1.gz file2.gz file3.gz
$ gzip * 
gzip: file1.gz already has .gz suffix -- unchanged
gzip: file2.gz already has .gz suffix -- unchanged
gzip: file3.gz already has .gz suffix -- unchanged
$ ls
file1.gz file2.gz file3.gz 

If the --force option is used, it will not skip already compressed files (note the double .gz)

$ gzip --force * 
$ ls
file1.gz.gz file2.gz.gz file3.gz.gz

Older gzip (1.3.3, for example)

Already gzipped files are skipped even when forcing:

$ gzip -V
gzip 1.3.3
...
$ ls
file1 file2 file3
$ gzip * 
$ ls
file1.gz file2.gz file3.gz
$ gzip * 
gzip: file1.gz already has .gz suffix -- unchanged
gzip: file2.gz already has .gz suffix -- unchanged
gzip: file3.gz already has .gz suffix -- unchanged
$ ls
file1.gz file2.gz file3.gz

If the --force option is used, it will skip already compressed files:

$ gzip --force * 
gzip: file1.gz already has .gz suffix -- unchanged
gzip: file2.gz already has .gz suffix -- unchanged
gzip: file3.gz already has .gz suffix -- unchanged
$ ls
file1.gz file2.gz file3.gz

All versions of bzip2 and xz

All versions of bzip2 and xz behave very similarly to older versions of gzip:

$ bzip2 -V
bzip2, a block-sorting file compressor.  Version 1.0.6, 6-Sept-2010.
...
$ touch test{1..3}
$ bzip2 *
$ bzip2 *
bzip2: Input file test1.bz2 already has .bz2 suffix.
bzip2: Input file test2.bz2 already has .bz2 suffix.
bzip2: Input file test3.bz2 already has .bz2 suffix.

If the --force option is used, it will skip already compressed files:

$ bzip2 --force *
bzip2: Input file test1.bz2 already has .bz2 suffix.
bzip2: Input file test2.bz2 already has .bz2 suffix.
bzip2: Input file test3.bz2 already has .bz2 suffix.

This solution is part of Red Hat’s fast-track publication program, providing a huge library of solutions that Red Hat engineers have created while supporting our customers. To give you the knowledge you need the instant it becomes available, these articles may be presented in a raw and unedited form.

Comments