RHBA-2016:0193 - Bug Fix Advisory

Topic

Red Hat Gluster Storage 3.1 Update 2, which fixes several bugs,
and adds various enhancements, is now available for Red Hat
Enterprise Linux 6.

Description

Red Hat Gluster Storage is a software only scale-out storage solution that
provides flexible and affordable unstructured data storage. It unifies data
storage and infrastructure, increases performance, and improves
availability and manageability to meet enterprise-level storage challenges.

Red Hat Gluster Storage's Unified File and Object Storage is built on
OpenStack's Object Storage (swift).

This update also fixes numerous bugs and adds various enhancements. Space
precludes documenting all of these changes in this advisory. Users are
directed to the Red Hat Gluster Storage 3.1 Technical Notes, linked to in
the References section, for information on the most significant of these
changes.

This advisory introduces the following new features:

• Writable Snapshots

Red Hat Gluster Storage snapshots can now be cloned and made writable by creating a new volume based on an existing snapshot. Clones are space efficient, as the cloned volume and original snapshot share the same logical volume back end, only consuming additional space as the clone diverges from the snapshot. For more information, see the Red Hat Gluster Storage 3.1 Administration Guide: https://access.redhat.com/documentation/en-US/Red_Hat_Storage/3.1/html/Administration_Guide/.

• RESTful Volume Management with Heketi [Technology Preview]

Heketi provides a RESTful management interface for managing Red Hat Gluster Storage volume lifecycles. This interface allows cloud services like OpenStack Manila, Kubernetes, and OpenShift to dynamically provision Red Hat Gluster Storage volumes. For details about this technology preview, see the Red Hat Gluster Storage 3.1 Administration Guide: https://access.redhat.com/documentation/en-US/Red_Hat_Storage/3.1/html/Administration_Guide/

• Red Hat Gluster Storage for Containers

With the Red Hat Gluster Storage 3.1 update 2 release a Red Hat Gluster Storage environment can be set up in a container. Containers use shared operating systems and are much more efficient than hypervisors in system resource terms. Containers rest on top of a single Linux instance and allows applications to use the same Linux kernel as the system that they're running on. This improves the overall efficiency and reduces the space consumption considerably. For more information, see the Red Hat Gluster Storage 3.1 Administration Guide: https://access.redhat.com/documentation/en-US/Red_Hat_Storage/3.1/html/Administration_Guide/.

• BitRot scrubber status

The BitRot scrubber command (gluster volume bitrot VOLNAME scrub status)
can now display scrub progress and list identified corrupted files,
allowing administrators to locate and repair corrupted files more easily.
See the Red Hat Gluster Storage 3.1 Administration Guide for details:
https://access.redhat.com/documentation/en-US/Red_Hat_Storage/3.1/html/Admi nistration_Guide/.

• Samba Asynchronous I/O

With this release, asynchronous I/O from Samba to Red Hat Gluster Storage is supported. The aio read size option is now enabled and set to 4096 by default. This increases the throughput when the client is multithreaded or there are multiple programs accessing the same share. If you have Linux clients using SMB 2.0 or higher, Red Hat recommends disabling asynchronous I/O (setting aio read size to 0).

All users of Red Hat Gluster Storage are advised to apply this update.

Solution

Before applying this update, make sure all previously released errata
relevant to your system have been applied.

For details on how to apply this update, refer to:

https://access.redhat.com/articles/11258

Affected Products

• Red Hat Enterprise Linux Server 6 x86_64
• Red Hat Gluster Storage Server for On-premise 3 for RHEL 6 x86_64
• Red Hat Gluster Storage Nagios Server 3 for RHEL 6 x86_64

Fixes

• BZ - 1018170 - quota: numbers of warning messages in nfs.log a single file itself
• BZ - 1060676 - [add-brick]: I/O on NFS fails when bricks are added to a distribute-replicate volume
• BZ - 1139193 - git operations fail when add-brick operation is done
• BZ - 1177592 - quota: rename of "dir" fails in case of quota space availability is around 1GB
• BZ - 1199033 - [epoll] Typo in the gluster volume set help message for server.event-threads and client.event-threads
• BZ - 1224064 - [Backup]: Glusterfind session entry persists even after volume is deleted
• BZ - 1224880 - [Backup]: Unable to delete session entry from glusterfind list
• BZ - 1228079 - [Backup]: Crash observed when keyboard interrupt is encountered in the middle of any glusterfind command
• BZ - 1228643 - I/O failure on attaching tier
• BZ - 1230540 - Quota list is not working on tiered volume.
• BZ - 1231144 - Data Tiering; Self heal deamon stops showing up in "vol status" once attach tier is done
• BZ - 1236020 - Data Tiering: Change the error message when a detach-tier status is issued on a non-tier volume
• BZ - 1236052 - Data Tiering:Throw a warning when user issues a detach-tier commit command
• BZ - 1236153 - setting enable-shared-storage without mentioning the domain, doesn't enables shared storage
• BZ - 1236503 - Disabling enable-shared-storage deletes the volume with the name - "gluster_shared_storage"
• BZ - 1237022 - Probing a new RHGS node, which is part of another cluster, should throw proper error message in logs and CLI
• BZ - 1237059 - DHT-rebalance: rebalance status shows failed when replica pair bricks are brought down in distrep volume while re-name of files going on
• BZ - 1238561 - FSAL_GLUSTER : nfs4_getfacl do not display DENY entries
• BZ - 1238634 - Though scrubber settings changed on one volume log shows all volumes scrubber information
• BZ - 1240502 - nfs-ganesha: remove the entry of the deleted node
• BZ - 1240918 - Quota: After rename operation , gluster v quota <volname> list-objects command give incorrect no. of files in output
• BZ - 1241436 - nfs-ganesha: refresh-config stdout output includes dbus messages "method return sender=:1.61 -> dest=:1.65 reply_serial=2"
• BZ - 1242022 - rdma : pending - porting log messages to a new framework
• BZ - 1242148 - With NFSv4 ACLs enabled, rename of a file/dir to an existing file/dir fails
• BZ - 1243534 - Error messages observed in cli.log
• BZ - 1243797 - quota/marker: dir count in inode quota is not atomic
• BZ - 1244792 - nfs-ganesha: nfs-ganesha debuginfo package has missing debug symbols
• BZ - 1247515 - [upgrade] Error messages seen in glusterd logs, while upgrading from RHGS 2.1.6 to RHGS 3.1
• BZ - 1247947 - [upgrade] After in-service software upgrade from RHGS 2.1 to RHGS 3.1, bumping up op-version failed
• BZ - 1248895 - [upgrade] After in-service software upgrade from RHGS 2.1.6 to RHGS 3.1, probing a new RHGS 3.1 node is moving the peer to rejected state
• BZ - 1251471 - FSAL_GLUSTER: Code clean up in acl implemenatation
• BZ - 1251477 - FSAL_GLUSTER : Removal of previous acl implementation
• BZ - 1257209 - With quota enabled, when files are created and deleted from mountpoint, error messages are seen in brick logs
• BZ - 1257343 - vol heal info fails when transport.socket.bind-address is set in glusterd
• BZ - 1257957 - nfs-ganesha: nfs-ganesha process gets killed while executing UNLOCK with a cthon test on vers=3
• BZ - 1258341 - Disperse volume: single file creation is generating many log messages
• BZ - 1260530 - Provide more meaningful errors on peer probe and peer detach
• BZ - 1261765 - NFS Ganesha export lost during IO on EC volume
• BZ - 1262191 - nfs-ganesha: having acls and quota enabled for volume and nfs-ganesha coredump while creating data
• BZ - 1262680 - IO hung on v4 ganesha mount
• BZ - 1264804 - ECVOL: glustershd log grows quickly and fills up the root volume
• BZ - 1265200 - quota: set quota version for files/directories
• BZ - 1267488 - [upgrade] Volume status doesn't show proper information when nodes are upgraded from 2.1.6 to 3.1.1
• BZ - 1269203 - regression : RHGS 3.0 introduced a maximum value length in the info files
• BZ - 1269557 - FUSE clients in a container environment hang and do not recover post losing connections to all bricks
• BZ - 1270321 - Add heketi package to product RH Gluster Storage 3
• BZ - 1271178 - rm -rf on /run/gluster/vol/<directory name>/ is not showing quota output header for other quota limit applied directories
• BZ - 1271184 - quota : display the size equivalent to the soft limit percentage in gluster v quota <volname> list* command
• BZ - 1271648 - tier/cli: number of bricks remains the same in v info --xml
• BZ - 1271659 - gluster v status --xml for a replicated hot tier volume
• BZ - 1271725 - Data Tiering: Disallow attach tier on a volume where any rebalance process is in progress to avoid deadlock(like remove brick commit pending etc)
• BZ - 1271727 - Tiering/glusted: volume status failed after detach tier start
• BZ - 1271733 - Tier/shd: Tracker bug for tier and shd compatibility
• BZ - 1271750 - glusterd: disable ping timer b/w glusterd and make epoll thread count default 1
• BZ - 1271999 - After upgrading to RHGS 3.1.2 build, the other peer was shown as disconnected
• BZ - 1272335 - [Heketi] Not all /etc/fstab enteries are cleaned up after volume delete
• BZ - 1272341 - Data Tiering:Promotions fail when brick of EC (disperse) cold layer are down
• BZ - 1272403 - [Tier]: man page of gluster should be updated to list tier commands
• BZ - 1272407 - Data Tiering:error "[2015-10-14 18:15:09.270483] E [MSGID: 122037] [ec-common.c:1502:ec_update_size_version_done] 0-tiervolume-disperse-1: Failed to update version and size [Input/output error]"
• BZ - 1272408 - Data Tiering:[2015-10-15 02:54:52.259879] E [MSGID: 109039] [dht-common.c:2833:dht_vgetxattr_cbk] 0-tiervolume-cold-dht: vgetxattr: Subvolume tiervolume-disperse-1 returned -1 [No such file or directory]
• BZ - 1272452 - Data Tiering:heat counters not getting reset and also internal ops seem to be heating the files
• BZ - 1273347 - [Tier]: glusterfs crashed --volfile-id rebalance/tiervolume
• BZ - 1273348 - [Tier]: lookup from client takes too long {~7m for 18k files}
• BZ - 1273385 - tiering + nfs-ganesha: tiering has a segfault
• BZ - 1273706 - build: package release in NVR should only be integral
• BZ - 1273711 - Disperse volume: df -h on a nfs mount throws Invalid argument error
• BZ - 1273728 - Crash while bringing down the bricks and self heal
• BZ - 1273850 - Replica pairs in a volume shouldn't be from the same node
• BZ - 1273868 - Heketi doesn't allow deleting nodes with drives missing/inaccessible
• BZ - 1275155 - [Tier]: Typo in the output while setting the wrong value of low/hi watermark
• BZ - 1275158 - Data Tiering:Getting lookup failed on files in hot tier, when volume is restarted
• BZ - 1275515 - Reduce 'CTR disabled' brick log message from ERROR to INFO/DEBUG
• BZ - 1275521 - Wrong value of snap-max-hard-limit observed in 'gluster volume info'.
• BZ - 1275525 - snap-max-hard-limit for snapshots always shows as 256 in info file.
• BZ - 1275633 - Clone creation should not be successful when the node participating in volume goes down.
• BZ - 1275751 - Data Tiering:File create terminates with "Input/output error" as split brain is observed
• BZ - 1275912 - AFR self-heal-daemon option is still set on volume though tier is detached
• BZ - 1275925 - [New] - Message displayed after attach tier is misleading
• BZ - 1275971 - [RFE] Geo-replication support for Volumes running in docker containers
• BZ - 1275998 - Data Tiering: "ls" count taking link files and promote/demote files into consideration both on fuse and nfs mount
• BZ - 1276051 - Data Tiering:inconsistent linkfile creation when lookups issued on cold tier files
• BZ - 1276227 - Data Tiering:delete command rm -rf not deleting files the linkto file(hashed) which are under migration and possible spit-brain observed and possible disk wastage
• BZ - 1276245 - [Tier]: Stopping and Starting tier volume triggers fixing layout which fails on local host
• BZ - 1276248 - [Tier]: restarting volume reports "insert/update failure" in cold brick logs
• BZ - 1276273 - [Tier]: start tier daemon using rebal tier start doesnt start tierd if it is failed on any of single node
• BZ - 1276330 - [heketi-cli] Incorrect error message when storage-host-name is missing while adding a node
• BZ - 1276334 - Data Tiering:tiering deamon crashes when trying to heat the file
• BZ - 1276340 - [heketi-cli] Inconsistency with the requirement for zone value
• BZ - 1276348 - nfs-ganesha: ACL issue after adding an ace for a user the file permissions gets modified
• BZ - 1276542 - RHGS-3.1.2 op-version need to be corrected
• BZ - 1276587 - [GlusterD]: After updating one of rhgs 2.1.6 node to 3.1.2 in two node cluster, volume status is failing
• BZ - 1276678 - CTR should be enabled on attach tier, disabled otherwise.
• BZ - 1277028 - SMB: share entry from smb.conf is not removed after setting user.cifs and user.smb to disable.
• BZ - 1277043 - Upgrading to 3.7.-5-5 has changed volume to distributed disperse
• BZ - 1277088 - Data Tiering:Rename of cold file to a hot file causing split brain and showing two copies of files in mount point
• BZ - 1277126 - [New] - Files in a tiered volume gets promoted when bitd signs them
• BZ - 1277316 - Data Tiering: fix lookup-unhashed for tiered volumes.
• BZ - 1277359 - Data Tiering:Filenames with spaces are not getting migrated at all
• BZ - 1277368 - Bit rot version and signature for the files on a tiered volume are missing after few promotions and demotions of the files.
• BZ - 1277659 - lookup and set xattr fails when bit rot is enabled on a tiered volume.
• BZ - 1277886 - FSAL_GLUSTER : if only DENY entry is set for a user/group, then it lost all its default permission
• BZ - 1277944 - "Transport endpoint not connected" in heal info though hot tier bricks are up
• BZ - 1278254 - [Snapshot]: Clone creation fails on tiered volume with pre-validation failed message
• BZ - 1278270 - [Tier]: "failed to reset target size back to 0" errors in tier logs while performing rename ops
• BZ - 1278279 - EC: File healing promotes it to hot tier
• BZ - 1278346 - Data Tiering:Regression:NFS crashed due to dht readdirp after attach tier
• BZ - 1278384 - 'ls' on client mount lists varying number of files while promotion/demotion
• BZ - 1278389 - Data Tiering: Tiering deamon is seeing each part of a file in a Disperse cold volume as a different file
• BZ - 1278390 - Data Tiering:Regression:Detach tier commit is passing when detach tier is in progress
• BZ - 1278399 - I/O failure on attaching tier on nfs client
• BZ - 1278408 - [Tier]: Volume start failed after tier attach to newly created stopped volume.
• BZ - 1278419 - Data Tiering:Data Loss:File migrations(flushing of data) to cold tier fails on detach tier with quota limits reached
• BZ - 1278723 - Tier : Move common functions into tier.rc
• BZ - 1278754 - Data Tiering:Metadata changes to a file should not heat/promote the file
• BZ - 1278798 - Few snapshot creation fails with pre-validation failed message on tiered volume.
• BZ - 1279314 - [Tier]: After volume restart, unable to stop the started detach tier
• BZ - 1279350 - [Tier]: Space is missed b/w the words in the detach tier stop error message
• BZ - 1279830 - File creation in nested folders fails when add-brick operation is done on a volume with exclusive file lock.
• BZ - 1280410 - ec-readdir.t is failing consistently
• BZ - 1281304 - sometimes files are not getting demoted from hot tier to cold tier
• BZ - 1281946 - Large system file distribution is broken
• BZ - 1282701 - build: compile error on RHEL5
• BZ - 1282729 - Creation of files on hot tier volume taking very long time
• BZ - 1283035 - [GlusterD]: Incorrect peer status showing if volume restart done before entire cluster update.
• BZ - 1283050 - self-heal won't work in disperse volumes when they are attached as tiers
• BZ - 1283057 - nfs-ganesha+tiering: the fs-sanity is taking is more than 24 hours to complete on nfs vers=3
• BZ - 1283410 - cache mode must be the default mode for tiered volumes
• BZ - 1283505 - when scrubber is scheduled scrubd moves the files to hot tier
• BZ - 1283563 - libgfapi to support set_volfile-server-transport type "unix"
• BZ - 1283566 - qupta/marker: backward compatibility with quota xattr vesrioning
• BZ - 1283608 - nfs-ganesha: Upcall sent on null gfid
• BZ - 1283940 - Data Tiering: new set of gluster v tier commands not working as expected
• BZ - 1283961 - Data Tiering:Change the default tiering values to optimize tiering settings
• BZ - 1284387 - Without detach tier commit, status changes back to tier migration
• BZ - 1284834 - tiering: Seeing error messages E "/usr/lib64/glusterfs/3.7.5/xlator/features/changetimerecorder.so(ctr_lookup+0x54f) [0x7f6c435c116f] ) 0-ctr: invalid argument: loc->name [Invalid argument] after attach tier
• BZ - 1285166 - Snapshot creation after attach-tier causes glusterd crash
• BZ - 1285226 - Masking the wrong values in Bitrot status command
• BZ - 1285238 - Corrupted objects list does not get cleared even after all the files in the volume are deleted and count increases as old + new count
• BZ - 1285281 - CTDB: yum update fails on RHEL6 for ctdb package with dependency on procps-ng and systemd-units
• BZ - 1285295 - [geo-rep]: Recommended Shared volume use on geo-replication is broken in latest build
• BZ - 1285306 - Unresolved dependencies on ctdb-4.2.4-7.el6rhs.x86_64
• BZ - 1285651 - [Tier]: Error: attempt to set internal xattr: trusted.ec.* [Operation not permitted]
• BZ - 1285678 - RHGS312 RHEL7.2 based ISO is not working. throws error like ImportError; no library named udev
• BZ - 1285783 - fops-during-migration.t fails if hot and cold tiers are dist-rep
• BZ - 1285797 - tiering: T files getting created , even after disk quota exceeds
• BZ - 1285958 - [GlusterD]: NFS service not running after layered installation of RHGS on RHEL7.x
• BZ - 1285998 - Possible memory leak in the tiered daemon
• BZ - 1286058 - Brick crashes because of race in bit-rot init
• BZ - 1286218 - Data Tiering:Watermark:File continuously trying to demote itself but failing " [dht-rebalance.c:608:__dht_rebalance_create_dst_file] 0-wmrk-tier-dht: chown failed for //AP.BH.avi on wmrk-cold-dht (No such file or directory)"
• BZ - 1286346 - Data Tiering:Don't allow or reset the frequency threshold values to zero when record counter features.record-counter is turned off
• BZ - 1286604 - glusterfsd to support volfile-server-transport type "unix"
• BZ - 1286605 - vol quota enable fails when transport.socket.bind-address is set in glusterd
• BZ - 1286637 - [geo-rep+tiering]: symlinks are not getting synced to slave on tiered master setup
• BZ - 1286654 - Data Tiering:Read heat not getting calculated and read operations not heating the file with counter enabled
• BZ - 1286927 - Tier: ec xattrs are set on a newly created file present in the non-ec hot tier
• BZ - 1287447 - remove watermark ie cluster.tier-mode from vol info after a detach tier is completed successfully
• BZ - 1287532 - After detach-tier start writes still go to hot tier
• BZ - 1287980 - [Quota]: Peer status is in "Rejected" state with Quota enabled volume
• BZ - 1287997 - tiering: quota list command is not working after attach or detach
• BZ - 1288490 - Good files does not promoted in a tiered volume when bitrot is enabled
• BZ - 1288509 - rm -rf is taking very long time
• BZ - 1288921 - Use after free bug in notify_kernel_loop in fuse-bridge code
• BZ - 1288988 - Getting errors while launching the selfheal
• BZ - 1289017 - Failed to show rebalance status if the volume name has 'tier' substring
• BZ - 1289071 - [Tier]: Failed to open "demotequeryfile-master-tier-dht" errors logged on the node having only cold bricks
• BZ - 1289092 - update redhat-release-server to the latest one available in rhel7.2
• BZ - 1289228 - [Tiering] + [DHT] - Detach tier fails to migrate the files when there are corrupted objects in hot tier.
• BZ - 1289423 - Regular files are listed as 'T' files on nfs mount
• BZ - 1289437 - [Tier]: rm -rf * from client during demotion causes a stale link file to remain in system with attributes as ?????
• BZ - 1289483 - FSAL_GLUSTER : Rename throws error in mount when acl is enabled
• BZ - 1289893 - Excessive "dict is NULL" logging
• BZ - 1289975 - Access to files fails with I/O error through uss for tiered volume
• BZ - 1290401 - File is not demoted after self heal (split-brain)
• BZ - 1291052 - [tiering]: read/write freq-threshold allows negative values
• BZ - 1291152 - [tiering]: cluster.tier-max-files option in tiering is not honored
• BZ - 1291195 - [georep+tiering]: Geo-replication sync is broken if cold tier is EC
• BZ - 1291560 - Renames/deletes failed with "No such file or directory" when few of the bricks from the hot tier went offline
• BZ - 1291566 - first file created after hot tier full fails to create, but later ends up as a stale erroneous file (file with ???????????)
• BZ - 1291969 - [Tiering]: When files are heated continuously, promotions are too aggressive that it promotes files way beyond high water mark
• BZ - 1292205 - When volume creation fails, gluster volume and brick lvs are not deleted
• BZ - 1292605 - (RHEL6) hook script for CTDB should not change Samba config
• BZ - 1292705 - gluster cli crashed while performing 'gluster vol bitrot <vol_name> scrub status'
• BZ - 1292773 - (RHEL6) S30Samba scripts do not work on systemd systems
• BZ - 1293228 - Disperse: Disperse volume (cold vol) crashes while writing files on tier volume
• BZ - 1293237 - [Tier]: "Bad file descriptor" on removal of symlink only on tiered volume
• BZ - 1293286 - heal info output shouldn't print number of entries processed when brick is unreachable.
• BZ - 1293380 - [tiering]: Tiering isn't started after attaching hot tier and hence no promotion/demotion
• BZ - 1293903 - [Tier]: can not delete symlinks from client using rm
• BZ - 1294073 - [tiering]: Incorrect display of 'gluster v tier help'
• BZ - 1294478 - quota: limit xattr not healed for a sub-directory on a newly added bricks
• BZ - 1294487 - glusterfsd crash while bouncing the bricks
• BZ - 1294594 - [Tier]: Killing glusterfs tier process doesn't reflect as failed/faulty in tier status
• BZ - 1294774 - Quota Aux mount crashed
• BZ - 1294816 - Unable to modify quota hard limit on tier volume after disk limit got exceeded
• BZ - 1295299 - glusterfs fuse mount session hangs indefinitely when a file create progressively fills up the hot tier completely
• BZ - 1295736 - "Operation not supported" error logs seen continuosly in brick logs
• BZ - 1296048 - Attach tier + nfs : Creates fail with invalid argument errors
• BZ - 1296134 - Rebalance crashed after detach tier.
• BZ - 1297004 - [write-behind] : Write/Append to a full volume causes fuse client to crash
• BZ - 1297300 - Stale stat information for corrupted objects (replicated volume)
• BZ - 1299724 - Excessive logging in mount when bricks of the replica are down
• BZ - 1299799 - Snapshot creation fails on a tiered volume
• BZ - 1300246 - [Tiering]: Values of watermarks, min free disk etc will be miscalculated with quota set on root directory of gluster volume
• BZ - 1300682 - [georep+tiering]: Hardlink sync is broken if master volume is tiered
• BZ - 1302901 - SMB: SMB crashes with AIO enabled on reads + vers=3.0
• BZ - 1303894 - promotions not happening when space is created on previously full hot tier
• BZ - 1304684 - [quota]: Incorrect disk usage shown on a tiered volume
• BZ - 1305172 - [Tier]: Endup in multiple entries of same file on client after rename which had a hardlinks

CVEs

(none)

References

• http://www.redhat.com/security/updates/classification/#normal