SCL Packaging Bugaboos

Latest response

So, I've got an application (RedMine) that wants a newer version of Ruby (2.2+) than is stock with RHEL 7. Silly me figured, "lets see what if SCL can help me." I do a quick search and see that there's rh-ruby22, rh-ruby23 and rh-ruby24.

"Dandy," I think to myself. This was before I started going down the rabbit hole of actually trying to make the SCL packaged version usable.

  • Created an /etc/profile.d/enable_rh-ruby2.sh script with:
#!/bin/bash
source /opt/rh/rh-ruby24/enable
export X_SCLS="$(scl enable rh-ruby24 'echo $X_SCLS')"

in it.

  • Go to run bundle and find that bundle is not in the PATH-elements that are activated by the /opt/rh/rh-ruby24/enable script. Have to update my profile.d script to add
export PATH=${PATH}:/opt/rh/rh-ruby24/root/usr/local/bin

so that bundle will work

  • Get ruby to create my baseline RedMine config and gems.
  • Create my Passenger.conf and verify it's good
  • Start httpd (no errors, yay!)
  • Browse to my URL and get an error - check my logs and find that the SCL-packaged Ruby was unable to find its libs because they're dynamically-linked and not in the default lib search path.
  • Create an appropriate file in /etc/ld.so.conf.d/ and run ldconfig. The SCL-packaged Ruby is now able to find its libs.

This isn't the only time that I or peers have run into issues with SCL-packaged software. Most have taken to either using offerings from other repos than trying to fight the SCL peculiarities. For whatever reason, I'm a bit more bloody-minded (not wanting to be defeated by software, I guess). The SCL struggles feel a lot like dealing with the early days of SunFreeware (Solaris) packagings ...or the even earlier days of Solaris when everything went into /opt/<PACKAGE>.

Am I missing something about the value of SCL packagings or do the SCL packagings need improvement?

Thomas Jones's picture
Log in to join the conversation

Responses

VO Red Hat Newbie 15 points
8 January 2018 9:27 AM

Go to run bundle and find that bundle is not in the PATH-elements

I can't reproduce this testing in RHEL7 mock buildroot. Your /etc/profile.d/enable_rh-ruby2.sh script does what is expected. But if I may suggest, I'd go rather with this script, if it makes any difference:

$ cat /etc/profile.d/enable_rh-ruby2.sh
#!/bin/bash
. scl_source enable rh-ruby24
Thomas Jones's picture Guru Community Member 3711 points
8 January 2018 11:07 AM

Thanks for the suggestion. Unfortunately, it doesn't really make a difference:

  • . is shorthand for source
  • scl_source enable rh-ruby24 is effectively shorthand for directly calling-and-sourceing /opt/rh/rh-ruby24/enable

The overall problem lies with the fact that the rh-ruby24 RPM's enable script is incomplete from a PATH-setting perspective (needs to include the .../local/bin dir). The packaging also fails to install the files needed by ldconfig. This omission makes it so that the rh-rub24-package ruby command can't find its libraries (more a problem when using Apache-spawned processes like Passenger than when running via interactive-shell).

VO Red Hat Newbie 15 points
8 January 2018 11:29 AM

This is content of the enable scriptlet:

# cat /opt/rh/rh-ruby24/enable 
export PATH=/opt/rh/rh-ruby24/root/usr/local/bin:/opt/rh/rh-ruby24/root/usr/bin${PATH:+:${PATH}}
export LD_LIBRARY_PATH=/opt/rh/rh-ruby24/root/usr/local/lib64:/opt/rh/rh-ruby24/root/usr/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}
export MANPATH=/opt/rh/rh-ruby24/root/usr/local/share/man:/opt/rh/rh-ruby24/root/usr/share/man:$MANPATH
export PKG_CONFIG_PATH=/opt/rh/rh-ruby24/root/usr/local/lib64/pkgconfig:/opt/rh/rh-ruby24/root/usr/lib64/pkgconfig${PKG_CONFIG_PATH:+:${PKG_CONFIG_PATH}}
export XDG_DATA_DIRS=/opt/rh/rh-ruby24/root/usr/local/share:/opt/rh/rh-ruby24/root/usr/share:${XDG_DATA_DIRS:-/usr/local/share:/usr/share}

The PATH clearly contains the "/opt/rh/rh-ruby24/root/usr/local/bin". The LD_LIBRARY_PATH is extended to contain the path to the libruby.so.

I am testing with:

$ rpm -q rh-ruby24-runtime
rh-ruby24-runtime-2.4-2.el7.x86_64
Thomas Jones's picture Guru Community Member 3711 points
8 January 2018 12:20 PM

Re-spun the instance this morning. Looks like the contents are different from what I was seeing last week. I'd say, "not 100% sure why, at this point, it was barking that it couldn't find bundle due to the .../local/bin not being in its path and not able to find the requisite shared libraries," but it actually comes down to how things were being invoked.

Non-interactive processes — stuff that isn't associated with a bash login - don't get anything from /etc/profile.d. Therefore, its contents are typically moot in the context of a daemon. Services like the Apache httpd and, therefore, Passenger, won't pick up PATH or LD_LIBRARY_PATH statements from any of the /etc/profile.d contents. These services need to fall back on other methods.

One such method is to update where the runtime-linker looks for libraries — by way of example, MariaDB:

# rpm -qf /etc/ld.so.conf.d/mariadb-x86_64.conf
mariadb-libs-5.5.56-2.el7.x86_64
# rpm --scripts -q $(rpm -qf /etc/ld.so.conf.d/mariadb-x86_64.conf)
postinstall program: /sbin/ldconfig
postuninstall program: /sbin/ldconfig

Per the above, when the MariaDB Libraries RPM is installed, it creates a file in /etc/ld.so.conf.d/ and uses an %post script to activate that files' contents. It's arguable whether it ought to do this, but it's a point of comparison on which (looking back at things, "faulty") expectations are built (since both the mariadb and rh-ruby24 RPMs come by way of the vendor's repositories). I'd initially used this method to work around Passenger not being able to find the rh-ruby24 shared libraries (method since changed).

At any rate... Since I was deploying RedMine via non-interactive processes (specifically, CloudFormation), the contents of /etc/profile.d were essentially moot in that context. This would be why my deployment-scripts weren't finding bundle.

I guess what I'm driving at is that, since SCL seems to be optimized for interactive shell use, there probably needs to be -server RPM bundles for the SCL packages. This might make using SCL packages less rage-inducing when you want to deploy an application/service that needs SCL-provided functionality.

Sidenote: yes, I'm aware that Apache lets you reset its internal library search path via a SetEnv directive in its /etc/httpd/conf.d/ files. It's just not something I normally have to deal with when using the antiquated libraries that Red Hat makes available via other-than-SCL means (heck, even EPEL puts its libs in the standard search paths).

VO Red Hat Newbie 15 points
8 January 2018 2:07 PM

"non-interactive process" <= this is unfortunately very broad term and there is many ways to execute such process.

In case of mariadb collection, this is actually systemd unit. If you check that unit files e.g. in rh-mariadb102-mariadb-server, you'll see that there is always called "scl enable".

In your case, you want to use Passenger. I am not an expert here, but since you said you have correct "Passenger.conf", I'm going to assume assume that your "PassengerRuby" points to "/opt/rh/rh-ruby24/root/usr/bin/ruby" which is not sufficient actually, because precisely the issues you mention. Not sure if using something like "scl enable rh-ruby24 ruby" would be acceptable instead.

Nevertheless, in rh-passenger RHSCL, there are available packages such as rh-passenger40-ruby22, which provides scripts such as:

$ cat /opt/rh/rh-passenger40/root/usr/libexec/passenger-ruby22
#!/bin/bash

. /opt/rh/rh-passenger40/service-environment
for sclname in $RH_PASSENGER40_RUBY22_SCLS_ENABLED ; do
    if [ -f /opt/rh/$sclname/enable ]; then
        . /opt/rh/$sclname/enable
        export X_SCLS="$X_SCLS $sclname"
    fi
done

export RUBYLIB="/opt/rh/rh-passenger40/root/usr/lib64/passenger22${RUBYLIB:+:${RUBYLIB}}"

ruby "$@"

There is obviously lot of cruft, but in essence, it is similar to calling ". scl_source enable rh-ruby24" and then running Ruby. For Passenger, you need script like this (you might be interested in rhbz#949833, which is related).

However, I am not sure how useful it would be to ship script like this in Ruby (and other) SCL. What should be the name of the script. How discoverable it would be. We used to have something like /usr/bin/ruby193-ruby, but providing wrappers for every executable does not scale and we abandoned this effort, since one of the original requirements was "don't install anything outside /opt"

Thomas Jones's picture Guru Community Member 3711 points
8 January 2018 2:28 PM

Like I said: SCL feels a lot like the bad old days of when Sun switched from BSD-style "everything in /usr/local" to SysV-style "everything in /opt/package" and the associated mile-long PATH (etcs) settings. Yeah, it can work, but, ultimately, it feels horrible - so much so that even Sun abandoned the over-isolation bits of add-on functionalities.

All I know is that, when dealing with co-workers, the feelings around SCL's implementations are not "charitable". The usual reaction to the prospect of using SCL is to ask "isn't there an alternative in EPEL or that doesn't require this degree of dicking around". Again, pretty much the same sentiments that were common when Sun decided "all packages now go in /opt/".

All of which bears out the adages:

  • The more things change, the more they stay the same.
  • Technology is cyclical
Close

Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.