Patch management solutions using Satellite

Latest response

Hello All,

I was hoping I could get some insight as to how you guys use the API (or GUI) to handle patch management. I see there are a lot of options but none of them seem to be THE way to go. I used to use the scheduleScriptRun method in Satellite to remotely run a "yum -update -y" but this caused issues. When the command times out it completely screws the server due to the yum update failing in the middle of removing/updating packages. Now, I use some logic to get relevant errata for servers indicated in the arguments. Then, apply each errata one at a time per server. I then check the scheduled actions queue to see if any of the jobs contain the phrase "Errata Update". If present, the script waits. If not, then the script reboots the servers one at a time. Is there a better way to do this? The issue I keep running in to is finding an accurate and reliable way to determine when the servers have finished patching using Satellite.

Responses

Hi Chris,

Ideally you should be able to do what you want with Apply Errata & Schedule Reboot functionality via the Satellite GUI itself, for instance:

  • Access system profile
  • Software->Errata->Select & 'Apply Errata'
  • Details->Overview->Schedule System Reboot

rhnsd (via regular check-ins) or osad (scheduled tasks via Messaging) should then process those tasks in order and do what you want.

Two other options that may be worth considering, using your old method, it may be possible to 'disown' the process from rhnsd/osad which would allow it to run beyond the timeout, or use 'at' to schedule the task via Satellite - for instance 'at now + 20 minutes -f <jobspec file>' and use config management to deploy a file read/write only to root to a specified location, with 'yum -y update' followed optionally by a 'reboot'.  This assumes that you haven't disabled 'at', but would 20 minutes after task was processed by rhnsd, would trigger yum -y update to be run by atd (think cron), after waiting 20 minutes.

I used to use this approach myself to turn off my work machines at the end of the day, they had to be switched off in a particular order, so I'd schedule at jobs on the two that had to be shutdown last, so I could walk away and let things happen automatically.  A very nifty tool.

I still cant find a god way to do it and i m using yum from each client. The GUI seems to be pretty tedious since it does not install errata + packages all at the same time . maybe im wrong but it seems you have to do lot of clicks and there is no visible log of what is going on that i know. 

Hello Nigel and thanks so much for the input!

However, I have 2 obstacles with the methods you presented and if I'm misunderstanding something please correct me. The first one is that the GUI only allows you to schedule one event at a time with no "repeat" logic. Meaning, I can't put a server on a reoccuring schedule very easily. We have 330+ RHEL servers so scheduling each one in advance with the given method is more than just tedious. It's also hard to manage. The second issue is that I don't like the idea of hard coding a wait time. For example, I can easily use the API to schedule reoccuring patching and just include a scheduled reboot 20 or 30 minutes later. Call me paranoid but I don't like the risk of rebooting a server in the middle of it patching. That has lead us to dependancy hell in the past due to servers taking much longer to patch than expected and rebooting mid-process. What I am looking for is a way to use the API (or GUI if I'm overlooking something) to patch a server and check to make sure that the server is finished before rebooting.

Here is a portion of my script so far:

***EDIT*** I have no idea why the code below won't format properly. 

 

 
ServerList = []
servers = sys.argv[1:]
for x in servers:
    names = client.system.search.hostname(key, x)
    var =  names[0]['id']
    ServerList.append(var)
    print ServerList
for i in ServerList:
    GetErrata = client.system.getRelevantErrata(key, i)
    if GetErrata == []:
     print "No Updates for "+ str(i)
    else:
     y = 0
     IDlist = []
     while y < len(GetErrata):
        errata =  GetErrata[y]['id']
        print errata
        IDlist.append(errata)
        print IDlist
        print i
        update = client.system.scheduleApplyErrata(key, i, errata )
        print update
        y = y + 1
 
def getActions():
    actions = []
    c = 0
    schedule = client.schedule.listInProgressActions(key)
    print schedule
    while c < len(schedule):
        type = schedule[c]['type']
        actions.append(type)
        print type
        print actions
        c = c + 1
    return actions
 
z = 0
while z < 1:
    actions = getActions()
    print actions
    if "Errata Update" not in actions:
        print "Finished patching. Servers are now rebooting..."
        z = z + 1
    else:
        time.sleep(1)
        print "still Patching"
 
 
client.auth.logout(key)
 
 

Hi Chris,

The second one (using at) should be okay, as you'd be using Satellite to schedule the 'update & reboot' script on the local machine.  As the script would be run by at, it'd run in order and not reboot until the first line of the script - 'yum -y update' is complete.  You are right about the first approach though, I've just tried it out on a test VM that I was about to wipe clean, I had scheduled ~30 pending Erratum, plus a system reboot, and it appears to have taken the actions in the reverse order.

I'm personally not aware of another way around this, especially in Satellite, what it sounds like you really want is the option when applying erratum to have a checkbox 'Reboot after applying these errata', which could be complicated by the fact that at the moment, it'd be akin to manually scheduling the reboot after each individual errata, which if rhn_check were to pick up tasks out of order, could cause issues.  There are a couple of other interesting things we'd need to look at as well, for instance what would be done if a single errata failed to apply, would we honour the reboot.

My recommendation is that if what I have described in my second paragraph is what you are looking for (it sounds to me that it is), then create a support case with us so we can look into this line further.

Thank you very much for your reply. I will definitely look into creating a support case for this. Just out of curiosity though, is there a "best preactices" guide to patching with Satellite? I wonder how they intended it to be used for patching. To me, it looks as if Satellite wasn't really created with the intention of being a true patch management solution IMO. 

Thanks agin!

Hey, did anyone got any more information on it ? Does anyone has any approach finalized if you can share please? any scripts

Close

Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.