wget Download Error when trying to download isos

Latest response

I'm trying to automate the iso downloads for the offline Red Hat Satellite, by using wget.

However if I try to download with the following:


[root@ameda4aisrx0232 tmp]# wget -c "https://content-web.rhn.redhat.com/rhn/isos/rhn-satellite-isos/channel-dumps/incremental-dumps/dump-rhel-x86_64-5/rhn-export-rhel-x86_64-5-20141005000000-20160116000000.0-01.iso?__gda__=1453230481_a492dd9b039977ee2587097b62ad7e1d&ext=.iso" --2016-01-19 14:00:00-- https://content-web.rhn.redhat.com/rhn/isos/rhn-satellite-isos/channel-dumps/incremental-dumps/dump-rhel-x86_64-5/rhn-export-rhel-x86_64-5-20141005000000-20160116000000.0-01.iso?__gda__=1453230481_a492dd9b039977ee2587097b62ad7e1d&ext=.iso Resolving content-web.rhn.redhat.com... 184.29.188.245 Connecting to content-web.rhn.redhat.com|184.29.188.245|:443... connected. HTTP request sent, awaiting response... 302 Moved Temporarily Location: https://rhn.redhat.com/rhn/DownloadError.do?code=403&uri=/rhn/isos/rhn-satellite-isos/channel-dumps/incremental-dumps/dump-rhel-x86_64-5/rhn-export-rhel-x86_64-5-20141005000000-20160116000000.0-01.iso&client=192.138.57.36&edge=184.28.127.72&timestamp=1453233600 [following] --2016-01-19 14:00:00-- https://rhn.redhat.com/rhn/DownloadError.do?code=403&uri=/rhn/isos/rhn-satellite-isos/channel-dumps/incremental-dumps/dump-rhel-x86_64-5/rhn-export-rhel-x86_64-5-20141005000000-20160116000000.0-01.iso&client=192.138.57.36&edge=184.28.127.72&timestamp=1453233600 Resolving rhn.redhat.com... 209.132.183.42 Connecting to rhn.redhat.com|209.132.183.42|:443... connected. HTTP request sent, awaiting response... 200 OK Length: unspecified [text/html] Saving to: `DownloadError.do?code=403&uri=%2Frhn%2Fisos%2Frhn-satellite-isos%2Fchannel-dumps%2Fincremental-dumps%2Fdump-rhel-x86_64-5%2Frhn-export-rhel-x86_64-5-20141005000000-20160116000000.0-01.iso&client=192.138.57.36&edge=184.28.127.72&timestamp=1453233600' [ <=> ] 40,898 --.-K/s in 0.1s 2016-01-19 14:00:01 (296 KB/s) - `DownloadError.do?code=403&uri=%2Frhn%2Fisos%2Frhn-satellite-isos%2Fchannel-dumps%2Fincremental-dumps%2Fdump-rhel-x86_64-5%2Frhn-export-rhel-x86_64-5-20141005000000-20160116000000.0-01.iso&client=192.138.57.36&edge=184.28.127.72&timestamp=1453233600' saved [40898] [root@ameda4aisrx0232 tmp]#

I end up getting a file that looks like the following:

DownloadError.do?code=403&uri=%2Frhn%2Fisos%2Frhn-satellite-isos%2Fchannel-dumps%2Fincremental-dumps%2Fdump-rhel-x86_64-5%2Frhn-export-rhel-x86_64-5-20141005000000-20160116000000.0-01.iso&client=192.138.57.36&edge=184.28.127.72&timestamp=1453233600

I'm not sure what this means. I have port 80 and 443 open on IPTables.

Also I can't even ping the rhn.redhat.com or 209.132.183.43 from either my Windows 7 Desktop or from the RHEL Server,

However I can use a tool like

http://www.tcpiputils.com/

Which seems to ping ok

http://www.tcpiputils.com/ping

I can traceroute out from desktop and server

desktop


server

[root@server ~]# traceroute 209.132.183.42
traceroute to 209.132.183.42 (209.132.183.42), 30 hops max, 40 byte packets
 1  dhcp83-220-1 (143.83.220.1)  0.424 ms  0.464 ms  0.516 ms
 2  10.241.3.67 (10.241.3.67)  0.371 ms  0.423 ms  0.482 ms
 3  192.138.57.20 (192.138.57.20)  4.990 ms  5.560 ms  5.261 ms
 4  mhsi-43-201.mhsi.med.osd.mil (214.2.43.201)  1.231 ms  1.343 ms  1.051 ms
 5  192.138.57.62 (192.138.57.62)  3.961 ms  3.683 ms  3.423 ms
 6  192.138.40.90 (192.138.40.90)  3.685 ms  2.593 ms  4.508 ms
 7  198.26.34.181 (198.26.34.181)  3.352 ms  3.331 ms  3.326 ms
 8  * * *
 9  * * *
10  * * *
11  * * *
12  * * *
13  * * *
14  * * *
15  * * *
16  * * *
17  * * *
18  rhn.redhat.com (209.132.183.42)  3.762 ms  4.039 ms  3.740 ms
19  rhn.redhat.com (209.132.183.42)  4.987 ms  5.221 ms  5.291 ms
20  * * *
21  * * *
22  140.14.36.41 (140.14.36.41)  4.607 ms  4.948 ms  4.934 ms
23  63-158-243-45.dia.static.qwest.net (63.158.243.45)  8.978 ms  8.726 ms  8.742 ms
24  * dal-pcor-05.inet.qwest.net (67.14.2.182)  18.443 ms  18.421 ms
25  be3034.ccr41.dfw03.atlas.cogentco.com (154.54.11.165)  14.334 ms  14.917 ms  15.280 ms
26  be2764.ccr22.dfw01.atlas.cogentco.com (154.54.47.213)  14.215 ms  14.422 ms  14.391 ms
27  be2443.ccr22.iah01.atlas.cogentco.com (154.54.44.229)  19.245 ms  19.378 ms be2441.ccr21.iah01.atlas.cogentco.com (154.54.41.65)  19.108 ms
28  be2292.ccr21.sat01.atlas.cogentco.com (154.54.1.82)  25.240 ms be2291.ccr21.sat01.atlas.cogentco.com (154.54.2.190)  24.613 ms  25.063 ms
29  be2301.ccr21.elp01.atlas.cogentco.com (154.54.5.174)  52.071 ms  51.632 ms  51.369 ms
30  be2414.ccr22.phx02.atlas.cogentco.com (154.54.42.105)  51.665 ms be2254.ccr21.phx02.atlas.cogentco.com (154.54.7.33)  54.060 ms be2414.ccr22.phx02.atlas.cogentco.com (154.54.42.105)  51.201 ms

thanks

Responses

Hello

Downloading files requires checking entitlements and so authentication based on certificates takes place. When you download an ISO (logged in using a browser) a temporary link is generated. This is all leads to: "The only supported way to download ISO's in an automated fashion is to use Red Hat Satellite 6."[1]

Please open a support request or contact your support representative to discuss this further.

[1] Download boot ISO from command line

Ok, I'll open a ticket and also contact my support person, thanks.

Just wanted to revisit this.

So what I've done is using Firefox, just right click on the URL from the following URL

https://rhn.redhat.com/rhn/software/channel/downloads/Download.do?cid=18952

Using VIM, I create a file, place the URL into it and from there run wget -i against the file.

The results aren't consistent. Sometimes its working and other times it isn't, and I'm trying to understand why.

I'm typically getting an error like this:

-rw-r--r-- 1 root     root          40801 Jan 25 15:00 DownloadError.do?code=403&uri=%2Frhn%2Fisos%2Frhn-satellite-isos%2Fchannel-dumps%2Fbase-dumps%2Fdump-rhel-i386-5                  %2Frhn-export-rhel-i386-5-20140928.0-06.iso&client=192.138.57.36&edge=23.79.255.69&timestamp=1453755648

Another thing to is that the website for these ISOs, if say I don't do anything with the website for a few minutes, there is an 'Unavailable' error and I have to refresh to get the page to display correctly again.

Why does Red Hat makes it so hard to get update ISOs for those of us who have Disconnected Red Hat Satellites. Again, my main goal is to automate the process using wget and something to me that seems to simple is proving to be alot of work.

The links that are on the download pages are time-sensitive, hence why they eventually time-out with the 403 error (403 is the HTTP code for 'unauthorized'). The timeout has recently been increased (I believe to 30 minutes) to make this a little less painful. What I'd do is start 2-6 downloads simultaneously, refresh the page and repeat as necessary.

The document above (How can I download the Red Hat Enterprise Linux ISO from CDN using the Entitlement Certificate? ) doesn't work with Satellite Content ISOs. Satellite Content ISOs are special in the sense that you require BOTH an valid subscription for Satellite AND the product you wish to download a content ISO for. (i.e., we cannot you to download a content ISO for RHEV if you do not posses a valid subscription for it).

So looks like from wget I will have to add

--certificate=<entitlement_certificate>.pem --ca-certificate=/etc/rhsm/ca/redhat-uep.pem 

When grabbing the ISOs. Also looks like I'm going to have to lower my expectations on when I grab ISOs from 2 to 6 at a time.

thanks

I've tried this out and its working. Thanks for the info.

EDIT: So I spoke too soon. Its not working.

I create a text file with multiple URLs and it might work for one and then it gives an error.

I've looked at the URL above to download the Entitlement Certificate, however I believe the site has changed and I don't see the Units URL where I can go.

So do I need to register my server that I want to download these ISOs to in order for this to work? The URL provided isn't making sense.

Close

Welcome! Check out the Getting Started with Red Hat page for quick tours and guides for common tasks.