systemd journal filling logs due to DEBUG_LOGLEVEL set to 8.

Solution Verified - Updated -

Issue

  • High CPU and memory utilisation observed on system.

  • 'journalctl' utilising more CPU.

  • atomic-openshift-node service flooding the logs inside messages file. Snippet for reference.

</snip>
Feb  5 16:39:04 xxxx.com atomic-openshift-node: I0205 16:39:04.371713    6487 round_trippers.go:414]     Date: Wed, 05 Feb 2020 11:06:17 GMT
Feb  5 16:39:04 xxxx.com atomic-openshift-node: I0205 16:39:04.372188    6487 request.go:895] Response Body:
Feb  5 16:39:04 xxxx.com atomic-openshift-node: 00000000  6b 38 73 00 0a 0a 0a 02  76 31 12 04 4e 6f 64 65  |k8s.....v1..Node|
Feb  5 16:39:04 xxxx.com atomic-openshift-node: 00000010  12 8c 2e 0a e0 03 0a 16  70 72 64 78 63 72 61 70  |........xxxx|
Feb  5 16:39:04 xxxx.com atomic-openshift-node: 00000020  70 61 70 70 30 31 2e 69  62 6c 2e 63 6f 6d 12 00  |.com..|
Feb  5 16:39:04 xxxx.com atomic-openshift-node: 00000030  1a 00 22 2b 2f 61 70 69  2f 76 31 2f 6e 6f 64 65  |.."+/api/v1/node|
Feb  5 16:39:04 xxxx.com atomic-openshift-node: 00000040  73 2f 70 72 64 78 63 72  61 70 70 61 70 70 30 31  |s/xxxx.com|
Feb  5 16:39:04 xxxx.com atomic-openshift-node: 00000050  2e 69 62 6c 2e 63 6f 6d  2f 73 74 61 74 75 73 2a  |.xxxx.com/status*|
Feb  5 16:39:04 xxxx.com atomic-openshift-node: 00000060  24 66 35 61 37 38 32 61  62 2d 65 65 31 36 2d 31  |$f5a782ab-ee16-1|
Feb  5 16:39:04 xxxx.com atomic-openshift-node: 00000070  31 65 38 2d 38 66 61 37  2d 30 30 35 30 35 36 39  |1e8-8fa7-0050569|
Feb  5 16:39:04 xxxx.com atomic-openshift-node: 00000080  66 32 62 62 35 32 08 39  33 30 39 34 36 33 37 38  |f2bb52.930946378|
Feb  5 16:39:04 xxxx.com atomic-openshift-node: 00000090  00 42 08 08 ad fb d8 df  05 10 00 5a 0b 0a 03 61  |.B.........Z...a|
Feb  5 16:39:04 xxxx.com atomic-openshift-node: 000000a0  70 70 12 04 78 63 72 6f  5a 20 0a 17 62 65 74 61  |pp..xcroZ ..beta|
Feb  5 16:39:04 xxxx.com atomic-openshift-node: 000000b0  2e 6b 75 62 65 72 6e 65  74 65 73 2e 69 6f 2f 61  |.kubernetes.io/a|
Feb  5 16:39:04 xxxx.com atomic-openshift-node: 000000c0  72 63 68 12 05 61 6d 64  36 34 5a 1e 0a 15 62 65  |rch..amd64Z...b [truncated 2827
2 chars]
</snip>

Environment

  • Openshift Container Platform v3.11

Subscriber exclusive content

A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.

Current Customers and Partners

Log in for full access

Log In

New to Red Hat?

Learn more about Red Hat subscriptions

Using a Red Hat product through a public cloud?

How to access this content