"clocksource: timekeeping watchdog on CPUxx: Marking clocksource 'tsc' as unstable because the skew is too large" 메시지가 표시된 후 소프트 잠금 발생
Issue
- "clocksource: timekeeping watchdog on CPUxx: Marking clocksource 'tsc' as unstable because the skew is too large" 메시지가 표시된 후 소프트 잠금 발생
Mar 2 08:59:50 hostname kernel: [48100603.536281] clocksource: timekeeping watchdog on CPU11: Marking clocksource 'tsc' as unstable because the skew is too large:
Mar 2 08:59:50 hostname kernel: [48100603.536292] clocksource: 'hpet' wd_now: f9df9741 wd_last: f8f7fc81 mask: ffffffff
Mar 2 08:59:50 hostname kernel: [48100603.536297] clocksource: 'tsc' cs_now: 177152954967e5a cs_last: 177152913b38b56 mask: ffffffffffffffff
Mar 2 08:59:50 hostname kernel: [48100603.536303] tsc: Marking TSC unstable due to clocksource watchdog
Mar 2 08:59:50 hostname kernel: [48100603.536317] TSC found unstable after boot, most likely due to broken BIOS. Use 'tsc=unstable'.
Mar 2 08:59:50 hostname kernel: [48100603.536319] sched_clock: Marking unstable (48104766144542359, -4162608173504)<-(48100603836282552,
...
Mar 2 10:31:05 hostname kernel: [48106077.512811] watchdog: BUG: soft lockup - CPU#0 stuck for 21s! [kworker/0:1:63753]
Mar 2 10:31:05 hostname kernel: [48106077.513139] Modules linked in: ...
Mar 2 10:31:05 hostname kernel: [48106077.513198] CPU: 0 PID: 63753 Comm: kworker/0:1 Not tainted 4.18.0-372.19.1.el8_6.x86_64 #1
Mar 2 10:31:05 hostname kernel: [48106077.513201] Hardware name: Dell Inc. PowerEdge R640/08HT8T, BIOS 1.6.63 01/28/2019
Mar 2 10:31:05 hostname kernel: [48106077.513203] Workqueue: events drm_fb_helper_damage_work [drm_kms_helper]
Mar 2 10:31:05 hostname kernel: [48106077.513226] RIP: 0010:memcpy_erms+0x6/0x10
Mar 2 10:31:05 hostname kernel: [48106077.513233] Code: 90 90 90 90 eb 1e 0f 1f 00 48 89 f8 48 89 d1 48 c1 e9 03 83 e2 07 f3 48 a5 89 d1 f3 a4 c3 66 0f 1f 44 00 00 48 89 f8 48 89 d1 <f3> a4 c3 0f 1f 80 00 00 00 00 48 89 f8 48 83 fa 20 72 7e 40 38 fe
Mar 2 10:31:05 hostname kernel: [48106077.513235] RSP: 0018:ffff9a3ea79dbca8 EFLAGS: 00010206 ORIG_RAX: ffffffffffffff13
Mar 2 10:31:05 hostname kernel: [48106077.513238] RAX: ffff9a3e8a1b6000 RBX: ffff9a3e89067000 RCX: 0000000000000200
Mar 2 10:31:05 hostname kernel: [48106077.513240] RDX: 0000000000000240 RSI: ffff9a3e89067040 RDI: ffff9a3e8a1b6040
Mar 2 10:31:05 hostname kernel: [48106077.513241] RBP: 0000000000000240 R08: 0000000000070000 R09: 0000000000000070
Mar 2 10:31:05 hostname kernel: [48106077.513243] R10: 0000000000000000 R11: 0000000000000001 R12: 0000000000000150
Mar 2 10:31:05 hostname kernel: [48106077.513244] R13: ffff897ec4134400 R14: 0000000000001000 R15: 0000000000000147
Mar 2 10:31:05 hostname kernel: [48106077.513245] FS: 0000000000000000(0000) GS:ffff898dffc00000(0000) knlGS:0000000000000000
Mar 2 10:31:05 hostname kernel: [48106077.513247] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Mar 2 10:31:05 hostname kernel: [48106077.513248] CR2: 00007f7048903ee0 CR3: 0000000bba610003 CR4: 00000000007706f0
Mar 2 10:31:05 hostname kernel: [48106077.513250] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Mar 2 10:31:05 hostname kernel: [48106077.513251] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Mar 2 10:31:05 hostname kernel: [48106077.513252] PKRU: 55555554
Mar 2 10:31:05 hostname kernel: [48106077.513253] Call Trace:
Mar 2 10:31:05 hostname kernel: [48106077.513256] drm_fb_memcpy_dstclip+0x60/0x80 [drm_kms_helper]
Mar 2 10:31:05 hostname kernel: [48106077.513271] mgag200_simple_display_pipe_update+0x7f/0xa0 [mgag200]
Mar 2 10:31:05 hostname kernel: [48106077.513275] drm_atomic_helper_commit_planes+0xb6/0x220 [drm_kms_helper]
Mar 2 10:31:05 hostname kernel: [48106077.513289] drm_atomic_helper_commit_tail+0x26/0x60 [drm_kms_helper]
Mar 2 10:31:05 hostname kernel: [48106077.513301] commit_tail+0xc6/0x110 [drm_kms_helper]
Mar 2 10:31:05 hostname kernel: [48106077.513314] drm_atomic_helper_commit+0x103/0x110 [drm_kms_helper]
Mar 2 10:31:05 hostname kernel: [48106077.513326] drm_atomic_helper_dirtyfb+0x20e/0x260 [drm_kms_helper]
Mar 2 10:31:05 hostname kernel: [48106077.513339] drm_fb_helper_damage_work+0x222/0x2c0 [drm_kms_helper]
Mar 2 10:31:05 hostname kernel: [48106077.513351] process_one_work+0x1a7/0x360
Mar 2 10:31:05 hostname kernel: [48106077.513354] ? create_worker+0x1a0/0x1a0
Mar 2 10:31:05 hostname kernel: [48106077.513356] worker_thread+0x30/0x390
Mar 2 10:31:05 hostname kernel: [48106077.513358] ? create_worker+0x1a0/0x1a0
Mar 2 10:31:05 hostname kernel: [48106077.513359] kthread+0x10a/0x120
Mar 2 10:31:05 hostname kernel: [48106077.513364] ? set_kthread_struct+0x40/0x40
Mar 2 10:31:05 hostname kernel: [48106077.513367] ret_from_fork+0x1f/0x40
...
Mar 2 10:32:32 hostname kernel: [48106126.089230] watchdog: BUG: soft lockup - CPU#18 stuck for 23s! [bind_exporter:140060]
Mar 2 10:32:32 hostname kernel: [48106126.089233] Modules linked in: ...
Mar 2 10:32:32 hostname kernel: [48106126.089274] CPU: 18 PID: 140060 Comm: bind_exporter Tainted: G L --------- - - 4.18.0-372.19.1.el8_6.x86_64 #1
Mar 2 10:32:32 hostname kernel: [48106126.089277] Hardware name: Dell Inc. PowerEdge R640/08HT8T, BIOS 1.6.63 01/28/2019
Mar 2 10:32:32 hostname kernel: [48106126.089278] RIP: 0010:read_hpet+0x31/0xc0
Mar 2 10:32:32 hostname kernel: [48106126.089281] Code: 05 8c a8 3a 71 a9 00 00 f0 00 75 65 48 8b 35 b6 ac 79 01 49 c7 c0 40 60 40 90 85 f6 74 1d 48 c1 ee 20 eb 04 85 c9 74 12 f3 90 <49> 8b 08 48 89 ca 48 c1 ea 20 89 d0 39 f2 74 ea c3 9c 58 0f 1f 44
Mar 2 10:32:32 hostname kernel: [48106126.089284] RSP: 0018:ffff9a3ea15d3cc0 EFLAGS: 00000202 ORIG_RAX: ffffffffffffff13
Mar 2 10:32:32 hostname kernel: [48106126.089286] RAX: 00000000d490f313 RBX: 00aaec0d48f6f52c RCX: d490f31300000001
Mar 2 10:32:32 hostname kernel: [48106126.089288] RDX: 00000000d490f313 RSI: 00000000d490f313 RDI: ffffffff90437d80
Mar 2 10:32:32 hostname kernel: [48106126.089289] RBP: 0000000000000000 R08: ffffffff90406040 R09: abcc77118461cefd
Mar 2 10:32:32 hostname kernel: [48106126.089290] R10: 0000000000000030 R11: ffff897ec6a1a3fe R12: 0000000044b2e20e
Mar 2 10:32:32 hostname kernel: [48106126.089291] R13: ffffffff911a5a00 R14: ffff897ec7491400 R15: 00000abc2e236349
Mar 2 10:32:32 hostname kernel: [48106126.089292] FS: 000000c000100090(0000) GS:ffff898e00080000(0000) knlGS:0000000000000000
Mar 2 10:32:32 hostname kernel: [48106126.089294] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Mar 2 10:32:32 hostname kernel: [48106126.089296] CR2: 000055b170280238 CR3: 00000010b9cce002 CR4: 00000000007706e0
Mar 2 10:32:32 hostname kernel: [48106126.089297] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Mar 2 10:32:32 hostname kernel: [48106126.089298] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Mar 2 10:32:32 hostname kernel: [48106126.089299] PKRU: 55555554
Mar 2 10:32:32 hostname kernel: [48106126.089300] Call Trace:
Mar 2 10:32:32 hostname kernel: [48106126.089301] ktime_get+0x3e/0xa0
Mar 2 10:32:32 hostname kernel: [48106126.089306] get_cpu_iowait_time_us+0x3c/0xb0
Mar 2 10:32:32 hostname kernel: [48106126.089311] get_iowait_time.isra.4+0x24/0x40
Mar 2 10:32:32 hostname kernel: [48106126.089315] show_stat+0x3b4/0x6e0
Mar 2 10:32:32 hostname kernel: [48106126.089318] seq_read+0x163/0x420
Mar 2 10:32:32 hostname kernel: [48106126.089323] proc_reg_read+0x39/0x60
Mar 2 10:32:32 hostname kernel: [48106126.089326] vfs_read+0x91/0x140
Mar 2 10:32:32 hostname kernel: [48106126.089330] ksys_read+0x4f/0xb0
Mar 2 10:32:32 hostname kernel: [48106126.089333] do_syscall_64+0x5b/0x1a0
Mar 2 10:32:32 hostname kernel: [48106126.089337] entry_SYSCALL_64_after_hwframe+0x65/0xca
...
- 이러한 소프트 잠금은 클록 소스인 TSC가 불안정해지면서 HPET로 전환되어 발생했을 가능성이 있나요?
Environment
- Red Hat Enterprise Linux 8
Subscriber exclusive content
A Red Hat subscription provides unlimited access to our knowledgebase, tools, and much more.