call trace 관련 log 분석
페이지 정보
본문
PM 작업을 위한 재부팅 시도 후 시스템 hang으로 인새 서버 강제 종료,
후 call trace 관련 log의 분석 및 원인파악 문의
관련 로그
-------------------------------------------------------------------------------------
Jul 1 00:27:00 npvlapp1 kernel: INFO: task sh:6179 blocked for more than 120 seconds.
Jul 1 00:27:00 npvlapp1 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Jul 1 00:27:00 npvlapp1 kernel: sh D 0000000000000004 0 6179 6149 0x00000080
Jul 1 00:27:00 npvlapp1 kernel: ffff88041b8f7ce8 0000000000000082 ffff8804369ccae0 ffff8804369ccae0
Jul 1 00:27:00 npvlapp1 kernel: ffff8804369ccae0 ffff880836c92380 ffff8804369ccae0 0000010100000070
Jul 1 00:27:00 npvlapp1 kernel: ffff8804369cd098 ffff88041b8f7fd8 000000000000fb88 ffff8804369cd098
Jul 1 00:27:00 npvlapp1 kernel: Call Trace:
Jul 1 00:27:00 npvlapp1 kernel: [<ffffffffa009ebf0>] ? ext4_file_open+0x0/0x130 [ext4]
Jul 1 00:27:00 npvlapp1 kernel: [<ffffffff8150e555>] schedule_timeout+0x215/0x2e0
Jul 1 00:27:00 npvlapp1 kernel: [<ffffffff8117e434>] ? nameidata_to_filp+0x54/0x70
Jul 1 00:27:00 npvlapp1 kernel: [<ffffffff812771e9>] ? cpumask_next_and+0x29/0x50
Jul 1 00:27:00 npvlapp1 kernel: [<ffffffff8150e1d3>] wait_for_common+0x123/0x180
Jul 1 00:27:00 npvlapp1 kernel: [<ffffffff81063310>] ? default_wake_function+0x0/0x20
Jul 1 00:27:00 npvlapp1 kernel: [<ffffffff8150e2ed>] wait_for_completion+0x1d/0x20
Jul 1 00:27:00 npvlapp1 kernel: [<ffffffff8106513c>] sched_exec+0xdc/0xe0
Jul 1 00:27:00 npvlapp1 kernel: [<ffffffff81189fc0>] do_execve+0xe0/0x2c0
Jul 1 00:27:00 npvlapp1 kernel: [<ffffffff810095ea>] sys_execve+0x4a/0x80
Jul 1 00:27:00 npvlapp1 kernel: [<ffffffff8100b4ca>] stub_execve+0x6a/0xc0
-------------------------------------------------------------------------------------
해당 시스템에서 발생한 이슈와 동일한 사례가 있어 해당 아래 문서에 따라 업데이트 후 해결
[1] Servers with Intel® Xeon® Processor E5, Intel® Xeon® Processor E5 v2, or Intel® Xeon® Processor E7 v2 and certain versions of Red Hat Enterprise Linux 6 kernels
become unresponsive/hung or incur a kernel panic
https://access.redhat.com/solutions/433883
후 call trace 관련 log의 분석 및 원인파악 문의
관련 로그
-------------------------------------------------------------------------------------
Jul 1 00:27:00 npvlapp1 kernel: INFO: task sh:6179 blocked for more than 120 seconds.
Jul 1 00:27:00 npvlapp1 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Jul 1 00:27:00 npvlapp1 kernel: sh D 0000000000000004 0 6179 6149 0x00000080
Jul 1 00:27:00 npvlapp1 kernel: ffff88041b8f7ce8 0000000000000082 ffff8804369ccae0 ffff8804369ccae0
Jul 1 00:27:00 npvlapp1 kernel: ffff8804369ccae0 ffff880836c92380 ffff8804369ccae0 0000010100000070
Jul 1 00:27:00 npvlapp1 kernel: ffff8804369cd098 ffff88041b8f7fd8 000000000000fb88 ffff8804369cd098
Jul 1 00:27:00 npvlapp1 kernel: Call Trace:
Jul 1 00:27:00 npvlapp1 kernel: [<ffffffffa009ebf0>] ? ext4_file_open+0x0/0x130 [ext4]
Jul 1 00:27:00 npvlapp1 kernel: [<ffffffff8150e555>] schedule_timeout+0x215/0x2e0
Jul 1 00:27:00 npvlapp1 kernel: [<ffffffff8117e434>] ? nameidata_to_filp+0x54/0x70
Jul 1 00:27:00 npvlapp1 kernel: [<ffffffff812771e9>] ? cpumask_next_and+0x29/0x50
Jul 1 00:27:00 npvlapp1 kernel: [<ffffffff8150e1d3>] wait_for_common+0x123/0x180
Jul 1 00:27:00 npvlapp1 kernel: [<ffffffff81063310>] ? default_wake_function+0x0/0x20
Jul 1 00:27:00 npvlapp1 kernel: [<ffffffff8150e2ed>] wait_for_completion+0x1d/0x20
Jul 1 00:27:00 npvlapp1 kernel: [<ffffffff8106513c>] sched_exec+0xdc/0xe0
Jul 1 00:27:00 npvlapp1 kernel: [<ffffffff81189fc0>] do_execve+0xe0/0x2c0
Jul 1 00:27:00 npvlapp1 kernel: [<ffffffff810095ea>] sys_execve+0x4a/0x80
Jul 1 00:27:00 npvlapp1 kernel: [<ffffffff8100b4ca>] stub_execve+0x6a/0xc0
-------------------------------------------------------------------------------------
해당 시스템에서 발생한 이슈와 동일한 사례가 있어 해당 아래 문서에 따라 업데이트 후 해결
[1] Servers with Intel® Xeon® Processor E5, Intel® Xeon® Processor E5 v2, or Intel® Xeon® Processor E7 v2 and certain versions of Red Hat Enterprise Linux 6 kernels
become unresponsive/hung or incur a kernel panic
https://access.redhat.com/solutions/433883
- 이전글시스템 구동 중 Thread Hangup 발생 이슈 20.10.20
- 다음글paged in/out, swapping 문의 20.10.20
댓글목록
유령님의 댓글
유령 작성일
Call Trace 가 주기적으로 발생하는 경우 kdump 를 생성 후 vmcore 파일을 분석을 하면 더 좋습니다.
방법은 아래 링크 참고 바랍니다.
https://access.redhat.com/solutions/767753