http://devgrapher.com

1. WinDbg랑 친해지기
김지훈
ZUMinternet
2013. 11. 29.
1

2. WinDbg??
MS에서 제공하는 다목적 Windows용 Debugger
Kernel Mode Debugging
User Mode Debugging
Post-mortem Debugging
2

3. WinDbg는 너무 어려워!
3

4. 4

5. 최대한 간단하게 써보자
5

6. 덤프 분석의 기본
6

7. 넌 이미 죽어있다…
7

8. !analyze –v
FAULTING_IP:
dumptest1!SecondThreadFunc+4f [c:usersdevgrapherdocumentsvisual studio 2010projectsdumptest1dumptest1.cpp @ 15]
00a0146f 031481
add
edx,dword ptr [ecx+eax*4]
EXCEPTION_RECORD: ffffffff -- (.exr 0xffffffffffffffff)
ExceptionAddress: 00a0146f (dumptest1!SecondThreadFunc+0x0000004f)
ExceptionCode: c0000005 (Access violation)
ExceptionFlags: 00000000
NumberParameters: 2
Parameter[0]: 00000000
Parameter[1]: 000d9000
Attempt to read from address 000d9000
DEFAULT_BUCKET_ID: INVALID_POINTER_READ
….
STACK_COMMAND: ~1s; .ecxr ; kb
….
….
8

9. DEFAULT_BUCKET_ID
발생한 오류의 종류
INVALID_POINTER_READ
APPLICATION_HANG
HEAP_CORRUPTION
BAD_INSTRUCTION_PTR
STATUS_BREAKPOINT
….
DEFAULT_BUCKET_ID: INVALID_POINTER_READ
9

10. STACK_COMMAND
오류가 발생한 Stack을 확인하기 위한 명령어 추천
STACK_COMMAND: ~1s; .ecxr ; kb
10

11. ~1s
스레드 컨텍스트 변경(switch)
~ : 스레드 정보 출력
0:001> ~
0 Id: 2060.b34 Suspend: 0 Teb: 7efdd000 Unfrozen
1 Id: 2060.1e04 Suspend: 0 Teb: 7efda000 Unfrozen
~[스레드번호]s : 스레드 컨텍스트 변경
~1s : 1번 스레드로 컨텍스트 변경
11

12. .ecxr (Exception Context Record)
예외가 발생한 컨텍스트로 변경
일단 넘어가자….
12

13. kb
현재 스레드의 Stack 출력
k 커맨드 의 변형 : k[b|p|P|v]
0:001> kb
# ChildEBP RetAddr Args to Child
00 00b1feec 6f85a273 000d87f0 c81c2e95 00000000 dumptest1!SecondThreadFunc+0x4f
01 00b1ff28 6f85a204 000d8c08 00b1ff40 7592336a MSVCR100D!_beginthreadex+0x243
02 00b1ff34 7592336a 000d8c08 00b1ff80 775e9f72 MSVCR100D!_beginthreadex+0x1d4
03 00b1ff40 775e9f72 004c8d80 76f645d5 00000000 kernel32!BaseThreadInitThunk+0xe
04 00b1ff80 775e9f45 6f85a180 004c8d80 00000000 ntdll!__RtlUserThreadStart+0x70
05 00b1ff98 00000000 6f85a180 004c8d80 00000000 ntdll!_RtlUserThreadStart+0x1b
프레임 EBP
번호
리턴 주소
함수 파라메터
함수 심볼
13

14. dv (Display Local Variables)
지역변수 확인
0:001> dv
i = 0n516
param = 0x000d87f0
sum = 0n1997962443
arr = 0x000d87f0
14

15. !address [주소]
[주소] 가 속하는 메모리 영역 정보 출력
0:001> !address 0x000d87f0
Usage:
Heap
Base Address:
000d0000
End Address:
000d9000
Region Size:
00009000
State:
<info not present at the target>
Protect:
<info not present at the target>
Type:
<info not present at the target>
Allocation Base:
<info not present at the target>
Allocation Protect: <info not present at the target>
More info:
heap owning the address: !heap 0xd0000
More info:
heap segment
More info:
heap entry containing the address: !heap -x 0xd87f0
15

16. ?
? 표현식 : 표현식을 평가
0:001> ? 0n520 * 4
Evaluate expression: 2080 = 00000820
? : 도움말 출력
0:001> ?
Open debugger.chm for complete debugger documentation
B[C|D|E][<bps>] - clear/disable/enable breakpoint(s)
BL - list breakpoints
BA <access> <size> <addr> - set processor breakpoint
BP <address> - set soft breakpoint
…..
Menu -> Help -> Index 참조
16

17. Windbg 환경설정
실전덤프 분석은 좀 이따가…
17

18. 유용한 옵션
미리설정된 Workspace 열기
c:program files....windbg.exe -W [workspace 이름]
Hyper Link 활성화
.prefer_dml 1
18

19. 플러그인
blwdbgue.dll
Syntex Highlighting
sdbgext.dll
STL 컨테이너 뷰어
!sdbgext.stlvector, !sdbgext.stllist, !sdbgext.stlmap
…
CMKD.dll
x64 스택 분석
패킷 분석
…
19

20. 바로가기 만들기
"C:Program Files (x86)Windows Kits8.0Debuggersx86windbg.exe"
-W working -c ".prefer_dml 1 ; .load blwdbgue.dll;.load cmkd.dll;.load
sdbgext.dll"
20

21. 깨진 스택을 살려보자
덤프는 다 알고 있다
21

22. Stack Frame 구조
void Print(int sum) {
printf("%d", sum);
}
int Sum(int arg1, int arg2) {
int local = 0;
local = arg1 + arg2;
Print(local);
return local;
}
int main(int argc, char* argv[]) {
EIP -> Sum(3, 4);
return 0;
}
109
105
101
xxxx
<- ESP,EBP
22

23. Stack Frame 구조
void Print(int sum) {
printf("%d", sum);
}
int Sum(int arg1, int arg2) {
int local = 0;
local = arg1 + arg2;
Print(local);
return local;
}
int main(int argc, char* argv[]) {
EIP -> Sum(3, 4);
return 0; <- addr1
}
109
105
addr1
<- ESP
3
4
101
xxxx
<- EBP
23

24. Stack Frame 구조
void Print(int sum) {
printf("%d", sum);
}
int Sum(int arg1, int arg2) {
EIP -> int local = 0;
local = arg1 + arg2;
Print(local);
return local;
}
int main(int argc, char* argv[]) {
Sum(3, 4);
return 0; <- addr1
}
109
105
101
<- ESP,EBP
addr1
3
4
101
xxxx
24

25. Stack Frame 구조
void Print(int sum) {
printf("%d", sum);
}
int Sum(int arg1, int arg2) {
EIP -> int local = 0;
local = arg1 + arg2;
Print(local);
return local;
}
int main(int argc, char* argv[]) {
Sum(3, 4);
return 0; <- addr1
}
109
0
105
<- ESP
101
<- EBP
addr1
3
4
101
xxxx
25

26. Stack Frame 구조
void Print(int sum) {
printf("%d", sum);
}
int Sum(int arg1, int arg2) {
int local = 0;
EIP -> local = arg1 + arg2;
Print(local);
return local;
}
int main(int argc, char* argv[]) {
Sum(3, 4);
return 0; <- addr1
}
109
7
105
<- ESP
101
<- EBP
addr1
3
4
101
xxxx
26

27. Stack Frame 구조
void Print(int sum) {
printf("%d", sum);
}
int Sum(int arg1, int arg2) {
int local = 0;
local = arg1 + arg2;
EIP -> Print(local);
return local; <- addr2
}
int main(int argc, char* argv[]) {
Sum(3, 4);
return 0; <- addr1
}
109
addr2
<- ESP
7
7
105
101
<- EBP
addr1
3
4
101
xxxx
27

28. Stack Frame 구조
void Print(int sum) {
EIP -> printf("%d", sum);
}
int Sum(int arg1, int arg2) {
int local = 0;
local = arg1 + arg2;
Print(local);
return local; <- addr2
}
int main(int argc, char* argv[]) {
Sum(3, 4);
return 0; <- addr1
}
109
105
<- ESP,EBP
addr2
7
7
105
101
addr1
3
4
101
xxxx
28

29. Stack Frame 구조
void Print(int sum) {
EIP -> printf("%d", sum);
}
int Sum(int arg1, int arg2) {
int local = 0;
local = arg1 + arg2;
Print(local);
return local; <- addr2
}
int main(int argc, char* argv[]) {
Sum(3, 4);
return 0; <- addr1
}
Print
frame
109
105
<- ESP,EBP
addr2
7
Sum
frame
7
105
101
addr1
3
Main
frame
4
101
xxxx
29

30. Stack Frame 구조
void Print(int sum) {
printf("%d", sum);
}
int Sum(int arg1, int arg2) {
int local = 0;
local = arg1 + arg2;
Print(local);
EIP -> return local; <- addr2
}
int main(int argc, char* argv[]) {
Sum(3, 4);
return 0; <- addr1
}
109
105
addr2
7
7
105
<- ESP
101
<- EBP
addr1
3
4
101
xxxx
30

31. Stack Frame 구조
void Print(int sum) {
printf("%d", sum);
}
int Sum(int arg1, int arg2) {
int local = 0;
local = arg1 + arg2;
Print(local);
return local; <- addr2
}
int main(int argc, char* argv[]) {
Sum(3, 4);
EIP -> return 0; <- addr1
}
109
105
addr2
7
7
105
101
addr1
3
4
101
xxxx
<- ESP,EBP
31

32. Manual Stack Trace
!teb : Thread Environment Block 정보 출력
스레드 스택 범위 확인 가능
0:000> !teb
TEB at 7ffdf000
ExceptionList:
StackBase:
StackLimit:
…..
001bd958
001c0000
001ae000
32

33. Manual Stack Trace
dds : 지정한 메모리 범위를 4바이트 단위로 일렬로 나열
심볼 연동
스택 범위 입력
dds 001ae000 001c0000
StackBase: 001c0000
StackLimit: 001ae000
스택 범위 추정하여 입력(추천)
dds esp esp+3000 <-임의의값
33

34. Manual Stack Trace
.frame /c =BasePtr StackPtr InstructionPtr
지정한 주소 기반으로 스택 재구성
07f5dff4 07f5e034
07f5dff8 00000000
07f5dffc 00000001
07f5e000 0ae900e8 ShellStreams+0xe8
07f5e004 07f5e020
07f5e008 77bff658 ntdll!RtlImageDirectoryEntryToData+0x1a
07f5e00c 0ae90000 ShellStreams
07f5e010 00000001
.frame /c =07f5e004 07f5e004-8 77bff658
34

35. 실전 덤프 분석
35

36. 상황
어플리케이션의 UI가 행이 걸려서 응답이 없는 상황
재현은 되지 않음
해당 상황에서 프로세스 덤프는 확보 되었음
36

37. Hang 분석 명령어
!analyze -v –hang
Hang 정보 추가 분석
!locks
Lock이 걸린 Critical Section 정보
DERIVED_WAIT_CHAIN:
Dl Eid Cid WaitType
-- --- ------- -------------------------0 1584.1588 Event
0:036> !locks
CritSec nlaapi!gLPCCS+0 at 747bd0bc
WaiterWoken
No
LockCount
0
RecursionCount 1
OwningThread
1588
EntryCount
0
ContentionCount 0
*** Locked
37

38. 무얼 대기하고 있나
0:036> !locks
CritSec ntdll!LdrpLoaderLock+0 at 77c77340
WaiterWoken
No
LockCount
13
RecursionCount 1
OwningThread
13b4
EntryCount
0
ContentionCount 9d
*** Locked
….. 중략 …..
38

39. 어디서 멈춰있나
0:036> .cxr
Resetting default scope
0:036> kb
# ChildEBP RetAddr Args to Child
00 07f5d76c 77be6a64 77bd2278 000020cc 00000000 ntdll!KiFastSystemCallRet
01 07f5d770 77bd2278 000020cc 00000000 00000000 ntdll!NtWaitForSingleObject+0xc
02 07f5d7d4 77bd215c 00000000 00000000 fffffffe ntdll!RtlpWaitOnCriticalSection+0x13e
03 07f5d7fc 777bce18 778192e0 1309a1c8 07f5d944 ntdll!RtlEnterCriticalSection+0x150
04 07f5d848 777bd130 00000020 00000020 00000001 user32!BitmapFromDIB+0x1f
05 07f5d8ac 777bd6db 01000000 00000020 00000020 user32!ConvertDIBBitmap+0x12b
06 07f5db74 777bccbf 0d2d735c 000010a8 0d2a0001 user32!ConvertDIBIcon+0x112
……
39

40. ntdll!RtlEnterCriticalSection
NTSTATUS RtlEnterCriticalSection
(
RTL_CRITICAL_SECTION* crit
);
40

41. 스택 복구
07f5d76c 77be6a64 77bd2278 000020cc 00000000 ntdll!KiFastSystemCallRet
07f5d770 77bd2278 000020cc 00000000 00000000 ntdll!NtWaitForSingleObject+0xc
07f5d7d4 77bd215c 00000000 00000000 fffffffe ntdll!RtlpWaitOnCriticalSection+0x13e
07f5d7fc 777bce18 778192e0 1309a1c8 07f5d944 ntdll!RtlEnterCriticalSection+0x150
07f5d848 777bd130 00000020 00000020 00000001 user32!BitmapFromDIB+0x1f
07f5d8ac 777bd6db 01000000 00000020 00000020 user32!ConvertDIBBitmap+0x12b
……
Shellstreams!xxxxxx
……
07f5e21c 77c02322 07f5e27c 07f5e248 00000000 ntdll!LdrpLoadDll+0x4d1
07f5e250 75c98c19 110c87ac 07f5e294 07f5e27c ntdll!LdrLoadDll+0x92
07f5e28c 75f69d43 00000000 00000000 110c87ac KERNELBASE!LoadLibraryExW+0x1d3
07f5e2a8 75f69cc7 00000000 07f5e324 00000008 ole32!LoadLibraryWithLogging+0x16
……
07f5fb2c 00000000 77bd03e9 002a4168 00000000 ntdll!_RtlUserThreadStart+0x1b
41

42. LoadLibrary
0:036> .frame 6
06 07f5e2a8 75f69cc7 ole32!LoadLibraryWithLogging+0x16
0:036> dv
why = LoadingInprocServer (0n0)
pwszFileName = 0x07f5e324 "C:Program FilesCommon
FilesAppleInternet ServicesShellStreams.dll"
dwFlags = 8
phMod = 0x07f5e2f8
42

43. 또 다른 Lock
0:036> !locks
….. 중략 …..
CritSec user32!gcsHdc+0 at 778192e0
WaiterWoken
No
LockCount
1
RecursionCount 1
OwningThread
1340
EntryCount
0
ContentionCount 1
*** Locked
….. 중략 ……
43

44. Lock이 걸린 다른 스레드
41 Id: 1584.1340 Suspend: 0 Teb: 7ff85000 Unfrozen
# ChildEBP RetAddr Args to Child
00 0d45d694 77be6a64 77bd2278 000000f0 00000000 ntdll!KiFastSystemCallRet
01 0d45d698 77bd2278 000000f0 00000000 00000000 ntdll!NtWaitForSingleObject+0xc
02 0d45d6fc 77bd215c 00000000 00000000 0d45d764 ntdll!RtlpWaitOnCriticalSection+0x13e
03 0d45d724 77bffa13 77c77340 7a8035dd 761255ec ntdll!RtlEnterCriticalSection+0x150
04 0d45d7c0 77c0229b 77060000 0d45d7fc 00000000 ntdll!LdrGetProcedureAddressEx+0x159
05 0d45d7dc 75c96d18 77060000 0d45d7fc 00000000 ntdll!LdrGetProcedureAddress+0x18
06 0d45d804 76134476 77060000 761255ec 0d45d85c KERNELBASE!GetProcAddress+0x44
07 0d45d814 761343d6 0b011d42 0000000f 0000000c usp10!GetFontAssocStatusInit+0x16
08 0d45d85c 7612794e 000020a2 00002000 0b011d42 usp10!IsFontAssociated+0xe6
…..중략 …..
44

45. Thread 36> 13b4
ntdll!KiFastSystemCallRet
ntdll!NtWaitForSingleObject
ntdll!RtlpWaitOnCriticalSection
ntdll!RtlEnterCriticalSection
user32!BitmapFromDIB
CS
user32!ConvertDIBBitmap
user32!ConvertDIBIcon
77c77340
……
ShellStreams!xxxxxx
778192e0
……
ntdll!LdrpHandleTlsData
.….
ntdll!LdrpLoadDll
ntdll!LdrLoadDll
KERNELBASE!LoadLibraryExW
ole32!LoadLibraryWithLogging
ole32!CClassCache::CDllPathEntry::LoadDll
ole32!CClassCache::CDllPathEntry::Create_rl
ole32!CClassCache::CClassEntry::CreateDllClassEntry_rl
Thread 41> 1340
user32?
ntdll!KiFastSystemCallRet
ntdll!NtWaitForSingleObject
ntdll!RtlpWaitOnCriticalSection
ntdll!RtlEnterCriticalSection
ntdll!LdrGetProcedureAddressEx
ntdll!LdrGetProcedureAddress
KERNELBASE!GetProcAddress
usp10!GetFontAssocStatusInit
usp10!IsFontAssociated
usp10!ScriptStringAnalyse
lpk!LpkStringAnalyse
lpk!LpkCharsetDraw
lpk!LpkDrawTextEx
user32!DT_DrawStr
user32!DT_GetLineBreak
user32!DrawTextExWorker
user32!DrawTextExW
user32!DrawTextW
45

46. 그래서 나쁜놈은 누구?
0:036> lmvm shellstreams
Browse full module list
start end
module name
0ae90000 0af63000 ShellStreams (export symbols)
ShellStreams.dll
Loaded symbol image file: ShellStreams.dll
Image path: C:Program FilesCommon FilesAppleInternet ServicesShellStreams.dll
Image name: ShellStreams.dll
Browse all global symbols functions data
Timestamp:
Thu Mar 21 04:17:25 2013 (514A0B45)
CheckSum:
000AB9E6
ImageSize:
000D3000
File version: 7.7.1.4
Product version: 0.0.0.0
File flags:
0 (Mask 0)
File OS:
0 Unknown Base
File type:
0.0 Unknown
File date:
00000000.00000000
Translations: 0000.0000
CompanyName: Apple Inc.
OriginalFilename: ShellStreams.dll
FileVersion: 7.7.1.4
FileDescription: ShellStreams.dll
LegalCopyright: © 2013 Apple Inc. All rights reserved.
46

47. 지역변수가 안나와요
바이너리가 최적화된경우
47

48. 최적화된 지역변수
unsigned __stdcall SecondThreadFunc(void* param)
{
int* arr = (int*)param;
int sum = 0;
for (int i = 0; i < 1000; ++i) {
sum += arr[i];
}
std::cout<< sum << std::endl;
_endthreadex(0);
return 0;
}
0:001> dv
param = 0x006c7be0
딸랑 이거밖에 없어?
48

49. 별 수 없어요…
그냥 어셈 보세요
49

50. 지역변수가 안나오는 이유
스택에 존재하지 않는다.
컴파일 타임에 최적화된 경우
이 경우 보통 레지스터가 들고 있다.
그러니 어셈 보세요….
힌트!!
this 포인터 : ecx
리턴값 : eax
50

51. 기타 유용한 커맨드
51

52. .reload /i
심볼이 안맞을때 강제로 맞추기
빌드 환경과 소스가 같다면 거의 같은 pdb가 생성되므로 강제로 매치시킬 수 있음
0:001> kc
WARNING: Frame IP not in any known module. Following frames may be wrong.
00 0x0
01 MSVCR100D!_beginthreadex
02 MSVCR100D!_beginthreadex
0:001> .reload /i dumptest1.exe
*** WARNING: Unable to verify checksum for dumptest1.exe
0:001> kc
00 dumptest1!SecondThreadFunc
01 MSVCR100D!_beginthreadex
02 MSVCR100D!_beginthreadex
52

53. 스택 문자열 뒤지기
ddu(UNICODE)
dda(ASCII)
du [주소]
da [주소]
0:021> ddu esp esp+1000
1ceffaa8 007e7498 "둼ȿen‫.ﬠ‬On.ʆitiali4"
1ceffaac 0077e800 "C:Documents and Settingsezadmin바탕
화면dotnetfx.exe"
1ceffab0 1ceffab4 ""
1ceffab4 2e800000 "婍.."
1ceffab8 0166bec0 ""
1ceffabc 0238e566 "‫"ڹ‬
1ceffac0 007e7498 "둼ȿen‫.ﬠ‬On.ʆitiali4"
1ceffac4 1ceffc20 "靖..w‫.ﳀ‬胠粖齰粖4"
1ceffac8 1ceffb38 ".ʆ"
1ceffacc b1f482ac
1ceffad0 024164c8 ".ȿ"
1ceffad4 007620d0 "C:Documents and Settingsezadmin바탕
화면dotnetfx.exe"
1ceffad8 0074a7dc ".t.t.t뱸u(c)97 "
53

54. assembly 보기
0:001> uf SecondThreadFunc
u, ub, uf [주소]
dumptest1!SecondThreadFunc
10 01341420 55
push ebp
10 01341421 8bec
mov ebp,esp
…..
dumptest1!SecondThreadFunc+0x34
13 01341454 8b45e0
mov eax,dword ptr [ebp-20h]
13 01341457 83c001
add eax,1
13 0134145a 8945e0
mov dword ptr [ebp-20h],eax
dumptest1!SecondThreadFunc+0x3d
13 0134145d 817de0e8030000 cmp dword ptr [ebp-20h],3E8h
13 01341464 7d11
jge dumptest1!SecondThreadFunc+0x57 (01341477) Branch
dumptest1!SecondThreadFunc+0x46
15 01341466 8b45e0
mov eax,dword ptr [ebp-20h]
15 01341469 8b4df8
mov ecx,dword ptr [ebp-8]
15 0134146c 8b55ec
mov edx,dword ptr [ebp-14h]
54

55. 감사합니다.
55