2019 臺灣資安大會 (CYBERSEC 2019) 03/21

1. AI 的逆襲，
眼⾒見見真的為憑嗎？
台灣資安⼤大會 iThome CYBERSEC 2019
Turkey & Melody

2. 台灣駭客協會 秘書長
HITCON GIRLS 共同創辦⼈人
台灣駭客協會 專案經理理
HITCON GIRLS 成員
ABOUT US
2
Turkey Melody

3. OUTLINE
3
• Deepfakes
• Lyrebirds
• 應⽤用
• 反制技術

4. DEEPFAKES MAKES IT POSSIBLE TO
CREATE FAKE VIDEOS OF ALMOST
EVERYONE.
4

5. 5
LET’S TALK ABOUT DEEPFAKES!
1. In December 2017, a user
named “DeepFakes”
announce the tool on
Reddit Community of
developers.
2. DeepFakes is a tool that
utilizes deep learning to
recognize and swap faces
in pictures and videos.
5

6. 6
HOW DEEPFAKES WORK?

7. HOW DEEPFAKES WORK?
https://github.com/iperov/DeepFaceLab
7

8. HOW DEEPFAKES WORK?
>DeepFakes Tools
1. Fakeapp
2. DeepFaceLab
3. Fakeswap
4. Openfaceswap
5. Myfakeapp
8

9. HOW DEEPFAKES WORK?
>DeepFakes Tools
1. Fakeapp
2.DeepFaceLab
3. Fakeswap
4. Openfaceswap
5. Myfakeapp
>
9

10. Source Target
10

11. HOW DEEPFAKES WORK?
>Steps for: DeepFaceLab
2 31
Extract image
from source
video.
Extract faces
from source
and target
image.
4
Manually
remove
error image.
5
Model
Training
6
Extract image
from target
video.
Debug & Convert
image to mp4
11

12. HOW DEEPFAKES WORK?
>Extract image from source＆target video.
FFmpeg
FFmpeg is the leading
multimedia framework,
able to decode, encode,
transcode, mux, demux,
stream, filter and play
pretty much anything
that humans and
machines have created.
ffmpeg -i clipname -vf fps=framerate -qscale:v 2 "imagename%04d.jpg"Command_
12

13. HOW DEEPFAKES WORK?
>Extract faces from source and target image
（Ref: Facial landmarks with dlib, OpenCV, and Python）
Divided a face into the following areas:
1. eyes(left/right)
2. eyebrows(left/right)
3. nose
4. chin
13

14. HOW DEEPFAKES WORK?
>Extract faces from source and target image
（Ref: Real-Time Face Pose Estimation ）
14

15. HOW DEEPFAKES WORK?
>Extract faces from source and target image
15

16. HOW DEEPFAKES WORK?
>Extract faces from source and target image
（Ref: One Millisecond Face Alignment with an Ensemble of Regression Trees）16

17. HOW DEEPFAKES WORK?
>Manually remove error image
17

18. HOW DEEPFAKES WORK?
>Manually remove error image
18

19. HOW DEEPFAKES WORK?
>Manually remove error image
19

20. HOW DEEPFAKES WORK?
>Model Training
(Ref: https://deepfakes.com.cn/)20

21. HOW DEEPFAKES WORK?
>Model Training
(Ref: https://deepfakes.com.cn/)21

22. HOW DEEPFAKES WORK?
>Model Training
(Ref: https://deepfakes.com.cn/)22

23. HOW DEEPFAKES WORK?
>Model Training
Models Types:
1. H64
2. H128
3. DF
4. LIAEF
5. LIAEF128YAW
6. MIAEF128
7. AVATAR
8. SAE H12823

24. HOW DEEPFAKES WORK?
>Model Training
Models Types:
1. H64
2. H128
3. DF
4. LIAEF
5. LIAEF128YAW
6. MIAEF128
7. AVATAR
8. SAE H128
Train 越久效果越好！
24

25. HOW DEEPFAKES WORK?
>Debug & Convert image to mp4
Debug Mode
Use predicted mask? 1
Erosion (-100 to +100): (default = 0) 1
Seamless Erosion (0 to 40): (default = 0) 20
Blur (-200 to +200): (default = 0) 40
Hist-match threshold (0 to 255): (default = 255) default
Face Scale (-50 to +50): (default = 0) default
Transfer Color from predicted face? (LCT/RCT/no): (default = no) RCT
Degrade Color Power of Final Image: (default = 0) default
25

26. HOW DEEPFAKES WORK?
>Debug & Convert image to mp4
Debug Mode
Use predicted mask? 1
Erosion (-100 to +100): (default = 0) 1
Seamless Erosion (0 to 40): (default = 0) 20
Blur (-200 to +200): (default = 0) 40
Hist-match threshold (0 to 255): (default = 255) default
Face Scale (-50 to +50): (default = 0) default
Transfer Color from predicted face? (LCT/RCT/no): (default = no) RCT
Degrade Color Power of Final Image: (default = 0) default
26

27. HOW DEEPFAKES WORK?
>Debug & Convert image to mp4
Debug Mode
Use predicted mask? 1
Erosion (-100 to +100): (default = 0) 1
Seamless Erosion (0 to 40): (default = 0) 20
Blur (-200 to +200): (default = 0) 40
Hist-match threshold (0 to 255): (default = 255) default
Face Scale (-50 to +50): (default = 0) default
Transfer Color from predicted face? (LCT/RCT/no): (default = no) RCT
Degrade Color Power of Final Image: (default = 0) default
27

28. HOW DEEPFAKES WORK?
>Debug & Convert image to mp4
Debug Mode
Use predicted mask? 1
Erosion (-100 to +100): (default = 0) 1
Seamless Erosion (0 to 40): (default = 0) 20
Blur (-200 to +200): (default = 0) 40
Hist-match threshold (0 to 255): (default = 255) default
Face Scale (-50 to +50): (default = 0) default
Transfer Color from predicted face? (LCT/RCT/no): (default = no) RCT
Degrade Color Power of Final Image: (default = 0) default
28

29. HOW DEEPFAKES WORK?
>Debug & Convert image to mp4
Debug Mode
Use predicted mask? 1
Erosion (-100 to +100): (default = 0) 1
Seamless Erosion (0 to 40): (default = 0) 20
Blur (-200 to +200): (default = 0) 40
Hist-match threshold (0 to 255): (default = 255) default
Face Scale (-50 to +50): (default = 0) default
Transfer Color from predicted face? (LCT/RCT/no): (default = no) RCT
Degrade Color Power of Final Image: (default = 0) default
29

30. HOW DEEPFAKES WORK?
>Debug & Convert image to mp4
Debug Mode
Use predicted mask? 1
Erosion (-100 to +100): (default = 0) 1
Seamless Erosion (0 to 40): (default = 0) 20
Blur (-200 to +200): (default = 0) 40
Hist-match threshold (0 to 255): (default = 255) default
Face Scale (-50 to +50): (default = 0) default
Transfer Color from predicted face? (LCT/RCT/no): (default = no) RCT
Degrade Color Power of Final Image: (default = 0) default
30

31. HOW DEEPFAKES WORK?
>Debug & Convert image to mp4
Debug Mode
Use predicted mask? 1
Erosion (-100 to +100): (default = 0) 1
Seamless Erosion (0 to 40): (default = 0) 20
Blur (-200 to +200): (default = 0) 40
Hist-match threshold (0 to 255): (default = 255) default
Face Scale (-50 to +50): (default = 0) default
Transfer Color from predicted face? (LCT/RCT/no): (default = no) RCT
Degrade Color Power of Final Image: (default = 0) default
31

32. HOW DEEPFAKES WORK?
>Debug & Convert image to mp4
ffmpeg -f image2 -i imagename%04d.jpg -vcodec libx264 -crf 15 -pix_fmt yuv420pCommand_
32

33. (Downlink: https://github.com/iperov/DeepFaceLab)
HOW DEEPFAKES WORK?
>Download DeepFaceLab
33

34. 34
LET’S DEMO !

35. HOW DEEPFAKES WORK?
>⼩小插曲
實測跑評：RTX 2080 ⼤大約需跑 6 個⼩小時以上會有不錯的成果
作者跪求 RTX
2080 Ti 中 XD
35

36. HOW DEEPFAKES WORK?
>不要讓貧窮限制你的 AI
＃讓 AI 聰明前
先飽滿你的⼝口袋
36

37. Source Target
37

38. 38
（Link:https://youtu.be/yfsdZ5gLuOo）

39. CAN AI MIMIC YOUR VOICE?
39

40. CAN YOU TELL A FAKE VOICE FROM A REAL ONE?
40
（Ref: https://lyrebird.ai/）

41. • create a digitized version of your own voice.
• Uses deep learning frameworks on AWS to develop and
train its AI models
What is Lyrebird?
Record & Upload
voice samples
Train by DL Framework
on AWS EC2 P3
Digital Voice
Generated
Optput audio of
requested dialog
（Ref: https://lyrebird.ai/）41

42. LYREBIRD LIVE DEMO
This is my honored to talk to everyone here.
However, can you trust your eyes and ears to perceive reality?
Machine learning can artificially mimic natural sounds to create digital voice.
Just like me!
42

43. Ｗhy my voice sounds awful?
43
（Ref: https://lyrebird.ai/）

44. Voice Cloning Market
Generation Is Coming
44

45. CHALLENGES FOR OUR FUTURE
45

46. Why do you believe what
I am telling you now?
NEGATIVE EFFECTS
46

47. POSITIVE EFFECTS
Create images and voices
from the past.
47
（Ref:apple）
（Ref:Fast & Furious 6）

48. （Ref:https://www.youtube.com/watch?v=5iZuffHPDAw）

49. FIGHT AGAINST FAKE VIDEOS!
49

50. FAKE VIDEO DETECT SOLUTION
>Detect malicious alterations with AI
•abnormal compression signatures
•lip sync analysis
•Video metadata analysis
•noise patterns analysis
50

51. FAKE VIDEO DETECT SOLUTION
> Fingerprint source videos & track
provenance with blockchain smart contracts
Create Hashing Blackchan record submitt
51

52. REFERENCE
• https://deepfakes.com.cn/
• https://www.youtube.com/watch?v=K98nTNjXkq8
• ttps://github.com/Fabsqrt/BitTigerLab
• https://github.com/iperov/DeepFaceLab
• https://www.youtube.com/watch?v=CR5Jr6Z1KI
• https://lyrebird.ai/
• https://www.darpa.mil/
• https://medium.com/amber-video
52

53. THANKS!
53