A miało być tak... bez wycieków

1. A miało być tak… bez wycieków! Konrad Kokosa

3. Jeden bajt

6. Pamięć po prostu jest?

7. Kiedyś

9. Kiedyś to było, kurła…

10. Teraz…

12. “MySQL instances created before April 2014 used an ext3 filesystem which has a 2TB file size limit” Brian Donohue @ Instapaper

14. “All non-trivial abstractions, to some degree, are leaky.”

15. Pamięć vs CPU

16. https://mechanical-sympathy.blogspot.com/2013/02/cpu-cache-flushing-fallacy.html

17. “Latency Numbers Every Programmer Should Know” https://gist.github.com/jboner/2841832 Norvig’s Nowadays (Haswell i7 @2.3Ghz) L1 cache reference 0.5 ns <2.0 ns L2 cache reference 7.0 ns 4.8 ns L3 cache reference 14.4 ns Main memory reference 100 ns 71.4 ns Disk seek 10 000 000 ns 150 000 ns

18. Hardware - ISA - System - Framework (JVM, CLR, Go, …) - Programista

19. int n = 5000; int m = 5000; int[,] tab = new int[n, m]; for (int i = 0; i < n; ++i) { for (int j = 0; j < m; ++j) { tab[j, i] = 1; } } int n = 5000; int m = 5000; int[,] tab = new int[n, m]; for (int i = 0; i < n; ++i) { for (int j = 0; j < m; ++j) { tab[i, j] = 1; } }

20. Method | Size | Mean | Error | StdDev | --------- |----- |---------------:|--------------:|--------------:| IJAccess | 50 | 3.350 us | 0.0435 us | 0.0407 us | JIAccess | 50 | 3.372 us | 0.0491 us | 0.0460 us |

21. Method | Size | Mean | Error | StdDev | --------- |----- |---------------:|--------------:|--------------:| IJAccess | 50 | 3.350 us | 0.0435 us | 0.0407 us | JIAccess | 50 | 3.372 us | 0.0491 us | 0.0460 us | IJAccess | 500 | 459.295 us | 8.8558 us | 7.3950 us | JIAccess | 500 | 616.643 us | 11.3232 us | 10.0377 us |

22. Method | Size | Mean | Error | StdDev | --------- |----- |---------------:|--------------:|--------------:| IJAccess | 50 | 3.350 us | 0.0435 us | 0.0407 us | JIAccess | 50 | 3.372 us | 0.0491 us | 0.0460 us | IJAccess | 500 | 459.295 us | 8.8558 us | 7.3950 us | JIAccess | 500 | 616.643 us | 11.3232 us | 10.0377 us | IJAccess | 5000 | 48,728.263 us | 273.4506 us | 255.7859 us | JIAccess | 5000 | 243,778.158 us | 6,232.9454 us | 5,525.3433 us |

23. public static int[] counters = new int[4]; ... Thread[] workers = new Thread[4]; for (int i = 0; i < 4; ++i) { workers[i] = new Thread(idx => { int index = (int)idx; for (int j = 0; j < 100_000_000; ++j) { counters[index] = counters[index] + 1; } }); }

25. 64B (cache-line)

31. Method | Mean | Error | StdDev | -------------- |---------:|----------:|----------:| DoSharingTest | 925.7 ms | 19.233 ms | 46.816 ms | DoSharingTest2 | 338.4 ms | 9.131 ms | 26.779 ms | DoSharingTest3 | 166.8 ms | 1.732 ms | 1.536 ms |

32. Data Oriented Design

33. input -> data transformation -> output “Problem Oriented Design” with an efficient data transformations

34. Array of Structs vs Struct of Arrays

35. class CustomerClassRepository { List<Customer> customers = new List<Customer>(); public void UpdateScorings() { foreach (var customer in customers) { customer.UpdateScoring(); } } } public class Customer { private double Earnings; private DateTime DateOfBirth; private bool IsSmoking; private double Scoring; private HealthData Health; private AuxiliaryData Auxiliary; private Company Employer; public void UpdateScoring() { Scoring = Earnings * (IsSmoking ? 0.8 : 1.0) * ProcessAge(DateOfBirth); } private double ProcessAge(DateTime dateOfBirth) => dateOfBirth.Year / 2048.0; }

36. class Program { // NuGet package ObjectLayoutInspector static void Main(string[] args) { TypeLayout.PrintLayout(typeof(Customer)); Console.ReadLine(); } public class Customer { private double Earnings; private DateTime DateOfBirth; private bool IsSmoking; private double Scoring; private HealthData Health; private AuxiliaryData Auxiliary; private Company Employer; } } public void UpdateScoring() { Scoring = Earnings * (IsSmoking ? 0.8 : 1.0) * ProcessAge(DateOfBirth); } Size: 56 bytes. Paddings: 7 bytes (%12 of empty space) |==========================================| | Object Header (8 bytes) | |------------------------------------------| | Method Table Ptr (8 bytes) | |==========================================| | 0-7: HealthData Health (8 bytes) | |------------------------------------------| | 8-15: AuxiliaryData Auxiliary (8 bytes) | |------------------------------------------| | 16-23: Company Employer (8 bytes) | |------------------------------------------| | 24-31: Double Earnings (8 bytes) | |------------------------------------------| | 32-39: Double Scoring (8 bytes) | |------------------------------------------| | 40: Boolean IsSmoking (1 byte) | |------------------------------------------| | 41-47: padding (7 bytes) | |------------------------------------------| | 48-55: DateTime DateOfBirth (8 bytes) | |==========================================|Cacheline#1acheline2

37. Method | Mean | Allocated | LlcMisses/Op | LlcReference/Op | ------------- |-----------:|----------:|-------------:|----------------:| ObjectStyle | 2,152.5 us | 0 B | 24680 | 30031 |

38. class CustomerRepositoryDOD { int NumberOfCustomers; double[] Scoring; double[] Earnings; bool[] IsSmoking; int[] YearOfBirth; DateTime[] DateOfBirth; // ... public void UpdateScorings() { for (int i = 0; i < NumberOfCustomers; ++i) { Scoring[i] = Earnings[i] * (IsSmoking[i] ? 0.8 : 1.0) * ProcessAge(YearOfBirth[i]); } } public double ProcessAge(int yearOfBirth) => yearOfBirth / 2048.0; } // No any Customer!!!

39. Method | Mean | Allocated | LlcMisses/Op | LlcReference/Op | ------------- |-----------:|----------:|-------------:|----------------:| ObjectStyle | 2,152.5 us | 0 B | 24680 | 30031 | DoDStyle | 213.9 us | 0 B | 272 | 816 |

40. Entity Component System Przykłady: Unity3D ECS, Entitas…

41. The Garbage Collector

42. [HttpGet] [Route("values/concatenated/{count}")] public string GetConcatenated(int count) { Random rand = new Random(); string result = string.Empty; for (int i = 0; i <= count; i++) { result += "<Customer Id=""; result += i.ToString(); result += "" lastUpdateDate=""; result += DateTime.Now.ToString(); result += "" branchId=""; result += i.ToString(); result += "" firstName=""; result += i.ToString(); ; result += "" lastName=""; result += "This is the customer with the Id: "; result += i.ToString(); result += "" ranking=""; result += rand.Next(100).ToString(); result += ""/>"; } result = "<Customers>" + result + "</Customers>"; return result; } [HttpGet] [Route("values/builder/{count}")] public string GetBuilder(int count) { Random rand = new Random(); StringBuilder sb = new StringBuilder("<Customers>"); for (int i = 0; i <= count; i++) { sb.Append("<Customer Id=""); sb.Append(i.ToString()); sb.Append("" lastUpdateDate=""); sb.Append(DateTime.Now.ToString()); sb.Append("" branchId=""); sb.Append(i.ToString()); sb.Append("" firstName=""); sb.Append(i.ToString()); sb.Append("" lastName=""); sb.Append("This is the customer with the Id: "); sb.Append(i.ToString()); sb.Append("" ranking=""); sb.Append(rand.Next(100).ToString()); sb.Append(""/>"); } sb.Append("</Customers>"); return sb.ToString(); }

46. All non-trivial abstractions, to some degree, are leaky.

52. -server -Xss4096k -Xms12G -Xmx12G -XX:MaxPermSize=512m -XX:+HeapDumpOnOutOfMemoryError -verbose:gc -Xmaxf1 -XX:+UseCompressedOops -XX:+DisableExplicitGC -XX:+AggressiveOpts -XX:+ScavengeBeforeFullGC -XX:CMSFullGCsBeforeCompaction=10 -XX:CMSInitiatingOccupancyFraction=80 -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:+CMSIncrementalMode -XX:+CMSIncrementalPacing -XX:+CMSParallelRemarkEnabled -XX:GCTimeRatio=19 -XX:+UseAdaptiveSizePolicy -XX:MaxGCPauseMillis=500 -XX:+PrintGCTaskTimeStamps -XX:+PrintGCApplicationStoppedTime -XX:+PrintHeapAtGC -XX:+PrintTenuringDistribution -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCApplicationConcurrentTime -XX:+PrintTenuringDistribution -Xloggc:gc.log -server -Xms24G -Xmx24G -XX:PermSize=512m -XX:+UseG1GC -XX:MaxGCPauseMillis=200 -XX:ParallelGCThreads=20 -XX:ConcGCThreads=5 -XX:InitiatingHeapOccupancyPercent=70

53. Czy to musi tak być…?

54. From: k...@rational.com (Kent Mitchell) Subject: Re: Does memory leak? Date: 1995/03/31 Norman H. Cohen (nco...@watson.ibm.com) wrote: : The only programs I know of with deliberate memory leaks are those whose : executions are short enough, and whose target machines have enough : virtual memory space, that running out of memory is not a concern. : (This class of programs includes many student programming exercises and : some simple applets and utilities; it includes few if any embedded or : safety-critical programs.) This sparked an interesting memory for me. I was once working with a customer who was producing on-board software for a missile. In my analysis of the code, I pointed out that they had a number of problems with storage leaks. Imagine my surprise when the customers chief software engineer said "Of course it leaks". He went on to point out that they had calculated the amount of memory the application would leak in the total possible flight time for the missile and then doubled that number. They added this much additional memory to the hardware to "support" the leaks. Since the missile will explode when it hits its target or at the end of its flight, the ultimate in garbage collection is performed without programmer intervention.

61. Epsilon GC

62. .NET Core 2.1+: COMPlus_GCName=F:CoreCLR.ZeroGCx64ReleaseOurCustomGC.dll

64. A czy można inaczej…?

66. Witaj Rust!

67. let s1 = String::from("hello"); let s2 = s1; println!("{}, world!", s1); error[E0382]: use of moved value: `s1` --> src/main.rs:5:28 | 3 | let s2 = s1; | -- value moved here 4 | 5 | println!("{}, world!", s1); | ^^ value used here after move | = note: move occurs because `s1` has type `std::string::String`, which does not implement the `Copy` trait

68. fn main() { let s = String::from("hello"); // s comes into scope takes_ownership(s); // s's value moves into the function... // ... and so is no longer valid here } fn takes_ownership(some_string: String) { // some_string comes into scope println!("{}", some_string); } // Here, some_string goes out of scope and `drop` is called. The backing // memory is freed.

69. fn main() { let s = String::from("hello"); use(&s); } fn use(some_string: &String) { // … }

70. fn main() { let s = String::from("hello"); change(&s); } fn change(some_string: &String) { some_string.push_str(", world"); } error[E0596]: cannot borrow immutable borrowed content `*some_string` as mutable --> error.rs:8:5 | 7 | fn change(some_string: &String) { | ------- use `&mut String` here to make mutable 8 | some_string.push_str(", world"); | ^^^^^^^^^^^ cannot borrow as mutable

71. fn main() { let mut s = String::from("hello"); change(&mut s); } fn change(some_string: &mut String) { some_string.push_str(", world"); }

72. let mut s = String::from("hello"); let r1 = &mut s; let r2 = &mut s; println!("{}, {}", r1, r2); error[E0499]: cannot borrow `s` as mutable more than once at a time --> src/main.rs:5:14 | 4 | let r1 = &mut s; | ------ first mutable borrow occurs here 5 | let r2 = &mut s; | ^^^^^^ second mutable borrow occurs here 6 | 7 | println!("{}, {}", r1, r2); | -- first borrow later used here

73. let mut s = String::from("hello"); let r1 = &s; // no problem let r2 = &s; // no problem let r3 = &mut s; // BIG PROBLEM println!("{}, {}, and {}", r1, r2, r3); error[E0502]: cannot borrow `s` as mutable because it is also borrowed as immutable --> src/main.rs:6:14 | 4 | let r1 = &s; // no problem | -- immutable borrow occurs here 5 | let r2 = &s; // no problem 6 | let r3 = &mut s; // BIG PROBLEM | ^^^^^^ mutable borrow occurs here 7 | 8 | println!("{}, {}, and {}", r1, r2, r3); | -- immutable borrow later used here

74. Mechanical Sympathy

75. “You don’t have to be an engineer to be a racing driver, but you do have to have Mechanical Sympathy.” Jackie Stewart, racing driver

78. Dziękuję! Pytania?

A miało być tak... bez wycieków

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

Similaire à A miało być tak... bez wycieków

Similaire à A miało być tak... bez wycieków (20)

Plus de Konrad Kokosa

Plus de Konrad Kokosa (7)

Dernier

Dernier (20)

A miało być tak... bez wycieków

Notes de l'éditeur