SlideShare une entreprise Scribd logo
1  sur  27
Tricorder: Diagnose and heal your software 
without the science fiction!
What’s a Tricorder?
How it began: the call for arms
Facing the unknown 
A very big project : 
more than 8 years old 
thousands of options, since it works on every host 
millions of downloads 
a very large userbase
Impossible tickets
Impossible solutions
The log
How logs are recorded
Plain text logging advantages 
very very fast 
low resource consuming 
easy to maintain 
human readable (kind of…) 
easy to analyze 
very reliable
Drowning by numbers
The training 
Trying to identify patterns: 
environment requirements 
runtime errors 
files to exclude 
detect a broken installation
The training
Finding the causes 
Groups of causes 
requirements 
filesystem issues 
runtime errors
Finding the causes 
Requirements 
PHP versions 
database type and version 
database permissions 
available memory
Finding the causes 
Filesystem issues 
big files 
multiple Joomla! installation
Finding the causes 
Runtime errors 
Corrupted install (some hosts blindly delete or rename user files) 
Error logs included in the backup 
Single big archive file while trying to upload it to a remote storage 
Timeouts 
Trying to backup huge tables (more than 1M records) 
Trying to backup tables from different installations 
Backup instance killed by the host
Conclusions 
Several groups 
Each group contains several checks 
Every check requires to open and read a (possibly) very big file 
Avoid using too many resources and hit the timeout limit 
This is the same scenario of a backup
Akeeba Log Inspector and Cause Eliminator
Akeeba Log Inspector and Cause Eliminator
Akeeba Log Inspector and Cause Eliminator
How can I put this inside my application? 
Log the actions taken during a failure prone process 
Analyze the log file 
Propose the correct solution with documentation links 
Keep the analyzer updated with the latest solutions
Real world example: file uploads 
Requested features 
Multiple upload 
Resize images 
Thumbnail creation 
Apply watermark
Real world example: file uploads 
Possible failure reasons 
Maximum upload size on the server 
Maximum POST size 
Number of files 
No upload directory 
Temporary directory unwriteable 
Final directory unwriteable
Real world example: file uploads 
Possible failure reasons 
Error while resizing the image 
Error while creating the thumbnail 
Error while applying the watermark 
Error while saving the image file 
Error while saving into the db
The results 
Fewer support requests 
Happier customers 
More time for coding instead of doing support

Contenu connexe

Similaire à Tricorder: Diagnose and heal your software (without science fiction)

Beyond the File System: Designing Large-Scale File Storage and Serving
 	Beyond the File System: Designing Large-Scale File Storage and Serving 	Beyond the File System: Designing Large-Scale File Storage and Serving
Beyond the File System: Designing Large-Scale File Storage and Servingmclee
 
Web20expo Filesystems
Web20expo FilesystemsWeb20expo Filesystems
Web20expo Filesystemsguest18a0f1
 
Web20expo Filesystems
Web20expo FilesystemsWeb20expo Filesystems
Web20expo Filesystemsroyans
 
Web20expo Filesystems
Web20expo FilesystemsWeb20expo Filesystems
Web20expo Filesystemsroyans
 
Scalable Apache for Beginners
Scalable Apache for BeginnersScalable Apache for Beginners
Scalable Apache for Beginnerswebhostingguy
 
Beyond the File System - Designing Large Scale File Storage and Serving
Beyond the File System - Designing Large Scale File Storage and ServingBeyond the File System - Designing Large Scale File Storage and Serving
Beyond the File System - Designing Large Scale File Storage and Servingmclee
 
Filesystems
FilesystemsFilesystems
Filesystemsroyans
 
AWS re:Invent 2016: Deep Dive on Amazon Elastic File System (STG202)
AWS re:Invent 2016: Deep Dive on Amazon Elastic File System (STG202)AWS re:Invent 2016: Deep Dive on Amazon Elastic File System (STG202)
AWS re:Invent 2016: Deep Dive on Amazon Elastic File System (STG202)Amazon Web Services
 
Session9-File Upload Security
Session9-File Upload SecuritySession9-File Upload Security
Session9-File Upload Securityzakieh alizadeh
 
Dan Crowley - Jack Of All Formats
Dan Crowley - Jack Of All FormatsDan Crowley - Jack Of All Formats
Dan Crowley - Jack Of All FormatsSource Conference
 
Deep Dive on Elastic File System - February 2017 AWS Online Tech Talks
Deep Dive on Elastic File System - February 2017 AWS Online Tech TalksDeep Dive on Elastic File System - February 2017 AWS Online Tech Talks
Deep Dive on Elastic File System - February 2017 AWS Online Tech TalksAmazon Web Services
 
Disk Image!...and then what? Strategies for sustainable long-term storage an...
Disk Image!...and then what?  Strategies for sustainable long-term storage an...Disk Image!...and then what?  Strategies for sustainable long-term storage an...
Disk Image!...and then what? Strategies for sustainable long-term storage an...Helen Bailey
 
Performance Analysis of Idle Programs
Performance Analysis of Idle ProgramsPerformance Analysis of Idle Programs
Performance Analysis of Idle Programsgreenwop
 
AWS re:Invent 2016: Case Study: How Spokeo Improved Web Application Response ...
AWS re:Invent 2016: Case Study: How Spokeo Improved Web Application Response ...AWS re:Invent 2016: Case Study: How Spokeo Improved Web Application Response ...
AWS re:Invent 2016: Case Study: How Spokeo Improved Web Application Response ...Amazon Web Services
 
Bp106 Worst Practices Final
Bp106   Worst Practices FinalBp106   Worst Practices Final
Bp106 Worst Practices FinalBill Buchan
 
Deep Dive on Amazon Elastic File System - June 2017 AWS Online Tech Talks
Deep Dive on Amazon Elastic File System - June 2017 AWS Online Tech TalksDeep Dive on Amazon Elastic File System - June 2017 AWS Online Tech Talks
Deep Dive on Amazon Elastic File System - June 2017 AWS Online Tech TalksAmazon Web Services
 
Bareos - Open Source Data Protection, by Philipp Storz
Bareos - Open Source Data Protection, by Philipp StorzBareos - Open Source Data Protection, by Philipp Storz
Bareos - Open Source Data Protection, by Philipp StorzNETWAYS
 
Web scale MySQL at Facebook (Domas Mituzas)
Web scale MySQL at Facebook (Domas Mituzas)Web scale MySQL at Facebook (Domas Mituzas)
Web scale MySQL at Facebook (Domas Mituzas)Ontico
 
Data management for TA's
Data management for TA'sData management for TA's
Data management for TA'saaroncollie
 
File Management (1).pptx
File Management (1).pptxFile Management (1).pptx
File Management (1).pptxSolomonAnab1
 

Similaire à Tricorder: Diagnose and heal your software (without science fiction) (20)

Beyond the File System: Designing Large-Scale File Storage and Serving
 	Beyond the File System: Designing Large-Scale File Storage and Serving 	Beyond the File System: Designing Large-Scale File Storage and Serving
Beyond the File System: Designing Large-Scale File Storage and Serving
 
Web20expo Filesystems
Web20expo FilesystemsWeb20expo Filesystems
Web20expo Filesystems
 
Web20expo Filesystems
Web20expo FilesystemsWeb20expo Filesystems
Web20expo Filesystems
 
Web20expo Filesystems
Web20expo FilesystemsWeb20expo Filesystems
Web20expo Filesystems
 
Scalable Apache for Beginners
Scalable Apache for BeginnersScalable Apache for Beginners
Scalable Apache for Beginners
 
Beyond the File System - Designing Large Scale File Storage and Serving
Beyond the File System - Designing Large Scale File Storage and ServingBeyond the File System - Designing Large Scale File Storage and Serving
Beyond the File System - Designing Large Scale File Storage and Serving
 
Filesystems
FilesystemsFilesystems
Filesystems
 
AWS re:Invent 2016: Deep Dive on Amazon Elastic File System (STG202)
AWS re:Invent 2016: Deep Dive on Amazon Elastic File System (STG202)AWS re:Invent 2016: Deep Dive on Amazon Elastic File System (STG202)
AWS re:Invent 2016: Deep Dive on Amazon Elastic File System (STG202)
 
Session9-File Upload Security
Session9-File Upload SecuritySession9-File Upload Security
Session9-File Upload Security
 
Dan Crowley - Jack Of All Formats
Dan Crowley - Jack Of All FormatsDan Crowley - Jack Of All Formats
Dan Crowley - Jack Of All Formats
 
Deep Dive on Elastic File System - February 2017 AWS Online Tech Talks
Deep Dive on Elastic File System - February 2017 AWS Online Tech TalksDeep Dive on Elastic File System - February 2017 AWS Online Tech Talks
Deep Dive on Elastic File System - February 2017 AWS Online Tech Talks
 
Disk Image!...and then what? Strategies for sustainable long-term storage an...
Disk Image!...and then what?  Strategies for sustainable long-term storage an...Disk Image!...and then what?  Strategies for sustainable long-term storage an...
Disk Image!...and then what? Strategies for sustainable long-term storage an...
 
Performance Analysis of Idle Programs
Performance Analysis of Idle ProgramsPerformance Analysis of Idle Programs
Performance Analysis of Idle Programs
 
AWS re:Invent 2016: Case Study: How Spokeo Improved Web Application Response ...
AWS re:Invent 2016: Case Study: How Spokeo Improved Web Application Response ...AWS re:Invent 2016: Case Study: How Spokeo Improved Web Application Response ...
AWS re:Invent 2016: Case Study: How Spokeo Improved Web Application Response ...
 
Bp106 Worst Practices Final
Bp106   Worst Practices FinalBp106   Worst Practices Final
Bp106 Worst Practices Final
 
Deep Dive on Amazon Elastic File System - June 2017 AWS Online Tech Talks
Deep Dive on Amazon Elastic File System - June 2017 AWS Online Tech TalksDeep Dive on Amazon Elastic File System - June 2017 AWS Online Tech Talks
Deep Dive on Amazon Elastic File System - June 2017 AWS Online Tech Talks
 
Bareos - Open Source Data Protection, by Philipp Storz
Bareos - Open Source Data Protection, by Philipp StorzBareos - Open Source Data Protection, by Philipp Storz
Bareos - Open Source Data Protection, by Philipp Storz
 
Web scale MySQL at Facebook (Domas Mituzas)
Web scale MySQL at Facebook (Domas Mituzas)Web scale MySQL at Facebook (Domas Mituzas)
Web scale MySQL at Facebook (Domas Mituzas)
 
Data management for TA's
Data management for TA'sData management for TA's
Data management for TA's
 
File Management (1).pptx
File Management (1).pptxFile Management (1).pptx
File Management (1).pptx
 

Dernier

GOING AOT WITH GRAALVM – DEVOXX GREECE.pdf
GOING AOT WITH GRAALVM – DEVOXX GREECE.pdfGOING AOT WITH GRAALVM – DEVOXX GREECE.pdf
GOING AOT WITH GRAALVM – DEVOXX GREECE.pdfAlina Yurenko
 
SpotFlow: Tracking Method Calls and States at Runtime
SpotFlow: Tracking Method Calls and States at RuntimeSpotFlow: Tracking Method Calls and States at Runtime
SpotFlow: Tracking Method Calls and States at Runtimeandrehoraa
 
CRM Contender Series: HubSpot vs. Salesforce
CRM Contender Series: HubSpot vs. SalesforceCRM Contender Series: HubSpot vs. Salesforce
CRM Contender Series: HubSpot vs. SalesforceBrainSell Technologies
 
英国UN学位证,北安普顿大学毕业证书1:1制作
英国UN学位证,北安普顿大学毕业证书1:1制作英国UN学位证,北安普顿大学毕业证书1:1制作
英国UN学位证,北安普顿大学毕业证书1:1制作qr0udbr0
 
Implementing Zero Trust strategy with Azure
Implementing Zero Trust strategy with AzureImplementing Zero Trust strategy with Azure
Implementing Zero Trust strategy with AzureDinusha Kumarasiri
 
Machine Learning Software Engineering Patterns and Their Engineering
Machine Learning Software Engineering Patterns and Their EngineeringMachine Learning Software Engineering Patterns and Their Engineering
Machine Learning Software Engineering Patterns and Their EngineeringHironori Washizaki
 
Unveiling the Future: Sylius 2.0 New Features
Unveiling the Future: Sylius 2.0 New FeaturesUnveiling the Future: Sylius 2.0 New Features
Unveiling the Future: Sylius 2.0 New FeaturesŁukasz Chruściel
 
Software Project Health Check: Best Practices and Techniques for Your Product...
Software Project Health Check: Best Practices and Techniques for Your Product...Software Project Health Check: Best Practices and Techniques for Your Product...
Software Project Health Check: Best Practices and Techniques for Your Product...Velvetech LLC
 
Salesforce Implementation Services PPT By ABSYZ
Salesforce Implementation Services PPT By ABSYZSalesforce Implementation Services PPT By ABSYZ
Salesforce Implementation Services PPT By ABSYZABSYZ Inc
 
Large Language Models for Test Case Evolution and Repair
Large Language Models for Test Case Evolution and RepairLarge Language Models for Test Case Evolution and Repair
Large Language Models for Test Case Evolution and RepairLionel Briand
 
Alfresco TTL#157 - Troubleshooting Made Easy: Deciphering Alfresco mTLS Confi...
Alfresco TTL#157 - Troubleshooting Made Easy: Deciphering Alfresco mTLS Confi...Alfresco TTL#157 - Troubleshooting Made Easy: Deciphering Alfresco mTLS Confi...
Alfresco TTL#157 - Troubleshooting Made Easy: Deciphering Alfresco mTLS Confi...Angel Borroy López
 
Post Quantum Cryptography – The Impact on Identity
Post Quantum Cryptography – The Impact on IdentityPost Quantum Cryptography – The Impact on Identity
Post Quantum Cryptography – The Impact on Identityteam-WIBU
 
Global Identity Enrolment and Verification Pro Solution - Cizo Technology Ser...
Global Identity Enrolment and Verification Pro Solution - Cizo Technology Ser...Global Identity Enrolment and Verification Pro Solution - Cizo Technology Ser...
Global Identity Enrolment and Verification Pro Solution - Cizo Technology Ser...Cizo Technology Services
 
UI5ers live - Custom Controls wrapping 3rd-party libs.pptx
UI5ers live - Custom Controls wrapping 3rd-party libs.pptxUI5ers live - Custom Controls wrapping 3rd-party libs.pptx
UI5ers live - Custom Controls wrapping 3rd-party libs.pptxAndreas Kunz
 
Introduction Computer Science - Software Design.pdf
Introduction Computer Science - Software Design.pdfIntroduction Computer Science - Software Design.pdf
Introduction Computer Science - Software Design.pdfFerryKemperman
 
20240415 [Container Plumbing Days] Usernetes Gen2 - Kubernetes in Rootless Do...
20240415 [Container Plumbing Days] Usernetes Gen2 - Kubernetes in Rootless Do...20240415 [Container Plumbing Days] Usernetes Gen2 - Kubernetes in Rootless Do...
20240415 [Container Plumbing Days] Usernetes Gen2 - Kubernetes in Rootless Do...Akihiro Suda
 
Simplifying Microservices & Apps - The art of effortless development - Meetup...
Simplifying Microservices & Apps - The art of effortless development - Meetup...Simplifying Microservices & Apps - The art of effortless development - Meetup...
Simplifying Microservices & Apps - The art of effortless development - Meetup...Rob Geurden
 
Real-time Tracking and Monitoring with Cargo Cloud Solutions.pptx
Real-time Tracking and Monitoring with Cargo Cloud Solutions.pptxReal-time Tracking and Monitoring with Cargo Cloud Solutions.pptx
Real-time Tracking and Monitoring with Cargo Cloud Solutions.pptxRTS corp
 
Maximizing Efficiency and Profitability with OnePlan’s Professional Service A...
Maximizing Efficiency and Profitability with OnePlan’s Professional Service A...Maximizing Efficiency and Profitability with OnePlan’s Professional Service A...
Maximizing Efficiency and Profitability with OnePlan’s Professional Service A...OnePlan Solutions
 

Dernier (20)

GOING AOT WITH GRAALVM – DEVOXX GREECE.pdf
GOING AOT WITH GRAALVM – DEVOXX GREECE.pdfGOING AOT WITH GRAALVM – DEVOXX GREECE.pdf
GOING AOT WITH GRAALVM – DEVOXX GREECE.pdf
 
SpotFlow: Tracking Method Calls and States at Runtime
SpotFlow: Tracking Method Calls and States at RuntimeSpotFlow: Tracking Method Calls and States at Runtime
SpotFlow: Tracking Method Calls and States at Runtime
 
CRM Contender Series: HubSpot vs. Salesforce
CRM Contender Series: HubSpot vs. SalesforceCRM Contender Series: HubSpot vs. Salesforce
CRM Contender Series: HubSpot vs. Salesforce
 
英国UN学位证,北安普顿大学毕业证书1:1制作
英国UN学位证,北安普顿大学毕业证书1:1制作英国UN学位证,北安普顿大学毕业证书1:1制作
英国UN学位证,北安普顿大学毕业证书1:1制作
 
Implementing Zero Trust strategy with Azure
Implementing Zero Trust strategy with AzureImplementing Zero Trust strategy with Azure
Implementing Zero Trust strategy with Azure
 
Machine Learning Software Engineering Patterns and Their Engineering
Machine Learning Software Engineering Patterns and Their EngineeringMachine Learning Software Engineering Patterns and Their Engineering
Machine Learning Software Engineering Patterns and Their Engineering
 
Unveiling the Future: Sylius 2.0 New Features
Unveiling the Future: Sylius 2.0 New FeaturesUnveiling the Future: Sylius 2.0 New Features
Unveiling the Future: Sylius 2.0 New Features
 
Software Project Health Check: Best Practices and Techniques for Your Product...
Software Project Health Check: Best Practices and Techniques for Your Product...Software Project Health Check: Best Practices and Techniques for Your Product...
Software Project Health Check: Best Practices and Techniques for Your Product...
 
Salesforce Implementation Services PPT By ABSYZ
Salesforce Implementation Services PPT By ABSYZSalesforce Implementation Services PPT By ABSYZ
Salesforce Implementation Services PPT By ABSYZ
 
Large Language Models for Test Case Evolution and Repair
Large Language Models for Test Case Evolution and RepairLarge Language Models for Test Case Evolution and Repair
Large Language Models for Test Case Evolution and Repair
 
Alfresco TTL#157 - Troubleshooting Made Easy: Deciphering Alfresco mTLS Confi...
Alfresco TTL#157 - Troubleshooting Made Easy: Deciphering Alfresco mTLS Confi...Alfresco TTL#157 - Troubleshooting Made Easy: Deciphering Alfresco mTLS Confi...
Alfresco TTL#157 - Troubleshooting Made Easy: Deciphering Alfresco mTLS Confi...
 
Post Quantum Cryptography – The Impact on Identity
Post Quantum Cryptography – The Impact on IdentityPost Quantum Cryptography – The Impact on Identity
Post Quantum Cryptography – The Impact on Identity
 
Global Identity Enrolment and Verification Pro Solution - Cizo Technology Ser...
Global Identity Enrolment and Verification Pro Solution - Cizo Technology Ser...Global Identity Enrolment and Verification Pro Solution - Cizo Technology Ser...
Global Identity Enrolment and Verification Pro Solution - Cizo Technology Ser...
 
UI5ers live - Custom Controls wrapping 3rd-party libs.pptx
UI5ers live - Custom Controls wrapping 3rd-party libs.pptxUI5ers live - Custom Controls wrapping 3rd-party libs.pptx
UI5ers live - Custom Controls wrapping 3rd-party libs.pptx
 
Introduction Computer Science - Software Design.pdf
Introduction Computer Science - Software Design.pdfIntroduction Computer Science - Software Design.pdf
Introduction Computer Science - Software Design.pdf
 
20240415 [Container Plumbing Days] Usernetes Gen2 - Kubernetes in Rootless Do...
20240415 [Container Plumbing Days] Usernetes Gen2 - Kubernetes in Rootless Do...20240415 [Container Plumbing Days] Usernetes Gen2 - Kubernetes in Rootless Do...
20240415 [Container Plumbing Days] Usernetes Gen2 - Kubernetes in Rootless Do...
 
Simplifying Microservices & Apps - The art of effortless development - Meetup...
Simplifying Microservices & Apps - The art of effortless development - Meetup...Simplifying Microservices & Apps - The art of effortless development - Meetup...
Simplifying Microservices & Apps - The art of effortless development - Meetup...
 
Real-time Tracking and Monitoring with Cargo Cloud Solutions.pptx
Real-time Tracking and Monitoring with Cargo Cloud Solutions.pptxReal-time Tracking and Monitoring with Cargo Cloud Solutions.pptx
Real-time Tracking and Monitoring with Cargo Cloud Solutions.pptx
 
2.pdf Ejercicios de programación competitiva
2.pdf Ejercicios de programación competitiva2.pdf Ejercicios de programación competitiva
2.pdf Ejercicios de programación competitiva
 
Maximizing Efficiency and Profitability with OnePlan’s Professional Service A...
Maximizing Efficiency and Profitability with OnePlan’s Professional Service A...Maximizing Efficiency and Profitability with OnePlan’s Professional Service A...
Maximizing Efficiency and Profitability with OnePlan’s Professional Service A...
 

Tricorder: Diagnose and heal your software (without science fiction)

  • 1. Tricorder: Diagnose and heal your software without the science fiction!
  • 3. How it began: the call for arms
  • 4. Facing the unknown A very big project : more than 8 years old thousands of options, since it works on every host millions of downloads a very large userbase
  • 8. How logs are recorded
  • 9. Plain text logging advantages very very fast low resource consuming easy to maintain human readable (kind of…) easy to analyze very reliable
  • 11.
  • 12. The training Trying to identify patterns: environment requirements runtime errors files to exclude detect a broken installation
  • 14.
  • 15. Finding the causes Groups of causes requirements filesystem issues runtime errors
  • 16. Finding the causes Requirements PHP versions database type and version database permissions available memory
  • 17. Finding the causes Filesystem issues big files multiple Joomla! installation
  • 18. Finding the causes Runtime errors Corrupted install (some hosts blindly delete or rename user files) Error logs included in the backup Single big archive file while trying to upload it to a remote storage Timeouts Trying to backup huge tables (more than 1M records) Trying to backup tables from different installations Backup instance killed by the host
  • 19. Conclusions Several groups Each group contains several checks Every check requires to open and read a (possibly) very big file Avoid using too many resources and hit the timeout limit This is the same scenario of a backup
  • 20. Akeeba Log Inspector and Cause Eliminator
  • 21. Akeeba Log Inspector and Cause Eliminator
  • 22. Akeeba Log Inspector and Cause Eliminator
  • 23. How can I put this inside my application? Log the actions taken during a failure prone process Analyze the log file Propose the correct solution with documentation links Keep the analyzer updated with the latest solutions
  • 24. Real world example: file uploads Requested features Multiple upload Resize images Thumbnail creation Apply watermark
  • 25. Real world example: file uploads Possible failure reasons Maximum upload size on the server Maximum POST size Number of files No upload directory Temporary directory unwriteable Final directory unwriteable
  • 26. Real world example: file uploads Possible failure reasons Error while resizing the image Error while creating the thumbnail Error while applying the watermark Error while saving the image file Error while saving into the db
  • 27. The results Fewer support requests Happier customers More time for coding instead of doing support