Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.

Formatting ForThe Masses

Code formatting is an opinionated beast. It always has been a matter of taste, and it always will be a matter of taste. This is the reason, why professional formatting tools, such as Eclipse JDT, offer a gazillion number of options. Which is still not sufficient enough. After all, you can override them inline with tag-comments to make the formatter shut up. Can't we do better than that? What if we could use machine learning techniques to detect the preferred code style that was use in a codebase so far? Turns out, we can.

The Antlr Codebuff project (https://github.com/antlr/codebuff) offers a generic formatter for pretty much any given language. As long as a grammar file exists, existing source can be analyzed to learn about the rules that have been applied while writing the code. Those can than be used to pretty print newly written code. No configuration required. And existing sources will stay as nicely formatted as they are. In the end, the primary purpose of code formatting is not to re-arrange all the keywords, but to make the source layout consistent.

In this talk, we will demonstrate the usage of the codebuff project and how it can be used to format the sources of your repo in a consistent way. We'll also show some other gems that have been revealed when toying around with the technology.

  • Identifiez-vous pour voir les commentaires

  • Soyez le premier à aimer ceci

Formatting ForThe Masses

  1. 1. EDCM_2017_{ ↦ Formattingn
 ↦ ↦ forn
 ↦ ↦ then
 ↦ Massesn }
  2. 2. HolgerSebastian
  3. 3. Formatting
  4. 4. Beautiful Code!
  5. 5. grammar org.eclipse.xtext.example.domainmodel.Domainmodel with org.eclipse.xtext.xbase.Xbase generate domainmodel "http://www.xtext.org/ example/Domainmodel" DomainModel: importSection=XImportSection? elements+=AbstractElement*; AbstractElement: PackageDeclaration| Entity;PackageDeclaration:'package' name=QualifiedName'{'elements+=AbstractElement*'}';Entity:'entity'name=ValidID(' extends'superType=JvmParameterizedTypeReference)?'{'features+=Feature*'}';Featur e:Property| Operation;Property:name=ValidID':'type=JvmTypeReference;Operation:'op'name=Valid ID'('(params+=FullJvmFormalParameter(','params+=FullJvmFormalParameter)*)?')'(': ' type=JvmTypeReference)?body=XBlockExpression; Valid, but not Readable!
  6. 6. grammar org.eclipse.xtext.example.domainmodel.Domainmodel with org.eclipse.xtext.xbase.Xbase generate domainmodel "http://www.xtext.org/example/Domainmodel" DomainModel: importSection=XImportSection? elements+=AbstractElement*; AbstractElement: PackageDeclaration | Entity; PackageDeclaration: 'package' name=QualifiedName '{' elements+=AbstractElement* '}'; Entity: 'entity' name=ValidID ('extends' superType=JvmParameterizedTypeReference)? '{' features+=Feature* '}'; Feature: Property | Operation; Property: name=ValidID ':' type=JvmTypeReference; Operation: 'op' name=ValidID '(' (params+=FullJvmFormalParameter (',' params+=FullJvmFormalParameter)*)? ')' (':' type=JvmTypeReference)? body=XBlockExpression; Same Code, but Readable!
  7. 7. grammar org.eclipse.xtext.example.domainmodel.Domainmodel with org.eclipse.xtext.xbase.Xbase generate domainmodel "http://www.xtext.org/example/Domainmodel" DomainModel: importSection=XImportSection? elements+=AbstractElement*; AbstractElement: PackageDeclaration | Entity; PackageDeclaration: 'package' name=QualifiedName '{' elements+=AbstractElement* '}'; Entity: 'entity' name=ValidID ('extends' superType=JvmParameterizedTypeReference)? '{' features+=Feature* '}'; Feature: Property | Operation; Property: name=ValidID ':' type=JvmTypeReference; Operation: 'op' name=ValidID '(' (params+=FullJvmFormalParameter (',' params+=FullJvmFormalParameter)*)? ')' (':' type=JvmTypeReference)? body=XBlockExpression; Same Code, but Readable! Spaces
  8. 8. grammar org.eclipse.xtext.example.domainmodel.Domainmodel with org.eclipse.xtext.xbase.Xbase generate domainmodel "http://www.xtext.org/example/Domainmodel" DomainModel: importSection=XImportSection? elements+=AbstractElement*; AbstractElement: PackageDeclaration | Entity; PackageDeclaration: 'package' name=QualifiedName '{' elements+=AbstractElement* '}'; Entity: 'entity' name=ValidID ('extends' superType=JvmParameterizedTypeReference)? '{' features+=Feature* '}'; Feature: Property | Operation; Property: name=ValidID ':' type=JvmTypeReference; Operation: 'op' name=ValidID '(' (params+=FullJvmFormalParameter (',' params+=FullJvmFormalParameter)*)? ')' (':' type=JvmTypeReference)? body=XBlockExpression; Linebreak Same Code, but Readable!
  9. 9. grammar org.eclipse.xtext.example.domainmodel.Domainmodel with org.eclipse.xtext.xbase.Xbase generate domainmodel "http://www.xtext.org/example/Domainmodel" DomainModel: importSection=XImportSection? elements+=AbstractElement*; AbstractElement: PackageDeclaration | Entity; PackageDeclaration: 'package' name=QualifiedName '{' elements+=AbstractElement* '}'; Entity: 'entity' name=ValidID ('extends' superType=JvmParameterizedTypeReference)? '{' features+=Feature* '}'; Feature: Property | Operation; Property: name=ValidID ':' type=JvmTypeReference; Operation: 'op' name=ValidID '(' (params+=FullJvmFormalParameter (',' params+=FullJvmFormalParameter)*)? ')' (':' type=JvmTypeReference)? body=XBlockExpression; Indentation Same Code, but Readable!
  10. 10. grammar org.eclipse.xtext.example.domainmodel.Domainmodel with org.eclipse.xtext.xbase.Xbase generate domainmodel "http://www.xtext.org/example/Domainmodel" DomainModel: importSection=XImportSection? elements+=AbstractElement*; AbstractElement: PackageDeclaration | Entity; PackageDeclaration: 'package' name=QualifiedName '{' elements+=AbstractElement* '}'; Entity: 'entity' name=ValidID ('extends' superType=JvmParameterizedTypeReference)? '{' features+=Feature* '}'; Feature: Property | Operation; Property: name=ValidID ':' type=JvmTypeReference; Operation: 'op' name=ValidID '(' (params+=FullJvmFormalParameter (',' params+=FullJvmFormalParameter)*)? ')' (':' type=JvmTypeReference)? body=XBlockExpression; Alignment Same Code, but Readable!
  11. 11. But there are already formatters out there!
  12. 12. But there are already formatters out there!
  13. 13. But there are already formatters out there!
  14. 14. What’s the Point?
  15. 15. Formatting is a Matter of Taste!
  16. 16. Natural Formatting public NotificationChain myMethod(EObject otherEnd, int featureID) { switch (featureID) { …. } return super.eInverseRemove(otherEnd, featureID); } public NotificationChain myMethod(EObject otherEnd, int featureID) { switch (featureID) { …. } return super.eInverseRemove(otherEnd, featureID); } EMF
  17. 17. Eclipse JDT
  18. 18. What, if there is no Option?
  19. 19. Manual Formatting FTW /** * @formatter:off */ public void foo() {}; /** * @formatter:on */
  20. 20. What, if there is no Formatter?
  21. 21. Domain Specific Languages
  22. 22. Bob Nystrom Author of dartfmt http://journal.stuffwithstuff.com/2015/09/08/the-hardest-program-ive-ever-written/ The Hardest Program
 I've Ever Written
  23. 23. Rich API to Define Formatters
  24. 24. Options are not coming for free!
  25. 25. What if formatting would work by using an example?
  26. 26. Towards a
 Universal Code Formatter 
 through
 Machine Learning Terence Parr, Jurgin Vinju 06/2016 https://arxiv.org/abs/1606.08866
  27. 27. CodeBuff
  28. 28. Mimic developer during formatting
  29. 29. (Invalid) Simplification of ML / AI () n1 n2 n3 .. nk ( )↦ m1 ..
  30. 30. CodeBuff ⊕
  31. 31. grammar org.eclipse.xtext.example.domainmodel.Domainmodel with org.eclipse.xtext.xbase.Xbase generate domainmodel "http://www.xtext.org/example/Domainmodel" DomainModel: importSection=XImportSection? elements+=AbstractElement*; AbstractElement: PackageDeclaration | Entity; PackageDeclaration: 'package' name=QualifiedName '{' elements+=AbstractElement* '}'; Entity: 'entity' name=ValidID ('extends' superType=JvmParameterizedTypeReference)? '{' features+=Feature* '}'; Feature: Property | Operation; Property: name=ValidID ':' type=JvmTypeReference; Operation: 'op' name=ValidID '(' (params+=FullJvmFormalParameter (',' params+=FullJvmFormalParameter)*)? ')' (':' type=JvmTypeReference)? body=XBlockExpression; Xtext Grammar Language
  32. 32. PackageDeclaration: 'package' name=QualifiedName '{' elements+=AbstractElement* '}';
  33. 33. PackageDeclaration: 'package' name=QualifiedName '{' elements+=AbstractElement* '}'; split(Document)↦Seq<Token>
  34. 34. PackageDeclaration: 'package' name=QualifiedName '{' elements+=AbstractElement* '}'; format (Token)↦TextChange
  35. 35. format (Token)↦TextChange PackageDeclaration: 'package' name=QualifiedName '{'n .... ....elements+=AbstractElement* '}';
  36. 36. match(TokenProp)↦WS apply(WS)↦NewText prop(Token)↦TokenProp format (Token)↦TextChange
  37. 37. 1. sp 2. nl 3. indent(Token) 4. align(Token) 5. nothing ) WS:=(
  38. 38. WS:=(sp, nl)
  39. 39. WS:=(8, 1) PackageDeclaration: 'package' name=QualifiedName '{'n .... ....elements+=AbstractElement* '}';
  40. 40. match(TokenProp)↦ (sp, nl) apply(sp, nl)↦NewText prop(Token)↦TokenProp
  41. 41. prop(Token)↦TokenProp
  42. 42. TokenProp:=Vector<..>
  43. 43. 1. Token Type (String, Identifier, …) 2. Prev Token, Next Token 3. Paired Token ( { + }, : + ;, [ + ] ) 4. Parent Node Type 5. List Index and List Size TokenProp:=Vector<..>
  44. 44. TokenProp:=V<21>
  45. 45. prop(Token)↦V<21> match (V<21>)↦ (sp, nl) apply(sp, nl)↦NewText
  46. 46. 1. Train Corpus Documents a. Compute prop(Token)↦tp b. Store (tp, sp, nl)
  47. 47. ( ) tpt1 1 tpt1 2 tpt1 3 .. tpt1 x
 spt1 1
 nlt1 2 .. tpt2 1 tpt2 2 tpt2 3 .. tpt2 x
 spt2 1
 nlt2 2 tptc 1 tptc 2 tptc 3 .. tptc x
 sptc 1
 nltc 2
  48. 48. 1. Train Corpus Documents a. Compute prop(Token)↦tp b. Store (tp, sp, nl) 2. Format Unknown Document
  49. 49. () tpu 1 tpu 2 tpu 3 .. tpu x
 ?
 ?
  50. 50. 1. Train Corpus Documents a. Compute prop(Token)↦tp b. Store (tp, sp, nl) 2. Format Unknown Document a. Compute prop(Token)↦tpu b. Find match(tpu)↦ (spu, nlu) c. Apply WS to Document
  51. 51. () tpu 1 tpu 2 tpu 3 .. tpu x ( )spu 1 nlu 2 ( ) tpt1 1 tpt1 2 tpt1 3 .. tpt1 x
 spt1 1
 nlt1 2 .. tpt2 1 tpt2 2 tpt2 3 .. tpt2 x
 spt2 1
 nlt2 2 tptc 1 tptc 2 tptc 3 .. tptc x
 sptc 1
 nltc 2 ↦⊕
  52. 52. match(tpu )↦(spu , nlu ) K Nearest Neighbour
  53. 53. KNN 1. Select k most similar V<21> as vs 2. Select best (spu , nlu )b from vs 
 by weigh function b 3. k = 11
  54. 54. ( ) tpk 1 tpk 2 tpk 3 .. tpk x
 spk 1
 nlk 2 () tpu 1 tpu 2 tpu 3 .. tpu x ( )spk’ 1 nlk’ 2 tpt1 1 tpt1 2 tpt1 3 .. tpt1 x
 spt1 1
 nlt1 2 .. tptc 1 tptc 2 tptc 3 .. tptc x
 sptc 1
 nltc 2 ↦⊕..
  55. 55. Demo
  56. 56. Download it now! 0 € Not ready to use!

×