ScalaMatsuri 2018の発表です

1. Extensible Effects
in Dotty
SanshiroYoshida @halcat0x15a
DWANGO Co.,Ltd.

2. Contents
• Dotty
• Introduction of language features
• Extensible Effects
• Explanation of the implementation
DottyとExtEffについて話します

3. About Dotty
• A next generation compiler for Scala
• Formalized in DOT
• Implemented new features
Dottyは次世代のScalaコンパイラです

4. DOT
• Dependent Object Types
• A calculus aimed as a new foundation of
Scala
• Featured path-dependent types, refinement
types, and abstract type members
DOTはScalaの基礎となる計算モデルです。

5. DOT
• Models Scala language features with
minimal calculus.
trait List { self =>
type A
def head: self.A; def tail: List { type A <: self.A }
}
def cons(x: { type A })(hd: x.A)(tl: List { type A <:
x.A }): List { type A <: x.A } =
new List { self =>
type A = x.A
def head = hd; def tail = tl
}
DOTは最小限の計算でScalaの言語機能をモデル化します

6. New language features
• Enums
• Type Lambdas
• Intersection Types
• Union Types
• Implicit Function Types
• etc.
Dottyの新しい言語機能です

7. Enums
• Syntax sugar for enumerations and ADTs.
• The companion object defines utility
methods.
enumは列挙型と代数的データ型を定義するための構文です

8. Enums
• The value of enums is tagged with Int.
enum Color {
case Red, Green, Blue
}
scala> Color.enumValue(0)
val res0: Color = Red
列挙型の値にはInt型のタグが付けられます

9. Enums
• The `enum` supports ADTs.
• ADTs can define fields and methods.
enumキーワードは代数的データ型をサポートします
enum Option[+A] {
case Some(a: A)
case None
def isEmpty: Boolean = this == None
}

10. Type Lambdas
• Representation of higher-kinded types.
Type lambdaは高階型を表現します
type Pair = [A] => (A, A)
val p: Pair[Int] = (0, 1)

11. Type Lambdas
• Possible to write partial application of type
directly
型の部分適用を直接書けるようになりました
// Scala2
type FreeMonad[F[_]] =
Monad[({ type λ[A] = Free[F, A] })#λ]
// Dotty
type FreeMonad[F[_]] = Monad[[A] => Free[F, A]]

12. Dotty
• More expressive types
• More easy-to-write syntax
• More suitable for functional programming
Dottyは関数型プログラミングにより適しています

13. About ExtEff
• Extensible Effects = Freer Monad + Open
Union + Type-aligned Queue
• Freer Monad = Free Monad + Free Functor
(Coyoneda)
• I will talk about these abstractions.
今日はこれらの抽象概念について話します

14. Monad
• Monad is a computation with side-effects.
• For example, Option monad is a
computation for which there may not exist
a value.
• You can write it procedurally with ‘for’
expression.
モナドは副作用付き計算のこと

15. Free
• Free f is a monad if f is a functor.
• Various monads can be represented using f.
Freeはファンクタによって様々なモナドを表現できます

16. Definition of Free
Pureは純粋な計算でImpureは副作用付きの計算を表します
• Pure is a pure computation.
• Impure is a computation with side-effects.
enum Free[F[_], A] {
case Pure(a: A)
case Impure(ffree: F[Free[F, A]])
}
def lift(fa: F[A])(implicit F: Functor[F]): Free[F, A] =
Impure(F.map(fa)(a => Pure(a)))

17. Free Monad
• flatMap has the constraint that F is Functor
flatMapはFがFunctorである制約を持ちます
enum Free[F[_], A] {
def flatMap[B](f: A => Free[F, B])(implicit F:
Functor[F]): Free[F, B] =
this match {
case Pure(a) => f(a)
case Impure(ffree) =>
F.map(ffree)(free => free.flatMap(f))
}
}

18. Free Writer
• Writer is a computation with another
output.
• For example, it is a computation that
outputs logs.
Writerは別の出力を持つ計算です

19. Free Writer
• Definition of Writer monad by Free.
FreeによるWriterモナドの定義です
type Writer[W, A] = Free[[T] => Tell[W, T], A]
// Effect is described CPS
case class Tell[W, A](w: W, a: A) {
def map[B](f: A => B) = Tell(w, f(a))
}
def tell[W](w: W): Writer[W, Unit] =
Free.lift(Tell(w, ()))

20. Free Writer
• An example of handler for Writer.
• Writer can output as List.
WriterはListとして出力することができます
def runAsList[W, A](free: Writer[W, A]): (List[W], A) =
free match {
case Free.Pure(a) => (Nil, a)
case Free.Impure(Tell(w, free)) =>
runAsList(free).map {
case (ws, a) => (w :: ws, a)
}
}

21. Free Writer
• Writer can output asVector.
WriterはVectorとしても出力できます
def runAsVec[W, A](free: Writer[W, A]): (Vector[W], A) =
{
def go(acc: Vector[W], free: Writer[W, A]):
(Vector[W], A) =
free match {
case Free.Pure(a) => (acc, a)
case Free.Impure(Tell(w, free)) =>
go(acc :+ w, free)
}
go(Vector.empty, free)
}

22. Free Writer
• Multiple interpretations can be made for
one expression.
一つの式に対して複数の解釈が可能です
val e = for {
_ <- tell("hoge")
_ <- tell("fuga")
} yield ()
scala> runAsList(e)
val res0: (List[String], Unit) = (List(hoge, fuga),())
scala> runAsVec(e)
val res1: (Vector[String], Unit) = (Vector(hoge, fuga),
())

23. Free
• Free represents various monads.
• Free has a functor constraint.
• Free is free to interpret.
Freeは様々なモナドを表現でき、自由に解釈できます

24. Freer
• Freer is Free applied to Coyoneda.
• Freer becomes a monad without
constraints.
• Freer uses a tree in flatMap.
Freerは制約なしにモナドになります

25. Coyoneda
• Coyoneda is Free Functor.
• For all f, Coyoneda f is a functor.
任意fについてCoyoneda fはファンクタです

26. Definition of Coyoneda
• FMap has a signature similar to map.
FMapはmapと似たシグネチャをもちます
enum Coyoneda[F[_], A] {
case FMap[F[_], A, B](fa: F[A], k: A => B) extends
Coyoneda[F, B]
}
def lift[F[_], A](fa: F[A]): Coyoneda[F, A] =
Coyoneda.FMap(fa, a => a)

27. Coyoneda Functor
• Coyoneda becomes a functor without
constraints.
Coyonedaは制約なしにファンクタになります
enum Coyoneda[F[_], A] {
def map[B](f: A => B): Coyoneda[F, B] =
this match {
case Coyoneda.FMap(fi, k) =>
Coyoneda.FMap(fi, k andThen f)
}
}

28. Coyoneda
• Using Coyoneda seems to be able to map
everything.
Coyonedaは全てをmapできるように見えます
case class Box[A](a: A)
val box = Coyoneda.lift(Box(0))
.map(i => i + 1)
.map(i => i.toString)

29. Coyoneda
• Coyoneda does not apply to the value of
the Box.
• Coyoneda#map is only composing
functions.
Coyonedaは関数の合成をしているだけで適用をしていません

30. Definition of Freer
• Expands the definition of Free Coyoneda.
• Impure has a signature similar to flatMap.
Free Coyonedaの定義を展開したものです
enum Freer[F[_], A] {
case Pure(a: A)
case Impure[F[_], A, B](fa: F[A], k: A => Freer[F, B])
extends Freer[F, B]
}
def lift[F[_], A](fa: F[A]): Freer[F, A] =
Impure(fa, a => Pure(a))

31. Freer Monad
• flatMap is a free from constraints of
Functor.
flatMapからFunctorの制約がなくなりました
enum Freer[F[_], A] {
def flatMap[B](f: A => Freer[F, B]): Freer[F, B] =
this match {
case Freer.Pure(a) => f(a)
case Freer.Impure(fa, k) =>
Freer.Impure(fa, a => k(a).flatMap(f))
}
}

32. Freer Monad
• This implementation of flatMap is slow.
• flatMap(f_0).flatMap(f_1)…flatMap(f_n) is
O(n^2).
このflatMapの実装は遅いです

33. Type-aligned Queue
• FTCQ is a sequence of functions.
• It is implemented with a tree.
FTCQは関数の列を表します
val `A => F[C]`: FTCQ[F, A, C] =
Node(
Leaf(f: A => F[B]),
Leaf(g: B => F[C])
)

34. Type-aligned Queue
• The composition of functions is constant-
time.
• The application of function is stack-safe.
合成は定数時間で、適用はスタックセーフに行われます
val `A => F[D]`: FTCQ[F, A, D] =
Node(
`A => F[C]`,
Leaf(h: C => F[D])
)

35. Definition of Fast Freer
• Impure represents a continuation with
FTCQ.
Impureは継続をFTCQで表現します
type Arrs[F, A, B] = FTCQ[[T] => Freer[F, T], A, B]
enum Freer[F[_], A] {
case Pure(a: A)
case Impure[F[_], A, B](fa: F[A], k: Arrs[F, A, B])
extends Freer[F, B]
}

36. Fast Freer Monad
• flatMap is constant-time.
flatMapは定数時間で実行されます
enum Freer[F[_], A] {
def flatMap[B](f: A => Freer[F, B]): Freer[F, B] =
this match {
case Freer.Pure(a) => f(a)
case Freer.Impure(fa, k) =>
Freer.Impure(fa, k :+ f)
}
}

37. Freer Reader
• Reader is a computation with environment.
• For example, it is a computation that takes
the configuration.
Readerは環境を持つような計算です

38. Freer Reader
• Definition of Reader monad by Freer.
FreerによるReaderモナドの定義です
type Reader[I, A] = Freer[[T] => Ask[I, T], A]
case class Ask[I, A](k: I => A)
def ask[I]: Reader[I, I] =
Freer.lift(Ask(i => i))

39. Freer Reader
• The handler of Freer applies continuation.
Freerのハンドラは継続の適用を行います
def runReader[I, A](freer: Reader[I, A], i: I): A =
freer match {
case Freer.Pure(a) => a
case Freer.Impure(Ask(f), k) =>
runReader(k(f(i)), i)
}

40. Freer Reader
• An example of a Reader monad.
Readerモナドの例です
val e = for {
x <- ask[Int]
y <- ask[Int]
} yield x + y
scala> runReader(e, 1)
val res0: Int = 2

41. Freer
• Freer has no constraints of Functor.
• Freer is faster than Free by using Type-
aligned Queue.
FreerはFunctorの制約がなくFreeよりも高速です

42. Extensible Effects
• ExtEff is Freer applied to Open Union.
• ExtEff can compose various effects using
Open Union.
ExtEffはOpen Unionを使って様々な副作用を合成できます

43. Open Union
• Representation of extensible sum of types
• Automatic construction by typeclass
Open Unionは拡張可能な型の和を表します

44. Open Union
• Union seems to be a higher-kinded type
version of Either.
Unionは高階型を使ったEitherのような定義です
enum Union[F[_], G[_], A] {
case Inl(value: F[A])
case Inr(value: G[A])
}

45. Open Union
• An alias for writing in infix notation is
useful.
中置記法で記述するための別名があると便利です
type :+:[F[_], G[_]] = [A] => Union[F, G, A]
type ListOrOption = List :+: Option :+: Nothing

46. Member
MemberはUnionに値をinjectします
trait Member[F[_], R[_]] {
def inject[A](fa: F[A]): R[A]
}
• Member injects a value into an union.

47. Member
• It is uniquely derived if a value is in Inl.
値がInlにあることでインスタンスを一意に導出できます
implicit def leftMember[F[_], G[_]] =
new Member[F, F :+: G] {
def inject[A](fa: F[A]) = Union.Inl(fa)
}
implicit def rightMember[F[_], G[_], H[_]](implicit F:
Member[F, H]) =
new Member[F, G :+: H] {
def inject[A](fa: F[A]) = Union.Inr(F.inject(fa))
}

48. Member
• An example of constructing Union using
Member.
Memberを使ってUnionを構成する例です
def inject[F[_], R[_], A](fa: F[A])(implicit F:
Member[F, R]): R[A] =
F.inject(fa)
scala> val opt: ListOrOption[Int] = inject(Option(0))
val opt: ListOrOption[Int] = Inr(Inl(Some(0)))

49. Definition of ExtEff
• lift injects effects using Member.
liftはMemberを使ってエフェクトを注入します
enum Eff[R[_], A] {
case Pure(a: A)
case Impure[R[_], A, B](union: R[A], k: Arrs[F, A, B])
extends Eff[R, B]
}
def lift[F[_], R[_], A](fa: F[A])(implicit F: Member[F,
R]): Eff[R, A] =
Impure(F.inject(fa), Arrs(a => Pure(a)))

50. ExtEff Writer
コンストラクタからモナドの値が決まります
• No longer necessary to write in CPS.
• A value of a monad is determined from the
constructor.
enum Writer[W, A] {
case Tell[W](w: W) extends Writer[W, Unit]
}
def tell[R[_], W](w: W)(implicit ev: Member[[A] =>
Writer[W, A], R]): Eff[R, Unit] = Eff.lift(Tell(w))

51. ExtEff Reader
プリミティブの関数はコンストラクタを持ち上げるだけです
• Primitive functions only lifts constructors.
enum Reader[I, A] {
case Ask[I]() extends Reader[I, I]
}
def ask[R[_], I](implicit ev: Member[[A] => Reader[I,
A], R]): Eff[R, I] = Eff.lift(Ask[I])

52. ExtEff Reader
• If another effect appears, transfer it.
他のエフェクトがあらわれた場合は処理を移譲します
def runReader[R[_], I, A](eff: Eff[([T] => Reader[I, T])
:+: R, A], i: I): Eff[R, A] =
eff match {
case Eff.Pure(a) => Free.Pure(a)
case Eff.Impure(Union.Inl(Reader.Ask()), k) =>
runReader(k(i), i)
case Eff.Impure(Union.Inr(r), k) =>
Eff.Impure(r, a => runReader(k(a), i))
}

53. ExtEff Handler
• Handlers can be generalized.
ハンドラは一般化することができます
def handleRelay[F[_], R[_], A, B]
(eff: Eff[F :+: R, A])
(pure: A => Eff[R, B])
(bind: F[A] => (A => Eff[R, B]) => Eff[R, B])
: Eff[R, B] = eff match {
case Eff.Pure(a) => pure(a)
case Eff.Impure(Union.Inl(fa), k) =>
bind(fa)(a => handleRelay(k(a))(pure)(bind))
case Eff.Impure(Union.Inr(r), k) =>
Eff.Impure(r, a => handleRelay(k(a))(pure)(bind))
}

54. ExtEff Writer
• Just write pure and flatMap with a handler.
handleRelayを使えばpureとflatMapを書くだけです。
def runWriter[R[_], W, A](eff: Eff[([T] => Writer[W, T])
:+: R, A]): Eff[R, (List[W], A)] =
handleRelay(eff)(a => (Nil, a)) {
case Writer.Tell(w) =>
k => k(()).map { case (ws, a) => (w :: ws, a) }
}

55. Run ExtEff
• If effects is Nothing, no instance of Impure
exists.
def run[A](eff: Eff[Nothing, A]): A =
eff match {
case Eff.Pure(a) => a
}
エフェクトがNothingならImpureのインスタンスは存在しない

56. ExtEff Example
• You can write two monads in one ‘for’
二つのモナドを一つのfor式に書けます
def e[R[_]](implicit r: Member[[T] => Reader[Int, T],
R], w: Member[[T] => Writer[Int, T], R]): Eff[R, Int] =
for {
x <- Reader.ask
_ <- Writer.tell(x + 1)
} yield x

57. ExtEff Example
• To run the Eff requires explicit monad stack
実行にはエフェクトスタックの明示が必要です
type Stack = ([T] => Reader[Int, T]) :+: ([T] =>
Writer[Int, T]) :+: Nothing
scala> run(runWriter(runReader(e[Stack], 0))))
val res0: (List[Int], Int) = (List(1),0)

58. Problems of ExtEff
• Requires description of type parameters
• We can not make use of type inference.
型パラメータを引き回す必要があります

59. ExtEff with Subtyping
• Make type parameters covariant.
• Replace Open Union with Dotty’s Union
types.
Open UnionをDottyのUnion typesで置き換えます

60. Union types
• Values of type A | B are all values of type A
and type B
A ¦ BはAとBの値すべてをとります
val x: String | Int =
if util.Random.nextBoolean() then "hoge" else 0

61. Tagged Union
• Tagged Union is tagged to identify the value
of Union types.
Tagged Unionは値を識別するためにタグが付けられます
case class Union[+A](tag: Tag[_], value: A)
object Union {
def apply[F[_], A](value: F[A])(implicit F: Tag[F]) =
new Union(F, value)
}

62. Tag
• Tag is implemented with ClassTag.
• It makes unique identifiers from types.
case class Tag[F[_]](value: String)
object Tag {
implicit def __[F[_, _], T](implicit F: ClassTag[F[_,
_]], T: ClassTag[T]): Tag[[A] => F[T, A]] =
Tag(s"${F}[${T}, _]")
}
型から一意な識別子を作ります

63. New ExtEff
• The type parameter R becomes covariant.
• Impure is replaced Open Union with Tagged
Union.
ImpureはOpen UnionをTagged Unionで置き換えます
enum Eff[+R[_], A] {
case Pure(a: A)
case Impure[R[_], A, B](union: Union[R[A]], k: Arrs[R,
A, B]) extends Eff[R, B]
}
def lift[F[_]: Tag, A](fa: F[A]): Eff[F, A] =
Impure(Union(fa), a => Pure(a))

64. New ExtEff
• flatMap returns an union of effects.
flatMapはエフェクトの和を返します
enum Eff[+R[_], A] {
def flatMap[S[_], B](f: A => Eff[S, B])
: Eff[[T] => R[T] | S[T], B] =
this match {
case Eff.Pure(a) => f(a)
case Eff.Impure(u, k) =>
Eff.Impure(u, k :+ f)
}
}

65. New Handler
• Uses Tag to identify an effect.
Tagを使ってエフェクトを識別します
def handleRelay[F[_], R[_], A, B]
(eff: Eff[[T] => F[T] | R[T], A])
(pure: A => Eff[R, B])
(flatMap: F[A] => (A => Eff[R, B]) => Eff[R, B])
(implicit F: Tag[F]): Eff[R, B] =
eff match {
case Eff.Pure(a) => pure(a)
case Eff.Impure(Union(`F`, fa: F[A]), k) =>
flatMap(fa)(a => handleRelay(k(a))(pure)(flatMap))
case Eff.Impure(u: Union[R[A]], k) =>
Eff.Impure(r, a => handleRelay(k(a))(pure)(flatMap))
}

66. New ExtEff Example
• You can write more simply.
• Type inference works.
よりシンプルな記述が可能になりました
val e: Eff[[T] => Reader[Int, T] | Writer[Int, T], Int]
=
for {
x <- Reader.ask[Int]
_ <- Writer.tell(x + 1)
} yield x
scala> run(runWriter(runReader(e, 0)))
val res0: (Int, Int) = (1,0)

67. Benchmarks
• Comparison of ExtEff and
MonadTransformer.
• Count up with State monad in 1,000,000
loops.
• The effect stack gets deeper.
ExtEffとMTを比較します

68. Benchmarks in ExtEff
def benchEff(ns: Seq[Int])
: Eff[[A] => State[Int, A], Int] =
ns.foldLeft(Eff.Pure(1)) { (acc, n) =>
if n % 5 == 0 then for {
acc <- acc
s <- Reader.ask[Int]
_ <- Writer.tell(s + 1)
} yield acc max n
else acc.map(_ max n)
}
ExtEffのベンチマークコードです

69. Benchmarks in MT
MTのベンチマークコードです
def benchTrans[F[_]: Monad](ns: Seq[Int])
: StateT[F, Int, Int] = {
val m = StateT.stateTMonadState[Int, F]
ns.foldLeft(StateT.stateT(1)) { (acc, n) =>
if n % 5 == 0 then for {
acc <- acc
s <- m.get
_ <- m.put(s + 1)
} yield acc max n
else acc.map(_ max n)
}
}

70. Benchmarks
• Overlays State effects
Stateモナドを重ねます
def benchEff(): (Int, Int) =
Eff.run(State.run(0)(Bench.benchEff(1 to N)))
def benchEffS(): (String, (Int, Int)) =
Eff.run(State.run("")(State.run(0)(Bench.benchEff(1 to
N))))
def benchTrans(): (Int, Int) =
Bench.benchTrans[Id](1 to N).runRec(0)
def benchTransS(): (String, (Int, Int)) =
Bench.benchTrans[[A] => StateT[Id, String, A]](1 to
N).runRec(0).runRec("")

71. Benchmarks
• Run JMH with 20 warmups,100 iterations.
ExtEffの実行は線形時間、MTの実行は二乗時間です
Benchmark Mode Cnt Score Error Units
BenchJmh.benchEffJmh thrpt 100 5.724 ± 0.220 ops/s
BenchJmh.benchEffSJmh thrpt 100 5.133 ± 0.154 ops/s
BenchJmh.benchEffSSJmh thrpt 100 4.578 ± 0.175 ops/s
BenchJmh.benchTransJmh thrpt 100 4.928 ± 0.155 ops/s
BenchJmh.benchTransSJmh thrpt 100 1.848 ± 0.107 ops/s
BenchJmh.benchTransSSJmh thrpt 100 0.852 ± 0.042 ops/s

72. Benchmarks
• The run-time of ExtEff is linear.
• The run-time of MT is quadratic.
ExtEffはUnion Typesを使うことでシンプルにかけます
ops/s
0
1.5
3
4.5
6
ExtEff MT

73. Conclusions
• You can compose multiple effects by ExtEff.
• Using the union types makes writing ExtEff
simpler.
• If the monad stack is deep, ExtEff is faster
than MT.
ExtEffはUnion Typesを使うことでシンプルにかけます

74. Thank you for listening