Expression trees in c#, Алексей Голубь (Svitla Systems)

Speaker: Alexey Golub @Tyrrrz
Expression trees in C#
I heard you like code, so we put code in your code so you can code while you code

/whois ${speaker}
• Open-source developer ✨
• Conference speaker & blogger 🌐️
• C#, F#, JavaScript 💻
• Cloud & web ☁️
• Automation & DevOps ⚙️

What is an expression tree?

+Constant (2) Constant (3)
Plus operator
2 3
Binary expression

!string.IsNullOrWhiteSpace(personName)
? "Greetings, " + personName
: null;
string? GetGreeting(string personName)
{
return
}

"Greetings, "
!
personName
string.IsNullOrWhiteSpace( )
null;:
? +
personName
OPERATOR "NOT" METHOD CALL
PARAMETER
PARAMETERCONSTANT OPERATOR "ADD"
CONSTANT

{ Ternary conditional }
{ + }
TRUE
{ null }
FALSE
{ Method call }
CONDITION
{ string.IsNullOrWhiteSpace }
{ personName }
{ personName }
{ "Greetings, " }
{ ! }

Expression Tree
describes the structure of an expression

Constructing expression
trees manually

Expression.Constant(...) ConstantExpression
Expression.New(...) NewExpression
Expression.Assign(...) BinaryExpression
Expression.Equal(...) BinaryExpression
Expression.Call(...) MethodCallExpression
Expression.Condition(...) ConditionalExpression
Expression.Loop(...) LoopExpression
...

!string.IsNullOrWhiteSpace(personName)
? "Greetings, " + personName
: null;
Let’s recreate our expression dynamically

public Func<string, string?> ConstructGreetingFunction()
{
var personNameParameter = Expression.Parameter(typeof(string), "personName");
var isNullOrWhiteSpaceMethod = typeof(string)
.GetMethod(nameof(string.IsNullOrWhiteSpace));
var condition = Expression.Not(
Expression.Call(isNullOrWhiteSpaceMethod, personNameParameter));
var trueClause = Expression.Add(
Expression.Constant("Greetings, "),
personNameParameter);
var falseClause = Expression.Constant(null, typeof(string));
var conditional = Expression.Condition(condition, trueClause, falseClause);
var lambda = Expression.Lambda<Func<string, string?>>(conditional, personNameParameter);
return lambda.Compile();
}

var getGreeting = ConstructGreetingFunction();
var greetingForJohn = getGreeting("John");
The binary operator Add is not defined for the types 'System.String' and 'S
ystem.String'.

We need to call string.Concat() directly
var concatMethod = typeof(string)
.GetMethod(nameof(string.Concat), new[] {typeof(string), typeof(string)});
var trueClause = Expression.Call(
concatMethod,
Expression.Constant("Greetings, "),
personNameParameter);

var getGreetings = ConstructGreetingFunction();
var greetingsForJohn = getGreetings("John");
var greetingsForNobody = getGreetings(" ");
// "Greetings, John"
// <null>

Not everything is an expression
but we are not limited by that

new StringBuilder()
.Append("Hello ")
.AppendLine("world!");
Statements
Expression
Console.Write("Hello ");
Console.WriteLine("world!");

public Expression CreateStatementBlock()
{
var consoleWriteMethod = typeof(Console)
.GetMethod(nameof(Console.Write), new[] {typeof(string)});
var consoleWriteLineMethod = typeof(Console)
.GetMethod(nameof(Console.WriteLine), new[] {typeof(string)});
return Expression.Block(
Expression.Call(consoleWriteMethod, Expression.Constant("Hello ")),
Expression.Call(consoleWriteLineMethod, Expression.Constant("world!")));
}
var block = CreateStatementBlock();
var lambda = Expression.Lambda<Action>(block).Compile();
lambda(); // Hello world!

public Expression CreateStatementBlock()
{
var variableA = Expression.Variable(typeof(string), "a");
var variableB = Expression.Variable(typeof(string), "b");
return Expression.Block(
new[] {variableA, variableB},
Expression.Assign(variableA, Expression.Constant("Foo ")),
Expression.Assign(variableB, Expression.Constant("bar")),
Expression.Call(consoleWriteMethod, variableA),
Expression.Call(consoleWriteLineMethod, variableB));
}
Declare variables
Assign values to
variables
Reference variables

Optimizing reflection-heavy
code

How can we invoke Execute() from the outside?
public class Command
{
private int Execute() => 42;
}
public static int CallExecute(Command command) =>
(int) typeof(Command)
.GetMethod("Execute", BindingFlags.NonPublic | BindingFlags.Instance)
.Invoke(command, null);

public static class ReflectionCached
{
private static MethodInfo ExecuteMethod { get; } = typeof(Command)
.GetMethod("Execute", BindingFlags.NonPublic | BindingFlags.Instance);
public static int CallExecute(Command command) =>
(int) ExecuteMethod.Invoke(command, null);
}
public static class ReflectionDelegate
{
private static Func<Command, int> Impl { get; } =
(Func<Command, int>) Delegate
.CreateDelegate(typeof(Func<Command, int>), ExecuteMethod);
public static int CallExecute(Command command) => Impl(command);
}
Using cached MethodInfo
Using Delegate.CreateDelegate

public static class ExpressionTrees
{
private static Func<Command, int> Impl { get; }
static ExpressionTrees()
{
var instance = Expression.Parameter(typeof(Command));
var call = Expression.Call(instance, ExecuteMethod);
Impl = Expression.Lambda<Func<Command, int>>(call, instance).Compile();
}
public static int CallExecute(Command command) => Impl(command);
}
Lazy thread-safe
initialization via static
constructor

public class Benchmarks
{
[Benchmark(Description = "Reflection", Baseline = true)]
public int Reflection() => (int) typeof(Command)
.GetMethod("Execute", BindingFlags.NonPublic | BindingFlags.Instance)
.Invoke(new Command(), null);
[Benchmark(Description = "Reflection (cached)")]
public int Cached() => ReflectionCached.CallExecute(new Command());
[Benchmark(Description = "Reflection (delegate)")]
public int Delegate() => ReflectionDelegate.CallExecute(new Command());
[Benchmark(Description = "Expressions")]
public int Expressions() => ExpressionTrees.CallExecute(new Command());
public static void Main() => BenchmarkRunner.Run<Benchmarks>();
}
| Method | Mean | Error | StdDev | Ratio |
|---------------------- |-----------:|----------:|----------:|------:|
| Reflection | 192.975 ns | 1.6802 ns | 1.4895 ns | 1.00 |
| Reflection (cached) | 123.762 ns | 1.1063 ns | 1.0349 ns | 0.64 |
| Reflection (delegate) | 6.419 ns | 0.0646 ns | 0.0605 ns | 0.03 |
| Expressions | 5.383 ns | 0.0433 ns | 0.0383 ns | 0.03 |

Generic operators

What can we do to support multiple types?
public int ThreeFourths(int x) => 3 * x / 4;
public int ThreeFourths(int x) => 3 * x / 4;
public long ThreeFourths(long x) => 3 * x / 4;
public float ThreeFourths(float x) => 3 * x / 4;
public double ThreeFourths(double x) => 3 * x / 4;
public decimal ThreeFourths(decimal x) => 3 * x / 4;
public T ThreeFourths<T>(T x) => 3 * x / 4;
But we actually want something like this instead

public T ThreeFourths<T>(T x)
{
var param = Expression.Parameter(typeof(T));
var three = Expression.Convert(Expression.Constant(3), typeof(T));
var four = Expression.Convert(Expression.Constant(4), typeof(T));
var operation = Expression.Divide(Expression.Multiply(param, three), four);
var lambda = Expression.Lambda<Func<T, T>>(operation, param);
var func = lambda.Compile();
return func(x);
}
var a = ThreeFourths(18); // 13
var b = ThreeFourths(6.66); // 4.995
var c = ThreeFourths(100M); // 75M

public dynamic ThreeFourths(dynamic x) => 3 * x / 4;
Wait, how is it different from this?

public static class ThreeFourths
{
private static class Impl<T>
{
public static Func<T, T> Of { get; }
static Impl()
{
var param = Expression.Parameter(typeof(T));
var three = Expression.Convert(Expression.Constant(3), typeof(T));
var four = Expression.Convert(Expression.Constant(4), typeof(T));
var operation = Expression.Divide(Expression.Multiply(param, three), four);
var lambda = Expression.Lambda<Func<T, T>>(operation, param);
Of = lambda.Compile();
}
}
public static T Of<T>(T x) => Impl<T>.Of(x);
}
Generic, thread-safe
lazy initialization

public class Benchmarks
{
[Benchmark(Description = "Static", Baseline = true)]
[Arguments(13.37)]
public double Static(double x) => 3 * x / 4;
[Benchmark(Description = "Expressions")]
[Arguments(13.37)]
public double Expressions(double x) => ThreeFourths.Of(x);
[Benchmark(Description = "Dynamic")]
[Arguments(13.37)]
public dynamic Dynamic(dynamic x) => 3 * x / 4;
public static void Main() => BenchmarkRunner.Run<Benchmarks>();
}
| Method | x | Mean | Error | StdDev | Ratio | RatioSD |
|------------ |------ |-----------:|----------:|----------:|------:|--------:|
| Static | 13.37 | 0.6077 ns | 0.0176 ns | 0.0147 ns | 1.00 | 0.00 |
| Expressions | 13.37 | 1.9510 ns | 0.0163 ns | 0.0145 ns | 3.21 | 0.08 |
| Dynamic | 13.37 | 19.3267 ns | 0.1512 ns | 0.1340 ns | 31.82 | 0.78 |

Compiled dictionary

public TValue Lookup(TKey key) => key.GetHashCode() switch
{
// No collisions
9254 => value1,
-101 => value2,
// Collision
777 => key switch
{
key3 => value3,
key4 => value4
},
// ...
// Not found
_ => throw new KeyNotFoundException(key.ToString())
};
Dictionary lookup in a nutshell

public class CompiledDictionary<TKey, TValue> : IDictionary<TKey, TValue>
{
private readonly IDictionary<TKey, TValue> _inner = new Dictionary<TKey, TValue>();
private Func<TKey, TValue> _lookup;
public void UpdateLookup()
{
// ...
}
public TValue this[TKey key]
{
get => _lookup(key);
set => _inner[key] = value;
}
// The rest of the interface implementation is omitted for brevity
}

public void UpdateLookup()
{
var keyParameter = Expression.Parameter(typeof(TKey));
var keyGetHashCodeCall = Expression.Call(
keyParameter,
typeof(object).GetMethod(nameof(GetHashCode)));
var keyToStringCall = Expression.Call(
keyParameter,
typeof(object).GetMethod(nameof(ToString)));
var exceptionCtor = typeof(KeyNotFoundException)
.GetConstructor(new[] {typeof(string)});
var throwException = Expression.Throw(
Expression.New(exceptionCtor, keyToStringCall),
typeof(TValue));
var body = Expression.Switch(
// ...
));
var lambda = Expression.Lambda<Func<TKey, TValue>>(body, keyParameter);
_lookup = lambda.Compile();
}

var body = Expression.Switch(
typeof(TValue),
keyGetHashCodeCall,
throwException,
null,
_inner
.GroupBy(p => p.Key.GetHashCode())
.Select(g =>
{
if (g.Count() == 1)
return Expression.SwitchCase(
Expression.Constant(g.Single().Value),
Expression.Constant(g.Key));
return Expression.SwitchCase(
Expression.Switch(
typeof(TValue),
keyParameter,
throwException,
null,
g.Select(p => Expression.SwitchCase(
Expression.Constant(p.Value),
Expression.Constant(p.Key)
))),
Expression.Constant(g.Key));
}));
No collision
Collision

| Method | Count | Mean | Error | StdDev | Ratio |
|-------------------- |------ |----------:|----------:|----------:|------:|
| Standard dictionary | 10 | 24.995 ns | 0.1821 ns | 0.1704 ns | 1.00 |
| Compiled dictionary | 10 | 9.366 ns | 0.0511 ns | 0.0478 ns | 0.37 |
| | | | | | |
| | | | | | |

Parsing into expression trees

public static class SimpleCalculator
{
private static readonly Parser<Expression> Constant =
Parse.DecimalInvariant
.Select(n => double.Parse(n, CultureInfo.InvariantCulture))
.Select(n => Expression.Constant(n, typeof(double)))
.Token();
private static readonly Parser<ExpressionType> Operator =
Parse.Char('+').Return(ExpressionType.Add)
.Or(Parse.Char('-').Return(ExpressionType.Subtract))
.Or(Parse.Char('*').Return(ExpressionType.Multiply))
.Or(Parse.Char('/').Return(ExpressionType.Divide));
private static readonly Parser<Expression> Operation =
Parse.ChainOperator(Operator, Constant, Expression.MakeBinary);
private static readonly Parser<Expression> FullExpression =
Operation.Or(Constant).End();
public static double Run(string expression)
{
var operation = FullExpression.Parse(expression);
var func = Expression.Lambda<Func<double>>(operation).Compile();
return func();
}
}

var a = SimpleCalculator.Run("2 + 2");
var b = SimpleCalculator.Run("3.15 * 5 + 2");
var c = SimpleCalculator.Run("1 / 2 * 3");
// 4
// 17.75
// 1.5

Inferring expression trees
from code

Func<int, int, int> div =
(a, b) => a / b;
Expression<Func<int, int, int>> divExpr =
(a, b) => a / b;
Same value, different type
Console.WriteLine(divExpr.Type);
// System.Func`3[System.Int32,System.Int32,System.Int32]

(a, b) => a / b;
(a, b) => a / b;
foreach (var param in divExpr.Parameters)
Console.WriteLine($"Param: {param.Name} ({param.Type.Name})");
// Param: a (Int32)
// Param: b (Int32)

(a, b) => a / b;
(a, b) => a / b;
var div = divExpr.Compile();
var c = div(10, 2); // 5

(a, b) => a / b;
(a, b) => a / b;
Product
Recipe

Limitations
Func<int, int, int> div = (a, b) => a / b;
Expression<Func<int, int, int>> divExpr = div;
Compilation error

Limitations
Expression<Func<int, int, int>> divExpr = (a, b) =>
{
var result = a / b;
return result;
}; Compilation error
Expression<Action> writeToConsole = () =>
{
Console.Write("Hello ");
Console.WriteLine("world!");
};
Compilation error

Limitations
• Null-coalescing operator (obj?.Prop)
• Dynamic variables (dynamic)
• Asynchronous code (async/await)
• Default or named parameters (func(a, b: 5), func(a))
• Parameters passed by reference (int.TryParse("123", out var i))
• Multi-dimensional array initializers (new int[2, 2] { { 1, 2 }, { 3, 4 } })
• Assignment operations (a = 5)
• Increment and decrement (a++, a--, --a, ++a)
• Base type access (base.Prop)
• Dictionary initialization (new Dictionary<string, int> { ["foo"] = 100 })
• Unsafe code (via unsafe)
• Throw expressions (throw new Exception())
• Tuple literals ((5, x))
Can’t use any of the following:

Identifying type members

How can we get PropertyInfo of Dto.Id?
public class Dto
{
public Guid Id { get; set; }
public string Name { get; set; }
}
var idProperty = typeof(Dto).GetProperty(nameof(Dto.Id));
Console.WriteLine($"Type: {idProperty.DeclaringType.Name}");
Console.WriteLine($"Property: {idProperty.Name} ({idProperty.PropertyType.Name})");
// Type: Dto
// Property: Id (Guid)

public class Validator<T>
{
// Add validation predicate to the list
public void AddValidation<TProp>(string propertyName, Func<TProp, bool> predicate)
{
var propertyInfo = typeof(T).GetProperty(propertyName);
if (propertyInfo is null)
throw new InvalidOperationException("Please provide a valid property name.");
// ...
}
// Evalute all predicates
public bool Validate(T obj) { /* ... */ }
/* ... */
}
var validator = new Validator<Dto>();
validator.AddValidation<Guid>(nameof(Dto.Id), id => id != Guid.Empty);
validator.AddValidation<string>(nameof(Dto.Name), name => !string.IsNullOrWhiteSpace(name));
var isValid = validator.Validate(new Dto { Id = Guid.NewGuid() }); // false

What if we wanted to change the property type?
public class Dto
{
public int Id { get; set; }
public string Name { get; set; }
}
validator.AddValidation<Guid>(nameof(Dto.Id), id => id != Guid.Empty);
Still compiles, even though there is now an error

public class Validator<T>
{
public void AddValidation<TProp>(
Expression<Func<T, TProp>> propertyExpression,
Func<TProp, bool> predicate)
{
var propertyInfo = (propertyExpression.Body as MemberExpression)?.Member as PropertyInfo;
if (propertyInfo is null)
throw new InvalidOperationException("Please provide a valid property expression.");
// ...
}
public bool Validate(T obj) { /* ... */ }
/* ... */
}
Expression is used to
identify a property
var validator = new Validator<Dto>();
validator.AddValidation(dto => dto.Id, id => id != Guid.Empty);
validator.AddValidation(dto => dto.Name, name => !string.IsNullOrWhiteSpace(name));
var isValid = validator.Validate(new Dto { Id = Guid.NewGuid() }); // false

Providing context to assertions

[Test]
public void IntTryParse_Test()
{
// Arrange
const string s = "123";
// Act
var result = int.TryParse(s, out var value);
// Assert
Assert.That(result, Is.True, "Parsing was unsuccessful");
Assert.That(value, Is.EqualTo(124), "Parsed value is incorrect");
}
X IntTryParse_Test [60ms]
Error Message:
Parsed value is incorrect
Expected: 124
But was: 123

public static class AssertEx
{
public static void Express(Expression<Action> expression)
{
var act = expression.Compile();
try
{
act();
}
catch (AssertionException ex)
{
throw new AssertionException(
expression.Body.ToReadableString() +
Environment.NewLine +
ex.Message);
}
}
}
Extension method from
ReadableExpressions package

X IntTryParse_Test [60ms]
Error Message:
Assert.That(value, Is.EqualTo(124))
Expected: 124
But was: 123
[Test]
public void IntTryParse_Test()
{
// Arrange
const string s = "123";
// Act
var result = int.TryParse(s, out var value);
// Assert
AssertEx.Express(() => Assert.That(result, Is.True));
AssertEx.Express(() => Assert.That(value, Is.EqualTo(124)));
}

Traversing and rewriting
expression trees

public class Visitor : ExpressionVisitor
{
protected override Expression VisitMethodCall(MethodCallExpression node)
{
Console.WriteLine($"Visited method call: {node}");
return base.VisitMethodCall(node);
}
protected override Expression VisitBinary(BinaryExpression node)
{
Console.WriteLine($"Visited binary expression: {node}");
return base.VisitBinary(node);
}
}

Expression<Func<double>> expr = () => Math.Sin(Guid.NewGuid().GetHashCode()) / 10;
new Visitor().Visit(expr);
// Visited binary expression: (Sin(Convert(NewGuid().GetHashCode(), Double)) / 10)
// Visited method call: Sin(Convert(NewGuid().GetHashCode(), Double))
// Visited method call: NewGuid().GetHashCode()
// Visited method call: NewGuid()

public class Visitor : ExpressionVisitor
{
{
var newMethodCall = node.Method == typeof(Math).GetMethod(nameof(Math.Sin))
? typeof(Math).GetMethod(nameof(Math.Cos))
: node.Method;
return Expression.Call(newMethodCall, node.Arguments);
}
}

Expression<Func<double>> expr = () => Math.Sin(Guid.NewGuid().GetHashCode()) / 10;
var result = expr.Compile()();
Console.WriteLine($"Old expression: {expr.ToReadableString()}");
Console.WriteLine($"Old result: {result}");
var newExpr = (Expression<Func<double>>) new Visitor().Visit(expr);
var newResult = newExpr.Compile()();
Console.WriteLine($"New expression: {newExpr.ToReadableString()}");
Console.WriteLine($"New result value: {newResult}");
// Old expression: () => Math.Sin((double)Guid.NewGuid().GetHashCode()) / 10d
// Old result: 0.09489518488876232
// New expression: () => Math.Cos((double)Guid.NewGuid().GetHashCode()) / 10d
// New result value: 0.07306426748550407

Transpiling code into a different
language

Expression<Action<int, int>> expr = (a, b) =>
Console.WriteLine("a + b = {0}", a + b));
var fsharpCode = FSharpTranspiler.Convert(expr);

public static class FSharpTranspiler
{
private class Visitor : ExpressionVisitor
{
private readonly StringBuilder _buffer;
public Visitor(StringBuilder buffer)
{
_buffer = buffer;
}
// ...
}
public static string Convert<T>(Expression<T> expression)
{
var buffer = new StringBuilder();
new Visitor(buffer).Visit(expression);
return buffer.ToString();
}
}

protected override Expression VisitLambda<T>(Expression<T> node)
{
_buffer.Append("fun (");
_buffer.AppendJoin(", ", node.Parameters.Select(p => p.Name));
_buffer.Append(") ->");
return base.VisitLambda(node);
}
{
if (node.Method.DeclaringType == typeof(Console) &&
node.Method.Name == nameof(Console.WriteLine))
{
_buffer.Append("printfn ");
if (node.Arguments.Count > 1)
{
var format = (string) ((ConstantExpression) node.Arguments[0]).Value;
var formatValues = node.Arguments.Skip(1).ToArray();
_buffer.Append(""").Append(Regex.Replace(format, @"{d+}", "%O")).Append("" ");
_buffer.AppendJoin(" ", formatValues.Select(v => $"({v.ToReadableString()})"));
}
}
return base.VisitMethodCall(node);
}

var fsharpCode = FSharpTranspiler.Convert<Action<int, int>>(
(a, b) => Console.WriteLine("a + b = {0}", a + b));
> let foo = fun (a, b) -> printfn "a + b = %O" (a + b)
val foo : a:int * b:int -> unit
> foo (3, 5)
a + b = 8
val it : unit = ()
// fun (a, b) -> printfn "a + b = %O" (a + b)

Summary
• Expression trees are fun
• We can make reflection-heavy code much faster
• We can do late-binding with almost no performance penalties
• We can write our own runtime-compiled DSL
• We can provide refactor-safe identification for type members
• We can analyze specified lambdas and reflect on their structure
• We can rewrite existing expressions to behave differently
• We can transpile code into other languages

Useful packages
• FastExpressionCompiler
https://github.com/dadhi/FastExpressionCompiler
Speeds up LambdaExpression.Compile()
• ReadableExpressions
https://github.com/agileobjects/ReadableExpressions
Converts expression to readable C# code

Learn more
• Working with expression trees in C# (by me)
https://tyrrrz.me/blog/expression-trees
• Introduction to expression trees (MS docs)
https://docs.microsoft.com/en-us/dotnet/csharp/expression-trees
• Expression trees in enterprise software (Maksim Arshinov)
https://youtube.com/watch?v=J2XzsCoJM4o

Thank you!

Expression trees in c#, Алексей Голубь (Svitla Systems)

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Expression trees in c#, Алексей Голубь (Svitla Systems)

Similar to Expression trees in c#, Алексей Голубь (Svitla Systems) (20)

More from Alina Vilk

More from Alina Vilk (20)

Recently uploaded

Recently uploaded (20)

Expression trees in c#, Алексей Голубь (Svitla Systems)