Part 1 - Expression Tree basics
Part 2 - IQueryable composition
Sample projects containing a demonstration of the theory explained in this article are available on GitHub:
GitHub

In part 1 we learned that you can swap parts of an Expression Tree to another compatible (i.e. with a matching return type) expression. Swapping is, in fact, the easiest thing to do - with a bit more work we can construct a serializable representation of almost any bit of C# code. This opens great avenues for Domain Driven Development and introducing hot-swappable, dynamic, yet safe parts of logic to your application.

One of the best examples of the power we get is shown by introducing a reusable expression function with LINQKit.

internal static class AddressSubqueries
{
internal static Expression<Func<string, string, string>> FormatCityAndProvince =
    (city, province) => "The glorious city of " + city 
                            + " of the wonderful province of " + province;
}
     
//used like this:
public IQueryable<string> GetStandardAddressDescription(int addressId)
{
    return DataContext
                .Addresses.AsExpandable() // this hooks in LINQKit 
                .Where(x => x.AddressId == addressId)
                .Join(
                    DataContext.StateProvinces,
                    adr => adr.StateProvinceId,
                    prov => prov.StateProvinceId,
                    (adr, prov) => AddressSubqueries.
                                        FormatCityAndProvince // <==
                                            .Invoke(adr.City, prov.Name))
                .FirstOrDefault();
}

As you can imagine, via .AsExpandable() hook LINQKit gets the chance to analyze the Expression Tree representing our query, and substitute all .Invoke calls to expression tree held inside the variables invoked. In fact, as you would usually want a reusable way to get your company’s 'standard' address description from just addressId in other queries, it makes sense to have the entire GetStandardAddressDescription available as an expression function:

internal static class AddressSubqueries
{
internal static Expression<Func<string, string, string>> 
                                             FormatCityAndProvince =
    (city, province) => "The glorious city of " + city + 
                            " of the wonderful province of " + province;

internal static Expression<Func<IAdventureWorksDataContext, int, string>> 
                                                 GetStandardAddressDescription =
(DataContext, addressId) => 
            DataContext
                .Addresses
                .AsExpandable()
                .Where(x => x.AddressId == addressId)
                .Join(
                    DataContext.StateProvinces,
                    adr => adr.StateProvinceId,
                    prov => prov.StateProvinceId,
                    (adr, prov) => FormatCityAndProvince
                                         .Invoke(adr.City, prov.Name))
                .FirstOrDefault();
}

//used like this
public IQueryable<string> GetLocationsByBusinessEntityId(int businessEntityId)
{
    return DataContext
        .BusinessEntityAddresses.AsExpandable()
        .Where(bae => bae.BusinessEntityId == businessEntityId)
        .Select(bae => AddressSubqueries
                            .GetStandardAddressDescription
                            .Invoke(DataContext, bae.AddressId));
}

We will get back to LINQKit example later. For now we will look at a typical task in enterprise application - 'we need this data shown to our managers in a grid, in web app, with smart filters on every column by tomorrow'. We assume you have a generic grid component which just needs a model description (names and types of properties) and a little configuration of column views and provides you with a nice grid where users can click the headers for flexible filtration and sorting options, which your grid than sends in form of JSON to your back-end.

Now you need to use this JSON, essentially, just a textual representation of grid options, with the IQueryable that services this grid. In older days we would have used Linq.Dynmaic for this, and use bare strings. But since that project looks to be abandoned, we will do it with our own code.

This task is a nice start, because it introduces us to Expression Tree construction and code generation with reflection. It is also easy, since we only add to the existing tree, so we don't need to walk it with a visitor or swap parts. Using the following dto that we would receive from front-end:

var req = new GridRequest()
{
    Skip = 2,
    Take = 2,
    Filter = new GridRequestFilter[]
    {
        new GridRequestFilter()
        {
            PropName = "ProductNumber",
            Operator = "Contains",
            JsonValue = "'-54'"
        }
    },
    Sort = new GridRequestSort[]
    {
        new GridRequestSort()
        {
            PropName = "ProductId",
            IsDescending = true
        },
    }
};

We apply filtering and sorting to our IQueryable, as if we were doing it in our code, i.e. using reflection we will reproduce the following code.

// lambda expression will be constructed dynamically
Expression<Funce<OurDto, bool>> lambdaFilter = 
                            (x) => x.ProductNumber.Contains("-54");

ourQueryable = ourQueryable.Where(lambdaFilter);

 // lambda expression will be constructed dynamically
Expression<Funce<OurDto, bool>> lambdaSort = (x) => x.ProductId;
                                  
// Since OrderBy and other ordering methods
// are generic and depend on the type of property used to order,
// concrete generic method will be constructed and invoked with reflection
ourQueryable = ourQueryable.OrderByDescending<OurDto, int>(lambdaSort);

Like this:

                                   
private static IQueryable<T> WrapFilter<T>(
                                    IQueryable<T> query,
                                    GridRequestFilter filter)
{
    var type = typeof(T);
    // get property from string name
    PropertyInfo prop = type.GetProperty(filter.PropName, 
                                        BindingFlags.Public 
                                        | BindingFlags.Instance 
                                        | BindingFlags.IgnoreCase);

    // prepare the lambda representing this filter
    ParameterExpression parameter = Expression.Parameter(type, "par");

    MemberExpression propAccessExpr = Expression
                                        .MakeMemberAccess(parameter, prop);

    // this assumes Newtonsoft.JSON for simplicity. 
    // in the production lib mentioned above, deserialization function
    // is injected by the user
    object value = JsonConvert
                        .DeserializeObject(filter.JsonValue, 
                                            prop.PropertyType);

    Expression<Func<T, bool>> filterExpr;

    // translate string like 'Eq' into corresponding expression
    Expression comparison = GetComparator(
                                filter.Operator, 
                                prop.PropertyType, 
                                propAccessExpr, 
                                Expression.Constant(value));

    // wrap in null-check if needed
    if (IsNullable(prop.PropertyType))
    {
        BinaryExpression nullCheck = Expression.NotEqual(
                                                propAccessExpr, 
                                                Expression.Constant(null));
        comparison = Expression.And(nullCheck,comparison);  
    }

    filterExpr = Expression.Lambda<Func<T, bool>>(
                                                comparison, 
                                                new[] { parameter });

    IQueryable<T> newQuery = query.Where(filterExpr);
    return newQuery;
}

private static IQueryable<T> WrapSort<T>(
                                    IQueryable<T> query,
                                    GridRequestSort sort,
                                    bool isFirst = false)
{
    // prepare lambda like (x) => x.y;
    LambdaExpression propAccessExpr = 
                    GetPropAccesssLambdaExpr(typeof(T), sort.PropName);

    var orderMethodName = "";

    if (isFirst)
    {
        orderMethodName = sort.IsDescending 
                                ? "OrderByDescending" 
                                : "OrderBy";
    } 
    else
    {
        orderMethodName = sort.IsDescending 
                                ? "ThenByDescending" 
                                : "ThenBy";
    }

    // find the right generic method definition
    MethodInfo method = typeof(Queryable)
                        .GetMethods()
                        .FirstOrDefault(m => m.Name == orderMethodName 
                                            && m.GetParameters().Length == 2);

    // make a concrete version of the method based on the type
    // of property returned by access lambda
    MethodInfo genericMethod = method
                                .MakeGenericMethod(typeof(T), 
                                            propAccessExpr.ReturnType);

    // invoke it to produce new IQueryable
    IQueryable<T> newQuery = 
            (IQueryable<T>)genericMethod.Invoke(
                                            null, 
                                            new object[] 
                                                { 
                                                    query, 
                                                    propAccessExpr 
                                                });
    return newQuery;
}

private static Expression GetComparator(
                                string Operator, 
                                Type propType, 
                                Expression left, 
                                Expression right)
{
    switch (Operator)
    {
        case "Gt":
            return Expression.GreaterThan(left, right);
        case "Eq":
            return Expression.Equal(left, right);
        // lots more cases
        default :
            // try to find a method by the name of Operator
            // just to show this is possible, not for production use
            var MethodInfo = propType.GetMethod(Operator, 
                                            BindingFlags.Public 
                                            | BindingFlags.Instance 
                                            | BindingFlags.IgnoreCase);
            return Expression.Call(left, MethodInfo, right);
    }
}

private static bool IsNullable (Type type)
{
return type.IsClass
            || (type.IsGenericType 
                && type.GetGenericTypeDefinition() == typeof(Nullable<>));
}

private static LambdaExpression GetPropAccesssLambdaExpr(Type type, string name)
{
    PropertyInfo prop = type.GetProperty(name, 
                                BindingFlags.Public 
                                | BindingFlags.Instance 
                                | BindingFlags.IgnoreCase);

    ParameterExpression param = Expression.Parameter(type);
    MemberExpression propAccess = Expression.Property(param, prop.Name);
    LambdaExpression expr = Expression.Lambda(propAccess, param);
    return expr;
}

Next we will take a look at how to examine an existing Expression Tree and possibly modify it. When we were talking about unit testing your complex Entity Framework 6 queries, we saw that some function like 'DbFunctions.DiffDays(workOrder.StartDate, workOrder.EndDate)' lacked a C# implementation, as they would be translated into their SQL equivalent call for query execution. In unit testing, such functions would throw a NotImplemented exception. We need to shim them.

For that purpose we will use an Expression Visitor. It is a class that copies/modifies a given expression tree by recursively traversing it. It allows us to override methods for 'visiting' a particular node in the old tree and either accept it for the new tree, or return a different expression tree in its place.

public static class QueryableExtensions
{
    // Extension method that will shim the calls we are interested in
    public static IQueryable<T> 
                        FixDbFunctionCalls<T>(this IQueryable<T> source)
    {
        var replacer = new MethodCallReplacer();
        Expression newExpresiion = replacer.Visit(source.Expression);
        IQueryable<T> newQueryable = source
                                        .Provider
                                        .CreateQuery<T>(newExpresiion);

        return newQueryable;
    }
}

// This class aggregates all available replacements
// and traverses the tree, substituting method calls.
// Every replacement class handles one particular method.
// The base class provided with .NET has a virtual method 
// for examining each node type
public class MethodCallReplacer : ExpressionVisitor
{
    private MethodReplacement[] AvailableReplacements { get; set; } =
        new MethodReplacement[]
        {
            new DiffDaysReplacement()
        };

    // we are interested in function calls
    protected override Expression VisitMethodCall(MethodCallExpression node)
    {
        // does the method being called fit any replacement? 
        var methodReplacement = AvailableReplacements
            .FirstOrDefault(x => node.Method == x.MethodToReplace);

        if(methodReplacement == null)
        {
            return base.VisitMethodCall(node);
        }
        
        // produce a new Expression 
        Expression newMethodCall = methodReplacement
                                        .ReplaceCall(this, node);

        // swap old one for the new one
        return newMethodCall;
    }
}

public abstract class MethodReplacement
{
    public MethodInfo MethodToReplace { get; protected set; }

    public abstract Expression ReplaceCall(
        ExpressionVisitor visitor, MethodCallExpression callToReplace);
}

class DiffDaysReplacement : MethodReplacement
{
    public DiffDaysReplacement()
    {
        // which method we want to replace
        MethodToReplace = typeof(DbFunctions).GetMethod(
            nameof(DbFunctions.DiffDays), new[] 
                                            { 
                                                typeof(DateTime?), 
                                                typeof(DateTime?) 
                                            });
    }

    // C# implementation to be used in its place
    public static int? DateDiffEquivalent(
                                DateTime? dateLeft, 
                                DateTime? dateRight)
    {
        return dateLeft.HasValue && dateRight.HasValue
                ? (int?)(dateRight.Value - dateLeft.Value).TotalDays
                : (int?) null;
    } 

    public override Expression ReplaceCall(
        ExpressionVisitor visitor, MethodCallExpression nodeToReplace)
    {
        // substitute the old method call for the new method call, 
        // preserving any argument expressions.
        var argumentsOfExistingCall =  nodeToReplace.Arguments;
                                        
        // Note, that argument expressions can themselves be complex,
        // they have to be visited themselves, 
        // and then plugged into the new call.
        Expression dateArgLeftExpr = 
                    visitor.Visit(argumentsOfExistingCall[0]);
        Expression dateArgRightExpr = 
                    visitor.Visit(argumentsOfExistingCall[1]);
        MethodInfo replacementMethod = 
                    this.GetType().GetMethod(nameof(DateDiffEquivalent));
        MethodCallExpression replacementMethodCallExpr = 
                                Expression.Call(replacementMethod, 
                                                dateArgLeftExpr, 
                                                dateArgRightExpr);

        return replacementMethodCallExpr;
    }
}

Now you can use the extension method to fix you IQueryables by shimming.

[TestMethod]
public void GetWorkOrderSummaries_Test()
{
    var context = Context();
    // populate fake context sets with test data

    var repo = new ProductsRepository(context);

    var result = repo
                    .GetWorkOrderSummaries()
                    .FixDbFunctionCalls()   // fix
                    .ToArray();

    Assert.IsTrue(result.Count() == 1);
    var item = result[0];
    Assert.IsTrue(item.ModelName == "MODELNAME");
    Assert.IsTrue(item.ProductModelId == 2);
    Assert.IsTrue(item.RoutingsCount == 2);
    Assert.IsTrue(item.DurationDays == 5);  //days diff
    //... more assertions
}

Returning to the example at the beginning of the article, you can imagine, that the functionality of LINQKit can be replicated by using an Expression Visitor that looks for occurrences of x.Invoke and substitutes the invocation by inlining the Expression Tree of x and substituting usage of its parameters for the actual argument expressions.

If you want to see quick production-tested examples of what else you can do with Expression Tress, check out my other projects:
Linq.AutoProject - during 'select' part of your queries, project one dto into another, with all compatible properties mapped automatically, and you only need to handle the non-trivial mappings.
DeepPropertyAccessor - extract a deeply nested property or field from complex DTOs with a detailed description of where the chain was interrupted (instead of just null with '?.' operator), for cases where you need to diagnose missing data, so the log has to be specific about how far the extraction got.

Any farther knowledge on Expression Trees is best obtained by building you own functionality around them. A few words of advice:

- When you are trying to construct an expression tree, and don't know where to start - write it out as a C# Expression<Func<....>> and examine that at runtime.

- Use Roslyn syntax visualizer to peek inside the structure of your methods. Remember though, similar as it may be to expression trees, it deals with the syntax tree before compilation, implicit things added during compilation will not have a representation in the visualizer. Check next paragraph, to see what I mean.

- Some things are implicit in C# code, but explicit after compilation. For example:
Expression<Func<Dto,object>> someFunc = (x) => x.y;
will invoke a boxing operation if type of y is a value type:
x => Convert(x.y)

- Also, subtle changes producemeaningful differences in the resulting Expression Tree:

// two lambdas look similar, but have a different
// expression for body
Expression<Func<Dto>> someFunc1 = () => new Dto(); // NewExpression
Expression<Func<Dto>> someFunc2 = () => new Dto{}; // MemberInitExpression

The piece of code also demonstrates a thing to remember - some code does not make sense, (empty '{}' bindings section in the constructor in second func), but is valid from C# point of view, and you have to think about such cases when designing your library. Since any failure you get will be in production, you must anticipate weird scenarios. Also, good idea is to pprogram defensively and give out verbose exceptions with a comprehensive description of what went wrong and what is a likely solution. If an exception gives out enough information to fix the problem without having to look in the documentation - you've done a good job.

- Pattern matching from C# 7 is a bliss when working with Expression Trees. Most of you code will look like this:

private MemberInitExpression ExtractMemberInitExpressionFromLambda(
                                                LambdaExpression expression)
{
    switch (expression.Body)
    {
        case NewExpression newExpr:
            return Expression.MemberInit(newExpr);
        case MemberInitExpression memberInitExpr:
            return memberInitExpr;
        default:
            throw new ArgumentException(
                $"Can't handle expression of type {expression.NodeType}");
    }
}