This project has moved. For the latest updates, please go here.

Expressions containing another expression

Aug 19, 2014 at 1:48 PM
Edited Aug 20, 2014 at 5:36 AM
Hi Rupert,

I want to use this library for evaluating dependent expressions as mentioned in pseudo code below:
A = "10"  
B = "A+10"       
C = "A+B"        

Eval("B")       // Expect 20
Eval("C")       // Expect 30

A = 20

Eval("B")       // Expect 30
Eval("C")       // Expect 40
Will this be possible in conjunction with scoping?
Developer
Aug 20, 2014 at 10:05 AM
Hi,

I'm a bit confused. A, B and C are expressions, i.e. strings? but at the same time you want to use the results of those expressions inside the other expressions. What you are saying is that A, B ,and C are actually functions that return values of expressions, not the values themselves. I assume that A, B and C should be able to be set by the end user.

I don't think what you are saying can be done in C# without using delegates.

Assuming A, B and C were all Func<int> properties on a class instance c, if I rewrote it like this it would work in C#:
            c.A = () => 10;
            c.B = () => c.A() + 10;
            c.C = () => c.A() + c.B();

            Debug.WriteLine(c.B());       // Expect 20
            Debug.WriteLine(c.C());       // Expect 30

            c.A = () => 20;

            Debug.WriteLine(c.B());       // Expect 30
            Debug.WriteLine(c.C());       // Expect 50
This way you are not simply getting the value, you would be executing an expression that returns a value.

lambdas are not currently supported in ExpressionEval so we cannot do this anyway. But I think I understand what you want to do and I will try to think of a possible approach.
Aug 20, 2014 at 12:36 PM
Hi,

I'm sorry for making it a bit confusing and vague. Let me try to lay it out this way.

My end users enter a formula (basically, a valid expression) through a gui, which I'm gonna persist in my database. Each expression would be referenced by a specific, unique (within a namespace) atom name. Other expressions can then refer to these atom names such that it allows us to reuse my expressions across the entire scope. This will require a dependency graph for each expression. An ideal candidate for such work would be a logical container, something like an evaluation engine, which manages all of these expressions and maintains their dependencies.

At the start of my programme, the engine would load, parse, compile all of these expressions through the container. When I ask the engine to evaluate an expression, it would also automatically evaluate all input expressions (on which this expression depends).

A very similar project exists on Flee. However, this project is pretty old, the developer is not even contactable and it contains a very serious limitation: the atom names can neither be scoped nor can the symbol replacements (value of the variables) be altered on the fly.

I'm guessing that current project has full potential to accommodate this work relatively easily as the basic expression evaluation framework is very solid. When I created this thread, I thought this function might be existing.

I can extend my personal support in bringing this forward as this functionality can bring this library to the next level. What are your views?

Thanks,
Kinjal
Developer
Aug 21, 2014 at 12:23 AM
Edited Aug 21, 2014 at 2:19 AM
The goal of this library is to provide runtime compilation of C# code. The feature you are suggesting isn't part of the C# specification so I don't think it should be part of the library itself. It is important that this library stay as close to C# specification as possible so that anyone can build on top of it easily.

That said, it is an interesting use case and I believe it can be implemented on top of this library rather easily.

I think the key thing here is to separate the atom name from the expression itself. The expressions should be an actual assignment to a scope variable, and then the atom name should just correlate to the left hand part of the expression. When a string expression is compiled, a LINQ Expression tree is created that encapsulates the expression as a code structure, an abstract syntax tree. We can analyze this tree as a dependency tree. For each node in the tree we would have to execute the associated assignment expression so that the corresponding scope variable is updated.

In your example:
Expression: "A = 10"
LINQ Expression Tree: $scope.A = 10 
Left-hand: $scope.A
Atom: "A"
We would then add this to a dictionary whose key is the atom and whose value is an aggregate of the string expression, the LINQ tree and the compiled function.
Similarly:
Expression: "B = A+10"
LINQ Expression Tree: $scope.B = $scope.A + 10 
Left-hand: $scope.B
Atom: "B"
and
Expression: "C = A+B"
LINQ Expression Tree: $scope.C = $scope.A + $scope.B 
Left-hand: $scope.C
Atom: "C"
Now your extended expression parser intercepts the expression, for example, "C", looks it up in the dictionary, recompiles it if it has changed, then takes the LINQ expression and traverses the right hand part. If it encounters a scope variable (PropertyExpression with $scope as the parent), it will look up that property name in the dictionary, and repeat until it reaches the leaves. It executes the compiled function at each node, ensuring that the corresponding assignment expression is executed so that the scope variable is updated.

So in your example, setting A = 20 will flag Atom A as being dirty and requiring recompilation. When "C" is evaluated, it will lookup the dictionary, get the expression, traverse the tree up to the assignment of A and recompile the expression "A = 20" and then execute it. This will ensure that the scope variable A is updated. Then atom B is evaluated. As the expression hasn't changed, you don't have to recompile (saves time) and just execute the function. Now Scope.B is updated. Finally it will execute the entire expression of assignment of C, the sum of scope.A and scope.B, and return the value.

The only problem I see is that you should not set values for the scope variables outside your expression parser, i.e. in normal code. The Scope object is simply a container for values. Any values set outside would be overwritten as the engine would execute the assignment expressions every run. You would also have to check for circular dependencies in your expressions.

I can probably come up with a working proof of concept today.

Does this method work for you?
Aug 21, 2014 at 5:16 AM
Edited Aug 21, 2014 at 5:29 AM
Absolutely! On the face of it, this looks like what I need. A POC will be great starter for me. I'm also trying to get my hands dirty into something similar.

I guess, along with all other items in the dictionary, a collection of input tokens can also be stored.

One of my other requirement that could play some part in this is the atom names should be unique only within a logical scope. This allows me to reuse them across other similar scopes and also to set values from a data store. Such that I can iteratively load values, set them, evaluate the expression across huge number of records.

Thanks,
Kinjal
Developer
Aug 21, 2014 at 8:19 AM
I have some working code, please take a look at it:

http://pastebin.com/7UJTb00f
Aug 21, 2014 at 11:49 AM
Edited Aug 21, 2014 at 12:19 PM
Hi Rupert,

Wow! That's really quick! Even guys who are paid for support do not reply with this speed and precision! Yep. This definitely suits me. I'm gonna put this to a more real world test and share my experience.

One thing that I could not figure out: How do I cache the compiled expression without tying it to a scope? In my case most of my expressions are repeated, but the scope is changed at runtime. It would be ideal if we could pass in the scope when we invoke eval.

I may require your further help, though I would really try to avoid troubling you further.

Many Thanks,
Kinjal
Aug 27, 2014 at 7:27 AM
Edited Aug 27, 2014 at 7:28 AM
Hi,

Regret stretching this thread this far, but I'm really stuck.

I'm almost done with my design. However, it's virtually impossible for me to know and design scope classes in advance. I'll have to use dynamic objects. That's when I get a type conversion exception:
Expression of type 'System.Object' cannot be used for return type 'System.Double'. 
I thought it had something to do with the new design. So, I tried basic scope compile. It seems, ScopeCompile is not comfortable with dynamic guys. I'm pasting my test code below:
        public class SimpleScope
        {
            public double Basic { get; set; }
            public double Bonus { get; set; }
        }

        public class DynamicScope : DynamicObject
        {
            protected readonly Dictionary<string, object> _members = new Dictionary<string,object>();

            public override bool TryGetMember(GetMemberBinder binder, out object result)
            {
                if (_members.ContainsKey(binder.Name))
                {
                    result = _members[binder.Name];
                    return true;
                }
                else
                {
                    return base.TryGetMember(binder, out result);
                }
            }

            public override bool TrySetMember(SetMemberBinder binder, object value)
            {
                _members[binder.Name] = value;
                return true;
            }

            public object this[string member]
            {
                get
                {
                    if (_members.ContainsKey(member))
                        return _members[member];
                    else
                        return null;
                }
                set
                {
                    _members[member] = value;
                }
            }
        }

        protected static void TestScopes()
        {
            var expression = "(Basic * 2) + Bonus";

            var expr = new CompiledExpression<double>(expression);
            
            var func = expr.ScopeCompile<SimpleScope>();
            var myscope = new SimpleScope()
            {
                Basic = 1000d,
                Bonus = 100d
            };
            Console.WriteLine(func(myscope));

            var newfunc = expr.ScopeCompile<DynamicScope>();
            var mynewscope = new DynamicScope();
            mynewscope["Basic"] = 2000d;
            mynewscope["Bonus"] = 200;
            Console.WriteLine(newfunc(mynewscope));
        }
I then referred a recent thread with similar problem, but it's related to normal compile. I could not really figure out what change do I have to make to have something similar effect.

Request your help.

Thanks,
Kinjal
Developer
Aug 27, 2014 at 11:50 PM
Edited Aug 27, 2014 at 11:57 PM
Dynamics for the most part, works fine.

Change this
    var expr = new CompiledExpression<double>(expression);
to this
    var expr = new CompiledExpression(expression);
(equivalent to CompiledExpression<object>) and cast the result of myfunc() to double or whatever you expect/need the output to be as the result of a dynamic operation is essentially a dynamic/object.
Sep 1, 2014 at 7:33 AM
Edited Sep 2, 2014 at 5:05 AM
Hi,

Thanks for that tip. It works perfectly now.

Another thing to note is that the ExpressionExtension and it's eval method mentioned in your pastebin code does not re-evaluate the expression with ExpandoObject scope. The reason seems to be because Expression is DynamicExpression. Traversing through a Dynamic expression's Arguments will be required for that and what are possible outcomes of the same is an unknown area for me.

Looking at the code, I think there are two ways to handle this. Your input on these will help me in selecting my future developments.
  1. Continue modifying the code of this structure so that it can confidently handle dynamic object.
  2. Based on your idea of caching compiledexpressions and using a custom DynamicObject, generate value of dependents through TryGetMember. Please take look at a Demo of that idea.
With this approach, I'm solving following problems:
  1. use dynamic and compile-time classes as scope.
  2. the scope variables are evaluated dynamically even if seed variables are manipulated outside the ExpressionExtension.
  3. cache scope-compiled expressions across the application in a logical container
  4. Assignment of the result is not embedded within the expression
However, indirect cyclic dependencies and validation are not handled with this approach.

Request your thoughts on this and subsequent performance.

Thanks,
Kinjal