This project has moved. For the latest updates, please go here.

ScopeCompile and IDictionary<string, object>

May 29, 2014 at 9:41 AM
Hey there,
I am trying to execute the following code:
var data = new Dictionary< string, object > { { "intVal", 4 } };
var expr = new CompiledExpression< bool >( @"intVal > 4" );
var func = expr.ScopeCompile< IDictionary< string, object > >();
var bb = func( data );
but I get an error Cannot resolve symbol "intVal"
Is there a way to express that intVal is not a property somehow or any suggestion what could be wrong?
I am using the latest commit in the Antlr branch.
Developer
May 29, 2014 at 11:35 AM
Hi,

when you use ScopeCompile, any identifiers that are not registered will try to be bound as properties against the scope object that you pass into the function.

So your expression "intVal > 4" becomes "scope.intVal > 4" where scope is a virtual identifier that is inserted by the compiler.

The object you are passing in is a dictionary, so unfortunately it won't work like that since the Dictionary class doesn't have an intVal property.

If you want to have dynamic variables, why don't you try using dynamics and the ExpandoObject class?

Do you need to have the variables stored in a Dictionary?




May 29, 2014 at 11:48 AM
Edited May 29, 2014 at 11:48 AM
Hi,

I am testing the code and its functionality. Even cleaned some parts of the code, which will commit and send a pull request if you wish.
The ExpandoObject is working fine, and I made some fixes for the CompiledExpression< TReturn >().ScopeCompile< TParam >() case.

The problem is that i think there is a memory leak with dynamic and ExpandoObject (or I am doing something wrong), that why I wanted to test out the IDictionary scenario with ScopeCompile.
After looking at the grammar i found that it doesn't understands that 'intVal' is not property but a key in collection.
Anyway a solution for me is to use TypeRegistry and regular compile, but maybe accessing the collection values as a regular members (if the scope implements ICollection or something) may be good addition to the library.

Regards,
Daniel Tsviatkov.
May 29, 2014 at 3:39 PM
More info, but not regarding ScopeCompile.

I tried this case:
var data = new Dictionary< string, object > { { "intVal", 4 } };
var tR = new TypeRegistry();
tR.RegisterSymbol( "data", data );
var expr = new CompiledExpression( @"(int)data[ ""intVal"" ] > 4" ){ TypeRegistry = tR };
var func = expr.Compile();
var bb = func();
It didn't worked out because the data object is not an array. It is collection.
To fix and make it work the following change have to be done:
In ExpressionHelper.cs on line 71
else
{
   return Expression.ArrayAccess(le, args);
}
should be changed to:
return type.IsArray ? Expression.ArrayAccess(le, args) : Expression.Property( le, "Item", args.ToArray() );
This way I was able to grab the value of the generics i tested (Dictionary< string, object >, List< object >).

Greetings,
Daniel Tsviatkov.
Developer
May 29, 2014 at 10:59 PM
Thanks! I noticed the same problem when trying a workaround for your question, but I hadn't gotten around to fixing it properly.

I'm hesitant to make changes like accessing a collection like an array since I will be implementing language features that aren't necessarily part of C#. Though I agree it makes expression evaluator more useful. BTW, How did you come across the idea that dynamic/ExpandoObject has a memory leak? It would be interesting to know.




May 30, 2014 at 6:44 AM
Hi,

The first thing I tested was the dynamic/ExpandoObject compination, since it is what I would need for a project. The project will be long running process with services etc. and memory consumption is important over time.

Created a case with the 4 possible combinations of CompiledExpression/ScopeCompile (CE< bool >().SC(), CE().SC< Expando >() ... ) and itterated 5000 times each to see which one is the most performant and with lowest memory consumption.

The performance was ok ( the best performing scenario is CE< bool >().SC< Expando >() with little modifications to the code to avoid some unnecessary conversions ).
Whatever I did the memory kept going up and up. Same case but with Dictionary performs like 10 times faster, and have a stable memory consumption.

The problem with ExpandoObject as much as I could tracked down was something with the binder.
As I said i prepared a commit with all the changes I did and will try to send it till the end of the week for your review if you wish to add it to the current state of the library.

Greetings,
Daniel Tsviatkov.
Developer
May 30, 2014 at 7:01 AM
Can you send me a sample of your test code?


May 30, 2014 at 7:35 AM
Hi,
Here is the code I used:
            dynamic data = new ExpandoObject();
            data.intVal = 5;

            const int repeat = 5000;

            Console.WriteLine( "Press Enter to start ... (Memory = {0:00000} KB;  Assemblies = {1})", Process.GetCurrentProcess().PrivateMemorySize64 / ( 1024 * 1 ), AppDomain.CurrentDomain.GetAssemblies().Length );
            Console.ReadLine();

            Stopwatch sw;
            //
            sw = new Stopwatch();
            sw.Start();

            for( int i = 0; i < repeat; i++ ) {
                var expr = new CompiledExpression< bool >( @"intVal > 4" );
                var func = expr.ScopeCompile< ExpandoObject >();
                var bb = func( data );
            }
            Console.WriteLine( "ExpandoEbject, Bool   => {0}, Memory = {1:00000} KB;  Assemblies = {2}", sw.ElapsedMilliseconds, Process.GetCurrentProcess().PrivateMemorySize64 / ( 1024 * 1 ), AppDomain.CurrentDomain.GetAssemblies().Length );
            Console.WriteLine( "Press Enter to continue ..." );
            Console.ReadLine();
            //
            sw = new Stopwatch();
            sw.Start();
            for( int i = 0; i < repeat; i++ ) {
                var expr = new CompiledExpression( @"intVal > 4" );
                var func = expr.ScopeCompile();
                var bb = func( data );
            }
            Console.WriteLine( "Object, Object        => {0}, Memory = {1:00000} KB;  Assemblies = {2}", sw.ElapsedMilliseconds, Process.GetCurrentProcess().PrivateMemorySize64 / ( 1024 * 1 ), AppDomain.CurrentDomain.GetAssemblies().Length );
            Console.WriteLine( "Press Enter to continue ..." );
            Console.ReadLine();
            //
            sw = new Stopwatch();
            sw.Start();

            for( int i = 0; i < repeat; i++ ) {
                var expr = new CompiledExpression( @"intVal > 4" );
                var func = expr.ScopeCompile< ExpandoObject >();
                var bb = func( data );
            }
            Console.WriteLine( "ExpandoEbject, Object => {0}, Memory = {1:00000} KB;  Assemblies = {2}", sw.ElapsedMilliseconds, Process.GetCurrentProcess().PrivateMemorySize64 / ( 1024 * 1 ), AppDomain.CurrentDomain.GetAssemblies().Length );
            Console.WriteLine( "Press Enter to continue ..." );
            Console.ReadLine();
            //
            sw = new Stopwatch();
            sw.Start();

            for( int i = 0; i < repeat; i++ ) {
                var expr = new CompiledExpression< bool >( @"intVal > 4" );
                var func = expr.ScopeCompile();
                var bb = func( data );
            }
            Console.WriteLine( "Object, Bool          => {0}, Memory = {1:00000} KB;  Assemblies = {2}", sw.ElapsedMilliseconds, Process.GetCurrentProcess().PrivateMemorySize64 / ( 1024 * 1 ), AppDomain.CurrentDomain.GetAssemblies().Length );

            Console.WriteLine( "Done" );
            Console.ReadLine();
Greetings,
Daniel Tsviatkov.
Developer
May 30, 2014 at 10:06 AM
Thanks for this.

By the way, have you considered a caching mechanism for your expressions? If you expect an expression to be called repeatedly, it is best to cache the generated function. I doubt that the 5000+ expressions will all be unique.




May 30, 2014 at 10:08 AM
I considered expression caching and I doubt that I will have more than 100-200 expressions max, but still tests are tests :)

I need to know that I can and what I can't do :)

Greetings,
Daniel Tsviatkov.
May 30, 2014 at 12:11 PM
Heya,

I made a pull request. Take a look and if the changes fit your plans apply it.

Greetings,
Daniel Tsviatkov.
May 30, 2014 at 2:56 PM
Hi again,

I am getting very spammy and sorry for that but here is another "fix" for IDictionary and ScopeCompile
In ExpressionHelper.cs on line 228
if (le.Type.Name == "RuntimeType")
should be changed to :
if( le.ToString() == "scope" && le.Type.GetInterfaces().Contains( typeof( IEnumerable ) ) ) {
                return GetPropertyIndex( le, new[] { Expression.Constant( membername ) } );
            } else if (le.Type.Name == "RuntimeType") {
This looks abit like a hack, but was the first thing that came in my mind, and worked pretty well.
Don't really know if its place is over there nor if it is right to do it like that, but works for my case for now.
A generic fix (if there is one) would be better. If I find one will let you know.
Keep up the good work.

Greetings,
Daniel Tsviatkov.