This project has moved and is read-only. For the latest updates, please go here.

Confused by nullable values in TypeRegistry

May 8, 2014 at 5:22 PM
Hi,

This is a fantastic library! I encountered something today that I'm confused about. Below is a scratch program that demonstrates it:
    using ExpressionEvaluator;

    class Program
    {
        static void Main(string[] args)
        {
            var expression = new CompiledExpression()
            {
                TypeRegistry = new TypeRegistry()
            };

            int? argument1 = 5;
            Fact argument2 = new Fact()
            {
                Count = 5
            };

            expression.TypeRegistry.RegisterSymbol("Argument1", argument1);
            expression.TypeRegistry.RegisterSymbol("Argument2", argument2);

            // Works
            expression.StringToParse = "Argument2.Count != null";
            expression.Eval();

            // Fails with NullReferenceException
            expression.StringToParse = "Argument1 != null";
            expression.Eval();
        }

        public class Fact
        {
            public int? Count
            {
                get;
                set;
            }
        }
Essentially, if I test against a nullable int when it is a property of something that's shoved into the TypeRegistry, it works fine, but if it's shoved directly into the TypeRegistry, it fails. My guess is it has something to do with type conversion. Is this expected behavior? I reckon the easiest workaround is to wrap everything in a parent object and test from there.
May 9, 2014 at 1:14 AM
Hello,

I just tried your code against my latest commit and it seems the issue has already been fixed. You're probably right, it must have something to do with type conversion. It's definitely not expected behavior, and bugs like this tend to creep through as the C# implementation is not 100% accurate yet.

Please get the latest commit from the AntlrParser branch and let me know if this fixes your problem.

Thanks for your interest in the project! If you have any questions, observations or bug reports, or API feature requests please feel free to let me know.


May 9, 2014 at 1:39 PM
Thanks for the response! I downloaded the AntlrParser branch ("935303483a7c") and ran the scratch program against it, and oddly now the problem scenario is flipped: "Argument1 != null" is fixed, but "Argument2.Count != null" now fails with the NullReferenceException.

In the AntlrParser branch, if I change the test to "(object)Argument2.Count != null", then it works again. Doing this seems to make the generated expression tree have an extra call to Convert(). It's "Convert(NotEqual(value(ConsoleApplication3.Fact).Count, null))" without the cast to object (which, again, fails with a NRE), and "Convert(NotEqual(Convert(value(ConsoleApplication3.Fact).Count), null))" with the cast (which works).

Of course, I have no real idea what any of these means. Thanks for any insight!
May 9, 2014 at 2:28 PM

Hmmm. That's strange. I'll check again.

May 13, 2014 at 12:34 PM
I have updated the codes to support nullable types better.

There is a caveat however: for some reason the type of the registered symbol that is a nullable type is set to the underlying type if it has a constant value, or object if it is null. This is probably an optimization by the C# compiler.

For properties the issue isn't apparent since it goes through the property type instead of the compiled value.

'''
    [TestMethod]
    public void NullableType()
    {
        var expression = new CompiledExpression()
        {
            TypeRegistry = new TypeRegistry()
        };

        int? argument1 = 5;
        var argument2 = new Fact()
        {
            Count = 5
        };

        /// argument1 is of type Int32 instead of Nullable<Int32>, so we need to force the type here...
        expression.TypeRegistry.RegisterSymbol("Argument1", argument1, typeof(int?));
        expression.TypeRegistry.RegisterSymbol("Argument2", argument2);

        // Works
        expression.StringToParse = "Argument2.Count != null";
        expression.Eval();

        // Fails with NullReferenceException
        expression.StringToParse = "Argument1 != null";
        expression.Eval();
    }
'''
May 13, 2014 at 7:51 PM
It works! Love it!

I've been trying it out in a port of myBATIS that makes it simple to generate safe, parameterized SQL queries based on an input object:
<?xml version="1.0" encoding="utf-8" ?>
<Statement>
  SELECT
    *
  FROM Core.Blogs AS b
  INNER JOIN Core.Authors AS a
  ON a.AuthorId = b.AuthorId
  <Where>
    <If test="State != null">
      b.State = #{State}
    </If>
    <If test="Title != null">
      AND b.Title LIKE #{Title}
    </If>
    <If test="Author != null">
      AND a.Name LIKE '%' + #{Author} + '%'
    </If>
  </Where>
</Statement>
where the XML bits control what SQL gets generated, and the #{} parts are evaluated and replaced with database parameters. It's really convenient for a page where you let a person filter optionally by a bunch of different values.

The main advantage is that it takes care of fixing up the WHERE and extraneous ANDs and ORs or closing parens as necessary, based on the input variables. It just so happens that a lot of my input parameters tend to be strings or nullable integers, which is where I hit this issue. Previously, I had been using the Mono compiler to evaluate the test expressions, but this library is WAY more performant. Thanks again!
May 13, 2014 at 9:12 PM
I lied! =)

I did find one issue with the latest commit when the value actually is null, but I think I've found a fix.

First, let's update the unit test to include some tests where the value is actually null. I added a Length property to the Fact (which as a nullable int that's null), and I added another "Argument3" that's null, using your handy new overload:
        [TestMethod]
        public void NullableType()
        {
            var expression = new CompiledExpression()
            {
                TypeRegistry = new TypeRegistry()
            };

            int? argument1 = 5;
            var argument2 = new Fact()
            {
                Count = 5,
                Length = null
            };

            expression.TypeRegistry.RegisterSymbol("Argument1", argument1, typeof(int?));
            expression.TypeRegistry.RegisterSymbol("Argument2", argument2);
            expression.TypeRegistry.RegisterSymbol("Argument3", null, typeof(int?));

            expression.StringToParse = "Argument1 != null";
            Assert.IsTrue((bool)expression.Eval());
          
            expression.StringToParse = "Argument2.Count != null";
            Assert.IsTrue((bool)expression.Eval());

            expression.StringToParse = "Argument2.Length != null";
            Assert.IsFalse((bool)expression.Eval());
            
            expression.StringToParse = "Argument3 != null";
            Assert.IsFalse((bool)expression.Eval());
        }
So what happens if we run this in the latest commitment, is the ones where the symbols aren't null work right, and the ones were the symbols ARE null (the last two cases above) get converted to "0 != null" and then evaluate to true, when they should have been false.

I'm no expert in this, but I traced it to the new NullLiteralConversion() method. The overload to select the constructor used for Activator.CreateInstance ends up passing the underlying type (which doesn't have a null value, and ends up using the default, in this case 0 for int). I fixed it by using the overload that says "use the parameterless constructor for Nullable<whatever>":
        //        6.1.5 Null literal conversions
        //An implicit conversion exists from the null literal to any nullable type. This conversion produces the null value (§4.1.10) of the given nullable type.
        public static Expression NullLiteralConverion(Expression dest, Expression src)
        {
            if (src.NodeType == ExpressionType.Constant 
                && src.Type == typeof(object) 
                && ((ConstantExpression)src).Value == null
                && Nullable.GetUnderlyingType(dest.Type) != null)
            {
                var value = Activator.CreateInstance(dest.Type);
                return Expression.Constant(value, dest.Type);
            }

            return src;
        }
And all tests seem to now pass. Hope this helps!
May 13, 2014 at 11:11 PM
Oh cool. I was wondering where that 0 was coming from. Thanks! I'll get that in right away
May 14, 2014 at 2:45 AM
Your fix has been committed. Thanks again!