Null reference Exception in Aggregate Serializer

Jan 4, 2011 at 3:28 AM

 I got this exception when i was trying to flush some dummy data. The serialize target was null and as a result the gettype called on that failed.

Thanks

stem.NullReferenceException was unhandled
  Message=Timeouts are not supported on this stream.
  StackTrace:
       at System.IO.Stream.get_WriteTimeout()
       at Wintellect.Sterling.Serialization.AggregateSerializer.Serialize(Object target, BinaryWriter writer)
       at Wintellect.Sterling.Indexes.IndexCollection`3._SerializeIndexes()
       at Wintellect.Sterling.Indexes.IndexCollection`3.Flush()
       at Wintellect.Sterling.Database.BaseDatabaseInstance.Flush()
       at ItemLocator.Mobile.ClientRepository.DummyDataHelper.CreateDummyData(ISterlingDatabaseInstance database)
       at ItemLocator.Mobile.ClientRepository.SterlingItemLocatorRepository..ctor()
       at ItemLocator.Mobile.UI.Utilities.RepositoryFactory.Create(String databaseName, Boolean isPersistentStore, Boolean useSterlingDb)
       at ItemLocator.Mobile.UI.App.CreateRepository()
       at ItemLocator.Mobile.UI.App.Application_Launching(Object sender, LaunchingEventArgs e)
       at Microsoft.Phone.Shell.PhoneApplicationService.FireLaunching()
       at Microsoft.Phone.Execution.NativeEmInterop.FireOnLaunching()

Jan 4, 2011 at 4:00 AM

Also, if i remove all the indexes from the tables, it works fine but I don't use the indexes. I really would like to use the indexes if possible. Any ideas. I am using the changeset 71547

This happens in the Flush method call

 return new List<ITableDefinition>() {
           
               CreateTableDefinition<Itemint>(i => i.ItemId)
                                    //.WithIndex<Item, int, int>(ITEM_BY_ITEMGROUPID, x => x.ItemGroupId.Value)
                                    //.WithIndex<Item, string, int>(ITEM_BY_NAME, x => x.Name)
                                    //.WithIndex<Item, string, int>(ITEM_BY_KEYWORD, x => x.Keyword)
                                    ,

               CreateTableDefinition<ItemGroupint>(i => i.ItemGroupId).WithIndex<ItemGroupstringint>(ITEMGROUP_BY_NAME, x => x.Name),

               CreateTableDefinition<Locationint>(i => i.LocationId)
                                                         //.WithIndex<Location, int, int>(LOCATION_BY_LOCATIONGROUPID, x => x.LocationGroupId.Value) 
                                                         //.WithIndex<Location, string, int>(LOCATION_BY_NAME, x => x.Name)
                                                         //.WithIndex<Location, string, int>(LOCATION_BY_KEYWORD, x => x.Keyword)
                                                         ,

               CreateTableDefinition<LocationGroupint>(i => i.LocationGroupId)
                                                        //.WithIndex<LocationGroup, string, int>(LOCATIONGROUP_BY_NAME, x => x.Name) 
                                                        ,

               CreateTableDefinition<ItemLocationDetailint>(i => i.ItemLocationDetailId)
                                                             //.WithIndex<ItemLocationDetail, int, int>(ITEMLOCATIONDETAIL_BY_ITEMID, x => x.ItemId)
                                                             //.WithIndex<ItemLocationDetail, int, int>(ITEMLOCATIONDETAIL_BY_LOCATIONID, x => x.LocationId)
                                                             ,

               CreateTableDefinition<Customerint>(i => i.CustomerId)
           }; 
Coordinator
Jan 4, 2011 at 11:12 AM

Can you show the code that threw the exception? Obviously the indexes/etc work in general as the tests pass, but there is likely a test condition we didn't consider - if I could see how the data was staged first it would help me better determine what fix is needed. Thanks!

Jan 5, 2011 at 12:22 AM
Here is my code to set up some dummy data. The database.flush is the user code 
database.Save<Customer>(new Customer() { CustomerId = 1, LastName = "blah", FirstName = "blah" });
            foreach (Item i in GetItems()) {

                database.Save<Item>(i);
            }

            foreach (ItemGroup i in GetItemGroups())
            { 
                database.Save<ItemGroup>(i);
            }

            foreach (LocationGroup i in GetLocationGroups())
            {
                database.Save<LocationGroup>(i);
            }

            foreach (Location i in GetLocations())
            {
                database.Save<Location>(i);
            }

            foreach (ItemLocationDetail i in GetItemLocationDetails())
            {
                database.Save<ItemLocationDetail>(i);
            }

            database.Flush();
The Sterling code is in the Aggregate Serializer and the target was null
     public override void Serialize(object target, BinaryWriter writer)
        {
            var type = target.GetType();
Jan 5, 2011 at 12:31 PM

Could it have something to do with some properties being nullable?  Let me know if you need more information.  This is holding me back from moving forward

Coordinator
Jan 5, 2011 at 12:38 PM

Still not enough information for me to troubleshoot. Are you using the latest source code download (not release 0.9, which has known bugs, but the latest actual source)?

Jan 5, 2011 at 1:52 PM

I am using the latest source code download and not release 0.9.  I can write a unit test with my entities & data to the sterling phone test project later this evening

Coordinator
Jan 5, 2011 at 2:01 PM

I appreciate that. If you want to send me the code for the unit test that reproduces the problem, I'll integrate it into the main trunk and resolve it.

Jan 6, 2011 at 12:42 AM

Ok. I was able to repro the problem. Here is the code for my test. Let me know if the test is incorrect but i did get the same exception.

using System.Windows.Media;
using System.Windows.Media.Animation;
using System.Windows.Shapes;
using Microsoft.VisualStudio.TestTools.UnitTesting;
using Wintellect.Sterling.IsolatedStorage;
using System.Collections.Generic;
using Wintellect.Sterling.Database;
using Microsoft.Silverlight.Testing;

namespace Wintellect.Sterling.WindowsPhone.Test.Database
{
    public partial class Item
    {


        public virtual int ItemId
        {
            get;
            set;
        }

        public virtual int CustomerId
        {
            get;
            set;
        }

        public virtual System.DateTime CreationDate
        {
            get;
            set;
        }


        public virtual System.DateTime LastModifiedDate
        {
            get;
            set;
        }

        public virtual string Name
        {
            get;
            set;
        }


        public virtual string Description
        {
            get;
            set;
        }

        public virtual string Keyword
        {
            get;
            set;
        }


        public virtual Nullable<int> ItemGroupId
        {
            get;
            set;
        }

    }

    public partial class ItemGroup
    {

        public virtual int ItemGroupId
        {
            get;
            set;
        }

        public virtual string Name
        {
            get;
            set;
        }

        public virtual string Description
        {
            get;
            set;
        }

        public virtual System.DateTime CreationDate
        {
            get;
            set;
        }

        public virtual System.DateTime LastModifiedDate
        {
            get;
            set;
        }

        public virtual int CustomerId
        {
            get;
            set;
        }

        public int TotalItems
        {

            get;
            internal set;
        }

        public string NameAndTotalItems
        {

            get
            {

                return string.Format("{0} ({1})", Name, TotalItems);
            }
        }
    }

  
    public class TestFlushOfTableWithIndexesInstance : BaseDatabaseInstance
    {
        public const string ITEM_BY_ITEMGROUPID = "ItemItemGroupId";
        public const string ITEM_BY_NAME = "ItemName";
        public const string ITEM_BY_KEYWORD = "ItemKeyword";

        public const string ITEMGROUP_BY_NAME = "ItemGroupName";

        public override string Name
        {
            get { return "Test"; }
        }

        protected override System.Collections.Generic.List<ITableDefinition> _RegisterTables()
        {
            return new List<ITableDefinition>() {
          
               CreateTableDefinition<Item, int>(i => i.ItemId)
                                    .WithIndex<Item, int, int>(ITEM_BY_ITEMGROUPID, x => x.ItemGroupId.Value)
                                    .WithIndex<Item, string, int>(ITEM_BY_NAME, x => x.Name)
                                    .WithIndex<Item, string, int>(ITEM_BY_KEYWORD, x => x.Keyword)
                                    ,

               CreateTableDefinition<ItemGroup, int>(i => i.ItemGroupId).WithIndex<ItemGroup, string, int>(ITEMGROUP_BY_NAME, x => x.Name),

            


           };


        }
    }

    [Tag("Database")]
    [TestClass]
    public class TestFlushofIndexes
    {
        private SterlingEngine _engine;
        private ISterlingDatabaseInstance _databaseInstance;

        [TestInitialize]
        public void TestInit()
        {
            _engine = new SterlingEngine();
            _engine.Activate();
            _databaseInstance = _engine.SterlingDatabase.RegisterDatabase<TestFlushOfTableWithIndexesInstance>();

        }

             [TestMethod]
        public void Test_Flush_Entity_With_Multiple_Indexes_Does_Not_Throw_Null_Reference_Exception()
        {

            GetItems().ForEach(x => _databaseInstance.Save(x));
            GetItemGroups().ForEach(x => _databaseInstance.Save(x));
            _databaseInstance.Flush();

        }

        [TestCleanup]
        public void TestCleanup()
        {
            _engine.Dispose();
            _databaseInstance = null;
            using (var iso = new IsoStorageHelper())
            {
                iso.Purge(PathProvider.BASE);
            }
        }


        /// <summary>
        /// Helper method to create items
        /// </summary>
        private static List<Item> GetItems()
        {

            List<Item> items = new List<Item>();

            items.Add(new Item() { ItemId = 1, ItemGroupId = 2, Name = "Item 1", Description = "" });
            items.Add(new Item() { ItemId = 2, ItemGroupId = 9, Name = "Item 2", Description = "" });
            items.Add(new Item() { ItemId = 3, ItemGroupId = 7, Name = "Item 3", Description = "" });
            items.Add(new Item() { ItemId = 4, ItemGroupId = 8, Name = "Item 4", Description = "" });
            items.Add(new Item() { ItemId = 5, ItemGroupId = 10, Name = "Item 5", Description = "" });
            items.Add(new Item() { ItemId = 6, ItemGroupId = 4, Name = "Item 6", Description = "" });
            items.Add(new Item() { ItemId = 7, ItemGroupId = 3, Name = "Item 7", Description = "" });
            items.Add(new Item() { ItemId = 8, ItemGroupId = 3, Name = "Item 8", Description = "" });
            items.Add(new Item() { ItemId = 9, ItemGroupId = 7, Name = "Item 9", Description = "" });
            items.Add(new Item() { ItemId = 10, ItemGroupId = 3, Name = "Item 10", Description = "" });
            items.Add(new Item() { ItemId = 11, ItemGroupId = 3, Name = "Item 11", Description = "" });
            items.Add(new Item() { ItemId = 12, ItemGroupId = 3, Name = "Item 12", Description = "" });
            items.Add(new Item() { ItemId = 13, ItemGroupId = 3, Name = "Item 13", Description = "" });
            items.Add(new Item() { ItemId = 14, ItemGroupId = 7, Name = "Item 14", Description = "" });
            items.Add(new Item() { ItemId = 15, ItemGroupId = 7, Name = "Item 15", Description = "" });

            return items;
        }

        /// <summary>
        /// Helper method to create item groups
        /// </summary>
        private static List<ItemGroup> GetItemGroups()
        {
            List<ItemGroup> itemGroups = new List<ItemGroup>();

            itemGroups.Add(new ItemGroup() { ItemGroupId = 1, Name = "Item Group 1", Description = "" });
            itemGroups.Add(new ItemGroup() { ItemGroupId = 2, Name = "Item Group 2", Description = "" });
            itemGroups.Add(new ItemGroup() { ItemGroupId = 3, Name = "Item Group 3", Description = "" });
            itemGroups.Add(new ItemGroup() { ItemGroupId = 4, Name = "Item Group 4", Description = "" });
            itemGroups.Add(new ItemGroup() { ItemGroupId = 5, Name = "Item Group 5", Description = "" });
            itemGroups.Add(new ItemGroup() { ItemGroupId = 6, Name = "Item Group 6", Description = "" });
            itemGroups.Add(new ItemGroup() { ItemGroupId = 7, Name = "Item Group 7", Description = "" });
            itemGroups.Add(new ItemGroup() { ItemGroupId = 8, Name = "Item Group 8", Description = "" });
            itemGroups.Add(new ItemGroup() { ItemGroupId = 9, Name = "Item Group 9", Description = "" });
            itemGroups.Add(new ItemGroup() { ItemGroupId = 10, Name = "Item Group 10", Description = "" });

            return itemGroups;
        }
    }
}

Coordinator
Jan 6, 2011 at 1:20 PM

Thanks for taking the time to do this.

Currently, Sterling doesn't support null index values. I'll be sure to update the documentation to reflect this. That includes strings. Ultimately, it makes sense perhaps - definitely never for keys, but for indexes perhaps. The problem is that the index serializer is higher in the stack to handle raw values and reflects a type and when it's a null value, it creates issues. I don't want to do a null check right now because that adds another byte for every index which makes them grow abnormally large and slows processing.

Fortunately, if you are fine with using default values for indexes when there is null, the problem is easily resolved - in your case, if you change the table definition to this, you should be fine - not how I cast nulls to a default value.

return new List<ITableDefinition>
                       {
                           CreateTableDefinition<Item, int>(i => i.ItemId)
                               .WithIndex<Item, int, int>(ITEM_BY_ITEMGROUPID, x => x.ItemGroupId ?? -1)
                               .WithIndex<Item, string, int>(ITEM_BY_NAME, x => x.Name ?? string.Empty)
                               .WithIndex<Item, string, int>(ITEM_BY_KEYWORD, x => x.Keyword ?? string.Empty)
                           ,
                           CreateTableDefinition<ItemGroup, int>(i => i.ItemGroupId).WithIndex<ItemGroup, string, int>(
                               ITEMGROUP_BY_NAME, x => x.Name ?? string.Empty),
                       };
Jan 7, 2011 at 2:00 AM

Thanks for confirming that. I will make a few changes and plow ahead to take this into account.