ArgumentOutOfRangeException in IsolatedStorageDriver.GetTypeAtIndex

Apr 20, 2011 at 4:09 AM

In the following test, the very last assert line causes an ArgumentOutOfRangeException in IsolatedStorageDriver.GetTypeAtIndex, under LazyValue.get_Value. TypeIndex has 8 items and the 9th is asked.

    [TestMethod]
    public void TestService()
    {
      var engine = new SterlingEngine();
      engine.Activate();

      try
      {
        var database = engine.SterlingDatabase.RegisterDatabase<DutchTabDatabase>( new IsolatedStorageDriver() );
        database.Purge();

        Bill bill = new Bill() { Name = "Test", Total = 45 };
        Person person = new Person() { Name = "Martin Plante" };

        database.Save( bill );
        database.Flush();

        database.Save( person );
        database.Flush();

        BilledPerson billedPerson = new BilledPerson() { Person = person, Paid = 20 };
        bill.BilledPeople.Add( billedPerson );

        database.Save( bill );
        database.Flush();
      }
      finally
      {
        engine.Dispose();
      }

      engine = new SterlingEngine();
      engine.Activate();

      try
      {
        var database = engine.SterlingDatabase.RegisterDatabase<DutchTabDatabase>( new IsolatedStorageDriver() );

        var bills = database.Query<Bill, Guid>();
        Assert.IsTrue( bills.Count == 1, "The database does not contain a single Bill" );

        var people = database.Query<Person, Guid>();
        Assert.IsTrue( people.Count == 1, "The database does not contain a single Person" );

        Assert.IsTrue( bills[ 0 ].LazyValue.Value.BilledPeople.Count == 1, "The Bill does not contain a single BilledPerson" );
      }
      finally
      {
        engine.Dispose();
      }
    }

The database tables are the following:

          this.CreateTableDefinition<Bill, Guid>( b => b.Id ),
          this.CreateTableDefinition<Person, Guid>( p => p.Id ),

If you remove the code related to the "BilledPerson", and change the last assert to still load that LazyValue, it works:

    [TestMethod]
    public void TestService2()
    {
      var engine = new SterlingEngine();
      engine.Activate();

      try
      {
        var database = engine.SterlingDatabase.RegisterDatabase<DutchTabDatabase>( new IsolatedStorageDriver() );
        database.Purge();

        Bill bill = new Bill() { Name = "Test", Total = 45 };
        Person person = new Person() { Name = "Martin Plante" };

        database.Save( bill );
        database.Flush();

        database.Save( person );
        database.Flush();
      }
      finally
      {
        engine.Dispose();
      }

      engine = new SterlingEngine();
      engine.Activate();

      try
      {
        var database = engine.SterlingDatabase.RegisterDatabase<DutchTabDatabase>( new IsolatedStorageDriver() );

        var bills = database.Query<Bill, Guid>();
        Assert.IsTrue( bills.Count == 1, "The database does not contain a single Bill" );

        var people = database.Query<Person, Guid>();
        Assert.IsTrue( people.Count == 1, "The database does not contain a single Person" );

        Assert.IsTrue( bills[ 0 ].LazyValue.Value.BilledPeople.Count == 0, "The Bill's BilledPeople is not empty" );
      }
      finally
      {
        engine.Dispose();
      }
    }

If you add a BilledPerson table to the database, both tests work.

          this.CreateTableDefinition<Bill, Guid>( b => b.Id ),
          this.CreateTableDefinition<Person, Guid>( p => p.Id ),
          this.CreateTableDefinition<BilledPerson, Guid>( bp => bp.Id ),

If you remove that third table and simply build instead of rebuild, even though there is a call to Purge on the database, both tests work! Doing a rebuild obviously wipes clean the isolated storage, and the error returns.

The BilledPeople property is of type IList<BilledPerson>, and is using an ObservableCollection<BilledPerson> underneath.

Thanks.

Coordinator
Apr 20, 2011 at 3:57 PM

Are you assigning a Guid to the person in the constructor? Common mistake is not setting up the id and then it wipes out the ability to load it back in? Let me know ... otherwise I'll create a test case and check it out. Thanks.

Apr 20, 2011 at 4:21 PM

All my model classes derive from a base class that explicitly assign a new Guid to every new instance. I don't mind sending you (in private) a copy of the projects if this can help you.

Coordinator
Apr 20, 2011 at 4:25 PM

My track record for troubleshooting just from discussion online is so-so. I'm 100% on finding problems when I have the solution in hand. Caveat is that I may not be able to get to it right away, but will definitely queue it up to look at.

Thanks,

Jeremy

Coordinator
Apr 20, 2011 at 4:26 PM

PS - no need to send the full project just yet. I think I have enough info above to recreate the test so let me go that route first, and if I can't duplicate it I'll reach out for more.

Coordinator
Apr 21, 2011 at 7:42 PM

OK, I created a test that I believe replicates what you have based on what I know. I cannot get the test to fail. Can you let me know what I might be missing here? This is the test, it is completely self-contained to run in the isolated storage tests for Sterling:

using System;
using System.Collections.Generic;
using Microsoft.Silverlight.Testing;
using Microsoft.VisualStudio.TestTools.UnitTesting;
using Wintellect.Sterling.Database;

namespace Wintellect.Sterling.IsolatedStorage.Test.Database
{
    [Tag("PersonBill")]
    [TestClass]
    public class TestPersonBill
    {
        public class Bill
        {
            public Bill()
            {
                BilledPeople = new List<BilledPerson>();
            }

            public Guid Id { get; set; }
            public string Name { get; set; }
            public int Total { get; set; }
            public IList<BilledPerson> BilledPeople { get; set; }
        }

        public class Person
        {
            public Guid Id { get; set; }
            public string Name { get; set; }
        }

        public class BilledPerson
        {
            public Person ThePerson { get; set; }
            public int Paid { get; set; }

        }

        public class BillPersonDatabase : BaseDatabaseInstance
        {
            public override string Name
            {
                get { return typeof(BillPersonDatabase).Name; }
            }

            protected override List<ITableDefinition> RegisterTables()
            {
                return new List<ITableDefinition>
                           {
                               CreateTableDefinition<Bill, Guid>(b => b.Id),
                               CreateTableDefinition<Person, Guid>(p => p.Id)
                           };
            }
        }
        
        [TestMethod]
        public void TestService()
        {
            var engine = new SterlingEngine();
            engine.Activate();

            try
            {
                var database = engine.SterlingDatabase.RegisterDatabase<BillPersonDatabase>(new IsolatedStorageDriver());
                database.Purge();

                var bill = new Bill { Name = "Test", Total = 45, Id = Guid.NewGuid() };
                var person = new Person { Name = "Martin Plante", Id = Guid.NewGuid() };

                database.Save(bill);
                database.Save(person);

                var bills = database.Query<Bill, Guid>();
                Assert.IsTrue(bills.Count == 1, "The database does not contain a single Bill");

                var people = database.Query<Person, Guid>();
                Assert.IsTrue(people.Count == 1, "The database does not contain a single Person");

                var billedPerson = new BilledPerson { ThePerson = person, Paid = 20 };
                bill.BilledPeople.Add(billedPerson);

                database.Save(bill);

                bills = database.Query<Bill, Guid>();
                Assert.IsTrue(bills.Count == 1, "The database does not contain a single Bill");

                people = database.Query<Person, Guid>();
                Assert.IsTrue(people.Count == 1, "The database does not contain a single Person");

                database.Flush();
            }
            finally
            {
                engine.Dispose();
            }

            engine = new SterlingEngine();
            engine.Activate();

            try
            {
                var database = engine.SterlingDatabase.RegisterDatabase<BillPersonDatabase>(new IsolatedStorageDriver());

                var bills = database.Query<Bill, Guid>();
                Assert.IsTrue(bills.Count == 1, "The database does not contain a single Bill");

                var people = database.Query<Person, Guid>();
                Assert.IsTrue(people.Count == 1, "The database does not contain a single Person");

                Assert.IsTrue(bills[0].LazyValue.Value.BilledPeople.Count == 1, "The Bill does not contain a single BilledPerson");
            }
            finally
            {
                engine.Dispose();
            }
        }
    }
}
Coordinator
Apr 21, 2011 at 7:43 PM

PS - I just realized one tweak I had to do was change "Person" to "ThePerson", i.e. so the property name was not the same as the class name. Can you see if that makes a difference in your own tests? It may be that handling a property with the same name as the class causes issues.

Apr 22, 2011 at 5:09 AM

It would have been great if it was the property name... but I still get the same exception, even after naming that property "ThePerson". I'll compare the tests more thoroughly.

Apr 22, 2011 at 6:33 AM

Ok, I know how to modify your test in order to reproduce the error. The problem is behind multiple calls to Flush.

a) Add a "database.Flush()" call after every "database.Save()"

b) You can remove the last "database.Flush()" at the end of the first "try".

 

Coordinator
Apr 22, 2011 at 1:39 PM

OK, that was a good one - thanks. The latest change set fixes it and I included a test for our isolated storage drivers and elevated trust drivers to avoid it in the future.

Basically Sterling using run-length encoding (RLE) to compress the types in the stream (imagine writing out a fully qualified type name every time a new class is encountered in the stream, vs. just an integer index that refers to a type). The driver is responsible for maintaining the "type master" and flushing it. In the isolated storage and elevated trust drivers, the types were being flushed the same time the keys were. This seemed like a good strategy. The problem is that only implemented types are ever indexed. In your scenario, the initial "save" of the bill and the person resulted in several types being generated, but because the list of billed persons was empty, the "billedperson" type was not indexed. When you updated it, the type was added. However, the "flush" never resulted in the type being written because the keys did not change. Because the key list was not dirty, it never called the key flush which means it never called the type flush. The last type was in memory, but not persisted to disk.

I modified it so whenever a type is added, it sets a dirty flag in the driver. On save, if the dirty flag is set at the end of the save, it resets it and flushes the type master. This ensures any update operation that results in new types will also flush the types to disk automatically.

Thanks for taking the time to come up with a solid test so I could fix this - I appreciate your efforts and patience!

Apr 22, 2011 at 2:10 PM

Thanks for the very quick fix!

Jun 1, 2011 at 8:50 PM

This is fixed in version 1.4 beta right ?

Coordinator
Jun 2, 2011 at 11:52 AM

Yes.

Jun 3, 2011 at 10:14 AM
Edited Jun 3, 2011 at 10:39 AM

Hi guys. Now after I have updated to 1.4 I don't receive this error and performance is drastically improved, but I still have issue when my application go to tombstoning entire database is deleted and I don't know how ? Do you have some idea ?

Coordinator
Jun 5, 2011 at 5:41 PM

See my other reply. You must be missing some step such as passing in the isolated storage driver. See the included example which is working by the teams' tests.

Jun 6, 2011 at 7:26 AM
Edited Jun 6, 2011 at 7:27 AM

I have downloaded and tested SterlingPhoneExample.

When I put messagebox in method LoadData  it seams that you constantly load test data not from Isolated storage.

Steps to reproduce :

1) Try to run this example not from debug mode but from Windows Phone.

2) Data is loaded becouse SetupData is called and Database.Save is called

3) Close application by clicking windows menu button

4) Reopen application

5) I am constantly receiving messageBox and hasKeys seems to be false. This is similar to problem that I have. When my application is tombstoned or closed I lose all data....

  public void LoadData()
        {
            bool hasKeys = false;
            foreach (var item in App.Database.Query<ItemViewModel, int>())
            {
                hasKeys = true;
                break;
            }

            if (!hasKeys)
            {
                MessageBox.Show("setup");
                _SetupData();
            }

Coordinator
Jun 6, 2011 at 11:49 AM

Ah, interesting. I tested the pre 1.5 on the phone, and the 1.5 in the emulator. Don't have a test phone with me - are you getting the same behavior in the emulator? can anyone else confirm whether or not the example works on the phone for them? Thanks!

Jun 6, 2011 at 11:55 AM

Yes I am getting same behavior on emulator...

Coordinator
Jun 6, 2011 at 12:02 PM

I've tested it several ways in the emulator. Whenever I tombstone (Windows menu button) and come back, I'm right where I left off. When I add something new and tombstone, when I come back in it is still there. When I add new recipes, completely exit, and come back, it is there. I am not able to replicate the issue you are listing.

Jun 6, 2011 at 12:41 PM
Edited Jun 6, 2011 at 12:51 PM

I have tested using SterlingPhoneExample not Recipes example. Can you please try reproduce using SterlingPhoneExample and just add messagebox in same place where I have added? I have large commercial project using Sterling and have save problem...

I have looked into Recipes sample, here it seams everything works fine. I will give deeper look into this sample trying to find difference...

Jun 6, 2011 at 1:02 PM

I think I am missing this line from recipe project :

RecipeDatabase.CheckAndCreate(Database);

Is this method _ParseFromResource

populate database from IsolatedStorage ?

Coordinator
Jun 6, 2011 at 1:05 PM

I know what your issue is. The example you're using just uses memory, not isolated storage. To use isolated storage, you need to modify the project to reference Wintellect.Sterling.WindowsPhone.IsolatedStorage.dll and then change the database line in App.xaml.cs to this:

_database = _engine.SterlingDatabase.RegisterDatabase<ItemDatabase>(new IsolatedStorageDriver

());

 

Then you should see the behavior you are looking for. By default, Sterling uses an in-memory database - you must pass a different driver for persistence.

Jun 6, 2011 at 1:25 PM
Edited Jun 6, 2011 at 1:45 PM

Thanks I also thing this is issue... I will give a try.

Thanks...

 

Update:

Thanks. That was issue. Now everything is working.

I only have little performance issues with Iso storage mode . For loading 2000 rows I need something about 50 sec to wait...