Sterling Load data failed after restoring from tombstoning

Apr 27, 2011 at 1:06 PM
Edited Apr 28, 2011 at 12:38 AM

I followed the WP7 tips for Sterling, and use it in my project.

But I found that when my app restoring back the code which trying to load data via ISterlingDatabaseInstance.Load would fail and got an exception, the query code ISterlingDatabaseInstance.Query will success, but failed after getting the Value property of the Lazy<T>.

And I noticed that in the tip, it says

Activation only requires loading keys and indexes into memory and is therefore not an expensive or slow operation. 

so does it mean that my database were not filled with data? if no data, how can I load the data from the database?

Coordinator
Apr 27, 2011 at 1:15 PM

Need more details - what version of Sterling are you using? What is the exception you are receiving? Have you used a tool like the Isolated Storage Explorer to validate the files on the phone? Are you using the recommendations for flushing the database? Have you run it in debug mode and looked at the debug output from Sterling and are there any unusual exceptions or errors displayed in that log?

Apr 27, 2011 at 1:21 PM
Edited Apr 27, 2011 at 1:23 PM

I'm using Sterling OODB v1.0.

The exception is ArgumentOutOfRangeException when getting the Lazy<T>.Value.

Yes, I'm using the flushing exactly same as the sample code in the WP7 tip.

I'll try the ISolatedStorageExplorer and check the debug output later, and post the details here.

 

Thanks

Apr 28, 2011 at 12:33 AM
Edited Apr 28, 2011 at 7:27 AM

Sorry, I post the wrong Exception in previous post.

It actually is IndexOutOfRangeException, and here is the stack.

 

   at Wintellect.Sterling.IsolatedStorage.PathProvider.GetTypeAtIndex(Int32 typeIndex)
   at Wintellect.Sterling.Database.BaseDatabaseInstance.<Load>b__e(Int32 i)
   at Wintellect.Sterling.Serialization.SerializationHelper._Deserialize(BinaryReader br, CycleCache cache)
   at Wintellect.Sterling.Serialization.SerializationHelper._Deserialize(BinaryReader br, CycleCache cache)
   at Wintellect.Sterling.Serialization.SerializationHelper.Load(Type type, Object key, BinaryReader br, CycleCache cache)
   at Wintellect.Sterling.Database.BaseDatabaseInstance.Load(Type type, Object key, CycleCache cache)
   at Wintellect.Sterling.Database.BaseDatabaseInstance.Load(Type type, Object key)
   at Wintellect.Sterling.Database.BaseDatabaseInstance.Load[T,TKey](Boolean key)
   at MyApp.ViewModels.ViewModelHelper.LoadData()
   at MyApp.App.CheckAndCreateDefaultData()
   at MyApp.App.Initialize(Object state)
   at System.Threading.ThreadPool.WorkItem.doWork(Object o)
   at System.Threading.Timer.ring()

I'm loading data from a thread in the pool which try to load the user options for my app.

The exception happened at

 

return MyApp.App.ApplicationDatabase.Load<OptionsModel, bool>(true);

 

And I changed it to loading data in the main thread, it still happens, feel less chance.

Does it mean that I should not loading data right after database activated?

=======

Update:

I keep make the app deactive/active back and forth, quite a few times later, the app stopped working...

or deactive/active back and forth, then active and press back button to leave the app, when you start the app again, it stopped launching and not work again.

While debugging, the log in out put is:

//'taskhost.exe' ... loading dlls...
4/28/2011 2:12:59 AM::Sterling::Information::Sterling is registering database MyApp.Database.ApplicationDatabase
4/28/2011 2:12:59 AM::Sterling::Information::Sterling serialized the master database nextDb=1 nextTable=0 databases=1
4/28/2011 2:12:59 AM::Sterling::Information::Sterling serialized the master database nextDb=1 nextTable=1 databases=1
4/28/2011 2:12:59 AM::Sterling::Information::Sterling serialized 1 table definitions to path Sterling/0/tables.dat for database ApplicationDatabase:
 0=MyApp.Models.OptionsModel, MyApp, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null
4/28/2011 2:12:59 AM::Sterling::Information::Sterling serialized the master database nextDb=1 nextTable=2 databases=1
4/28/2011 2:12:59 AM::Sterling::Information::Sterling serialized 2 table definitions to path Sterling/0/tables.dat for database ApplicationDatabase:
 0=MyApp.Models.OptionsModel, MyApp, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null  1=MyApp.Books, MyApp, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null
4/28/2011 2:12:59 AM::Sterling::Information::Sterling serialized the master type list types=1
4/28/2011 2:13:05 AM::Sterling::Information::Sterling serialized the master type list types=2
4/28/2011 2:13:05 AM::Sterling::Information::Sterling serialized the master type list types=3
4/28/2011 2:13:05 AM::Sterling::Information::Sterling serialized the master type list types=4
4/28/2011 2:13:05 AM::Sterling::Information::Sterling serialized the master type list types=5
4/28/2011 2:13:05 AM::Sterling::Information::Sterling serialized the master type list types=6
4/28/2011 2:13:05 AM::Sterling::Information::Sterling serialized the master type list types=7
4/28/2011 2:13:05 AM::Sterling::Information::Sterling serialized the master type list types=8
4/28/2011 2:13:05 AM::Sterling::Information::Sterling serialized the master type list types=9
4/28/2011 2:13:05 AM::Sterling::Information::Sterling serialized the master type list types=10

// ---------Tombstoning from here -------------------//
'taskhost.exe' (Managed): Loaded 'System.SR.dll'
A first chance exception of type 'System.IO.IsolatedStorage.IsolatedStorageException' occurred in mscorlib.dll
A first chance exception of type 'System.Threading.ThreadAbortException' occurred in mscorlib.dll
A first chance exception of type 'System.Threading.ThreadAbortException' occurred in IsolatedStorageExplorer.dll
A first chance exception of type 'System.Threading.ThreadAbortException' occurred in System.dll
A first chance exception of type 'System.Threading.ThreadAbortException' occurred in mscorlib.dll
A first chance exception of type 'System.Threading.ThreadAbortException' occurred in IsolatedStorageExplorer.dll
A first chance exception of type 'System.Threading.ThreadAbortException' occurred in IsolatedStorageExplorer.dll
A first chance exception of type 'System.Threading.ThreadAbortException' occurred in System.dll
The thread '<No Name>' (0xc22003e) has exited with code 0 (0x0).
The thread '<No Name>' (0xcbb003a) has exited with code 0 (0x0).
The thread '<No Name>' (0xd840062) has exited with code 0 (0x0).
The thread '<No Name>' (0xd970052) has exited with code 0 (0x0).
The thread '<No Name>' (0xd6e006e) has exited with code 0 (0x0).
The thread '<No Name>' (0xcc6006e) has exited with code 0 (0x0).
The thread '<No Name>' (0xde1005e) has exited with code 0 (0x0).

// ---------Normal restoring from here -------------------//

// loading dlls.
4/28/2011 2:19:08 AM::Sterling::Information::Sterling is registering database MyApp.Database.ApplicationDatabase
4/28/2011 2:19:09 AM::Sterling::Information::Sterling de-serialized 2 table definitions from path Sterling/0/tables.dat:
 0=MyApp.Models.OptionsModel, MyApp, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null  1=MyApp.Books, MyApp, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null

sometimes the debugger stopped, and I start the debugger again, the app might restore by chance, sometimes, if the debugger stopped it may failed to restore.

I found an IsolatedStorageException message in the Output every time the tombstoning occured, and it seems like the Known WP7 Issue of IsolatedStorageSettings.Save() is invoked automatically and caused this exception, and there're some ThreadAbortedExceptions after it, and I think it's not caused by my code.

Some times after tombstoning, I press Back button to restore, but the debugger stopped, and failed to restore the app.

Then I start the debugger again, the previous exception IndexOutOfRangeException is shown and handled by Sterling, then leads to my loading failure.

After checking the log of sterling,

I Found that the type list is 8 when I started the debugger after the restore failure, and when my app works the type list is 10.

And I also checked the code, i believe that the types.dat is corrupt when tombstoing/restore many times. ( the BinaryReader reads a Int32, an it is used as the index of the type, but the type list is less than it, so there's an IndexOutOfRangeException.

So I wonder if you have any suggestion on this issue?

Thanks a lot.

// loading dlls.
4/28/2011 2:19:08 AM::Sterling::Information::Sterling is registering database MyApp.Database.ApplicationDatabase
4/28/2011 2:19:09 AM::Sterling::Information::Sterling de-serialized 2 table definitions from path Sterling/0/tables.dat:
 0=MyApp.Models.OptionsModel, MyApp, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null  1=MyApp.Books, MyApp, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null
Apr 28, 2011 at 7:19 AM

I've tested with a new application, it still failed when I deactivate / activate the app.

It indicates that before the database finished flush, the flush process is stopped by the OS, and that leads to the corrupted files which caused the failure of the loading process when activating the app again.

Coordinator
Apr 28, 2011 at 11:09 AM

It sounds like you are trying to work with more data than the phone can handle. Unfortunately, isolated storage on the phone is notoriously slow so you have to chunk data and flush regularly. What volumes of data are you working with and how often do you flush when you are saving within the application?

Apr 28, 2011 at 11:26 AM

Actually, I created a new application, which only created one Database, and I just added one item of my data to it.

Just quickly deactivate and activate, the app, and the app may crash.

 

I just use the WP7 Tip Sample code which handling the App tombstoning events, and load data when App Launching like following ( I noticed that this should be avoid )

var options = App.Database.Load<OptionsModel, bool>(true);
            if (options == null)
            {
                App.Database.Save<OptionsModel, bool>(DefaultOption);
            }

But actually the Save only happened in the first run, if that is successful, the following tombstoning activate/deactivate failed to write the correct data in the IsolatedStorage.

Coordinator
Apr 28, 2011 at 11:29 AM

There's definitely something missing there - might make sense if you can use the contact form to send me the project. I have dozens of people with products in the marketplace using this so I know it's not an issue on the phone, there must be something either with the definition or the process that is getting disconnected. If you are able to share the project feel free to ship it via the contact form. Thanks.

Apr 28, 2011 at 11:38 AM

Sorry, I never used a contact form, where can I find it?

Coordinator
Apr 28, 2011 at 12:22 PM

Click on my name, then click on the contact. That will generate an email, and I can reply with my email for you to send the zipped project to. Thanks.

Apr 28, 2011 at 12:50 PM

Thanks, i've sent you a message.

Coordinator
Apr 28, 2011 at 3:32 PM

I don't see where you are flushing at all, based on the guidelines/best practices. I changed your save code to this:

var options = App.Database.Load<OptionsModel, bool>(true);
if (options == null)
{
    App.Database.Save<OptionsModel, bool>(DefaultOption);
    App.Database.Flush();
}

... and go into it, back out, etc, and it works Best practice is to flush after saves - if doing a lot of saves, do the bulk then flush, if doing a single save, flush directly after.

Apr 28, 2011 at 4:23 PM

 

OK, but the sample code in the wp7 tip says,

_engine.Dispose(); // this flushes to isolated storage
I assumed that will do the flush when deactivating...
As you replied that I should explicitly invoke the db.Flush() after some db.Save(), so how about that _engine.Dispose() line?
Does it flushes data to IsolatedStorage again?
Coordinator
Apr 28, 2011 at 4:29 PM

It will call flush again, but flush tracks internally so if nothing changed since the last save it will do nothing. You want to flush often before the application is tombstoned to minimize the time it takes to dispose the engine.

Apr 28, 2011 at 4:42 PM

OK, that's cool !

The reason I'm asking this is that I found a discussion here similar to my issue --- ArgumentOutOfRangeException in IsolatedStorageDriver.GetTypeAtIndex

So I was worrying about too many flush may lead to this ArugmentOutOfRangeException...

What version in his project? form the code, it seems to be v1.4, so it's ok in v1.0 right?

Coordinator
Apr 28, 2011 at 4:46 PM

You're fine w/ v1.0.

Apr 28, 2011 at 4:53 PM

Thanks for your help, I really appreciate it.

And again thanks for your work on Sterling.

I'll try flush in my real app later and post the result here.

Apr 29, 2011 at 4:06 AM

After I tried your code in my sample application, still get the IndexOutOfRangeException then the app crashed.. after several time activate/deactivate and exit the app.

Seems the flush in Application_Closing or Application_Deactivated not complete writing the database back to the IsolatedStorage...

 

>   Wintellect.Sterling.WindowsPhone.dll!Wintellect.Sterling.IsolatedStorage.PathProvider.GetTypeAtIndex(int typeIndex) Line 132    C#
     Wintellect.Sterling.WindowsPhone.dll!Wintellect.Sterling.Database.BaseDatabaseInstance.Load.AnonymousMethod__e(int i) Line 726 + 0x7 bytes    C#
     Wintellect.Sterling.WindowsPhone.dll!Wintellect.Sterling.Serialization.SerializationHelper._Deserialize(System.IO.BinaryReader br, Wintellect.Sterling.Database.CycleCache cache) Line 376 + 0xc bytes    C#
     Wintellect.Sterling.WindowsPhone.dll!Wintellect.Sterling.Serialization.SerializationHelper._Deserialize(System.IO.BinaryReader br, Wintellect.Sterling.Database.CycleCache cache) Line 404    C#
     Wintellect.Sterling.WindowsPhone.dll!Wintellect.Sterling.Serialization.SerializationHelper.Load(System.Type type, object key, System.IO.BinaryReader br, Wintellect.Sterling.Database.CycleCache cache) Line 368 + 0x7 bytes    C#
     Wintellect.Sterling.WindowsPhone.dll!Wintellect.Sterling.Database.BaseDatabaseInstance.Load(System.Type type, object key, Wintellect.Sterling.Database.CycleCache cache) Line 741 + 0x7 bytes    C#
     Wintellect.Sterling.WindowsPhone.dll!Wintellect.Sterling.Database.BaseDatabaseInstance.Load(System.Type type, object key) Line 655 + 0x8 bytes    C#
     Wintellect.Sterling.WindowsPhone.dll!Wintellect.Sterling.Database.BaseDatabaseInstance.Load<WindowsPhonePanoramaApplication3.OptionsModel,bool>(bool key) Line 633 + 0xb bytes    C#
     WindowsPhonePanoramaApplication3.dll!WindowsPhonePanoramaApplication3.App.Application_Launching(object sender, Microsoft.Phone.Shell.LaunchingEventArgs e) Line 79 + 0x6 bytes    C#
     Microsoft.Phone.dll!Microsoft.Phone.Shell.PhoneApplicationService.FireLaunching() + 0x16 bytes   
     Microsoft.Phone.Interop.dll!Microsoft.Phone.Execution.NativeEmInterop.FireOnLaunching() + 0xb bytes   
     [Native to Managed Transition]   

Apr 29, 2011 at 4:57 AM

I guess if I removed the line _engine.Dispose(), will it be ok when _ActivateEngine ?

        private void _ActivateEngine()
        {
            _engine = new SterlingEngine();
            _logger = new SterlingDefaultLogger(SterlingLogLevel.Information);
            _engine.Activate();
            _database = _engine.SterlingDatabase.RegisterDatabase<MyDatabase>();
        }

        private void _DeactivateEngine()
        {
            _logger.Detach();
            //_engine.Dispose();
            _database = null;
            _engine = null;
        }

or i should modify the code of Dispose since the flush of the tables is done, and the _engine.Dispose just ignore the table flush?

Apr 29, 2011 at 6:03 AM

I found the issue is always caused by the types. When the BinaryReader reads the "types.dat" file, it will get a wrong type index, so that makes the db failed to load.

After reading some piece of code, I found that the PathProvider.Serialize will write the "types.dat" file when tombstoning or application exiting,

First try, I removed the line SterlingFactory.GetPathProvider().Serialize(); in Sterling code of the class SterlingDatabase.Deactivate(),

        /// <summary>
        /// Performs application-defined tasks associated with freeing, releasing, or resetting unmanaged resources.
        /// </summary>
        public void Deactivate()
        {
            lock(Lock)
            {
                _activated = false;
                //SterlingFactory.GetPathProvider().Serialize();
                _Unload();
                _databases.Clear();
                BaseDatabaseInstance.Deactivate();
                _serializer = new AggregateSerializer();
            }

            return;
        }

But sometimes I got an end of file error when loading data.

 

The 2nd try, I removed the PathProvider.SerializeTypes(); line in Sterling code of the class PathProvider.Serialize(),

        public void Serialize()
        {
            _SerializeDatabases();
            //SerializeTypes();
            foreach(var database in _databaseMaster.Keys)
            {
                _SerializeTables(database);
            }
        }

This is because I found this Serialize() method only invoded by the db.Deactivate() which only come from the engine.Dispose().

If I won't create tables at after the Sterling is activated, so I guess I don't need to serialize the types right?

And I think may be some of my types might cause some error handling or time consuming task during serialization which leads to running out of the time which MS limits for the application.

Does it make sense?

I'll keep testing my app after removed this line of SerializeTypes, currently it work ok, I'll update this post after some amount of test.

Apr 29, 2011 at 10:56 AM
Edited Apr 29, 2011 at 12:01 PM

It works ok with the Emulator, but not perfect on the device, guess I need to check other issues.

==================

update:

Actually the app sometimes stops working in the Emulator too 

May 3, 2011 at 1:45 PM

I moved to v1.4,  I tried the IsolatedStorage mode, but still failed so I use the in memory mode, just save and load xml file myself for loading/saving tables.

It works great. Thanks a lot

Jun 1, 2011 at 7:45 PM

I am having same issue as you. Have you solved this issue ?

Jun 2, 2011 at 1:27 AM

yes, the problem solved with memory mode sterling database in v1.4. And I just created/saved the db file (.xml) when the data is modified. But memory mode sterling leads to that you have to reload the data from xml to memory when the app restores from tombstoning.

Coordinator
Jun 2, 2011 at 10:21 AM

If you can provide me with a test on the current 1.4 using an isolated storage driver I'll be happy to debug it. Currently all of the tests for that scenario work, so I don't see where there is an issue - not saying it doesn't exist, but difficult to troubleshoot without having a test to duplicate the problem.

Jun 5, 2011 at 2:06 PM

Hi guys. Now after I have updated to 1.4 I don't receive this error and performance is drastically improved, but I still have issue when my application go to tombstoning entire database is deleted and I don't know how ? Do you have some idea ?

Coordinator
Jun 5, 2011 at 4:40 PM

Take a look at the Windows Phone 7 recipes app included in the latest download. I've tested that multiple times with tombstoning and see no issue whatsoever - what are you doing differently?

Jun 7, 2011 at 2:15 AM
Edited Jun 14, 2011 at 12:56 AM

guys,

I think I used too much types for sterling to handle when tombstoning since I checked the files it created and the data shows the incomplete types list.

While debugging, I also found that every time it failed loading the data is caused by the reading the type file, the count of the types mismatch.

This is my case, and how about yours?