Expected Performance for IsolatedStorage Writes Using Save on Windows Phone 7.8

Feb 18, 2013 at 5:02 AM
I'm using the IsolatedStorage driver and I notice that saving around 200-300 entities to sterling from an empty database takes longer than a few seconds (10+). I'm wondering what the expected approximate time is? I'm using indexes but I've tried with and without and still the time remains a bit high. My object has 7 or so properties that are not complex types. I was using SQLCE before and the time wasn't this long (is this even a fair comparison?).
Feb 18, 2013 at 7:26 AM
I'm not entirely sure. It sound plausible though. Indexes will speed up reading, but I don't think it will speed up writing.
Feb 26, 2013 at 5:40 PM

I'm sure you've seen a few references to LexDB in the forums and it is something I have looked at (especially /w Sterling so dead in the water until recently) but something that jumped out on his blog was a reference to how slow isolated storage is and that perhaps there was a better design to accommodate it.

I'm using Silverlight and pulling down a lot of data and I am really seeing that the HD access (I have an SSD) is really hammered. It does seem like save/loading the data is the slowest part. This is a production application so removing Sterling isn't really a short-term option but my local caching implementation is pretty slow.

Do you think that perhaps there are some optimizations that could be done to the isolated storage driver?




Sterling : This one I liked a lot, some nice workarounds to avoid implementation of full-blown Linq provider especially, but performance and storage were unacceptably slow. Knowing that Silverlight does quota-check on every isolated storage file IO, database access was implemented absolutely wrong technically.
Feb 27, 2013 at 7:24 AM
Hi jaddington,

I have indeed read about Lex.DB and it looks promising. I for one can only encourage the project. It's not exactly clear to me yet how Lex.DB optimizes the isolated storage, but I haven't really looked into it in detail. I'll try to do that and see if Sterling can be optimized in any way.
Feb 27, 2013 at 8:04 AM
I'm frantically pushing out a prod release but it is on my list to take a look. I have a feeling that part of it is shrinking the number of files so that there is less overhead... I haven't even tried implementing anything although I am curious.

Never hurts to get some new ideas.


Feb 27, 2013 at 11:52 AM
Thanks for bringing this up. It's a key point.

The provided isolated storage driver does use a model that segregates files to keep the implementation simple, but it does come at a cost, both with size and performance. The quota checks and wrappers mean the individual file access is slow, and the way the file system works there are blocks of space allocated for each file regardless of the size so the apparent size on disk grows quickly.

This is a known issue and it will be resolved when we release a driver that is a single-file driver. This is, of course, a far more complicated driver to write. It means using random access in a single stream, synchronizing access to the stream, and writing a custom file allocation table so that when records are deleted the driver is smart enough to fill available space if and when it is available to avoid over-fragmentation. It's not trivial so at this stage it's been just a question of time.

It is, however, a priority, and I know for a fact once we have a single file driver we'll experience orders of magnitude of performance improvements and the reported storage allocations will diminish significantly. I have two community contributors now where for awhile I was the main driver, so that's helping tremendously.

I appreciate your patience and willingness to share feedback and insights to help improve the project - it is very helpful and we are listening. Over the next few weeks you will see new versions of Sterling released for new platforms, new drivers, and many more improvements that I believe will help address the majority of needs the community has expressed.

Thanks again,

Feb 28, 2013 at 3:49 AM
I've been doing some research on storage for my WP app and was turned off LocalDatabase due to the fact I have to pollute (or map) my object model with EntitySets and stuff like that. It's not a huge deal, but I also need to define tables and columns and everything Sterling doesn't force me to do. I use EF Code First in my web apps and I like it, but I'm writing a phone client for a webapp whose API returns objects perfect for an OODB... in fact, I wrote a small storage mechanism for Caliburn to store stuff in isolated storage... but I'd basically need to write everything Sterling offers; which is why I started to research. Caliburn takes care of tombstoning for me well, I just need a local cache and storage on the phone to minimize network calls (and offer offline support).

At work, we use Lucene.NET (port of Java Lucene) on a few projects and it's incredibly fast. I wondered if they had added SL support but they haven't yet (though someone's made progress). I mention it because it's open source and may provide some hints on how to optimize single-file storage (though when optimized it only uses 3 files, a couple marker files and then the actual index file). It's great because you can read from the index as it's being written to, and they've implemented all the logic to handle streaming and such.

Something to explore when you're thinking of making the single-file change you talked about, Jeremy.
Feb 28, 2013 at 8:56 AM
@jaddington: Would you like to share what application you've put in production? I'm curious :)
Mar 1, 2013 at 4:42 PM

I'm using it in Silverlight application (electronic Project Manager) used primarily by a single client (a mental health research consortium) to handle the data management side of their research projects databases. It has an Oracle backend with WCF Data Services and CSLA to transfer data. I use Sterling to cache many of the DTO's, all the lookup code tables, and now my ViewModels.

I have two Sterling instances, a memory and a local storage. I basically try to follow this premise:

GetData() - In memory cache ? return value : InLocalCache ? Load/SaveToMemory/Return : CallDbService/SaveToMemory/SaveToLocal

I have some checks and balances and some data will return the cached value and then async check for newer data etc.

I have lots of improvements to make in the caching model (I'm trying to be too fancy at the moment) but it is an improvement. I am taking a massive hit on the login up front though where I refresh/rebuild everything (hence the current discussion questions) and obviously my client's PC range from bad to worse when compared to my development machine and SSD.

Mar 1, 2013 at 9:03 PM
@jaddington: Hah, my app is not as crazy as that, but it's funny how our data retrieval is essentially the same thing (though right now I don't have an override to skip the cache). Good to know I'm not a total dolt. :)