AJ's blog

March 24, 2007

List the List

Filed under: .NET, .NET Framework, C#, Design Time — ajdotnet @ 5:51 pm

This post is again going deep down to the bits (writing on high-level topics takes so much more time…).

Suppose (again) you were writing some kind of generic serializer or databinding code. Sooner or later you would have to deal with lists. Collections. Arrays. In other words, you would have to deal with a situation like this: 

public class MyObject
    // …

public class MyCollection : CollectionBase
    // …

public class Data
    public MyObject[] MyObjectArray { /* … */ }
    public MyCollection CollectionOfMyObject { /* … */ }
    public IList<MyObject> GenericListOfMyObject { /* … */ }

    public ArrayList ListOfMyObjects { /* … */ }
    public object ThisCouldBeAListOfMyObjects { /* … */ }

In order to analyze some arbitrary object (say an instance of Data), you would use either type.GetProperties() (more suited for serializers) or TypeDescriptor.GetProperties(type) (the better choice for databinding and design time related stuff). You would then look at each property’s type, recognize it is a collection type, and somehow deduce the type of the collection elements (to create them dynamically or to read their properties to create list columns during databinding).

Let’s have a look on what our code could be presented with:

  1. Arrays. They are the most simple collection type, embedded in the language, and are often used by code generation tools. Supporting them is a must.
  2. Collection classes derived from CollectionBase. MSDN states that
        “This base class [CollectionBase] is provided to make it easier for implementers to create a strongly typed custom collection. Implementers should extend this base class instead of creating their own.”
    Therefore CollectionBase was the means of choice before we had generics. Please note that this class comes with a pattern that implies type safe methods in the derived class.
  3. Collections implementing ICollection or IList. This is a more generic approach than using CollectionBase. We will have to look closer at this, but if it worked, it would automatically cover the ColectionBase approach.
  4. Generic collections, implementing ICollection<T> or IList<T>. This is propably the way new code will present collections to our code. Please note that a bunch of methods (like Add, Remove, etc.) that are in the non-generic version part of IList have been pushed down to ICollection<T> in the generic version.
  5. The predefined collection classes in the System.Collections namespace, notably ArrayList, will also have been used quite often.
  6. There is a special interface ITypedList, meant to support databinding. This may help (or it may not.)
  7. Finally we may have to deal with collections that may be present in some untyped property.

Now let’s see which of these cases we can support to what degree:

Arrays: You can check if it’s an array using Type.IsArray and use Type.GetElementType() to get the type of the elements.
➡ Supporting arrays is mandatory and no sweat at all. 100% done.

CollectionBase, ICollection/IList: Neither CollectionBase nor one of the interfaces (also implemented by CollectionBase) tell you something about the element type. The usage of CollectionBase however implies a pattern that will have the implementor support type safe overloads of the usuall collection methods. What we can do is get hold of one of those members (e.g. the Add method or the indexer) and analyze its type.
➡ Supporting arbitrary ICollection classes can be done if they adhere to some pattern (implied by but not restricted to CollectionBase). Let’s call that 90% covered.

ICollection<T>/IList<T>: This case is as easy as arrays are. Well, appart from figuring out the interface. But let’s ignore this exotic cases and settle with, say 99% coverage? Once you got hold of the interface it’s just a matter of calling type.GetGenericArguments().
➡ Supporting generic collections is mandatory and no sweat at all. 99% done.

ArrayList: Here we will raise the white flag. The type of ArrayList does not tell us anything about the element type and no way to get it working. Can we live with that? ArrayList is “not the best choice” as property type, so this restriction might be the encouragement the developer needed to improve his data structures… (allways point out the positive aspects 😉 )
➡ Supporting ArrayList stays at 0%.

ITypedList: ITypedList will give you direct access to the element’s properties (similar to TypeDescriptor.GetProperties(type)). This may be usefull for databinding and design time features — in fact I would regard that as a must, since it is part of the databinding infrastructure of .NET.
For serializers might be used to get the properties and guess the component type (“von hinten durch die Brust ins Auge” — german proverb, literally “from the back through the cest into the eye”, used for arkward indirect ways to achive something). I would consider that only if I absolutely had to.
➡ Supporting ITypedList depends on the purpose of our code. For databinding it should be considered (100% coverage), for serializers it may be a fallback chance, though unreliable. No more than 50% coverage.

Untyped property: No type, no chance to even know it’s a collection.
➡ 0%.

Further complications…

So far we’ve looked at collection types, not at elements. If the collection type does not tell us enough, we may look at the first element in the collection. Asuming that there is one. If not, a serializer might have no problem, yet a databinding scenario might — which is the very reason Microsoft came up with ITypedList.

Another aspect has so far been ignored: We … (OK, I) asumed homogenous collections, i.e. collections of elements of the same type. Collections containing elements of different types (they may have a common base class, or be completely arbitrary) will pose a whole new bunch of problems. This is probably beyond what databinding can support, serializers would have to make sure that each list entry is stored along with type information.

Where are we?

If you take a look at what can be supported and what can’t, you’ll notice that it is simply not possible to cover 100% of the theoretically possible cases. Even some feasible cases will only be covered by 80%. However, if you look closer, those 80% may very well be all you’ll ever need. And if you really stumble over one of the 20% cases (ArrayList might be one of those), don’t try to guess out of the blue; think of some way to feed additional meta information into your serializer.

That’s all for now folks,

kick it on DotNetKicks.com

March 17, 2007

Get view state off __VIEWSTATE!

Filed under: .NET, .NET Framework, ASP.NET, C#, Software Development — ajdotnet @ 4:59 pm

As I announced in a previous post there is something to say about storing the view state on the server…

ASP.NET saves transient control data across postbacks in the “view state”. The means for persisting that data is a hidden field in the pages HTML code. Take some complex controls — especially grids that save the complete data they are working on in view state — the amount of view state data can easily reach, even exceed, the size of the remaining HTML. This can severely affect response times, not to mention the processing time on the server to de/serialize the data and to protect against tampering and other view state attacks.

Apart from enabling HTTP compression (which you should do anyway) and spending some thought on whether you need that particular view state data at all (not a bad idea either) the obvious solution for these issues is not to send the view state data over the wire at all. Let’s take a closer look at that option.


In ASP.NET 1.1. the page class supports customization of view state persistence with two methods:

  • void Page.SavePageStateToPersistenceMedium(object viewState) (see MSDN)
  • object Page.LoadPageStateFromPersistenceMedium() (see MSDN)

This has to be done for every page, but I would recommend using a common base class for all pages anyway.

The default implementation of SavePageStateToPersistenceMedium simply stores the view state in a member variable, actually storing it is defered until Page.OnFormRender is called. OnFormRender uses a LosFormatter to serialize the data into a string representation and puts it into a hidden field. LoadPageStateFromPersistenceMedium uses a LosFormatter to do the reverse. The role of that LosFormatter is noteworthy; the documentation states:

The limited object serialization (LOS) formatter is designed for highly compact ASCII format serialization. This class supports serializing any object graph, but is optimized for those containing strings, arrays, and hash tables. It offers second order optimization for many of the .NET Framework primitive types.

One should pay special attention to the fact that LosFormatter can deal with objects that are not marked as seriablizable, and thus cannot be handled by the standard .NET serialization.

If you want to take the view state data out of the hidden field you can do this simply by overriding those two methods. The things to take into account include:

  • In order to properly support the browser history you need to track the particular page request and link it to the view state data. E.g. you may still use a hidden field to store some kind of view state ID.
  • You must decide how much view state packages you maintain on the server. It’s not feasible to store all view states as long as the user session is alive. Store just one and the second “browser back” in a row will fail.
  • You need some persistence medium to store the view state. The obvious choice would be the user session. However, since your session should be held out-of-proc (i.e. managed by the ASP.NET State Service) you cannot simply put the view state object into the state — see the serialization problem mentioned above — but you may use a LosFormatter and put that string in the session.

As a sidenote: Other articles about this can be found on the internet. They usually propose the ASP.NET cache or a file based approach.
The cache suffers from the same problems that in-proc session state has (see here). It also may remove cache entries far earlier than suitable for view state (e.g. under load conditions).
The file based approach will need some additional cleanup mechanism but has the best potential to support view state with a limitless browser history. It also consumes the least amount of memory.
The session approach suffers from the fact that all session content (triggered by and including all maintained view state packages) has to be serialized with every request. Cleanup however is done automatically.

If I had to think about a fully-fledged high-throughput solution, I would probably try out a combination of database (with a cleanup job) and caching.

I’m not going to provide code, since the algorithm can easily be taken form ASP.NET 2.0 and be applied to ASP.NET 1.1 (i.e. attached to SavePageStateToPersistenceMedium and LoadPageStateFromPersistenceMedium). The good news is that this implementation will work as well if you move to ASP.NET 2.0.


With ASP.NET 2.0 there is no change in principle, just some additional concepts. Make that a huge additional concept. And pittfalls.

Fact one: ASP.NET 2.0 introduced the concept of adapters which are registered in ”.browser”-files for bowser types. Adapters can interfere with different control aspects, such as rendering — and view state.

Fact two: The Page class now has a virtual property PageStatePersister. The default implementation of the ASP.NET 1.1 methods mentioned above now delegate the work to the instance returned by this property. And while HiddenFieldPageStatePersister still is the default persister, one may also return SessionPageStatePersister. Or a custom class.

Fact three: The documented way to replace the persister is to a) write an adapter and b) register it in a .browser file. And c) I don’t like this. First, .browser files do a lot more than just replacing the adapter. Second, how should anybody realize how this happens (the guy who has to maintain my code after I left). Third, in what way is this replacement browser specific? And anyway, I really don’t care for browser differences.

But why not simply forget about the “documented way” and just overwrite the property?

PageStatePersister _pageStatePersister;

protected override PageStatePersister PageStatePersister
        if (_pageStatePersister == null)
            _pageStatePersister = new SessionPageStatePersister(this);
        return _pageStatePersister;

Great, just a few lines of code and I can throw away all the work I did for ASP.NET 1.1 … . ? What’s that? A serialisation exception? Oh boy, I know that problem. OK, I’ll derive my own persister … let’s see… persisters have a method CreateStateFormatter … which is internal? OK, … the formatter isn’t used anyway? The view state is simply put into the session state? And the method doing this is … well, of course it’s not virtual!

So, this leaves two options:

  1. If you already have a homegrown APS.NET 1.1 algorithm, just leave it as it is. It works, so why bother.
  2. If you start something new, try “copy and mend”, i.e. use your favourite help tool (Reflector!) to get the code from SessionPageStatePersister and fix the problems (just use a LosFormatter before putting the view state in the session). Don’t bother with virtual methods, you are the user of a framework, not the implementor.
    And this can easily be ported back to ASP.NET 1.1 if the need arises.

So, finally there. And not too much work either. However this little experience has again raised the question whether (ASP).NET is meant to be extended and tailored to one’s needs — or just to be used as-is.

Thats all for now folks,

kick it on DotNetKicks.com

March 10, 2007

New version of my AddIn…

Filed under: .NET, .NET Framework, C#, Design Time, Software Developers — ajdotnet @ 9:25 pm

BrowseCurrentFile.jpgI just put a new version of my addin on my web site (for a first introduction see this post). Here are the major changes (apart from bug fixing):

  • Browse current file: Methods now show signatures
  • Browse current file: Generics shown correctly
  • Browse (all): Support of progress bar
  • Browse (all): Persistent window size
  • AddIn: first tests under Vista

There is currently an issue under vista with language packs. Menu icons are not shown and shortcut keys are not assigned.

Obviously MS has changed the loading scheme of the resource DLL — I need to fix that, once I get an idea how to do that). They also made the same mistake the once made with localized VBA languages (remember your VBA keywords being translated from IF-THEN-ELSE to WENN-DANN-SONST?). Now they translate the key codes (“Ctrl-Up Arrow” to “Strg-NACH-OBEN-TASTE”) as well as the commands. Can you believe that?

Anyway, I took care that the AddIn works, but nothing more. The missing icons are no vital part. Regarding shortcuts you need to assign them manually until I have fixed that issue. 

I think I have addressed some of the feedback I got (if only in the FAQ help page). Other feedback has been placed on my todo list (including flattened presentation, i.e. a list control rather than a tree control for browse).

I hope you enjoy it.

That’s all for now folks,

kick it on DotNetKicks.com

March 3, 2007

Bury in-proc session state!

Filed under: .NET, .NET Framework, ASP.NET, C#, Software Architecture, Software Development — ajdotnet @ 10:29 pm

In ASP.NET session state can be held in 3+ locations. The choice is between in-proc (i.e. managed in the app domain), out-of-proc (with either ASP.NET state service or in SQL Server), or custom built.

When you create a new web application, in-proc is used by default – which is the worst choice one could make. Apparantly this is the case for simplicity, because it does not need additional setup and works out of the box. ASP.NET state service needs a running service (which is by default not the case), and SQL Server or custom built solutions obviously need even more setup work.

What you should do immediately is to switch to out-of-proc mode!

Why? Well, not for the usually mentioned reasons. Not because using out-of-proc session state increases robustness, i.e. the state will survive if the application restarts, allegedly after some kind of error. Not because MSDN advises against in-proc session state for web garden scenarios (see „caution“). And also not because there is a serialization issue (see „note“).

Don‘t get me wrong, those are all valid reasons. Especially the serialization issue is the very reason why you should use out-of-proc mode right from the beginning rather than switching later. But the reason you should switch is more simple and evene more serious: In-proc session state does not work!

„Are you nuts? I always used it and never had any problems…“. Yes I know. But sorry, you were just lucky. Just because the problem didn’t manifest itself doesn’t mean you didn’t have it. Actually Microsoft tells all the facts one needs to know (they just forgot to tell about the consequence):

„The worker process is subject to a feature named process recycling. Process recycling consists in the aspnet_isapi ability of automatically starting a new process when the existing one is consuming too much memory, responds too slowly, or just hangs. When this happens, new requests are serviced by the new instance, which becomes the new active process.“

And what happens to the in-proc state? Right. Nothing. In other words it dies with the app domain. Session state lost, sorry. And by the way, this is by intention.

„OK, I can allways switch in production, no big deal!“ Yes you can. But don‘t make too many plans for the weekend, you might have to track down that serialization issue. That one is particularly nasty, because it happens out of sync. You can put anything from integers to objects to french fries into the state. ASP.NET just won‘t complain. Only at the end of the request life cycle the session state is serialized and only if some non-serializable object is in the state you (or rather the end user) will be „served“ with a serialization exception. Try to find the root cause for that exception in a non-trivial application and without knowing what the user did beforehand.
Therefore you should wok out-of-proc during development. If you did not do that you are probably better off with in-proc session state and loosing the sessions now and then.

That’s all for now folks,

kick it on DotNetKicks.com

Create a free website or blog at WordPress.com.