Silverlight is for RIA, and the domain of RIA applications is largely intranet or closed/controlled extranet user groups. This again is what is usually found in larger enterprise companies. Companies that usually have a vested interest in controlling their environment. And in terms of bringing software into production and of operations and maintenance afterwards, every new platform is one platform to many.

So, the odd developer comes along and talks about this great new technology. Does the management care? Probably not. What does it care about? Simple. Money! Money, as in costs for deployment and user support, hardware and licenses to get the stuff up and running, operations and developer training, maintenance. And money as in savings in the respective areas and – the cornerstone, as the business usually pays the bill – impact on the business. All usually subsumed under the term ROI.

About a year ago, I finished an analysis looking into RIA with Silverlight, conducted for a major customer. Not from the point of view of the developer, but that of business people, operations, and IT management:

So, let’s look briefly at each aspect…

User/Business perspective…

The business doesn’t exactly care for the platform Silverlight itself; it cares for its business benefits. Benefits as in improved user experience, streamlined business workflows, office integration, and so on. And since we had some lighthouse projects with Silverlight we were able to collect some customers’ voices:

“This [streamlining with Silverlight] would reduce a [...] business process [...] from ~10 min to less than a minute.”

“Advanced user experience of Silverlight UI helps raising acceptance of new CRM system in business units”

“I was very impressed of the prototype implementation […] with Silverlight 3. Having analyzed the benefits of this technology I came to the conclusion that I want the […] development team to start using Silverlight as soon as possible. [...]”

This is also confirmed by the typical research companies, like Gartner or Forrester:

“Firms that measure the business impact of their RIAs say that rich applications meet or exceed their goals” (Forrester)

Operations perspective…

In production, the benefit of Silverlight applications (compared with respective conventional web based applications) is reduced server and network utilization.

For example, we had a (small but none-trivial) reference application at our disposal, which was implemented in ASP.NET as well as Silverlight (as part of an analysis to check the feasibility of Silverlight for LOB applications). We measured a particular use case with both implementations – starting the application and going through 10 steps, including navigation, searches, and selections. Both applications were used after a warm-up phase, meaning that the .xap file, as well as images and other static files had already been cached.

The particular numbers don’t matter, what matters is the difference between the amount of data that has been exchanged for each step (in case of navigations none at all for Silverlight). For the single steps:

And accumulated over time:

A ratio of roughly a tenth of the network utilization is quite some achievement – considering that the Silverlight application wasn’t even optimized to use local session state and caching, it should be even higher.

This should have a direct impact on the number of machines you need in your web farm. Add the fact that session state management on the client drastically reduces the demand for ASP.NET session state – usually realized with a SQL Server (Cluster) – there is yet another entry on the savings list.

On the down side is the deployment of the Silverlight plugin. For managed clients – especially if outsourcing the infrastructure comes into play – this may very well become a showstopper.

IT Management perspective…

With respect to development and maintenance, what IT Management should care about includes things like ability to deliver the business demands, development productivity, bug rates in production, costs for developer training, and so on.

Actually all areas in which Silverlight can shine, compared with other RIA technologies, and with the typical mix of web technologies as well:

• Rich, consistent, homogenous platform
  • .NET Framework (client and server), Visual Studio, Debugger, C#
  • Reduced technology mix, less technology gaps, less broad skill demands
• Improved code correctness and quality…
  • compiler checks, unit testing, code coverage, debugging, static code analysis, in-source-documentation, …
• Improved architecture and code
  • Clean concepts, coding patterns, clear separation of client code, lead to better architectures
  • Powerful abstractions lead to less code (up to 50% in one project), less complexity, less errors

Customers’ voices in this area:

“between our desktop app and the website, we estimate 50% re-use of code”

“a .NET developer can pretty much be dropped into a SL project. […] This is a huge deal […]”

“As alternative for Silverlight we considered Flash. […] only Silverlight could provide a consistent development platform (.NET/C#). […]”

Conclusion…

Taking all this together, and considering that enterprise companies usually have the tooling and test environments (well…) readily available, this all adds up to something like the following bill:

Whether the bill looks the same for your company or for one particular project, of course, depends on many things. Especially nowadays with all the hubbub around HTML5 and mobile applications (without any relevant Silverlight support). But if RIA is what you need, the Silverlight will quite often yield far more benefits than any other option.

Still, you need to do your own evaluation. However, I hope to have given you some hints on what you might focus on, if you want to sell technology to the people who make platform decisions in your company.

The actual analysis was fairly detailed and customer specific. But we also prepared a neutralized/anonymized version, which we just made available for download (pdf). (Also directly at SDX.)

That’s all for now folks,
AJ.NET

The last post was about the chances and benefits of HTML5. But there are also some exaggerated expectations that HTML5 cannot fulfill. Mainly this concerns the term “RIA”, and the effect HTML5 will have in this area.

Or as someone wrote:

“Will HTML 5 one day make Flash, Silverlight and other plug-in technologies obsolete?” (link)

Actually this is a very regular question I have to answer. (Often enough it doesn’t come as question, but as statement. Or accusation. Sic!)

I already detailed how HTML5’s video and canvas will take over tasks that have formerly been solved by other technologies, namely Flash. And some people seem to infer that if Flash is a RIA technology, and HTML5 obsoletes Flash, then HTML5 must obviously make RIA technologies obsolete in general.

Quite wrong. Actually the fact that Flash is used for video and 2D graphics has nothing to do with Flash being a RIA technology. Flash simply happened to be able to delivering these features, both in terms of technical capability as well as broad availability. That it also happens to be a RIA technology is more or less happenstance.

But before moving on we need to clarify the terms “RIA” and “RIA technology”…

My personal definition of RIA technologies relates to the following attributes: Stateful programming model, with some kind of page model, for applications running in a browser sandbox. This includes Flash, JavaFx, Silverlight as a browser plugin (but not in its WP7-platform variation).

Wikipedia applies the terms to Flash, Java (not JavaFx! Sic!), and Silverlight. Sill, this is debatable, and a year ago even Wikipedia had a far broader definition, but in my experience this actually covers the common understanding. Curiously Adobe claims that AIR is their RIA technology, not Flash. But me, Wikipedia, and general consensus agree that Flash indeed is a valid RIA technology.

By the way: Leaving HTML+AJAX out of this picture is by no means meant to be deprecatory, it just reflects common understanding. Wikipedia actually makes the distinction based on the (lack of) necessity to install an additional software framework.

And a final tidbit: Once upon a time Microsoft advertised Silverlight as Flash replacement, addressing video and graphics, just like in the typical Flash use cases. However, even with growing adoption of the Silverlight plugin, Silverlight never became a serious competitor for Flash in that area. (This may actually have played a role in Microsoft’s commitment to HTML5…) Still, Silverlight has long outgrown this narrow definition and later versions has put more emphasis on business features.

So, let’s have a look at where HTML5 reaches its limits and where RIA technologies might kick in. I’ll look at regular web applications before moving on to RIA applications.

Web Applications

HTML5 is going to address standard demands of web applications, including those addressed today by Flash. This will have a crowding-out effect on Flash and RIA technologies in general. But once the demands go beyond being “standard”, RIA technologies will find their niche even in web applications.

On example could be premium video delivery: Some vendors will probably be eager to offer unique selling propositions in the emerging markets of WebTV and high quality HD video content (probably involving DRM).

Since Flash can no longer play the trump of being the only or the broadest available platform in this area, this will also change the picture among the RIA technologies. Especially Silverlight has been very successful in this area recently. Take the Olympic Games or maxdome.

Other examples include complex user interactions that are not feasible with canvas and script, e.g. Mazda’s car configurator, and similarly dynamic data visualizations.

Finally there is the support of additional features. RIA technology vendors certainly have shorter innovation cycles than standards bodies. This especially includes hardware support (camera, game controller, hardware accelerated animations and 3D).

These scenarios all require the user to accept the plugin – which might become a more severe issue if this necessity is less ubiquitous. Thus for the web site provider this always incurs the question whether he can compel his users to use his offering despite that nuisance, or whether he may have to provide a (perhaps simplified and less capable) HTML based version.

RIA Applications

HTML5 won’t turn HTML into a RIA technology. It doesn’t come with a new programming model, doesn’t change server side processing, page model, and postbacks, doesn’t change that fact that the HTML ecosystem really is a conglomerate of diverse technologies.

Many applications – be it multimedia, some kind of hardware dependency, or line of business – simply require the potential of rich client applications. For these, HTML simply cannot deliver what is necessary. Typical demands include:

• Data centric applications: Large amounts of data; data input with complex validations, lists with extended feature sets, …
• Usability: Immediate feedback (not possible with postbacks), dynamic UIs with forms built dynamically or changing depending on user input, …
• Business features: Printing, graphical presentation, Office integration, …
• Offline capability…
• Connectivity: Communication patterns such as peer-to-peer, server pushes, …
• Multimedia and hardware support: Animations, camera, microphone, multitouch, …
• Rich platform: Stateful programming model, component model, feature rich controls, rich databinding capabilities, …

While these are certainly not the demands for typical web applications. But intranet applications and applications addressing some distinct or closed user group on the web are very well within this category. Prominent example is SAP, one can also think of WebTV portals, home banking, or other.

In the past java applets were often used to cover these demands. Recently AJAX approach have spread, but while this worked to some degree, it often falls short of meeting the demands completely. From a technical perspective, RIA technologies are the adequate choice in these scenarios. And (in my opinion), Microsoft Silverlight is currently the best technology available in that area. Adobe AIR lacks availability and adoption, Flash alone is not sufficient, and JavaFx seems to die a slow death. 

Conclusion

HTML5 will push RIA technologies out of their makeshift role (video and canvas). However this doesn’t affect the feasibility of employing RIA technologies on their own turf, i.e. beyond-HTML-capability demands in web applications and fully fledged RIA applications.

However, since this “pushing out of RIA technologies” mainly affects Flash, HTML5 has an interesting effect on the RIA market: Broad availability is no longer a strong USP for Flash, which is to the benefit of Silverlight. Add the hazy prospect of JavaFx and the fact that Silverlight is not only a RIA platform, but also enters devices (WP7, WebTV), and HTML5 may actually further the adoption of Silverlight – not as cross-platform tool, as it was once intended, but in all areas not covered by HTML5.

The one argument in favor of HTML5 – which no RIA technology is likely to ever achieve – is its universal availability across all platforms, even if that comes at a cost.

Where are we?

The conclusion of this little series may be as follows: The conflict, or enmity, between HTML5 and RIA that some people see (or exaggerate) doesn’t really exist. There may be a competition between HTML5 and Flash, but even that may turn out differently form what people expect.

Actually HTML5 and RIA complement each other. There are areas in which one technology certainly make more sense than the other, other areas in which there is a choice, again other areas in which a combination of both may work best; even areas in which neither is an ideal choice. E.g.…

• A web application addressing the broadest available audience? HTML5.
• An LOB application with high usability demands? Silverlight.
• A mobile application addressing a broad audience? HTML5. As long as not tighter device integration is necessary, in which case one has to address several mobile OSes…

And between these black-and-white examples there’s a lot gray areas, to be decided on a case-by-case basis. And usually the important thing is not exactly which technology you favor. The important thing is to make an informed decision, aware of the pros and cons, and not solely based on political opinions of certain apologists.

That’s all for now folks,
AJ.NET

HTML5 is a lot of things for a lot of people. For some it is a vehicle to rid the web of patent laden video formats. For Apple it is a means to keep Flash off iPhone and iPad. Microsoft uses it to push IE9. Some people are using it to – at least try to – bury RIA. Anything else…? Oh yes, some people actually look into HTML5 technologies.

In an attempt to join the last group, here is my opinion on what HTML5 is and what it will mean.

The Standard

Formally, HTML5 is just the next version of HTML. (Period!) In terms of scope however it covers some more ground than “HTML” did before. Included is not only the markup language we know as HTML 4.1, but also the formerly distinct standards XHTML and DOM.

Given that HTML 4.0 became a recommendation in 1997 one can only say (shout!) “It’s about time!”. The fact that HTML 4.x cannot be considered “state of the art” for quite some time now is exactly the reason why browser vendors chose to implement proprietary extensions and why technologies like Flash – even ActiveX – have to fill in the gaps.

HTML5 is available as working draft (latest version today from January 2011) and all major browser venders are more or less committed to implementing it:

• Apple Safari
• Google Chrome
• Microsoft Internet Explorer
• Mozilla Firefox
• Opera

Regarding general support for HTML5, the industry for once agrees on something. Surprising enough. Regarding the details however, one has to be careful, which part of HTML5 is supported by which browser to what degree. It may be going to take some time – and more than one version number – until we get broad HTML5 support on all browsers.

For some time the critics gloated at the fact that HTML5 has barely reached “working draft” status and that “recommendation” status is not expected before 2022. Even if that were the case, it would have been totally irrelevant, as the industry is implementing and therefore standardizing parts of HTML5 right now. Additionally the W3C group is just now reconsidering its approach and looking for a more timely “delivery”. TODO

So far for the term, now for the content…

The Details

In the details, HTML5 is a conglomerate of different – and not necessarily related – improvements and new features. Of course this includes the two features mentioned mostly: video and 2D graphics (a.k.a. “canvas”).

Video is nothing spectacular: It simply displays a video, albeit without depending on some 3^rd party plugin (namely Flash). Benefit for web developers: a standardized API based on HTML and JavaScript (rather than having to learn another technology). Benefit for the user: He can watch the video, independently on the device and plugin availability.

Canvas allows 2D graphics, even (with JavaScript) animated. This again allows (in principle) to address use cases formerly the domain of Flash. Diagrams for usage statistics, stock exchange rates, and so on. Contrary to simple (server rendered) images this could be including user interaction. This even may include more complex things, like web games; the technology is up to it, as this classic proves.

Regarding multimedia, one probably hast to mention audio support for a complete picture.

Video and canvas are mentioned quite often, probably because they constitute the use cases today most often addressed using Flash. Still, it would be unfair to reduce HTML5 to these two.

If it comes to plain old markup code HTML5 offers some improvements regarding java script control, as well as semantic tags like “header” and “footer”.

Regarding user interaction HTML5 offers new types of input fields (e.g. date picker), complete with validations (e.g. textbox with email address). Also add APIs for drag-and-drop, browser history management, and some other.

You may find this in more detail at the W3C or on Wikipedia. A nicely illustrated introduction can be found here (German, but the pictures shouldn’t need translation ).

Yet this still doesn’t conclude the list. For a complete picture one has to name topics closely related to (even if not formally part of) HTML5: CSS3, Web Storage, and (especially for mobile applications) Geolocation.

It should be noted that the recent improvements in JavaScript execution in all major browsers (well… – IE starting with IE9) also contribute to HTML5: Many of the features unfold their full potential only with scripting. It’s a fair assumption that HTML5 will cause a further growth in scripting in general, and probably the adoption of respective AJAX frameworks.

The Impact

At a closer look HTML5 is a mixture of rather different and unrelated things – none of them especially outstanding. Basically HTML is only upgraded to accommodate current state-of-the-art technologies. All together a (long overdue and therefore a little extensive) evolution, but certainly no revolution or “groundbreaking upgrade”, as some like to think.

Therefor the relevance of HTML5 is not the functionality itself. It stems from two facts:

1. The broad support from all major browser vendors. No matter why they do it, that fact that they agree on HTML5 in such an unequivocal and joined fashion is without precedence. This will likely ensure that all browsers level on comparable features and capabilities. Which is important to break todays disparities and incompatibilities. HTML5 shows all promises of becoming a platform for the web that is state-of-the-art as well as broadly available. Something HTML has increasingly failed at in recent years.
2. The timely appearance together with the emerging mobile ecosystem (smart phones, pads). In this area we have a far more diverse and inhomogeneous platform landscape than on desktops (iOS, Android, WP7, other mobile OSes, desktop OS adaptions). No platform has a dominance similar to Windows, thus vendors need to address more than one platform. And web applications built for mobile devices are the only feasible cross platform approach available. Even if HTML5 lacks full device integration (e.g. phone integration or multitouch), it goes a long way with web storage, geolocation, and rudimentary offline capabilities.

To conclude: HTML5 is not going to be some academic standard, it will be a true and broadly supported industry standard. Together with browser improvements, especially regarding script engines, HTML will become an adequate development platform for state-of-the-art web applications and the emerging mobile area.

For web developers HTML5 is a good thing. At least as long as the agreement among the browser vendors holds – and as long as we don’t have to wait another 10 years for the next version.

That’s all for now folks,
AJ.NET

META: This blog got a little sleepy recently. But actually I’ve been spending quite some time writing blog posts, albeit for our company blog. And I’m planning to reuse some of that content here (this post is actually the first one). Also I’ve been busy in several new areas, including WP7, and I may have something to say about those, too. So, this blog is still alive and kicking.

There’s a small hype around HTML5 for some time now. Ironically the reason were more political than for technical ones, since browsers begin to support HTML5 only just now. Be it organizational hassles, discussions about a video format, or a certain company having issues with Flash on their iPlatform. And since Microsoft has announced its decision to make HTML5 their cross platform strategy, the last big browser vendor has joined the camp.

And this is a good thing! Not only homogenizes HTML5 the web platform again, it is also the only feasible platform for cross platform development in the mobile world, which is much more diverse than our desktop eco system.

On the other hand I am regularly irritated about what people think HTML5 will be able to accomplish. Especially in the relation to RIA applications that are all but dead, according to various sources:

“HTML5, a groundbreaking upgrade to the prominent Web presentation specification, could become a game-changer in Web application development, one that might even make obsolete such plug-in-based rich Internet application (RIA) technologies as Adobe Flash, Microsoft Silverlight, and Sun JavaFX.” (link)

“Will HTML 5 one day make Flash, Silverlight and other plug-in technologies obsolete? Most likely, but unfortunately that day is still quite a way off.” (link)

“Sure maybe today, we have to rely on these proprietary browser plugins to deliver content to users, but the real innovative developers and companies are going to standard on HTML 5 and in turn revolutionize how users interact with data.  We all want faster web applications and the only way to deliver this is to use HTML 5.” (link)

To put it bluntly: HTML is no RIA technology and HTML5 is not going to change that. Thus Silverlight is a valid choice for any RIA application today, and it will be one tomorrow.

On the other hand, HTML5 is certainly going to deliver features that today are the domain of RIA technologies, namely Flash. And this will affect RIA technologies in some way.

Then how will HTML5 affect RIA? Well, I’m afraid, it’s not that simple and there is no short and sufficient answer that I’m aware of. In order to decide – probably on a case by case basis – what HTML5 can do, in which use cases HTML5 is the right answer, and in which cases RIA technologies still are the better choices, we need to take a closer look at some details…

To keep it a little more concise, I’m going to break with my usual habit of very long posts. I’m splitting the remainder into two additional not-quite-so-long posts:

• Part II: HTML5 – New Standard for Web Applications
• Part III: HTML5 – The Limits

Stay tuned,
AJ.NET

The following post is by courtesy of wordpress. The reported content is by courtesy of my readers — in other words: YOU!

Thank you. And a happy new year.

The stats helper monkeys at WordPress.com mulled over how this blog did in 2010, and here’s a high level summary of its overall blog health:

The Blog-Health-o-Meter™ reads Wow.

Crunchy numbers

About 3 million people visit the Taj Mahal every year. This blog was viewed about 40,000 times in 2010. If it were the Taj Mahal, it would take about 5 days for that many people to see it.

In 2010, there were 21 new posts, growing the total archive of this blog to 130 posts. There were 81 pictures uploaded, taking up a total of 2mb. That’s about 2 pictures per week.

The busiest day of the year was May 21st with 453 views. The most popular post that day was Visual Studio 2010 Architecture Edition.

Where did they come from?

The top referring sites in 2010 were forums.silverlight.net, dotnetkicks.com, forums.asp.net, google.com, and google.co.in.

Some visitors came searching, mostly for silverlight 4 validation, visual studio 2010 class diagram, visual studio 2010 architecture, visual studio 2010 modeling project, and visual studio 2010 architecture explorer.

Attractions in 2010

These are the posts and pages that got the most views in 2010.

Visual Studio 2010 Architecture Edition March 2009
12 comments

Understanding Validation in Silverlight February 2010
4 comments

ASP.NET 2.0 DataBinding Examined July 2006
7 comments

Silverlight Bits&Pieces: The First Steps with Visual State Manager January 2010
3 comments and 1 Like on WordPress.com,

Command line tool vs. PowerShell Cmdlet January 2008
3 comments

During the last PDC Microsoft announced a next innovation: Async C#, currently available as CTP. There is also a respective white paper; I recommend reading it, if you need an introduction.

Just for the record, here’s what we are talking about:

This roughly translates to:

As you can see, the compiler mainly translates an await into a new task and the remainder of the method into a continuation for that task.

To tell the truth, the actual generated code looks quite different; this is semantically equal, though.

Simple enough, but that’s only a simple case. Actually there’s more:

• If we are talking about a UI thread, the runtime ensures that the continuation runs on the UI thread as well. The task version would need to marshal the call to the UI thread explicitly, e.g. by calling Dispatcher.BeginInvoke.
• If the wait is contained within a loop of some kind, the compiler also handles that, cutting the loop into asynchronous bits and pieces. Similarly for conditional calls.

That’s a nice set of features and there is no doubt that this approach works (there is a CTP after all). And reception within the community so far has been very positive.

However!

If this is really such a compelling feature, it should be quite easy to present a bunch of examples in which it pays off. Instead, what I found was a bunch of examples that made me wonder. Therefore I decided to put it to some…

Litmus tests…

To test whether async/await actually merits the praise, I set out to recheck and question the examples provided by various Microsoft people:

• Soma Segar’s introduction (actually taken from the white paper mentioned above)
• Netflix (used by Anders Hejlsberg during his PDC talk and available as sample within the CTP)
• Eric Lippert’s discussion

Note: Examples have a tendency to oversimplify. I chose this set to overcome that effect: Soma provides a simple introduction, Netflix is more “real world”, and Eric looks at it form a language designer point of view. Three different samples, three different people, three different perspectives.

Note 2: The code I present is from my test solution (download link at the end), thus it is not exactly a citation, rather it’s complemented with my bookkeeping code. I also didn’t address every question that may arise (such as exception handling) explicitly. This post got long enough as it is.

Soma Segar’s blog

Let’s begin with Soma’s introduction, as it is the most simple one:

He provides a „conventional“ asynchronous solution, that looks… quite convoluted, actually. Note that I prepended the necessary code to synchronize the call.

And his respective solution using the async/await looks quite nice in comparison:

Easier to write, and easier to understand, no question about that.

However: Looking at the awfully convoluted task based version, I asked myself: „Who would write such code in the first place?“.

The use case is to load some web pages and sum up their size. And to do this asynchronously, probably to avoid freezing the main thread. And the solution – awful and async/await alike – insists on getting the value for one web page asynchronously, but coming back to the UI thread to update the sum. Then starting the next fetch asynchronously, and again come back to the UI thread to update the sum.

But then, why not put the whole loop in a separate thread? The only caveat would be the marshaling of UI calls (i.e. a call to Dispatcher.BeginInvoke, which I left out here in favor of simple tracing, but it’s addressed in the next example). Somewhat like this:

This is very close to the await/async version, nearly equally easy to write, and certainly as easy to understand. And it did not need a language addition, merely a little thinking about what it is that shall actually be accomplished.

OK, one example that didn’t really convince me.

Netflix (sample in the CTP)

The Netflix sample is part of the CTP and was also used by Anders Hejlsberg during his PDC talk. The sample solution comes with synchronous version, conventional asynchronous version, and new async/await version.

The synchronous version looks straight forward, with 43 of relatively harmless LOC for the relevant two methods (LoadMovies and QueryMovies):

The conventional asynchronous version – do I even have to mention it? – is in comparison an ugly little piece of code. For the sake of completeness:

Twice as much code (85 LOC), a mixture of event handler registrations with nested lambdas and invokes. It takes considerable time to fully understand it. No one in his right mind would want to have to write this.

The async/await version – again – turns the synchronous version into a similarly straight forward looking asynchronous version:

Again the initial verdict: The async/await version looks nice and clean compared to the ugly task based version, and very similar to the synchronous one. Hail to async/await.

But again, looking at the ugly asynchronous version, I asked myself, “Who would actually write such code? And why? Masochistic tendencies?”

Let’s take a step back and look at the use case: Call a web service asynchronously to avoid freezing the UI (as the synchronous version does), update the UI with the result, and call the web service again if there is more information available.

The task based version tries to manually dismantle this loop into a sequence of asynchronous call followed by callback on the main thread to update the UI, followed again by the next asynchronous call. Add cancelation and exception handling and it is surprising that it isn’t even more ugly than it already is.

But! Why even try to dismantle the loop? Why not put the whole loop in a separate thread? Yes, that would solve the use case, and yes, it would be just as clean as the synchronous version. The only caveat is that each interaction with the UI has to be marshaled into the main thread, but I can live with that. Here it is:

I left the QueryMovies Method out for brevity, as it is similar to the synchronous version above.

Again this is very close to the await/async version, again nearly equally easy to write, and – again – certainly as easy to understand. And – again (pardon the repetition) – it did not need a language addition. Merely – ag… ok, ok – a little thinking about what it is that shall actually be accomplished.

Another example that didn’t convince me. But two is still coincidence.

Eric Lippert‘s discussion

Eric has a slightly different use case: For a list of web addresses, fetch the web page and archive it. There should be only one fetch at a time, and only one archive operation, but fetch and archive may overlap (described here). This is more about overlapping different operations, rather than avoid freezing the UI, as in the previous examples.

The synchronous version is this simple:

He constructed the asynchronous version (not even mentioning async/await) as state machine, described in the post I just mentioned. I’ll spare you the reiteration this time; follow the link if you have doubts that it is far more convoluted than the simple loop above. But of course, async/await comes to the rescue:

Nice and simple. But can you deduce the logic behind this? I have to admit it required some mind twists on my part. Now, to make it short, here is the task version:

Not quite as simple as the async/await version, but if you look closely, it’s actually a very close translation into continuations. And… equally inefficient. You may have noticed the comments I put in there, stating that there is some unnecessary synchronization between a fetch and an earlier archive operation.

So, this time I actually went a little bit further: This is why I complemented a little test data, bookkeeping, and a nice little printout to visualize the sequence. Here’s what the async/await version (and similarly the task version) produces:

“‘f” stands for fetch, “a” for archive. The lower part shows the operations according to their start and end time. You can depict nicely how tasks follow other tasks, and how fetch and archive run in parallel. No fetch overlaps another, no archive overlaps another archive operation. Well. Have a look at line 3. See how the fetch starts after the archive in line 1 was finished? It could have started earlier, couldn’t it? Same in line 6. That’s the unnecessary synchronization I mentioned above.

A little working on the task version solves that by chaining the fetches and archives onto their respective tasks:

However, now I becomes a little more complex and hard to understand.

Going back to the drawing-board. What again is the use case? It’s actually a pipeline, with multiple stages (two in this case), each stage throttled to process only a certain amount of input values (one in this case). The cookbook solution is to put a queue in front of each stage, and have separate workers (again, in our case just one) take input from this queue, process it, and place the result in the next stages queue. Like this:

Two queues of lambdas, two tasks processing these queues, the first lambda placing new entries into the second queue upon completion. If you ask me, this is way closer to the proposed use case than either the task based versions or the async/await version. And the logic behind it – intention as well as control flow –, which in this case is a little more complicated than in the previous examples, is also far easier to comprehend. And the best part: Look at the output:

Fetch and archive start independently of each other, making the solution more effective than the async/await version. Why? Because I actually thought about the problem and the solution, rather than simply throwing asynchronicity on the code.

If anything, this example raises even more concerns than the other ones.

Verdict

Three different samples. Three times presenting some conventional task based asynchronous code – code that is rather convoluted, ugly, and hard to understand. Three times a nice async/await version that clearly makes the code nicer.

But also three times it “only” took a little thinking to come up with a far nicer task based version than the proposed convolution. Two times very close to the async/await version. The third time actually more efficient than that.

One could actually get the impression that the convoluted task based code has been deliberately made ugly in order to provide a better motivation for the new features. I wouldn’t go that far, but still, it raises some questions, as to how these examples came into being. Perhaps a back-port of what the compiler produces for async/await?

Granted, each example I examined may have its own compromises which slightly invalidates it (as is often the case with examples); had I only found one, I wouldn’t have bothered writing this post. But: One is an incident, two is coincidence, three is a trend. And I did not stumble over one example yielding a different result.

So, to summarize my concerns:

1. I showed that the value of async/await is less compelling than presented. This does raise the question, whether it is still compelling enough to merit a language extension.
   

   I judge that value by its ability to making code shorter, easier to write, easier to comprehend. Or to quote Eric Lippert:

   “Features have to be so compelling that they are worth the enormous dollar costs of designing, implementing, testing, documenting and shipping the feature. They have to be worth the cost of complicating the language and making it more difficult to design other features in the future.” http://blogs.msdn.com/b/ericlippert/archive/2008/10/08/the-future-of-c-part-one.aspx

2. The way I verified my concerns was actually to take one step back and think about the demand. I hate to think that async/await actually encourages developers to think less and in turn write less than optimal code – especially the less experienced developers who’ll have a hard time understanding the implications of those keywords in the first place.

To make that clear: I did not show anything that suggests that the async/await language feature is per se wrong. Nor did I prove that there is no use case that does not suffer from my concerns (only that Microsoft failed to present them yet – or I failed to find them).

Conclusion

Here my conclusion: I could live with concern #1, but #2 is something that gives me the creeps. There’s two things I would like Microsoft to do to make the feature work (or drop it altogether):

1. Make more obvious what the code actually does (e.g. see the comments here for some naming suggestions – although my concern goes beyond simple naming issues).
2. Rethink the task based samples you are basing your reasoning on. We don’t need a feature that makes convoluted code simpler; we need features that make code simple, we would write in the first place.

BTW: Whether async/await is backed into the language or not, this endeavor showed one thing quite clearly: Asynchronicity is important, and becoming even more so. And Microsoft certainly did accomplish something great with the TPL, on which all these discussions built without even mentioning it anymore. And as Reed pointed out, Tasks play a major role in the CTP as well.

Finally: Here is the download link for those who’d like to recheck my findings.

Merry Christmas!

That’s all for now folks,
AJ.NET

It took some time for Microsoft to realize what fatal message they had sent, but eventually Muglia backpedaled, and from there one it seems that every other Microsoftie and close associate came out to deny the imminent death of Silverlight. So far I’ve stumbled over:

• Steve Ballmer (CEO): http://www.microsoft.com/presspass/press/2010/nov10/11-01Statement.mspx
• Bob Muglia (President, Server and Tools Business): http://team.silverlight.net/announcement/pdc-and-silverlight/
• S. Somasegar (Senior Vice President, Developer Division): http://blogs.msdn.com/b/somasegar/archive/2010/11/01/silverlight-lighting-up-the-client.aspx
• Scott Guthrie (Corporate Vice President, .NET Developer Platform): http://weblogs.asp.net/scottgu/archive/2010/11/04/silverlight-questions.aspx
• Tim Heuer (Program Manager for Microsoft Silverlight): http://timheuer.com/blog/archive/2010/11/01/silverlight-is-dead-long-live-silverlight.aspx
• Jesse Liberty (Senior Community Program Manager in Microsoft Developer Guidance): http://jesseliberty.com/2010/11/02/silverlight-alive-and-well/

Among the non-Microsofties where

• Mary-Jo herself: http://www.zdnet.com/blog/microsoft/microsofts-muglia-reconfirms-html-is-microsofts-cross-platform-play/7854
• Dan Wahlin (MVP ASP.NET): http://weblogs.asp.net/dwahlin/archive/2010/11/01/silverlight-is-dead-the-moon-is-made-of-cheese-and-html-5-is-ready-for-prime-time.aspx
• Jeremy Likness (MVP Silverlight): http://csharperimage.jeremylikness.com/2010/11/silverlight-and-html5-microsofts.html
• Oh, me, by the way: http://flurfunk.sdx-ag.de/2010/11/silverlight-technologie-abkundigung.html (German)

So, just for the record: I sincerely believe that Microsoft is still very much committed to Silverlight as RIA technology for regular (read non-WP7) clients.

Still, as relieved as I am that this mess unfolded that way, it kept me thinking. What if? I mean, what if Microsoft actually had dropped Silverlight on the client…?

What if?

Just imagine… What would happen if Microsoft actually had changed their strategy? What if Silverlight was really dead on the client, and “only” the development platform for WP7?

• For the vast majority of regular web applications: Nothing much would have happened. Use cases here include mostly video and advertisement. And due to the availability of the plugin this is the domain of Adobe Flash. With the advent of HTML5, and it’s coverage of video and graphics (canvas) – and none the least the backing it gets from Apple – it should have been clear to everybody with open eyes that HTML5 will be the future in that area. But that will happen at the expense of Flash, not Silverlight!

But from there it would go downhill, and Microsoft would start losing…

• Microsoft would lose a platform for non-typical demands on the web. Demands that go far beyond what HTML5 can deliver. Complex UIs such as car configurators (Mazda), HD video streaming (maxdome). And Silverlight it gaining momentum in the market, not Flash or some other technology.
   
• Microsoft would lose their platform for RIA applications. In this area HTML5 is of no further relevance at all, rather Adobe AIR and JavaFx are the competition. And in this area Silverlight is way ahead of the completion, both technically in terms of business features, as well as by adoption (usually intranet applications, but also SAP).
   
• Microsoft would lose the developer base it relies on for Windows Phone 7, and with it WP7 itself. One of the big selling points for WP7 is the fact that WP7 uses the very same platform as is used for RIA, thus every developer using Silverlight instantly becomes a WP7 developer. Ironically, focusing Silverlight on WP7 would take away that advantage. Silverlight would become a platform you have to learn before doing phone development. And since WP7 is just taking off, the future not yet certain, why take the risk? Why not learn Android instead? Who would then build the apps Microsoft needs?
   
• It gets worse: Microsoft would lose credibility, and the trust of the developer community. This is not a technology at the end of its life cycle we’re talking about. It’s a technology just beginning to take off, and a technology that they told us were strategic! A technology more and more developers are just beginning to adopt, to invest in. If Microsoft dropped Silverlight – without any warning I might add –, how would those developers react? How could they trust Microsoft to be true to what they call “strategic” in the future? I know what I would think.
   
• Needless to say that this would affect their partners and customers in the same way. Who would invest in any platform if he cannot be sure the platform is maintained for a reasonable time (rather than being dropped at the spur of a moment). And if the vendor cannot be trusted? The platform may be as good as it wants, the first thing to care about is to protect my investments.
   
• In the end this would include every API, every platform, every offering they have. This specially includes Azure – the very platform Microsoft is betting the company on. Which developer would work against that API? Which ISV would build his software on Azure? Which partner would counsel his customers to use Azure? Which enterprise would rely on Azure with his applications and his data? It’s a strategic platform for Microsoft, sure. But with dropping Silverlight they would just have taught us what that means.

Ultimately Microsoft could lose… Microsoft. Because at the end of the day, credibility is the most important thing. That’s what made this whole thing a marketing fiasco. Not the fact that a bunch of developers and companies sat on the wrong bandwagon. Lose credibility and you lose the company.

Now, all this is hypothetical – I hope I made that clear with my statement above. And it is also just an opinion and certainly exaggerated in some points. But it may very well be the kind of trash and FUD that Microsoft will be experiencing for some time. Which is why I believe that Microsoft will continue to have to do damage control for quite some time to come.

Yours sincerely,
AJ.NET

The use case was simple: A tool chain, processing some files and spitting out others (host service descriptions as input and configuration and proxy code as output – is doesn’t really matter though). At one stage I needed to process a file line by line. And, as usually, my code was nice, clean, tested, maintainable, performing quite well, robust, and unbreakable. In one word: Just perfect. Well, until someone broke it. All he did was feeding a “slightly” larger code fragment — about 5MB file size for a corresponding text file — and he encountered OutOfMemoryExceptions. Sik!

So, what on earth could have happened here? Well, one of the least things I would have expected: string.Split and StringBuilder – despite the fact that I of course had employed them correctly, concise with the general recommendations! 

The Cost of String.Split

It turned out that string.Split trades memory for performance…. Take a string of a certain length, say 100, and split into, say, 10 parts. The result you’ll get will consist of the array (4 bytes per entry, 10 entries in our example), and the content fragments. This is roughly the same memory consumption for the result as for the input string (see also on MSDN).

Note: I‘m leaving out the overhead for internal management of strings and memory allocation in general.

That’s the memory consumption that is obvious and cannot be avoided (OK, I’ll get back to that one!). But what does String.Split add to that cost, if only temporarily?

String.Split internally creates an array of integers, with the length of the string as size. In other words for an input string of 100 characters it creates an int[100], using up 400 bytes! This array is used to remember the index of the found separator, by writing the found index at the next free position. That implies that only the first slots are actually used, the same number as the number of parts the string is split into. In our example 90 slots are simply wasted. This is a usage ratio of 10%! It would only need this array completely in the extreme case of a string only consisting of separators.

In case of StringSplitOptions.RemoveEmptyEntries and at least one empty entry, another temporary array for the result is created, add another array the size of the number of parts. Seems things can only get worse.

Once more: For a any given string as input String.Split returns something more than that as result – and on top of that it needs twice as much memory temporarily!

OK, so 100 characters, 200 bytes, produce 240 bytes result (40 bytes for the array) and 400 bytes temporarily on top. Who cares? But wait, remember the 5MB text file? The resulting string needed 10MB and the result ~11MB. Additionally ~20MB temporarily, of which only 4% – less than 1MB – where actually used. And all this is subject to the Large Object Heap. Well? Who cares now?

Mending…

Granted, that’s an extreme case. But how often did I use String.Split in other cases? How often within loops? How big where the input strings? How much unnecessary overhead and memory pressure did that introduce?

What if I could get rid of the temporary overhead? Great. What if I could even get rid of the result? Wait… what?

Matter of fact, quite often the result from String.Split is processed once, entry per entry, in a loop. And that’s what enumerators are for. So, if I had a respective implementation, instead of writing

string[] lines = text.Split(Environment.NewLine[0]);
foreach(string line in lines)
    …

I could call a method that returns an enumerator:

var lines= text.SplitString(Environment.NewLine[0]);
foreach(string line in lines)
    …

And with the lazy evaluation of enumerators I would get one part at a time. And once I’m done processing it, I drop it before going on to the next part. Granted, I still need to touch every part, but no longer all at once. When memory get’s tight, chances are all those parts are in gen 0 when the GC hits.

Meaning I would do no longer consume more than three times the memory of the input string at once. A string of 100 characters/200 bytes as input would not acquire additional 600 bytes. A 10MB string would not need additional 31MB (and only using 12MB of that anyway, subject to the LOH).

Well, I had a minute to spare… . I wrote a helper class for this post, tested to behave exactly like String.Split, so it should be possible to just replace every call to Split with the SplitString extension method, like above. You can find the code for download here.

That’s all for now folks,
AJ.NET

This error essentially tells you that a server call somehow messed up – which is obvious – and nothing beyond, much less anything useful to diagnose the issue.

This not exactly rocket science, but the question comes up regularly at the Silverlight forums, so I’m trying to convey the complete picture once and for all (and only point to it, once the question comes up again – and it will!).

I’m also violating my rule not to write about something that is readily available somewhere else. But I have the feeling that the available information is either limited to certain aspects, not conveying the complete picture, or hard to come by or to understand. Why else would the question come up that regularly?

The Root Cause

So, why the “NotFound”, anyway? Any HTTP response contains a numeric code. 200 (OK) if everything went well, others for errors, redirections, caching, etc.; a list can be found at wikipedia. Any error whatsoever results in a different result code, say 401 (Unauthorized), 404 (NotFound), or 503 (Service unavailable).

Any plugin using the browser network stack (as Silverlight does by default) however is also subject to some restrictions the browser imposes in the name of security: The browser will only pass 200 in good cases or 404 without any further information in any other case to the plugin. And the plugin can do exactly NIL about it, as it never gets around to see the original response.

Note: This is not Silverlight specific, but happens to every plugin that makes use of the browser network stack.

Generally speaking there are two different groups of issues that are reported as errors:

1. Service errors: The services throws some kind of exception.
2. Infrastructure issues: The service cannot be reached at all.

Since those two groups of issues have very different root causes, it makes sense to be able to at least tell them apart, if nothing else. This is already half of the diagnosis.

Handling Service Errors

Any exception thrown by a WCF service is by default returns as service error (i.e. SOAP fault) with HTTP response code 500 (Internal Server Errors). And as we have established above, the Silverlight plugin never get’s to see that error.

The recommended way to handle this situation is to tweak the HTTP response code to 200 (OK) and expect the Silverlight client code to be able to distinguish error from valid result. Actually this is already backed into WCF: A generated client proxy will deliver errors via the AsyncCompletedEventArgs.Error property – if we tweak the response code that is. Fortunately the extensible nature of WCF allows us to do just that using a behavior, which you can find readily available here.

Once we get errors through to Silverlight we can go ahead and make actual use of the error to further distinguish server errors:

1. Business errors (e.g. server side validations) with additional information (like the property that was invalid).
2. Generic business errors with no additional information.
3. Technical errors on the server (database not available, NullReferenceException, …).

It’s the technical errors that will reveal more diagnostic information about the issue at hand, but let’ go through them one by one…

Business errors with additional information are actually part of the service’s contract, more to the point, the additional information constitutes the fault contract:

These faults are also called declared faults for the very reason that they are part of the contract and declared in advance. Declared faults are thrown and handled as FaultException<T> (available as full blown .NET version on the server, and as respective counterpart in Silverlight), with the additional information as generic type parameter:

Note: There’s no need to construct the FaultException from another exception. And of course this StandardFault class is rather simplistic, not covering more fine-grained information, e.g. invalid properties – which you may need in order to plug into the client side validation infrastructure. But that’s another post.

On the client side this information is available in a similar way, and can be used to give the user feedback:

Generic business errors are not part of the service contract, hence they are called undeclared faults, and they cannot contain additional information beyond what the already got. From a coding perspective they are represented by FaultExceptions (the non-generic version, .NET and Silverlight) and thrown and handled similarly to declared faults:

However, the documentation states…

“In a service, use the FaultException class to create an untyped fault to return to the client for debugging purposes. […]

In general, it is strongly recommended that you use the FaultContractAttribute to design your services to return strongly typed SOAP faults (and not managed exception objects) for all fault cases in which you decide the client requires fault information. […]”

MSDN

That leaves arbitrary exceptions thrown for whatever reason in your service. WCF also translates them to (undeclared) faults, yet it uses the generic version of FaultException, with the predefined type ExceptionDetails. This way, any exception in the service can (or rather could) be picked up on the client:

However, while ExceptionDetails contains information about exception type, stack trace, and so on, that fault contains by default only a generic text, stating “The server was unable to process the request due to an internal error.”. This is exactly as it should be in production, where any further information might give the wrong person too much information. During development however it may make sense to get more information, to be able to diagnose these issues more quickly. To do that, the configuration has to be changed:

And now the returned information contains various details about the original exception:

BTW: To complete the error handling on the client, you need to to address the situation were the issue was on the client itself, in which case the exception would not be of some FaultException type:

This covers any exception thrown from the WCF service, provided it could be reached at all.

Alternative routes…

As I said, tweaking the HTTP response code is the recommended way to handle these errors. This is still a compromise on the protocol level to work around the browser network stack limitation. However, there are other workarounds to do that, and for the sake of completeness:

1. Compromise on the service’s contract: Rather than using the fault contract, one could include the error information in the regular data contract. This is typical for REST style services, e.g. Amazon works that way. For my application services I am generally reluctant to make that compromise. The down side is that is doesn’t cover technical errors, but that can be remedied with a global try/catch in your service method.
2. Avoid the browser network stack. Silverlight offers its own network stack implementation (client HTTP handling), though it defaults to using the browser stack. Using client HTTP handling, one can handle any HTTP response code, as well as offering more freedom regarding HTTP headers and methods. The downside however is, that we lose some of the features the browser adds to its network stack. Cookie handling and browser cache come to mind.

Handling Infrastructure Issues

If some issue prevented the service to be called at all, there is obviously no way for it to tweak the response. And unless we revert to client HTTP handling (which would be a rather drastic means, given the implications), the Silverlight client gets no chance to look at it either. Hence, we cannot do anything about our CommunicationException: NotFound.

However, by tweaking the response code for service exceptions as proposed above, we at least make it immediately obvious (if only by indirect reasoning) that the remaining CommunicationException: NotFound is indeed an infrastructure issue.

The good news is that infrastructure issues usually carry enough information by themselves. Also they appear rarely, but if they do they usually are quite obvious (including obviously linked to some recent change), affect any call (not just some), and are easily reproducibly. Hence using Fiddler one can get information about the issue very easily (even in the localhost scenario).

The fact that the issue is pretty obvious pretty fast, in turn makes it usually quite easy to attribute it to the actual cause – it must have been a change made very recently. Typical candidates are easy to track down:

• Switching between Cassini and IIS. I have written about that here.
• Changing some application URLs, e.g. giving the web project a root folder for Cassini, without updating the service references.
• Generating or updating service proxies, but forgetting to change the generated URLs to relative addresses.
• Visual Studio sometimes assigns a new port to Cassini if the project settings say “auto-assign port”, and the last used port is somehow blocked. This may happen if another Cassini instance still lingers around from the last debugging session.
• Any change recently made to the protocol or IIS configuration.

This only get’s dirty if the change was made by some team member and you have no way of knowing what he actually changed. But since this will likely affect the whole team, you will be in good company

Wrap up

There are two main issues with CommunicationException: NotFound:

1. It doesn’t tell you anything and the slew of possible reasons makes it unnecessarily hard to diagnose the root cause.
2. It prevents legitimate handling of business errors in a WCF/SOAP conformant way.

Both issues are addressed sufficiently by tweaking the HTTP response code of exceptions thrown within the service, which is simple enough. Hence the respective WCF endpoint behavior should be part of every Silverlight web project. And in case this is not possible for some reason, you can revert to client HTTP handling.

Much if not all of this information is available somewhere within the Silverlight documentation. However, each link I found only covered certain aspects or symptoms, and I hope I have provided a more complete picture on how to tackle (for the last time) CommunicationException: NotFound.

That’s all fro now folks,
AJ.NET

I have my Silverlight application sitting on an .aspx page, occupying all real estate. I work a while. And suddenly there is this scrollbar appearing on the right side:

To hunt it down, I create a new Silverlight project. No scrollbar. I compare the .aspx‘’. No difference to speak of. I deconstruct the Silverlight part of my application in hope the scrollbar will disappear. It won’t.

Finally I stumbled by accident over the reason (which is – in retrospect – quite obvious):

Here’s the .aspx fragment that creates the Silverlight plugin:

Created by VS, I changed some code above this frament, and reformatted it in VS for better readability. And you know what? It’s already broken!

And I broke it by… drum-roll please… reformatting it in VS! The issue is the (irrelevant, according to any standard I’m aware of) whitespaces between the tags. Manually changing the code back to the original (closing object tag and iframe element at the end)…

… will solve the problem and the scrollbar is gone:

Just one more reason to prefer Silverlight over HTML, if you ask me!

That’s all for now folks,
AJ.NET

Pakistan is encountering a "disaster [that] is one of the most challenging that any country has faced in recent years” (BBC). The infrastructure is already down, and yet another flood is announced (CNN). Food, clean water, and medicine are unavailable in large areas, diseases are imminent (BBC).  

At the same time the Red Cross (ICRC) reports “relief ‘not in sufficient quanitites’” (BBC) and the UN states that “massive scale-up [is] needed in Pakistan flood relief operation” (UN).

The German “tagesschau” reports that donations for Pakistan have been 620,000 €. During the same amount of time after the Haiti earth quake, donations have been as high as 20,100,000 € – 32 times as much (ARD, tagesschau, 12.08.10, 3:16). And this is apparently a general trend.

If you can read this blog post, chances are you’re better off than many people and certainly those affected by this catastrophe!

DO SOMETHING!

• International Committee of the Red Cross
• UNICEF and UNHCR
• Caritas 
• Or your preferred national relief organization… 

If you need some additional motivation try this photo series or youtube.

AJ.NET

Note: Apart from driving me crazy, I’ve been stumbling over this issue quite a few times in the Silverlight forums. Thus I’m going through this in detail, explaining one or the other seemingly obvious point…

Many ASP.NET LOB applications run on the intranet with Windows integrated authentication (see also here). This way the user is instantly available from HttpContext.User, e.g. for display, and can be subjected to application security via a RoleProvider. Silverlight on the other hand runs on the client. I have written about making the user and his roles available on the client before. However, the more important part is to have this information available in the WCF services serving the data and initiating server side processing. And being WCF, they work a little different from ASP.NET, or not, or only sometimes….

Starting with Cassini…

Let’s assume we are developing a Silverlight application, using the defaults, i.e. Cassini, and the templates Visual Studio offers for new items. When a “Silverlight-enabled WCF service” is created, it uses the following settings:

Now there’s (already) a choice to make: Use ASP.NET compatibility? Or stay WCF only? (That question may be worth a separate post…). With ASP.NET compatibility, HttpContext.* is available within the service, including HttpContext.User. The WCF pendant of the user is OperationContext.Current.ServiceSecurityContext.PrimaryIdentity. Take the following sample implementation to see which information is available during a respective call:

The client code to test that service is simple as can be:

The XAML is boilerplate enough, but for the sake of completeness:

I choose compatibility mode, and as the client shows, HttpContext.User is available out of the box:

Great, just what an ASP.NET developer is used to. But compatibility or not, it also shows that the WCF user is not available. But! WCF is configurable, and all we have to do is set the correct configuration. In this case we have to choose Ntlm as authentication scheme:

And look what we’ve got:

Great, no problem at all. Now we have the baseline and are ready to move on to IIS.

Now for IIS…

Moving to IIS on the developer machine is simple. Just go to the web project settings, tab “Web”, choose “Use Local IIS Web Server”, optionally creating the respective virtual directory from here. In order to work against IIS, Visual Studio needs to run with administrative permissions.

Moving from Cassini to IIS however has a slew of additional pitfalls:

• The service URL
• The IIS configuration for authentication
• WCF service activation issues
• The WCF authentication scheme
• localhost vs. machine name

Usually they show up as team (which obviously doesn’t help), but let’s look at them one by one.

The service URL

There’s one difference between how Cassini and IIS are being addressed by the web project: Projects usually run in Cassini in the root (i.e. localhost:12345/default.aspx), while in IIS they run in a virtual directory (e.g. localhost/MyApplication/default.aspx). This may affect you whenever you are dealing with absolute and relative URLs. It will at least cause the generated service URLs to differ more than just by the port information. Of course you can recreate the service references at that point, but you don’t want to do that every time you switch between Cassini and IIS, do you?

BTW: There’s a similar issue if you are running against IIS, using localhost, and you create a service reference: This may write the machine name into the ServiceReferences.ClientConfig (depending on the proxy configuration), e.g. mymachine.networkname.com/application, rather than localhost. While these are semantically the same URLs, for Silverlight is qualifies as a cross domain call. Consequently it will look for a clientaccesspolicy.xml file which is probably not there and react with a respective security exception.

The solution with Silverlight 3 is to dynamically adjust the endpoint of the client proxy in your code to point to the service within the web, the Silverlight application was started from:

Silverlight 4 supports relative URLs out of the box:

Coming versions of the tooling will probably generate relative URLs in the first place; until then you’ll have to remember to adjust them every time you add or update a service reference.

The IIS configuration for authentication

This one may be obvious, but in combination with others, it may still bite you. Initially starting the application will result in the notorious NotFound exception:

Note: To be able to handle server exceptions in Silverlight you’ll have to overcome some IE plugin limitations, inhibiting access to http response 500. This can be achieved via a behavior, as described on MSDN. However this addresses exceptions thrown by the service implementation and won’t help in the infrastructure related errors I’m talking about here.

The eventlog actually contains the necessary information:

No windows authentication? Well, while Cassini automatically runs in the context of the current user, IIS needs explicitly being told that Windows Authentication is required. This is simple in IIS configuration, just enable Windows Authentication and disable Anonymous Authentication in the IIS configuration for the respective virtual directory.

WCF service activation issues

Running again will apparently not have changed anything at all, displaying the same error. With just a seemingly minor difference in the eventlog entry:

That’s right. The service demands to run anonymous after it just demanded to run authenticated. Call it schizophrenic.

To make a long story short, our service has two endpoints with conflicting demands: The regular service endpoint requires Windows Authentication, while the “mex” endpoint for service meta information requires Anonymous access. OK, we might re-enable anonymous access, but that wasn’t indented, so the way to work around this activation issue is to keep anonymous access disabled and to remove the “mex” endpoint from the web.config:

Curiously generating the service reference still works in Visual Studio (perhaps with login dialogs, but still)…

The WCF authentication scheme

We’re still not there. The next issue when running the application might be a login credentials dialog when the service is called. And no matter what you type in, it still won’t work anyway, again with the NotFound exception. Unfortunately this time without an eventlog entry.

Again, to make it short, IIS doesn’t support Ntlm as authentication scheme, we need to switch to Negotiate… .

And now it works:

Do I have to say that this configuration doesn’t run in Cassini? Right, every time you switch between IIS and Cassini you have to remember to adjust the configuration. There is another enumeration value for the authentication scheme, named IntegratedWindowsAuthentication, which would be nice – if it worked. Unfortunately those two values, Ntlm and Negotiate, are the only ones that work, under Cassini and IIS respectively.

localhost vs. machine name

Now it works, we get the user information as needed. For a complete picture however, we need to look at the difference between addressing the local web server via localhost or via the machine name: Calls against localhost are optimized by the operating system to bypass some of the network protocol stack and work directly against the kernel mode driver (HTTP.SYS). This affects caching as well as http sniffers like Fiddler, which both work only via the machine name.

Note: This may actually be the very reason to switch to IIS early during development, when you need Fiddler as debugger (to check the actually exchanged information). Otherwise it’s later on, when you need it as profiler (to measure network utilization). Of course you’ll want http caching enabled and working by that time.

Of course you can put the machine name in the project settings, yet this would affect all team members. Perhaps a better idea is to have the page redirect dynamically:

Another word on caching: In IIS6 caching has to be explicitly set for the .xap file, using 1 day as default cache time and allowing 1 minute at the minimum. During development this may be an issue. With IIS7 caching should be automatically set to CacheUntilChange and you may also set the cache time with a resolution in seconds.

Where are we?

OK, that was quite a list of pitfalls and differences between Cassini and IIS, even IIS6 and IIS7. Some of this may go away with the new IIS Express. Some will stay and remain a nuisance. Visual Studio initially guides you towards using Cassini. At one time however you’ll have to switch to IIS. And since you cannot have both at the same time, this may be an issue especially in development teams. My recommendation would be: Start with IIS right away or plan the switch as a concerted action within your team.

That’s all for now folks,
AJ.NET

<content locale=”de; de-DE”>

… für die der deutschen Sprache mächtigen Leser: SDX hat den firmeneigenen Blog „Flurfunk” gestartet.

Die Themen sind ein bunter Mix aus kleineren technischen Artikeln (und Kollegen die das mit dem “klein” noch nicht verstanden haben ), netten Hinweisen aus dem Back-Office  , aktuelles aus dem Büro, Veranstaltungshinweise – und nicht zuletzt Marketing.
OK, auch nicht ganz so ernst gemeintes ist dabei ;o). Schön bunt ist‘s auf jeden Fall!

Einfach mal reinschauen und guten Tag sagen

</content>

That’s all for now folks,
AJ.NET

Now for the practical implications of this new pattern. Let’s look at an example to get the idea, and a better understanding of the consequences in code…

Let’s assume a component that is instantiated, does some work (allegedly asynchronous, and notifying about the progress) and provides the result at the end via an event. This is akin to making a server call, showing a confirmation message box, or the way the BackgroundWorker component works.

Example 1: Using events

First, implementing the a component the classical way would look somewhat like this:

The result event needs a respective EventArgs class, the declaration and the trigger method:

public class WorkResultEventArgs : EventArgs
{
    public object ResultData { get; set; }
}

public class MyWorkingComponent1
{
    public event EventHandler<WorkResultEventArgs> WorkResult;
    
    protected virtual void OnWorkResult(object resultData)
    {
        if (WorkResult != null)
            WorkResult(this, new WorkResultEventArgs() { ResultData = resultData });
    } 
    …
}

A work progress event should go a little further and provide cancelation support:

public class WorkProgressEventArgs : CancelEventArgs
{
    public int Progress { get; set; }
    public object SomeData { get; set; }
}

public class MyWorkingComponent1
{ 
    public event EventHandler<WorkProgressEventArgs> WorkProgress;
    
    protected virtual bool OnWorkProgress(int progress, object someData)
    {
        if (WorkProgress == null)
            return true;
        
        var ea = new WorkProgressEventArgs() { Progress = progress, SomeData = someData, Cancel = false };
        WorkProgress(this, ea);
        return ea.Cancel ? false : true;
    }
    …
}

Now we only need the actual worker method:

public class MyWorkingComponent1
{ 
    …
     
    public void StartWork()
    {
        int sum = 0;
        for (int i = 0; i < 10; ++i)
        {
            sum += i;
            if (!OnWorkProgress(i, sum))
                return;
        }
        OnWorkResult(sum);
    }
}

Again, we may assume that there is some asynchronicity involved, e.g. the loop could contain a web request or something. But this example should do for the sake of the argument.

The usage (as by support form Visual Studio to create the event handlers) would look like this:

public void Test1()
{
    var worker = new MyWorkingComponent1();
    worker.WorkProgress += new EventHandler<WorkProgressEventArgs>(Worker_WorkProgress);
    worker.WorkResult += new EventHandler<WorkResultEventArgs>(Worker_WorkResult);
    
    worker.StartWork();
}

void Worker_WorkProgress(object sender, WorkProgressEventArgs e)
{
    Console.WriteLine(e.Progress + ":" + e.SomeData);
}

void Worker_WorkResult(object sender, WorkResultEventArgs e)
{
    Console.WriteLine("Result:" + e.ResultData);
}

Creating the component, registering the event handlers, running the task, throw the component away. The fact that events are multi cast capable is never used at all (and never will, as the component is rather short-lived).

I guess we can agree that this is all very boilerplate. And all in all, that’s quite some overhead, from the component perspective as well as from the client code.

Example 2: Using callbacks

Now let’s try the new approach. Rather than defining an event, I pass in two callbacks. The information that was carried in the EventArgs is moved to the parameter lists, thus no need for these classes. The Cancel property is replaced by the return value of the callback. And since the client code always follows the same idiom, I expect the callbacks as constructor parameters, eliminating a source of errors along the way — something that is not possible with event handlers:

public class MyWorkingComponent2
{ 
    public Action<MyWorkingComponent2, object> WorkResult {get;set;} 
    public Func<MyWorkingComponent2, int, object, bool> WorkProgress { get; set; } 
         
    public MyWorkingComponent2( 
        Action<MyWorkingComponent2, object> workResult, 
        Func<MyWorkingComponent2, int, object, bool> workProgress) 
    { 
        WorkResult = workResult; 
        WorkProgress= workProgress; 
    } 
    …
}

The worker method changes only slightly:

public class MyWorkingComponent2
{ 
    …
         
    public void StartWork() 
    { 
        int sum = 0; 
        for (int i = 0; i < 10; ++i) 
        { 
            sum += i; 
            if (!WorkProgress(this, i, sum)) 
                return; 
        } 
        WorkResult(this, sum); 
    } 
}

That’s it. No EventArgs classes, no events, no respective OnEventHappened methods. Granted, the callback declarations are a little more complex, and their parameters also lack intellisense providing information about the semantics of each parameter. But otherwise? Way shorter, way more concise, way less overhead. The actual worker method hasn’t changed at all, but all the event related overhead is gone, which amounted to only 40% LOC.

Now the client code, first only slightly adapted:

public void Test1()
{
    var worker = new MyWorkingComponent2(
        (sender, resultData) => Worker_WorkResult(sender, resultData),
        (sender, progress, someData) => Worker_WorkProgress(sender, progress, someData)
    );
    
    worker.StartWork();
}

bool Worker_WorkProgress(object sender, int progress, object someData)
{
    Console.WriteLine(progress + ":" + someData);
    return true;
}

void Worker_WorkResult(object sender, object resultData)
{
    Console.WriteLine("Result:" + resultData);
}

As you can see, it didn’t change that much. But passing in the lambdas via the constructor fits the use case far better than events, and it is even more robust, as I cannot forget to pass in a callback via the constructor, the way I can forget to register an event handler.

Speaking of lambdas, and since the implementation is that simple, we can even simplify the client code further by omitting those two handler methods:

public void Test1()
{
    var worker = new MyWorkingComponent2(
        (sender, resultData) => { Console.WriteLine("Result:" + resultData); },
        (sender, progress, someData) => { Console.WriteLine(progress + ":" + someData); return true; }
    );
    
    worker.StartWork();
}

Alright, this would have been possible with events as well if you used anonymous methods. But Visual Studio guides you otherwise and early examples of anonymous methods (before we had lambdas) where rather ugly, so I doubt that can be seen as valid counterargument. Here however lambdas can be seen as typical means of choice.

Verdict

Neat? Net result:

• I’m writing less code on the event source side, including no longer declaring EventArgs classes.
• I’m writing less code on the event sink side.
• The handler methods can use clean parameter lists (rather than EventArgs).
• I’m eliminating the risk of forgetting to register event handlers by making the callbacks explicit parameters.
• I’m elimination the danger of leaks due to failing to consistently deregistering event handlers.
  • (That was not addressed in the example, but still.)
• When chaining together several of these steps I can make the logic – especially conditional processing – more explicit and concise.
  • Events would either require setting up beforehand (partly unnecessary overhead), or setup on demand, cluttering the handler with registration and deregistration code.

All in all, this is way more readable, way more robust, and way more efficient than using events.

I for one have begun to adopt this scheme quite liberally. My Silverlight bookshelf application has wrappers for service calls that translate the event to callbacks (several actually, including error handling and other demands). My dialogs always take callbacks for OK and Cancel. I so far have two ICommand implementations, both take callbacks (one with parameters, the other without). I even have a PropertyObserver class that translates a PropertyChanged event into a callback.
Actual event handlers? Apart from the wrappers enabling what I just presented, only a few reacting to control events.

In other words: This is not just an interesting technical detail. It really changes the way I’m addressing certain demands.

That’s all for now folks,
AJ.NET

That got me thinking. Why did I recommend callbacks? When and why did I abandon events? What’s wrong with events anyway?

Well, there’s nothing wrong with events at all. It’s just that Silverlight, asynchronous, and multithreaded processing have changed the picture (and I may have worked too much in those areas lately ). And this is the deal:

1. Until recently I used to “see” (read: write code for/against) events mainly from the event consumer side. WinForms, WebForms; register handler, react to something. That kind of stuff.
   • Since “recently” I kind of had to do that less often. Why? Silverlight data binding solved many of the demands I previously used to address with event handlers. Making a control invisible for example. (Events still drive databinding, but at the same time databinding shields them away from me.)
2. Also since “recently” I have had to implement the providing part quite a bit more often. Why? Silverlight databinding relies on certain interfaces that contain event declarations, namely INotifyPropertyChanged, INotifyCollectionChanged, and INotifyDataErrorInfo.
3. And the “event-based asynchronous pattern”. Yep. We’ll get to that one.

OK, let’s try to classify these scenarios.

Broadcasts

The first two points are just two sides of the same coin: The radio broadcasting scenario.

• Some component wants to advertise interactions or state changes; hence it broadcasts them by way of events.
• Some client code needs to get notified about one or the other of these events; hence it subscribes to the event by way of an respective event handler, to consume it from there on.

Same as radio, the broadcaster broadcasts and doesn’t care whether anyone listens. Same as radio, the receiver is turned one and listens as long as something comes in. Well, the analogy stops at the life time: Event source and consumers tend to have similar life time.

Passing the Baton

The 3rd point is actually a quite different scenario: Start some work and have an event notify me about the result (and sometimes about intermediate state). Once I receive the result I let go of the participant and pass the baton on to the next piece of work.

Same as in a relay run, each participant does one job and once it’s done, he is out of business. Same as in a relay run, participation is obligatory – take someone out (or put something in his way) and the whole chain collapses.

Needless to say that this is nothing like the broadcasting scenario…

Usually the reason for the event approach (rather than simple return values) is asynchronous processing; and in fact this is not a particularly new pattern – BackgroundWorker works accordingly. On the other hand the pattern is still evolving, as the usual pattern for asynchronous work has been no pattern at all (i.e. leave it to the developer, as Thread or ThreadPool do), or the IAsyncResult pattern (relying on a wait handle). New developments however start to employ events more often, and Microsoft has actually dubbed this pattern as “event-based asynchronous pattern” (see MSDN).

One area which relies heavily on this pattern is server requests in Silverlight, via WebClient or generated proxies. But it doesn’t stop there, as Silverlight is asynchronous by nature, rather than by exception: Showing a modal dialog, navigating to another page, (down-)loading an assembly. And quite often these single incidences are chained together to form a bigger logical program flow, for example:

• The user clicks the delete button –> the application shows the confirmation dialog –> a call to the delete operation is made –> IF it succeeds (the code navigates to a list page –> …) OTHERWISE (an error message box is shown –> …)

Any arrow represents an event based “hop” to bridge some “asynchronicity gap” – essentially turning the logically sequential chain into a decoupled, temporary register and deregister event nightmare.

Coming back to the beginning of the post: This is the scenario I was discussion with my colleague. And doing this with a whole bunch of events and respective handler methods is simply awkward, especially if you even have to provide the event sources, usually with respective EventArgs classes. And the issue of having to consistently deregister the event handlers in order to avoid memory leaks becomes more prevalent.

Changing the Picture…

Inevitably I got annoyed with the setup/teardown orgies, and eventually I began to abandon events in this case and started passing simple callbacks along. Like this:

void DeleteBook(Book book)
{
    string msg= "Delete book #" + book.Inventory + ", ‘" + book.Title + "’ ?" ;
    MessageBoxes.Instance.ShowConfirm(null, msg, 
        ok => { if (ok) BeginDeleteBook(book); }
    );
}

void BeginDeleteBook(Book book)
{
    this.DeleteBookCall.Invoke(book.BookID,
        ea => NavigateToBooks());
}

And actually I’m not the only one following this approach. The Task Parallel Library TPL for example has already started to make heavy use of callbacks. So this is definitely not limited to Silverlight…

Note: Also this lays the ground for a next evolutionary step: Coroutines.

Caliburn has a nice example of what this looks like; a little weird at first glance actually, but it collects all that logically sequential but technically asynchronous control flow in one method. Jeremy digs a little deeper into the topic in his post “Sequential Asynchronous Workflows in Silverlight using Coroutines”. 

Anyway, even without going into coroutines, the callbacks over events approach has its merits in terms of coding efficiency. I’ll provide a motivating example, next post.

That’s all for now folks,
AJ.NET

This is actually a defining feature of the next evolutionary step of UI, namely Natural User Interfaces (NUI). As wikipedia states…

A NUI relies on a user being able to carry out relatively natural motions, movements or gestures that they quickly discover control the computer application or manipulate the on-screen content. The most descriptive identifier of a NUI is the lack of a physical keyboard and/or mouse. (wikipedia)

While Apple seems to take the lead in public perception, Microsoft has a rather mixed lineup: With smart phones Windows Mobile 7 seems a bit like “taking the last chance”, even if the move to Silverlight as a platform is a bold one and (IMO) a good one. On the other hand they just managed to drop the very promising – by itself as well as positioned against the iPad – Courier project. As a colleague stated in your internal company blog: “I’m frustrated. Period.” And lastly Microsoft has Surface which has no competition I’m aware of at all (unless you want to build one yourself).

Surface is not only commercially available, it also adds the capability to detect objects placed on the table and thus goes beyond plain multi touch. And it is subject to further research, as this excerpt from PDC09 shows: (better quality here, at 83:00)

Looking Ahead

Well, this is kind of what we have today. If you would like to see where this might be heading, have a look at the Microsoft Gives Glimpse Into the Future talk Stephen Elop held early ‘09. It’s a 36 minutes video, but you may jump to 14:00 and watch the presentation of "Glimpse in the future". What’s presented there is impressive: Live translations enabling people talking with each other in different languages. Surface like tables interacting directly with iPad-like multi touch tablets placed on it. Minority Report like control. Augmented reality. …. It’s even more impressive since everything is backed afterwards by actually existing (if in early stages) technology. There’s a shortened and also an extended version available on youtube:

Speaking of Minority Report. Another great video comes from John Underkoffler; John has been the science adviser for that movie and he does the whole presentation with exactly that technology!

This talk is certainly worth watching, as he makes some very interesting observations (in fact, watching this video triggered this post; thanks Daniel). His final prediction is … ambitious: “I think in 5 years time, when you buy a computer, you’ll get this.”

Is that cool or what?

Second Thoughts 

Well, as they say:

“Prediction is very difficult, especially about the future.” (various)

There’s one thing I don’t like about those predictions. They are (deliberately?) incomplete. They certainly shine in new fields of applications for computers, new degrees of collaboration, new ways of interaction. Like home integration, meeting areas with huge collaboration screens, geo services and augmented reality, or simply navigating and reshaping existing data. But in their aim to show new ways of doing things, they neglect the “old”, conventional demands, demands that won’t go away.

The very fact that these NUI approaches – touch, gestures, even voice – are defined by “the lack of a physical keyboard and/or mouse” (and in case you didn’t notice – NONE of the above videos hat a keyboard in it!) renders them inappropriate for a whole bunch of scenarios. Can you imagine a secretary typing on a virtual keyboard? A call center clerk waving at his screen while he talks to a customer? A banker shooing stock rates up and down? A programmer snipping his code into place? Cool as all that Minority Report and other stuff may seem, I have a hard time imagining anyone whose daily job today requires a keyboard to a substantial degree using some other “device” instead.

In the end we’ll probably see both. NUI approaches are going to spread, new devices targeted at different scenarios simply require different notions of user interaction. But they are not going to replace today’s conventional computers, they are going to be a complement, actually even a necessary one. Another necessary complement is the mutual integration with each other, the internet/cloud, and social platforms, but that’s a different story.

For us developers this will be the actual challenge: developing on conventional machines for devices and environments that have totally different ideas of how an application should look like and interact with its surroundings. Testing is going to be a bitch.

That’s all for now folks,
AJ.NET

Again, I took the upgrade exams, but since they distinguish the topics pretty well (each topic is covered in a separated section during the questionnaire and has to be completed before moving on to the next), the assessments should be valid for single exams as well – even if the single exams may shift emphasis somewhat.

Logistics

The upgrade is available to those who have the MCPD/EA on .NET 2.0. (This implies that there is no direct upgrade from the older MCSD, sorry Daniel.)

There are two exams that cover the ground of 5 single exams:

• Exam 70-568: UPGRADE: Transition your MCPD Enterprise Application Developer Skills to MCPD Enterprise Application Developer 3.5, Part 1
• Exam 70-569: UPGRADE: Transition your MCPD Enterprise Application Developer Skills to MCPD Enterprise Application Developer 3.5, Part 2

You can learn more about these and other upgrades here.

Part 1 includes the following three parts and respective certifications:

• Microsoft Certified Technology Specialist: .NET Framework 3.5, ADO.NET Applications
• Microsoft Certified Technology Specialist: .NET Framework 3.5, ASP.NET Applications
• Microsoft Certified Technology Specialist: .NET Framework 3.5, Windows Forms Applications

Part 2 adds another Technology Specialist part and the “Designing and Developing” part that concludes the MCPD:

• Microsoft Certified Technology Specialist: .NET Framework 3.5, Windows Communication Foundation Applications
• Designing and Developing Enterprise Applications Using the Microsoft .NET Framework 3.5

All of which concludes the “Microsoft Certified Professional Developer: Enterprise Application Developer 3.5”

More information about single exams is available here.

The Exam Parts

While the last upgrade included a major remapping of the certification scheme, this one is merely keeping up with technological changes. The single parts vary in the amount of change they have undergone and the degree to which they can be considered being up-to-date.

• The ASP.NET part was no big surprise, containing additional questions for ASP.NET AJAX. I’ll call that “reasonably up-to-date”.
   
• The WinForms part hasn’t really changed its content at all. Hardly surprising, since WinForms itself haven’t changed. What has changed with .NET 3.5 though, is the introduction of WPF, a whole new UI technology. WPF is completely missing from the 3.5 exams. Hence, seen as Windows development part, this can hardly be called up-to-date.
   
• The ADO.NET part. Well. I would call that one “unreasonably up-to-date”. It is still mainly focused on DataSets in great detail. Now, I can understand that LINQ2SQL doesn’t qualify for an exam with ADO.NET Entity Framework around the corner. But still! DataSets have gone out-off-fashion even before LINQ2SQL and I really had to scratch some layers of dust from that knowledge.
   
• The WCF Part is probably the most advanced. This used to be the “Building Services” exam, covering remoting and WSE, but those topics are completely gone. This exam is solely focused on WCF, and you have to have a solid understanding about hosting options, configuration, and security to pass this exam.
   
• Designing and Developing hasn’t changed in style, but it incorporates current technologies. Unlike the respective other exams, it actually addresses WPF and ADO.NET Entity Framework as technology choices.

Generally speaking, each and every of these exam parts can be solved by people with reasonable experience in the topic. That should cover ASP.NET, WinForms, and “Designing and Developing” quite nicely. Whether you put ADO.NET in that bag is up to you, but dusting the DataSet knowledge shouldn’t be a showstopper anyway. The only part that is more demanding is the WCF part. Be aware that the questions cover WCF capabilities in general; having created one or the other WCF service or reference in Visual Studio and twisted the configuration somewhat is certainly not going to get you through this exam.

Looking Ahead

.NET 4.0 and Visual Studio 2010 have been released weeks ago and usually it takes Microsoft about a year to prepare the next increment of certifications. So the current 3.5 certification is going to stay valuable for quite some time. Still, a little crystal ball gazing might be fun:

In terms of technologies, .NET 4.0 does not only contain brand new stuff, but also incorporates some technologies that have been around for some time, but not formally part of the .NET Framework. Together with the gaps noted above we have the following list of suspects:

• In the ADO.NET area:
  • LINQ 2 SQL (leaving it out so far may have been lack of time rather than a deliberate choice after all)
  • ADO.NET Entity Framework
• In the ASP.NET area:
  • ASP.NET MVC
  • ASP.NET Dynamic Data
  • perhaps jQuery
• In the Windows Development area:
  • WPF
  • Parallel Extensions (PLINQ, TPL)
• In the Designing and Developing area:
• Visual Studio Architecture features
• Testing features

I do hope that we will see a substitution of content in the ADO.NET area. Please Microsoft, no more DataSets! And it wouldn’t be a surprise at all if Microsoft dropped WinForms in favor of WPF, adding a little parallel stuff. ASP.NET on the other hand will certainly grow, as ASP.NET WebForms are not substituted but complemented by the new additions. The new Visual Studio features may result in in just one or the other additional question. But then, they might not, as they are not universally available.

Anyway, this is reasonably manageable for those people trying to keep their certificates up-to-date. For people trying to get their first certification it certainly adds up; but then, they may start with Technology Specialist or the more specific MCPD certificates and build on that.

What parts didn’t I mention? Silverlight is obviously missing from the list. And don’t tell me, it’s not relevant for enterprise development, for it certainly is! But I still can live with that, for it isn’t formally part of the .NET Framework and you have to draw the line somewhere. The same applies to Team Foundation Server.

On the other hand, how can you call yourself a certified “Enterprise Application Developer”, not knowing about TFS or Silverlight?

That’s all for now folks,
AJ.NET

The code can be downloaded here: AmazonSample.zip

After downloading the solution (VS2010!), you need to open the appSettings.config file in the web project and provide your personal Amazon credentials. Afterwards F5 should be all you need.

Calls via the server use a simple cache implementation that stores the XML returned from Amazon in a local directory. This way one can have a more detailed look at the information available from Amazon. This is intended for debugging purposes and to avoid flooding Amazon during development – it is not suitable for production code!

The related blog posts are available here:

• Calling Amazon – Part 1
• Calling Amazon – Part 2

That’s all for now folks,
AJ.NET

1. I’m going to make REST calls, as they are more flexible than SOAP calls.
2. I’ll make the calls to Amazon from the client.

It’s important to note that these decisions depend to a very high degree on the particular service I’m calling. Amazon offering a policy file, the structure of the API allowing to keep the secrets on my server, the fact that Amazon actually offers REST calls in the first place. Any other service might need a completely different approach. (That’s what the last post covered.)

As a reminder, here is the relevant image:

So how about actually implementing that?

The Server Part 

The server has to build and sign the URL for the Amazon call. Implementing that is straight forward. The AmazonApi class maintains the Amazon configuration in the appSettings.config:

The BuildItemSearchRequestUrl method first calls a method to prepare the query parameters, then another method to build a respective URL:

The called methods are equally simple. BuildRequestParams translates the typed query parameters into a dictionary, adding some other necessary parameters along the way. The parameter names can be found in the developer guide:

In order to build the URL I need the SignedRequestHelper class, extracted from Amazon’s REST sample:

This method is made available to the client via a WCF service, but I’ll leave that one out, its straight forward and boilerplate enough.

Calling Amazon from the Client

On the SL client we have a two step process: First, call the server with the filter criteria, and get the prepared URL. Second, make the call to Amazon, using the URL. The first call is no different than any other call to my own server application, no need to elaborate on that. The second one uses the WebClient class to make the REST call:

The ParseItemSearchResponse translates the XML into a respective object structure. Boilerplate, boring, and kind of longish if you do it manually.

View Model Stuff

Now that the details are in place, I “only” need to wire them into the UI.

The calls from the SL client to its housing server application are straight forward. First, call the server with the filter criteria, and get the prepared URL. Second, make the call to Amazon, using the URL. First the bookkeeping:

The BuildItemSearchRequestUrlCall class encapsulates the calls to the BuildItemSearchRequestUrl service operation shown earlier, AmazonClientApi does the same for Amazon and is also shown above.

Now the actual implementation, kind of leapfrogging from one method to the next by way of asynchronous events and lambdas I pass in for that purpose:

That should get the first 10 results from Amazon – and the proof that I can actually make the call:

The ShowAmazonResponseErrors simply iterates over the returned error collection and shows a respective message box. Amazon will return an error if it couldn’t find anything:

I have now solved the basic technical demands, yet the user may be a little more demanding, since…

Employing Paging

… 10 items is usually not sufficient. Hence I need to make more calls, read paging. Paging technically only requires an ItemPage parameter to be set to a value bigger than 1 (the page index is 1-based). However on the view model, some additional questions arise.

First question is whether the subsequent pages should be loaded right away, constantly filling the result grid in the background. This could be done by triggering the next call, once the previous one has returned, until all available results have arrived. Leapfrogging in a loop. Of course, if the user triggered a new search somewhen in-between, I would have to cancel that chain of calls. Or I could let the user trigger the loading explicitly, e.g. with some „load more“ button (which is what I’ll do).

In any case I have to deal with the user changing the filter criteria or interacting with the result, e.g. resorting it. This is obvious for the second case, but even automatically loading all data in chunks takes time.

Therefore I need to distinguish between the first call, and subsequent calls. The first call initiates a new search, replacing any previous search result. Subsequent calls have to use the same filter criteria, just with another page, and the result is appended to the previous ones. Now, if the filter criteria is bound to the UI and used as parameter to the service call, the user might change the filter and then click the „load more“ button (or the automatic loading might kick off at that time). To prevent that I need a copy of my request property. Similarly I need to maintain my result in a separate property, otherwise the call would overwrite any previous result data.

BeginSearchAmazon and EndSearchAmazon now only handle the first call, initiating a new search, and have to be changed accordingly:

The chain for subsequent calls looks similar in structure, but preserves the values in the separate copy properties:

The next image shows the dialog after having loaded 3 pages and in the process of loading the fourth: 

Great? By the way, the details link jumps straight to Amazon, showing the respective book.

Roundup

Whether you are going to call Amazon or some other external service, these two posts should give you some hints on what to take into account form the infrastructure and architectural perspective. On the client you’ll have to look into Silverlight security and cross domain calls, on the server you might run into firewall or proxy authentication issues.

Also the Amazon API with its approach to paging may give you some hints on how to implement paging over larger result sets with Silverlight. While server calls are asynchronous, SL doesn’t provide the option of processing results while they arrive. For a large result set it might take some time to download the data, and the user might notice the time lag. It could be the better user experience to load the data in chunks, as shown here.

One hint at last: Jon Galloway has a good explanation on the rationale behind policy files on the called server, see here.

That’s all for now folks,
AJ.NET

Note: This post will dig into the Amazon API and some general infrastructure questions. Actually implementing this will be the topic of the next post.

The Preconditions

Amazon isn’t exactly forthcoming with its product catalogue API. Starting at http://aws.amazon.com/ points to about any Amazon service offering there is — except the product catalogue API. Well, some bings later, by way of other articles and blog posts, and once you know that the correct name is „Product Advertising API“, you’ll find the entry point. From there it is reasonably well documented.

First thing is to register oneself as developer. This will result in various information, one can pick up on his user profile page:

• The AWS Account ID is the user ID
• A variable number of  pairs of an AWS Access Key ID and the respective AWS Secret Access Key itself. You need one for the REST API.
• A variable number of X.509 certificates, one of which you need to make secure SOAP requests.

Understanding the API

As a quick guide to the documentation: The entry to documentation is here. Under “Documentation Archive” you can pick the latest version of the Getting Started Guide and the Developer Guide.

The logical next step is to find some samples, get them working and understand the details of calling Amazon. There is one sample using the REST API, and one for SOAP using… WSE?. Well. WCF is actually quite new and since almost everyone is still using WSE, why update… . Anyway, the example is simple to migrate – and it doesn’t work at all, could never work actually, since it doesn’t address security.

You can find descriptions on how to get security for SOAP calls working here. I never checked that out, though (I ended up using the REST API, see below), and I couldn’t find any samples using WCF.

The REST API is used by putting parameters in a dictionary, supplementing the user information, and letting a helper class (SignedRequestHelper.cs) produce a URL. The respective request with that URL will return some XML, one has to parse.

You’ll need that helper class; unfortunately it is, again, not that easy to find, as most links will lead you the online test page, not the download. You could even download that page here. But I never found a download for the class by itself and ended up extracting it form the example named above.

The WSDL is also available and can be used to create a WCF client. Even if you employ the REST API, the data classes from the WCF client may help you parsing the returned XML.

The API itself is simple and straight forward. You send some query parameters and you get some result. The query parameters include the operation, and the operation determines the valid set of other parameters. The operation I’m going to be interested in is ItemSearch.

The query parameters also include the ResponseGroup parameter that describes what kind of output I would like. Amazon doesn’t return each and every detail about, say, any book in a book search result. It returns just some major fields like title, detail URL, etc. by default. One could use this information to populate a search result list and load the book details on demand, thus relieving Amazon from doing too much unnecessary work in the first place (and reducing network load). But in cases where more information is needed right away, one can tell Amazon to include other sets of information, like images, in the returned search result.

Another parameter is ItemPage for the partition of results. Amazon returns 10 items with each search request, and no way to change that value. To get the next 10 values, one has to make separate calls for page 2, 3, and so on, 400 at most.

The Infrastructure Question

Now there are a few choices to make (or rather to rule out). We have the REST and the SOAP API at our disposal, and we can make the call from our server, or from our Silverlight client. Note also that the REST call can be split into two independent parts: building the URL (which includes signing), and making the actual call against Amazon. In theory this leads to the following options:

1. SOAP call from the server
2. SOAP call from the Silverlight client
3. REST URL built at the server, call made from the server
4. REST URL built at the server, call made from the SL client
5. REST URL built at the Silverlight client, call made from the SL client

What are the forces restricting these options?

• One restriction is that I‘m not going to send my private Amazon secret key or certificate – my eyes only, signed with the blood of a black cat killed on full moon on the grave of a convicted murderer – to the Silverlight client. It’s not that I don’t trust you… Well, it is, and I don’t. That invalidates option 2 and 5.
• Another possible restriction is the server infrastructure. Depending on proxy or firewall configuration, you cannot call outbound from your server to the internet. In case of a proxy it might be possible, but it‘d take unreasonable effort. That puts at least a question mark behind options 1 and 3.

Regarding proxy authentication: ASP.NET applications (including .asmx services) have the CredentialCache.DefaultNetworkCredentials property to get the current user’s credentials to pass on. WCF services don’t have that option which makes it unreasonably hard to make the subsequent call using the current user’s security context. Tell me if I’m missing something! 

• To make the picture complete: Our SL client is also subject to security restrictions. The called service has to explicitly allow the call from our client, offering a policy file. Fortunately Amazon does that, so this won’t be an issue for now. If you are planning on using other services, make sure to check this out, for this puts a block on the call form the client.

Note: I‘d like to stress the fact that it is the providing service, Amazon in this case, who has to opt in for client calls. There is nothing that can be done on the client side about it. This is quite a common misconception…

The corrected list of options:

1. (SOAP call from the server)
2. SOAP call from the Silverlight client
3. (REST URL built at the server, call made from the server)
4. REST URL built at the server, call made from the SL client
5. REST URL built at the Silverlight client, call made from the SL client

Since the REST API offers client side calls (option 4) and still leaves the option of making server calls (option 3), the SOAP option never came up again. I actually started with server calls.

Calling Amazon from the Server

In this scenario all security related issues are the server’s problem:

The client passes the filter criteria to the server, the server creates a signed URL, invokes the REST call, parses the XML, and returns the result. The call from the server to Amazon may have to go through proxies, firewalls, or other intermediaries.

Doing the work on the server had certain advantages: It was easy to implement, I could use unit tests, I added some persistent caching in files (actually to avoid flooding Amazon with my debug calls, but caching would of course improve multiple clients). Also in this scenario the server does the job of parsing the returned XML into decent entities, and only those are returned to the client, which may be a factor depending in the WAN structure.

Calling Amazon from the Client

In this scenario the client still calls the server, but rather than making the call to Amazon, the server just builds the signed URL and hands it back to the client. The client calls Amazon and it also has to parse the resulting XML.

Calling the service from the client is only possible if the called serve permits it (as Amazon does). And if it does, it is more fragile as one cannot always foresee under which circumstance the code will run. If something goes wrong, the chances of getting diagnostics information are bad.

On the other hand the proxy issue is nicely circumvented and in case you have to support different authentication schemes, say openID, you may be better off telling the user that you cannot record his credential information on the server if the server never sees it.

Initially I implemented the client side call more out of curiosity, to evaluate the implications. But when I eventually did run into the proxy issue, I only had to switch my view model to get it working again.

Calling What from Where?

I’d like to stress that point: The decision whether to call the external service from my own server or from the client is extremely depended on very different influences: Infrastructure, security related, available API, did the service opt in to client calls, etc.. The decision may consequently be completely different in other cases. It may even be the case that there is no simple solution. For example, had Amazon neglected to opt in for client calls (the policy file), I would have been forced to make the call form the server. Had I then run into the proxy or some other firewall issue I would have had some hard tasks to face.

There may be some workarounds, like falling back to .asmx for the credentials or some browser script workaround – but none is especially nice.

Anyway, since in my case I have a working approach, I can now set out to actually implementing it. Next post…

That’s all for now folks,
AJ.NET

Starting a little internet research will reveal two variations:

• Putting validation code into property setters, throwing exceptions (covered by Jesse Liberty).
• Decorating properties with validation attributes, either checked by the data grid, or manually, again eventually based on exceptions (see MSDN).

OK, obviously you don’t need me to repeat what has been described by others in sufficient detail.

However, neither the property setter approach, nor validation attributes work well with code generated classes (which are employed quite regularly with SL). Would property changed event handler solve that? How do value converters fit into the picture? What about mutual dependent properties? Null values? Actually, there’s no overall concept that would help answer these questions.

Note: WCF RIA Services (see also here; formerly known as RIA Services) addresses the „generated code“ issue by copying the validation from the server to the client. However I refrain from using WCF Services in this learning experience, since the aim is to understand what’s going on under the hood. Additionally I can easily envision cases in which I may want different (i.e. more restricting) validations on the frontend than on the server.

Defining “Validation”

Let’s clarify what we are talking about. Validation (more to the point, input validation) deals with checking some user input, however it is made, against, let’s call it syntactical and semantical rules. Syntactical rules include all that is necessary to satisfy the type of the backing data, e.g. ensuring the string actually is a valid time (provided the property is of type DateTime). Semantical rules deal with additional restrictions, usually business related, say whether some value is in a certain range.

Note: Validation always happens after the user has made his input, making sure the input was correct. The alternative approach (though not always feasible) is to let him only make valid input in the first place, e.g. with date pickers, numeric up/down, or masked edit boxes. This renders at least the need for syntactic validation obsolete, semantic validation may or may not be covered. Still, it may provide the better user experience.

Syntactic rules are usually enforced implicitly during the actual conversion. The need for a conversion depends on what data the control actually provides (e.g. a date picker control supplies a DateTime ready to use, no need for validation in this case). In other cases syntactic validation is covered by the databinding mechanism of Silverlight.

Semantical rules are up to the developer.

There’s another difference between syntactic and semantic validation: Syntactic validation has to occur before changing the data (since it’s a precondition to a successful type conversion), thus it is also tied to the UI and the databinding process. Semantic validation on the other hand can also happen after the data has already been changed. And it doesn’t have to happen in the UI either. Matter of fact, some validations may not even be possible in the UI, but have to be enforced by the business logic or even the database. Heck, they may happen out-of-band at any time, say by asking some other system asynchronously.

Note: Within my Silverlight UI I don’t care who actually does the validation, but the fact that it may happen outside of the usual sequence (databinding, triggered by some user input) is important.

Typical use cases for validation include:

syntactic validation
A) ensuring a string represents a certain type (numeric, date,…)

semantic validation
B) ensuring some input, i.e. required fields
C) ensuring a certain text length
D) ensuring a value matches some criterion (range check, regular expression, date in the future, etc.)

special cases
E) mutually dependent fields (mutual exclusion, value dependencies, etc.)
F) distinguishing between input and no input (i.e. null values)

Validation “mechanics” in SL

Validation is tied to data binding and to get it working the binding properties have to be set respectively:

Now let’s take a closer look at the data binding process. Here’s a (somewhat simplified) sequence diagram of the relevant code for validation, after a control changed its value. Basically it all depends on exceptions:

(Note: this is not entirely correct regarding the exception handling, but is conveys the actual meaning better.)

Let’s go over the locations I numbered:

1: The value converter you could declare in the binding expression is called – yet it does not take part in the validation handling. Meaning, any exception thrown here won’t appear as validation error, but as actual application error. This renders value converters useless in cases that need syntactic validation (like use case A).

2: Right after the value converter, a try marks the area in which every exception will be treated as validation error.

Personally I don’t understand why the value converters have been left out of this area (seems to be a conscious decision). They would have offered some easy to use validation mechanism…

3: The type converter of the property’s type is called, which translates the value from the type passed in by the control (a textbox passes a string, other controls may pass in other types) to the target type, say Int32. Exceptions thrown here will show up as something like „Input string was not in a correct format.“, and of course the property retains its original value. Hence the type converter does syntactic validation (use case A). There’s however no feasibly way to customize the conversion, like what a value converter could do.

Personally I don’t understand why the value converters… . Ah, said that already, didn’t I?

4: The property value is set, and in turn ticks off any validation code put in the property setter for semantic validation. The samples on the internet usually have the validation code before actually changing the property value, thus the property still retains its original value in case of a validation error, read exception. Anyway, use cases B (ignoring null values for the moment), C, and D are covered here.

5: The setter will also raise the PropertyChangedEvent, and in turn any exception thrown in a respective handler will also take part in the validation handling. However if we get that far, the property now has the invalid data already set. Still, this may be another location for use cases B, C, and D.

6: Any validation error is announced to the control via the BindingValidationError event. Many controls have an error state and will show a red border and an error tooltip. Alternatively there is a ValidationSummary control that takes care of presenting the feedback:

7: Finally there’s some missing parts in the sequence diagram:

• The databinding mechanism doesn’t check the validation attributes, so who does? The DataGrid is the only control that actively validates the row in edit mode, but one can trigger the validation himself.
  If done before writing the data, this approach bypasses any value and type conversion (another reason why value converters have no part in validation). Theoretically one could validate a property this way without setting the values at all. However due to the type conversion issue, the only sensible approach is validate afterwards (which is what the DataGrid does), and let the data binding mechanism care about conversions. This way one could trigger that validation generically in the PropertyChanged event handler.
• Nobody cares about errors (exceptions) in property setters and event handlers (point 4 and 5) outside of this sequence. Meaning any other code manipulation the properties and causing an exception will tear the whole application down.

As a corollary: Validation works one binding at a time. This implies that mutually dependent properties (use case E) are not part of the equation. If property X caused an error that may be fixed by changing property Y, SL doesn’t help. One can work around this (by a combination of UI design and simulated PropertyChanged events), but it’s ugly work.

I left out use case F (null values) so far: Data intensive applications may have to distinguish between no entry (empty string, null value) and the 0-value for the data type (e.g. „0“, „00:00“). The data type of the property would be a nullable value type, e.g. Nullable<Int32> or Nullable<DateTime>. The sad part: Type converters don’t handle null values, neither does some other part of the data binding mechanism. nullable types are treated like their non-nullable counterpart, thus no empty strings are allowed, worse, an empty string even usually causes an exception, thus validation error. The best way to solve this is a separate property of type string that handles null and 0 representation and does the type conversion. However it takes some effort to keep those two properties in sync and propagate PropertyChanged events of the other property.

Consequences & Conclusions

Now that the input validation mechanism is understood (I hope), I can draw some conclusions:

To set or not to set…

There is some inconsistency between whether an invalid value will actually be set in the data property or not: Type converter (syntactic validation) and property setter issues (semantic validation) generally leave the value as is. Validations in property changed notifications – including validation attributes if implemented that way – (also semantic validations) will set the property to the invalid value and notify only afterwards. This in turn means that our view model logic has to take the validity of the data into account, i.e. a save button should check some kind of IsValid property on the model.

To throw or not to throw…

In SL3 the validation mechanism relies on exceptions. Adverse effect is that the first validation error hides subsequent errors, which may obscure the feedback for the user. Secondly this code is also present when I set the properties from code, with no data binding infrastructure readily available to catch the exceptions. Hence I have to be very careful, or some innocent code may take my whole application down.

To depend or not to depend…

Validation in SL3 doesn’t cover mutually dependent properties properly. Some nasty stunts and compromises may get you a working solution, but it hurts to write that code (been there, done that!).

Anyway, with SL4 around the corner (see below) I‘d refrain from putting too much effort into this issue right now.

To convert or not to convert…

As said before, value converters do not exactly work with validation, at least if the conversion itself may fail. Thus I’d refrain from using them altogether in case validation is also needed. I’d rather put the conversion logic into „shadow properties“, i.e. properties that replace/complement the original property, changing only the property type (usually to string). Another reason for these shadow properties would be the null/0 issue mentioned above.

To keep these two properties connected and synchronized takes some effort. For example if one property changes, the other should raise the PropertyChanged event as well.

Silverlight 4

With SL4 we’ll get IDataErrorInfo and INotifyDataErrorInfo for asynchronous validation, as announced by Tim Heuer, and described in more detail by Fredrik Normén.

Contrary to SL3 this isn’t tied to exceptions any more. This will change the picture completely:

• The first validation error won’t necessarily prevent other validations
• Other code setting the value (outside of databinding) won’t have to deal with exceptions (and still partake in validation).
• Asynchronous validation is covered.
• Mutual dependencies can be addressed.

The one notable gap is the still missing MetadataTypAttribute that would enable the usage of validation attributes for code generated classes. The only other aspect not covered is the null values and value converter issue. But strictly speaking this is more an issue of type conversion than of the validation itself. (Still a pesky issue.)

If SL4 changes nearly everything, then why this post in the first place? Well, apart from the fact that existing code doesn’t migrate itself, while the contents presented here are less relevant, they are still valid.

Final Verdict

With SL3 I’d say about 80% of my validation demands are covered. But as always, the remaining 20% don’t appear all that often, but if they do, it hurts.

Looking ahead, SL4 will provide a better foundation for validation, and solve some issues — yet it still won’t cover every aspect. Still, having looked at SL4 may give some hints on what to implement today for SL3 (and what not), with a clean migration path to SL4.

That’s all for now folks,
AJ.NET

There is a lot of information about VSM available, e.g. a quick introduction at silverlight.net, and when I first started to tackle VSM I read it all and then some (felt that way, anyway). Still, my first experiments with VSM failed miserably—and it did so because of a lack of understanding. Because the one main issue for me was that all the articles and screencasts explained what the VSM does and what great effects one (well, someone else) could achieve with it, yet all with the emphasis on ‚what‘ (and usually all at once), not on ‚how‘ (in small digestible chunks).

So, if you have looked into VSM and didn’t quite get it, but only just, then this post may be for you. First I‘m going to dive into some code, afterwards I’ll try to offer a few hints that should help you getting started with VSM.

The Crash Course

Controls have states (like normal, pressed, focused, for a button); states are represented by VSM as Visual States, organized in distinct State Groups. State Groups separate mutually independent Visual States (e.g. pressed state or mouse over are independent of the focus state). Silverlight allows to define these states in templates, along with State Transitions that define how the state change is to happen (e.g. instant change or some animation).

Silverlight also provides an attribute, namely TemplateVisualStateAttribute, to declare the supported Visual States and Groups on a control. Keep in mind however that this is merely for tool support and perhaps documentation. At runtime, the presence or absence of these attributes is of no consequence at all.

The Sample

OK, let’s see some code. I’ll build on the image button from my last post. It should support three different images, as well as a focused rectangle. (I’ll leave out the text though. I don’t need it and it would complicate matters for this post without gain.) The button class already defines the Visual States and Groups, I’ll stick with that.

First I extended the image button control class to support three dependency properties, namely NormalImage, HooverImage, and DisabledImage. (I could have added a ClickedImage, but I‘ll solve that otherwise.) To make a long story short, here is the custom class, defining the necessary dependency properties:

The button inherits the VSM attributes from its base class, thus I don’t have to reiterate them here.

Having done that I can already use these properties to set them in XAML:

Of course it still uses only the one image, I gave it last time. So, the next step is to extend the template with images, and other parts, to accommodate the Visual Styles. If I can live with XAML, I could do this using Visual Studio 2008. To do it in a design view I need Visual Studio 2010 or Blend.

In any case, this is conventional designing, it’s not yet time to look in blend at the “States” panel in Blend! And when it comes to this, VS2010 is also out of the game (at least in beta 2).

The resulting XAML looks somewhat like that:

Note that I used a Grid to stack the images on top of each other. Note also that the default settings for all parts are compliant with my „normal“ button state, i.e. the second and third image are invisible, so is the focus rectangle.

Now that my control contains all the primitives I need, it’s time to enter VSM. To prepare for that, I manually provided the State Groups and Visual States. This ensures that I get all states (Blend would only add the ones it manipulates, and since the normal state is going to be empty, it would always be missing), and in the order I prefer.

Now is the time to enter blend, select the button, then the current template, and have a look at the “States” pane.

Note that the “States” pane contains the State Groups with their respective Visual States. Blend gets this information from the TemplateVisualStateAttribute on the class, but also includes additional states and groups it finds in the template XAML. Additionally there is a “pseudo-state” named “Base”, which is simply the “state” in which the control is without putting it in a distinct state.

Now I went ahead, selected the state in question in Blend and changed the controls to match my design. Since I had the desired design figured out before I started the VSM – up to which properties to change for a transition – this was as simple as can be. For the mouse over state:

Note how Blend shows the design area with a red border and a “recording mode” sign. Every change to the template is now recorded as state change for the selected state, mouse over in this case. (You could switch recording off by clicking on the red dot in the upper left, and manipulate the properties ordinarily; yet selecting another state will switch it back on, so this is OK for some quick fixes, but too error prone for general editing.)
Note also that the “Objects” pane shows not only the controls, but marks those affected by the currently manipulated state with a red dot and puts the manipulated properties beneath. In case you accidentally manipulated the wrong property, you should remove this entry, rather than simply change it back, otherwise the (trivial) transition will remain in the XAML.

Just setting the visibility of two images results in some verbose and (at first sight) rather confusing XAML:

The disabled state looks similar. The click state is represented by the hover image which is moved slightly off center to achieve the click effect (“Properties” pane, “Transform”).

And here’s the resulting button in action, showing normal, hover, disabled, and clicked state:

Lessons Learned

What I just presented was a fast forward replay of employing the VSM. Using VSM minimalistic to the extreme actually, since I have left out quite a bit of VSM functionality, most notably transitions with animations. Still, I have applied some guidelines I learned to value when using VSM, that I’d like to point out.

So, here are some of the twists that made VSM useful to (rather by) me… (some learned the hard way).

Hint 0 (The most general hint): States need careful planning.

If you don’t know yet what the control should look like in the various states, you should shy away from the “States” pane in Blend. Start with conventionally designing the control, It may even help to design a separate control template per state, and merge them only after the design has reached a stable state.

Hint 1: Don’t look at existing controls.

It’s tempting to look at the existing templates, with Blend the XAML is only a mouse click away. Don’t. The button template has ~100 LOC, and I’ve seen others with more than 300 LOC. And what’s more, they are fully styled, meaning that they employed probably all the features, caring for sophisticated visual effects but not exactly for the poor developer trying to deduce the workings from looking at the XAML.

Hint 2: Start simple.

Many samples quickly jump at animations used for transitions, easing functions, and slicing French Fries. For me one key to understanding VSM was to stick to the minimum at first. States. Transitions only as simple as possible. Period.

Hint 3: VSM is not about creating states. It is about styling them.

My initial thinking was „I have a button with a normal image; in disabled mode I need to have a disabled image…“. This led to all kinds of mind twists, like „how do I create an image control during a state change?“ „Should I rather replace the image URL of the existing control, and how?“, and others. It was a crucial part of understanding when I realized that I do not have a button with one image in one state and another image in another state. What I have is a button with three images in all states, as presented above. The difference between the states is merely which of these images is visible and which is not.

Hint 4: When designing a new control, avoid the VSM “States” pane for quite some time.

There is one pitfall I managed to hit several times in the beginning. I started Blend, selected the particular state I‘m interested in, and tried to design my control for that state. This is futile, because Blend does not actually design the control (as in setting property values), rather it designs the transitions to these values. (You could switch off recoding mode, but Blend really insists on switching it on again and again and again.)

Therefore I generally design my control „conventionally“. I.e. I place the normal image in a grid and style it; then I make it invisible (kind of a manual state change) and do the same with the next image; and so forth for all states. Only when I‘m done with this I allow myself to even look at the VSM support in Blend.

Hint 5: The visual state for the normal state is always there. And always empty.

Worse, you’ll have to include it manually in XAML, since Blend doesn’t put it there…

„Normal state“ is the default state of the state group. Each state group has one; it doesn’t have be be named „normal“, but it has to exist. This is the state in which the control is by default, after initially displaying the control, and before VSM has even touched it. The one that is denoted as “Base” in the “States” pane.

The „normal“ state has to be declared, because otherwise the control will not be drawn correctly after it has been in a different state, say normal –> hover –> normal. And it has to be empty because otherwise the control would show up in an undefined state, at least according to VSM, which can never again be reached, once the control was in a different state. This would lead to all kinds of inconsistencies.

Lemma: All controls in the template initially have property values compliant with the normal state. In the image button example: The normal image is visible, the other images invisible.

Hint 6: VSM is not about designing states. It’s about designing differences between the state in question and the normal state.

Suppose I have the control designed the „conventional“ way with the looks of the normal state; I also have yet invisible controls for the other states. Now is the time to enter Blend and the “State” pane. Choose the state in question, e.g. mouse over, and manipulate exactly those properties that constitute the difference between normal state and mouse over state. I.e. set the normal image to invisible and the hover image to visible. Blend will record the respective transitions.

It’s always this difference, always normal state vs. the state in question. Only if you achieved the first belt in using VSM and signed the no-liability warrant should you go ahead and attack transitions between specific states, for complexity will explode.

Hint 7: State groups are mutually independent. And the same is mandatory for the state differences.

Never let different state groups manipulate the same properties. For example the button addresses common states and focus states independently. It would not work to implement the focused state by setting the hover image visible, as this would collide with the mouse over state and eventually result in undefined behavior. The focused state could show a focus rectangle. Or it could actually even manipulate the hover image, as long as it is not the visibility property used by the mouse over. (Whether that makes is a different question, though.)

Hint 8: Visual states are not put in stone.

Controls usually have visual states defined via attributes. However this is just some information, used by some tools (such as Blend), but of no consequence otherwise. VisualStateManager.GoToState is what triggers a transition, and it may or may not be called from the control itself. The visual states and groups defined in the template are merely backing information used at runtime. Should the need arise, I could define a new state group, say „Age“, with two visual states „YoungAge“ and „OldAge“ in XAML. Then I could go ahead and call the VSM from the code behind file of my page class to change the state. And after 5 minutes of inactivity my button could grow a beard.

Wrap-Up

So far the hints. But what about more complex demands? I have barely touched the eye catching features at all.

In my opinion, what I just presented will cover the first steps and provide a sound understanding of the core VSM principles. Once this level of understanding VSM is mastered, one can go ahead and explore other areas.

And there certainly are “other areas”. I already mentioned state specific transitions; animated transitions are another topic. If you need an example of what’s possible have a look at this online sample. This is VSM in action, admittedly complemented with some code, but surprisingly little. (You can dig into it starting Blend and opening the sample project „ColorSwatchSL“.)

Some other useful links:

• VSM designing tips: http://blogs.msdn.com/expression/archive/2009/10/02/visual-state-manager-tips-for-design-and-authoring.aspx
• Additional styling tips (for different controls): http://blogs.msdn.com/expression/pages/styling-tips.aspx
• A more complex example: http://scorbs.com/2008/06/11/parts-states-model-with-visualstatemanager-part-1-of/

And from now on it’s no longer a lack of understanding that keeps me from doing things. It’s my incompetence as designer…

That’s all for now folks,
AJ.NET

My use case: I wanted/needed a simple image button, one that simply takes the image as a property, rather than having to manipulate the content for every button anew. So what do I need?

1. A derived class.
2. A dependency property for the image URL.
3. A new default template.

Deriving a SL control is just a matter of clicking add/new item in the solution explorer, and choosing „Silverlight Templated Control“. This will actually create 2 things (and address the default template requirement as well…):

1. A class derived form Control, placed in a .cs file in the folder I used to create the new item.
2. A XAML file named Themes/Generic.xaml is created (or extended if it already exists) and contains a style and template for the new control.

Now, the implied behavior is that a custom control sets its DefaultStyleKey property to a type (usually its own). At runtime SL will determine the default style of a control by using this type’s assembly to read the Themes/Generic.xaml content and pick the style that has the type as target type.

Note how the style in Themes/Generic.xaml uses an XML namespace to map the class name to a C# namespace:

Note: One consequence of this is that all styles of all custom controls in an assembly will end up in the same generic.xaml file. This is usually not an issue, even if implementation and style/template reside in relatively remote files. However, if the assembly grows to accommodate a bigger number of controls, it might make sense to put the control specific resources into separate .xaml files right beside the implementation. Loading the template from a .xaml resource is no big deal, all you need is GetManifestResourceStream and XamlReader.Load, in more detail here.

The next step is to change the base class — I want to extend the Button, not write it completely anew — and to provide the dependency property for the image. Having a peek at the Source property of the Image control tells me that ImageSource is the adequate type.

Now, let’s customize the appearance. Unfortunately Blend cannot deduce the dependency between the control an the style in Themes/Generic.xaml. Therefore its easier to create an instance of the ImageButton, assign a temporary style with template in blend, placed it into the same page’s XAML:

and the respective button:

Of course I need to have the respective image…

I can now use Blend to work on the template (assigned within the style):

This will change to editing context to the template rather than the control:

I changed it to include the image, placed it beside the text. (Actually, placed beside what I choose to have in the content property, using a content presenter.)

Now I want the image control to show what I have in the NormalImage property I just wrote. Blend is aware of the type of my class, so I can bind the Image.Source property using a template binding to the property of my class.

and clicking it:

The temporary style with template finally looks like this:

Template bindings can be used to bind against existing properties (of appropriate type), as well as any new property I choose to provide. Actually for a complete implementation I would probably have to map alignments, and other properties accordingly to parts of my template, to provide full customizability for my control.

Now that I’m done designing my button, I can save it, copy the resulting template into the default style for my control in Themes/Generic.xaml, recompile – and then just use it:

Just an image, no text; and at runtime:

Alright, that’s the basics of a custom control. Essentially what I’ve done is

• replacing the default template with one that includes an image
• providing a dependency property for the image, actually nothing more than a mirror of the respective property of the image in my template.

This is all very boilerplate on one hand, yet extremely flexible at the same time.

Now, the image of the button does not „feel“ very buttonish, i.e. it does not reflect mouse over, disabled state, or clicked state. This is the domain of VSM. Next post…

That’s all for now folks,
AJ.NET

And since controls are such an ingrained feature of SL, I can safely refer to others for the bread-and-butter explanations and focus on the key points afterwards. (Still, is post is going to be about fairly basic stuff.)

The basics…

• Control Customization (MSDN) does a good job of naming the core concepts, namely Style, Template, Parts, States and State Groups.
• Many Softies blogs about SL, I found Dan Wahlin particularly valuable (and he’ll point you to others.)
• Various hints can be found on silverlight.net and silverlightshow.net, e.g. about Styles and Templates.

Done

OK, this would be a short post, if that where indeed it.

Actually this is how I started. Using controls the way I did in WinForms or ASP.NET, setting properties, layouting, styling. It is all so common and natural, I didn’t spend much thought on controls per se at first.

However, treating SL controls like WinForms controls doesn’t do them credit and neglects quite a bit of the potential. Actually, in my opinion, the SL control system is one step ahead of all previous systems.

The Key Point About SL Controls…

… is this: Silverlight clearly and cleanly distinguishes and separates behavior (control type), shape (control template), and style (control style).

And the glue holding this triumvirate together is the properties, more accurately dependency properties. The following image is an attempt to describe this relationship:

• Controls define their properties, and only the properties defined in a control are accessible for manipulation via styles or to be referenced by templates.
• Templates make up the shape of the control, by assembling Parts into a new compound representation.
• And styles set property values that affect either the behavior or are referenced by the template to affect the appearance or content of a Part.

Add the fact that properties may be subject to databinding and you get a very concise system.

In More Detail…

Control…

SL controls come as classes (surprise!), but their job is only to define behavior (unlike WinForms or WebForms controls that also do the rendering). E.g. a button control essentially does three things:

• It defines behavior. I.e. it reacts on mouse over and mouse click to render a hover effect or trigger the click event.
• It defines (dependency) properties. The properties offer a way of controlling the behavior and appearance – if the control code or the template chose to refer to them.
• It triggers the rendering process, deferring it to the template(s). Usually controls bring their own default template within the resources, many let the template be replaced either partly (usually by setting a Content property), or completely by setting some Template property.

Style…

Styles are essentially a way to set a couple of properties of a control. Period.

To be more elaborate, Styles don’t care whether those values actually style the control in the sense of designing it. For example I can set the selection mode of a data grid, which affects the behavior rather than the design. Styles can set virtually any property of any control, including the one I haven’t even written yet. Styles are also not restricted to certain (and in these cases typically simple) types; leveraging the power of XAML, styles can assign virtually any value of any type.

Lemma: Styles can only manipulate properties. Meaning if I need a button with an image I cannot simply set the image via a style, because the standard button control simply does not have a respective property. I need to derive a control and define a respective dependency property.

Since those properties are class specific, styles are consequently type specific (contrary to CSS), and have to unequivocally state that type. Hence one cannot define a style (say setting font type and size) and apply it to a textbox as well as to a link control. To one who is used to CSS this may come as a restriction.

Template…

Basically a template is the XAML representation of the Parts that together constitute the visual appearance of a control. This can be a mixture of primitives, but it could also incorporate other controls. Also there is some degree of “behavior” in the sense that this may include animations, visual state manager information, and other dynamic content. Depending on the control, a template can be partly set via some content property or completely via some template property. Some controls may offer more than one template, e.g. for header and cells in a data grid. Templates can refer to properties, by way of TemplateBinding, and those properties can afterwards be customized via styles.

Depending on the control (which carries the logic to “activate” them), templates can offer a wide spectrum of extensibility. As an example consider the following textbox, using an alternate template:

This template adds a small glyph to the textbox and sets its tooltip to the textbox content. And this is the result:

This is not bullet proof (changing the text won’t update the tooltip), but it should give a first hint on the potential of customization via templates.

Add visual state manager and states for mouse, selection, validation errors, put animations on top, and you get an unprecedented customizability of even the most basic controls. The down side is the unprecedented complexity, for example the template for the date picker control (extracted via blend) has about 150 LOC, and I have seen other with over 300 LOC.   

Some Conventions…

SL controls don’t adhere to some global style, i.e. it is not possible to somehow state that every textbox should use a certain font within a given application (this is one of the new SL4 features, though). To make my default style assignments easier I like to have standard styles named after class (i.e. "Textbox” – not “DefaultTextbox” or “TextboxStyle”, or some other convention). One could replace the standard classes with derived classes or use some other generic mechanism, yet so far Blend and manual assignment worked fine for me.

Regarding templates, I generally don’t set the Template property within the page XAML. Rather I like to assign the template indirectly via a style. This way I won’t have to think about setting both properties, style and template, and worry about dependencies. Applied to the example above:

And the respective style which also sets the property:

If styles and templates are that powerful one could assume that the need to deriving controls was less prevalent. Matter of fact I do derive controls as often as before, only the reasons have changed: Mostly I derive controls to define properties that I need within custom templates and styles.

And I thought this might be a shorter post…. Anyway, the next post will lay out some additional basic stuff before I’m going to tackle derived controls and VSM.

That’s all for now folks,
AJ.NET

Matter of fact, there are two incarnations of MVVM. In order to distinguish them, let’s call them MVVM/PM and MVVM/SC respectively.

As a quick recap: The relevant difference between the Presentation Model pattern and the Supervising Controller Patter (and the MVVM derivates respectively) is whether the View gets access to the Model’s data model or not. The patterns are very similar, but they differ in this one important aspect.

See my preparatory posts for details. Another descriptions of the two patterns with the same distinction can be found in the Composite Application Guidance for WPF and Silverlight documentation.

One could argue that this difference is superficial, even that Supervising Controller is actually a special case of Presentation Model (if you promote Fowlers exception for read-only data to the be the norm). In fact, when I discussed this whole topic with other people, about half of them fell into the camp that didn’t care at all and rather took the pragmatic approach, like “Who cares, I use MVVM the way it suites me, no matter what some useless theoretician thinks I should be doing.”. But even they didn’t argue the fact that there actually is something wrong with the theory.

The Problem with the Theory…

Theoretical musings aside – and presuming we agree that there are two different MVVM patterns – why is that a problem?

Well, I see several problems:

First, if someone tells you, he is using MVVM, he communicates something.

This is the most important achievement of the GoF with their groundbreaking work on patterns: Simply by providing a catalogue of abstract relationships of software artifacts and naming them, they established a vocabulary. This vocabulary is a means of communication, a way of talking about abstract concepts. What this has done to foster knowledge reuse, software quality, and documentation aspects can hardly be underestimated.

However, having a term with contradictory meanings will destroy that means perhaps more severely than not having a respective term in the first place.

Second, having a solid theoretical foundation, a clear concept, leads to a better understanding in general.

It is the concept that guides me if I have to face a situation that is not really addressed by the current implementation. And implementation has to follow the concept, not vice versa.

If I know what the theoretical foundation is, it can help me answering certain questions. Or it can tell me clearly that a particular topic isn’t covered by this pattern, hence I will know that I’ll have to look elsewhere. Without a solid foundation I won’t get any guidance, I would have to solve every problem on my own, and I would be in danger of violating principles that I’m not even aware of.

Third, on the practical side, we need to clarify the theoretical foundation to tackle the actual issue: Neither approach does actually help by itself.

As an example take validation. Validation in SL3 works by putting validation code into property setters or applying validation attributes to them. However, neither option is feasible with the code generated service proxy classes. Alternative approaches I found, like attaching code to the property changed event, are not very satisfying to say the least. Other demands include thoughts about generic caching or offline support.

• MVVM/PM solves this issue quite easily within the PM. But having to replicate the data model leads to a maintenance nightmare. And (at least in the Silverlight area) it’s hardly employed, anyway.
• MVVM/SC (the usually employed pattern because it works very well with the tooling) simply stops short of answering questions regarding validation.

Actually it has been the issue of validation and thoughts about caching and offline support that triggered all these thoughts and sent me back to the drawing board, investigating the concept, and trying to find out what the theory tells me. Well, I found two different MVVMs…

Verdict

So, I did wreck MVVM, didn’t I? Sorry. Couldn’t be helped.

In my opinion, the only solution for this mess is a redefinition of the term MVVM. This in inevitable, I’m just not sure how this will happen. The concept could evolve and form a better theoretical foundation. But then, people could also continue to neglect the theory and come up with “pragmatic solutions” – meaning that MVVM becomes a fancy umbrella term for anything that looks remotely like PM or SC.

Personally, I would certainly prefer a sound theoretical foundation, probably one that emancipates MVVM from its two parents. Actually I would go where the tooling goes (see Dan’s post), because this is what people will do in the long run, and with good reason. Thus I would start with MVVM/SC as MVVM. From there I would either extend MVVM or complement it with other patterns to address validation, caching, synchronization for offline applications, whatever.

Yes, this would imply redefining MVVM to stem from SC rather than PM, and to accept VM as misnomer for the sake of continuity. And additional concepts to fill the gaps, that may yet have to evolve. However, this is what people think they are doing, anyway. It’s simpler to adjust the theory than people’s perceptions.

One important note: I deliberately ignored WPF! While MVVM started there and was reused with SL, it may be the case that the special demands of an SL application may cause MVVM to evolve differently for the two technologies. (It may not. Still, I’d like to keep that door open.)

So far my criticism. I’m aware that I just redefined (or rather dumped?) a well-established term, and I’m not someone like Fowler who does that on a daily basis. So I would be interested in opinions. Just tell me if I went over the top.

That’s all for now folks,
AJ.NET

The Supervising Controller pattern (previously known as Supervising Presenter pattern) goes back to Fowler ([Fowler-SC]). What makes it relevant for Silverlight development is that it is the usually employed pattern.

Similarly to the Presentation Model (PM) pattern, the Supervising Controller pattern essentially contains three parts: View, Model, and Supervising Controller (SC). The SC pattern provides “the ability to easily map between the view and model, often using some kind of Data Binding”. The mediator between the View and the Model is the SC: “A Supervising Controller has two primary responsibilities: input response and partial view/model synchronization.” ([Fowler-SC])

While the SC pattern is very similar to the PM pattern, the relationship and responsibilities of the parts are somewhat different. For an SL application it looks like this:

Compared with the PM pattern, the difference is seemingly minor: The SC lacks the responsibility to establish a data model, rather there is an additional arrow from View to Model. However, this is actually (at least in comparison to the PM pattern) the defining characteristic of the SC pattern: The View interacts directly with the Model (contrary to the PM pattern). The SC acts only as a mediator, once the connection between View and Model is established, the SC is no longer involved.

The SC has the following responsibilities (leaving aside view entities, as they constitute only a minor part of the data model):

• Handling the controller logic part
• Talking to the service

Obviously the SC as less responsibilities than the Presentation Model, resulting in less complexity, and better separation of concern. On the other hand it is also less flexible and thus less applicable in complex cases.

The Relation to MVVM

Matter if fact, this is the way MVVM for Silverlight is usually presented: If you find a more recent description of Silverlight with MVVM (which is curiously not that easy), it is probably similar to my own introduction. Dan Wahlin has recently written about the topic, focusing on SL, and it’s probably the most concise introduction to MVVM with SL I’ve found so far. And again, his code snippets and text descriptions clearly follow the SC pattern. (Curiously his images follow the PM pattern, but since he never names it and takes a rather pragmatic approach anyway, I’ll leave it at that. )

Notwithstanding the verdict: From a purely formalistic point of view, this is simply wrong! If MVVM is defined as a derivate of the PM pattern (as it was originally introduced), it cannot follow the SC patter at the same time. Both patterns follow contradictory approaches regarding the data model and the resulting ViewModel responsibilities. Actually it is even worse: Since it is not even a responsibility of the SC to define a data model in the first place, the term ViewModel is grossly misleading.

However, given that the Model for SL applications is usually code generated and supports everything that is needed for databinding, the beauty of this approach is its ease of use. At first. (In other words, stay tuned…)

That’s all for now folks,
AJ.NET

The Presentation Model pattern goes back to Fowler ([Fowler-PM]). What makes it relevant for Silverlight development is that it is often cited as the underlying pattern for the Model-View-ViewModel pattern.

Essentially the Presentation Model pattern contains View, Presentation Model (PM) and Model. And it actually comes in two flavors: “An important implementation detail of Presentation Model is whether the View should reference the Presentation Model or the Presentation Model should reference the View. Both implementations provide pros and cons.” ([Fowler-PM])

Interesting (but irrelevant for this post): “A Presentation Model that references a view generally maintains the synchronization code in the Presentation Model. […] The views implement interfaces allowing for easy stubbing when testing the Presentation Model.” 
This leads straight to the Passive View incarnation of the MVP pattern, also supported by the WCSF. The other incarnation of MVP is the Supervising Controller (which will be the topic of another post). The pattern ecosystem certainly has some relations…

Now, the View referencing the PM (Fowler explicitly mentions databinding) leads to MVVM/PM. Given Fowlers rejection of DTO’s its fair to assume that he had a Model constituted as BO’s in mind, with the BO’s talking directly to the next layer. However, I’ll skip that and stay with DTO’s as Model. Personal opinions aside, the common case with SL applications is using code generated service proxies, especially the service contract, as Model.

Hence the relationship of the participants in an SL application would look like this:

Fowler states it himself: “The essence of a Presentation Model is of a fully self-contained class that represents all the data and behavior of the UI window,” ([Fowler-PM]), meaning that the PM constitutes its own data model, and the View does not get access to the Model.

This is actually the defining characteristic of the PM pattern: The PM provides – constitutes – is – a data model, separate from the Model; the View is shielded away from the Model and doesn’t get access to it or its types. (It consequently follows that the pattern requires a duplication of the data model – hence the name Presentation Model – that is largely readily available with the Model.)

Thus the PM has several responsibilities:

• Defining its own data model
• Mapping the data between the two data models (using wrappers or other means)
• Handling the controller logic part
• Talking to the service (with BO’s rather than DTO’s, it would be their job to do this).

The Relation to MVVM

John Gossman’s post “Introduction to Model/View/ViewModel pattern for building WPF apps” (written 2005!) is generally referred to as the “introducing post”, bringing MVVM into existence. If only somewhat between the lines, but he introduces and defines MVVM as derivate of the PM pattern (without actually naming it); Wikipedia declares MVVM “as a specialization of the PresentationModel design pattern”, and the often quoted MSDN article “WPF Apps With The Model-View-ViewModel Design Pattern” follows the same principles.

Hence MVVM also suffers from the need to replicate the data model. John Gossman states that “only a small subset of application UI can be data bound directly to the Model“, and thus it is feasible to pay for this effort and the implications. However, if you don’t agree with this assessment (like me, for one), you could argue that this leads to meaningless replication.

Note: Fowler actually acknowledges that and states: “One area where data binding can work very well is with read-only data, […] The presentation model in this case reveals its internal data set as a property. This allows the form to data bind directly to the cells in the data set.”

However, this is a shortcut (kind of a degeneration) for read-only cases, in order to leverage data binding capabilities.

And by the way: If this is not exactly what you do if you say “I do Silverlight with MVVM…”, welcome to the club. That’s what started this post in the first place. More on this in the next posts…

That’s all for now folks,
AJ.NET

Whenever a service call reports an error I want some message box telling me about it (rather than simply swallowing it, which is the default behavior). Whenever the user does something potentially devastating I want some explicit confirmation, read message box, that he knows what he’s doing. MessageBox.Show does all I need (well, it is restricted to OK and OK/Cancel, but one can live with that). Only… these system message boxes are dull, boring, and not at all a shiny example for a Silverlight application. Enter the message box service provider…

Basic implementation of a message box service

The basic implementation will get the infrastructure up and running.

The first step is defining the service contract. Show this and that and a query method. The first (and naive) version looks like this:

The default implementation of our application extension service turned service provider (AES/SP) would use the dull system message boxes to implement that. The code is actually quite straight forward:

Now, I could demand that the app.xaml has this one (or any other service implementing my interface) registered. However, I like to be correct by default, thus my accessor will fall back on this implementation if none is registered – and I can be sure that there will always be a respective service.

All that is left is a search&replace for all calls to MessageBox.Show… E.g. to show an error:

… and to get confirmation, in this case to return a book:

And, of course, it works as expected:

Replacing the Dialog

Second act. Get rid of those dull things.

Create a new “Silverlight Child Window” and style it to look like a message box. I „borrowed“ the images from the Visual Studio Image Library (on my machine under C:\Program Files\Microsoft Visual Studio 9.0\Common7\VS2008ImageLibrary\1033\VS2008ImageLibrary\Objects\png_format\WinVista\) and simply placed all possible images in the dialog. A textbox, two buttons, that’s it. Here is the styled XAML:

Some code is needed for the initialization. The message has to be set, the correct image made visible, etc.. I could probably have done this with less coding, using some tricks and elaborate databinding. But who cares, it’s straight forward and comprehensible (unlike what I probably would have come up with).

Setting the DialogResult property also closes the dialog (sik!).

Finally I need a replacement AES/SP. The main method to show the dialog looks like this:

Great? Great! … GOT YOU! (Fell into the trap myself, actually… :-/ )

Fixing the Bug

Remember that in Silverlight everything is asynchronous? Well, everything except MessageBox.Show? And ‘everything’ includes ChildWindow.Show! Meaning my confirm method will not work this way. To overcome this I decided to pass a delegate to the dialog constructor and made sure it’s called in the OK case:

And to be able to pass the delegate I changed the existing AES as well (and the interface respectively):

Of course I had to adjust the default implementation using a messagebox:

The calling code changes respectively, passing a lambda:

Done. Now my application looks nice, even if it has to show a message box:

ANFSCD…

This endeavor served actually three purposes:

• First, I wanted/needed the feature
• Second, I wanted to see/demonstrate the service provider pattern from a user’s point of view.
• And third – as you may have noticed from some screenshots – I used this implementation to check out VS2010 beta.

A quick verdict about VS 2010 beta (not really worth a separate post)…

The core system, i.e. the shell, the C# code editor, build system, etc. feels very good. No apparent bugs, quite fast, including intellisense, and close enough to VS2008 to feel familiar. Considering that big parts of this are complete rewrites, this is quite an achievement.

The visual designer for (Silverlight) XAML works nice for user controls. Designing grids, the property pane, and other tasks, is at first glance en par with Blend, but comes in a more familiar „Visual Studio flavor“; still it feels more rich and mature than VS2008.
However, there are some notable gaps. Editing of styles and templates, animations, and visual state manager are not covered. Thus my guess is that Blend will remain a necessary complement to VS, even if one has to switch less often. BTW: Contrary to what Tim wrote, I could work with Blend on VS2010 solutions (the project that cannot be loaded is only the web project), I just refrained from manipulating my project files with Blend.

Other areas I touched briefly have been less satisfying. IntelliTrace didn’t work, but I didn’t spend too much time on that. The architecture and modeling area for example has changed, but is by no means bug free (to the point of “not yet usable”). The profiler has evolved, but IMO still lacks what DevPartner offered nearly 10 years ago: a decent call graph.

Oh, one bright spot for any dev lead: code analysis (FxCop) rules are now maintained in separate files, projects reference these files by name.

Anyway, I have been using VS2010 beta since I installed it and was never compelled to switch back to VS2008. I’m going to have to reinstall my machine anytime soon, and I’m planning on going along with VS2010 beta, not installing VS2008 at all.

That’s all for now folks,
AJ.NET

Note: This is part of a series, you can find the related posts here…

The last post used the new Application Extension Services (AES for short) to include security into the application. This time I’m going to take AES one step further, laying out yet another piece of basic infrastructure.

As a quick recap: An AES is a simple class, implementing a simple interface (IApplicationService) and optionally another one (IApplicationLifetimeAware). It is then registered by the developer via the app.xaml. SL3 instantiates the AES at runtime and calls the respective callback methods on said interfaces, including StartService, StopService, and others. The recommended way to use these services is to maintain a static property and reference the class accordingly.

Basically AES solve one problem: How do I extend the global application class, without actually replacing it, i.e. without providing a derived class?

The problem here is that many libraries need some kind of global anchor and in the past, tool developers often chose to provide this by subclassing the next available central artifact (e.g. the page class or the application). And the next tool developer doing the same rendered those two libraries mutually exclusive, just by employing an adverse implementation strategy.

However, SL3 solves only the providing part of the equation, the part related to the application class and the instantiation of the AES. But look again at the consumer code:

Following the recommendations, I get singleton access to some object. Worse, the calling code is directly and tightly coupled to the AES class. But frankly, do I need SL3 to implement a singleton? Certainly not. Then why use AES in the first place?

What I’d love is to have the consumer code depend on some service (read interface), not on the actual implementation. And the ability to swap those services in and out, without affecting the caller. Matter of fact, the calling code doesn’t and shouldn’t care whether I use a SecurityServiceThatGetsItsInformationFromTheServer or a SecurityServiceThatGetsItsInformationFromWindowsAzure (or google account, open id, whatever).

If I could get that, AES would become a very valuable feature… . 

Introducing Service Providers

OK, what the calling code needs is some service in terms of a contract, say ISecurityService, that it can ask for. And the same is true for any crosscutting concern, such as error reporting, tracing, caching, you name it. And actually .NET has already addressed this need with the service provider pattern. This pattern has been used for example in WF (e.g. to introduce workflow instance persistence), and quite extensively in the Visual Studio design time infrastructure.

OK, I can hear you crying out DI. And the chorus chanting Unity or Ninject (to name just two that support SL). But think again. Would you (or rather a library developer) mandate a specific DI container without reason? And you can always wire your pet container into this pattern by providing a IFactory service using whatever container you like. After all, the pattern handles access to services, not instantiation of them. (Which is what AES does, but again, Microsoft chose not to provide a fully fledged DI container…)

What do I need to make this approach tick? I need a GetService method that iterates all AES – they are available via Application.ApplicationLifetimeObjects. And I need something to attach this method to. Using an extension method I can actually attach it to the application class:

The method iterates all AES, checks whether one implements the requested (interface) type. It also checks whether any AES itself follows the pattern and implements IServiceProvider, and respectively forwards the request if it does. (This allows me to build up a chain of providers.)

That’s it. Long talk, short implementation, all set

Reimplementing the SecurityService

Now I need the rework the SecurityService and the calling code to comply with the pattern. The service interface is simple enough:

And my revised security service looks like this:

The base class ApplicationServiceBase implements the two interfaces with respective empty virtual methods, thus I only had to overwrite StartService. Aside from some reformatting the most notable difference with the previous implementation is the absence of the static Current property to access the service instance at runtime. I also renamed the class to reflect what it does. Since the calling code won’t refer to that name any more, this is now feasible.

Accessing the service at runtime is done – drum roll please – using the service provider pattern. This (admittedly not exactly nice) code can be hidden in a simple static property, adding a little convenience. I did this with a class that provides a static property, but also allows calling an extension method on the Application class:

BTW: Wouldn’t it be nice to have extension properties that I could attach to the application class…?

And the calling code, i.e. the access to the user object changes to:

Now the calling code is completely decoupled from the actual implementation of the security service, meaning I could replace it without affecting the calling code.

A look ahead: Building on the pattern

There are several use cases for this pattern that I will employ (they may or may not be addressed in later posts, but at least this list should give you some ideas):

• Last chance exception handling: A service will react to the unhandled exception event and gather context information. It will then use another service to report the error.
• Message boxes: I will need message boxes for errors, information, and confirmation. The system MessageBox is however somewhat dull. A service will cover that and a later implementation will replace the boring system dialogs with nice and shiny replacements.
• Logging and tracing: At some point I will have to tackle these demands.
• …

I’m sure you can think of other examples, like navigation with parameter passing, global state, caching, …. Anyway, I think this is motivation enough to roll out some additional infrastructure.

As a side note: Actually I had a completely homegrown implementation of this pattern for SL2. When SL3 came out and offered AES I quickly jumped on the bandwagon and threw away most of that code. If only Microsoft had not stopped one step short of my needs… . I’d rather have the platform support that out-of-the-box. And given the simplicity of the remaining implementation this shouldn’t have been too much of an issue.

That’s all for now folks,
AJ.NET

Note: This is part of a series, you can find the related posts here…

The last post (pre-Kili ) provided me with the necessary infrastructure to address the next topic: application security.

Silverlight is “secure by design”. It runs in a sandbox, only allowing restricted access to the local machine. Regarding server calls it supports HTTPS and windows authentication for intranets, it restricts URL access and allows only calls to servers explicitly opted in for. From a .NET perspective it has a changed programming model, no longer supporting CAS, but a more simple model that takes the restrictions into account that are already in place. It has the well-known interfaces IPrincipal and IIdentity in place, yet no implementation and no anchor to ask for them, like HttpContext.User in web applications.

Silverlight is so secure, it doesn’t even tell you, who you are.

From a purely technical perspective this is perfectly in order. The services I call are running on the server and it’s their job to make sure no one does something or gets to know something he isn’t entitled to.
On the other hand, many intranet applications need to know the user name, if only to display it. And they need the permissions granted to the user to hide buttons, menu entries or make edit fields read only – none of that for actual security reasons (the server would take care that the user couldn’t do any harm, anyway), but to improve the user experience, not letting him do things and telling him afterwards that he has had no permission to do what he did in the first place.

User and roles are available at the server at the ASP.NET runtime, and one “only” needs to make it available to the client. Matter of fact, ASP.NET provides the necessary service implementation readily available. This is also the way to integrate with ASP.NET forms authentication.

There are two caveats with this approach, though: These ASP.NET services provide roles and all, but they just don’t include the user name. Also, being services, I would have to call them asynchronously, which would incur at least a little time lag after starting the application, that I would have to deal with. A time lag that is unnecessary in intranet applications where the user is determined by windows authentication.

Thus my solution is as follows:

1. On the server include the information about user name and his roles in the initparams, so that it is available for the SL application right form the start, without any lag.
2. On the client pick up that information and mimic HttpContext.User to have a similar developer experience.

Of course, someone could fake this data, which might at first glance be a security flaw. But as I said, this information is only used to improve the user experience. Even if someone masqueraded as a different user, the sandbox on the client side still uses the actual user’s restrictions. And the same is true for the services that still have the responsibility to enforcing security anyway. I’m also not revealing sensitive information, since user name and permissions are not exactly protected data. And finally I could load the web page containing this data via HTTPS. Thus, form a security perspective I’m on the save side.

Server side

The server side is easy enough, since the necessary work has already been done. Step one is to include the information in the initparams within the ASP.NET page:

Step to is the respective implementation:

Client side

Mimic HttpContext.User? Well, there is no HttpContext, but we have the Application object which can be seen as a pendant of sorts. (I wouldn’t relate it to the HttpApplication, though, since an application in the ASP.NET sense spans all users.) Yet I’m reluctant to provide a base class to inject my pet feature, because that would bring me into conflict with any other library that may one day choose to do the same.

Luckily SL3 introduced the new concept of Application Extension Services (AES for short). AES have to implement a simple interface and are registered via the app.xaml. SL3 instantiates them and calls the StartService and StopService method to provide the hooks for initialization and shutdown. There will be another post about AES, thus I’ll leave it at that for now. Anyway, here’s the respective implementation:

It’s straight forward, it merely parses the information provided via the initparams. And the registration that will create the service at runtime is quite simple:

This is backed by boilerplate implementations of IPrincipal…

… and IIdentity:

From here I can use the user information at leisure, as I need it, and as I’m used to do. For example:

That’s all for now folks,
AJ.NET

Note: This is part of a series, you can find the related posts here…

Configuration of ASP.NET applications is quite potential. Simple application settings, database connection strings, complex configuration data. All editable by the administrator after deployment.

Configuration of SL applications is virtually non-existent.

While a Silverlight application may have less need for configuration than a web application – part of the logic still remains server side in respective services anyway – it’s not as if there is no need at all.

The service URLs (or a base URL) themselves, for example. Service proxies are created with the very URL that was used to provide the meta information baked into the code. Since this is probably a local address on the developers machine it needs to be adjusted during runtime. And preferably the administrator should be able to reconfigure the SL application if the service URL changes.
Other use cases include the usual: Whatever you want to put in some appSettings replacement. Enabling debug features. Just have a look at your last web application’s configuration and you’ll probably find some good candidates.

Silverlight configuration

Looking at the object tag for a Silverlight control on a web page (the ASP.NET server side control we had with SL2 is gone with SL3), there is some technical parameterization available:

You can find a complete list of available parameters here and below.

Of particular interest right now is the initParams property. It allows me to pass arbitrary information to the SL client. It is however not suited to be maintained by an administrator, as I certainly don’t want him to mangle my web page. However nobody says I cannot provide code that reads the information from somewhere else, like so:

and the respective code, obviously intended to read in the contents of a config file:

That config file is a simple analogon to the standard ASP.NET appSettings:

Note: I could have returned the web.config appSettings section. Yet using a separate file helps keeping client and server configuration cleanly separated.

So, what’s left is the actual implementation of that InitParams class, I’ve been using…

Implementation

The InitParams class is merely a little helper, providing a fluent interface over a dictionary and convenience methods, as I needed them:

Reading the config file is equally simple with XLinq at our disposal:

And I will complement that with additional helpers in upcoming posts.

What’s it worth?

So now I have the ability to provide configuration to my client, the information readily available to be picked up during the startup event or anytime from Application.Host. For simple values I have the configuration file, for other information I can extend my helper class anytime I want (and I will!), the infrastructure is there.

Could that have been done otherwise? Well, a respective services would have been another option. But providing the information via the initparams has the advantage of being readily available to the client without delay. And configuring the configuration service via the configuration service…? Well.

That’s all for now folks,
AJ.NET

I had thought for a longer time about posting some pictures from PDC08 as kind of a retrospective – but you know how it is, other things coming up, not much time, well.

And then this nut showed up, hijacked my blog and posted this nonsense about some crime that should have happened last PDC? Come on! Here’s the list of what I can only call a real good example of real dumb accusations:

• PDC08 – the unrevealed crime (featuring Anders Hejlsberg)
• Part 2: Management Panics (featuring Ray Ozzie)
• Part 3: Early Responses (featuring Bob Muglia, Steven Sinofski, and others)
• Part 4: One Special Response (featuring Chris Anderson and Don Box)
• Part 5: The Investigation (featuring Channel 9 and the audience!)
• Part 6: A Copy Cat (featuring Scott Guthrie)
• Part 7: The Revelation (featuring Bob Muglia again)
• Part 8: The Guilty (featuring Don Box – talk about nuts)
• Part 9: Aftermath! (featuring PDC, L.A. and Halloween)

Let me stress that again: There is no proof whatsoever to this story! As I see it, all these accusations are groundless, all people addressed are innocent victims. So, please, do not accuse anyone of installing LINUX on somebody’s machine if you don’t have proof. Much less on the grounds of this story!

All that remains for me is some complementary pictures:

 Windows Azure was THE big announcement at PDC08, and it’s going to be one of the major topic for PDC09. Microsoft really has something here that might change a lot in the long run. If they remove the two road blockers, that is: One, they need to make this infrastructure available for enterprise customers to lower the barrier to entry. Two, they need to do what they always did: attract developers and create a prosperous community of enthusiasts around Azure. To achieve that, they need to offer the platform for free. All with reasonable constraints of course. And this is just my opinion.   

Oslo, and M, was another major topic. But I can’t help thinking that this was mainly because it was the “Don Box-Topic”. It’s been relatively quiet around it since then, and unfortunately this also includes the somewhat related “Dublin”, which I think is a very important step in the services area.

Hands-On-Labs was as always the best way to lay your hands on the stuff you heard about for the first time just this very morning. If you had the time.

Talking about the location, the first and the last thing you saw about PDC was L.A. …

the Convention Center…

… and if you didn’t use the shuttle system, it was probably Pico Station, featuring a great view at Downtown.

However, I have been attending 3 PDCs so far, all of them in L.A. I’d really like to see some other city next time? (Microsoft, I could give you a list of areas in the U.S., I haven’t visited so far ).

So much for my little retrospective. CU at PDCnext… .

That’s all for now folks,
AJ.NET

I tried various channels:  

Officials didn’t know (or reveal) anything!

Other attendees searched for some kind of evidence – no success either.

  The local muscle knew nothing.

Law enforcement was as confused as I was.

When the PDC closed the doors for the last time, after sunset in L.A., I know that everything was lost.

The next day was October, 31st, and I still remember this strange girl, claiming to know something, but not really telling. Beyond some slightly insane giggle…

PS: This is it. I told it all. Obviously I will not come to this years PDC, not to watch an even more horrid crime unfold. To those still planning to go there: I wish you all a good time and – above all – an uneventful stay and a save return home!

Anonymous

For one thing they gave each of them a… mentor. 

“Someone stole my belt…” “No Don, no one stole your belt!”

“Chris was supposed to be here but he ‘volunteered’ for toilet cleaning!” “Now Don, isn’t that nice? Doing something positive?”

Apparently Don didn’t take that all very good. He reverted to drinking… 

And then happened what made me shiver, and, ultimately, break the silence…  Don’s mood changed…

“You know what?”

“I just got a wonderful idea….” “Don? Don’t start something foolish. Again. You know what happened…”

“YES! But you will not catch me NEXT TIME. Wait until PDC 09!”

I can tell you, I was in shock. I don’t remember how I got back to the hotel. My colleagues have been wondering about my apparent paranoia ever since then. I can’t help it, I always have this… image… of … LINUX … running on my … and that of other innocent and less prepared victims… oh no, I can’t take it no longer. I had to tell you and I’m sorry I had to give you nightmares. But you have to be prepared. Someone had to warn you…

Anonymous

Anyway, finally he broke the silence. Not only did he reveal who was behind THE CRIME, but he also made clear how high this conspiracy ranked, if he himself knew about it…

“I, Bob M., do confess that I knew from the very beginning about…”

“Chris and Don are not bad people, really, …”

THERE! SEE! HE NAMED THEM!

“… they are just misguided, maybe hung out with the wrong people…”

HE EVEN DEFENDS THEM! 

“… we have to help them. Make them see the wrong in their doing.”

Of course this part was cut out of the official recordings of the key notes. I also couldn’t ask him personally later on because he supposedly started his vacation earlier than planned. Yeah, right. But he is still alive, I mean, they didn’t go that far… so far.

Anonymous

“That hideous crime … scratch that, I’m not allowed to talk about that any longer. Anyway, I can live with LINUX. I mean, as long as I can install Visual Studio it could be running on DOS….”

“What… What is ‘Booting Linux Kernel’ supposed to mean?  … Wait! Where’s my Visual Studio? …”

  Fortunately the forensics where still around. They secured the evidence and got the machine running in no time. Real pros, I have to say.

Later the “problem” was explained as some ‘glitch’ in an early Windows 7 version. (There was some talk about a run-in, Scott supposedly had with Steven later that day, but I could never confirm that.)

Stay tuned, I’m getting closer. You’ll never believe the truth…

Anonymous

Anyway, it was not long until the professionals arrived.

Some forensics specialists…

Some, let’s call them detectives,… – see that innocent looking guy right in front? Did you notice that he is obviously watching the audience in front of him, rather than what’s going on on the stage? See? And two rows behind him, probably his assistant, looking directly at ME?  

But most important was the arrival of THE INVESTIGATOR. Nobody knew him. At first nobody even knew he was there; he was inconspicuous to the point of invisibility. But he had a reputation of sneaking up on the criminal and put him in handcuffs before I even knows what happened to him… me…

He would spot the suspect in a crowd like a blood hound. Hunt him down… hunt me… OH NO!!

… and finally confront me. “Excuse me?… … Who me, sir?… … No sir, I know nothing about no LIN… uhm… what did you call that?”

I took a while and I had to admit some of my lesser crimes (like installing OS/2 when I was young, or laughing at Bill Gates), but finally I could convince him that I was an innocent bystander. Yet I still shudder from the experience…. It was so…. Sorry, I’m too stirred up right now to continue…

Anonymous

“This is plain disgusting…” “Oh yeah!”

  “Could you imagine something more vile?” “No never. Do you think they’ll catch him?”

  “No, I don’t think so. They’ll never find out that w…” “Uhmm!” “… ahm, I mean, we will never find out who did it!” ”He he.” “YES!”

“Oh, I’m still sooo shocked.“ “I have no words…”

“Let’s write some letter of condolence…” “OK, shot! I’ll write.” “You can’t write.” “Yes, I can!” “OK…”

“Dear Mr. Assassin…” “D-e-a-r… M… where is the M…”

I was deeply move by such caring guys. It was really touching.

But soon thereafter the whole affair became dirty. But I’d better switch my location again…

Anonymous

Anyway, about the reactions I managed to catch:

“What on earth is a human being thinking, installing LINUX on a machine of an innocent person…”

“… never understood those freaks. Must be some personality disorder…”

“You heard what happened?” “Yes, I was told that there were three of them…” “Hush, they’re going to cover-up. This is a marketing disaster.”

“Now personally I can understand that someone would get rid of Vista. But Anders actually had Windows 7 running and THAT is the real crime here….”

There were some more reactions, but I have to leave…

Anonymous

Just after the incident nobody thought about cover-up. That came later. Rather THE CRIME quickly got management attention. The shock obviously went deep.

“My dear friends. Before I get to the finest newest technology in the world, I have to bring you some very sad news…”

“… I’m still deeply shocked myself…”

“… I implore you to help us solve this… this… lèse-majesté!”

That rambling went on for some time, but Ray certainly kept his dignity. In the end he even offered some consolation to the equally shocked audience.

“But rest assured! We WILL find this criminal and we WILL bring him to justice…”

“We shall not allow this to happen again!”

I also got some reactions…. Excuse me? Uhm, no, this is just a personal e-mail… (Sorry, have to leave…)

Anonymous

You never heard about it? No wonder. Ask someone who was at the last PDC to tell you something about THE CRIME. You’ll see people hunching down, backing off, finding excuses to leave, or otherwise evading the topic. It’s the dirty secret we all share and that to the best of my knowledge, nobody has told yet. It’s a hideous crime, and the cove-up that ensued is on par with the fake moon landing. (You know the moon landing never happened, don’t you?)

But this time, it’s going to be different. This time the truth will be told. I will tell it, for I cannot bear the burden of endangering the attendees of the upcoming PDC on my conscience.

THE CRIME

It happened to the most revered, and well regarded, father of Turbo Pascal. (And some other languages, I guess. He never managed Modula, though, but that’s really NOT AN EXCUSE!): Anders Hejlsberg!

I was a witness and I happened to have my camera ready to document THE CRIME first-hand (and I sneaked it out when all cameras where confiscated!). See for yourself:

 “What’s that…?” “No… that can’t be…? Must be some hardware problem…”

 “Really it has to do with these cables…?”
“No Anders, look at your screen. If the problem is not on the screen, it’s probably in front of it.”

Note: In the interest of the innocent I’ll leave out the rather pitiful scenes and groundless accusations that followed. Suffice it to say that Anders eventually had to accept the fact that somebody installed LINUX on his machine! Imagine the insolence! The effrontery! the sheer impudence! A penguin mocking a distinguished person like Anders, and along with him the whole assembled Microsoft celebrity!

Of course that didn’t amuse Anders in the slightest way…

 “I’m usually a nice and likable guy, …”

 “… but let me get my hands on this culprit, this evildoer, this…” (again, I have to protect the innocent.)

And yet, this… abomination… was only the beginning of the story…. But I have to go, that guy over there is eying me suspiciously. More when I’m save again…

Anonymous

Note: This is part of a series, you can find the related posts here…

This time, building on the last post, I’m going over the basics of getting data to the client. The order of the day is just get it working, I’ll come back later to address some issues.

The Service

I have to provide the service first. This is easiest done by adding a Silverlight-enabled WCF service, located in the “Create New Item” dialog, under the Silverlight group. It comes with the necessary configuration, ready to use. After defining the EF model, a first service operation delivering recently acquired books may look like this:

Note the Sleep. I like to stress the asynchronous nature of the SL client and I want the client code to handle this properly, thus the Sleep makes the time lag very apparent.

Note: This way I know very early where I should provide visual feedback, or disable controls. This is especially important considering that on my development machine I will hardly spot any delay, but once the application goes into production and network performance and latency demand their toll, this will likely change.
Thus I recommend you do the same, perhaps surrounding with #if DEBUG and making it configurable.

Calling a service

After running the web application, Cassini provided a live version, which I needed to create the client side proxy. This generates respective code, of course including the data defined in the contract. Those classes already (by default) implement INotifyPropertyChanged and use ObservableCollection<T>. It certainly will give you a head start if you can use these classes as your model.

For every service operation there is a respective MyOperationAsync() method that initiates the asynchronous call and a MyOperationCompleted event that delivers the result. (No synchronous version by design!)

One hint: Creating the client proxy also created a ServiceReferences.ClientConfig file in the service assembly. This file is needed in the UI project to be available for the client. It works quite well to include it in the UI project be setting a reference, rather than just copying the file (“add existing item”, “add as link”), and this way you won’t have to keep a copy in sync.

Wiring it Together

The idiom is documented well enough, thus without further ado, my first version looks like this:

The client instance is also used to detect whether the last call is still under way, and to suppress further calls.

The UI includes a button to trigger the server call (just for now, in this case the call will eventually be made from the c’tor), and the DataGrid to present the result:

The respective event handler is trivial as well:

Now, running the app looks like this:

Clicking the button… waiting… that’s why I put the Sleep in… . Oh, and I can click the button as I like? … Finally the result appears:

Regarding the “I can still click the button…”, I wanted to make this point specifically, because I have already seen people ignore that time lag too easily. It no only makes your application “feel unresponsive” in case of longer operations, because the user doesn’t get any indication whether the application has actually accepted his button click. It also may makes your logic more complex if it had to deal with multiple calls at the same time. My code simply ignored them, but there is no reason why they shouldn’t be processed as they come in. Even canceling the previous call may make sense, because it may be obsolete due to changed parameters (most operations are certainly not as simple as the one used in this example). You may want to cancel if the criteria actually changed and ignore the subsequent call if the criteria stayed the same… . See? Quickly the complexity adds up. The easy way around is disabling the button and making the user experience a sequential one.

Providing visual feedback

So, the logical step is to add a respective property CanLoadBooks and use it for the binding…

After that, disabling the button can be done via data binding the IsEnabled property:

And nicely my application provides visual feedback and at the same time prevents the user from redoing what he did:

The stage…

This post sort of completes the initial setup of the stage for my Silverlight application. I have a solution, the basic layout is in place, the general application architecture and data binding strategy is set up, and with this post I can talk to the server.
From here I will go exploring various aspects, probably at random, but now I have the means to do so.
And there is a lot of aspects to cover anyway. The services part I addressed in this post is by no means covered exhaustively, neither is data binding. Other topics haven’t yet been mentioned at all, like basic application services, visual state manager, navigation, or configuration.

Anyway, I will have to work on it before I can write about it. So this series is certainly not coming to an end, however further posts may take some more time. And I have a small by-project running, some mischief I’m up too

That’s all for now folks,
AJ.NET

Note: This is part of a series, you can find the related posts here…

Displaying and manipulating data on the client – the one and only purpose of any LOB application – includes two things if it comes to Silverlight: a) The asynchronous server calls and b) the client architecture. I will be going over them in a cursory fashion and come back later to address each one in more depth. This time the client architecture.

View Model basics

The client code is largely guided by the employment of the Model-View-ViewModel (M-V-VM, or MVVM) approach, which is the predominant architectural approach used with SL and WPF. The reason probably being that it is a natural counterpart to the WPF and Silverlight data binding features, the two work extremely well together.

Cook book style:

• For every page (the view) there is a respective class (the view model) that manages the data (the model).
• For each control on the view that shall be populated dynamically with data, the view model has a corresponding property providing the model.
• For each control state, such as enabled or visible, that shall be controlled by the logic, the view model has a corresponding property, probably boolean.
• For each action triggered by the UI the view model has a respective method.

The view model is very closely associated with its view, so there is not much reuse here, but then, it largely consists of properties and forwards to service calls. Or so the theory says. (Subsequent posts will gave deal with the shortcomings….)

Please note that it is debatable whether the data provided by the view model actually is the model. Another approach would be to expose a view related data model, namely view entities. The view model would have to map them to the data model (the model) the lower layer exposes, probably some service proxy.

Both approaches have pros and cons, however you’ll mostly see the former approach, since it’s well supported by the tools (service proxy generation, etc.). Still, it has some cons…

Anyway, the only technical demand for view models in SL is due to the intended use for data binding: Classes subject to fully fledged data binding need to support the INotifyPropertyChanged interface for simple properties, while collections have to support INotifyCollectionChanged. The later one comes for free if you use ObservableCollection<T> consequently. (Please note that data binding works with conventional properties, but only to a limited degree.)

For INotifyPropertyChanged a little base class comes in handy:

Note the generic overload. That’s a little trick to avoid typos in the property name argument.

With this class as base a property implementation usually follows this idiom:

(Without that trick one would have to pass the property name as string. A source of errors due to typos, and a pitfall during refactoring.) 

BookFilter is a data class that, again, follows the same pattern, i.e. it supports INotifyPropertyChanged.

Hint: This begs for a code snippet!

The View Model

Any book shelf has a collection of books, so does my application. The book list page should provide a means to filter the book list (two text boxes), a button to trigger the search, and to show the result (a data grid):

Not especially nice and the grid is a little degenerated for now, but that will change. In XAML:

Thus the first view model implementation may look like this (including some simple test data, the next post will deal with the server call):

Databinding

For the actual binding I need to wire that up with the page. The usually presented manual way looks like this:

The view model class is created in the c’tor, and assigned to the DataContext. A property provides a more convenient access to it. This is already used in the button event handler that triggers the book search.
However, I recommend against this way. Rather I did the data binding in Blend…

Opening the page, selecting the first textbox, finding the Text property in the properties and clicking the text box (or that tiny little dot to the right) brings up the context menu.

Then I choose data binding, the Data Field tab, and the +CLR Object button:

That left finding the view model class and selecting it:

This way I could add various “data sources”, yet I only want one for now, and according to M-V-VM, for ever. Afterwards the dialog lets me browse the class structure and select the property to bind against:

On second thought, I selected the StackPanel which contains the filter textboxes and bound it against the BookFilter property, in order to narrow the available context. Afterwards the textboxes could be bound via the Explicit Data Context tab:

I had to expand the lower area to set the binding to TwoWay.

But I didn’t have to go through the dialog armada for every field. Blend also provides the data tab that lets me browse through the available data sources. Drag’n’drop of field simply works and generally uses the last settings from the dialog, i.e. it includes the TwoWay setting. Also it binds by against the default property, but if I hold the shift key down it lets me choose the property. And some other stuff I leave to you to explore. Really nice.

Anyway, this is what Blend just created for us in markup speak (just the relevant part):

It registered the namespace to locate the view model class, created the view model as resource, set the DataContext of the LayoutRoot element to this resource, and it added the usual binding to the subsequent controls.

Changing the generated prefix to viewModel was all I did. Otherwise I could live very well with that, given that is achieves all I need and it allows me to do my data binding much more efficient and less error prone in Blend. The manual way was opaque to Blend, thus it couldn’t assist me in any way.

The only thing left was removing the manual instantiation from the c’tor and changing the ViewModel property to use the LayoutRoot control. I may have moved the binding to the page instead, but I prefer to work with the tools, not against them.

After running the application and clicking on the button, it shows the respective test data:

The next post will deal with the actual server call.

That’s all for now folks,
AJ.NET

Note: This is part of a series, you can find the related posts here…

One thing is as true with SL as it was with HTML and CSS: Starting with a basic layout of the pages and a “site map” really helps a lot. (Trying to work with a style system that you don’t know doesn’t.)

The application created by the template looks like this:

It’s a navigation application with head area (including menu) and remaining work area. It uses a Grid control, yet only as kind of canvas, all controls are placed via margins and alignments. I find this, well, curious, there is a canvas control after all. But the layout doesn’t suit me anyway.

So the first step is to get rid of all styles, i.e. I cleaned Assets/Styles.xaml. That included removing all references to these styles, as in this fragment:

Searching with a little regular expression solves that problem. Now for the intended layout:

I want to have a head area with application title and other information. A left area containing the menu. A bottom line with legal information. And finally the remaining area for our pages. And some space between for some visual separation.

In SL this is done with a Grid. Curiously enough, for this is akin to table layout in HTML. I wonder when the CSS gang will show up and cry wolf

Most samples show and explain how to write the markup to define rows and columns. But then, most samples are pre-created and reiterated, and I have problems “mind rendering” the markup. With the Silverlight 2 SDK there was at least a kind of preview in VS, but with SL3 that preview does not work (the document outline may help to navigate larger XAML files, though):

So I chose a different way and put Blend to its first use…

Grid layout

Defining the desired Grid layout can be done easily in Blend. Placing rows and columns is just a mouse click away: Clicking on the areas to the left or the top adds new rows and columns. Clicking on the symbols changes the type. If you don’t see them, change the layout mode by clicking in the upper left icon.

Once the rough layout is done, it can be saved and one can switch back to VS. The resulting markup is very clean, Blend generally does a good job producing clean markup and at the same time leaving your markup as it was.

Placing the contents

Next step was creating controls for the three areas (head, menu, footer) and move the respective content from the page there (copy & paste in VS). The markup now only contains the navigation frame, which is the working area. Drag & drop and some more mouse jostling in Blend positions that control in the correct cell:

And sizing it correctly:

Also – after recompiling the solution – Blend picked up the user controls and offered them as assets if you select Project. Again some mouse jostling later (and with different backgrounds colors for each user control to distinguish them) the result looks like this:

Blend automatically generated a namespace definition as prefix for the controls, using a veeerrrryyyy long prefix name. But that’s easy to solve and to replace with control. The final markup is, again, very clean:

After some working on the user controls and some styling… alright, it may take some time, but had it ready from some internal application, and besides it was done by a colleague, for if I had done it myself if would probably cause eye cataracts… , the end result looks like this:

This may all seem trivial if you’ve been through this experience once. Bottom line I guess is the way I worked through this (which may or may not work for you). It’s the combination of VS and Blend, working with both tools at the same time. Even if you are a markup guy rather than design time worker in ASP.NET, my recommendation is that you give Blend a chance. It’s far more stable than the ASP.NET designer in VS ever was. I’m not saying you should switch to Blend, just get the best of both, VS and Blend.

The next post will contain some real code, promise

That’s all for now folks,
AJ.NET

Note: This is part of a series…. Well, it will be  
You can find the related posts here…

Alright, let’s start looking into some Silverlight 3 (SL3) development. Specifically of business applications.

First step is a respective solution. I created a new “Silverlight Navigation Application”, with SL hosted in a web site, and got the typical solution structure. Needless to say, that this structure doesn’t meet my requirements.

First, I’m going to have a bunch more assemblies. And since SL assemblies are different from “ordinary” assemblies – they work against a different runtime and use different libraries – I like to have them clearly distinguishable. I did this a) with respective solution folders and b) with an added namespace part “SL3”.

The Server Part

On the server side I added two assemblies:

One data access assembly. This contains the ADO.NET Entity Framework access to the database. For the purpose of this application, this constitutes the business layer. (A little simplistic, but bear with me, the focus is on the client.)

A “Common” assembly. There will be server side parts of logic that may be reused in other applications.

I also made some adjustments to the web application (project properties/web): I changed virtual path to “/xBookshelf”, I don’t like my web apps to occupy the root. And I set the port to “specific port”. It doesn’t matter which one, it’s just a little annoying to have to update service references when the port is changed by VS every now and then. Speaking of services, I’m going to put them in a separate folder.

The Client Part

My naming convention for SL3 assemblies is Company.Application.SL3… . Specifically I renamed the SL assembly containing the XAML files to SDX.Bookshelf.SL3.UI. Renaming the .xap file affects the web project and the hosting pages. I also changed the pages to have Page as postfix.

And there are a few additional assemblies as well:

SDX.Bookshelf.SL3.Model contains the view models, as I’m going to use the Model-View-ViewModel pattern.

SDX.Bookshelf.SL3.Service contains all service references. Since this is all generated code, I thought it prudent to cleanly separate it.

SDX.SL3.Common is the counterpart to the server side common assembly. It will contain everything that may be reused in other application.

Setting the project dependencies and getting it all to run again is just a bit tedious, especially because the startup project is the web project, but the application in question is the UI project.

Blend

The final test for the reworked solution is opening it in Blend – which may be not as trivial as it sounds

You will at least get one warning dialog telling you that blend does not support solution folders. Never mind, that’s of no consequence.

In one project I also run into a serious issue: Blend refused to open the solution with the following dialog:

Also subsequently I got a large message box with an exception stack trace.

To work around this problem, I removed all projects from the solution and added them again in Blend. I was able to provoke that bug by reordering the sequence of the projects within the .sln file, hence it’s probably due to some project dependencies that blend can’t solve correctly.

Anyway, don’t let that bias your perception. Blend may be awkward, and it takes time getting used to – but is worth the effort.

That’s all for now,
AJ.NET

• I never got to look into Silverlight 1, so I avoided the scripting mess
• For Silverlight 2 I participated in some research efforts on our company and I worked on a business solution (as in business solution, not just a “simple” control). This did not only include the obvious stuff like drawing and driving the UI, but also some necessary groundwork.
• Microsoft launched Silverlight 3 officially on Friday, July, 10th; on the following Monday, July, 13th, I was officially in a customer’s Silverlight 3 project. We had just waited for the availability

And to relay the probably most important experience for me so far: Working with Silverlight 3 and Blend is fun!

It’s easier to get what you want than with web applications, the programming model is much more concise and powerful than ASP.NET, Blend has a far better user experience than the WebForms designers ever had (after being nearly a decade and several versions old!).
For me, working with Silverlight has a thrill factor comparable to what I encountered when I first started to develop with .NET.

Note: From now on, I’ll refer to Silverlight 3 simply as SL.

So, I may have a little head start with SL business applications and their demands, but I’m sure that will wear down over time. Anyway, I thought I might share some of the experiences, insights, and ways I discovered.

This is not going to be me providing a tutorial, or me telling you what to do. It’s about me telling you what worked for me. Some may be outright trivial (especially the first ones, laying the foundation), some may be about the how to do it, rather than the what to do. But then, I also had to tackle a few things that go beyond the usual “My first Silverlight Hello World Application” type of samples.

Disclaimer: This is as much about telling you how I did things as it is a learning experience for me. I may be wrong at times. I may tell you in one post to go left and a few posts later that you had better not listened. No warranties here, sorry.

And I’ll try something different and keep the posts short and to a certain point (and I admit that this post is already in violation with this intention…).

Preconditions

Just for the record: I’ll base the work on SL as you can get it here. That includes:

• Microsoft Silverlight Tools for Visual Studio 2008 (includes developer runtime and SDK)
• Microsoft Expression Blend 3 + Sketchflow

I refrained from using RIA services, because I want to understand the technology bare boned. That being said, I think that RIA services has really great potential and for those interested I recommend Brad’s series. I also haven’t thought about including the control toolkit, yet.

Setting the stage

My way of learning a new technology is usually a) read about it to understand the idea behind it, b) do some technical prototypes to dive into one or the other feature, and c) take a real world example and see how the technology fares. In my experience, point c is the most important, because it goes beyond the usual samples that are built to suit the technology. Rather it stresses the technology’s abilities to meet actual needs. This is usually when you’ll encounter the pitfalls.

My real world example for SL was kind of a library or book management application. At SDX we maintain our books in a simple SharePoint list, so every colleague knows what books are actually available, where to find them, who has borrowed it currently. This list is more intended to keep people informed, than controlling whether they return the books on time. Weren’t our colleagues dispersed at various customers’ sites, a simple bookshelf with no bureaucracy at all would be sufficient. (And I wouldn’t have to go hunting for lost books every now and then .)

The Database

The database I’m going to use is simple enough. It contains books, information about who borrowed it, and user information:

Additionally I like to create views as I need them (rather than using EF for this). For example BookInventory contains all books as well as the related information on the currently borrowed books and employees.

And with these conditions set, we are ready to start. The next post will start at the beginning, stay tuned

That’s all for now folks,
AJ.NET
 

There are certainly references to Guard classes. For example this one hinting at an implementation from MS Patterns&Practices, although an outdated one and no online documentation available. And this one talking about using Guard classes in combination with LINQ.

Still, nothing explaining the concept, nothing on Wikipedia, nothing anywhere. Can it be that Guard classes are far too simple to be worth mentioning? I don’t think so, because while the implementation certainly is simple, I wouldn’t have had to explain the concept that often if that where the reason.

So, what are Guard classes?

First of all, let’s make sure we’re taking about the same thing. Judging from the name, Guard classes are about guarding against something, but this can mean just about anything. Actually I found two incarnations: One, guarding against concurrent access, i.e. some kind of locks. And two, guarding against the passing of invalid parameters into a method – which is what this post is about. (You may have come across Guard classes by the name of ArgChecker, ArgumentHelper, or something similar, or as single helper methods appearing where they don’t belong.)

Straight to the matter: Have a look at the following example:

IEnumerable<Employee> FindCustomers1(Guid companyID, EmployeeFilter filter)
{
    IEnumerable<Employee> result= null;
    if (companyID!=Guid.Empty) // only hit DB if we have to
    {
        using (DatabaseDataContext context = new DatabaseDataContext())
        {
            result = from e in context.Employees
                         where e.FirstName == filter.FirstName && e.LastName == filter.LastName
                         select e;
        }
    }
    return result;
}

While probably doing its job properly, this method isn’t exactly defensive. The issue should be apparent: What happens if the caller passed null for the filter? Not that he is expected to, quite the opposite, but anyway?

You could try to handle that in code and it would probably lead to some ugly code, as Abby rightly complained about. And to be blunt, that approach is wrong right from the start. If caller is not supposed to pass null values into our method, why should we clutter our code with conditions that aren’t supposed to happen anyway.

The better solution is getting rid of unwanted null values by making them illegal. And not only by politely asking in the documentation (and please RTFM); rather make it unmistakably apparent, punish any violation, make it impossible to bypass. In other words, check the parameter and throw an ArgumentException.

On second thought, the same reasoning applies also to the properties of our filter object, so we may end up writing the following code, prone to becomes ugly itself:

if (filter == null)
    throw new ArgumentNullException(„filter“);
if (filter.FirstName == null)
    throw new ArgumentNullException(„filter.FirstName“);
if (filter.LastName == null)
    throw new ArgumentNullException(„filter.LastName“);

And we actually did expect some content, so why not go ahead and do some other vanity checks:

if (filter.FirstName == „“)
    throw new ArgumentNullException(„filter.FirstName“);
if (filter.LastName == „“)
    throw new ArgumentNullException(„filter.LastName“);

But wait. An empty string is certainly not null. Unfortunately there is no StringEmptyArgumentException. Should we use an ArgumentOutOfRangeException? Or an InvalidOperationException, not an ArgumentException at all? Derive a new one? ArgumentOutOfRangeException may look nicely to me, but my team mate just settled with ArgumentNullException, which is at the least an inconsistency… .

And what the heck, this is way too much code for nothing anyway. And so we end up not checking our parameters and hoping for the best…?

Now consider this code:

IEnumerable<Employee> FindCustomers2(Guid companyID, EmployeeFilter filter)
{
    Guard.AssertNotNull(filter, „filter“);
    Guard.AssertNotEmpty(filter.FirstName, „filter.FirstName“);
    Guard.AssertNotEmpty(filter.LastName, „filter.LastName“);
    
    IEnumerable<Employee> result = null;
    if (companyID != Guid.Empty) // only hit DB if we have to
    {
        using (DatabaseDataContext context = new DatabaseDataContext())
        {
            result = from e in context.Employees
                         where e.FirstName == filter.FirstName && e.LastName == filter.LastName
                         select e;
        }
    }
    return result;
}

We just replaced 5 ugly conditions (10 LOC) with 3 calls. Let me rephrase that: We replaced a bunch of code detailing some implementation with 3 lines revealing the intention – way more readable. Wow! By centralizing the checks, we were able to provide more convenience (putting the null and string empty check in one method), we ensured consistent behavior, we improved the calling code considerably. And finally, we even added some more information to the exception, as you will see in the implementation:

[DebuggerStepThrough]
static public void AssertNotEmpty(string arg, string paramName)
{
    if (arg == null)
        throw new ArgumentNullException(paramName,
            „Argument ‚“ + (paramName ?? „<missing>“) + „‘ should not be NULL!“);

    if (arg.Length == 0)
        throw new ArgumentOutOfRangeException(paramName, arg,
            „Argument ‚“ + (paramName ?? „<missing>“) + „‘ should not be empty!“);
}

You don’t want to put that code in every method. But you do want to call the Guard method, don’t you?

Now that we have established what Guard classes are, let’s look at …

What are Guard classes about?

So far I introduced Guard classes form the coding level, merely as convenient helper classes. But there’s some broader meaning to them, namely in the following areas:

• Contracting
• Self preservation
• Robustness

Contracting.

Contracting as a means has gained attention as important aspect of SOA service design, and also as development approach, namely design by contract. Contracting may be a major undertaking if it comes to SOA services, yet it can also be employed on a much lower level, designing the public interface of a class.

A little simplified, a contract of a class or service consists of

• Functional contract: Specifies the operations, the semantics, required call sequences, runtime behavior, error conditions and behavior. Some of this may be expressed using code, some may not.
• Data contract: Details parameters and return values, in other words the data that is exchanged during the operations. It defines the legal shape and values of the data, restricting it first by data types, secondly by additional demands, such as stating which parameter is mandatory, which content is required, and which values it is limited to beyond what the data type mandates. Also interdependencies between fields, and so on.
• Some other aspects, but that’s not relevant right now.

As you see, the data contract goes beyond what the type system is capable to enforce. And Guard classes may serve as a means to express some of those aspects in code. Thus, Guard classes serve to enforce the contract of a class. And since these calls are that obvious, the code is self-documenting its preconditions quite nicely.

Note 1: Of course this makes only sense with regard to the technical contract that has to be met by the calling code, i.e. the developer, not a business contact that has to be met by the user. See my post on error handling and responsibilities for more on that topic.

Note 2: In terms of design by contract, Guard classes express the preconditions of an operation, something that has been baked into other programming languages right from the start.

Self preservation.

If you are working in a team, you inadvertedly have to work with other team members. Good ones, bad ones. And you may want to protect yourself against the later… .

Face it: If a NullReferenceException arises, the developer who wrote that particular code fragment – mayhap you – is held responsible (shot on first sight, so to speak). Even if it turned out later that the calling code passed some illegal null value… . Well, problem solved, but the stigma sticks, and anyone will only remember that the exception was thrown in your code.

How do you protect yourself against that? Guard classes. If the calling code just passed some crap data – despite you having told that guy what to pass a hundred times – just throw it right back into his face. That’s what ArgumentExceptions are for: complaining loud and clear about someone unduly misusing your code. Be the accusator, not the accused!

BTW: If you’re on the callers side and the method you just called returned a value, you may achieve something similar with Debug.Assert.

For example if you had written that code above, and I knew you to be a sloppy developer, I would certainly check the return value. And if I were as mean as you are sloppy, I would deliberately call it passing Guid.Empty as companyID. And if you dared to hand back that pesky null instead of an empty enumeration I would … .

Well, never mind. You‘re not sloppy, I‘m not mean, and I only put that bug in to stress the fact that Guard classes only work up the call chain, not down. (And yes, it’s a bug to return null in the above case. Methods supposed to be used in a LINQ context have to comply with LINQ demands, which are based on functional programming, which boils down to „no nulls please!“)

Robustness (or: making errors apparent).

Someone passed null or some other unwanted value to a method. There are actually two bad things that might happen:

1. The application may crash further down.
2. The wrong value may affect some data, but the application continues to work.

Number one may be bad, but at least the application crashes (which is a good thing!). Still, in this case Guard classes will help you identify the error far earlier, possibly avoiding a situation where you’ll have to hunt for the root cause of a problem. This should at least help diagnosing the problem.

Number two is far worse. It’s actually the very situation that you would want to avoid at any cost: A bug that stays unnoticed. A bug that occasionally produces wrong data, and that lingers around long enough to seriously affect data consistency to a degree that renders all data useless, because you have no way of knowing which data was affected and which not. Technically just a tiny slip, but in the long run these bugs are the most harmful.

To guard against these bugs Guard classes put up another safety net. Make sure the data coming in matches the specification.

Essentially both issues neatly demonstrate what „fail early, fails fast“ is about. Making errors apparent, prevent them from being obscured and from causing damage further down the line.

And that’s it…

Quite some task for a tiny little helper – but really worth the effort. Should you need some kick-start to employ this concept yourself, I posted the code for a Guard class here. Use it, enhance it, employ it to make you life easier.

HIH!

That’s all for now folks,
AJ.NET

Note: What follows is merely a sample implementation, just to drive the ball home, but it should turn my earlier musings into a little more practical advice. And since this is only meant to highlight the principle, not to provide a production ready solution, I’ll keep it simple.

Exception classes

First let’s try to classify our exceptions, depending on the supposed effect. This classification is sort of the driving force behind the strategy: who is responsible, or more to the point how to tell the responsible people…

A quick reminder on what our goal is:

1. The user has to solve everything that results from his user user interaction in due process. This includes invalid input, insufficient permissions, or other business conditions that interrupt the current functionality.
2. The administrator is responsible for infrastructure issues. This includes every call into the „outside world“, such as databases, services, file system. Issues here are manifold, including:
   • unavailable database or service
   • errors during some operation, e.g. unexpected a constraint violations
   • erroneous configuration
3. Everything else: Bugs. Developer. You!

So consequently:

• If the user is responsible, we just tell him on the page and let him choose what to do.
• If the administrator is responsible, we show a general error page, telling the user that the application is not available, and of course we write some event log entry.
• If the developer is responsible, we do the same, but we tell the administrator explicitly to forward the event log information to the development team.

I’ll spare you the boilerplate exception implementation details, suffice it to say that we have a UserIsResponsibleException for “user exceptions” and a AdminIsResponsibleException for “admin exceptions”, respectively. Names that wouldn’t make it into my production code, but that’s what their intention is, anyway. Oh, and add a ConcurrencyException exception for respective issues, they may or may not be user exceptions, depending on the context.

Now let’s put the handlers in place. These reside in the UI layer, and from there we can work through the architectural layers towards the databases.

User error handlers

User exceptions have to be handled on the page level. Whenever you call into some business layer method, say from a button click event handler, you wrap the call in a try/catch(respective exceptions). The exception is then presented to the user, and eaten, meaning it is not thrown along. Something like this:

protected void btnConcurrentUpdate_Click(object sender, EventArgs e)
{
    try
    {
        BusinessLogic bl = new BusinessLogic();
        bl.UpdateDatabaseWithConcurrencyIssue("some content");
        lblConcurrentUpdate.Text = "did it.";
    }
    catch (UserIsResponsibleException ex)
    {
        // just give feedback and we’re done
        PresentError(ex);
    }
    catch (ConcurrencyException ex)
    {
        // just give feedback and we’re done
        PresentError(ex);
    }
}

The way of presenting the error message is something that has to be decided per application. You could write the message to some status area on the page, you could send some java script to pop up a message box. You could even provide a validator that checks for a recent user exception and works with the validation summary for the feedback.

Statistics:

• Exception classes: Write once.
• Presentation: Write once.

The try/catch code is simple enough, yet you have to do it again and again, and should the need for another user exception arise (e.g. some user errors can be handled on the page, some others require leaving the page), the logic is spread across our pages. Thus it makes sense to factor the actual handling into a separate method, in a helper class, or a page base class. Now, catching specific exceptions is still part of our job and that cannot be factored out. To solve this we have three options:

1. Build the exception hierarchy accordingly and catch the base class for user exceptions.
2. Catch all exceptions and throw the unwanted exceptions on.
3. Live with the problem.

I decided against option 1 because I didn’t want the concurrency exception to be derived from some user exception. Option 3 is what we wanted to avoid in the first place. So my solution is number 2, in the name of ease of use and maintenance, but at the expense of language supported exception handling features:

protected void btnConcurrentUpdate_Click(object sender, EventArgs e)
{
    try
    {
        BusinessLogic bl = new BusinessLogic();
        bl.UpdateDatabaseWithConcurrencyIssue("some content");
        lblConcurrentUpdate.Text = "did it.";
    }
    catch (Exception ex)
    {
        // since C# does not support exception filters, we need to catch
        // all exceptions for generic error handling; throw those on, that
        // have not been handled…
        if (!HandleUserExceptions(ex))
            throw;
    }
}

This is boilerplate and unspecific enough to copy&paste it around. The generic handler may look like this:

bool HandleUserExceptions(Exception ex)
{
    if (ex is ConcurrencyException)
    {
        // just give feedback and we’re done
        PresentError(ex);
        return true;
    }
    
    if (ex is UserIsResponsibleException)
    {
        // just give feedback and we’re done
        PresentError(ex);
        return true;
    }
    
    // other errors throw on…
    return false;
}

Statistics:

• Handler method: Write once.
• Catching exceptions: Every relevant location, boilerplate copy&paste

Please note that exception filters might make this a little less coarse, but this is one of the rare occasions, where a CLR feature is not available to C# programmers. Anyway, we just factored all logic into a centralized method which achieves exactly what we asked for.

But wait… . Wouldn’t the Page.Error event provide a better anchor? It’s already there and no need to add try/catch around every line of code. Look how nice:

protected void btnConcurrentUpdate_Click(object sender, EventArgs e)
{ 
    BusinessLogic bl = new BusinessLogic(); 
    bl.UpdateDatabaseWithConcurrencyIssue("some content"); 
    lblConcurrentUpdate.Text = "did it.";
}

protected override void OnError(EventArgs e)
{
    // raise events
    base.OnError(e);
    
    var ex = Server.GetLastError();
    
    // if we don’t clear the error, ASP.NET
    // will continue and call Application.Error
    if (HandleUserExceptions(ex))
        Server.ClearError();
}

Unfortunately at the time the error event is raised, the exception will already have disrupted the page life cycle and rendering has been broken. It may help in case you need a redirect, but not if the page should continue to work.

“Other” error handlers

So, the page handles user exceptions, what about admin and system exceptions? The page doesn’t care, so they wind up the chain and ASP.NET will eventually pass them to the application, via the Application.Error event, available in the global.asax. This is the “last chance exception handler” or LCEH for short.

One thing we did want to do was writing the exception into the event log. I’ll leave actually writing into the event log to you, just remember that creating an event log source requires respective privileges, e.g. running as admin. Just a minor deployment hurdle.

protected void Application_Error(object sender, EventArgs e)
{
    Exception ex = Server.GetLastError();
    
    // unpack the exception
    if (ex is HttpUnhandledException)
        ex = ex.InnerException;
    Session["exception"] = ex;
    
    // write detailed information to the event log…
    WriteInformationToEventLog(ex);
    
    // tell user something went wrong
    Server.Transfer("Error.aspx");
}

private void WriteInformationToEventLog(Exception ex)
{
    // an ID as reference for the user and the admin
    ex.Data["ErrorID"] = GenerateEasyToRememberErrorID();
    
    string whatToDoInformation;
    if (ex is AdminIsResponsibleException)
        whatToDoInformation = "Admin, you better took care of this!";
    else
        whatToDoInformation = "Admin, just sent the information to the developer!";
    
    // put as much information as possible into the eventlog,
    // e.g. write all exception properties into one big exception report,
    // including the Exception.Data dictionary and inner exceptions.
    // also include information about the request, the user, etc.

    …
}

As you can see, there are some details to keep in mind. One, the exception has been wrapped by the ASP.NET runtime, and that exception class doesn’t tell you anything worthwhile. Second, passing information to the error page may be a little tricky in cases where the request has not yet (or not at all) established the session state. The easier approach is to let ASP.NET handle the redirection, yet at the loss of information. Three, be careful regarding the requests: depending on the IIS version you may get only .aspx requests, or any request including static resources. Thus, with this simplistic implementation a missing image will send you to the error page depending on the IIS version and registered handlers.

I would also like to encourage you to spend some time on collecting information for the event log message. A simple ex.ToString() doesn’t reveal that much information if you look closely. You should invest some time to add information about the current request (URL, etc.), the user (name, permissions), and server state (app domain name, to detect recycling issues). And use reflection to get all information from the exception, especially including the Exception.Data dictionary. Also go recursively through the inner exceptions. The more information you provide, the better.

I also hinted at an error ID. It is probably a good idea to give the user some ID to refer to when he talks to the admin. This way, the admin will know exactly what to look for in the event log (especially in multiple event logs of a web farm).

Finally, the exception class tells us whether we should include an “Admin, do your job!” or “Kindly forward this issue to your fellow developer, brownies welcome, too.” message.

And by the way: This is the one and only location in our application that writes to the event log!

Statistics:

• Global error handler: Write once.

Done. We have the handlers in place. Handlers for user exceptions, admin issues, and any other exception that might be thrown. Now we need to take care that error causes are mapped to the respective exception class.

Business layer

The business logic is being called from the UI layer, and calls into the data access layer. Thus it has to check parameters coming in, as well as data handed back from the data source.

Data coming in hast to comply with the contract of the business class, and I’d like to stress the fact that not every data is legal. Meaning the business logic is not expected to accept every crap the UI layer might want to pass. A search method that takes a filter class as parameter? No harm in defining this parameter mandatory and require the UI to pass an instance, even if it is empty. And a violation is not the user’s problem, it’s the programmer’s fault! In these cases we throw an ArgumentException, and that’s it. The LCEH will take care of the rest, blaming the UI developer that is.

public IEnumerable<Customer> SearchCustomer(SearchCustomerFilter filter)
{
    // preconditions have to be met by the calling code
    Guard.AssertNotNull(filter, "filter");
    
    // input values have to be provided by the user
    if (string.IsNullOrEmpty(filter.Name))
        throw new UserIsResponsibleException("provide a filter value");
    
    DatabaseAccess db = new DatabaseAccess();
    return db.SearchCustomer(filter);
}

Any issue with the content of the data, i.e. the information the user provides, is a user issue. Usually this includes data validations and if you follow the rule book you’ll have to repeat everything the UI already achieved via validators. Whether you do that or just provide additional checks, e.g. dependencies between fields, is something I leave to you. Anyway, throw a user exception if the validation fails.

Calling the data access layer may also result in thrown exceptions. Usually there is no need to catch them at all. Period.

Finally the data you get from the database may ask for some interruption. A customer database may tell you that this particular customer doesn’t pay his dues, thus no business with him. In this case you throw a user exception in order to interrupt the current business operation.

And that’s it.

Statistics:

• Business validations and interruptions: As the business logic demands…

“Uhh, that’s it? And I used to spend so much time on error handlers in my business logic…”. Nice, isn’t it? Just business logic, no unrelated technical demand.

Data access layer

The data access layer is where many exceptions come from. If you call a data source (web service, database, file system, whatever), the call may fail because the service is unavailable. Or it may fail because the called system reports an error (constraint violation, sharing violation, …). It may time out. Some exceptions won’t require any special handling and you can leave them alone. Some need to be promoted to admin or user exceptions, or they may require special handling for some reason. You should  translate those exceptions to respective application exceptions. The reason is  that you don’t want to bleed classes specific to a data source technology (e.g. LINQ 2 SQL exceptions) into the upper layers, introducing undue dependencies.

Luckily all this tends to be as boilerplate as can be. Call/catch/translate. With always the same catches. So, again, we can provide a “once and for all” (at least per data source technology) exception handler, like this one for the ADO.NET Entity Framework:

public IEnumerable<Customer> SearchCustomer(SearchCustomerFilter filter)
{
    try
    {
        using (DatabaseContainer db = new DatabaseContainer())
        {
            //…
        }
    }
    catch (DataException ex)
    {
        TranslateException(ex);
        // any exception not translated is thrown on…
        throw;
    }
}

And the respective translation method:

static void TranslateException(Exception ex)
{
    // database not available, command execution errors, mapping errors, …
    if (ex is EntityException)
        throw new AdminIsResponsibleException("…", ex);
    
    // translate EF concurrency exception to application concurrency exception
    if (ex is OptimisticConcurrencyException)
        throw new ConcurrencyException("…", ex);
    
    // constraint violations, etc.
    // this may be caused by inconsistent data
    if (ex is UpdateException)
        throw new AdminIsResponsibleException("…", ex);
}

Statistics:

• Translation method: Write once.
• Catching exceptions: Every relevant location, boilerplate copy&paste

Conclusion:

Done. Let’s sum up the statistics:

• Write once (at least partly reusable in other applications):
  • Application exception classes
  • Last chance exception handler (LCEH)
  • Handler method for user errors, including presenting the error
  • Exception translation method in DAL
• Boilerplate copy&paste
  • Catching exceptions in the UI, call handler
  • Catching exceptions in the DAL, call translation
• Application code that still needs a brain
  • Business validations and interruptions: As the business logic demands…

And this is all we have to do with regard to exception handling in our application. I’ll let that speak for itself… .

This post concludes this little series about the state of affairs regarding error handling in an application. I do not claim to have presented something particularly new or surprising. Rather the opposite, this or some similar strategy should be considered best practice. The reason to even start this series was the fact that I regularly encounter code that does exception handling … questionably. And I also regularly met people who look at exception handling only in the scope of the current code fragment, not in light of a broader approach that spans the application.

If you need further motivation to employ proper error handling in the first place, look here.
A little more on error management in long running, asynchronous scenarios can be found here.

That’s all for now folks,
AJ.NET

Now, I cannot provide THE strategy, for there is no such thing. But I may present one feasibly approach that has served me well, quite a few times. I’ll simplify quite a bit, but it is nonetheless a complete approach that may serve as starting point for more complex demands.

OK, let’s play…

To define the playing field: The following is about classical n-Tier ASP.NET web applications, and I have decided to represent user induced issues, like business validation errors, as exceptions.

Some Groundwork

Exceptions are used for different error conditions, but also for non-erroneous situations, which leads to a surprising variety:

• Exceptions being thrown as result to infrastructure issues. E.g. IOException, SecurityException. They may be caused by invalid configurations, by unavailable infrastructure resources, or for other reasons.
• Exceptions representing errors reported by external services, like database constraint violations or SOAP exceptions.
• Exceptions being thrown because the caller did not meet the expectations of the called code, e.g. a violation of a contract. (Usually ArgumentExceptions).
• Exceptions being thrown because a developer’s lapse. NullReferenceException.
• Exceptions being thrown to sidestep the current flow of control, e.g. HttpResponse.Redirect() throws a ThreadAbortException to prevent an ASP.NET page from further unnecessary processing.
• Business validation errors. The business logic decides that – despite proper input validation – something is not quite as it should be. Not exactly an error in a technical sense, but the processing simply cannot go on.
• User concurrency issues. Like some user wants to update some data that has been changed by someone else in the meantime. And “last one wins” may not be good enough.

Please note that while Basic Rule Number 1 for exception handling (no abuse of exceptions for non-exceptional conditions) is as valid as ever, there are certainly exceptions (no pun intended) to that rule. For example, aborting the request upon redirection certainly makes sense; (ab)using an exception to do so may violate said rule, yet it is far more efficient and convenient than any other alternative. The same may be true for business validation errors. Not exceptional, yet exceptions may be the most convenient way to accommodate them.

This is a (perhaps surprising) variety and may even be incomplete – actually that variety is a major contributor to the complexity of exception handling. And yet, it is only half of the equation. Exceptions have cause and effect, and I found that tackling exception handling from the opposite angle, the effect that is, has certain advantages.

But before we come to that, there’s a question that has to be answered…

What is the purpose, the intention of throwing exceptions? (Seriously!)

This is a question one should ask at least once, even so it may seem a bit superfluous. For the answer does not include some of the usually mentioned suspects: Preventing data inconsistencies? Let’s simplify a bit and state that this is the job of transactions. Gracefully shut down? An error happened, what do I care how the application goes down.

Actually it’s far simpler: An error happened! Something which the developer didn’t anticipate or couldn’t cope with. The primary purpose of exceptions is to communicate that situation. And exception handling in turn it is about notifying everyone involved in the appropriate way, telling them what they are supposed to do now, either to compensate the issue or to prevent it from happening again. In other words: It’s about responsibilities emerging out of this error.

And this is what my strategy revolves around: Responsibilities.

An Exception Handling Strategy

Back to cause and effect, and let’s start by the later one. More to the point, the desired effect, driven by what is actually asked for in case of an error (rather than what could be done). This will lead to a strategy that is more simple, easier to understand, and easier to implement.

As I mentioned ‘responsibilities’ the first question is kind of obvious:

Question #1: Who should be responsible for what?

This is no esoteric question aiming at pieces of architecture or code. Rather it’s asking which person has to shoulder the work (or the blame, if you like):

1. The Administrator: He has to solve problems caused by infrastructure issues (such as invalid configuration entries, unavailable databases, and so on).
2. The User himself: Let him deal with errors caused by invalid input, other users working simultaneously with the same data, or in some other way inherent to the intended usage of the product. RTFM!
3. The developer (YOU!): Anything else falls back on the developer’s desk.

Obviously the last point is the inconvenient part: Every exception defaults to that category, unless the developer explicitly “tells” someone else that he is responsible. Let’s stress that: An error caused by an invalid configuration entry is the developer’s problem – unless he explicitly tells the administrator that he should take care of this.

So, now we are back where we started right at the beginning, at the developer’s desk. The difference is that the developer now has some specific goal: Lay the blame on someone else’s doorstep!

This is actually a very important point. Making someone else responsible is in the developer’s own interest. It helps him avoiding unnecessary and annoying future work. Quite motivating if you ask me.

This makes the next question obvious:

Question #2: How does the developer tell the user or the administrator that he is to take responsibility?

The answer is simple: Provide the necessary feedback, e.g. a message box or an event log entry; in more detail:

• In case of a user error you tell him with a message box, a popup, within some message area or on a special “you did wrong!” page.
  You don’t tell the admin, because he doesn’t care for typos. Neither do you as developer.
• In case of an infrastructure issue you show a standard message to the user, telling him to call the admin and come back later. You don’t want to include details, because they might include security relevant information, such as the always cited database connection string.
  Additionally you have to give the admin all information he needs, usually within the event log.

In both cases it’s important that you provide not only the information that an error has occurred (that much is obvious, isn’t it?). You need to include information on how to solve it. Otherwise it will eventually again be up to you, the developer, to solve the issue. That’s why it makes sense not to stop at “couldn’t save data” or “data source not available”. Rather invest some work – in your own self-interest – in providing “Customer data could not be save because it is locked by user ‘XYZ’” or “Database not available, connection string:=…”.

• Remember the last case: In case of technical issues for which the developer is responsible, you still have to tell the user and the admin something. The user should again get some standard error page (not the ASP.NET error page!), the admin should get an event log entry that tells him to notify the developer. This entry should include as much information as possible for a post mortem diagnosis.

So, essentially this paragraph revolved around presenting an exception, at the UI level where the information leaves the area of your application code. It’s quite simple to implement that, because you can rely on centralized features: ASP.NET provides a global error handler that can be used to provide standard event log entries. Presenting user errors on a page should be boilerplate as well, the only distinction being whether the current page and data is still valid (just show a message box) or not (redirect to some business error page). Nice, easy, and no rocket science.

Of course in order to do this we need a means of distinguishing those three cases. For this, all you need is respective exception classes, two actually. One BlameTheUserException and another BlameTheAdminException. You don’t need a BlameTheDeveloperException, because he is already to blame for every other exception, no need to stress that .

Question #3: Who to put exceptions to work… ?

If the developer’s goal is to blame someone else, then this goal can guide him answering the more detailed questions, such as, where in his code should he be catching exceptions (where not)? When should he throw exceptions (when not)? What about logging? And so on. Everything else follows as consequence…

First about throwing exceptions. Yes, not only the system throws exceptions; you may do the same, and no need to be shy about it. Sprinkle your methods with guard calls throwing argument exceptions, they imply coding errors on the callers side. If the code allows certain conditions that are logically impossible, guard yourself against that impossibility. Throwing exceptions in exceptional cases is no problem at all. Actually it is by far the preferred way to react this harshly, rather than obscuring erroneous conditions because of falsely understood modesty. Fail early, fail often!

Handling exceptions, as in solving the problem once and for ever is of course possible. If it is possible – which is in my experience rarely the case. (It had to be mentioned, though.)

Enriching exception information is a more frequent task. Whenever it makes sense for the post mortem diagnosis, you should add context information to the exception. Adding the actual SQL statement and the connection string to a database error, adding the user name and the accessed resource to security exceptions. This is valuable, if not necessary, to diagnose the root cause of the problem. It’s not even necessary to wrap the exception to do that, since the exception class has a dictionary Data for this very purpose.

Promoting exceptions: Once your code is able to decide that a certain error is an administration or user problem, you should wrap it in the respective class, thus “promoting” it.
This sentence implies the fact that not every location is suited to make that decision. The database layer should probably never decide whether a particular error is the consequence of some erroneous user action; this usually depends on the business context in which it is called, and should be decided there. Infrastructure issues on the other hand happen very low in the call chain and couldn’t even be decided further up.
The real work here lies in actually putting in the necessary handlers. However they tend to gather in classes that deal with the outside world, the database, configuration, some service, or whatever. They also tend to be very boilerplate.

That’s it. … Wait! … That’s not possible!

Question #4: Can that really be all?

Actually this is it. No “every method should…”. No “layer specific exceptions…”. No “log it just in case…”. Actually whenever I tell people how to implement an exception handling strategy in an existing application, the majority of the work is removing exception handlers. Sometimes life can be so easy.

In case it escaped you: If the developer doesn’t have to do something to employ proper exception management, he can’t do anything wrong. We don’t have to rely on him to properly handle his own bugs. We just solved a paradoxon!

Going further, I even consider it an issue by itself if you did too much exception handling!

For example “shouldn’t we at least log the error, just in case?” No, you shouldn’t. The global error handler is responsible for doing that, so there is no need to log it. Worse, if the calling code decided to handle the situation and carry on, there is by definition no application issue at all. And yet the event log would show one, waking the admins for no reason, or flooding the event log and obscuring the one important entry.

So what? That’s a strategy?

Yes it is. Check the demands the initial questions if you like, they’re all accounted for. Take some file that doesn’t exist. Either you did nothing to anticipate that situation, then an IOException will ensue. Neither user nor admin are taking care of this, so it’s back on your desk. Or you did handle it and threw a BlameTheAdminException.

Of course I simplified quite a bit, but not at the expense of the concept, only in terms of features. For example you may need to have different reactions for the user, some errors calling for a message box, other calling for a redirect. Similarly some unavailable service may allow retrying rather than requiring aborting the whole application. Or you may need additional exception classes to handle them in code. Anyway, nothing of this affects the strategy as such.

Once you’ve established such a strategy, every decision you make on the code level becomes easier, and more confident. And what’s more, in my experience it streamlines error handling, makes it more robust, and frequently simplifies the code by removing unnecessary exception handling. Because quite often if someone asks for exception handling, the answer is “already taken care of, nothing to do”.

Getting back to baseball, every player on the field now knows exactly what he is supposed to do. Some do all the work, most do nothing most of the time… .

PS: The ending anecdote: http://leedumond.com/blog/the-greatest-exception-handling-wtf-of-all-time/

That’s all for now folks,
AJ.NET

It’s actually quite frustrating how often that happens. What’s more, many developers don’t even notice that there is something amiss; they’re quite content dealing with exception on a method by method basis, without taking the bigger picture into account. Or they are very well aware that there is something amiss, but they don’t get it. They know they have to do something, so they set out to do a bit here, a bit there, but not with a clear goal in mind, nothing consistent, only aiming to treat the symptoms and quite regularly overdoing it.

I really don’t remember what first brought the analogy with baseball, of all things, to my mind. Perhaps it was the notion that ‘try’, ‘catch’ and ‘throw’ could be outcries during some ball game, combined with my lack of understanding of baseball (a bit stretching, but that’s how mind twists work). Perhaps it was the idea that many developers do exception handling like they would play baseball if they had never heard of the game.

Stretching or not, that analogy is something I can work with. (And my apologies to everyone who loves and understands that game for everything that follows! )

A Quick Evaluation of … Baseball.

Baseball, like all games has rules. Rules that put the various players into roles they have to fulfill. Most importantly, rules you have to learn if you want to play along. Some are fairly basic “mechanics” of the game: The pitcher throws in the general direction of the batter and the catcher does what his title implies (or so wikipedia tells me).
Some rules are obvious by looking at the game: If the batter hits that ball, he drops the bat and runs, while those guys on the field are trying to catch the ball.
Some rules are rather convoluted and you may have to study some booklet to grasp them; like what the hell is an inning and how do they determine who is winning?

That’s about as much as I understand about baseball. But then, would I want to play baseball, I knew I had a lot to learn before even thinking of entering the field. What rules apply to pitching. What roles do those guys on the field have, what are their responsibilities. How to score points. When to move on to the next round. When does the game end. Under what conditions do the rules change. What is considered foul play and when does it lead to aborting the game.

And this is a surprisingly similar situation to exception handling… . Except that many who play that game play it as if there isn’t even the need for rules, much less asked for them (or have been told about them – which is telling something, too).

Back to Exceptions

The mechanics of exceptions revolve around try/catch/throw and the implied syntax, exception hierarchies, and understanding the runtime behavior. Knowing that much can be expected from every developer and is usually not an issue at all. It is however only just the mechanics, and merely a starting point to proper exception handling.

In the same way the current situation in a game does play a role in baseball (e.g. the batter’s decisions on whether to play safely or to take some risk), the location and situation of a particular code fragment plays a role in the decision on what to do with regard to exceptions. Dealing with them only in the scope of the current code fragment is simply too shortsighted. One has to ask about the overall application strategy to exception handling and how that respective fragment fits into that strategy.

The caveat is: You have to have such a strategy. And defining that strategy should be part of coming up with an application architecture. Yes, exception handling is an architectural topic, not a coding issue.

Exceptions vs. Errors vs. Complexity

As I said, exception handling is an aspect of static and dynamic application architecture, just like security, transaction handling, state management, whatever. But there’s one thing that sets exception handling apart, and that’s dealing with the unexpected, the contradiction of having to deal with what you didn’t think of in the first place.

Exceptions come into play under three different conditions:

• First, as a means to break the current control flow, say to abort a request upon redirection to another page. This is intended behavior in a regular situation, not even an error.
• Secondly in cases that disrupt the logically correct operation, but you actually planned for that disruption. Say you rely on a file to exist but you know that someone might screw up, so you handled that case, despite the fact that during regular operations it shouldn’t happen at all. This is an error, but an intended behavior in an expected irregular situation.
• The third condition is the worst of all: Something unintended happens, something you cannot do anything about. It may arise as ArgumentException (telling you that the developer screwed up) or may be as TypeLoadException (telling you that the environment is on quicksand). In any case this is a purely unintended error and the only thing you can assume about the state of your application afterwards is, that it is not safe to assume anything.

An application architecture should provide provide rules on how to handle these conditions. Add questions like how to deal with the consequences, or deal breaker like parallelism, asynchronicity, or loose coupling, as well as reasonable demands regarding data consistency and error compensation… . And this becomes quite a complex task.

The paradoxical situation however is: Complexity is the very source of errors. Adding complexity is like pouring gasoline into the fire. As developers we already have to deal with the complexity of the problem domain. An exception management strategy should kick in if we fail in that regard. And if he is already doing something wrong, how can we rely on him to do something other (that isn’t even in his line of thinking) right? We can’t!

So, a good exception handling strategy needs to solve that contradiction: The need to deal with different exception conditions on one side. Making things easy for the developers, at the same time not relying on them to do things correctly on the other. Not a small task, indeed. And perhaps this is the very reason that exception handling is usually dealt with so poorly.

Stay Tuned…

OK, this was a lengthy introduction and the actual story is even longer. However I thought it prudent to point out the bigger problem and generally raise the awareness for the topic. Of course I also enjoyed playing with that baseball analogy… . Anyway, in the interest of some readers not interested in such musings I split this one in two distinct posts. This one to explain the problem and make you curious, the next one to provide a solution. Stay tuned if I caught your attention.

PS: Special thanks to Karl for the permission to use his pictures…

PPS: There’s actually more to say about baseball than just baseball. Have a look here, German readers may find a more thorough explanation here. And no, it’s got nothing to do with software development

That’s all for now folks,
AJ.NET

During the last 4 weeks a team of my colleagues has put together a working application, simulating an actual business scenario. Starting at point zero, coming up with a meaningful idea – in terms of a cloud solution, and putting together a working application, is quite some achievement. Congrats, guys.

Now business scenario sounds a little drab, and banking, especially related to credits, may not actually cause people to applaud hilariously these days. But frankly, since the winner is decided by a community voting and we never thought about that application as particularly “sexy”, we never aimed for that goal. Our aim was to demo the potential of the cloud. Nothing less.

And that’s what the application does. It takes big amounts of data (storage), runs advanced statistical calculations (Monte Carlo simulations) (compute), and delivers the results. Since these are peek time usages, we may use several worker roles to scale out the calculations, cutting back when they are not needed (dynamic provisioning). Exactly what the cloud is about.

And what I really do like is that it doesn’t stop there. Service Bus may be leveraged to integrate this application with “local” solutions, calling back when the calculation has finished. SQL Data Services (once they become available with reporting) may be used for longtime data mining and trend analysis. One can even think of producing information needed to calculate a single credit contract and using Live Mesh to distribute it among the sales people.

So, check it out: http://ratingsimulator.cloudapp.net/  

That’s all for now folks,
AJ.NET

The main new things within VSArch include the Architecture Explorer, UML Support, and the Layer Diagram.

Architecture Explorer

Note: To make the following more tangible I loaded a sample project I use regularly as test harness and took respective screenshots while I analyzed it. Click the images for a larger view…

The Architecture Explorer is about getting a better view into existing code. Whether you join a project that is under way, whether you have lost control over your code, or whether you just need to spice up your documentation. Architecture Explorer helps you by visualizing your solution artifacts and dependencies. Artifacts include the classical code artifacts (classes, interfaces, etc.), as well as whole assemblies, files, and namespaces.

Architecture Explorer lets you select those artifacts, display graphs with dependencies, and even navigate along those dependencies and in and out of detail levels.

The following screenshot shows VSArch. The docked pane on the bottom contains the Architecture Explorer that acts as “navigation and control center”. This is where you select your artifacts and visualizations. It could certainly use some improvement from a usability perspective, but it does the job anyway.

The screenshot shows two different visualizations of the assembly dependencies in my solution, a matrix view and a directed graph. Just to stress the fact: This was generated out of the solution, by analyzing the project dependencies.

The next screenshot shows a mixture of various artifacts, including classes, interfaces, even files, across parts of or the whole solution.

Depending on what filters you set, this graph could give you a high level overview of certain artifacts and their dependencies. For example you could easily spot hot spots, like the one class your whole system depends upon. Or make sure the dependencies are nicely managed via interfaces and find undue relationships. Even spot unreferenced and therefore dead graphs.

Once you go one level deeper, you may want to cluster the artifacts by some category.

The image shows again artifacts and their dependencies, but this time grouped by the project to which they belong. It also shows what kind of relationship a line represents and lets you navigate along that dependency.

The Architecture Explorer should help getting a better understanding of your code. It helps you to detect code smells or may guide your refactoring.

UML Support

Yes, UML like in, well UML. Not extensively, but it includes activity diagram, component diagram, (logical) class diagram, sequence diagram, and use case diagram. I didn’t spend much time investigating them, just drew some diagrams in order to take the screen shots. Generally I can say that Microsoft can draw boxes and lines (big surprise here) but there is a lingering feeling that those diagram editors may not be finished yet (again, hardly surprising on a CTP).

Creating a new diagram is easy enough. Just create a new project of type “Modeling Project” and add an item:

Everything starts with a use case, so here is our use case diagram:

One can draw the diagram as he likes. As you can see from the context menu, there is something being worked on. Namely the “Link to Artifacts” entry shows the Architecture Explorer, yet I couldn’t quite figure out what’s behind this. Also note the validate entries which didn’t do very much, but we’ll see them later in the Layer Diagram.

Next on the list is activity diagrams:

Works as expected, no surprises, no hidden gems that I’ve found.

The same is true for the component diagram:

Just a diagram, no surprises.

The logical class diagram gets more interesting:

As you can see, it contains very .NETy stuff like enumerations. It also has these menu entries that hint on more to come in the future — right now the selected menu entry brings up the error message asking for a stereotype, yet I didn’t even find a way to set those. Also the editor may still need some work, e.g. one cannot drag classes in and out of packages.

As a side note: The relation between this logical class diagram and the already existing class diagram escapes me. At least they are a little redundant.

Next on the list is the sequence diagram. Rather than drawing one myself I reverse engineered the existing code:

Quite nice and again, used this way it can help you documenting or just plain understanding existing code.

Note: If you want to try that yourself, the CTP has a bug: You need to have a modeling project and at least one diagram before the menu entry “Generate Sequence Diagram” appears. And while you will be presented with a dialog asking what call depth to analyze, it usually works only for one level.

Layer Diagram.

Now for the most dreadfully looking diagram (though Microsoft has a more colorful one on its site…): Some boring connected blocks, meant to represent the layers of your architecture.

Actually this is one of the most interesting features for any architect and dev lead: It’s a living beast!  

You can add assemblies as well as other artifacts to the bleak boxes. Afterwards you can actually validate whether the dependencies between those artifacts match or violate the dependencies implied by the diagram. In the screenshot you can see that I deliberately misplaced an assembly and consequently got a respective error. Using this feature an architect can ensure that all layer related architectural decisions are honored during development.

To conclude…

The Architecture Explorer is certainly a worthwhile feature and I also like the validation feature of the Layer Diagram. That’s certainly something new and not to be found in other products.

Generating sequence diagrams is nice but it remains to be seen whether this will allow roundtrip engineering. The logical class diagram doesn’t yet meet my expectations and it’s not quite clear to me how it will evolve. The other diagrams? Well, they just work. However in this group is nothing exciting for you if you already have another modeling tool like Enterprise Architect (no advertising intended, just happens to be the one I’ve used recently…). And a dedicated tool probably will provide a more complete UML coverage. UML 2.0 has 13 types of diagrams, including state diagrams, which is in my opinion the biggest gap in VSArch UML support.

Anyway, if that caught your attention and your interested in more details there are two options: One, download the CTP and try for yourself. Two, if you want it more condensed and avoid the hassle with a VPC, watch a video with VSArch at work. For that there are two links I can provide:

1. Peter Provost’s talk at the PDC. Go to the timeline on the PDC site, search for TL15 and you should find “TL15 Architecture without Big Design Up Front”, which is about VSArch, despite the title. His talk was the role model for my analysis of VSArch, yet seeing it live could still give better insights.
2. Visual Studio Team System 2010 Week on Channel 9 has a bunch of videos, especially the “Architecture Day” ones. “top down” and “bottom up” show VSArch at work.

The final question however will be if all those features are compelling enough to actually buy the … Visual Studio Team Suite (i.e. the “you get everything” package). Why not the Architecture Edition? Well, if you are a developer as well as an architect, the Architecture Edition lacks too much in the other areas. Given that there is usually quite a monetary gap between dev edition and team suite, that gap might very well be used to buy a 3rd party UML tool instead… .

That’s all for now folks,
AJ.NET

Doesn’t today’s multithreaded software profit from more cores automatically?

The answer is a definite yes and no. Server software like IIS or SQL Server has always been optimized for throughput (performance only secondary) and as long as the processing is CPU bound it will certainly put more cores to good use. That should result in better throughput (i.e. more requests handled at the same time), but not necessarily in better performance (i.e. the time to complete one request).

The more interesting observation however is on the client: Client applications follow different use cases for multithreading, actually mainly two of them:

1. Avoid blocking of some task while processing something else. Examples include keeping the Windows UIs responsive (i.e. work around the UI’s thread affinity), Windows Services (their service control handler is invoked by the SCM and has to return in a certain amount of time).
2. Be able to spend processing time on something valuable while one task is blocked waiting for something to happen. Blocking regularly happens during some kind of I/O, especially network calls. This is the „call WebService asynchronously and do something worthwhile until the call returns“ scenario.

Just different sides of the same coin actually. Please note that neither use case has been about actual performance, as in doing things really faster. The first one is about perceived performance by reducing latency for some favored task, the second one improves performance, albeit not by doing things faster but by avoiding unnecessary wait times. Anyway, this works quite well even (or rather especially) on single core machines.

Let’s dig deeper into an example: Say a lengthy CPU intensive calculation is triggered by a button. The time spend on updating the UI and doing the calculation is denoted in the following picture, the red line represents the executing thread:

The calculation is done on the UI thread, giving it peek performance, but at the same time freezing the UI until the calculation has finished. The typical “optimization” is putting the calculation on a second thread, e.g. using the BackgroundWorker component. That way, the UI keeps updating itself rather than degenerating to one of those white and unresponsive windows:

The UI and the calculation run on different threads (yet still on one core), thus the UI can update itself even during the calculation is still running. However, now it has to deal with that intermediary state (denoted by the striped blocks). And every time the UI thread uses the core, that time is lost to the calculation, so the time to completion will actually be longer than with A.

Now switch gears to our new, say quad core, machine…

What happens on a quad core is that both tasks now get executed in parallel:

As you can see, actual processing time is now back to the same it was with A, yet it manages to avoid freezing the UI as in B; again at the expense of having to deal with the intermediary state. However that’s about as good as it becomes. But the application doesn’t get faster, neither will a third or fourth core be utilized at all (other than ensuring that other applications can run there rather then interfering with the cores we just occupied).

So, while the potential in “classical multithreading” lies in shifting or arranging calculations of disparate tasks, the potential of parallelism with many cores lies in doing more calculations of CPU bound tasks at the same time. Like so:

This time the calculation has been split into parts and been distributed on different threads on top of the UI thread. This eventually caused better performance than A. However this is not the way today’s multithreaded applications are written, because on a single core it actually leads to worse performance, due to memory pressure if used excessively, thread context switches, synchronization and contention, etc..

The verdict…

Today’s multithreaded client software may profit from more cores, but to a far lower degree than one might think at first. And what’s more: It might even stop running altogether because actual parallelism opens the door for error conditions that were simply theoretical threats on a single core system but become a reality on multi cores. Something like accessing some memory while another core is still in the middle of an instruction doing the same.

PS: And by the way: the future is now!

I just started in a new project and got my new machine last week: An HP wx6600, with 2 quad cores @ 3GHz, 4GB

This is a task manager screenshot, showing 8 nice little cores, sitting under my desk, waiting for me to send them on one or the other errand…

A single core is such a lonesome entity…

That’s all for now folks,
AJ.NET

Last week we had some customer events, containing some talks about PDC and other stuff and how it will affect the near term future. Among other things I tried to describe the many core shift as well as its consequences. Curiously the audience was largely aware of the facts, yet the consequences had yet escaped the vast majority. So let me try to repeat the gist and see whether this is maybe a common symptom…

What’s the many core shift, anyway?

Moore’s Law states that the number of transistors on an integrated circuits doubles every two years. Until not long ago, and accompanied by more complex designs and higher clock speeds, that meant faster CPUs. This was sometimes called the „free lunch for developers“, because if one happened to write a slow application… not that anyone ever would , all he had to do was wait two years and it ran twice as fast.

However this evolution has reached its physical limits (clock speed, power consumption, etc.). Yet, still the number doubles… .
So instead of building faster and more complex CPUs, the manufacturers started placing more CPUs, read cores, on a chip. It started 2006 with Intel’s dual cores, today you won’t find a single core desktop machine anymore. High end consumer machines come with quad cores, and servers with 16 cores (delivered as 4 quad cores). Have a look at the extrapolation:

The “lower” line shows cores on a CPU, starting 2006 with 2 cores, while the steeper one assumes 4 CPU sockets. And just in case the conclusion escaped you: Five years from now we will have between 32 and 128 cores. And remember, we are talking “consumer grade stuff”, that is the box under your desk, not something special! Impressive?

So that’s the many core shift. But what does it mean?

Well, it probably means that today’s software runs a bit faster. Not much, mind you, certainly not the 32 times faster a 64 core machine is supposed to be compared to my dual core. Why is that? Well, have a look at the following task manager of a 64 core machine:

(That’s a fake of course, but have a look at Marks’s blog for a real one.)

Now look at your own desktop and count the open applications. Outlook, Word, perhaps PowerPoint, Internet Explorer, Acrobat Reader? OK, say half a dozen applications, add 10 more for the OS stuff actually doing something. That’s 16 applications, using the upper row of cores, perhaps even to 100% and yearning for more, while the other 3 rows just sit there and twiddle their thumbs. That sad truth is: Most of today’s applications simply are not capable of employing these cores appropriately. Consequence: In order to leverage these cores we have to change the way we write our software!

Two questions come to mind: Do we actually need that kind of processing power? And if so, how do we open it up?

Seriously, does the average user need 64 cores?

Well, yes he does. If for no other reason, he did need the increase in processing power during the last years, why should that change?

So, what does he need it for? Gamers are always at the forefront of processing power demand. We have the generally increasing demand in UI technology: 3D, animations, visual effects are becoming mainstream with windows, WPF and Silverlight. The trend to digital photography has had its effect on the demand for graphics software. On my dual core, DXO needs about 1 hour per 150 pictures, so there’s certainly room for improvement (I brought 2500 pics from my last vacation in Tanzania. Do the math ). Background encryption, compression, virus scanning, etc. also add up.

Even if you are an “ordinary business user”: Word just sits and waits for your input most of the time? Well, open a non trivial 100 page document and see how long Word takes for pagination, spell checking, or updating the TOC. Change the formatting and watch again. So while Word mostly does nothing exciting, there are „burst times“ when it could really need those cores.

And I did not even mention Visual Studio and the compiler yet…

How do I, Joe Developer, put 64 cores to good use?
And how do I make sure the app doesn’t degrade on an old dual core beyond reasonable limits?

Here we are right at the center of the problem: Multithreading is not exactly something new, we’ve had that for more than 20 years on PC’s now. So why do I even have to ask that question? It’s because we didn’t actually use multithreading within our applications if we didn’t have to. Because it’s laborious, error prone, awkward. You have to deal with thread synchronization, race conditions, dead locks, error management, communication between threads. You can’t debug it, tracing doesn’t help very much either. In short: It’s a pain in the …, well.

So let’s face it: Most developers have avoided multithreading altogether (perhaps the lucky ones). And those who did do multithreading probably did it just for optimizations in very distinct areas.

But what we need to leverage those cores is quite the opposite: We need multithreading to become mainstream, kind of ubiquitous. For that it needs to be easier to employ parallelism. Complexity has to be pushed out of our code into the platform. Somewhat like nobody thinks any longer about virtual memory (while some of us are old enough to remember the days of physical addressing).

In other words: In order to deal with the parallelization demands, we need new patterns, libraries, and tools.

Microsoft is going to give us a first delivery on that with the next Visual Studio 2010 and .NET wave. Optimized runtimes (e.g. the thread pool), better tools (e.g. the debugger) and not the least, new libraries, introducing new patterns (e.g. Task Parallel Library). There’s more in the unmanaged world (e.g. Parallel Patterns Library), more on the server side (CCR), more on the language front (F#), more in the research area (transactional memory).
Microsoft even devoted a whole “developer center” to parallel computing (look there for more details). And quite rightly so, because there is no single solution to parallelization, it comes in different flavors (e.g. data parallelism vs. task based parallelism) and we can expect further developments in this area in the future.

Also it’s noteworthy that the OSes, namely Windows Server 2008 R2 and Windows 7 which share the same kernel, can manage 256 cores. Compared to what they supported before this is quite a jump.

Conclusion

So, parallelization is here to stay and we are going to have to deal with it. If anything, the trend is going to accelerate. It’s reasonable to assume that eventually processor manufacturers will trade single core performance for number of cores, i.e. put more but less capable cores on a chip, in order to save power consumption (green IT and mobility being two other major trends).

Looking even further, the many core shift may reach a break even where standard desktop systems will cease to profit from additional cores (how parallel can you become after all?), the problems of memory access may limit the amount of cores. Asynchronous multi cores may evolve, e.g. having cores optimized for certain tasks…

See, there’s a lot to look forward to. Our profession certainly remains interesting

PS: See Being Parallel? for some thoughts on whether today’s multithreaded client software will profit from additional cores…

That’s all for now folks,
AJ.NET

Not that it is not necessary to do so. It’s just that I expected a lot of bloggers, especially from Microsoft itself, trying to spread the news and foster understanding of what is ahead of us. Well, the Microsoft folks became kind of hushed, as if ducking down and counting the shrapnels after having thrown the bomb at the PDC. So I changed my mind…

Preface…

Given what Microsoft unveiled at the PDC — A new vision, a new strategy, a new technology stack, and a mish mash of existing, sometimes overlapping, not yet consistent, much less complete applications and services — it’s no wonder that it took me some time to grasp the idea. And when I thought I might have understood the gist I was still, well, unsure of whether I had gotten it all right.
The breakthrough came at the ask-the-experts. More to the point I had the chance to talk to Ori Amiga (the guy that did the “BB04 Live Services: A Lap around the Live Framework and Mesh Services” talk). Other than joking about the various ways to pronounce “Azure” and the fact that Americans always manage to get it wrong (sorry guys — and sorry Ori, hope I didn’t give you away too badly ) ), this little chat really turned “suspected functionality” into “understood technology” (at least I do hope so…).

Explaining Azure worked not on explaining the ever present Azure picture as is. It worked on developing the pieces of the picture, bit by bit, and relating it to other concepts. And since I worked for me, I thought I might share the gist of that conversation in much the same style, hoping I’m providing new insights rather than reiterating already available information. Actually I’m going to use the very sheet of paper we (mostly Ori) drew throughout said discussion. 

Enough of the preliminaries, here we go…

First of all, keep in mind that there’s two Azures: Windows Azure and the Azure Services Platform. They are not the same and neither presents the full picture. I’ll try to dissect that picture layer by layer, like a cream gateau…

It starts with the cake bottom: Windows Azure

Windows Azure is the basic “infrastructure” (to avoid the term “Operating System” for now) to run applications on, highlighted in the following picture in red:

That includes computing capabilities, basic storage, management of applications (i.e. deployment, including upgrading), and operations (e.g. handling failures). These concepts are abstractions from the underlying OS (Windows 2008 actually), machines, and storage devices.

The terms to think of are service instances rather than processes or (virtual) machines. This is similar to the way virtual memory abstracts physical memory. While any memory access obviously has to happen on physical memory, the virtual memory manager is free to relocate it, even to swap it out to disk. This not only makes applications independent of the amount of physical memory, it also optimizes resource usage, allowing other applications to use the memory my application has claimed but does not access at the moment.

Yet, while Microsoft Azure is one level of abstraction above the machine’s OS, it has similar concepts (highlighted in the above picture in blue):

• computing ~ job scheduling, etc. (NT Kernel);
• storage ~ file system (NTFS);
• management ~ application installer;
• operations ~ task manager, event manager, etc..

Thus it is quite feasible to call Windows Azure an Operating System for the datacenter (or for the cloud if you’re a marketing guy), even if that may not match exactly what you learned in university about operating systems.

The implications of deploying an application to Windows Azure (i.e. how the application has to be built and how the fabric manages it) is actually quite interesting … but a whole new blog post, thus I will skip that for now.

Let’s move on to the second layer of the cake: The Azure Services Platform

A barebone OS like Windows Azure would be of limited use if it were not completed with other general purpose services. Which services exactly that includes may be debatable, yet, again, the similarity with our local environment may help depicting the features we as developers have come to expect from the platform we are developing on: database service, user accounts, IPC, etc..

Again, the following picture highlights the services in red and similarities in blue:

Microsoft decided that the following services may be good ones to start with:

• .NET Services: basic infrastructure services for application security (access control), application communication (Service Bus), and workflow (three guesses…?).
• SQL Services: database related stuff; not exactly a SQL Server, but aspiring to be…
• Live Services: All around social applications (community, devices, etc.)
• Core application services: This is a set of higher level application services, such as SharePoint Services and CRM Services (explicitly not including the UI!). In my opinion they are there because they were readily available, not because they are particularly necessary.

Oh, and not to forget the reoccurring three dots in the PDC slides; those dots tell us that these are not closed and sealed sets. Actually Microsoft said that every major server application will eventually be made available on Azure.

Now for the chocolate and the cream: the applications running on the Azure Services Platform

While Ori included the application layer in the platform, any PDC slide puts it on top:

Where you put the label is of no consequence anyway, because this is no more than a logical hierarchy. However, please don’t misinterpret this hierarchy by assuming that those applications have to run on Azure and only applications running on Azure can leverage the Azure services! Au contraire!

If you have an application deployed on Azure it is (technically speaking) no different from other services. The difference only lies in the purpose, or the consumer if you wish. And still applications and services are free to call any services, not just those running on Azure. Likewise if your application is running on your local machine or network it can use services deployed on Azure to store data or integrate with whatever, that’s fine as well. Actually the best example for this flexibility is coming from Microsoft itself: Live Mesh.

The cherry on the cream: Live Mesh

Technically speaking the Live Mesh Desktop and Live Services are just another set of applications and services running on the Azure Services Platform, complemented with applications running somewhere else and using those services. This limited view however would miss much of the capabilities of Live Mesh, and the way it enhances the platform.

Live Mesh aims no less than to connect people, devices, and applications. Live Services contains services for identity (LiveID), presence, etc., and Mesh Services to maintain users, devices, applications, and — a corner stone — synchronization. The resource model organizes mesh objects (data, news, etc.) in feeds and entries, which in turn are subject to synchronization among the applications being “deployed” to Live Mesh. “Deploying” an application means either actual deployment on Azure, or storing it for (seamless) installation on your local device, via Live Mesh Client, offline capable if built to be.

Still with me? Well, to make this mess, pardon mesh, a little more tangible, let’s recap the example of Ori’s talk (PPTX WMV, images taken from there):

He had his media center PC connected to the Live Mesh, advertising its meta data, like favorites, recordings, etc.. That information was synchronized to the Live Desktop (running in the browser), so he could pick a TV show there and “start” recording. (Actually that “start” was some little piece of data, synchronized back to the media center PC which in turn did, surprise!, start recording.) He then started a locally installed application that showed the TV guide and had the typical red recording sign right at the respective TV show. That application was offline capable, so he could have planed his TV recordings on the airplane and have it synchronized when he gets back online. Finally he also showed the same TV Guide on his mobile phone simulation.
The only thing missing was integration with other people, but I think it was in a key note where they showed an application that allowed sharing of film critics with some friends.
All in all, that’s what I call ubiquitous computing!

Spicing the cake: Developer tools

This part is actually not in the picture, but it’s no less essential: Where does your application or 3^rd party code fit in? How does it get there? Those parts in red may be your application or service:

It’s actually quite easy: You can write applications that run on Azure and provide services (just like the basic services Microsoft provides), or an UI (just like Microsoft’s applications). You can access any of those services from the cloud or your local application, no matter whether Microsoft provided it or someone else. And you can Mesh-enable any of these applications and services as you like. This is an open platform!

Also Microsoft provides a simulated local environment for Azure, called development fabric, along with Visual Studio integration. Thus it is possible to develop your application locally, test it locally, and only afterwards deploy it to the cloud.
Regarding Live Mesh Matthias has more information on developing with Live Framework.

Postscript…

That’s it. You can find the complete untampered drawing here. That scrawl at the bottom of the drawing is actually Ori’s signature, but you should attribute any error, misunderstanding and adverse opinion in this post to me.

Finally two links for some alternate explanations (already repeated a thousand times over, but what the heck, they’re good):

• “Manuvir Das: Introducing Windows Azure”: Manuvir explains a little deeper how Windows Azure works.
  http://channel9.msdn.com/posts/Charles/Manuvir-Das-Introducing-Windows-Azure/
• “Windows Azure (aka “Red Dog”) explained in 145 seconds”: funny and enlightening (presented on a site you might find interesting as well)
  http://www.azurejournal.com/2008/11/windows-azure-aka-red-dog-explained-in-145-seconds/

That’s all for now folks,
AJ.NET

Now for this fool’s current view…

I’ll not try to explain it in technical details (not this time anyway, start at here or here to come up to speed — anyway, there are probably books being written about it right now). And I’m certainly not going to join the debate, whether Azure is an operating system or not.

I was still at the PDC when I started a Google search to see what reactions Azure triggered. That particular question was a predominant topic — and utterly without any consequence. (I couldn’t find that again though, it’s buried under millions of near identical pages describing Azure.)

So let’s talk about what Azure is about. For me, for you, for the companies we work for, for the industry.

Hailstorm — or what Azure is not…

When I first heard about some of the Azure stuff I immediate thought “Wait, didn’t they announce something very similar in 2001? Called Hailstorm a.k.a .NET Building Blocks a.k.a. My Services?”. It turns out that I was not the only one with this particular déjà-vu…

Taking a closer look, however, there are some significant differences in strategy between Hailstorm and Azure: Hailstorm was about Microsoft providing services to manage your data. Azure is about providing a platform for your services. In other words: With Hailstorm Microsoft said “give us your data and identity, keep the apps”, with Azure they say “give us your apps, you may keep the data and the identity (or you may choose to let us handle that as well)”. This is certainly a noteworthy change in attitude.

There’s another thing that changed since 2001. In 2001 Hailstorm was a new approach. It was visionary. It was the first time that someone asked me to let him manage my identity, appointments, even my wallet! And it was the evil empire that asked this very question… .
Today, hosting providers are a common thing and trusting Google or online communities with personal information is quite normal. Going even further, Amazon already offers a service platform with AWS, based on virtual machines (EC2), storage (S3 and SimpleDB), and a queuing service (SQS) as messaging bus. Google App Engine provides scalable application hosting (Python apps) and complements that with their identity system (Google Accounts). So, the good news for Microsoft is that they are for once not the bad guys. Of course, Microsoft being Microsoft, that probably only means that they will be accused of being the copycat again… .

What Azure may eventually become…

Still, what’s the big deal about Azure? I mean, doesn’t my hosting provider offer virtual or dedicated servers? Aren’t there enough storage offerings, many of them for free? Aren’t there Internet communities, online applications, service providers, …?

Well, looking at Azure piece by piece, there’s nothing new, nothing especially exciting. But take a step back and look at the big picture, at the strategy. With Azure…

Microsoft becomes a service hoster and operator. Not just a server hoster. They don’t stop at booting a VM. They operate your applications, load balance them, restart them on failures, scale them, provision them, provide an upgrade mechanism that doesn’t disrupt 24/7.

Microsoft becomes an application service provider as well as a service provider. They do not stop at providing fully fledged business applications like CRM or other. They offer basic infrastructure services like SQL Data Services or Workflow Services. You could even build your own CRM System, run it on Azure, competing with Microsoft’s CRM — and Microsoft couldn’t do anything about it (for if they would, that would put them out of business at the speed of light).

Microsoft becomes a service integrator. They don’t stop at getting applications into the cloud. They offer bridges to your company’s private network. Using Access Control you can leverage your local AD to control security. Using Service Bus you can integrate with any application on premise or in the cloud, even let business partners integrate with you.

Microsoft becomes a device integrator. They don’t stop at central storage or some device specific app for synchronizing contacts and appointments. The let you sync any application data on any Live enabled device on the fly, including handling offline cases.

Microsoft becomes a people integrator. They don’t stop at providing some social application. They allow you to collaborate with other people within your own application. “People” being users authenticated by your AD, LiveID, OpenID, and probably any other well-known ID provider. All based on “claims” in order to build trust between app and user.

And most important: They do all this at once! Putting it all together Microsoft offers a far more complete, comprehensive, and concrete vision of what the hazy term “cloud computing” means than anybody else out there. It’s the vision of interconnected, intertwined applications, devices, networks, … the next level of ubiquitous computing.

OK, I may have been carried away. Currently Azure is rudimentary, not yet version 1 (and we know what version 1 usually means at Microsoft) and many questions remain unanswered. I cannot help but think that the mixture of various existing apps and services with hosting and a big bunch of announcements, all wrapped up in a nice name, is more like a stew, thrown together but not yet readily cooked.

But then I might not. After all, Ray Ozzie said it loud and clear:

“[…] the systems that we’re building right now for cloud-based computing are setting the stage for the next 50 years of systems, both outside and inside the enterprise.”

and

“[..] we’re betting on Azure ourselves, and as the system scales out, we’ll be bringing more and more of our own key apps and key services onto Windows Azure because it will be our highest scale, highest availability, most economical, and most environmentally sensitive way of hosting services in the cloud.”

That’s what I call an announcement.

Does Azure stand a chance?

Cloud computing and Software-as-a-Service is widely seen as a coming trend, Azure or not. Microsoft is entering this market with vision, data centers, and service offerings. And a huge developer community and tooling support. This will change the IT industry and not many companies will be able to compete with that.

But a small statistic may be funny:

Google search for „Windows 7“: about 35,000,000 hits
Google search for „Windows Azure“: about 1,830,000 hits
Google search for „Amazon AWS“: about 13,000,000 hits
(measured 12. Nov.´08; add one for this post )

That’s 20 times more hits for Windows 7 than for Azure. Quite a poor result for the thing that denotes a change in strategy for the worlds biggest software company, the thing that Microsoft is betting its future on.
On the other hand, 2 million hits is “not too shabby” just two weeks after its announcement on the PDC. And it starts to shine if compared to the mere 7 times more hits for Amazon’s AWS, the offering that’s probably the most similar to Azure and one that dates back to 2004(!).

My opinion? Don’t expect Azure to play a big role next year or the year after. But then, don’t expect it to play no role in 10 years.

That’s all for now folks,
AJ.NET

There are two defining characteristics of LINQ 2 SQL that may affect the architecture of a web application:

1. The DataContext class, being “the main entry point for the LINQ to SQL framework.” (as the documentation states.)
2. The way code generation works.

While the first point affects what features of LINQ 2 SQL we are able to use in web applications, the second one affects the application architecture.

Note: if you need to catch up with LINQ 2 SQL have a look at the my earlier post.

DataContext Effects

In this post I’ll focus on the DataContext class generated by LINQ 2 SQL.

The DataContext class is not only an anchor to access the tables, views, and stored procedures. It also offers more interesting features like change tracking, maintaining object identities, optimistic concurrency, and deferred loading. Consequently this is what most people I talked to attributed to LINQ 2 SQL: The possibility of reading some data, manipulating it, and call SubmitChanges, while LINQ 2 SQL handles all the gory details of mapping the data, efficiently talking to the database (including load on demand), dealing with concurrent changes, and so on.

This read/manipulate/submit approach is what the documentation describes as being “designed to last for one ‘unit of work’.” And in case the consequence escaped you: This this implies a statefull programming model. The very DataContext instance that was used to read the data has to be available until every potential change or deferred access has been settled.

This does generally not agree with web applications, as read/manipulate/submit regularly spans several requests: The data is read, presented to the user, and then the request dies, and in a stateless scenario the DataContext as well. The next request comes in, say an update request, but there is no DataContext available, carrying information about the data that was read before. In order to work with the laid out programming model, one would have to create a new DataContext, reread the data, and then apply the changes and submit them. With optimistic locking being the first victim, because for that the information from the previous request is mandatory. And delayed loading degenerates to “optional loading”, because either you need the data (no benefit in delaying then) or you don’t.

How about maintaining the DataContext instance across requests? No way either; since it is not serializable, you can’t put it into the session. (And that’s just a technical reason, whether one should do that even if it were possible is a different matter altogether.)

To make a long story short, let me rephrase and emphasize the first sentence:
For web applications the generated DataContext class is only an anchor to access the tables, views, and stored procedures.

BTW: Services are frequently bound to be stateless, too. Thus the same reasoning applies to them as well. Additionally they face another problem: They hand out the entities and get fresh ones from the outside world via serialization. Thereby they are removed from the DataContext, which is kind of unforgiving.

Less is More

Now we have concluded that the statefull approach doesn’t work. Reverting to a stateless employment of LINQ 2 SQL however leaves out a lot and raises the question whether there is something left at all. Well, there certainly is. We still have code generation for entity classes (based on tables and views) and for stored procedures and functions.

May web applications are really quite simple in that they are mainly “browsers over a (one) data store”. The complexity of those kind of applications lies more in sophisticated presentation rather than coding the business logic. In this scenario, and to really benefit from the productiveness of LINQ 2 SQL, I have had good experience using LINQ 2 SQL with just two simple tweaks:

• Limited number of result set types (i.e. the set of columns):
  Rather than having my SELECTs (e.g. in stored procedures) produce different projections (i.e. tailored to the particular stored procedures task), I let them produce the same set of columns if they work on the same table. This way I will get a Customer entity and respective instances rather than a CustomerWithAddress, CustomerWithRating, CustomerWithName, CustomerWithAllInfo, … . And if I get that customer, I can always be sure it’s not some half filled entity.
• Explicit result set types:
  It’s quite common to work on JOINed data. LINQ 2 SQL handles this quite well and will figure out the entity type even for stored procedures as “auto-generated type”. In order to make this type explicit in the database I would define a view containing the JOIN and have the stored procedure work against this view. And of course I use the view to generate a respective entity with LINQ 2 SQL. 
• I also refrain from having stored procedures return more than one result set or varying result set types depending on some condition. (I don’t consider this to be a tweak, because I would do that anyway.)

Of course the database may need some tuning, as some of these tweaks may hurt the performance. But optimization when optimization is due… . And while I like the stored procedure approach, those tweaks will also work with free statements in code.

These tweaks are not meant as rules but as guidelines. Feel free to break them at leisure , their intention is merrily to support the LINQ 2 SQL code generation. However… I experienced that the define-your-entities-and-the-operations-on-top approach caused more OO-like thinking than the traditional define-your-operations-and-let-them-return-the-result-at-hand approach.

So, after dragging and dropping the tables, views, and stored procedures LINQ 2 SQL will happily generated the respective classes and methods. Getting type safe data and DB access was never easier.

If you are in the “use CRUD statements to access the database” camp, the usual round trip for an update would be: read the data that is to be changed, change it, submit changes. Like this:

using (MyDataContext dc = new MyDataContext())
{
    var customer = (from c in dc.Customer where c.ID == id select c).First();
    customer.Name = “new name”;
    dc.SubmitChanges();
}

Compared to ADO.NET this still eases db access a lot.

If you follow the “use stored procedures to access the database” approach it gets even simpler:

using (MyDataContext dc = new MyDataContext())
{
    dc.ChangeCustomerName(id, name);
}

This is kind of “anti-sophisticated” but quite efficient. And it allows to do concurrency in stored procedures if you maintain the version field on the client and provide it during the update call. I found this to be much more concise and better suited to the way web applications work.

One word of warning (or rather a disclaimer if you like): This works for me. And it works for a certain kind of web applications. And it only works because I’m willing to trade some academic architectural demands for the KISS principle, productivity, and maintainability. So please don’t follow this advice blindly.

Oh, by the way: I used this approach in a project using an Oracle database as well. Writing a simple code generator that read the system tables and generated wrappers around stored procedures was no rocket science, the only caveat was dealing with REF CURSORs.

The next post will take a closer look at LINQ 2 SQL code generation and how it affects the architecture.

That’s all for now folks,
AJ.NET

I’ve said it in an earlier post: I like working for SDX. In fact my work has become even more challenging due to a changed job description (with the unfortunate side effect that I have less time to spend on this blog). Anyway, we are looking for people (don’t look at the date of the post; if you are reading this in a year the statement will still be true). Well, here’s a short description of my latest project to give you an idea on some of the work we do for our customers…

I just completed a pilot study for one of our major customers, running about 3/4 of a year. Starting point was a system consisting of a non-trivial database and of two VB6/ActiveX/DHTML applications. Initially created a decade ago as replacement for a simple MS Access application, it had grown organically into a multi-purpose-application and a very vital, central data pool for “quite a few” stakeholders.

Since Microsoft has decommissioned VB6, the customer decided to do a VB6-to-.NET migration. My job as responsible architect was designing the target architecture, and coming up with a migration path that supported parallel operations of old and new world during the anticipated two years time the migration would take.

Of course in a decade the world has changed, so has the business and the way software has to be built to honor that. Globalization, European Union, changed customer’s perceptions, constantly changing demands, the way software is built, maintained, and operated.
So the job quickly grew to include a new domain model, adjusted business processes, better business alignment. So far the demands.

The agreed on strategy we came up with is an ecosystem of applications. One core application implements the main parts of a new domain data model, ensures consistency and — essential in any distributed system and surprisingly difficult to achieve at times — provides stable identifiers. Circled around this core application is a number of other mutually independent applications. (The possibility to do this of course relies on the business requirements being mutually independent.)

This approach is driven by

• the separation of the monolith in as many as possible disparate applications (pardon the exaggeration, but it is a fair number),
• defined ways of interaction between these applications (including off-the-shelf-software),
• and providing a sound basis to support building the custom-made family members.

This strategy provides the flexibility to build each application according to demand of specific stakeholders, even allowing for contradicting business demands of different departments. The set of applications can change in the future to address changed needs. Future technology changes can be applied in small steps (lessening the chance that will build the next Moloch). Maintenance should get better, because each application is less complex (simpler architecture, less business functions). And it conforms with Roger Sessions works on complexity of software systems.

Of course, a small but rigid set of rules and governance has to be implemented to ensure peaceful coexistence and avoid “chaotic growth” of our application family in the future. Especially the growth of interdependencies between the applications has to be watched carefully.

Next steps will be applying for the budget at the top management and then hopefully building that system. Of course with my and SDX participation .

Now, if that sounds like a challenging and interesting project; if you happen to be willing to work in the Rhein/Main area, Germany; if you could envision yourself working for the same company as I do … (not necessarily in that project!). Well then pay a look at http://www.sdx-ag.de/jobs.aspx (German language skills mandatory!).

That’s all for now folks,
AJ.SDX

While I still work on LINQ 2 SQL some people started an open discussion, rant, or rail (depending on one’s view and taken offense) about the upcoming Entity Framework.

In all frankness, an “ADO .NET Entity Framework Vote of No Confidence” (VoNC from now on, see here) is certainly capable to triggering off some very strong reactions.

What I disagree with is the way in which it’s presented. The language of “Vote of No Confidence” is intended to start a fight and is long hand for “You suck”. [...]

I’m not upset that people are giving feedback. I want them to give feedback. But here’s the deal – do it in a grown up way. My way or the highway doesn’t sit well with anyone on the receiving end. There’s a WIDE path between being constructively critical and downright offensive.

(Josh Holmes, Ranting and Raving or Getting Results?)

Also quite funny: While the VoNC has probably been popularized by Mary-Jo Foley, she instantly got her … feedback …. from one of the VoNC’s authors:

The signatories list represents leading developers, architects, and practice leads within the Microsoft community. The representation of the signatories as testers is misleading and inaccurate.

We would appreciate an immediate retraction and correction of this misrepresentation.

(Scott Bellware, MVP, see here)

Seems people got a little testy. Anyway, Scott also points out “the history of the feedback and relationship between the community of experts and the Entity Framework team going back to April of 2007″, which raises the question why they still had to publish the VoNC in the first place, despite that relationship.

Given all that, Microsoft’s response was a fine example of “cooling things down” and trying to focus in the facts. Well, essentially Tim more or less tells us that all the VoNC asked for will be kind of there in EF v2.

But already the VoNC is experiencing string headwind. Most is aimed at the, well, call it appearance, of the VoNC and a nice listing can be found here. There is even an “An Open Letter to the ALT.NET Community” (see here).

My opinion?

Thought I would spare you? Silly you

What instantly caught my attention, if just because it was the first point made in the VoNC, was the statement that “entities are more significantly behavioral objects from the perspective of entity-oriented applications” and the claim that this is a “key architectural enabler and distinction”.

Having some experience with EJB those statements vividly reminded me of entity beans. I have expressed my opinion on the notion of “intelligent data objects” (for lack of a better term) more than half a decade ago:

My opinion is that the support of statefull components (i.e. entity beans) in J2EE is one of its biggest weaknesses. Not because it’s technically bad but it leads to wrong architectural design decisions [...]
(microsoft.public.dotnet.general, 4/9/2001, see here)

The reasons for this opinion in 2001 (scalability, reliability, etc.) are still valid. Additionally there are a bunch of other reasons: Trends to lightweight architectures have prospered in Java and even affected .NET (back then, DataSets were the means of choice, today entity classes are very common). The Service notion put emphasis on data shape and operations on that data. Business process engines move data from system to system, shaping and reshaping it along the way. In the distributed, sometimes disconnected world that our enterprise environments have become I could build cases against behavioral objects around security, maintenance, outsourcing scenarios, versioning, testing (which is ironically so favored in the VoNC), software complexity, future safety, and so on.

The VoNC seems to imply that if you don’t have “behavioral objects from the perspective of entity-oriented applications”, then you are stuck with “data objects from the perspective of data storage and data storage technologies” (as I read it more or less simple mappings from database tables to objects). Well, correct me if I’m wrong, but data can be reshaped without putting behavior in it. That’s exactly what I expect from an ORM tool like EF.

I’ll stop before I get carried away. Much to my own relief, I’m not the only one having these reservations, though: Ian noted the similarity with EJB as well and got deeper in this topic. Ward makes a good point on the issues of “Persistence Ignorance” which I share and thus won’t reiterate here.

Where are we?

As I see it, the ongoing discussions are circling around the issue on whether entities should follow the data centric pattern of value objects, or whether they should be complemented with behavior to become business objects. That is largely a matter of taste.

On the other hand, the discussion is somewhat misleading. The topic has got nothing to do with domain models, which in itself is agnostic of those two approaches. And the VoNC’s reference to the anemic domain model (i.e. a domain model based on value objects) only muddies the water. The very verdict that this is an anti-pattern, and a horrific one at that, is due to a judgment by Martin Fowler and only justified by the fact that it violates OO principles (which is obvious) and people think (yes, they do, according to Martin) that value objects are real objects. No wonder the VoNC calls this a “degraded architecture”.

So, two opposing forces, standing in muddied water, and Microsoft caught between them. The PDC is going to be interesting…

BTW: Please note that I never said, Microsoft should not do what the VoNC asks for. There are bright people among the signatories and they should not be ignored. Microsoft as tool vendor should provide a tool than can be used in any architecture.

That’s all for now folks,
AJ.NET