Even a chimp can write code

Thursday, April 15, 2010

Of Crashing and Sometimes Burning

I don't feel we've narrated the story about error and exception handling in Silverlight very well. We started out in v1.0 with a relatively sucky error reporting and diagnostics story and only did real work in building a robust internal infrastructure and adding diagnostics support incrementally, starting with Silverlight 2. We've since made big strides.

Our core tenet is to always provide a robust and reliable operating experience for apps.

What conditions can cause crashes?

Crashes are rare occurences. Ideally we'd avoid them altogether. They are symptoms of one of the following conditions:

  1. A bug in the platform
  2. A security mitigation being acted out
  3. The app doing an illegal reentrant call

Crashes due to platform bugs
These are rare because few bugs of that severity ever get past our quality gates. This remedy is used within Silverlight only in situations where there's a reasonable chance the platform is in uncertain state and going ahead might be fraught with problems. Within this category, there are two variants: shutting down the app and then the runtime; and tearing down the host process.

Of the two variants, the former is the more common approach. It is typical for Silverlight to notify the app via Application.UnhandledException, and most times followed by raising the OnError DOM event for the OBJECT tag prior to shutdown. If you don't see either of these happening, then that might also indicate a bug, and you should report it.

In some situations Silverlight may not consider it tenable to continue processing. Instead it takes recourse to a failfast - aka a forceful teardown. Examples include when the internal state machine appears to be corrupt or required data is missing (an ExecutionEngineException thrown but not necessarily).

If you think you've found a platform bug that is causing crashes, we'd love to get our hands on a simplified repro app with sources. If the bug does not reproduce reliably, we'd appreciate crash dumps.

We will fix most of these issues as and when they're reported.

Crashes due to an in-built security mitigation
When Silverlight detects an Access Violation (aka segfault) or a Stack Overflow, or a Buffer Overflow, the runtime's built-in failfast routines kick in. These are deliberately designed to disallow app code to catch/suppress/rethrow these exceptions, and instead prevent any exploit from taking effect. Doing that would be folly... therein lie known attack vectors. Even in the most benign situations Silverlight assumes past performance is an indicator of future performance (no, we wouldn't be good investors with that mindset!) and finds its extreme action justifiable.

We will not "fix" issues in this category.

Crashes due to reentrancy
Silverlight does not deal well with reentrant behavior. Simply put, if Silverlight calls app code in a synchronous callout, it expects the call to be blocking, and it does not expect to be called back via that or another avenue until the blocking call exits. But when it finds itself in this situation, Silverlight will teardown with extreme prejudice, bringing down app, runtime and host (browser) together with a Null AccessViolation. A Null AV is different from a regular AV.

Here's how to tell if this is happening to you. The simple way is to look for "Null AccessViolation" in your dump. A more contrived way requires symbols and involves looking at the callstack:

KernelBase.dll!DebugBreak()  Line 81             C
npctrl.dll!CWindowsServices::DebugBreak() Line 4270 + 0x8 bytes C++
npctrl.dll!DisplayDebugDialog(unsigned int uClass=1, unsigned short * pFileName=0x5ef4ef78, int iLine=64, int iValue=0, unsigned short * pTestString=0x5ef36dec, unsigned short * pMessage=0x037d9a0c) Line 926 C++
npctrl.dll!XcpVTrace(unsigned int uClass=1, unsigned short * pFileName=0x5ef4ef78, int iLine=64, int iValue=0, unsigned short * pTestString=0x5ef36dec, unsigned short * pMessageString=0x5ef31e78, void * pVArgs=0x037d9c00) Line 1036 C++
npctrl.dll!CWindowsServices::XcpTrace(unsigned int uClass=1, unsigned short * pFilename=0x5ef4ef78, int iLine=64, int iValue=0, unsigned short * pTestString=0x5ef36dec, unsigned short * pMessageString=0x5ef31e78, ...) Line 7258 C++
npctrl.dll!CReentrancyGuard::CheckReentrancy(int bNullAvOnReentrancy=1) Line 64 + 0x36 bytes C++
npctrl.dll!CXcpDispatcher::OnReentrancyProtectedWindowMessage(HWND__ * hwnd=0x004b039a, unsigned int msg=1026, unsigned int wParam=0, long lParam=0) Line 900 + 0xa bytes C++
npctrl.dll!CXcpDispatcher::WindowProc(HWND__ * hwnd=0x004b039a, unsigned int msg=1026, unsigned int wParam=0, long lParam=0) Line 807 + 0x18 bytes C++
user32.dll!_InternalCallWinProc@20() Line 106 Asm
user32.dll!UserCallWinProcCheckWow(_ACTIVATION_CONTEXT * pActCtx=0x00000000, long (HWND__ *, unsigned int, unsigned int, long)* pfn=0x5f10bc90, HWND__ * hwnd=0x004b039a, _WM_VALUE msg=1026, unsigned int wParam=0, long lParam=0, void * pww=0x00e267f8, int fEnableLiteHooks=1) Line 163 + 0x12 bytes C
user32.dll!DispatchMessageWorker(tagMSG * pmsg=0x5f10bc90, int fAnsi=0) Line 2591 + 0x1e bytes C
user32.dll!DispatchMessageW(const tagMSG * lpMsg=0x037d9dec) Line 999 C
user32.dll!DialogBox2(HWND__ * hwnd=0x002902ac, HWND__ * hwndOwner=0x000b0924, int fDisabled=0, int fOwnerIsActiveWindow=0) Line 1150 C
user32.dll!InternalDialogBox(void * hModule=0x760b0000, DLGTEMPLATE * lpdt=0x04923388, HWND__ * hwndOwner=0x000708fe, int (HWND__ *, unsigned int, unsigned int, long)* pfnDialog=0x7611eec8, long lParam=58564704, unsigned int fSCDLGFlags=0) Line 1314 + 0x9 bytes C
user32.dll!SoftModalMessageBox(_MSGBOXDATA * lpmb=0x00000030) Line 1237 + 0x18 bytes C
user32.dll!MessageBoxWorker(_MSGBOXDATA * pMsgBoxParams=0x037da060) Line 791 C
user32.dll!MessageBoxIndirectW(const tagMSGBOXPARAMSW * lpmbp=0x037da0d4) Line 528 + 0xe bytes C
ieframe.dll!6132ecd5()
The presence of CReentrancyGuard::CheckReentrancy indicates the reentrancy guard; message 1026 indicates WM_INTERNAL_TICK the veritable sign of the apocalypse reentrancy; and the presence of a (non-Silverlight, in this case from NTUser) MessageBox at the bottom indicates an actor that pumps messages including Silverlight messages. Note that Silverlight's own MessageBox will not cause this reentrancy - it and the reentrancy guard were specifically designed to play well with each other. So I recommend using that any place you'd consider using an alert.

Now consider that SizeChanged and LayoutUpdated events are raised synchronously as part of a layout pass. (I've previously expressed my firm preference for all things async -- I'd have loved for these events to be async but lost that argument since it'd have broken WPF compat and eliminated a couple scenarios. But I digress.) Now if you were to use the DOM Bridge within a SizeChanged event handler to, say, do an alert(), that'd align all the stars. The alert would pump messages; Silverlight would be in a blocking call, not expecting any messages in its internal queue; its little "reentrancy guard" would see this as reentrant behavior, and initiate adverse action terminating with a Null AV.

Silverlight's behavior in this situation was intended to be moral equivalent of a shock collar. We wanted app authors to detect these situations at development time and fix their code, so it never shipped with reentrant behavior.

This condition is the most common cause for crashes in Silverlight, and is indicative of programming error in the app.

We're constantly evaluating whether this sort of positive reinforcement through operant conditioning actually works. If you have anecdotes or feedback, please drop a line. Bear in mind that we have done a poor job at advertising this behavior, and I do sincerely apologize. So the fact that it took you a good while longer than average to debug such an issue is unfortunate but explained away by that.

Where to look for crash dumps?
While developing apps on Windows, with Windows Error Reporting (Watson) you should see the path in the UI shown upon a crash. Alternately look under %Temp% for dirs with "WER" prefix, and look for a .mdmp file prefixed by process name. If this is an out of browser app, you should see "sllauncher.exe.mdmp" instead of one for the browser's executable.

Further reading

With the context from this post, I hope you will see the MSDN doc on Silverlight Error Handling in new light. Some of that content originated from my original spec on the topic written circa Silverlight 2. It will help you understand control flow. In addition, this topic page will help you understand how best to handle exceptions in your own Silverlight app code.



A word on diagnostics

If we could rewrite the past and change something, we'd have tweaked our initial conditions to one that enforced error reporting best practices in our internal development of the runtime. This would have eliminated having to perform triage and surgery on a rapidly evolving runtime as we added better diagnostics. There are still too many E_FAILs in our code that bubble up and result in the totally unhelpful "Unknown error" and the wildly melodramatic "Catastrophic error". There's a bit of dark humor in the backstory, but that's for another time.

Our work is not done though. We continue to seek feedback from app authors and fix our oversights. Keeping Silverlight easy to develop against is a giant priority for our team. Making sure you have great diagnostics and get past issues fast is one way we can make that happen.

Labels:

Email this | Bookmark this

Four scores!

Silverlight 4 is now live! Here's what's new.

The team is taking a much-deserved couple days off to recharge. Besides, the sun's out in Seattle this week - after what seems like decades.

Labels:

Email this | Bookmark this

Can you access a loose XAML file from LoadComponent?

The answer is no. The job of Application.LoadComponent in Silverlight is to “merge” a specified XAML file with a specified object.

public void Application.LoadComponent(object, Uri);

In doing so, it does a series of validations:

  • checks that neither parameter is null
  • checks whether the specified XAML URI is absolute or relative. Absolute URIs are not allowed.
  • checks whether the specified XAML is embedded within an assembly. If not, LoadComponent fails.
  • checks whether the root element declared in the specified XAML is actually the same type as that of the specified object. If not, LoadComponent fails.

and only then attempts to merge the two in a custom parsing step.

Well, why not?

The last bullet point above is consequential in why we don’t allow loose XAML. LoadComponent thinks of an assembly as a logical boundary. In the presence of XAML with contents like this:

<UserControl x:Class=”Foo”> …

the only unambiguous way for Silverlight to identify and locate “Foo” is if it knew the DLL where that type was defined. Iterating through all DLLs loaded in the AppDomain isn’t an optimal solution at all. This is why it insists that the XAML be embedded within a DLL. Then it knows the un-qualified type is defined within that DLL. In practice, the URI specified on the LoadComponent call ends up being something like:

/SomeAssemblyName;component/Page.xaml

which has metadata on the provenance of said XAML i.e. that it came from an assembly with short name “SomeAssemblyName”. Silverlight can now happily match it against the type backing the other specified parameter – the object.

How does one work around this?

A workaround exists, but it has some limitations. You can use XamlReader.Load() to have Silverlight create an object tree from your specified markup. The caveat being that said XAML cannot have code-behind associations i.e. a backing type as root element, or event handler declarations. This approach is ideal if you’re dealing with sprites that are injected into the app, with other plumbing hooked up programmatically.

Aside: This is part of a recycling bits series I've been considering for a while. I frequently find myself answering the same questions over email or other forums, and figure I'd give some of those FAQs airtime on this blog, so as to multiply any benefits the information might have.

Labels:

Email this | Bookmark this

Thursday, March 18, 2010

Stepping outside the browser with Silverlight 4

Thanks to all those who attended the talk "CL10 - Stepping outside the browser with Silverlight 4" at the Mix 2010 conference or caught it via the recorded session online. I talked about the evolution of Silverlight into a general purpose development platform, far away from its origins as a browser plugin. I walked through features that light up outside the browser through our deliberate nested doll design approach - both in Silverlight 3 and continuing into Silverlight 4 now. I also talked about trusted apps and emphasized how important getting the right user experience and removing friction from app deployment was to us in the Silverlight team.

Here is the slide deck from the talk. Source code for the basic sandboxed app is archived here; and code for the trusted app can be found here. Hope you find it valuable.

The Q&A session at the end was very useful. I appreciate all the interest in this space and was happy to learn of the apps people were building. We could only get here "with a little help from our friends" in the ecosystem. We can continue that conversation on this blog, or you can send me email (ashsh).


Update: as a postscript, the slide deck I linked to here looks different from the one I used in the talk. Due to "technical difficulties" the podium machine had an older copy of the deck from one of my previous uploads, and I only found out once I was into the talk (silly me for not checking). So if I looked like I crapped a brick at the beginning, now you know why. Live and learn.

Labels: , , ,

Email this | Bookmark this

Saturday, July 25, 2009

Silverlight out-of-browser apps: Remove

Removing a Silverlight out-of-browser app is as easy as installing it was previously. From within the app, just right-click and choose “Remove this application…”.

Just so you don’t wipe away apps by accident, Silverlight will ask you to confirm, and when you do, the app is uninstalled and its binaries are scavenged from the offline application cache.

Data persisted by the app into local data store (Isolated Storage) is not removed. It is still consumable by the in-browser version of the app.

On Mac OS X, dragging the app bundle into the Trash can is another way to remove the app.

In line with our guiding principles which involve keeping the consumer end user firmly in control, keeping it simple and staying out of the way, the Remove experience does not allow the app to programmatically delay, defer or prevent its own uninstall.

Previous posts in this series:

Labels: , , ,

Email this | Bookmark this

Friday, July 10, 2009

Silverlight out-of-browser apps: How Updates Work

Silverlight makes updating your out-of-browser XAP a breeze. It does the heavy lifting for you, asking that you merely invoke the update logic from within your app at a time or frequency of your choosing, and respond to events raised by Silverlight.

If an updated XAP has been published to your web server, Silverlight will download and induct it to the offline application cache. The next time your app is activated, Silverlight will ensure the updated bits are activated and the old bits are scavenged.

What the app author needs to do

Employ the async-pattern APIs exposed for updates in your out-of-browser apps. The trigger method is the Application.CheckAndDownloadUpdateAsync() method. Hook up an event handler for the CheckAndDownloadUpdateCompleted event. See, there’s not much to it! No messing with version numbers or the like.

Gotchas?

Well the only thing to watch out for is the situation when a new version of Silverlight comes out (say v4) and your app is built against it, but the consumer is running an older runtime (say, v3) and an appropriate older version of your app. In this situation, the app update cannot be applied because it requires a runtime update first. Silverlight notifies your app of this situation by setting the event arg’s Error property value to System.PlatformNotSupportedException.

All in all, here’s the code you’re going to write:

private void SomeMethod()
{
Application.Current.CheckAndDownloadUpdateAsync();
}

private void App_CheckAndDownloadUpdateCompleted(object sender,

CheckAndDownloadUpdateCompletedEventArgs e)
{
if (e.UpdateAvailable)
{
MessageBox.Show("An update has been downloaded. " +
"Restart the application to run the new version.");
}
else if (e.Error != null &&
e.Error is PlatformNotSupportedException)
{
MessageBox.Show("An application update is available, " +
"but it requires a new version of Silverlight. " +
"Visit the application home page to upgrade.");
}
else
{
MessageBox.Show("There is no update available.");
}
}


What the app publisher needs to do

If you handle application deployments or administer web servers, all you need to do is xcopy the new XAP (plus any external content files it needs) in place of the old XAP. Silverlight will take it from there.

 

In the end, a word to the wise... Make sure you clearly articulate your app's update policy to your end users. Whether you auto-update, or require user-initiation; whether you update upon every app launch or at another discrete frequency; let your app's consumer audience know. Ideally you'd grant them the ability to override your defaults. This is the right thing to do. The user must always be in control.

 

Previous posts in this series:

Labels: , , ,

Email this | Bookmark this

Three Years and Three Versions

Silverlight 3 just shipped! 3 is so much bigger than 2; and here's what's new. See Scott's post announcing the release to web (RTW) and outlining the major features.

Microsoft.com/silverlight gets a facelift. Goodbye dusk (theme), hello dawn. Be sure to check out the overview of Silverlight 3 there.

Thanks to all of you for your persistent feedback and support in making this such a powerhouse of a release. Keep that feedback coming - we're already working on 4.

More soon...

Labels:

Email this | Bookmark this

Saturday, May 09, 2009

User-initiated operations in Silverlight

User-initiated operations in Silverlight:
  1. must be initiated by the user via an input device like keyboard, mouse or accessibility client...
  2. ...within Silverlight
  3. have an in-built timeout
  4. have atomicity of transaction
  5. cannot be marshalled across the specific Silverlight plugin's boundaries

#1 is obvious.

#2 isn't. Silverlight will not honor user-initiated operations that start from an HTML element, have a JavaScript handler which calls into a Silverlight scriptable API or other such endpoint.

#3, the in-built timeout in a user-initiated operation exists to eliminate long running, blocking, iterative, recursive tasks within one user-initiated method call. The timeout value is sort of a magic number, chosen to be small enough to no be a pain point for most developers and their scenarios, and large enough not to compromise on its goals.

#4 indicates that you cannot combine two operations in Silverlight which both require user initation into one transaction.

#5 means that the user-initiated "bit" doesn't traverse the Silverlight plugin boundaries, via IPC calls such as local messaging or even intra-process calls via DOM bridge or Scriptable object features.

These may look like quirks but are actually "features" that exist in Silverlight as secure defaults and to provide consistency of behavior across its supported browser and platform matrix. It is important for every Silverlight designer and developer to understand these nuances.

Labels:

Email this | Bookmark this

Monday, April 27, 2009

Silverlight out-of-browser apps: Local Data Store

Silverlight applications – whether in-browser or out-of-browser, and whether running online or offline - can persist data into a local data store referred to as “Isolated Storage”. With the new offline/out-of-browser support in Silverlight 3, the platform further ensures that an app has access to the same Isolated Storage data regardless of whether it was activated in-browser or locally. This way, apps can share data between the in-browser and out of browser versions. As I’d mentioned in a previous post, Silverlight will enforce the provenance URI of the XAP in either case, ensuring that when each queries for its local store, it gets the very same artifact.

What can you do with this power?

Scenario #0: Local playback sure beats network playback.

Scenario #1: Assuming you have consumer use cases that involve using your in-browser and out-of-browser apps interchangeably, you now have a way to share data between them.

Scenario #2: If you need to share data between your in-browser app and your newly minted out-of-browser app immediately upon install (detach), you can use this local store as your custom “cookie store”.

Scenario #3: If your out-of-browser app detects no network connection, it can still take user input but persist that into local store, to later playback when connectivity is regained. 

Internals

Silverlight’s Isolated Storage feature provides a virtual file system in which apps can create and store both files and directories with arbitrary data.

Apps are abstracted from the lower level file I/O functionality; they do not (need to) know where the files are stored or what sort of internal formats and directory hierarchies are in place. While apps can use file paths and dot-out syntax, these paths cannot escape the bounds of the virtual file system. This sort of virtualization provides isolation of app data, preventing other apps from inadvertently reading from or writing to your app’s store. Hence the term “Isolated” in the name. Apps and app authors do need to recognize that data stored in this persistence store is per-user and per-domain.

The Isolated Storage feature in Silverlight persists app data within a directory in the user profile (\%AppData%\LocalLow\Microsoft\Silverlight\is on Windows Vista and up and /Users/<user>/Library/Application Support/Microsoft/Silverlight/is on Mac OS X). The AppData folder is a hidden folder. This has the beneficial side effect of hiding your data from from OS search utilities (Windows desktop search and Spotlight) as well as the casual user. This should not be seen as a security boundary – only as a perpetuating the isolation of data.

Quotas

The space allocated for local data storage in a Silverlight app is bounded by a default quota assigned to that app’s domain of origin. At install (detach) time out-of-browser Silverlight apps will have 25MB space available within the Isolated Storage by default. Contrast this to the default quota of 1MB enjoyed by conventional in-browser Silverlight apps.

Needless to say, more space is available to the app upon user consent when the app invokes the System.IO.IsolatedStorage.IsolatedStorageFile.IncreaseQuotaTo() method. That method invocation itself needs to be in response to a user-initiated action such as mouse click or key press.

When your out-of-browser app is uninstalled, Silverlight does not reclaim your special quota or erase data persisted there.

Encryption

Although the isolated storage maintains data in an obscure location, it should not be confused with secure or encrypted data. However apps can easily encrypt and decrypt isolated storage files; they may also sign and validate the signature of an isolated storage file using the SHA256 hash function.

Gotchas

When saving to a virtual file system, one needs to be aware of inherent limits on file path size. This is imposed on you because the Isolated Storage’s root dir itself occupies several characters in the path (151 on Vista/Win7, 183 on XP/2K, 160 on OS X) effectively leaving you only the remainder (109, 77 and 100) respectively for your own file and directory names before Silverlight’s limit of 260 chars for a file path is hit.

Saving files outside of Isolated Storage

Silverlight 3 allows for saving files outside of isolated storage. Saving to any location on the client-side file system requires user initiation and the use of the SaveFileDialog control - a new “safe” control which allows saving one file at a time via a stream with your app code not having access to the file system or structure. Saving data to the server via an HTTP postback is another option.

----

This was #6 in my ongoing series of posts on Silverlight’s new offline and out of browser support. Hope it is helping you gear up.

Previous posts in this series:

Related links:

Labels: , , ,

Email this | Bookmark this

Wednesday, April 22, 2009

Silverlight out-of-browser apps: Network Awareness

There will be times when your Silverlight app is run but the end user does not have a network connection. A key attribute of making a functional offline Silverlight app is building in network awareness. In this post, we’ll look at how easy Silverlight 3 makes this for you.

The API

Silverlight provides current network status via the NetworkInterface.GetIsNetworkAvailable() method. Apps can be notified of changes in network availability via the NetworkChange.NetworkAddressChanged event. The NetworkInterface and NetworkChange types are both declared in the System.Net.NetworkInformation namespace.

Here’s sample usage of the network detection APIs:

...

    NetworkChange.NetworkAddressChanged += 
                      new EventHandler (OnNetworkChange);

}


void
OnNetworkChange (object sender, EventArgs e)

{

    if (NetworkInterface.GetIsNetworkAvailable())

    {

        nwIndicatorEllipse.Fill =
                              new SolidColorBrush(Colors.Green);

        // and do something

    }

    else

    {

        nwIndicatorEllipse.Fill =
                              new SolidColorBrush(Colors.Red);

        // and do something else

    }

}

What This Feature Is

The NetworkAddressChanged event – as the name suggests - advertises a change in IP address (availability). It is to be used in concert with the NetworkInformation.GetIsNetworkAvailable() to identify whether the recent change resulted in the system obtaining an IP address or losing one. The IP address is seen as a good primordial proxy for availability of network connection or lack thereof.

This feature is an answer to the question “Is this is a good time to make an outbound network request?”. Silverlight plays an enabling role in detecting offline status, telling you whether it’s worth your while to make an outbound call, or alternately to cache and persist data locally until network connectivity is regained.

What this Feature Isn’t

This feature is not an answer to the question “Is my site (or web service) up and running at this moment?”. The way I see it, Silverlight doesn’t play an enabling role in that situation – that is something your app can achieve with 10 lines of code. And such a service indicator is better off taking the actual URI and returning an actual response, rather than being a glorified HTTP HEAD wrapper.

Gotchas

Say you have 2 adapters – a wireless and wireline Ethernet – both of which are connected. If your wireless connection drops, the NetworkAddressChanged event is raised but in the handler when you query NetworkInformation.GetIsNetworkAvailable(), you still get true because your wireline Ethernet connection is still valid. When that goes down as well, the event is raised yet again and the NetworkInformation.GetIsNetworkAvailable() returns false. Always remember: from your NetworkAddressChanged handler, you must query GetIsNetworkAvailable to definitively find out the connectivity situation.

Other gotchas include the “illusion of connectivity”. This happens when you have some connectivity (say you’re on a WiFi network in an airport or a coffee shop) but not any Internet connectivity until you sign in or pay up. Your app may see GetIsNetworkAvailable return true, but should still try that actual outbound request to ascertain this.

In some cases virtual machines (e.g. Hyper-V running on your Windows Server or running Windows with Parallels on your Mac) may interfere with this feature and your app may see false positives.

---

Previous posts in this series:

Related links:

Labels: , , ,

Email this | Bookmark this