Silverlight 2: Layout and Rendering engines

A layout engine which does precise and optimal size or position calculations is critical in any user interface framework for building responsive, interactive and high-performance applications. The term "layout engine" is often interchangeably used with "rendering engine" to mean code that takes markup content along with styles, templates and other formatting instructions to display content on screen. As prominent examples, you know of Trident, the layout engine in Internet Explorer, or Gecko which performs a similar function in Netscape/Mozilla family browsers.

In Silverlight, we typically look at the layout and rendering engines as two distinct things rather than one amorphous entity. The former handles the math which dictates how UI elements are sized and positioned on screen, and how they may be constrained or clipped in order to fit into the bounding box. The latter deals with the logic of taking bits representing graphical objects and showing them on screen. Lets do a quick lap around these features.

The Silverlight layout engine

Among the goals of the layout engine is to:

• Enable highly complex and dynamic user interfaces where elements can be automatically positioned
• Provide a gradient of Panel choices in which to place UI elements, as well a mix of knobs to achieve very precise positioning: absolute/relative sizes, margins, padding, static or specified sizes as well as dynamic or calculated sizes
• Allow app code to participate in layout. This can be as simple as being notified on layout events, or as niche as extending the layout behavior by actively participating in the process

With the design patterns and public APIs of the layout engine in Silverlight, we've stayed close to the behavior displayed by Windows Presentation Foundation (WPF), even if the implementation is custom keeping in mind the needs and constraints of Silverlight.

Very simply put, layout in WPF or Silverlight is a recursive operation that involves two passes, measurement and arrangement. During measurement, a child element is effectively told by its parent element "Here's how much room is available to all you kids and your 'gear'. Tell me how much you really need." Once all appropriate elements in the tree have been measured in this way, the arrangement pass kicks in. At this time, the elements' sizes are finalized. The parent element can choose to say either "Sure you can have your own room" or if the demands are profligate, then "Oh, the kids these days! I'd like champagne on a beer budget too. But no can do. Get your things together - you and your brother are rooming up." After arrangement, the elements in the tree are ready to be rendered.

Example of a grid with constrained size. The rectangle within it asks the parent for all its got -- and gets it: 
<Grid Width="250" Height="600">
      <Rectangle Fill="Red" />
</Grid>

Example of a stack panel with constrained size. The first rectangle gets what it desires. The second rectangle gets clipped because it desires more than the parent has room for:

<StackPanel Width="250" Height="600">
      <Rectangle Fill="Red" Width="250" Height="300"/>
      <Rectangle Fill="Green" Width="300" Height="300" />
</StackPanel>

Needless to say, the layout algorithm employs optimizations to ensure layout doesn't happen until absolutely necessary, and when it does, there selectiveness on which items need to be considered. I won't go into too much detail but primarily this keys off the knowledge that certain properties such as Width, Height, Visibility, Stretch, Alignments, Margins etc. are deemed layout-affecting and a change to one or more of these invalidates the current layout, causing Silverlight to re-layout i.e. measure and arrange its UI elements.

 

The Silverlight rendering engine

Silverlight uses vector graphics as its rendering data format, giving its graphics the ability to scale to any size or display resolution.

In WPF, the Visual class is the basic abstraction that provides rendering support. UI controls like Button, ListBox etc. derive from UIElement and FrameworkElement, which derive from Visual. In Silverlight we've dispensed of the Visual type, folding its functionality into its child types. This way we can keep the object hierarchy shorter, have one less type to deal with, small size reductions and still be a subset of WPF both conceptually and behaviorally. Silverlight also generally discards the notion of distinct visual and logical trees, effectively reducing to one tree where visual/drawing/logical info is persisted. This tree is the effective consolidation of elements you created in markup or code, containing all rendering info needed to output to screen. The tree determines rendering order as well - starting with the root visual (the top-most node), progressing to its children from left to right. If an element has children, then they are traversed before its siblings.

Silverlight uses retained mode rendering and employs a custom rendering pipeline, not relying on OS libraries like GDI/GDI+/WPF for better consistency and performance across platforms. It uses occlusion culling among other approaches to prevent overdraw. This is particularly significant if you consider that a typical application has 3-4x overdraw and perhaps 10x if effects are employed. Another very significant (but not commonly known) fact about Silverlight's rendering engine is its many-core support; specifically how it was architected to utilize the raw power of multiple cores and subdivide rendering tasks to be processed in parallel by CPU cores as they have available time. This provides phenomenal linear speed increases when you have dual, quad or 8 core machines. In this regard, Silverlight has few peers. Of course, one must not gloss over the detail that other functions such as tree walks, layout, template expansions etc. do not necessarily scale linearly.

Post-script:

Blog posts are never a substitute for good documentation (and we're fortunate to have a great team of doc writers). My intent with this post was to mix the "how" with the "why". One of my mental models is to view teams based on what I call "team DNA". In this respect, the UI platform team in Silverlight traces its DNA to the User32, GDI/GDI+, Trident, WinForms and WPF teams (Note: I'm way oversimplifying - we have a far more diverse team in ideas, outlooks and heritage than I can describe with words; but bear with me here). I think the team's DNA gives it a rich corpus of experience in blending the web and desktop, plus the ability to distil things on the "what works", "what doesn't work" and "what didn't work then, but will now" buckets. In the spirit of evolution and due to the unique size and platform requirements, the layout and rendering engines as other areas in Silverlight, introduce optimizations and use different algorithms than their WPF cousins. Feedback is always appreciated.

Further reading:

• Object positioning and layout in Silverlight 2
• The WPF Layout System
• Overview of WPF Graphics Rendering

Email this | Bookmark this