I haven't posted a lot in the last few weeks, and there's a succinct reason for that.  In fact, I can name it in three words:  Full Test Pass. The concept is close to what it sounds like.  During the earlier stages of product planning, the test team for ASP.NET and the web development profile for Visual Studio used the time to plan its test coverage.  In total, we wrote descriptions of over 40,000 test cases that we wanted to run.  In the span of about four weeks ending earlier this month, we took the time to acutally execute nearly all of them, many for the first time. Don't take that to mean that we whittled away all the hours between whiles drinking beverages and playing foosball -- we are professionals, after all.  We used the time to automate a large subset of our tests, including our acceptance tests (tests that we run on each daily build of the product to get a rough sense of its quality and catch really the big bugs).  Automation infrastructure -- the libraries and components our testcases use to simplify automation and manage the process of reporting results to our testcase management database (named Mad Dog, if you can believe it) -- also took up a lot of time.   But 40,000 is a big number, and in that number were a lot of scenarios that we just hadn't tried yet.  For example, one of the features I own is Control Designers, which is the mechanism a custom control developer uses to control the appearance and behavior of her control in Visual Studio's web design environment.  As part of this, I test several new host services that are exposed by Visual Studio which the authors of Control Designers can use to simplify their work.  One of these is an interface with the name IWebFormsBuilderUIService, which exposes methods for creating common UI dialogs, among them a URL editor.  In Visual Basic .NET, the code a control author would need to write in order to use this would be something like this: Imports System.ComponentModelImports System.ComponentModel.DesignImports System.Web.UI.DesignImports System.Web.UIImports System <Designer(GetType(ExampleControlDesigner)), . . . > Public Class ExampleControl    Inherits System.Web.UI.WebControls.WebControl     Dim _url As String     <Bindable(True), Category("Appearance"), DefaultValue("")> Property URL() As String        Get            Return _url        End Get         Set(ByVal Value As String)            _url = Value        End Set    End Property     . . .End Class Public Class ExampleControlDesigner    Inherits ControlDesigner     Protected Overrides Sub OnClick(ByVal e As DesignerRegionEventArgs)        Dim iwfbuis as IWebFormsBuilderUIService        Dim ctl As ExampleControl         Mybase.OnClick(e)        ctl = Component        iwfbuis = GetService(GetType(IWebFormsBuilderUIService))        try            ctl.URL = iwfbuis.BuildUrl(Nothing, ctl.URL, RootDesigner.DocumentUrl, "Please Choose a URL", "", UrlBuilderOptions.None)        catch ex as exception            Msgbox(ex.Message)        end try        UpdateDesignTimeHtml()    End Sub     . . .End Class The above code would bring up Visual Studio's URL dialog when the user clicked on the control, and set the control's URL property to the user's selection. I don't have any acceptance-level tests for IWebFormsBuilderUIService.BuildUrl, because even if this method were completely busted beyond usability, it wouldn't especially hurt the other testers on our team or the other teams that depend on ASP.NET.  However, I do have plenty of regular functional tests for the feature that I plan to automate, and the plan for this area looks like this: sub document_onclick() dim lWTop, lWLeft, e, lWidth, lHeight lWidth = 600: lHeight = 350 set e = window.event.srcElement lWTop = (window.screen.Height-lHeight) / 2 lWLeft = (window.screen.Width-lWidth) / 2 if left(e.id,2) = "tc" then window.open "/MadDog/Reports/Testcase/frmTestcase.asp?lTestcaseID=" & mid(e.id,3), "_blank", "resizable=yes,scrollbars=yes,height=" & lHeight & ",width=" & lWidth & ",status=no,toolbar=no,menubar=no,top=" & lWTop & ",left=" & lWLeft & ",location=no" elseif left(e.id,3) = "hdr" then if document.all("dv" & mid(e.id,4)).style.display = "none" then document.all("dv" & mid(e.id,4)).style.display = "" else document.all("dv" & mid(e.id,4)).style.display = "none" end if end if end sub sub document_onmouseover() if left(window.event.srcElement.id,2) = "tc" then window.event.srcElement.style.fontweight = 700 end if end sub sub document_onmouseout() if left(window.event.srcElement.id,2) = "tc" then window.event.srcElement.style.fontweight = 400 end if end sub document.body.style.cursor = "wait" Testcase Breakout for BuildUrl Hierarchy: All Testcases -> Web.NET -> Page & Controls -> Page Framework -> Designers -> Control Designers Extensibility -> Host Services -> IWebFormsBuilderUIService BuildUrl Web.NET -> Page & Controls -> Page Framework -> Designers -> Control Designers Extensibility -> Host Services -> IWebFormsBuilderUIService -> BuildUrl" ClayCo td.header {border:1px solid; background-color:ivory; width:150px; font-weight:bold} td.content {border:1px solid} Area Description Host-implemented method for bringing up its own URI-picking dialog. BuildUrl01 - Call and (from Venus) cancel the dialog Pri 2, Coding 1) Verify initialURL is returned. BuildUrl02 - Call (from ControlDesigner) and enter (from Venus) an absolute URL Pri 2, Coding Abstract:All scenarios assume UrlBuilderOptions is set to NoAbsolute unless stated otherwise. Scenarios:1) Use http://localhost/<currentapp>, verify relative conversion2) Call same with UrlBuilderOptions set to None, verify no conversion3) Use http://localhost/<currentapp>/, verify relative conversion4) Use http://localhost/<currentapp>/Page2.aspx, verify relative conversion5) Use https://localhost/AnotherApp/Page.aspx, verify relative conversion6) Use ftp://ftp.is.co.za/rfc/rfc1808.txt7) Use gopher://spinaltap.micro.umn.edu/00/Weather/California/Los%20Angeles8) Use mailto:clayco@microsoft.com9) Use news:comp.infosystems.www.servers.unix10) Use telnet://melvyl.ucop.edu/11) Use http://localhost/a/./g.aspx, verify relative conversion12) Call same with UrlBuilderOptions set to None, verify no conversion13) Use http://localhost/a/b/../g.aspx, verify relative conversion14) Call same with UrlBuilderOptions set to None, verify no conversion15) Use http://localhost/test., verify relative conversion16) Call same with UrlBuilderOptions set to None, verify no conversion17) Use http://localhost/.aspx, verify relative conversion18) Call same with UrlBuilderOptions set to None, verify no conversion19) Use http://localhost/..aspx, verify relative conversion20) Call same with UrlBuilderOptions set to None, verify no conversion21) Use 256-character URI22) Use string containing Arabic Letter Khah and U+4E2E (catch). BuildUrl03 - Call (from ControlDesigner) and enter (from Venus) a relative URL Pri 2, Coding Abstract:All scenarios assume UrlBuilderOptions is set to NoAbsolute unless stated otherwise. Scenarios:1) Use //localhost/<currentapp>/Page2.aspx, verify an app-relative URL is returned2) Call same with UrlBuilderOptions set to None, verify absolute conversion3) Use /<currentapp>/Page2.aspx, verify an app-relative URL is returned4) Call same with UrlBuilderOptions set to None, verify absolute conversion5) Use Page2.aspx6) Call same with UrlBuilderOptions set to None, verify absolute conversion7) Use //www.yahoo.com:80/, verify an absolute URL is returned8) Use /./Page2.aspx, verify an app-relative URL is returned9) Call same with UrlBuilderOptions set to None, verify absolute conversion10) Use /../Page2.aspx11) Call same with UrlBuilderOptions set to None, verify absolute conversion12) Use http.aspx13) Call same with UrlBuilderOptions set to None, verify absolute conversion14) Use test.15) Call same with UrlBuilderOptions set to None, verify absolute conversion16) Use .aspx17) Call same with UrlBuilderOptions set to None, verify absolute conversion18) Use test..19) Call same with UrlBuilderOptions set to None, verify absolute conversion20) Use ..aspx21) Call same with UrlBuilderOptions set to None, verify absolute conversion22) Use ./../test.aspx, verify an app-relative URL is returned23) Call same with UrlBuilderOptions set to None, verify absolute conversion24) Use ./SubDir/., verify an app-relative URL is returned25) Call same with UrlBuilderOptions set to None, verify absolute conversion26) Use SubDir/./Page2.aspx27) Call same with UrlBuilderOptions set to None, verify absolute conversion28) Use SubDir/../Page2.aspx29) Call same with UrlBuilderOptions set to None, verify absolute conversion30) Use Test;x=1/./Page2.aspx31) Call same with UrlBuilderOptions set to None, verify absolute conversion32) Use Test;x=1/../Page2.aspx33) Call same with UrlBuilderOptions set to None, verify absolute conversion34) Use ., verify an app-relative URL is returned35) Call same with UrlBuilderOptions set to None, verify absolute conversion36) Use ./, verify an app-relative URL is returned37) Call same with UrlBuilderOptions set to None, verify absolute conversion38) Use ..39) Call same with UrlBuilderOptions set to None, verify absolute conversion40) Use ../41) Call same with UrlBuilderOptions set to None, verify absolute conversion42) Use ../Page2.aspx43) Call same with UrlBuilderOptions set to None, verify absolute conversion44) Use ../..45) Call same with UrlBuilderOptions set to None, verify absolute conversion46) Use ../../47) Call same with UrlBuilderOptions set to None, verify absolute conversion48) Use ../../Page2.aspx49) Call same with UrlBuilderOptions set to None, verify absolute conversion50) Use 256-character URI51) Use string containing Arabic Letter Ghain and U+3A2E (lift up). BuildUrl04 - InitialUrl paramerter scenarios Pri 2, Coding 1) Use current web application folder's absolute URL, verify UI in Venus2) Use current web application folder's relative URL, verify UI in Venus3) Use current Page's URL, verify UI in Venus4) Use image file URL, verify UI in Venus5) Use absolute path to virtual sibling directory, verify UI in Venus6) Use non-existent absolute path to sibling directory, verify UI in Venus7) Use absolute path to virtual parent directory, verify UI in Venus8) Use absolute path to virtual child directory, verify UI in Venus9) Use relative path to virtual sibling directory, verify UI in Venus10) Use relative path to virtual parent directory, verify UI in Venus11) Use relative path to virtual child directory, verify UI in Venus12) Use http://, ftp://, and file:// URLs to test machine, verify UI in Venus13) Use URL to another machine, verify UI in Venus14) Use http://www.microsoft.com/, verify error BuildUrl05 - BaseUrl parameter scenarios Pri 2, Coding Abstract:All scenarios assume UrlBuilderOptions is set to NoAbsolute unless stated otherwise. Scenarios:1) Use current web application folder's absolute URL, verify relative URL calculation2) Use current web application folder's relative URL, verify relative URL calculation3) Use current Page's URL, verify relative URL calculation4) Use image file URL, verify relative URL calculation5) Use absolute path to virtual sibling directory, verify relative URL calculation6) Use absolute path to virtual parent directory, verify relative URL calculation7) Use absolute path to virtual child directory, verify relative URL calculation8) Use relative path to virtual sibling directory, verify relative URL calculation9) Use relative path to virtual parent directory, verify relative URL calculation10) Use relative path to virtual child directory, verify relative URL calculation11) Use http://, ftp://, and file:// URLs to the same machine, verify absolute URLs are used12) Use URL to another machine, verify relative and absolute paths on local and remote machines.13) Use http://www.microsoft.com/, verify absolute paths get used.14) Use String.Empty, verify absolute paths get used.15) Use current web application folder's absolute URL when options is None, verify absolute URL16) Use current web application folder's relative URL when options is None, verify absolute URL17) Use current Page's URL when options is None, verify absolute URL18) Use image file URL when options is None when options is None, verify absolute URL19) Use absolute path to virtual sibling directory when options is None, verify absolute URL20) Use absolute path to virtual parent directory when options is None, verify absolute URL21) Use absolute path to virtual child directory when options is None, verify absolute URL22) Use relative path to virtual sibling directory when options is None, verify absolute URL23) Use relative path to virtual parent directory when options is None, verify absolute URL24) Use relative path to virtual child directory when options is None, verify absolute URL BuildUrl06 - Caption parameters Pri 2, Coding Abstract:Perform standard string tests and verify in Venus Scenarios:1) S1) Verify default property value2) S2) Verify attribute value is not in HTML view3) S3) Set to all numbers4) S4) Set attribute to (nothing) in HTML view5) S5) Set to 1 character6) S6) Set attribute to space in HTML view7) S7) Set attribute without quotes in HTML view8) S8) Set property to DblQuote-space-DblQuote in Design view9) S9) Set property to Quote-space-Quote in Design view10) S10) set property to: DblQuote-Quote-&-Quote-DblQuote; verify11) S11) set attribute to: 3DblQuote-'''abc'''-3DblQuote; verify12) S12) set attribute to: DblQuote-@#$%-CrLf-GreaterThan-)(*^-DblQuote; verify13) S13) Set property to "dadadada" in prop grid14) S14) Set to long string15) S15) Set to a random intl string and verify16) S16) Set to an Intl string of problem characters17) S17) Set to a random unicode string BuildUrl07 - Filter parameter scenarios Pri 2, Coding Abstract:Use Standard String tests and verify expected value in Venus Scenarios:1) S1) Verify default property value2) S2) Verify attribute value is not in HTML view3) S3) Set to all numbers4) S4) Set attribute to (nothing) in HTML view5) S5) Set to 1 character6) S6) Set attribute to space in HTML view7) S7) Set attribute without quotes in HTML view8) S8) Set property to DblQuote-space-DblQuote in Design view9) S9) Set property to Quote-space-Quote in Design view10) S10) set property to: DblQuote-Quote-&-Quote-DblQuote; verify11) S11) set attribute to: 3DblQuote-'''abc'''-3DblQuote; verify12) S12) set attribute to: DblQuote-@#$%-CrLf-GreaterThan-)(*^-DblQuote; verify13) S13) Set property to "dadadada" in prop grid14) S14) Set to long string15) S15) Set to a random intl string and verify16) S16) Set to an Intl string of problem characters17) S17) Set to a random unicode string Testcase Summary Info: Design: Pending Scenario Review: Coding: 7 Pending Code Review: Complete: Total: 7 This is typical -- every feature area has a lot more tests defined for it than get will ever be run on any given build.  We rely on testers using the builds for ad-hoc testing and writing automation to find bugs, and rely on automated tests to catch regressions in later builds.  We also use full test passes like this to get a deeper sense of the product's quality at a few key intervals (such as earlier this month -- we wanted to catch isses in advance of the first large-scale Beta release of the product and get a sense of how much more work is needed to stabilize the product's quality before we ship). And we did find bugs.  In the above test plan, the very first test case failed during the full test pass.  If a user clicked “Cancel” in the URL dialog, an empty string was returned by the method, rather than the initialURL parameter value.  This has already been fixed, by the way.

Those of you who know about RFC 3066 know that it's the standard which describes the tags that you should use to identify the language of an HTML or XML document's contents.  For example, in XHTML you'd code something like: . . .<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="fr" >. . .</html> to tell the world that your page is in French.  RFC 3066 (albeit with a few differences) is also the syntax of the culture names used by System.Globalization.CultureInfo.  What you may not have known is that there's a way to request new tags in case you need to extend the existing language tagging mechanism.  This can be very useful if you need to indicate a language variety that isn't easily described with a language-and-country combination.  On the other hand, well, I'll let today's request speak for itself: LANGUAGE TAG REGISTRATION FORM Name of requester          : Jon HannaE-mail address of requester: jon [at] hackcraft.netTag to be registered       : en-Spam-porn English name of language   : Pornographic Spam English Native name of language (transcribed into ASCII):Pr0n Freespeech ohbsehkzq! Reference to published description of the language (book or article):Your deleted mail folder Any other relevant information:The relationship between this dialect and other forms of English istenuous and perhaps more a matter of historical accident than anystronger linguistic relationship. Some would argue that it is aseparate language, and some that it isn't a language at all butilliterate gibberish, however considering it to be a dialect of Englishis perhaps the most balanced position. Interestingly, it is the probably the only dialect of any language thatcan be encoded entirely in US-ASCII without doing any (further) violenceto the language. Conversely, it is also probably the only language thatcan be inscribed with the use of a diacritical mark on every singlecharacter, even spaces. The dialect appears to have very complicated punctuation rules, socomplicated that there is strong evidence that no one fully understandsthem. The exclamation mark (U+0021, and no doubt U+203C once theydiscover it) appears not to be a punctuation character within thedialect, but a symbol of some sort of religious or superstitiousfascination within the user community. The anthropologist might referto this as a "fetish", though he or she would be wise not to lead anymember of the user community to believe that he or she had any interestin fetishes. Despite widespread belief to the contrary it appears that some peoplereally do have this as their first, and indeed only, language. Thesepeople would be to be pitied, if we could have the luxury of doing sowithout actually coming into contact with them or their dialect. The pun is considered the highest form of expression within the dialect,although no user of the dialect has ever demonstrated any talent atcreating them. Special Treatment - Traditionally the dialect is encoded with whatever charset is believedmost likely to confuse a filter. Adoption of the UCS is increasinglycommon amongst the user community as it provides a source of yet morecharacters that look a bit like other characters. Semantic software should not attempt to differentiate between definiteand indefinite articles or should treat them specially. These are notused as they are in other dialects of English where they specify aparticular or a general example of the associated noun, but rather toreflect a self-deluding belief in universality of a fetish item as theuser community has a strong religious belief that we are all equallypathetic perverts to themselves. This religious belief contains astrong emphasis on proselytising for which the user community eagerlyadapt new communication technologies including those produced by theIETF. In rich-text formats in which the text can be coloured there is astrong preference towards red. I'll readily admit that this is probably the most articulate request for a new language tag in a long time, and that reading it left me with only one question:  Arê yØu`l00ki.ng 4_ä gõod t|mé? For those of you wondering how serious this request could be, consider today's date.  For everyone else, please remember that you have two weeks (starting today) to comment on the request.

In response to my first post, Scott asked a very topical question: Not sure if this falls in your basket or not, but I would be interest in any tips or tricks for memory profiling ASP.NET applications. Well, Scott, I'm pleased that you're interested in this subject, because it's a very good way to introduce how we're testing the performance of ASP.NET under stress.  For a grounding on the subject, I encourage anyone interested in this subject to start by reading this.  It's an article written by Thomas Marquardt, a developer (and former tester) on our team who did a lot of work creating the test infrastructure we're using today.  It should also handily answer your question. The other reason why this I want to point you to this article is that the source code for one of the test tools used for stress automation, QQQ.exe, can be downloaded there.  QQQ is a tool that monitors performance counters for the ASP.NET worker process and automatically attaches a debugger if any of a few different critical thresholds are reached.  We use this with the Web Application Stress Tool (which, in the long tradition of giving things names that will freak out everybody in marketing, is also called “Homer”) to simulate the process of multiple users making large quantities of requests to the server and catching the problems that arise.  These tools, combined with some debugging tools, scripts for automatically re-imaging the test machines and launching stress, and (naturally) the test pages themselves, are the way in which we stress-test the product. The tests themselves cover a pretty broad swath of scenarios.  As you'd expect, we have tests simulating high traffic in a production environment with all managed code, but we also have tests for pages that inter-operate with COM components and call data methods.  We also check how ASP.NET handles some more esoteric scenarios.  What happens if  the user changes machine.config every few minutes, causing the application to restart?  What if there's a memory leak in the user's applications?  What if the worker process crashes?  The process model, of course, should gracefully recover from these hiccups.  In all of these scenarios, the pages we use for stress testing are also functional tests that we use for regular verification of the product, so we can get as close to simulating real-life workloads and traffic as possible. I used to debug stress, so a lot of this comes from memory (it's like riding a bicycle), but special thanks go to (fellow tester) Vladimir Kritchko for pointing me to the latest stress infrastructure documentation. 

I'm a software tester, and I work on ASP.NET. Are you still reading?  Interesting. ASP.NET's test team isn't something you hear much about, and nobody who tests the product ranks among the best known team members in our product group (the organizational unit that includes testers, developers, and program managers working on the same closely-related set of products).  However, the importance of the test team around here is hardly minimal.  Just to give you some sense of proportion, testers make up over half of the product group's members. I'm going to start this weblog by describing in hopefully not too much detail the way in which ASP.NET is getting tested; as I think you'll learn, this is different from the kind and level of testing you may have seen (or done) in other products.  We have our reasons for testing the way we do, and maybe you can find value in hearing about how we do it. I also want to talk about ASP.NET itself, with attention on the new features in it that I test.  As soon as I'm done with my overview of how we test ASP.NET in all of its glory, I hope to share with you something useful about Smart Navigation, the ContentPager control, control designers, and the other nifty things I happen to test for a living. Of course, your suggestions are important; I want to share with you the things you want to know about to the greatest of my ability, so don't be afraid to give me your feedback.