owenG
home learn tableau about
divider
SDLC VS 2010 Coded UI Test Coded UI Test and Fitnesse MSBuild 4.0 MSBuild and IronPython MSBuild and IronPython - TFS checkins MSBuild and IronPython - Custom SQL Data








previous next

Coded UI Test and Fitnesse Part 4

Looping Coded UI to Fitnesse

Two big steps remain: 1) getting Fitnesse to generate a data source .csv for CUIT, and 2) running the CUIT test automagically. For the first step we begin by hooking into one of the defined methods that fitSharp/Fitnesse accepts. In the current scenario List<object> Table(List<List<String>> table), which gets called once per table, is the one to work with. Modify the StringManipulatorTests class like so:

C#
    public class StringManipulatorTests
    {
        string _firstString = "";
        string _secondString = "";
        string _csvData = "";

        ...

        public List<object> Table(List<List<String>> table)
        {
            int rowCount = 0;
            foreach (List<String> child in table)
            {
                for (int i = 0; i < child.Count; i++)
                {
                    string cell = child[i];
                    if (rowCount == 0)
                    {
                        cell = cell.Replace(" ", "");
                        cell = cell.Replace("?", "");
                        Console.WriteLine("Field Header: " + cell);
                    }
                    else
                    {
                        Console.WriteLine("cell data " + i + ":" + child[i]);
                    }
                    _csvData += cell + ",";
                }
                _csvData += "\r\n";
                Console.WriteLine();
                rowCount++;
            }
            WriteDataAndInvokeMsTest(_csvData);
            return null;
        }
        
        private void WriteDataAndInvokeMsTest(string _csvData)
        {
            // TODO: implement
        }

by adding a _csvData string field to the class along with the Table method. The calls to Console.WriteLine do not add any functionality per se but can be very helpful for troubleshooting. If we run the Fitnesse test again, the standard execution status in the upper-right hand corner of results page (assuming test ran OK) will change from 'Test Executed OK' to 'Output Captured:

Test OK arrow Test Output Captured

and after clicking that link, lower-half of the ErrorLogs page will display the output of the Console.WriteLines:


Standard Output:

Field Header: FirstPieceOfText
Field Header: SecondPieceOfText
Field Header: JoinTextTogether
Field Header: SwapTextOrderAndJoin
Field Header: ReverseLettersInFirstPieceOfTextAndJoin

cell data 0:Hello
cell data 1:World
cell data 2:HelloWorld
cell data 3:WorldHello
cell data 4:olleHWorld

cell data 0:New
cell data 1:York
cell data 2:NewYork
cell data 3:YorkNew
cell data 4:weNYork

cell data 0:apple
cell data 1:zebra
cell data 2:applezebra
cell data 3:zebraapple
cell data 4:elppazebra

Now to fill out the WriteDataAndInvokeMsTest implementation, which is set to be called right after the table has been processed. After first adding using directives for System.IO and System.Diagnostics:

C#
    private void WriteDataAndInvokeMsTest(string _csvData)
    {
        using (StreamWriter sw = new StreamWriter(@"\\localhost\TestData\StringManip\input.csv"))
        {
            sw.Write(_csvData);
        }

        Process proc = new Process();
        string vsToolsPath = Environment.GetEnvironmentVariable("VS100COMNTOOLS");
        string vsIdePath = vsToolsPath.Replace(@"\Tools\", @"\IDE\");
        string msTestPath = Path.Combine(vsIdePath, "mstest.exe");
        string testArgs = @"/testcontainer:C:\owenG\projects\FitnesseTestHarness\CuitStringManipWebTest\
            bin\Debug\CuitStringManipWebTest.dll /test:StringManipulationTestMethod1DataDriven";
        ProcessStartInfo psi = new ProcessStartInfo(msTestPath, testArgs);
        psi.UseShellExecute = false;
        proc.StartInfo = psi;
        proc.Start();
    }

Csv file is written to the location expected by the StringManipulationTestMethod1DataDriven test and then that same test is run by launching mstest.exe. When Test       is clicked the workflow should now look like:

  1. contents of the String Manipulator Decision table are read
  2. csv is written and the mstest.exe process started (the Table method is called before the rest of the methods)
  3. Fitnesse actually tests the table contents and success/failure is registered in the wiki page

Of course the amount of table data is quite small and the tests simple, so the Coded UI test kicked off by mstest.exe doesn't get going until after Fitnesse has returned results in the browser window. And in this current scenario the initial Fitnesse test has been kicked off manually, with the CUIT test running on the same machine but there are other possibilities. The Fitnesse test can be run from the Command Line and perhaps initiated via an msbuild task. With the help of PsExec MSTest.exe could be run on a machine different from that hosting Fitnesse. As an aside, I had initially saved the csv content in a class level field and then made the call to WriteDataAndInvokeMsTest in the class destructor. It seemed to work fine but ...

What about Coded UI without MSTest?

With some of the other GUI testing frameworks that can be accessed via a Fitnesse-supported language the UI automation code could actually go right into the primary test .dll. An example would be Selenium, where there is even a Selenesse project to allow markup in the wiki to drive the automation process. However, the Coded UI playback appears to be more or less tied to the overall test framework, where it should be initiated either through the IDE or via MSTest.exe. That doesn't mean it isn't possible though, merely unsupported and probably a pretty bad idea.

The Attempt

First hurdle is getting fitSharp to speak to a dll targetting the v4.0 Framework, which would at some point be necessary since CUIT is only available in 4.0. Since I already had a test .dll, I chose to call the Coded UI test method from there, as opposed to trying to add the methods being called by Fitnesse to the Coded UI Test .dll. There are no doubt other approaches but I chose to get everything involved up to 4.0, beginning with fitsharptStringTest.dll. Next were fitsharp binaries I was working with - the 1.6 version I downloaded was set for v3.5 of framework, so I first needed to get the C# source code. That involved downloading and installing some incorrect Git, followed by a correct Git, followed by a quick bash and clone. Upgraded projects for fit, fitSharp, Runner, and RunnerW to v4.0, fixed some build environment errors re ambiguous calls to System.Action, and copied output to a Slim1.6_v4 directory. Added reference to CuitStringManipWebTest project to the fitsharpStringTests project and updated the WriteDataAndInvokeMsTest method to call the original single-run Coded UI Test method instead of kicking off MSTest.exe:

C#
    CuitStringManipWebTest.CodedUITest1 cuit = new CuitStringManipWebTest.CodedUITest1();
    cuit.StringManipulationTestMethod1DataDriven();

Created a new Vs2010StringManipulationTest Fitnesse page with same contents as the original page but updated with the new Slim1.6_v4 path. Run the Fitnesse test and although it passes there are Exceptions written to results page. With the earlier Fit, as opposed to Slim, setting the TEST_RUNNER to RunnerW.exe would often be the next step when a test hung or failed for unknown reasons but Slim is writing a good amount of exception detail directly to the web page. In this case the error is Could not load file or assembly 'Microsoft.VisualStudio.TestTools.UITest.Playback' and not too surprising. Find that .dll in C:\Program Files\Microsoft Visual Studio 10.0\Common7\IDE\PrivateAssemblies and copy to the Slim1.6_v4 directory. Get another error re missing files and in the end wind up copying a number TestTool dlls in order to get this test running. In the end the list of DLLs copied ran to below, where at least one was living in \PublicAssemblies:

    Microsoft.VisualStudio.QualityTools.CodedUITestFramework.dll
    Microsoft.VisualStudio.TestTools.UITest.Common.dll
    Microsoft.VisualStudio.TestTools.UITest.Extension.dll
    Microsoft.VisualStudio.TestTools.UITest.Extension.IE.Communication.Interop.dll
    Microsoft.VisualStudio.TestTools.UITest.Extension.IE.dll
    Microsoft.VisualStudio.TestTools.UITest.Extension.MSAA.dll
    Microsoft.VisualStudio.TestTools.UITest.Extension.Uia.dll
    Microsoft.VisualStudio.TestTools.UITest.Framework.dll
    Microsoft.VisualStudio.TestTools.UITest.Playback.dll
    Microsoft.VisualStudio.TestTools.UITesting.dll

Of course a different CUIT, targetting different an app with different technology, might require a different set. Running the Fitness test now came up with a new error, the standard 'System.NullReferenceException: Object reference not set to an instance of an object'. Was able to trace it back to the first UI interaction in the Coded UI Test, a Mouse.Click, where the HtmlInputButton control was null. Some research led to the need for initiating Playback manually in situations where one needs access to methods in Microsoft.VisualStudio.TeamTest.UITesting namespace when coding within the Coded UI framework, but not within the [TestMethod] itself. Gave that a try in my fitsharp test dll by adding a reference, and using directive, for that .UITesting .dll. Then wrapped the call to the CUIT test like so:

C#
    Playback.Initialize();
    try
    {
        // Your one time per class initialization code goes here
        CuitStringManipWebTest.CodedUITest1 cuit = new CuitStringManipWebTest.CodedUITest1();
        cuit.StringManipulationTestMethod1DataDriven();
    }
    finally
    {
        Playback.Cleanup();
    }

Back to testing in Fitnesse and ... it works! The web page is cleared, "foo" and "bar" entered, etc. With that completing, try to call the CUIT that implements the csv, StringManipulationTestMethod1DataDriven. That does NOT work, get another null reference exception. Which turns out to be from the TestContext, or null status of such, in the CUIT test method at run-time. I tried a few workarounds but didn't think it worthwhile for serious investigation since the base scenario was unsupported in the first place.

THE END.

previous next