第 158 页

Test Run Configuration window in the Controller and Agent node you can select to run the tests

remotely and select the configured controller.

database unit test

You already looked at unit testing in Chapter 11; this section expands the topic to databases. This

kind of test is useful to verify the functionality and design of your schema objects and can work

hand-in-hand with your Data Generation Plans and Schema Database Projects. To create a new

Database Unit Test, from the Tests menu select New Test and choose the template for Database

Unit Test, then create a new project if needed. For the first DB Unit Test in the project you will be

prompted for a database to run the tests against. You can change this later from the Database Test

Configuration option under the Test menu. It’s highly recommended to have a dedicated database

for test purposes for each developer. You can also select to deploy the project before running the

tests. This will guarantee you are always running the tests against the latest version of the schema.

Finally, you can select to use a Data Generation Plan to re-create the data for the unit tests.

To create and edit the tests, you have a designer (Figure 56-7). From there you can select the different test

methods or create new ones and add new test conditions. The conditions available are Empty ResultSet,

Execution Time, Inconclusive, Not Empty ResultSet, Row Count, and Scalar Value. When selected, you

automated Tests .

1101

can configure them from the properties windows — for example, to set the value for the row count.

You can also create your own test conditions. The Data Checksum and Expected Schema test conditions

that were previously released as a part of the Power Tools pack are now also available by default.

fiGure 56-7

You have two other ways to verify your tests. One is by raising errors from your T- SQL code. The

RAISERROR function will take an error message (this will be displayed on the Test Results window in case

of failure), error severity, and error state. If error severity is above 10, it will cause the unit test to fail.

The other way to verify tests is from your C# or VB.NET script. If you right-click your test and

select View Code, you will see it’s structured as a regular unit test as explained in Chapter 11 and

simply has methods that call SQL scripts. When you execute a script, you will get back in your

.NET code an ExecutionResult array; each ExecutionResult object will contain a DataSet,

number of rows affected, and the Execution Time. Using those properties you can make your

assertions. The Execute method of the TestService also takes SqlParameters that could be

passed to your script and can be returned to your C# code for further verification.

coded ui test

Sometimes the best way to test an application is to drive it from the outside as a user would.

When you create a new Coded UI Test, it starts the Coded UI

Test Builder (Figure 56-8). Once you click the Start Recording

button, the Coded UI Test Builder tracks all of the actions that

you take with the mouse and keyboard.

fiGure 56-8

Open your application and use it to get into the state that

you’d like to test, then click the Generate Code button. This prompts you to name your recorded

method, which will be saved in the test project as a part of the UI Map. This map is a description of

actions and assertions that you can use to automate and test your application.

Each test project contains a single UI Map, which all of the Coded UI Tests share.

1102 .

chaPter 56 ViSuAl STudio ulTimATe For TeSTerS

Once your application is in the desired state you can create assertions about different parts of the

user interface. To do this, drag the cross-hair icon from the Coded UI Test Builder over the part

of the UI that you want to make an assertion about. When you release the mouse button, the Add

Assertions dialog is displayed as in Figure 56-9.

fiGure 56-9

On the left is a collapsible panel showing the UI control map, which displays the hierarchy of all

controls that have been identified so far. On the right is a list of properties that the Coded UI Test

Builder has been able to identify along with their values. To make an assertion about one of these

properties, you can right-click it and select Add Assertion. Each assertion has a comparator and a

comparison value to be tested against.

Generic tests

Not every kind of test is covered in Team System. This is why Microsoft included the concept of

Generic Tests so that you can easily use custom tests, but still be able to use the rest of the features

like Test Results, Test List, Assign Work Items, and Publish Test Results.

To configure a Generic Test, you need to specify an existing program and optionally specify its

command-line arguments, additional files to deploy, and environment variables. The external application

can communicate the test result back to Team System in two ways. One is with the Error Level, where a

value of 0 indicates success and anything else is considered a failure. The other is to return an XML file

that conforms to the SummaryResult.xsd schema located in Visual Studio’s installation path. In MSDN

you can find information about this schema and how to report detailed errors using XML.

ordered test

Ordered Tests are used when you need to group tests and run them as a whole, or if tests have

dependencies on each other and need to be run in a particular order. It’s a good practice to create

atomic Unit Tests to be able to run them in isolation with repeatable results. I don’t recommend

using Ordered Tests just to deal with dependencies between Unit Tests. A good reason for creating

Ordered Tests could be to create a performance session for more than one test.

In the Ordered Test Editor you will have a list of the available tests that you can add to the Ordered

Test — the same test can be added more than once. You can also choose to continue after a failure.

When the test is run, it executes each of the selected tests in the specified order.

relating Code and Tests .

1103

relatinG code and tests

Tests and code are heavily interconnected. Tests have no reason to exist without the code that they

verify and code that is not verified by tests is potentially incorrect. Visual Studio contains two tools

designed to make the link between tests and code more explicit. Code Coverage is able to determine

which areas of your code are executed during a test run, which tells you if you need to add more

tests to your solution. Test Impact Analysis enables you to determine which tests need to be re-run

based on the areas of code that you have modified.

Both Code Coverage and Test Impact Analysis are available only for the Premium

and Ultimate editions of Visual Studio 2010.

code coverage

This is a very useful tool. It will instrument the code being tested to help you see which lines of code

are really being executed. First, you need to have a Test Project on your solution. To demonstrate

this, you can refer to the example described under “Your First Test Case” in Chapter 11. Assuming

you have already created the SubscriptionTest class and CurrentStatusTest is passing, you will

now activate Code Coverage.

To open the Test Run properties window, you can double - click the Local.testsettings

file in Solution

Explorer or go to the menu Test . Edit Test Run Configurations and select your active configuration.

The settings for Code Coverage (Figure 56 - 10) are located under the Data and Diagnostics page.

fiGure 56-10

1104 .

chaPter 56 ViSuAl STudio ulTimATe For TeSTerS

You need to select the assemblies to instrument. In case you are signing your assemblies, similar to

the procedure when you are profiling using instrumentation, you need to resign them.

Now, simply run your test and from the Test Results window right-click it and select Code Coverage.

Figure 56-11 shows the Code Coverage window, indicating the not-covered and covered blocks for

each assembly, namespace, type, and member. Double-clicking a member opens the code file with

the executed lines highlighted in blue (light shading in the figure) and untouched lines in red (darker

shading in the figure) as shown in Figure 56-12.

fiGure 56-11

As you can see in Figure 56-11, the

get_CurrentStatus() has 73.68 percent in

not-covered blocks. The first option is evaluated

and returned, so the remaining branches are never

being touched. This is an indication that you will

need additional test cases. This was covered in the

“Data” section in Chapter 11, where you specify a

DataSource with the additional input.

When you have code that is never touched, this can

lead you to think three things:

.

It is code that really isn’t used and is only getting in your way, decreasing your project’s

maintainability. Solution: simply delete it.

.

That code isn’t being tested. Solution: create a new test.

.

The code is so simple that there’s probably no need to test it. Think twice about this,

even for simple properties or methods. The code is likely to be referenced elsewhere in

your application, in which case any errors in the code may cause issues elsewhere in your

application. This is a good reason to write a new test case. Right-clicking the member

and selecting Find All References can help you see if this is unused code, but it won’t find

references from data-bound properties or projects not in the solution.

It’s not necessary to take this practice to an extreme and look for 100 percent Code Coverage. In

many projects it’s not worth it, especially if you have legacy code or you didn’t start using unit

testing. What you need to keep in mind is not to let your code coverage go down. Iteration after

iteration, or better yet (if you are using continuous integration), check-in after check-in, your

percentage should increase, or at least remain constant. Most important, rather than looking at

the numbers at a test suite level, it’s useful to look at them at the code level to see if you’re missing

critical test cases.

fiGure 56-12

Visual studio Test Management .

1105

test impact analysis

It is a good idea to run all of your unit tests fairly

regularly as you develop new features and fix

bugs to ensure you have not broken anything. Having

said that, not every test needs to be re-run each time you

make a change to the code. Once it has been enabled,

Test Impact Analysis is able to determine which tests will

be affected by changes in your code base. The Test Impact

View (Figure 56-13) is split into two sections. The top

section shows the list of tests that need to be re-run based

on the changes you have made to the source code. When

you select a test (or tests) in the top section, the bottom

section shows the method or methods that have changed.

Test impact data is not kept up to date with every change that you make, because this would be

too much of a performance drain on your development environment. This data is instead refreshed

every time you build your solution and when you run tests. When test impact data is out of date, a

refresh button becomes available on the Test Impact View.

fiGure 56-13

The Test Impact View has a run button that allows you to run all of the

impacted tests.

Visual studio test ManaGeMent

The easiest way to manage your tests in Visual Studio is through the Test List Editor explained in

Chapter 11. There you can add the tests to lists, select the tests or lists to run, sort, group, filter, and

enable or disable them. To display the Test List Editor, select it from the Test Windows menu.

Another option is to use the Test View window, which is simpler than the Test List Editor. From

there you can also run tests and filter tests, but can’t group them in test lists.

When you run a set of tests you are presented with the Test Results window. In case it doesn’t

appear automatically, you can open it from the Test .

Windows menu. There you can sort the

tests and results and see error messages. You can select what tests to re-run; by default failing,

inconclusive, and not-executed tests will be checked, and passed tests will be unchecked. Because

the list of results can be big, you can use sorting, filtering, and grouping, or even change the view to

display a left pane with the Test Lists and use the same organization defined in the Test List Editor.

From the Test Results window, you can export the Test Result to save a .trx file containing all the

details and a folder with the test output. You can simply send the files to someone else who can

import them from the Test Results window by selecting the .trx file. This person can see the same

results the tester saw and even rerun the tests to reproduce the error on his or her machine. The

latter is possible because the binaries are included with every test output.

1106 .

chaPter 56 ViSuAl STudio ulTimATe For TeSTerS

Instead of passing Test Result files from one computer to another, it would be better to publish them

to Team Foundation Server. This option is available in the Test Results window’s toolbar. You will

be prompted to select a team project. The team project must have a build configured, as you will see

in Chapter 57 in the section “Team Foundation Build.” The benefit of publishing, besides making

the data available to other people, is that the data can also be used for reports.

test and lab ManaGer

Test and Lab Manager (Figure 56-14) is a new tool for you to plan, execute, and track your testing

activities. It integrates directly with a Team Foundation Server and enables you to create, update,

and query work items directly.

fiGure 56-14

There are two UIs for Test and Lab Manager. When you first open the application it will look like

Figure 56-14. This is the Testing Center. It has four tabs along the top that relate to the basic types

of activities that the Testing Center provides for creating and running suites of tests.

testing center

When you first start Testing Center you are asked to create a new plan. All tests in Testing Center

are organized into test plans and it is recommended that you create a test plan for each releasable