Mike Borozdin (mike.borozdin at cloudreach)
Introduction
This article gives a general overview of behaviour-driven development (BDD), talks about .NET tools for BDD (SpecFlow) and UI testing (White) and proceeds with a working example giving hands on BDD in .NET.
Behaviour-Driven Development (BDD) and Outside-in Development
Behaviour-Driven Development (BDD) is a philosophy of software development where the focus, as the name suggests, is on implementing behaviour. First proposed by
Dan North, this approach to development does away with a lot of the confusion — exemplified by questions like 'What do I test?', 'Where do I start testing?', and 'Where do I stop testing?' — that developers who set out to use TDD are faced with.
Outside-in development is a complementary technique that requires developers to focus primarily on defining customer requirements expressed as acceptance criteria, and work inwards towards building an implementation from there. BDD facilitates outside-in development by requiring the description of acceptance criteria as desired behaviours.
Let's take a look at a quick example. Consider the following user requirement:
As a user of this awesome application
I want to log in using my authentication credentials
So that not every Tom, Dick, and Harry can look at my precious data
One of the acceptance criteria for this story would be:
Given I am on the / page of the web application
When I enter my (awesome) authentication credentials
Then I should be presented with the home page
This criterion, written in plain English, describes the desired behaviour. This acts as an ideal starting point for us to write our acceptance tests. Once the acceptance tests have been written, a developer may then move inwards, writing integration and unit tests for the implementation that is required. It is important to note here that BDD does not end with the acceptance tests, but should be used throughout as we move inwards (see how Dan North suggests using shouldExhibitBehaviour(...) to describe tests rather than testExpectedFunctionality(...)).
Next, we show how to implement the outermost step of the BDD process in a .NET project. To do this, we use SpecFlow, a tool that allows you to translate acceptance criteria written in plain English into executable tests.
SpecFlow
SpecFlow translates human-language acceptance tests into runnable .NET code. What is even more interesting is that it is capable of generating test reports that can be understood by non-technical stakeholders like business analysts.
Acceptance tests (sometimes called scenarios in the literature) are specified in
Gherkin, a plain text DSL that adds some semantics to plain English. An acceptance test describes a sequence of events in the form:
Given [pre-condition], when [action], then [effect], when [another action], then [another effect], and so on
. For example,
Given I am in the New Customer Form
When I leave the company name field blank
Then I should be presented with an error saying so
Here's a more complex example with multiple actions:
Given I am in the Login Form
When I enter a valid e-mail in the e-mail field
When I enter a valid password in the password field
Then I should see the Dashboard window
Testing Desktop UIs with White
As an acceptance test describes behaviours that are visible to the end-user, they typically require a means of instrumenting interactions with the application's UI; this allows you to automate actions like clicking on buttons, filling in text fields, and searching for displayed text.
In this post, we describe the use of White, a UI testing tool that allows you to automate interactions with desktop applications written in .NET (WPF and WinForms), as well as native Windows applications.
Working Example
Let's use SpecFlow and White to write a simple desktop application.
Stories and Acceptance Criteria
The example application we shall look at in this post is a simple app that takes a customer's details and saves it to a database.
To start us off, our business analyst has provided us with this story,
Feature: Saving Customer Information
As a sales manager
I want to save a customer’s details
So that I can contact them later
and these acceptance criteria.
Scenario: HappyPath
Given I am in the New Customer window
When I enter a company name in the company name field
When I enter a contact e-mail address in the e-mail field
When I click on the Next button
Then I should see a newly added customer in the list of customers filed by their company name
Scenario: Blank Company Name
Given I am in the New Customer window
When I leave the company name field blank
When I click on the Save button
Then I should be presented with an error saying so
These criteria are not exhaustive, but should suffice for the purposes of this post.
Installing SpecFlow and White
SpecFlow is available on NuGet. However, at the time of this going to the metaphorical press, the latest version of White isn't. You can work around this by
downloading the White assemblies and then adding references to White.Core.dll and White.NUnit.dll.
When writing acceptance tests, it is a good idea to isolate them from the code under test by creating a separate class library project in your Visual Studio solution.
Gettings Started with SpecFlow
The acceptance criteria as well as the user story definition go into a SpecFlow feature file. You can create one on Visual Studio (Add-> New Item -> SpecFlow Feature file). SpecFlow feature files may contain an arbitrary number of acceptance tests. Moreover, acceptance tests for the same feature can be split across multiple feature files.
A SpecFlow step definition file that is used to instrument your tests is generated whenever you create a SpecFlow feature file. This is where you specify the instrumentation that allows you to perform the actions described using your DSL.
You can use the stub below to write your own SpecFlow step definition file for the acceptance criterion described in our example.
[Binding]
public class SavingCustomerInformation
{
[Given("I am in the New Customer window")]
public void GivenNewCustomerWindow()
{
}
[When("I enter a company name in the company name field")]
public void WhenEnterCompanyName()
{
}
[When("I enter a contact e-mail address in the e-mail field")]
public void WhenEnterEmail()
{
}
[When("I click on the Save button")]
public void WhenClickSaveButton()
{
}
[Then("I should see a newly added customer in the list of customers filed by their company name")]
public void ThenNewCustomerInList()
{
}
[When("I enter a company name that has been already added in the company name field")]
public void WhenEnterExistingCompanyName()
{
}
[Then("I should be presented with an error saying so")]
public void ThenError()
{
}
}
The snippet given below is a plain piece of C# code, the only interesting bit are the special annotations [Given], [When], and [Then] that specify that the annotated method will be executed upon running a statement declared in the annotation.
White to the Rescue
It is reasonable to expect that such a method should perform UI manipulations such as populating forms and clicking on buttons. Working with White starts with launching or attaching to an existing process of an application we want to test.
Application applicationUnderTest = Application.Launch(pathToApplication);
Every acceptance tests of ours has the same pre-condition block:
Given I am in the New Customer window
That translates into finding an appropriate window:
//the window object will be used in many methods, so it is reasonable to make it a member variable
private Window newCustomerWindow;
[Given("I am in the New Customer window")]
public void GivenNewCustomerWindow()
{
this.newCustomerWindow = SavingCustomerInformation.applicationUnderTest.GetWindow("New Customer");
}
Our scenarios also involve populating some fields which can be also easily done with White. Essentially, when you need to manipulate any control with White you first find it by either its name or any other criteria and they you perform manipulations on it. The two methods below illustrate this technique applied to writing in a text field and clicking on a button.
[When("I enter a contact e-mail address in the e-mail field")]
public void WhenEnterEmail()
{
TextBox emailTextBox = this.newCustomerWindow.Get<TextBox>("emailTextBox");
emailTextBox.Enter("some@email.com");
}
[When("I click on the Save button")]
public void WhenClickSaveButton()
{
Button saveButton = this.newCustomerWindow.Get<Button>("saveButton");
saveButton.Click();
}
Putting It All Together
Speaking about some certain aspects of using SpecFlow and White, it is still necessary to have an entire picture of the whole testing suite, that is given below.
[Binding]
[Binding]
public class SavingCustomerInformation
{
private static Application applicationUnderTest;
private Window newCustomerWindow;
private const string COMPANY = "Some Company";
[BeforeFeature]
public static void LaunchApplication()
{
SavingCustomerInformation.applicationUnderTest = Application.Launch(@"..\..\..\Application\bin\Debug\Application.exe");
}
[Given("I am in the New Customer window")]
public void GivenNewCustomerWindow()
{
this.newCustomerWindow = SavingCustomerInformation.applicationUnderTest.GetWindow("New Customer");
}
[When("I enter a company name in the company name field")]
public void WhenEnterCompanyName()
{
TextBox companyTextBox = this.newCustomerWindow.Get<TextBox>("companyTextBox");
companyTextBox.Enter(COMPANY);
}
[When("I enter a contact e-mail address in the e-mail field")]
public void WhenEnterEmail()
{
TextBox emailTextBox = this.newCustomerWindow.Get<TextBox>("emailTextBox");
emailTextBox.Enter("some@email.com");
}
[When("I click on the Save button")]
public void WhenClickSaveButton()
{
Button saveButton = this.newCustomerWindow.Get<Button>("saveButton");
saveButton.Click();
}
[Then("I should see a newly added customer in the list of customers filed by their company name")]
public void ThenNewCustomerInList()
{
ListBox customersList = this.newCustomerWindow.Get<ListBox>("customersListView");
Assert.NotNull(customersList.Items.Find(x => x.Text == COMPANY));
}
[When("I leave the company name field blank")]
public void WhenLeaveBlankCompanyName()
{
TextBox companyTextBox = this.newCustomerWindow.Get<TextBox>("companyTextBox");
companyTextBox.Enter(string.Empty);
}
[Then("I should be presented with an error saying so")]
public void ThenError()
{
Assert.NotNull(this.newCustomerWindow.Get(SearchCriteria.ByText("Company name is empty")));
}
[AfterFeature]
public static void KillApplication()
{
SavingCustomerInformation.applicationUnderTest.Kill();
}
}
Having the whole acceptance test code at a glance, it is evident that actual testing occurs in methods annotated with [Then]. Such methods contains Assert expressions and it is possible to make use either MSTest or NUnit.
Methods that have attributes of [BeforeFeature] and [AfterFeature] are executed before and after a feature respectively. In the code snippet above, such methods are used for launching and killing the application. Since the methods carrying such annotations are required to be static, the member variable containing a reference to the application under test is also declared as static.
Running your Tests
Since SpecFlow tests are compiled into DLL files, it is not possible to run them on their own. There are multiple tools that can do this for you, ranging from the built-in in Visual Studio MSTests to SpecRunner, which comes as a separate dependency in NuGet, and NUnit.
SpecRunner produces neat and most human readable HTML reports, while NUnit allows you to run individual scenarios. However, when using NUnit, you should bear in mind that the [BeforeFeature] annotation behaves like [BeforeScenario]. Thus, if you are launching an application under test in such methods, it will be executed on each acceptance test instead being run once for all the tests of a given user story.
If you'd like to switch runner, you can either go to Tools -> Options -> SpecFlow -> General -> Test Runner Tool or just modify app.config file of a test project. If you want to use NUnit as a runner, then the only option is modifying app.config, as NUnit is not present in the Tools drop-down menu.
<specflow>
<unittestprovider name="NUnit">
</unittestprovider>
</specflow>
More SpecFlow Goodies
SpecFlow step definition annotations can also use regex matching, thereby allowing you to pass parameters into generic step definitions. For example, instead of simply stating:
When I enter a company name in the company name field
You can write:
When I enter ‘Acme Ltd.’ in the company name field
Then I should see a message saying ‘A new company has been added’
And then receive these parameter values in your code:
[When("I enter ‘(.*)’ in the company name field")]
public void WhenEnterCompanyName(string companyName)
{
TextBox emailTextBox = this.newCustomerWindow.Get<TextBox>("emailTextBox");
emailTextBox.Enter(companyName);
}
[Then("I should see a message saying ‘(.*)’")]
public void ThenEnterCompanyName(string message)
{
Assert.NotNull(this.newCustomerWindow.Get(SearchCriteria.ByText(message)));
}
Of course, it’s not necessary to have acceptance test written in this way, as it might make them brittle and less robust, but it’s nice to have such an opportunity, just in case.
Now Write Some Code for Your Application
If you were to run your tests now, they would fail (or to be precise, they'd throw an exception as the application to test still doesn't exist). That is not a problem, as the whole point of TDD is to use your tests to guide your code. You can now write your code (writing any tests that may entail first), writing just enough code to make the tests pass.
You can download all of the source code for the example application, including the tests and the application,
here.
