Close [x]

Introduction to Magento Testing Framework

Edit this page on GitHub

Preface

This guide provides instructions on installing and configuring the Magento Testing Framework (MTF). With the Magento Testing Framework you can create and run functional tests to make it easier to perform basic acceptance testing, smoke testing, regression testing, etc. MTF does not contain tests. All functional tests are within Magento code in <magento2>/dev/tests/functional/. This MTF guide is also applicable to the Magento 1 platform.

MTF is an open source cross-platform solution (that is, does not depend on a specific operating system).

MTF enables you to quickly develop functional tests for the Magento application. These tests can be performed at any time.

You can run a single test independently, many tests together (that is, a test suite), or you can run all available tests.

What tools should I use to run tests with MTF?

What do I have as output after running tests with MTF?

  • Tested application

  • Basic PHPUnit results

  • Screenshots of failures

  • Logs of failures

Audience

This guide is intended to be used by any Magento developer. In addition, it can be used by software engineers such as QA specialists, PHP developers, and system integrators.

Goal

Facilitate functional testing and minimize efforts to perform regression testing.

Scope

MTF is purposed to test user interactions with web application under test.

MTF works with functional tests located in <magento2>/dev/tests/functional/.

Out-of-the-box tests cover basic functionality. Extended functionality can be tested using customized tests, created with MTF.

Relative to your software development lifecycle, the MTF can help you:

  1. During the development phase, test any changes of functionality (new modules, update modules, fix bugs).

  2. During the maintenance phase, for periodic automated regression testing.

MTF use cases examples

  1. As Magento developer I want to cover implemented functionality with new tests (for example, added attribute on Customer Form, extended Search functionality, added tags for Products etc).

  2. As a software engineer I want to perform regression testing before release to be confident that Magento works as expected with new functionality.

Non-functional testing

MTF works with tests from <magento2>/dev/tests/functional only.

For other tests please see the following topics:

Follow the MTF project and contribute on GitHub https://github.com/magento/mtf.