This is the next paper in our series of articles on future technology directions at Macrosoft. Our research teams are always on the lookout for new technologies that will automate and enhance quality of our development and QA processes. We are continually evaluating and moving to best in class technologies to better serve our clients.
This paper describes our evaluation and beginning implementation of Cypress.io for automated web application testing. We started taking a close look at Cypress about 3 months ago and are now convinced this is the way to go for front-end web application testing. There are several key advantages that we describe in this paper. We discuss these major benefits of using Cypress over our current testing tool Selenium for front-end web application testing. We feel strongly that shifting over to Cypress will significantly enhance and automate our web application work on behalf of our clients.
This case study on Cypress is one example among many of the ways our technical teams actively seek out new and better methods for every step in our development and testing processes. Others will be highlighted in subsequent papers. Our enterprise development and testing processes and methodologies are continually undergoing innovation, and our developers and testers are continually undergoing new training to fully take advantage of these new tools and technologies.
Cypress is an open-source test automation tool for testing web applications. Unlike other automation tools, Cypress can also interact directly with web browsers and is capable of testing modern web applications based on React and AngularJS frameworks as well.
Cypress uses its own architecture and can directly interact with web elements. It doesn’t use Selenium at its backend as with other UI testing tools we have evaluated in the market today. Cypress has its own rich set of powerful commands which it executes inside the browser itself, whereas Selenium executes its commands remotely using a server.
Below we list the major features of Cypress that make it an excellent option for us to use in our testing work on web applications.
- Using Cypress, we can perform all kinds of testing: End-to-End, Unit, Integration, etc.
- Cypress supports most types of commonly used browsers like Chrome, Firefox, Edge, Electron, etc. It interacts with these browsers directly without help of any browser drivers.
- Cypress can take automatic screenshots of the application during the test run. This feature is called Time Travel. Once the test run has been completed, we are then able to see the captured screenshots in a section called Command Log.
- Cypress has a unique debugging feature: we can see the state of an application element before and after the command execution using a simple click of a button.
- The chances of getting flaky or irrelevant tests are very minimal.
- Cypress uses 2 modes of executing tests: GUI mode and Command-line mode.
Testing is one of the critical processes in successful application development. The success or failure of the application entirely depends on it. However, website testing is totally different from conventional software testing, and the complexity has increased greatly from what it was even 3-5 years ago. In the face of this much greater complexity, staying with the old testing tools and methods leads to much greater testing efforts and quality challenges.
Below are some of the hurdles and challenges to modern web testing efforts that now make web testing extremely challenging for testers, often causing delays and additional rounds of testing.
- Dealing with XHR calls and web services
- Short deployment sprints that often overlap with the major time periods involved in testing
- Security of data
- Very expensive to maintain the web applications due to lack of a well-defined infrastructure for testing these web applications
- Dynamic behavior of web application due to modern development frameworks
These are some of the problems and challenges associated with Selenium. Selenium has been a major player in E2E web application testing for a decade now, but the modern web is different today. To overcome these shortcomings of Selenium, Cypress comes into the picture here.
The six-month download trend comparison of Selenium WebDriver and Cypress clearly indicates that the Selenium vs Cypress battle has been intensifying in recent months. As shown, Cypress has been widening its lead in downloads over Selenium WebDriver over the last six months. Clearly this trend is indicative of the added usefulness and functionality of the Cypress test automation framework.
In the table below, we compare Cypress with Selenium across most of the major features relevant to modern web application testing. While the first 5-6 features seem to indicate limited functionality for Cypress compared to Selenium, it is in the later set of features that the true value of Cypress comes to the fore.
|Browser Support||Chrome, Firefox, Internet Explorer, Microsoft Edge, Safari||Chrome, Edge, Firefox, Electron.|
|Integrations||Wide range of integration options — CI/CD tools, reporting tools, and more.||Limited integration support with CI/CD tools when compared to Selenium.|
|Driver Dependencies||An appropriate browser driver must be installed so that the test script can talk to the corresponding web browser.||No driver dependency.|
|Multi Tabs||Supported.||Not supported.|
|Multiple browser instances||Supported.||Not supported.|
|Execution Speed||Slow.||Fast as Cypress scripts are executed within the browser.|
|Time Travel||Not supported.||Supported.|
|Real-time reloads||Not supported.||Tests are reloaded when any change is made in the test implementation.|
|Automatic Waiting||Not supported.||No requirement to add waits or sleep in the tests. Cypress automatically waits for commands and assertions before moving to the next instruction.|
|Default Screenshots and Videos||Not available by default, the developer must write code to achieve the same.||Available by default.|
|Mobile Testing||Mobile Testing with Appium.||Not supported.|
|Test Flakiness||Tests can be flaky.||With Cypress, tests are expected to be non-flaky.|
The side-by-side charts below show the simplifications achievable with Cypress compared with our current web application testing environment based on Selenium tool. Clearly, as shown, very significant simplifications are possible with Cypress, and it is those advantages that have attracted us to start using Cypress for selected web application testing work.
Installing Cypress inti your test environment is straightforward and simple. It is all about the following two commands:
- npm init
- npm install cypress –save-dev
Installing Cypress will take around 2 to 3 minutes, based on network speed.
The first command creates a package.json file and the second command installs Cypress as a devDependencies array.
Cypress is a powerful web test automation tool for testing modern web applications. It supports the testing of all types of web elements easily using powerful in-built commands. It also supports the Page Object Model framework.
Macrosoft’s testing teams are now beginning to use Cypress in their web application testing and it is clear to us that with Cypress we are achieving very significant gains in automation, and in reducing the overall tester efforts, while achieving greater levels of quality testing. For us it is a clear advantage, one that we can pass onto our clients. Please feel free to contact us to discuss the advantages we see in using Cypress, we will be glad to share our experiences to date. Stay tuned for more papers from Macrosoft highlighting our advancements in tools and technology for use in all aspects of our company’s development and testing processes. We are constantly seeking new best-in-class tools and techniques that better the enterprise development work we do for our clients.
By Venkatraman Rajaram, G.N. Shah, Ronald Mueller | August 25th, 2021 | Enterprise Services