For about a year now I’ve been leading our mobile UI automated tests unit in Xpand-IT leveraging Xamarin Test Cloud, researching and developing mobile UI tests for clients. We came a long way since then, strengthening our relationship with Xamarin by becoming one of the few Xamarin Test Cloud Consulting Partner worldwide (after our Xamarin Premier Consulting Partner mark) and providing tests development services and Xamarin Test Cloud subscription alternatives to the market. Does it show how proud I am of our work? 🙂
Today I’m taking the oportunity to share some Xamarin UI Test / Xamarin Test Cloud knowledge, following Xamarin Evolve 2016 contest. In this article you’ll see how we can automate any Android app (we’ll use Twitter)!
Xamarin UI Test and Xamarin Test Cloud
Quoting the official Xamarin UI Test (XUT) description:
Xamarin.UITest [is] an Automated UI Acceptance Testing framework based on Calabash that allows programmers to write and execute tests in C# and NUnit that validate the functionality of iOS and Android Apps.
Code wise, it’s all about querying the UI and acting on the UI elements:
app.WaitForElement("fooButtonId"); app.Tap("fooButtonId"); app.Screenshot("Take a screenshot to create a new step in XTC!");
The official documentation has grown to be the best source of information, you’ll find everything there to get you started. One thing you need to have in mind though, you can test ANY Android app without recompiling but, iOS apps need to be compiled with Xamarin Test Cloud Agent. This tests can run locally on emulators and physical devices. Currently, Windows Phone is not supported but, Xamarin is working on it.
Xamarin Test Cloud (XTC) is a device cloud service that enables you to run your tests in physical devices somewhere and when the test execution finishes, get the final report of each step in each device with screenshots, performance information, console output and error logging in case something fails.
Since XTC has more than 1700 devices, you can use all of this power to testdrive your app in devices with different resolutions and OS versions, iOS and Android.
Xamarin UI Test Sample: Twitter
Now, on to the good stuff! Our first test script will be for the Twitter (android) app, we will:
- Tweets about the contest! #TweetCeption
- Run the test script in several devices
Feel free to browse the results, one image doesn’t despict the true awesomeness of this service. Later on I’ll do a walkthrough the code but, following this sample you can learn:
- Crossplatform test arquitecture for iOS and Android
- One test script to rule them all!
- How to leverage multiple environments configurations (DEV, QA, PROD)
- Project structure to test any Android app
- Query Xamarin Test Cloud environment variable with the device index
- Structure helper methods to reduce code repetition for common tasks like “Wait for an element and Tap it“
For laughs and giggles, heres what our tests did:
Test Results Analysis
In the above test run you can see that there was an error in a one step and just by seeing the screenshot of the device I knew what went wrong: The app presented an alert message that the test wasn’t expecting!
More specifically, the alert message informed the user that the location services were disabled and the user would need to enable them manually.
Notice that this only happened in 1 out of 10 devices, I wish I could say this was intended but, it’s the reason why Xamarin Test Cloud is huge for mobile development! With this information you could check if your app was correctly dealing with this scenario if the user pressed Cancel.
Have you used XUT or XTC? What do you think about it?