Xamarin Test Box, the first local test server.
Every day we are more involved in DevOps, which allow us to run processes automatically
DevOps gives us the chance to focus on the development, since we know that Build, Testing and Delivery will be done without much effort. Meaning that testing plays a very important role on this process and within this we find Unit tests and also UI tests. Therefore, we can run both of them locally using Visual Studio or Xamarin Test Cloud (subscription needed) for the UI tests.
Some time ago, I was looking for an uncoupled tool which could run all my applications’s tests without being tied to Visual Studio or XT Cloud and that could also be… simple. Unfortunately, I could not find something like this. For this reason, the need of having a tool hosted in a local machine and responsible for the testing phase increased.
So let me show you how we could go through a testing process with only 2 clicks…
What is Xamarin Test Box?
For now they are not more than 3 scripts: 2 Batch scripts (Run.bat and StartTesting.bat) and 1 Bash script (pull-git-repo.sh). They work together with Git Bash, MsBuild, xUnit and NUnit to pull your code from the git repository, compile Android, UnitTest (using xUnit) and UITest projects, then run the Unit tests and also the UI tests in a real connected device.
In this first version Xamarin Test Box can manage only one Solution folder and the UI project can be run on Android device (I am working to support iOS devices too).
How it works?
First of all check the requirements and be sure you does not lack any of them.
- Visual Studio for Xamarin
- NUnit Console for Windows – http://nunit.org/download/
- Git Bash – https://git-for-windows.github.io/
- Xamarin Solution with UI Test Project and Unit Test Projects( with xunit) within a git repository in your file system.
- Add xUnit console package in your UnitTest project – by Package Manager in VS or https://www.nuget.org/packages/xunit.runner.console
- UITest project with a real device configured (e.g: check Test.Cs section in http://sebacabrera.com/automated-xamarin-ui-tests-and-recorder/)
Visual Studio solution
If you have never compile your solution on this machine, do it!
NuGet packages should be in the file system so be sure they were restored after building. [Enabling and disabling package restore](https://docs.microsoft.com/en-us/nuget/consume-packages/package-restore#enabling-and-disabling-package-restore).
If you have not used the bash, please login with your credentials because you will need it later to get the last version of the tests.
Give read/write permission to TestResult.xml (located into NUnit console folder)
After that you should configure the files: pull-git-repo.sh and StartTesting.bat
- In pull-git-repo.sh: Set your solution folder (associated with the git repository)
- In StartTesting.bat set:
- Project Name
- Android Project file path
- XUnit Console path (it should be in your solution´s packages folder)
- UnitTest Project file path
- UnitTest Release library path
- NUnit Console path
- UITest Project file path
- UITest Release library path
Once you have all the configuration ready, you only have to open Run.bat and the process will appear like that:
1. Git Pull on the current branch of the project
2. Next step: Building
-Android project (If your git pulls take much longer time, you can increase the waiting time for compiling Android: into Run.bat – line 10)
3. Running Unit tests in secondary console.
4. Running UI tests in the connected device
Mirroring a connected Android device.
UI Test Results
Despite of having to configure a folder for each Xamarin solution, you have the advantage of testing all of your applications without needing to use Visual Studio or Xamarin Test Cloud. Personally, I have shortcuts on my desktop relating to the different Run.bat files for each project.
Only 2 clicks or a scheduled task to run all your tests.
Clear Outputs: 1 console for Unit tests and another one for UI tests.
Maximize Testers skills: it can be installed on Testers’s machines and it will help them to cover more kind of tests.
- Run Unit tests created with NUnit
- Restore NuGet packages automatically
- Support multiple Visual Studio solutions
- RunUI tests in iOS device
- Mac support
- Run UI tests in multiple connected devices.
- Continuous Integration with GitHub Repos using WebHooks and Node.js
To sum up
Although I recognize that it has many limitations, I think it allows us to cover our testing phase in parallel with the development. Also, I know there’s a long way ahead of us, but this is a kickoff to have a tool that connects to a continuous integration flow and runs on multiple physical devices (iOs and Android).
Check the Xamarin Test Box in GitHub
I hope it can help you as much as it has helped me.