End to End Testing

Might look into http://appium.io/
Would love to hear what you ultimately settle on :slight_smile:

thanks @thetc! I will take a look at appium

1 Like

I use Appium for testing my applications which I build with EXPO.
But it takes a lot of time to create new build every time when I want to test my app.
Maybe exists some way to run expo app in Appium or something similar way.
For example expo.(apk|app) on my machine.

Hi @maxboltik,

Just to share some of our experience here.
We built a set of automated tests with Appium for our Expo App, then pushed the Expo app to a special accont that we created for testing only.

Then we built the standalone binaries for the IOS and Android, be careful that you need to make a binary for simulator for IOS.

We then upload those expo binary to appetize.io, which has its built in Appium server for each mobile instance. One thing to note is that the expo you run on Appetize won’t update itself on start, so you have to trigger a reload with Appium to take advantage of the auto update in Expo.

The plus side of this is that you can run you Appium tests with any online continuous integration provider like Codeship.

Appetize also supports record video, the only downside of it is that it only has 100 free minutes per month, but it is not that expensive anyways.

Alternatively, you can run Appium tests with Expo apps on AWS Device farm. But only if your Appium tests are written in Python or Java. We haven’t explored much there so I don’t have that much more to tell you there.

1 Like

Is this still working for you? When I try to connect to appetize.io with IOS I am getting ‘Invalid osVersion for appium’

What are you using for your device and osVersion keys? This is happening to me using their Java and Node samples.

Hi @leimengda That binary that you built. I suppose is an “.IPA”? In your capabilities of appium tests, what do you put in the app root? Do you have to convert the .ipa to .app? or do you put the .ipa? Could you give me some code example? Thank you so much.

@leimengda Were you able to get this working with an app that was ejected to ExpoKit? Did you need to eject the app at all?

Hi Allen, our app was not ejected, but I suppose you can still get App working.

Get Outlook for Android

@leimengda can you please provide some detail on how you were able to do this on your local machine for iOS? When I ran my Appium Script in the iOS simulator I was able to load the Expo app but not able to load my React-Native app within Expo (think you need to specify the URL of the local server that’s hosting your app’s JS bundle). Also, you need to generate a debug build for Appium to work. How can Appium work if this the .ipa file hasn’t been codesigned with an Apple Developer Certificate???

For more informations see question: ‘Do you modify my app?’

Just created a ticket on AWS Device Farm: Forums

I keep meaning to try out Cavy (https://github.com/pixielabs/cavy). It obviously can’t do everything that Appium does, but it seems a lot easier to work with, and there’s no reason it shouldn’t work even with a non-detached Expo project. Seems like you could fully-automate testing via CI by publishing an Expo bundle with the Cavy scripts enabled and building a simulator build that points to it.

Actually I’m using detox in parallel to test on simulator too: https://github.com/wix/detox

But to test on device automatically Appium + AWS Device Farm is the key solution. (Android test work fine but ios still have an issue on launch, maybe re-sign from ADF is the reason)

Hello! I’d like to understand what are the E2E testing possibilities using Expo at the moment. Detox seems like an interesting solution, but running tests on iOS devices is not possible yet, which rules out (AWS) Device Farm testing. How about Appium, this seems like a viable option, especially now that AWS has added Appium JS support. But how does an Expo app play with Appium/AWS Device Farm? Is ejecting to ExpoKit/RN required or could one manage without ejecting…? Cheers!

There is already a tutorial for e2e testing with Detox in Expo’s blog:

1 Like

Thanks @bankify_expo_admin, it’s a great article! However, it covers only Detox which unfortunately is not an option if you want to test on real devices in AWS Device Farm for example, at least to my understanding.

Note to people trying to read that Detox article in Medium — the embedded gists don’t work in Safari, so you’ll have to read it in Chrome or Firefox.

For anyone still looking for potential solutions, I’m just finishing the beta version of a way to quickly do an E2E “smoke test” by uploading the apk/ipa, no appium setup or writing tests. It’s android only at the moment with iOS coming soon: https://www.smoketest.ai

It uses Appium and attempts to crawl your app and hit as much functionality as it can, then return test results and video. It also works with AWS Device Farm, but I haven’t enabled that option as it costs money to run. If anyone is interested, I’m happy to offer a free account and even write custom tests for you in exchange for feedback to improve it.

Thanks for the support and this great project!

We converted our app from RN to Expo and now we faced the issue that for iOS we cannot sign the certificate as development certificate. Which means it cannot run on a real device unless via test flight or the store.

So there is currently no option to run Appium tests on real devices (e.g. AWS Device Farm), right?

As described here:

you need an adhoc provisioning profile. This works with AWS Device Farm for us.