TestEngineering/Performance/NimbleDroid
At a glance
NimbleDroid provides functional performance testing for Android and iOS via a service. We currently use NimbleDroid exclusively for testing the performance of our Android applications on real devices.
| Application | Upload | #Cold startup | #Page load |
|---|---|---|---|
| Fennec | None | ? | ? |
| GeckoView Example | #Manual | ✅ | ✅ |
| Reference Browser | #Automated (push to master) | ✅ | ? |
| Fenix | #Automated (push to master) | ✅ | ✅ |
| Chrome | #Manual | ✅ | ✅ |
Test scenarios
Cold startup
The cold startup scenario is profiled for every app. See the NimbleDroid documentation for details on what and how they measure for cold startup.
Page load
Custom scenarios are used to test page load for the following sites:
- https://abcnews.go.com
- https://amazon.com
- https://amazon.com/s/ref=nb_sb_noss_2/139-6317191-5622045?url=search-alias%3Daps&field-keywords=mobile+phone
- https://blogger.com/about
- https://booking.com
- https://booking.com/searchresults.en-gb.html?city=20088325
- https://buzzfeed.com/daves4/the-new-york-times-makes-the-nerdiest-correction-e?utm_term=.tlL2KE08L6#.hwrBglaV2A
- https://cnn.com
- https://cnn.com/ampstories/us/why-hurricane-michael-is-a-monster-unlike-any-other
- https://edition.cnn.com/ampstories/us/still-missing-cassini-heres-what-else-is-out-there
- https://en.m.wikipedia.org/wiki/Main_Page
- https://en.m.wikipedia.org/wiki/Mozilla_Corporation
- https://eurosport.de
- https://eurosport.de/fussball/fifa-wm/2018/wm-2018-spielplan-russland-alle-dfb-spiele-wm-2018-gruppen-wm-2018-termine_sto6691356/story.shtml
- https://expedia.com/Hotel-Search?destination=New+York%2C+New+York&latLong=40.756680%2C-73.986470®ionId=178293&startDate=&endDate=&rooms=1&_xpid=11905%7C1&adults=2
- https://fashionbeans.com
- https://fashionbeans.com/article/coolest-menswear-stores-in-the-world
- https://glassdoor.com/index.htm
- https://glassdoor.com/Job/jobs.htm?suggestCount=0&suggestChosen=true&clickSource=searchBtn&typedKeyword=Mozilla&sc.keyword=Mozilla&locT=C&locId=1147431&jobType=
- https://google.com
- https://google.com/maps?force=pwa
- https://google.com/search?q=restaurants+near+me
- https://guardian.co.uk
- https://healthyway.com
- https://instagram.com
- https://m.360.cn
- https://m.ebay-kleinanzeigen.de
- https://m.ebay-kleinanzeigen.de/s-anzeigen/auf-zeit-wg-berlin/zimmer/c199-l3331
- https://m.facebook.com
- https://m.facebook.com/Cristiano
- https://m.ranker.com
- https://m.ranker.com/list/hunger-games-book-vs-movie-comparisons/lisa-waugh?ref=browse_list_5&l=2&pos=1
- https://m.vk.com
- https://m.weibo.cn
- https://m.yelp.com
- https://m.yelp.de
- https://m.yelp.de/biz/burgeramt-berlin-2
- https://marvel.wikia.com/wiki/Black_Panther
- https://medium.com
- https://medium.com/s/coincidences-are-a-lie/could-america-have-also-been-the-birthplace-of-impressionism-cb3d31a2e22d
- https://people.com/amp-stories/royal-a-to-z
- https://rumble.com/v3c44t-foxes-jumping-on-my-trampoline.html
- https://sbnation.com/2018/2/12/16719476/college-football-playoff-rankings-explained
- https://sima.com.cn
- https://tripadvisor.com
- https://tripadvisor.com/Hotels-g187147-Paris_Ile_de_France-Hotels.html
- https://urbandictionary.com
- https://urbandictionary.com/define.php?term=awesome%20sauce
- https://washingtonpost.com
- https://washingtonpost.com/news/the-switch/wp/2017/06/06/amazon-kickstarter-reddit-and-mozilla-are-staging-a-net-neutrality-online-protest/?noredirect=on&utm_term=.c2cc1fa4862a
- https://wired.com
- https://wired.com/story/vehicle-future-bike
- https://youtube.com
- https://zalando.de
- https://zalando.de/herrenschuhe/?exclusive=true
Uploading apps
Automated
[TODO]
Manual
[TODO]
Results
The test results can be seen in Firefox Health and NimbleDroid's dashboard.
Alerting
[TODO]