TestEngineering/Performance/NimbleDroid: Difference between revisions
Jump to navigation
Jump to search
(→At a glance: updated table) |
No edit summary |
||
Line 6: | Line 6: | ||
! Application !! Upload !! [[#Cold startup]] !! [[#Page load]] | ! Application !! Upload !! [[#Cold startup]] !! [[#Page load]] | ||
|- | |- | ||
| Fennec || | | Fennec || None || ? || ? | ||
|- | |- | ||
| GeckoView Example || Manual || ✅ || ✅ | | GeckoView Example || [[#Manual]] || ✅ || ✅ | ||
|- | |- | ||
| Reference Browser || | | Reference Browser || [[#Automated]] (push to master) || ✅ || ? | ||
|- | |- | ||
| Fenix || | | Fenix || [[#Automated]] (push to master) || ✅ || ✅ | ||
|- | |- | ||
| Chrome || Manual || ✅ || ✅ | | Chrome || [[#Manual]] || ✅ || ✅ | ||
|} | |} | ||
Revision as of 12:39, 18 September 2019
At a glance
NimbleDroid provides functional performance testing for Android and iOS via a service. We currently use NimbleDroid exclusively for testing the performance of our Android applications on real devices.
Application | Upload | #Cold startup | #Page load |
---|---|---|---|
Fennec | None | ? | ? |
GeckoView Example | #Manual | ✅ | ✅ |
Reference Browser | #Automated (push to master) | ✅ | ? |
Fenix | #Automated (push to master) | ✅ | ✅ |
Chrome | #Manual | ✅ | ✅ |
Test scenarios
Cold startup
The cold startup scenario is profiled for every app. See the NimbleDroid documentation for details on what and how they measure for cold startup.
Page load
Custom scenarios are used to test page load for the following sites:
- https://abcnews.go.com
- https://amazon.com
- https://amazon.com/s/ref=nb_sb_noss_2/139-6317191-5622045?url=search-alias%3Daps&field-keywords=mobile+phone
- https://blogger.com/about
- https://booking.com
- https://booking.com/searchresults.en-gb.html?city=20088325
- https://buzzfeed.com/daves4/the-new-york-times-makes-the-nerdiest-correction-e?utm_term=.tlL2KE08L6#.hwrBglaV2A
- https://cnn.com
- https://cnn.com/ampstories/us/why-hurricane-michael-is-a-monster-unlike-any-other
- https://edition.cnn.com/ampstories/us/still-missing-cassini-heres-what-else-is-out-there
- https://en.m.wikipedia.org/wiki/Main_Page
- https://en.m.wikipedia.org/wiki/Mozilla_Corporation
- https://eurosport.de
- https://eurosport.de/fussball/fifa-wm/2018/wm-2018-spielplan-russland-alle-dfb-spiele-wm-2018-gruppen-wm-2018-termine_sto6691356/story.shtml
- https://expedia.com/Hotel-Search?destination=New+York%2C+New+York&latLong=40.756680%2C-73.986470®ionId=178293&startDate=&endDate=&rooms=1&_xpid=11905%7C1&adults=2
- https://fashionbeans.com
- https://fashionbeans.com/article/coolest-menswear-stores-in-the-world
- https://glassdoor.com/index.htm
- https://glassdoor.com/Job/jobs.htm?suggestCount=0&suggestChosen=true&clickSource=searchBtn&typedKeyword=Mozilla&sc.keyword=Mozilla&locT=C&locId=1147431&jobType=
- https://google.com
- https://google.com/maps?force=pwa
- https://google.com/search?q=restaurants+near+me
- https://guardian.co.uk
- https://healthyway.com
- https://instagram.com
- https://m.360.cn
- https://m.ebay-kleinanzeigen.de
- https://m.ebay-kleinanzeigen.de/s-anzeigen/auf-zeit-wg-berlin/zimmer/c199-l3331
- https://m.facebook.com
- https://m.facebook.com/Cristiano
- https://m.ranker.com
- https://m.ranker.com/list/hunger-games-book-vs-movie-comparisons/lisa-waugh?ref=browse_list_5&l=2&pos=1
- https://m.vk.com
- https://m.weibo.cn
- https://m.yelp.com
- https://m.yelp.de
- https://m.yelp.de/biz/burgeramt-berlin-2
- https://marvel.wikia.com/wiki/Black_Panther
- https://medium.com
- https://medium.com/s/coincidences-are-a-lie/could-america-have-also-been-the-birthplace-of-impressionism-cb3d31a2e22d
- https://people.com/amp-stories/royal-a-to-z
- https://rumble.com/v3c44t-foxes-jumping-on-my-trampoline.html
- https://sbnation.com/2018/2/12/16719476/college-football-playoff-rankings-explained
- https://sima.com.cn
- https://tripadvisor.com
- https://tripadvisor.com/Hotels-g187147-Paris_Ile_de_France-Hotels.html
- https://urbandictionary.com
- https://urbandictionary.com/define.php?term=awesome%20sauce
- https://washingtonpost.com
- https://washingtonpost.com/news/the-switch/wp/2017/06/06/amazon-kickstarter-reddit-and-mozilla-are-staging-a-net-neutrality-online-protest/?noredirect=on&utm_term=.c2cc1fa4862a
- https://wired.com
- https://wired.com/story/vehicle-future-bike
- https://youtube.com
- https://zalando.de
- https://zalando.de/herrenschuhe/?exclusive=true
Uploading apps
Automated
[TODO]
Manual
[TODO]
Results
The test results can be seen in Firefox Health and NimbleDroid's dashboard.
Alerting
[TODO]