|
|
| (7 intermediate revisions by 4 users not shown) |
| Line 1: |
Line 1: |
| = At a glance = | | {{delete|Project is no longer active|date=2020-09-17}} |
| [https://nimbledroid.com/ NimbleDroid] provides functional performance testing for Android and iOS via a service. We currently use NimbleDroid exclusively for testing the performance of our Android applications on real devices.
| |
|
| |
|
| {| class="wikitable"
| | As of around May 2020, we no longer use Nimbledroid: https://github.com/mozilla-mobile/fenix/issues/11050 |
| |-
| |
| ! Application !! Upload !! [[#Cold startup]] !! [[#Page load]]
| |
| |-
| |
| | Fennec || {{todo|}} || ✅ || {{todo|}}
| |
| |-
| |
| | GeckoView Example || {{todo|}} || ✅ || {{todo|}}
| |
| |-
| |
| | Reference Browser || {{todo|}} || ✅ || {{todo|}}
| |
| |-
| |
| | Fenix || {{todo|}} || ✅ || {{todo|}}
| |
| |-
| |
| | Chrome || {{todo|}} || ✅ || {{todo|}}
| |
| |}
| |
|
| |
|
| = Test scenarios =
| | We've switched to an in-house system instead (FNPRMS, to be replaced by mach perftest) because: |
| | | * it's difficult to debug performance when working with a black box |
| == Cold startup ==
| | * Nimbledroid tests run with <pre>debuggable=true</pre> and that provides unreliable performance characteristics |
| The cold startup scenario is profiled for every app. See the [http://docs.nimbledroid.com/#cold-startup NimbleDroid documentation] for details on what and how they measure for cold startup.
| |
| | |
| == Page load ==
| |
| Custom scenarios are used to test page load for the following sites:
| |
| # https://abcnews.go.com
| |
| # https://amazon.com
| |
| # https://amazon.com/s/ref=nb_sb_noss_2/139-6317191-5622045?url=search-alias%3Daps&field-keywords=mobile+phone
| |
| # https://blogger.com/about
| |
| # https://booking.com
| |
| # https://booking.com/searchresults.en-gb.html?city=20088325
| |
| # https://buzzfeed.com/daves4/the-new-york-times-makes-the-nerdiest-correction-e?utm_term=.tlL2KE08L6#.hwrBglaV2A
| |
| # https://cnn.com
| |
| # https://cnn.com/ampstories/us/why-hurricane-michael-is-a-monster-unlike-any-other
| |
| # https://edition.cnn.com/ampstories/us/still-missing-cassini-heres-what-else-is-out-there
| |
| # https://en.m.wikipedia.org/wiki/Main_Page
| |
| # https://en.m.wikipedia.org/wiki/Mozilla_Corporation
| |
| # https://eurosport.de
| |
| # https://eurosport.de/fussball/fifa-wm/2018/wm-2018-spielplan-russland-alle-dfb-spiele-wm-2018-gruppen-wm-2018-termine_sto6691356/story.shtml
| |
| # https://expedia.com/Hotel-Search?destination=New+York%2C+New+York&latLong=40.756680%2C-73.986470®ionId=178293&startDate=&endDate=&rooms=1&_xpid=11905%7C1&adults=2
| |
| # https://fashionbeans.com
| |
| # https://fashionbeans.com/article/coolest-menswear-stores-in-the-world
| |
| # https://glassdoor.com/index.htm
| |
| # https://glassdoor.com/Job/jobs.htm?suggestCount=0&suggestChosen=true&clickSource=searchBtn&typedKeyword=Mozilla&sc.keyword=Mozilla&locT=C&locId=1147431&jobType=
| |
| # https://google.com
| |
| # https://google.com/maps?force=pwa
| |
| # https://google.com/search?q=restaurants+near+me
| |
| # https://guardian.co.uk
| |
| # https://healthyway.com
| |
| # https://instagram.com
| |
| # https://m.360.cn
| |
| # https://m.ebay-kleinanzeigen.de
| |
| # https://m.ebay-kleinanzeigen.de/s-anzeigen/auf-zeit-wg-berlin/zimmer/c199-l3331
| |
| # https://m.facebook.com
| |
| # https://m.facebook.com/Cristiano
| |
| # https://m.ranker.com
| |
| # https://m.ranker.com/list/hunger-games-book-vs-movie-comparisons/lisa-waugh?ref=browse_list_5&l=2&pos=1
| |
| # https://m.vk.com
| |
| # https://m.weibo.cn
| |
| # https://m.yelp.com
| |
| # https://m.yelp.de
| |
| # https://m.yelp.de/biz/burgeramt-berlin-2
| |
| # https://marvel.wikia.com/wiki/Black_Panther
| |
| # https://medium.com
| |
| # https://medium.com/s/coincidences-are-a-lie/could-america-have-also-been-the-birthplace-of-impressionism-cb3d31a2e22d
| |
| # https://people.com/amp-stories/royal-a-to-z
| |
| # https://rumble.com/v3c44t-foxes-jumping-on-my-trampoline.html
| |
| # https://sbnation.com/2018/2/12/16719476/college-football-playoff-rankings-explained
| |
| # https://sima.com.cn
| |
| # https://tripadvisor.com
| |
| # https://tripadvisor.com/Hotels-g187147-Paris_Ile_de_France-Hotels.html
| |
| # https://urbandictionary.com
| |
| # https://urbandictionary.com/define.php?term=awesome%20sauce
| |
| # https://washingtonpost.com
| |
| # https://washingtonpost.com/news/the-switch/wp/2017/06/06/amazon-kickstarter-reddit-and-mozilla-are-staging-a-net-neutrality-online-protest/?noredirect=on&utm_term=.c2cc1fa4862a
| |
| # https://wired.com
| |
| # https://wired.com/story/vehicle-future-bike
| |
| # https://youtube.com
| |
| # https://zalando.de
| |
| # https://zalando.de/herrenschuhe/?exclusive=true
| |
| | |
| = Uploading apps =
| |
| | |
| == Automated ==
| |
| {{todo|}}
| |
| | |
| == Manual ==
| |
| {{todo|}}
| |
| | |
| = Results =
| |
| The test results can be seen in [[../Results#Firefox_Health|Firefox Health]] and [[../Results#Nimbledroid|NimbleDroid]]'s dashboard.
| |
| | |
| = Alerting =
| |
| {{todo|}}
| |