1,295
edits
Build:TryServer:Suggestions: Difference between revisions
Jump to navigation
Jump to search
→Try Server Suggestions and Feature Requests
| Line 15: | Line 15: | ||
* Ability to just build installer/zip/dmg/tar without uploading symbols, running tests, etc. | * Ability to just build installer/zip/dmg/tar without uploading symbols, running tests, etc. | ||
* Compare talos numbers relative to mozilla-central numbers in proximity hg DAG-wise, see http://blog.mozilla.com/axel/2010/06/07/trying-talos-on-node-js/ for a proof of concept. | * Compare talos numbers relative to mozilla-central numbers in proximity hg DAG-wise, see http://blog.mozilla.com/axel/2010/06/07/trying-talos-on-node-js/ for a proof of concept. | ||
* (roc) We have a problem that try server doesn't degrade gracefully. When it's loaded, you lose results, or the results lose relevance when they're really old and far away from where trunk is now. So we get into situations where people are pushing but all the work done for the push is basically wasted. In these kinds of situations we need some kind of admission control so that fewer people can push but those who can actually get useful results. | |||
= Preventing you from using tryserver effectively for your needs = | = Preventing you from using tryserver effectively for your needs = | ||
* (bz) Being able to actually get results in sane amounts of time. It's now been over 24 hours since I pushed a patch, and I have yet to see a single mac test result, Linux debug test result, 32-bit Linux optimized test result, anything at all on Win7. The Win2k3 optimized build died with an infrastructure error... In other words, a bunch of machine time was wasted, and I still can't get the data I wanted. | * (bz) Being able to actually get results in sane amounts of time. It's now been over 24 hours since I pushed a patch, and I have yet to see a single mac test result, Linux debug test result, 32-bit Linux optimized test result, anything at all on Win7. The Win2k3 optimized build died with an infrastructure error... In other words, a bunch of machine time was wasted, and I still can't get the data I wanted. | ||
* (roc) Understanding tryserver performance results is hard. The tbpl comparator is somewhat helpful, but only if you can identify a changeset on the same page of results that would be a good baseline for comparison, and that is error-prone and often impossible. You also don't get a fine-grained breakdown of test results. I generally end up screen-scraping the old-style Tinderbox with a script I wrote. The process is even slower and harder when your test results appear over several days, if at all (see above). | * (roc) Understanding tryserver performance results is hard. The tbpl comparator is somewhat helpful, but only if you can identify a changeset on the same page of results that would be a good baseline for comparison, and that is error-prone and often impossible. You also don't get a fine-grained breakdown of test results. I generally end up screen-scraping the old-style Tinderbox with a script I wrote. The process is even slower and harder when your test results appear over several days, if at all (see above). | ||