QA/TCM/Releases/.1/Feedback

1. Do you like the direction of the initial .1 release?

• Yes - 2
• No - 2

2. What did you like about the features in the .1 release?

• good error dialogs for sign in page. I love the red/green highlights
• the workflow is nice and simple. its clean looking
• Pretty clear and intuitive.
• Wasn't able to sign up, so don't know :-(
  • this is fixed, see below. sorry for the hassle!
• nothing at all
  • sorry to hear that - this was a very early preview with a great deal missing, but suggestions for changes in direction are most welcome.

3. Is there anything you did not like about the features in the .1 release? If so, what?

• why cant i see the existing tests without having to sign in? I think it would be good if we just made it read only view for non-logged in
  • filed as https://bugzilla.mozilla.org/show_bug.cgi?id=635025
• after registering a new account, i needed to login again. it'd be better if it logs me in after i just created an account.
  • in future releases, email address confirmation will be required after registration before logging in, so if we did immediate-login now it would soon become irrelevant. see https://bugzilla.mozilla.org/show_bug.cgi?id=630592
• allow username as a login, and not just the whole email address
  • agreed! filed as https://bugzilla.mozilla.org/show_bug.cgi?id=635028
• it's very... WHITE. can we get a little more 2011 css webdesign around the site?
  • as Aakash's email mentioned, .1 release is just wireframes. Branding, naming, and actual design are all still in progress and not reflected here at all.
• can we get a folder heirarchy view in a left side column? show/hide structure
  • good idea! filed as https://bugzilla.mozilla.org/show_bug.cgi?id=635029 (if I understood the suggestion correctly)
• dont understand why there's a fail button per testcase instruction. i would make sense it was one button, sitting on the same line as the "invalid" button?
  • because the backend API wants to know which step failed when a testcase fails. the text field you fill in for a fail is supposed to be the "actual results" for that particular step, as opposed to the "expected results" listed for that instruction step. If we had just one fail button, we'd have to make the tester select the failed step via a dropdown or some such, which seems less humane. Suggestions for improvement here welcome.
• once i "pass" the testcase, there's no way to go back and change my response to "Fail?"
  • yes, I'd assumed this would be added eventually, it just wasn't in the .1 spec. filed as https://bugzilla.mozilla.org/show_bug.cgi?id=635032
• what is the difference between "Pass" vs "Passed!"?
  • the former is a button you can press to declare the test case passed, the latter is a status marker indicating that it has already been marked passed. These are distinguished visually and by tense, but we'll work on better visual distinctions.
• Add test Case: by default, there's a red highlight and a green highlight. i recommend getting rid of the colors and the "X"s on first launch, cause it signals to me that i may have did something wrong. When in fact i havent done anything yet. only after someone submits a testcase with errors, should we flag red boxes.
  • On initial load the red highlight just indicates a required field. we'll consider if there would be better ways to represent this. the red Xes don't represent an error at all, they are a control for removing an environment constraint or a test-case instruction step. we'll consider if there'd be an alternative delete icon that wouldn't imply an error condition.
• The UI seems to be taking almost too much real estate. Maybe better as more features come online and crowd the page somewhat, but now it feels to spread out.
  • yes, there are a number of features still planned for addition to the test-running UI, so it's intentionally generous with space at the moment, so as to avoid becoming overcrowded later.
• [16:47:09.904] POST https://tcm.oddsites.net/account/register/ [HTTP/1.1

500 INTERNAL SERVER ERROR 552ms]. The username I tried to register was "stephendonner"

• cannot comment as i could not log in
  • this was an issue with the backend API not accepting usernames longer than 12 characters. It's now fixed, the API will accept up to 20 and the UI gracefully validates the limit rather than puking. thanks for catching the bug!
• Unclear that a user needs to set their environment preferences
  • It defaults to Linux.