MozillaWiki

Calendar:QA TestDay:2006-10-17: Difference between revisions

Calendar Discussion

Calendar:QA TestDay:2006-10-17 (view source)

Revision as of 16:34, 19 October 2006

2,953 bytes added ,  19 October 2006
→‎RESULTS!
 
(17 intermediate revisions by 2 users not shown)
Line 2: Line 2:


<span style="color:red"> Welcome to the first Sunbird/Lightning Inter-Operability testday! </span> See below for more information.
<span style="color:red"> Welcome to the first Sunbird/Lightning Inter-Operability testday! </span> See below for more information.
<font color=red>
= RESULTS! =
* 9 Bugs Filed!
* 9 New People Attended!
* 8 Third Party Products Tested!
</font>
Thanks to everyone who worked on this test day. To Sebo, especially, who put together this page, and the known issues page. Thanks to ssitter, xFallenAngel, celina63, Damian, jminta, lilmatt, dmose, zach, and bbbrowning for continuing to step up and help out every time we have one of these days. Thanks.
This was a very different test day. We asked people to test interoperability which, by its very nature, requires some amount of technical skill to understand and complete. I think there could have been more that we could have done to help people with this testing. I think that the day would have also been more successful if we had been more proactive about engaging people as they logged int the Channel and we walked through interoperability tests with them. Also, and I admit I thought of this too late, but if we do another of these "interop" test days, then we should advertise it on the CalConnect.org's listserv. That would get us a ton of visibility.
These caveats aside, there were some very good things from this test day.
We had 9 bugs filed! Of those, two were duplicates, and one was not an import-export bug. Of the import export bugs, two are worth mentioning: {{bug|356967}} which I don't believe we have seen previously. And {{bug|356799}} which may have been previously reported, or may be a new manifestation of an existing issue.
We tested nine products:
* Apple iCal
* Evolution
* KOrganizer
* Google Calendar
* Trumba (www.trumba.com - software that integrates calendars into your website)
* the new Google Calendar Provider by our own xFallenAngel (not really a third party product)
* Groupwise 7
* Microsoft Outlook 2003
We continue to have several new people show up to our test days. This is really good, but I think that we are not engaging the new people well enough and when the test day is over, we lose them. We need to improve on this. I've denoted the new folks with a "*" in our list.
Thanks to everyone involved!
* sandos*, tering tuby, sopues*, dmose,
* j9*, xFallenAngel, julie*, Lisa,
* worzel, Mc, mat75*, Damian,
* sebo, woodturner*, hamm*, unlogic
* celina63, thebofh*, fopper*, Andreas,
* ssitter, zach, jminta, lilmatt, bbbrowning
And Congratulations to our two winners! Our prizes are once again provided by the folks at [http://linpro.no | Linpro AS]
* Sidney takes first place for best bug found {{bug|356967}}
* Worzel takes second for his testing of Outlook 2003.
== When is the testday? ==
The test day will run from [http://www.timeanddate.com/worldclock/fixedtime.html?month=10&day=17&year=2006&hour=0&min=0&sec=0&p1=0| 00:00 UTC on October 17 (click to get that in your timezone)] to [http://www.timeanddate.com/worldclock/fixedtime.html?month=10&day=18&year=2006&hour=1&min=0&sec=0&p1=0| 01:00 UTC on October 18 (click to get that in your timezone)]
Alternatively, you can import this iCalendar to see the start and end times in your timezone: [[Media:Inter-operability_Testday.ics]] (right click and save as..., then import into your calendar)


== Where do I start? ==
== Where do I start? ==
Line 23: Line 65:
* dmose
* dmose
* xFallenAngel
* xFallenAngel
 
* jminta
'''^^^to be updated'''
* Sebo


== What build should I test? ==
== What build should I test? ==
Line 53: Line 95:
We need to test against as many other calendar applications and calendar servers as possible. So, any test you can think up that measures the interoperability of your specific calendar servers and clients with Sunbird/Lightning will be appreciated.  
We need to test against as many other calendar applications and calendar servers as possible. So, any test you can think up that measures the interoperability of your specific calendar servers and clients with Sunbird/Lightning will be appreciated.  


=====Bugzilla Testing=====
See [[Calendar:Known Interop Issues]] for known issues on interoperability and feel free to add bugs that you find during your search in bugzilla.
Verify that the [https://bugzilla.mozilla.org/buglist.cgi?query_format=advanced&short_desc_type=allwordssubstr&short_desc=&product=Calendar&component=Alarms&component=Calendar+Views&component=General&component=Import+and+Export&component=Internal+Components&component=Lightning+Only&component=Printing&component=Provider%3A+CalDav&component=Provider%3A+ICS%2FWebdav&component=Provider%3A+Local+Storage&component=Provider%3A+WCAP&component=Security&component=Sunbird+Only&component=Tasks&component=Website&long_desc_type=allwordssubstr&long_desc=&bug_file_loc_type=allwordssubstr&bug_file_loc=&status_whiteboard_type=allwordssubstr&status_whiteboard=&keywords_type=allwords&keywords=&bug_status=RESOLVED&resolution=FIXED&emailtype1=substring&email1=&emailtype2=substring&email2=&bugidtype=include&bug_id=&votes=&chfieldfrom=-1m&chfieldto=Now&chfield=resolution&chfieldvalue=&cmdtype=doit&order=Reuse+same+sort+as+last+time&field0-0-0=noop&type0-0-0=noop&value0-0-0= fixed bugs] are addressed by these nightly builds.
 
=====Litmus Testing=====
There are four test suites in [http://litmus.mozilla.org Litmus] for Calendar. We want to run as many of these Litmus tests as possible for each product.
* Sunbird Basic Functional Tests - basic tests for Sunbird
* Sunbird Full Functional Tests - in depth tests for Sunbird
* Lightning Basic Functional Tests - basic tests for Lightning
* Lightning Full Functional Tests - in depth tests for Lightning


Also see [[Calendar:QA_TestDay:2006-10-17:Matrix|Testing Ideas]] for specific test cases to guide you and to enable us to track your progress.


=====Bug Verification Testing (optional)=====
Verify that the [https://bugzilla.mozilla.org/buglist.cgi?query_format=advanced&short_desc_type=allwordssubstr&short_desc=&product=Calendar&component=Alarms&component=Calendar+Views&component=General&component=Import+and+Export&component=Internal+Components&component=Lightning+Only&component=Printing&component=Provider%3A+CalDav&component=Provider%3A+ICS%2FWebdav&component=Provider%3A+Local+Storage&component=Provider%3A+WCAP&component=Security&component=Sunbird+Only&component=Tasks&component=Website&long_desc_type=allwordssubstr&long_desc=&bug_file_loc_type=allwordssubstr&bug_file_loc=&status_whiteboard_type=allwordssubstr&status_whiteboard=&keywords_type=allwords&keywords=&bug_status=RESOLVED&resolution=FIXED&emailtype1=substring&email1=&emailtype2=substring&email2=&bugidtype=include&bug_id=&votes=&chfieldfrom=-1m&chfieldto=Now&chfield=resolution&chfieldvalue=&cmdtype=doit&order=Reuse+same+sort+as+last+time&field0-0-0=noop&type0-0-0=noop&value0-0-0= fixed bugs] are addressed by these nightly builds.


<font color=red>
<font color=red>
Line 97: Line 134:
== Testing Instructions ==
== Testing Instructions ==


---''needs some refacturing, not final''---
Unfortunately, due to the nature of interoperability testing, these instructions will be very vague. The primary goal is to:
 
'''Move calendar data from application x into Sunbird/Lightning and ensure that Sunbird/Lightning interprets that data correctly'''.
 
There are many ways to do this.
1. Email calendar invitations to a Lightning enabled Thunderbird.
2. Import ICS calendar files into Lightning and Sunbird
3. Use the web interface to Sun's WCAP server and ensure that the data comes into a WCAP-enabled Lightning.
4. Attempt to do Copy/Paste between other applications and Sunbird
5. Use another application to publish calendar data to a WebDav or CalDav server, and then use Sunbird/Lightning to receive that data.
6. Use Sunbird/Lightning to change data that has been published to a WebDav or CalDav server and ensure that the other application can still interpret the event correctly.
 
Just some of the clients we would like to test interoperability with:
* KOrganizer
* Apple iCAL
* Microsoft Outlook 2000, 2002, 2003
* Microsoft Vista Calendar
* Lotus Notes
* Evolution
* Chandler
* Google Calendar
 
And some of the servers we would like to test with:
* Oracle CalDav
* Cosmo CalDav (latest build if possible, as well as the public demo build)
* Apple CalDav
* FTP (Simply plug an FTP server in as a "webdav" server)
* WebDav
* Sun WCAP
 
This is not a complete list, if you have a client that speaks ICS calendar format, we'd like to test with it in this test day.
 
===Testing Ideas===
This page lists some ideas and testcases to help get you started. Please update the page with your results:[[Calendar:QA_TestDay:2006-10-17:Matrix | Testing Ideas and Known Issues]]


# Download Sunbird or Lightning from the above URLs
'''Still Confused?'''
# Create your testing profile (see above)
'''THEN - Verify Bugzilla Fixed Bugs'''
# Click on the above Bugzilla bug list link and verify Fixed bugs. Read how the bug was produced and try to recreate it to verify the error doesn't happen anymore. 
# If the error is no longer occurring, select the "Mark bug as Verified" radio button and click the Commit button.  If the  error still occurs, select Reopen Bug and enter comments as to why the bug is being reopened (i.e. what is still failing), then click the Commit button.
'''OR - Run Litmus Test Cases'''


Build ID for October 17 Test Day
If you're still confused, log into #calendar-qa and ask us for help. We'll help you as best we can.
--------------------------------
OS      | Sunbird    | Lightning
Linux  | 20061016 | 20061016
Mac    | 20061016 | 20061016
Windows | 20061016 | 20061016


'''^^^to be updated'''


# Go to [http://litmus.mozilla.org Litmus].
##Select Run Tests, the Product is Calendar. Your buildId should match the one in the table above. Verify that it does. Sunbird's buildId is in the "About Sunbird" window. Lightning's buildId is in its description in the Add-ons Manager.
##Choose a Test group and subgroup to test on (choose one that has a low percentage of Community Coverage).
##Follow the test steps and verify the function under test works as described in Expected Results.
## If the test case performs as expected, mark it as a "Pass".  If not, mark it as a "Fail" and then enter a bug (see below).
## When you've finished one test, go to the next test by clicking the test case name.
## Once you've completed all the tests in a group, click the "Submit All Results" button at the bottom.
'''OR - Perform Usability Testing'''
# Use Sunbird and Lightning like you normally would. Try to do everything an end-user might do, ranging from simple calendar operations to power user features. Try to think of actions that are not typical since you will most likely find a bug there.
'''BUG FOUND!'''
'''BUG FOUND!'''
# Be sure to check to see if it's already on our [[Calendar:Known_Interop_Issues|list]] (we're updating it as we go)
# When you think you've found a bug, try to reproduce it. Restart Sunbird/Lightning and do the same steps again. The most valuable bugs are those that can be reproduced.
# When you think you've found a bug, try to reproduce it. Restart Sunbird/Lightning and do the same steps again. The most valuable bugs are those that can be reproduced.
# Once the bug has been reproduced, search for it or something similar on [https://bugzilla.mozilla.org Bugzilla]. This can easily be done by searching for the error message, window name, or some other identifying feature of the bug. If you found an existing bug that is just like the bug you found, read the comments. Maybe your situation is slightly different and your input could help developers find the origin of the bug.  
# Once the bug has been reproduced, search for it or something similar on [https://bugzilla.mozilla.org Bugzilla]. This can easily be done by searching for the error message, window name, or some other identifying feature of the bug. If you found an existing bug that is just like the bug you found, read the comments. Maybe your situation is slightly different and your input could help developers find the origin of the bug.  
Ctalbert
Confirmed users
3,816

edits

Retrieved from "https://wiki.allizom.org/Special:MobileDiff/36165...38436"