Calendar:QA TestDay:2006-10-17

<< Back to Calendar QA TestDays

Welcome to the first Sunbird/Lightning Inter-Operability testday! See below for more information.

RESULTS!

• 9 Bugs Filed!
• 9 New People Attended!
• 8 Third Party Products Tested!

Thanks to everyone who worked on this test day. To Sebo, especially, who put together this page, and the known issues page. Thanks to ssitter, xFallenAngel, celina63, Damian, jminta, lilmatt, dmose, zach, and bbbrowning for continuing to step up and help out every time we have one of these days. Thanks.

This was a very different test day. We asked people to test interoperability which, by its very nature, requires some amount of technical skill to understand and complete. I think there could have been more that we could have done to help people with this testing. I think that the day would have also been more successful if we had been more proactive about engaging people as they logged int the Channel and we walked through interoperability tests with them. Also, and I admit I thought of this too late, but if we do another of these "interop" test days, then we should advertise it on the CalConnect.org's listserv. That would get us a ton of visibility.

These caveats aside, there were some very good things from this test day. We had 9 bugs filed! Of those, two were duplicates, and one was not an import-export bug. Of the import export bugs, two are worth mentioning: bug 356967 which I don't believe we have seen previously. And bug 356799 which may have been previously reported, or may be a new manifestation of an existing issue.

We tested nine products:

• Apple iCal
• Evolution
• KOrganizer
• Google Calendar
• Trumba (www.trumba.com - software that integrates calendars into your website)
• the new Google Calendar Provider by our own xFallenAngel (not really a third party product)
• Groupwise 7
• Microsoft Outlook 2003

We continue to have several new people show up to our test days. This is really good, but I think that we are not engaging the new people well enough and when the test day is over, we lose them. We need to improve on this. I've denoted the new folks with a "*" in our list.

Thanks to everyone involved!

• sandos*, tering tuby, sopues*, dmose,
• j9*, xFallenAngel, julie*, Lisa,
• worzel, Mc, mat75*, Damian,
• sebo, woodturner*, hamm*, unlogic
• celina63, thebofh*, fopper*, Andreas,
• ssitter, zach, jminta, lilmatt, bbbrowning

And Congratulations to our two winners! Our prizes are once again provided by the folks at | Linpro AS

• Sidney takes first place for best bug found bug 356967
• Worzel takes second for his testing of Outlook 2003.

When is the testday?

The test day will run from 00:00 UTC on October 17 (click to get that in your timezone) to 01:00 UTC on October 18 (click to get that in your timezone)

Alternatively, you can import this iCalendar to see the start and end times in your timezone: Media:Inter-operability_Testday.ics (right click and save as..., then import into your calendar)

Where do I start?

• Open up your favorite IRC client:
  • Windows clients
  • MacOS clients
  • Linux clients
  • ChatZilla (Firefox extension)
• Connect to irc.mozilla.org
• Join the #calendar-qa channel

OK, I am in the channel, now what?

Let the moderators know if you have any questions or encounter any problems. If you are not familiar with our bugzilla bug tracking system, feel free to report your problems to a moderator and they can log any bugs for you.

Who will be moderating the #calendar-qa channel on the Test Days?

The moderator's nicks for this test day will be:

• ctalbert
• ssitter
• lilmatt
• dmose
• xFallenAngel
• jminta
• Sebo

What build should I test?

We will be testing the Sunbird 0.4a1 and Lightning 0.4a1 nightly builds:

Sunbird:

http://ftp.mozilla.org/pub/mozilla.org/calendar/sunbird/nightly/latest-trunk/

Lightning (for Thunderbird 1.5.x and 2.0 candidates):

http://ftp.mozilla.org/pub/mozilla.org/calendar/lightning/nightly/latest-mozilla1.8/

What specifically will we be testing?

In light of the recent 0.3 release, we've seen quite a few bugs coming through as people try to load data into Lightning and Sunbird from their favorite clients. We want this test day focused on interoperability testing. Among other things, this means:

Testing For Lightning:

• Test receiving email invitations from other clients and adding them to a Lightning calendar
• Test importing and exporting Lightning calendars to/from other calendar applications
• Test how Sunbird works with published calendars that were generated from other applications
• Test how Lightning works with WCAP data from Sun WCAP servers (requires Lightning WCAP extension)
• Are there other interoperability tests you can dream up for Lightning?

Testing For Sunbird:

• Test importing and exporting Sunbird calendars to/from other calendar applications
• Test how Sunbird works with published calendars that were generated from other applications
• Are there other interoperability tests you can dream up for Sunbird?

We need to test against as many other calendar applications and calendar servers as possible. So, any test you can think up that measures the interoperability of your specific calendar servers and clients with Sunbird/Lightning will be appreciated.

See Calendar:Known Interop Issues for known issues on interoperability and feel free to add bugs that you find during your search in bugzilla.

Also see Testing Ideas for specific test cases to guide you and to enable us to track your progress.

Bug Verification Testing (optional)

Verify that the fixed bugs are addressed by these nightly builds.

BEFORE YOU START--BACKUP YOUR PROFILE

Once you have downloaded the version of the application you want to test, but before you start it, you MUST back up your profile and create a testing profile, otherwise, you run a very good chance of losing your data.

Backing up a profile is easy:
1. Find your profile directory:

OS	Thunderbird Profile	Sunbird Profile
Windows	%AppData%\Thunderbird	%AppData%\Mozilla\Sunbird
Linux	~/.thunderbird	~/.mozilla/sunbird
MacOSX	~/Library/Thunderbird	~/Library/Application Support/Sunbird

2. Copy the root of it somewhere else, for example copy the Thunderbird or Sunbird directory and everything below it.

Creating a testing profile is also easy:

1. Launch your application with a -P option
2. Create a New Profile by clicking the Create a New Profile button
3. Click Next, Give it a name like "Testing Profile", click Finish
4. Select the testing profile and click the "Start Calendar" (Or Start Mail in Thunderbird) button.

Remember that you will have to start the application with the -P option during testing so you can pick your Testing profile

Once you've finished testing, you can delete the extra profiles and you'll be back to normal.

If something goes terribly wrong (although we don't expect it to), close the application and just replace the profile with the backed up copy. That will reset the application to the way it was before you started testing.

Testing Instructions

Unfortunately, due to the nature of interoperability testing, these instructions will be very vague. The primary goal is to:

Move calendar data from application x into Sunbird/Lightning and ensure that Sunbird/Lightning interprets that data correctly.

There are many ways to do this. 1. Email calendar invitations to a Lightning enabled Thunderbird. 2. Import ICS calendar files into Lightning and Sunbird 3. Use the web interface to Sun's WCAP server and ensure that the data comes into a WCAP-enabled Lightning. 4. Attempt to do Copy/Paste between other applications and Sunbird 5. Use another application to publish calendar data to a WebDav or CalDav server, and then use Sunbird/Lightning to receive that data. 6. Use Sunbird/Lightning to change data that has been published to a WebDav or CalDav server and ensure that the other application can still interpret the event correctly.

Just some of the clients we would like to test interoperability with:

• KOrganizer
• Apple iCAL
• Microsoft Outlook 2000, 2002, 2003
• Microsoft Vista Calendar
• Lotus Notes
• Evolution
• Chandler
• Google Calendar

And some of the servers we would like to test with:

• Oracle CalDav
• Cosmo CalDav (latest build if possible, as well as the public demo build)
• Apple CalDav
• FTP (Simply plug an FTP server in as a "webdav" server)
• WebDav
• Sun WCAP

This is not a complete list, if you have a client that speaks ICS calendar format, we'd like to test with it in this test day.

Testing Ideas

This page lists some ideas and testcases to help get you started. Please update the page with your results: Testing Ideas and Known Issues

Still Confused?

If you're still confused, log into #calendar-qa and ask us for help. We'll help you as best we can.

BUG FOUND!

1. Be sure to check to see if it's already on our list (we're updating it as we go)
2. When you think you've found a bug, try to reproduce it. Restart Sunbird/Lightning and do the same steps again. The most valuable bugs are those that can be reproduced.
3. Once the bug has been reproduced, search for it or something similar on Bugzilla. This can easily be done by searching for the error message, window name, or some other identifying feature of the bug. If you found an existing bug that is just like the bug you found, read the comments. Maybe your situation is slightly different and your input could help developers find the origin of the bug.
4. If you can't find the bug in Bugzilla, talk about it in the channel. Maybe others can confirm your bug or find a similar bug in bugzilla. This also gives moderators a chance to guide you to either provide more details, update an existing bug or file a new bug.
5. If you are certain you found a unique bug and have read the Bug Writing Guidelines and Bugzilla Etiquette, go ahead and file the bug in Bugzilla. Remember to be as specific as possible.

Some other ways you can help:

• Think of some test cases that aren't already included in Litmus (we love suggestions!).
• Ask your friends to join you. The more people help the better!

Where do I find Litmus and Bugzilla?

• Litmus can be found at http://litmus.mozilla.org/
• A list of Calendar specific test cases can be found via the search function

• Bugzilla can be found at http://bugzilla.mozilla.org/
• A list of Calendar specific bugs (without enhancements) can be found here
• A list of already filed enhancement requests can be found here

Tips:

• Make sure you enter the correct information for your platform, version of Thunderbird, Lightning etc. Please include the full build identifier (i.e Thunderbird 1.5.0.7 (20060909) with Lightning 0.3 (2006100206)).
• Be sure to read the Testing Instructions above before filing a bug.
• Feel free to ask on IRC if you need help or have any more questions.

How do I send feedback?

Please email us at dev-apps-calendar at lists.mozilla.org to send feedback about how the testday went, any overall thoughts about Sunbird and Lightning, and to follow up on issues found and bugs filed. We really appreciate your help today and your feedback is very valuable.

Rewards!

There will be two prizes available for the two people that make the largest contribution to the test day. For this test day, that will mean the people that find the best bugs.

Rewards are kindly provided by Linpro AS

I only know most of you by your IRC nickname, so to ensure your bugzilla activity gets counted, be certain to email cmtalbert at gmail dot com with your bugzilla user ID. Please use "TEST DAY" in the subject line of the mail.

• As always, know that we appreciate everyone's contribution and you all have our heart felt gratitude. Thank you.