QA/Firefox3.1/FontFace TestStatus

« QA/Firefox3.1/FontFace TestPlan

Notes

@font-face test status Test Plan: @font-face TestPlan

Executed Tests

(Tests Already Executed)

These tests were executed after applying patchv.0.9h from bug441473

Tests on Mac OS X

Executed Tests

file	description	status	failure cause
downloadable-font-conflictname.html	When the family name is same as a font family name in user's environment	Passes
downloadable-font-diacritic.html	Check if the diacritic marks are displayed correctly	Passes
downloadable-font-fallback.html	Check the fallback functions	Sometimes Fails	Font not loaded (it happens pretty often)
downloadable-font-familywithseveral.html	Check what happens when multiple @font-face rules are used for one font-family	Sometimes fails	Font is not loaded (it happens quite often)
downloadable-font-iframe.html	Check if the downloaded fonts can be used from other documents (Answer should be no)	Passes
downloadable-font-japanese.html	Incorrect rendering of some Japanese fonts	Fails	see Bugzilla for detail [bug451426]
downloadable-font-layouttest.html	Check the layout of downloaded fonts	Fails	Layout of default font used. Mentioned in [bug441473 #33]
downloadable-font-leavepagetest.html	Check if the firefox crashes when leaving the page during the font download	Sometimes crash	Crash only seems to happen when you invoke firefox with -reftest option. And when crash happens, it always happen after you load the next test HTML in reftest, not when you loaded this leavepagetest.html. But I'm not sure exactly when this crash happens, because this only happens about once in two trials. Still investigating the exact cause. [bug451462]
downloadable-font-ligature.html	Check if the ligatures are displayed correctly	Passes
downloadable-font-loadtest.html	Check how much downloadable fonts can be handled in a page without any crash	Sometimes fails	Fonts are not loaded. Even when the fonts are loaded, layout bug can be seen sometimes.
downloadable-font-markupdifference.html	Check what happens when applying HTML markups to downloaded fonts	Fails	For example, when you apply <em> to a text rendered with downloaded fonts, the italic FACE IS CALCULATED. However, when you apply <em> to a text rendered with font in user's environment, the ITALIC FACE of that font-family is used. The former calculated face and latter italic face is different, so they are rendered slightly differently.
downloadable-font-opentype.html	Check if the opentype fonts are rendered correctly	Passes
downloadable-font-printtest.html	Check if the downloaded fonts are used when you print out the document	Passes
downloadable-font-sameorigin.html	Check if the same-site restriction is working properly	Sometimes Fails	Fonts not loaded

LoadTest (stress test)

• even 100 @font-face rules per page seems to be pretty scalable. A web page with 200 @font-face rules will be loaded in around 5 seconds. When 250 or more @font-face rules are used in a document, firefox will crash.

Tests on Windows XP SP3

Executed Tests

file	description	status	failure cause
downloadable-font-conflictname.html	When the family name is same as a font family name in user's environment	Passes
downloadable-font-diacritic.html	Check if the diacritic marks are displayed correctly	Passes
downloadable-font-fallback.html	Check the fallback functions	Fails	The priority of fonts in fallback function seems to be wrong. When a rule was like @font-face { font-family:test; src: ; } .test { font-family:test, fantasy; } it displays the default font, not the fantasy font. Please be careful that fallback function has different problems in Windows and Mac.
downloadable-font-familywithseveral.html	Check what happens when multiple @font-face rules are used for one font-family	Fails	When you specify a non-italic face with a font-style:italic rule, it is rendered in italic. It should not be in italic. It should be rendered using the exact font face you specified.
downloadable-font-iframe.html	Check if the downloaded fonts can be used from other documents (Answer should be no)	Passes
downloadable-font-japanese.html	Incorrect rendering of some Japanese fonts	Passes
downloadable-font-layouttest.html	Check the layout of downloaded fonts	Fails	Layout of default font used. Mentioned in [bug441473 #33]
downloadable-font-leavepagetest.html	Check if the firefox crashes when leaving the page during the font download	Sometimes Crashes	Crash happens occasionally. It is not restricted to when firefox is invoked with -reftest option. When you load leavepagetest.html and loadtest.html back and force, it will crash after several tries. When this page is loaded, the following assertion is displayed: ASSERTION: Null pres shell: 'mShell', file ../../dist/include/layout/nsPresContext.h, line 185
downloadable-font-ligature.html	Check if the ligatures are displayed correctly	Passes
downloadable-font-loadtest.html	Check how much downloadable fonts can be handled in a page without any crash	Sometimes fails	Layout bug [bug441473 #33] It seem to happen when it fails to download the font before the first rendering. But I'm not sure when it happens. The place it happens changes every time.
downloadable-font-markupdifference.html	Check what happens when applying HTML markups to downloaded fonts	Fails	For example, when you apply <em> to a text rendered with downloaded fonts, the italic FACE IS CALCULATED. However, when you apply <em> to a text rendered with font in user's environment, the ITALIC FACE of that font-family is used. The former calculated face and latter italic face is different, so they are rendered slightly differently.
downloadable-font-opentype.html	Check if the OpenType fonts are rendered correctly	Fails	OpenType font is ignored. Default font is used.
downloadable-font-printtest.html	Check if the downloaded fonts are used when you print out the document	Fails	Layout of default font used. Mentioned in [bug441473 #33]
downloadable-font-sameorigin.html	Check if the same-site restriction is working properly	Passes

LoadTest (stress test)

• On Windows, it seems even more scalable. A web page with 100 @font-face rules will be loaded in around 1 second. But when more than 150 @font-face rules are used in a single page, it takes much more time to load. A web page with 200 @font-face rules will be loaded in around 20 seconds, which is a lot more than that on Mac.
• It does not crash even if a page had more than 1000 @font-face rules, but it would take more than 5 minutes until the text is rendered, which is not endurable for general use.

Planned (but not executed) Tests

(Tests Not Executed Yet)

Unicode-Range Tests

• Basic Tests
  • single code point
    • ex. U+A5, U+301C
  • interval value
    • ex. U+3040-30FF, U+999-998 (does this fail?)
• What happens if I put many 0s before the number?
  • ex. U+F, U+0F, U+00F, U+000F, U+00000000F, ... do they mean the same thing?
• What happens if I use ? in the front/middle?
  • ex. U+?4, U+3?5, ...
• What happens if I use multiple unicode-range rules to consist a single font-family?
  • ex. @font-face{ font-family:testA; src:url(fontA.ttf); unicode-range:U+0-U+2FF; } @font-face{ font-family:testA; src:url(fontB.otf); unicode-range:U+300-U+FFF }
• What happens if I specified only a single character in the rule and tried to display other character? Would there be a fallback function?
  • ex. @font-face { font-family:smallA; src:url(smallA.ttf); unicode-range:U+1; }
  • <span class="font-family:smallA">ABC</span>?
• What happens if the specified font did not have a character of the specified unicode range? Does it display nothing? Would there be a fallback function?

CSS Mutation from JavaScript tests

• What happens if you download the same font again? Would it use cache?
• Is it possible to change the DOM tree of other documents?
  • Is downloading another font for other documents prohibited? or is it allowed?
• Consider what will happen when 2 pages try to change the DOM of each other
  • stateful

Possible Risks

(Known bugs, not-yet-implemented features, questions)

Bugs already filed in Bugzilla

• Firefox crashes when you leave the page before the font download finishes.
  • For example, the following test page should crash your firefox (only if have applied the patch for bug 441473).
  • http://people.mozilla.org/~ctalbert/fontface/downloadable-font-crashtest.html
  • Now filed as bug 451462
  • test case: downloadable-font-leavepagecrash.html
• Unknown gryph change happens to some Japanese characters when font (which does not contain Japanese characters) is downloaded and rendered.
  • Now filed as bug 451426
  • It happens when the file is encoded in Japanese encoding (Shift_JIS, EUC-jp). But it also happened on files encoded in UTF-8.
  • As written in bug 451426, this feature can be divided into two features
    • Some characters are rendered incorrectly from the moment it's loaded
    • Some characters are rendered differently after you re-focus on the firefox window
  • The latter feature reproduced on firefox 3.0.x, even on the pages that was not using downloadable fonts
  • The former feature reproduced only on nightly builds with the patch applied.
  • Both features happen on Mac OS only
  • test case (for former feature only): downloadable-font-japanese.html

Bugs not filed in Bugzilla

• The layout after the new fonts load seems to be based on the old layout, as is mentioned in bug 441473 #33
  • test case: downloadable-font-layouttest.html
• Fallback function problem
  • Mac OS X
    • When it fails to download the font from every URI written in src, it just doesn't display anything ... shouldn't it display in default font?
    • Also, when the url written in src did exist but was not in a proper format, it displays nothing. Even if they had other candidates after that url, those candidates are ignored.
    • test case: downloadable-font-fallback.html
  • Windows
    • When family name defined by @font-face rule and generic family name is assigned for a class (ex. @font-face { font-family: AAA;} .test { font-family: AAA, fantasy }), if downloadable font was not downloaded, the default font is used instead of fantasy fonts.
    • test case: downloadable-font-fallback.html
• Printing problem when using src:url()
  • Mac OS X
    • Downloaded fonts are not used in printing.
    • It can be checked from Print preview. But it can not be checked from reftest-print.
  • Windows
    • Layout is wrong, and some characters are displayed on top of the others.
    • I suppose this problem is related to the layout problem mentioned in bugzilla 441473 #33.
• Some OpenType fonts are not displayed correctly when using @font-face rules on Windows
  • See the list below for details
  • Still not sure exactly when this happens
    • Maybe when we use OpenType fonts which is based on Type1(PostScript) fonts?
  • test case: not ready yet; still looking for OpenType fonts we can use for tests

Font face	Patched Build / @font-face	Patched Build / font-family	Firefox 3.0.1 / font-family
BKM-cmb10 (.otf)	✕	✓	✓
Hiragino Kaku Gothic ProN W3 (.otf)	✕	✓	✓
Verdana (.ttf)	✓	✓	✓

   ✓ Successfully rendered using the correct font face
   ✕ Failed. default font used

Bugs found after applying patch v.0.9f

• Graphic bug when using the same downloadable font previously used in a different page
  • For details, see bug441473 #66
• Firefox will crash when different types of src:url()s are used in more than two @font-face rules in a single page
  • When you use more than two @font-face rule with the following url()s, it will crash firefox
    • @font-face RULE1 with src:url(A) /// A: absolute URL, violates same-origin restriction
    • @font-face RULE2 with src:url(B) /// B: relative URL, does not violate same-origin restriction
• It takes much more time to load a font than it used to
  • As a consequence, the possibility of download failure increased
  • Unfortunately, I didn't take any statistics, so I can't compare them by exact numbers

They were fixed in patch v.0.9h :-)

Suspected Features

• It takes so much time to download and render texts when many @font-face rules are written in one document
  • test case: downloadable-font-loadtest.html
• When applying markups such as <strong> or <em> to the text rendered using downloaded fonts, it looks differently from when same markups were applied to same text rendered using system fonts.
  • For the former case, firefox calculates how strong or how italic the font should be. But for the latter case, firefox will look up the family and finds a bold face or italic face of the font. The calculated face and installed face occasionally look different.
  • test case: downloadable-font-markupdifference.html
• When using src:local(), we have to use different names for Windows and Mac
  • Windows: full name of font
  • Mac: PostScript name of font
  • For details, see bug441473 #53

Expected Features

• No support for EOT, dataFork TrueType, Bitmap Fonts

Bugs that will reproduce on Firefox 3.0.x

(that means, these bugs are not coming from the patch code, but are coming from the codes already implemented and landed)

Some fonts are not rendered on Windows

• Some fonts are not applied when using the characters not included in the font. This problem reproduces on Firefox 3.0.1
• List of fonts and the result

Font face	Patched Build / @font-face	Patched Build / font-family	Firefox 3.0.1 / font-family
Verdana	✓	✓	✓
Chalkboard	✕	★	★

   ✓ Successfully rendered using the correct font face
   ✕ Failed. default font used
   ★ Fail on some occasions. Details described under the table

• font-family: Chalkboard;
  • When using only the ordinary Latin alphabets, sentence is rendered using the chalkboard face, which is correct
  • When using the characters that are not included in Chalkboard, such as Ắ or 日, the whole sentence is rendered using the default font face
    • Expected result: The sentence is rendered using chalkboard characters, EXCEPT for the characters which is not included in chalkboard font
    • Internet Explorer 7 renders the text as expected

Unknown gryph change happens to some Japanese characters

• Unknown gryph change happens to some Japanese characters when font (which does not contain Japanese characters) is downloaded and rendered.
  • This problem happens when a page includes a line mixed of Japanese and Chinese charaters.
    • Chinese characters: Some characters are common between Japanese Kanji and Chinese characters. In this case, Chinese characters mean the characters used only in China.
  • Step to reproduce:
    • display the page
    • unfocus from the firefox window
    • wait for about a minute
    • focus again on the firefox window, and Japanese character will suddenly be rendered using Chinese fonts
  • test case: not ready yet

Fonts I use

(A list of fonts that I use in my test cases I developed so far)

Fonts I downloaded from Internet

• Bakoma cmb10.otf
  • http://www.ctan.org/tex-archive/fonts/cm/ps-type1/bakoma/
• Bitstream Vera Fonts
  • http://www.bitstream.com/font_rendering/products/dev_fonts/vera.html
  • http://www.gnome.org/fonts/
• Sazanami Fonts
  • http://sourceforge.jp/projects/efont/
  • http://sourceforge.jp/projects/efont/files/
• Scheherazade
  • http://scripts.sil.org/cms/scripts/page.php?site_id=nrsi&item_id=ArabicFonts_Download