QA/Firefox3.1/FontFace TestStatus: Difference between revisions

From MozillaWiki
< QA‎ | Firefox3.1
Jump to navigation Jump to search
Line 94: Line 94:
|}
|}


LoadTest (done on patch v.0.9d ... not on patch v.0.9h yet)
LoadTest (stress test)
* even 100 @font-face rules per page seems to be pretty scalable. When 210 (230 for the first page you load) or more @font-face rules are used in a document, firefox will crash.
* even 100 @font-face rules per page seems to be pretty scalable. A web page with 200 @font-face rules will be loaded in around 5 seconds. When 250 or more @font-face rules are used in a document, firefox will crash.


== Tests on Windows XP SP3 ==
== Tests on Windows XP SP3 ==

Revision as of 22:38, 12 September 2008

« QA/Firefox3.1/FontFace TestPlan

Notes

@font-face test status Test Plan: @font-face TestPlan

Executed Tests

(Tests Already Executed)

These tests were executed after applying patchv.0.9h from bug441473

Tests on Mac OS X

Executed Tests

file description status failure cause
downloadable-font-conflictname.html When the family name is same as a font family name in user's environment Passes
downloadable-font-diacritic.html Check if the diacritic marks are displayed correctly Passes
downloadable-font-fallback.html Check the fallback functions Sometimes Fails Font not loaded (it happens pretty often)
downloadable-font-familywithseveral.html Check what happens when multiple @font-face rules are used for one font-family Sometimes fails Font is not loaded (it happens quite often)
downloadable-font-iframe.html Check if the downloaded fonts can be used from other documents (Answer should be no) Passes
downloadable-font-japanese.html Incorrect rendering of some Japanese fonts Fails see Bugzilla for detail [bug451426]
downloadable-font-layouttest.html Check the layout of downloaded fonts Fails Layout of default font used. Mentioned in [bug441473 #33]
downloadable-font-leavepagetest.html Check if the firefox crashes when leaving the page during the font download Sometimes crash Crash only seems to happen when you invoke firefox with -reftest option. And when crash happens, it always happen after you load the next test HTML in reftest, not when you loaded this leavepagetest.html. But I'm not sure exactly when this crash happens, because this only happens about once in two trials. Still investigating the exact cause. [bug451462]
downloadable-font-ligature.html Check if the ligatures are displayed correctly Passes
downloadable-font-loadtest.html Check how much downloadable fonts can be handled in a page without any crash Sometimes fails Fonts are not loaded. Even when the fonts are loaded, layout bug can be seen sometimes.
downloadable-font-markupdifference.html Check what happens when applying HTML markups to downloaded fonts Fails For example, when you apply <em> to a text rendered with downloaded fonts, the italic FACE IS CALCULATED. However, when you apply <em> to a text rendered with font in user's environment, the ITALIC FACE of that font-family is used. The former calculated face and latter italic face is different, so they are rendered slightly differently.
downloadable-font-opentype.html Check if the opentype fonts are rendered correctly Passes
downloadable-font-printtest.html Check if the downloaded fonts are used when you print out the document Passes
downloadable-font-sameorigin.html Check if the same-site restriction is working properly Sometimes Fails Fonts not loaded

LoadTest (stress test)

  • even 100 @font-face rules per page seems to be pretty scalable. A web page with 200 @font-face rules will be loaded in around 5 seconds. When 250 or more @font-face rules are used in a document, firefox will crash.

Tests on Windows XP SP3

Executed Tests

file description status failure cause
downloadable-font-conflictname.html When the family name is same as a font family name in user's environment Passes
downloadable-font-diacritic.html Check if the diacritic marks are displayed correctly Passes
downloadable-font-fallback.html Check the fallback functions Fails The priority of fonts in fallback function seems to be wrong.

When a rule was like

@font-face { font-family:test; src: ; } .test { font-family:test, fantasy; }

it displays the default font, not the fantasy font.

Please be careful that fallback function has different problems in Windows and Mac.

downloadable-font-familywithseveral.html Check what happens when multiple @font-face rules are used for one font-family Fails When you specify a non-italic face with a font-style:italic rule, it is rendered in italic. It should not be in italic. It should be rendered using the exact font face you specified.
downloadable-font-iframe.html Check if the downloaded fonts can be used from other documents (Answer should be no) Passes
downloadable-font-japanese.html Incorrect rendering of some Japanese fonts Passes
downloadable-font-layouttest.html Check the layout of downloaded fonts Fails Layout of default font used. Mentioned in [bug441473 #33]
downloadable-font-leavepagetest.html Check if the firefox crashes when leaving the page during the font download Sometimes Crashes Crash happens occasionally. It is not restricted to when firefox is invoked with -reftest option. When you load leavepagetest.html and loadtest.html back and force, it will crash after several tries.

When this page is loaded, the following assertion is displayed:

ASSERTION: Null pres shell: 'mShell', file ../../dist/include/layout/nsPresContext.h, line 185

downloadable-font-ligature.html Check if the ligatures are displayed correctly Passes
downloadable-font-loadtest.html Check how much downloadable fonts can be handled in a page without any crash Sometimes fails Layout bug [bug441473 #33]

It seem to happen when it fails to download the font before the first rendering. But I'm not sure when it happens. The place it happens changes every time.

downloadable-font-markupdifference.html Check what happens when applying HTML markups to downloaded fonts Fails For example, when you apply <em> to a text rendered with downloaded fonts, the italic FACE IS CALCULATED. However, when you apply <em> to a text rendered with font in user's environment, the ITALIC FACE of that font-family is used. The former calculated face and latter italic face is different, so they are rendered slightly differently.
downloadable-font-opentype.html Check if the OpenType fonts are rendered correctly Fails OpenType font is ignored. Default font is used.
downloadable-font-printtest.html Check if the downloaded fonts are used when you print out the document Fails Layout of default font used. Mentioned in [bug441473 #33]
downloadable-font-sameorigin.html Check if the same-site restriction is working properly Passes

LoadTest (done with patch v.0.9d, not with patch v.0.9h yet)

  • 20 @font-face rules per page is pretty scalable. When you scroll the page or do some action during the downloading and rendering of a font, rendering will take a lot more time. So if more than 50 @font-face rules are used, it would take too much time for a user to wait (nearly 10 seconds, but not measured)
  • It does not crash even if a page had more than 1000 @font-face rules, but it would take more than 5 minutes until the text is rendered, which is not endurable for general use.
    • The more @font-face rules you have in a page, the more time is needed to its rendering. It seems that the time it takes is O(n^2) or something

Planned (but not executed) Tests

(Tests Not Executed Yet)

Unicode-Range Tests

  • Basic Tests
    • single code point
      • ex. U+A5, U+301C
    • interval value
      • ex. U+3040-30FF, U+999-998 (does this fail?)
  • What happens if I put many 0s before the number?
    • ex. U+F, U+0F, U+00F, U+000F, U+00000000F, ... do they mean the same thing?
  • What happens if I use ? in the front/middle?
    • ex. U+?4, U+3?5, ...
  • What happens if I use multiple unicode-range rules to consist a single font-family?
    • ex. @font-face{ font-family:testA; src:url(fontA.ttf); unicode-range:U+0-U+2FF; } @font-face{ font-family:testA; src:url(fontB.otf); unicode-range:U+300-U+FFF }
  • What happens if I specified only a single character in the rule and tried to display other character? Would there be a fallback function?
    • ex. @font-face { font-family:smallA; src:url(smallA.ttf); unicode-range:U+1; }
    • <span class="font-family:smallA">ABC</span>?
  • What happens if the specified font did not have a character of the specified unicode range? Does it display nothing? Would there be a fallback function?

CSS Mutation from JavaScript tests

  • What happens if you download the same font again? Would it use cache?
  • Is it possible to change the DOM tree of other documents?
    • Is downloading another font for other documents prohibited? or is it allowed?
  • Consider what will happen when 2 pages try to change the DOM of each other
    • stateful

Possible Risks

(Known bugs, not-yet-implemented features, questions)

Bugs already filed in Bugzilla

  • Firefox crashes when you leave the page before the font download finishes.
  • Unknown gryph change happens to some Japanese characters when font (which does not contain Japanese characters) is downloaded and rendered.
    • Now filed as bug 451426
    • It happens when the file is encoded in Japanese encoding (Shift_JIS, EUC-jp). But it also happened on files encoded in UTF-8.
    • As written in bug 451426, this feature can be divided into two features
      • Some characters are rendered incorrectly from the moment it's loaded
      • Some characters are rendered differently after you re-focus on the firefox window
    • The latter feature reproduced on firefox 3.0.x, even on the pages that was not using downloadable fonts
    • The former feature reproduced only on nightly builds with the patch applied.
    • Both features happen on Mac OS only
    • test case (for former feature only): downloadable-font-japanese.html

Bugs not filed in Bugzilla

  • The layout after the new fonts load seems to be based on the old layout, as is mentioned in bug 441473 #33
    • test case: downloadable-font-layouttest.html
  • Fallback function problem
    • Mac OS X
      • When it fails to download the font from every URI written in src, it just doesn't display anything ... shouldn't it display in default font?
      • Also, when the url written in src did exist but was not in a proper format, it displays nothing. Even if they had other candidates after that url, those candidates are ignored.
      • test case: downloadable-font-fallback.html
    • Windows
      • When family name defined by @font-face rule and generic family name is assigned for a class (ex. @font-face { font-family: AAA;} .test { font-family: AAA, fantasy }), if downloadable font was not downloaded, the default font is used instead of fantasy fonts.
      • test case: downloadable-font-fallback.html
  • Printing problem when using src:url()
    • Mac OS X
      • Downloaded fonts are not used in printing.
      • It can be checked from Print preview. But it can not be checked from reftest-print.
    • Windows
      • Layout is wrong, and some characters are displayed on top of the others.
      • I suppose this problem is related to the layout problem mentioned in bugzilla 441473 #33.
  • Some OpenType fonts are not displayed correctly when using @font-face rules on Windows
    • See the list below for details
    • Still not sure exactly when this happens
      • Maybe when we use OpenType fonts which is based on Type1(PostScript) fonts?
    • test case: not ready yet; still looking for OpenType fonts we can use for tests
Font face Patched Build / @font-face Patched Build / font-family Firefox 3.0.1 / font-family
BKM-cmb10 (.otf) ✕ ✓ ✓
Hiragino Kaku Gothic ProN W3 (.otf) ✕ ✓ ✓
Verdana (.ttf) ✓ ✓ ✓ 
   ✓ Successfully rendered using the correct font face
   ✕ Failed. default font used

Bugs found after applying patch v.0.9f

  • Graphic bug when using the same downloadable font previously used in a different page
  • Firefox will crash when different types of src:url()s are used in more than two @font-face rules in a single page
    • When you use more than two @font-face rule with the following url()s, it will crash firefox
      • @font-face RULE1 with src:url(A) /// A: absolute URL, violates same-origin restriction
      • @font-face RULE2 with src:url(B) /// B: relative URL, does not violate same-origin restriction
  • It takes much more time to load a font than it used to
    • As a consequence, the possibility of download failure increased
    • Unfortunately, I didn't take any statistics, so I can't compare them by exact numbers

They were fixed in patch v.0.9h :-)

Suspected Features

  • It takes so much time to download and render texts when many @font-face rules are written in one document
    • test case: downloadable-font-loadtest.html
  • When applying markups such as <strong> or <em> to the text rendered using downloaded fonts, it looks differently from when same markups were applied to same text rendered using system fonts.
    • For the former case, firefox calculates how strong or how italic the font should be. But for the latter case, firefox will look up the family and finds a bold face or italic face of the font. The calculated face and installed face occasionally look different.
    • test case: downloadable-font-markupdifference.html
  • When using src:local(), we have to use different names for Windows and Mac
    • Windows: full name of font
    • Mac: PostScript name of font
    • For details, see bug441473 #53

Expected Features

  • No support for EOT, dataFork TrueType, Bitmap Fonts

Bugs that will reproduce on Firefox 3.0.x

(that means, these bugs are not coming from the patch code, but are coming from the codes already implemented and landed)

Some fonts are not rendered on Windows

  • Some fonts are not applied when using the characters not included in the font. This problem reproduces on Firefox 3.0.1
  • List of fonts and the result
Font face Patched Build / @font-face Patched Build / font-family Firefox 3.0.1 / font-family
Verdana ✓ ✓ ✓
Chalkboard ✕ ★ ★ 
   ✓ Successfully rendered using the correct font face
   ✕ Failed. default font used
   ★ Fail on some occasions. Details described under the table
  • font-family: Chalkboard;
    • When using only the ordinary Latin alphabets, sentence is rendered using the chalkboard face, which is correct
    • When using the characters that are not included in Chalkboard, such as Ắ or 日, the whole sentence is rendered using the default font face
      • Expected result: The sentence is rendered using chalkboard characters, EXCEPT for the characters which is not included in chalkboard font
      • Internet Explorer 7 renders the text as expected

Unknown gryph change happens to some Japanese characters

  • Unknown gryph change happens to some Japanese characters when font (which does not contain Japanese characters) is downloaded and rendered.
    • This problem happens when a page includes a line mixed of Japanese and Chinese charaters.
      • Chinese characters: Some characters are common between Japanese Kanji and Chinese characters. In this case, Chinese characters mean the characters used only in China.
    • Step to reproduce:
      • display the page
      • unfocus from the firefox window
      • wait for about a minute
      • focus again on the firefox window, and Japanese character will suddenly be rendered using Chinese fonts
    • test case: not ready yet

Fonts I use

(A list of fonts that I use in my test cases I developed so far)

Fonts I downloaded from Internet

Retrieved from "https://wiki.allizom.org/index.php?title=QA/Firefox3.1/FontFace_TestStatus&oldid=108679"