Very interesting. I think the accessible tree creation stuff is great. I worry a bit about how it overrides markup. There is a complex hierarchy between native html elements, aria, and now this. It becomes even more confusing for the developer how a name or role and other semantics are computed.
From a perspective of authoring ATs with this, we need to have some larger architectural thoughts. For example, in FxOS it would be hard to keep the current top-level IPC design. instead, we may need to have a way to have scripts injected into each content process and yet somehow work in harmony and not bother each other (for example, hijacking input or speech output interruptions). Also, if we inject 3rd party scripts like that, we would need to have an appropriate security model.
Here are some random questions and thoughts in no particular order:
AccessibleElement
- What is the difference between the parent attribute and the logical parent relationship? Is it like aria-owns?
- What is a widget relationship? Is the widget an ancestor that is the root of the composite widget?
AccessibleValue Pattern Example
- How would one set the value or step up/down? via a method in the pattern or through actions?
- I guess this question should be more generalized: When does something fit a pattern or an action?
Actions
- I like how focus is folded into these methods.
- I would harmonize with IndieUI.
- How can you have two 'activate' actions if ActionSet is a name/object mapping?
- Additional actions:
- Zoom
- Drag and drop?
- expand/collapse
- select/unselect
- check/uncheck
- undo/redo
- value step up/down
- The relationship between interactions and actions is confusing, both a AccessibleElement.interactions attribute, and one for each action? Does the sum of all the action interactions equal the set of top-level interactions?
Traversal
I'm still wrapping my head around this proposal, it seems very different than what we have, and has potential. But really hard to concretetly understand how our current current design would functionally map to this.
- Does a move() call change the 'anchor' attribute? Could it also change the wherebit? How?
- I don't see a common use case of cyclic forward/backward.
- How do you query an AccessiblePos for the offset of the position within the anchor's text? Do you always need another AccessiblePos to compare with?
- I could see a use for AccessiblePos.clone() + AccessiblePos.move() to get move deltas.
- Is this what you see as replacing nsIAccessibleText?
- The document.position is changed by assigning to it a new AccessiblePos?, or by calling document.position.move()? If the latter, could the move() method change the wherebit?
- I am assuming that the cursor_moved event would provide the old AccessiblePos? Lets say we want to listen for cursor_change and outline the current position. For text nav, that would probably work well with doc.outline(oldPosition, newPosition). But what if we are navigating by roles like links? It will outline the entire area between the two links, no?
- I don't understand TextUnit.bit, is it related to the wherebit?
- Can you pass null to getBoundingRect to get the bounds of the anchor?
- Like yura says, positionFromPoint needs a similar argument to pos.move(). Use cases: (1) Ignore layout containers. (2) Speak word or sentence under finger/pointer.
AccessibleSource Etc.
- I am assuming after you do canvas.addHitRegion that all the bounds methods would work on it, like outline(), getBoundingRect(), etc? Plus canvas.scrollPathIntoView()?
- Speaking of scrolling, the relationship between layout, canvas and custom 'scroll' actions will be interesting.
- The taxonomy extensibility is awesome.