MozillaWiki

DXR Future Work Plan

Revision as of 18:01, 9 March 2010 by David.humphrey (talk | contribs) (→‎Indexing, HTML Generation)

What is DXR?

DXR is a two things. First, it’s a method for collecting type, member, statement, macro, etc. information about C++, IDL, and soon JavaScript using Mozilla’s static analysis tools (e.g., Dehydra), a hacked build system, and a bunch of scripts. Second, it’s a web-based tool for mapping this information back onto source code, and allowing users to query and look-up this information.

You can read more about it here.

DXR was written and is being actively developed by David Humphrey.

DXR Bugs

Current Task List

The current set of tasks can be summarized into the following high-level categories:

  • Improve the DXR web-app backend
  • Improve the indexing of source code and creation of the database and HTML
  • Improve user access to data stored in the database, making it easier to query, search, etc.
  • Improve the presentation and accuracy of data presented within the source code itself

Each of these three have sub-tasks, many of which interrelate.

Web App Back-end Work

  • Redo existing python to make it easier to extend
  • Switch python web cgi to use django
  • Audit web app for security
  • Start using caching
  • Investigate switching from SQLite to a db server
  • Investigate adding some basic auditing and user feedback:
    • What are people searching for most often?
    • Allow users to star or otherwise mark things that are wrong in the HTML data
    • Perhaps allow users to leave notes, links, etc. in various parts of the code

Indexing, HTML Generation

  • Figure out best method for doing continuous re-indexing of the tree
  • Improve post-processing speed, investigate better parallelism
  • Add proper IDL data to source code such that clicking types, members, etc. shows a popup with links to implementations, callers, etc.
  • Fix C++ snippets in IDL to use cpp position indicators (# line file). Add any other missing IDL features.
    • Switch from using xpidl to xpidl.py and add __FILE__ and __LINE__ data to IDL C++ snippets
  • Look for speed-ups in how analysis builds are done
  • Add links for #include header files
  • Add support for C source
  • Add support for static functions (see nsContentUtils)
  • Add support for JS source. Consider using jshydra to build static HTML. See js token info in jsscan.h.
  • Add support for Obj-C/C++ source
  • Show compiler warnings
  • Integrate callgraph: convert to using treehydra, investigate --with-static-checking in mozilla's build sys, include dwitte's callgraph.js and my dxr.js (include "callgraph.js").
  • Tokenize macros so that embedded macros are clickable (e.g., NS_ProcessNextEvent resolves to NS_ProcessNextEvent_P)
  • Investigate ways to deal with naming collisions. For example, where types, methods, or macros share the same name, sometimes we pick the wrong one for purposes of linkifying (e.g., NS_ProcessNextEvent).
  • Make IDL pages, and other decl locations do a better job at providing links
  • Fix srcdir vs. objdir links (e.g., PRInt32)
  • Remove .html from source file names (mod_rewrite?)
  • Make dxr scripts path agnostic and more portable (python path stuff)
  • Fix ctor html linkifying code so Foo::Foo works properly
  • Figure out how to deal with code being ifdef'ed out (e.g., debug vs. non-debug build)
  • Audit dehydra script for improvements
  • Treat functions as well as methods
  • Add a download-db link
  • Link to the hg revision of the indexed db (i.e., a timestamp)
  • Add simple hooks and ways for new types of data to be layered into the DB and web app
  • Investigate minifying html - http://perfectionkills.com/experimenting-with-html-minifier/

Querying, Search

  • Make Users queries differentiate decls/defs
  • Add ability to do Users:nsIFrameFrame::GetDocShell
  • Add base:type search query
  • Show Classname:: as part of member sidebar on the right (for searches)
  • Make search results clickable anywhere vs. just on line number and make line number plain text vs. link.
  • Make search sidebars links so they can be turned into searches. For example, members sidebar could be a link that takes you to a member:foo query
  • Search UI should encourage using built-in queries, for example, suggesting other keywords (members:) that could be added.
  • When searching of nsFoo, display a little summary of the class bases,derived, members
  • For Type search results, give a link to do a type:: search to show all members (if there are any) -- nsFoo (class) [members]
  • In search results, favour c++ style name ordering (e.g., class nsFoo vs. nsFoo (class) ). Make the name more prominent.
  • In search results, make all types and member names clickable
  • Remove side bar from searches, and copy Google Code search results by showing other queries at top of results (Narrow your search: 53 Types, 74 Members, ...)
  • In decl/def search results (e.g., member) only show line number once.
  • Add counts to all search results (glimpse and otherwise)
  • Add pagination to search results
  • Limit search nav divs to some lower number (12?) and give a link to see the whole thing.

Source Code View

  • Pin Popup when scrolling (Jeff and jdm want)
  • Make source code nav bar narrow if contents are narrower, and expand to a max size (current size).
  • When navigating to search result in code for a type or member, sync the source code navigation pane accordingly
  • Lazily keep source code nav and source view in sync. Clicking any stmt should highlight/scroll into view/etc the containing member/class
  • Make source code nav panel sortable (alpha, file order)
  • Make types clickable in source code nav bar, perhaps open the popup to show type info
  • Make types in source code nav bar always clickable, and add icon to take you to decl (should be able to jump to header file from cpp)
  • Add hrefs to links in source code dom nodes on hover
  • Standardize on look and feel of source code nav bar for members and popups, etc (gray, icons, font)
  • Tighten-up look of popup box
  • Fix pop-up position so it isn't off the screen
  • Fix Users query so it includes uses of the type directly in addition to uses of derived types (e.g., nsIFoo might have direct users if not an idl type)
  • Add Callees and Callers searches to all popups keyed on loc
  • Add info to popups that indicates that empty popups are bugs in data/dxr or TODOs (maybe with link to this wiki page or bug filing info).
  • Collapse Type Info and Bases tabs in popups
  • Convert popup to use fewer/no tabs, and dynamic/tree regions to show data like members, bases, etc.
  • Make http://dxr.proximity.on.ca/dxr/mozilla-central/xpcom/ds/nsHashtable.h.html#144/Put work

Future Ideas (not currently in scope)

  • Integrate other tools:
  • Add Macro decl locations
  • Investigate strategy for increment re-indexing vs. full clobbers
  • Add support for html/xml, perhaps with links to MDC doc pages
  • Multi-platform indexing
  • Add gcov support
  • Add build dependency map info - "what do I have to remake if I change something in content/html/document/src?"
  • Add control mechanism for running dxr continuously. Maybe web based? Maybe as part of buildbot with waterfall?
  • Patch/Review features: try-server type system that checks the patch for issues, but perhaps dxr should be used to visualize the results. Show things like, new warnings/issues/deadcode. Also, given a patch, show everything DXR knows about the patch and code affected by it.
Retrieved from "https://wiki.allizom.org/index.php?title=DXR_Future_Work_Plan&oldid=207078"