Platform/Porting to unified sources

From MozillaWiki
Jump to navigation Jump to search

This page is here to help coordinate the effort around porting Gecko's build system to UNIFIED_SOURCES, a mechanism that enables faster builds by concatenating sources files to have fewer, larger translation units.

Ongoing porting projects

To avoid stepping on each other's toes while porting, please update this table when you start on porting a new source directory to UNIFIED_SOURCES, and remove it when it's landed.

Directory Status Owner Bug
content/canvas/src in progress bjacob bug 937886
gfx/layers in progress bjacob bug 938970
content/svg in progress ehsan bug 937908
layout/svg in progress ehsan bug 938864

Script to track porting status

Here is a script; run it from the root of your source directory to generate a report on the porting status and number of cpp files under each source directory.

#!/bin/bash

MOZBUILDFILES="`find . -name moz.build | grep -v \\./obj`"

logfile="unified-conversion-status.log"

filescount=`echo "$MOZBUILDFILES" | wc -l`
fileindex=0

(while read -r mozbuildfile
do
  fileindex=$((fileindex + 1))
  mozbuilddir="`dirname "$mozbuildfile"`"
  numcpp=`find $mozbuilddir -name "*.cpp" -o -name "*.C" | wc -l`
  if [ $numcpp -eq 0 ]
  then
    continue
  fi
  if ! grep -q SOURCES "$mozbuildfile"
  then
    continue
  fi
  echo -ne "\r$((100 * fileindex / filescount))%    $mozbuilddir\033[0K" \
    1>&2
  outputline="$numcpp $mozbuilddir"
  unified_msg=
  if grep -q UNIFIED_SOURCES "$mozbuildfile"
  then
    outputline="$outputline \o/ UNIFIED_SOURCES \o/"
  fi
  echo "$outputline"
done <<< "$MOZBUILDFILES") | sort -rn > "$logfile"

echo -e "\rResults written to $logfile\033[0K"

Current porting status

This is just the output of the above script; use it to find a new directory to work on.

The first column is the number of cpp files below each directory, so the higher number hint loosely at greater potential benefits.

Directories that are already ported have \o/ UNIFIED_SOURCES \o/ written next to them.

 586 ./gfx/skia
 290 ./js/src
 229 ./widget
 167 ./content/media
 143 ./content/svg/content/src
 126 ./gfx/angle
 111 ./gfx/layers
 91 ./content/base/src
 80 ./xpcom/tests
 80 ./xpcom/reflect/xptcall/src
 76 ./content/html/content/src
 73 ./gfx/thebes
 70 ./xpcom/reflect/xptcall/src/md/unix
 64 ./netwerk/base/src
 64 ./js/src/jsapi-tests
 61 ./widget/windows
 56 ./gfx/angle/src/libGLESv2
 55 ./security/manager/ssl/src
 53 ./content/canvas/src
 50 ./content/media/webaudio
 47 ./layout/style
 47 ./gfx/2d
 45 ./layout/generic
 42 ./widget/gonk
 41 ./layout/xul/base/src
 40 ./netwerk/test
 40 ./content/events/src
 39 ./hal
 38 ./content/xslt/src/xpath
 37 ./xpcom/glue
 37 ./dom/base
 36 ./layout/base
 36 ./ipc/ipdl/test/cxx
 36 ./gfx/graphite2/src
 35 ./layout/svg
 33 ./parser/html \o/ UNIFIED_SOURCES \o/
 33 ./netwerk/protocol/http
 33 ./build/stlport
 32 ./js/xpconnect/src
 32 ./dom/plugins/base
 30 ./content/xslt/src/xslt
 28 ./widget/xpwidgets
 27 ./widget/gtk
 26 ./widget/qt
 26 ./content/xul/templates/src
 25 ./xpcom/io
 25 ./ipc/glue
 25 ./gfx/gl
 25 ./extensions/universalchardet/src/base
 25 ./dom/plugins/ipc
 24 ./xpcom/base
 24 ./uriloader/exthandler
 24 ./toolkit/xre
 24 ./image/src
 24 ./dom/system
 24 ./dom/indexedDB
 24 ./content/smil
 24 ./accessible/src/base
 23 ./storage/src
 23 ./netwerk/protocol/rtsp
 23 ./intl/locale/src
 23 ./dom/bluetooth
 22 ./xpcom/ds
 22 ./layout/mathml
 22 ./editor/libeditor/base \o/ UNIFIED_SOURCES \o/
 19 ./accessible/src/windows/msaa
 19 ./accessible/src/atk
 18 ./xpcom/string/src
 18 ./widget/os2
 18 ./widget/android
 18 ./tools/profiler
 18 ./layout/forms
 18 ./content/media/webaudio/blink
 17 ./toolkit/mozapps/update
 17 ./toolkit/components/places
 17 ./netwerk/cache
 17 ./editor/libeditor/html \o/ UNIFIED_SOURCES \o/
 17 ./dom/system/gonk
 17 ./dom/mobilemessage/src
 17 ./dom/file
 17 ./content/xbl/src
 16 ./netwerk/cache2
 16 ./intl/uconv/src \o/ UNIFIED_SOURCES \o/
 16 ./dom/workers
 16 ./dom/camera
 15 ./js/jsd
 15 ./image/decoders
 15 ./dom/ipc
 14 ./rdf/base/src
 13 ./widget/windows/winrt
 13 ./parser/htmlparser/src \o/ UNIFIED_SOURCES \o/
 13 ./modules/libjar \o/ UNIFIED_SOURCES \o/
 13 ./media/omx-plugin
 13 ./gfx/tests/gtest
 13 ./gfx/src
 13 ./extensions/spellcheck/hunspell/src
 12 ./toolkit/crashreporter
 12 ./toolkit/components/url-classifier
 12 ./layout/tables
 12 ./docshell/base
 12 ./accessible/src/xul
 12 ./accessible/src/windows/ia2
 12 ./accessible/src/generic
 11 ./xpcom/threads
 11 ./xpcom/build
 11 ./netwerk/streamconv/converters
 11 ./gfx/ycbcr
 11 ./dom/plugins/base/android
 11 ./content/media/omx
 10 ./toolkit/mozapps/update/updater
 10 ./content/media/webspeech/synth
 9 ./media/libsoundtouch/src \o/ UNIFIED_SOURCES \o/
 9 ./layout/xul/tree
 9 ./layout/xul/grid
 9 ./layout/inspector/src
 9 ./dom/telephony
 9 ./dom/network/src
 9 ./content/media/webspeech/recognition
 8 ./xpfe/appshell/src
 8 ./js/src/gdb
 8 ./image/decoders/icon
 8 ./extensions/spellcheck/src
 8 ./editor/composer/src \o/ UNIFIED_SOURCES \o/
 8 ./caps/src
 8 ./browser/components
 8 ./accessible/src/html
 7 ./tools/jprof
 7 ./security/sandbox
 7 ./netwerk/wifi
 7 ./mozglue/build
 7 ./intl/lwbrk/src
 7 ./content/media/directshow
 7 ./content/html/document/src
 6 ./xpcom/reflect/xptcall/src/md/win32
 6 ./widget/gonk/nativewindow
 6 ./uriloader/prefetch
 6 ./tools/trace-malloc
 6 ./toolkit/system/gnome
 6 ./toolkit/components/maintenanceservice
 6 ./security/manager/pki/src
 6 ./security/manager/boot/src
 6 ./netwerk/protocol/ftp
 6 ./netwerk/ipc
 6 ./mozglue/linker
 6 ./modules/libjar/zipwriter/src \o/ UNIFIED_SOURCES \o/
 6 ./js/xpconnect/wrappers
 6 ./extensions/cookie
 6 ./extensions/auth
 6 ./embedding/browser/webBrowser
 6 ./editor/libeditor/text \o/ UNIFIED_SOURCES \o/
 6 ./dom/src/storage
 6 ./dom/fmradio
 6 ./content/xul/document/src
 6 ./content/media/wmf
 6 ./content/media/webrtc
 5 ./widget/gonk/libdisplay
 5 ./toolkit/crashreporter/client
 5 ./toolkit/components/startup
 5 ./other-licenses/skia-npapi
 5 ./netwerk/protocol/wyciwyg
 5 ./netwerk/protocol/about
 5 ./netwerk/dns
 5 ./modules/libpref/src
 5 ./memory/mozalloc
 5 ./layout/printing
 5 ./intl/unicharutil/util \o/ UNIFIED_SOURCES \o/
 5 ./intl/unicharutil/src \o/ UNIFIED_SOURCES \o/
 5 ./intl/uconv/tests
 5 ./intl/locale/src/unix
 5 ./gfx/angle/src/libEGL
 5 ./embedding/components/commandhandler/src
 5 ./editor/txmgr/src \o/ UNIFIED_SOURCES \o/
 5 ./content/xslt/src/base
 5 ./content/media/omx/mediaresourcemanager
 5 ./content/media/mediasource
 5 ./content/media/gstreamer
 4 ./xpcom/reflect/xptinfo/src
 4 ./xpcom/components
 4 ./toolkit/components/downloads
 4 ./toolkit/components/ctypes
 4 ./startupcache
 4 ./python/mozbuild/mozbuild/test/frontend/data/variable-passthru
 4 ./python/mozbuild/mozbuild/test/backend/data/variable_passthru
 4 ./netwerk/socket
 4 ./netwerk/protocol/websocket
 4 ./netwerk/protocol/device
 4 ./netwerk/cookie
 4 ./mozglue/android
 4 ./layout/tools/layout-debug/src
 4 ./intl/locale/src/windows
 4 ./intl/locale/src/os2
 4 ./gfx/cairo/cairo/src
 4 ./extensions/pref/autoconfig/src
 4 ./embedding/components/printingui/src/win
 4 ./dom/time
 4 ./dom/quota
 4 ./dom/media
 4 ./dom/fmradio/ipc
 4 ./dom/encoding
 4 ./dom/audiochannel
 4 ./docshell/shistory/src
 4 ./content/xml/document/src
 4 ./content/xml/content/src
 4 ./content/media/plugins
 4 ./content/media/ogg
 4 ./chrome/src
 4 ./accessible/src/xpcom
 3 ./xpcom/sample
 3 ./widget/shared
 3 ./tools/trace-malloc/lib
 3 ./toolkit/system/dbus
 3 ./toolkit/library
 3 ./toolkit/components/remote
 3 ./toolkit/components/ctypes/tests
 3 ./testing/gtest
 3 ./rdf/tests/dsds
 3 ./parser/xml/src
 3 ./netwerk/protocol/data
 3 ./mobile/android/components/build
 3 ./mobile/android/components
 3 ./layout/style/test
 3 ./layout/build
 3 ./js/xpconnect/tests/components/native
 3 ./js/xpconnect/loader
 3 ./js/src/shell
 3 ./js/ipc
 3 ./ipc/testshell
 3 ./ipc/dbus
 3 ./intl/locale/src/mac
 3 ./intl/hyphenation/src
 3 ./gfx/ipc
 3 ./embedding/tests/winEmbed
 3 ./embedding/components/windowwatcher/src
 3 ./embedding/components/printingui/src/unixshared
 3 ./embedding/components/printingui/src/os2
 3 ./editor/txtsvc/src \o/ UNIFIED_SOURCES \o/
 3 ./dom/wifi
 3 ./dom/power
 3 ./dom/gamepad
 3 ./dom/devicestorage
 3 ./content/xul/content/src
 3 ./content/media/webm
 3 ./content/media/encoder
 3 ./build/win32
 3 ./browser/components/shell/src
 3 ./accessible/src/windows/sdn
 2 ./xpcom/reflect/xptcall/src/md/test
 2 ./widget/qt/faststartupqt
 2 ./view/src
 2 ./uriloader/base
 2 ./toolkit/xre/test/win
 2 ./toolkit/system/unixproxy
 2 ./toolkit/mozapps/update/tests
 2 ./toolkit/mozapps/plugins/tests
 2 ./toolkit/crashreporter/test
 2 ./toolkit/components/mediasniffer
 2 ./toolkit/components/autocomplete
 2 ./toolkit/components/alerts
 2 ./testing/tools/screenshot
 2 ./security/manager/ssl/tests/unit/tlsserver/lib
 2 ./rdf/datasource/src
 2 ./profile/dirserviceprovider/src
 2 ./netwerk/system/qt
 2 ./netwerk/streamconv/test
 2 ./netwerk/protocol/viewsource
 2 ./netwerk/protocol/file
 2 ./mozglue/tests
 2 ./layout/ipc
 2 ./ipc/app
 2 ./intl/unicharutil/tests
 2 ./intl/strres/src
 2 ./intl/chardet/src
 2 ./extensions/universalchardet/src/xpcom
 2 ./extensions/permissions
 2 ./embedding/components/printingui/src/mac
 2 ./embedding/components/find/src
 2 ./dom/system/unix
 2 ./dom/system/android
 2 ./dom/src/notification
 2 ./dom/src/geolocation
 2 ./dom/promise
 2 ./dom/plugins/ipc/hangui
 2 ./dom/indexedDB/ipc
 2 ./content/xslt/src/xml
 2 ./content/media/webspeech/synth/pico
 2 ./content/media/wave
 2 ./content/media/raw
 2 ./content/media/apple
 2 ./content/mathml/content/src
 2 ./build/unix/elfhack
 2 ./browser/metro/shell/commandexecutehandler
 2 ./accessible/src/other
 1 ./xulrunner/tools/redit
 1 ./xulrunner/stub
 1 ./xulrunner/examples/simple/components/src
 1 ./xulrunner/app
 1 ./xpfe/components/windowds
 1 ./xpfe/components/directory
 1 ./xpfe/components/build
 1 ./xpcom/windbgdlg
 1 ./xpcom/tests/external
 1 ./xpcom/tests/component_no_aslr
 1 ./xpcom/tests/component
 1 ./xpcom/tests/bug656331_component
 1 ./xpcom/sample/program
 1 ./xpcom/reflect/xptinfo/tests
 1 ./xpcom/reflect/xptcall/tests
 1 ./xpcom/reflect/xptcall/src/md/os2
 1 ./xpcom/glue/tests/gtest
 1 ./xpcom/glue/standalone
 1 ./webapprt/win
 1 ./webapprt/gtk2
 1 ./uriloader/exthandler/tests
 1 ./tools/jprof/stub
 1 ./toolkit/system/windowsproxy
 1 ./toolkit/system/androidproxy
 1 ./toolkit/profile
 1 ./toolkit/library/winvccorlib
 1 ./toolkit/identity
 1 ./toolkit/devtools/server
 1 ./toolkit/crashreporter/injector
 1 ./toolkit/components/typeaheadfind
 1 ./toolkit/components/telemetry
 1 ./toolkit/components/statusfilter
 1 ./toolkit/components/satchel
 1 ./toolkit/components/reflect
 1 ./toolkit/components/perf
 1 ./toolkit/components/parentalcontrols
 1 ./toolkit/components/jsdownloads/src
 1 ./toolkit/components/intl
 1 ./toolkit/components/find
 1 ./toolkit/components/finalizationwitness
 1 ./toolkit/components/filepicker
 1 ./toolkit/components/diskspacewatcher
 1 ./toolkit/components/commandlines
 1 ./toolkit/components/build
 1 ./testing/mochitest/ssltunnel
 1 ./storage/build
 1 ./services/crypto/component
 1 ./security/sandbox/win/src/sandboxbroker
 1 ./security/sandbox/linux
 1 ./security/manager/ssl/tests/unit/tlsserver/cmd
 1 ./security/manager/ssl/tests/gtest
 1 ./rdf/util/src
 1 ./rdf/tests/triplescat
 1 ./rdf/tests/rdfpoll
 1 ./rdf/tests/rdfcat
 1 ./rdf/build
 1 ./netwerk/system/win32
 1 ./netwerk/system/android
 1 ./netwerk/streamconv/src
 1 ./netwerk/sctp/src
 1 ./netwerk/sctp/datachannel
 1 ./netwerk/protocol/res
 1 ./netwerk/protocol/app
 1 ./netwerk/mime
 1 ./netwerk/build
 1 ./memory/replace/dmd
 1 ./media/omx-plugin/lib/ics/libutils
 1 ./media/omx-plugin/lib/ics/libstagefright
 1 ./media/omx-plugin/lib/hc/libstagefright
 1 ./media/omx-plugin/lib/gb/libutils
 1 ./media/omx-plugin/lib/gb/libstagefright_color_conversion
 1 ./media/omx-plugin/lib/gb/libstagefright
 1 ./media/omx-plugin/lib/gb235/libstagefright
 1 ./media/omx-plugin/lib/froyo/libstagefright
 1 ./media/omx-plugin/hc
 1 ./media/omx-plugin/gb235
 1 ./media/omx-plugin/gb
 1 ./media/omx-plugin/froyo
 1 ./media/libspeex_resampler/src
 1 ./media/libcubeb/src
 1 ./js/xpconnect/shell
 1 ./js/ductwork/debugger
 1 ./ipc/unixsocket
 1 ./ipc/ril
 1 ./ipc/nfc
 1 ./ipc/netd
 1 ./ipc/keystore
 1 ./ipc/ipdl/test/cxx/app
 1 ./intl/build
 1 ./image/encoders/png
 1 ./image/encoders/jpeg
 1 ./image/encoders/ico
 1 ./image/encoders/bmp
 1 ./image/decoders/icon/win
 1 ./image/decoders/icon/qt
 1 ./image/decoders/icon/os2
 1 ./image/decoders/icon/gtk
 1 ./image/decoders/icon/android
 1 ./image/build
 1 ./extensions/widgetutils/src
 1 ./extensions/gnomevfs
 1 ./extensions/gio
 1 ./embedding/components/webbrowserpersist/src
 1 ./embedding/components/build
 1 ./embedding/components/appstartup/src
 1 ./embedding/browser/build
 1 ./dom/voicemail
 1 ./dom/system/windows
 1 ./dom/src/offline
 1 ./dom/src/jsurl
 1 ./dom/src/json
 1 ./dom/src/events
 1 ./dom/nfc
 1 ./dom/media/bridge
 1 ./dom/icc/src
 1 ./dom/cellbroadcast/src
 1 ./dom/browser-element
 1 ./dom/battery
 1 ./dom/apps/src
 1 ./dom/alarm
 1 ./dom/activities/src
 1 ./docshell/build
 1 ./content/svg/document/src
 1 ./config
 1 ./build/win32/vmwarerecordinghelper
 1 ./build/win32/crashinjectdll
 1 ./build/unix/stdc++compat
 1 ./browser/metro/shell/testing
 1 ./browser/metro/shell/linktool
 1 ./browser/components/migration/src
 1 ./browser/components/feeds/src
 1 ./browser/components/dirprovider
 1 ./browser/components/build
 1 ./browser/components/about
 1 ./browser/app
 1 ./b2g/gaia
 1 ./b2g/app
 1 ./accessible/src/windows/uia