Webmaker/Code: Difference between revisions
No edit summary |
No edit summary |
||
| Line 125: | Line 125: | ||
===3. Understand Webmaker Code=== | ===3. Understand Webmaker Code=== | ||
The Webmaker code is spread across a large number of repositories, modules, libraries, apps, and web sites. Trying to locate the code for a particular part of Webmaker can be a challenge. The following is a | The Webmaker code is spread across a large number of repositories, modules, libraries, apps, and web sites. Trying to locate the code for a particular part of Webmaker can be a challenge. The following is a high level introduction to the code. | ||
====Technical Overview of Webmaker==== | |||
Users typically begin at https://webmaker.org, and so our discussion will start there too. The webmaker.org site has four main features, including: | |||
# allowing users to access Webmaker tools, including '''Popcorn Maker''', '''Thimble''', and '''X-Ray Goggles'''. | |||
# allowing users to search for and browse various "makes," which are objects in the '''Make API''' made with one of the Webmaker tools, or by a third-party web tool. | |||
# allowing users to create or find Webmaker Events, using a Google Maps driven sub-application often referred to as '''Webmaker Events''' or just '''Events'''. | |||
# allowing users to create a Webmaker (and by extension, Persona) account. | |||
====A Tour of Webmaker Code==== | ====A Tour of Webmaker Code==== | ||
| Line 166: | Line 177: | ||
* list of all our git repos, owners/peers | * list of all our git repos, owners/peers | ||
* high-level diagram/description of how the pieces of Webmaker fit together | * high-level diagram/description of how the pieces of Webmaker fit together | ||
====Webmaker Style Guide==== | |||
Anyone working on Webmaker's front-end, or otherwise altering aspects of user-facing UI/UX should be aware of the [https://wiki.mozilla.org/Webmaker/styleguide Webmaker Style Guide]. The Style Guide provides important information about Webmaker's design philosophy and branding, typography, logos, colours, thumbnails, etc. | |||
===4. Learn the Webmaker Development Workflow=== | ===4. Learn the Webmaker Development Workflow=== | ||
Revision as of 17:27, 11 October 2013
NOTE: this page is under active development, and will be changing rapidly over the coming days. Apologies for any chaos you discover at the moment.
Introduction
Webmaker is about empowering everyone to become makers of the web instead of just users. It's a mix of web tools, teaching material, and a social platform for teaching, learning, and sharing what you make. And, it's a lot of fun to use. It's even more fun to develop!
Webmaker is built using modern web technologies including Node.js, HTML5, CSS, and JavaScript (lots of JavaScript!)--many of the same technologies Webmaker teaches. If you know these technologies, or are interested in learning them more deeply, you might be interested in helping us develop Webmaker.
Getting Involved
There are many ways you can get involved in building the Webmaker project. Here are a few ideas:
- If you know JavaScript, you could help fix bugs or add features to our servers (e.g., the MakeAPI), JS libraries and modules (e.g., popcorn.js or our localization tools), or web tools.
- If you know HTML/CSS, you could work on our web sites and web tools, for example webmaker.org, Popcorn Maker, Thimble, etc.
- If you know MySQL/MongoDB/ElasticSearch and are interested in databases and data, you could help us work on the MakeAPI or Login server.
- If you're interested in Bug Triage, you could help us keep the Webmaker Product on bugzilla neat and tidy.
- If you know how to speak more than one language, you could work on localization and help us maintain our localization code.
- If you know how to work on Firefox/FirefoxOS or another Mozilla product/project, you could help us find ways to integrate Webmaker, for example building browser addons.
- If you know about automation/scripting/packaging/deployment, you could work with developer operations group to help deploy and manage Webmaker.
- If you're good at finding issues, confirming bugs, or otherwise breaking things, you could help us test Webmaker code on our staging and production servers.
- If you know how to do UI/UX work, you could join our design and front-end group to design and implement new user-facing aspects of our sites and tools.
These are just a few ideas to get you thinking--there's many ways that someone who wants to work on Webmaker can get involved. If you're passionate and interested in getting started, let us know and we'll try to find something that fits your interests and skills.
Communication
In order to get started on Webmaker code, you need to get involved with our community. Here are some ways to introduce yourself and get help:
- IRC: The #webmaker channel on moznet is our primary dev channel. If you're new to irc, please see the docs on learning how to use it. NOTE: our development team is spread across the world, with primary activity happening in Toronto, Vancouver, London, Berlin, New York, and San Francisco. If you're in a timezone that doesn't overlap with these, the mailing lists might be a better way to reach out.
- Mailing Lists: There are two primary lists: webmaker@lists.mozilla.org for general Webmaker communication; and webmaker-dev@mozilla.org for developer communication.
- Bug Tracker: We use Mozilla's Bugzilla to track our work, see the Webmaker Product in Bugzilla.
- Blog: Updates about the development of the Webmaker Product go on our Webmaker blog.
Developing Webmaker
Webmaker is a big project, and it can take some time to get set up and contributing. The following is a guide to our workflow, technologies, repositories, etc. Please read this documentation to help you get started, and ask follow-up questions on irc or the webmaker-dev list.
1. Set up a Webmaker Development Environment
Before you can test or make improvements to Webmaker, you need to get it running locally. This involves setting up your OS to host the Webmaker servers and apps, and installing various development tools you'll need. We strongly recommend a Unix-like OS, such as Linux or OS X, but it is also possible to use Windows.
Dependencies
Webmaker relies on a number of technologies and tools, which must first be installed. The following list needs to be installed and working before you can run or test Webmaker locally:
- node.js and npm
- bower: after you install npm, you can type
npm install -g bower(may require admin/sudo rights) - grunt: after you install npm, type
npm install -g grunt-cli(may require admin/sudo rights) - python 2.7
- pip
A number of node modules we use (e.g., SQLite) also require a working C/C++ toolchain. You should make sure that you have a working build environment for your OS (see the section of the Webmaker Suite docs on this issue).
Optionally, if you plan to work on, or run the MakeAPI locally, you will also require the following:
- MongoDB (NOTE: there is some discussion about dropping MongoDB altogether, but at the time of writing, this hasn't happened yet)
- Java
- Elastic Search
Setup Option 1: Webmaker Suite
By far the easiest way to get all of Webmaker set up and installed locally, on any platform, is to use Webmaker Suite. This is a set of automation scripts written in node.js by one of our lead developers, and is meant to provide turn-key installation by automatically downloading all the code, and setting default environment variables.
Complete instructions are available in the README for Webmaker Suite on github.
Setup Option 2: Local Native Installation
Installing and running Webmaker on your local OS requires you to firstsetup
Setup Option 3: Using Vagrant (i.e., Ubuntu-based VM)
TODO
Setup Option 4: Heroku
TODO
2. Find or File a Webmaker Bug
All the work we do is tracked in Bugzilla, and knowing how to find exiting bugs and how to file new bugs is important.
Why Bugzilla?
This question comes up sometimes, and it's worth understanding our reasons for choosing Bugzilla over Github Issues or some other tool--we have gone through a long process of discussing and exploring other options, and Bugzilla is what works best for our use case.
Webmaker isn't developed as a single code-base. This is partly due to historic reasons (i.e., Webmaker was assembled out of different, existing projects) and partly due to our deployment needs (i.e., needing to be able to scale parts of Webmaker differently in production). Webmaker is also larger than the sum of its code, and encompasses a large teaching, mentoring, and event-based community, all of which is tied into the development of the tools and apps.
As a result of the highly distributed yet interconnected nature of the Webmaker code, taking a per-repository view of of the issues/bugs has proven difficult. First, a bug might touch multiple parts of the product, and need to be spread across many repositories--it's not uncommon for a single bug to touch 5 or 6 different repositories. Second, often a bug that appears to belong to a tool like Thimble actually needs to get filed and fixed in a repository that isn't named Thimble at all, and users (and many developers not familiar with the code) can't find their way through all the Webmaker repositories.
We have taken a philosophy that favours users and bug-fillers over developers, and one which tries to provide a wide net for collecting feedback. People filing Webmaker bugs, unless they are on the development team, typically don't know our code or where bugs belong. If a bug gets filed in Github Issues, it's not possible to move it to another repository. In Bugzilla this is trivial, and it means that we can triage bug reports, CC the right people, and get our work done faster across all of Webmaker. We also favour an approach that allows us to lump bugs about code, content, and community activities into a single bucket, since Webmaker is more than just HTML, CSS, and JavaScript.
Another advantage to using the Mozilla's Bugzilla has been the opportunity to interface with other areas of Mozilla. A good example is the Security Team, who routinely helps us with security-critical bugs. In Bugzilla one can flag these such that they aren't open to the public, and therefore don't document exploits. This isn't (currently) possible in Github.
When it comes to issue trackers, there is no silver bullet, and it's impossible to please everyone. By using both Github and Bugzilla, the former for pull requests and code review, and the latter for global projecct issue tracking, we think we've struck the right balance.
Using Bugzilla
If you haven't used Bugzilla before, you need to create an account. A good first guide to Bugzilla for Webmaker users is available here: http://blog.webmaker.org/bugzilla. You should also watch [blog.johnath.com/2010/02/04/bugzilla-for-humans/ Bugzilla for Humans], and read this post. There are many good resources on learning Bugzilla.
Bugzilla is organized into Products, which in turn have Components. All Webmaker bugs live under the Webmaker product, and in one of its components (see the complete list and description of each). Developers often watch particular components, which means an email will be sent for all bugs filed or updated in that component.
New bugs
File a Webmaker bug: https://bugzilla.mozilla.org/enter_bug.cgi?product=Webmaker
TODO:
- using Bugzilla - http://blog.webmaker.org/bugzilla
- Webmaker Product tour (components, what's in each, who owns them)
- Useful bug queries to find things to work on
- good-first-bug, student-project, mentored-bug?
3. Understand Webmaker Code
The Webmaker code is spread across a large number of repositories, modules, libraries, apps, and web sites. Trying to locate the code for a particular part of Webmaker can be a challenge. The following is a high level introduction to the code.
Technical Overview of Webmaker
Users typically begin at https://webmaker.org, and so our discussion will start there too. The webmaker.org site has four main features, including:
- allowing users to access Webmaker tools, including Popcorn Maker, Thimble, and X-Ray Goggles.
- allowing users to search for and browse various "makes," which are objects in the Make API made with one of the Webmaker tools, or by a third-party web tool.
- allowing users to create or find Webmaker Events, using a Google Maps driven sub-application often referred to as Webmaker Events or just Events.
- allowing users to create a Webmaker (and by extension, Persona) account.
A Tour of Webmaker Code
For the most part, Webmaker code lives in one of many repositories hosted under the Mozilla Github Organization. We also contribute to a number of upstream projects, not all of which are listed below (e.g., node.js modules or libraries we use, but don't maintain directly). There are a few exceptions, but we try to graduate repositories we rely on to the Mozilla Organization instead of hosting them under individual github accounts. For more information about any of these repositories, you are encouraged to consult each their READMEs.
- Webmaker.org: https://github.com/mozilla/webmaker.org - the node.js app and front-end code for https://webmaker.org, including things like Events and the Web Literacy Standard.
- Thimble: a friendly real-time updating code editor, includes:
- thimble.webmaker.org: https://github.com/mozilla/thimble.webmaker.org - the node.js app and main/shell front-end code.
- friendlycode: https://github.com/mozilla/friendlycode/ - the code editor used by Thimble
- slowparse: https://github.com/mozilla/slowparse - an HTML5 parser with extra metadata and error handling
- togetherjs: https://github.com/mozilla/togetherjs/ - TogetherJS real-time collaboration library
- Popcorn Maker: a web-based media editing tool, includes:
- popcorn.webmaker.org: https://github.com/mozilla/popcorn.webmaker.org - the node.js app and front-end for Popcorn Maker
- popcorn.js: https://github.com/mozilla/popcorn-js - the Popcorn.js HTML5 media library
- mediasync: https://github.com/mozilla/webmaker-mediasync - sync/query services for media APIs
- X-Ray Goggles: https://github.com/mozilla/goggles.webmaker.org - the node.js app and front-end bookmarklet code for X-Ray Goggles, a tool for visually exploring and changing the underlying structure of web pages
- login.webmaker.org: https://github.com/mozilla/login.webmaker.org - node.js single-sign-on (SSO) server and identity provider for Webmaker apps, as well as front-end UI components for authentication, based on Mozilla's Persona
- MakeAPI: https://github.com/mozilla/makeapi - node.js metadata server for storing and indexing information about "makes" (e.g., things made on the web with Webmaker tools or 3rd party tools).
- Make Valet: https://github.com/mozilla/make-valet - node.js app for hosting information about makes (e.g., Make Details).
- Webmaker Profile: https://github.com/mozilla/webmaker-profile - node.js app and front-end for username.makes.org profile pages.
- webmaker-profile-service: https://github.com/mozilla/webmaker-profile-service - RESTful API for profile
- Common Node/JS Modules used by many Webmaker apps and code:
- webmaker-sso: https://github.com/jbuck/node-webmaker-sso - Single-Sign-On solution for Persona
- makeapi-client: https://github.com/mozilla/makeapi-client - node.js and browser-js client lib for communicating with the MakeAPI
- webmaker-i18n: https://github.com/mozilla/node-webmaker-i18n - Localization and Internationalization code for node.js and HTML.
- webmaker-loginapi: https://github.com/mozilla/node-webmaker-loginapi - client lib for accessing Login server from node.js
- webmaker-postalservice: https://github.com/mozilla/node-webmaker-postalservice - Webmaker mailer for apps requiring email services
- mox-server: https://github.com/Pomax/mox-server - mock implementation of the Amazon S3 service in node.
TODO
- list of all our git repos, owners/peers
- high-level diagram/description of how the pieces of Webmaker fit together
Webmaker Style Guide
Anyone working on Webmaker's front-end, or otherwise altering aspects of user-facing UI/UX should be aware of the Webmaker Style Guide. The Style Guide provides important information about Webmaker's design philosophy and branding, typography, logos, colours, thumbnails, etc.
4. Learn the Webmaker Development Workflow
Every project has its own way of working, and we're no exception. This guide will help you navigate our development workflow so you can be successful as a new contributor.
TODO:
- learning/using git
- bugzilla and github
- code review
- pull requests, inline comments
- rebasing patches when they are ready to land
- flags/whiteboard things to know about (e.g., l10n string changes)
- getting r+'ed patches landed
- tagging
- pushing to staging/production
TODO -- old stuff below to integrate....
Want to report a bug?
Reporting a bug goes in Github or Lighthouse, depending on the project. Links follow:
Thimble - https://github.com/mozilla/webpagemaker/issues Open Badges (OBI) - https://github.com/mozilla/openbadges/issues Open Badger (OBr) - https://github.com/mozilla/openbadger/issues Collusion - https://github.com/mozilla/collusion/issues Webmaker.org:https://mozila-webmakers.lighthouseapp.com/dashboard webmaker.org (non-events) - https://mozila-webmakers.lighthouseapp.com/projects/95184-webmakers-phase-2/overview webmaker.org/events - https://mozila-webmakers.lighthouseapp.com/projects/94308-webmakers/overview Popcorn Maker:https://webmademovies.lighthouseapp.com/projects/65733-popcorn-maker/overview Popcorn.js:https://webmademovies.lighthouseapp.com/projects/63272-popcornjs/overview
Software
"Apps that teach." Mozilla Webmaker tools like Popcorn, Thimble and X-Ray Goggles help users a) make something amazing on the web, and b) bake in learning as they go. They're designed to teach digital literacy and web mechanics like HTML, CSS and Javascript through making and doing. Here's how you can get involved:
Popcorn
Mozilla Popcorn makes video pop. The Popcorn app and Javascript library make it easy for end-users and developers to make videos interactive, pulling in content from the rest of the web. More info here.
| More Info | |
| Code Repo | |
| Issue Tracking | |
| Primary Language/Framework | JS |
| IRC | irc.mozilla.org #popcorn |
| Mailing list | |
| Dev Cycles | Popcorn Maker: Monthly; Popcorn.js: bi-monthly |
| More on how to contribute |
Thimble
Thimble makes it ridiculously simple to create and share your own web pages: Write and edit HTML and CSS right in your browser, instantly preview your work and then host and share finished projects with a single click. More info here.
| More Info | |
| Code Repo | |
| Dev site | |
| Issue Tracking | |
| Primary Language/Framework | Python on the server, JS on the client |
| IRC | irc.mozilla.org #MoFoDev, #thimble |
| Mailing list | |
| Dev Cycles | TBD |
Open Badges
Learning today happens everywhere. But it's often difficult to get recognition for skills and achievements that happen online or out of school. Mozilla Open Badges helps solve that problem, making it easy for any organization to issue, manage and display digital badges across the web. More info here.
| More Info | |
| Code Repo | |
| Issue Tracking | |
| Primary Language/Framework | Node.js |
| IRC | irc.mozilla.org #badges, #MoFoDev |
| Mailing list | |
| Dev Cycles | Bi-monthly |
X-Ray Goggles
The X-Ray Goggles make it easy to see and mess around with the building blocks that make up the web. Activate the Goggles to inspect the code behind any web page, from the New York Times to your own blog. Then remix elements with a single click, swapping in your own text, images and more. More info here.
| More Info | |
| Code Repo | |
| Issue Tracking | |
| Primary Language/Framework | JS |
| IRC | irc.mozilla.org #MoFoDev |
| Mailing list |