Webmaker/Code/Dev/Environment

Set up a Webmaker Development Environment

Before you can test or make improvements to Webmaker, you need to get it running locally. This involves setting up your OS to host the Webmaker servers and apps, and installing various development tools you'll need. We strongly recommend a Unix-like OS, such as Linux or OS X, but it is also possible to use Windows.

Option 1: Installing Webmaker Locally

Webmaker can be configured locally to allow for development and testing. This process can be accomplished in a number of ways, but they all require all dependencies to be working before they can succeed.

External Dependencies

Webmaker relies on a number of technologies and tools, which must first be installed. The following list needs to be installed and working before you can run or test Webmaker locally, and we provide instructions for doing so on Linux, Mac OS X and Windows below.

• node.js and npm
• bower:
• grunt:
• python 2.7
• pip
• MongoDB (NOTE: there is some discussion about dropping MongoDB altogether, but at the time of writing, this hasn't happened yet)
• Java
• Elastic Search
• A working C/C++ toolchain
• A stable build environment for your OS

Setting up your dev environment in Mac OS X

This is the easiest method of setting up a Webmaker dev environment in Mac OS X, but requires the use of the Homebrew package manager.

1. Install Xcode command line tools by running xcode-select --install from the terminal
2. Install the Homebrew package manager by following the instructions at their website.
3. Run brew doctor to complete Homebrew's installation
4. Install NodeJs with brew install node
5. Install the JDK (Java Development Kit) from their website
6. Install elasticsearch with brew install elasticsearch
7. Install mongodb with brew install mongodb

Setting up your dev environment in Ubuntu Linux

This is the easiest way to setup a Webmaker dev environment on Ubuntu Linux.

1. Install git and a C/C++ toolchain with sudo apt-get -y -q install git g++ make pkg-config
2. Install the Java Development Kit with sudo apt-get -y -q install openjdk-7-jre-headless
3. Install the MongoDB server
   1. Install the package with sudo apt-get -y -q install mongodb-server
   2. Ensure MongoDB is running with service mongodb status
   3. Otherwise, start MongoDB with sudo service mongodb start
4. Install Elasticsearch
   1. Download Elasticsearch as a .deb file and install from the software centre
   2. Ensure Elasticsearch has started with service elasticsearch status
   3. Otherwise, start Elasticsearch with sudo service elasticsearch start
5. Install Nodejs from Ubuntu PPA
   1. Add the Node.js PPA using sudo add-apt-repository ppa:chris-lea/node.js
   2. Update your packages using sudo apt-get update
   3. Install Node.js using sudo apt-get install nodejs

Setting up your dev environment in Fedora Linux

1. Install git and a C/C++ toolchain with sudo yum install -y git gcc-c++ make pkgconfig
2. Install the Java Development Kit with sudo yum install -y java-1.7.0-openjdk-headless
3. Install the MongoDB server
   1. Install the package with sudo yum install -y mongodb-server
4. Install Elasticsearch
   1. Download Elasticsearch as a .rpm file and install it with sudo rpm -i elasticsearch*.rpm
   2. You may need to run a few commands:
      1. sudo /bin/systemctl daemon-reload
      2. sudo /bin/systemctl enable elasticsearch.service
   3. You can then start it with sudo /bin/systemctl start elasticsearch.service
5. Install Nodejs with sudo yum install -y nodejs npm

Setting up your dev environment in Windows 7

Our Webmaker-suite npm module has excellent documentation on preparing a Windows development environment. You can find it here.

NodeJS dependencies

If you followed the Windows-specific external-dependency instructions, this step has already been completed. Otherwise, once your Development environment is set up, you must install some global NPM packages.

1. Install bower with sudo npm install -g bower
2. Install grunt with sudo npm install -g grunt
3. Install grunt-cli with sudo npm install -g grunt-cli

Installing Webmaker Option 1: Webmaker-suite

By far the easiest way to get all of Webmaker set up and installed locally, on any platform, is to use Webmaker Suite. This is a set of automation scripts written in node.js by one of our lead developers, and is meant to provide turn-key installation by automatically downloading all the code, and setting default environment variables.

Complete instructions are available in the README for Webmaker Suite on github.

Installing Webmaker Option 2: Manual Installation

Each server that makes up a part of the Webmaker.org ecosystem is configured with .env or environment files. The servers come with a sample version of this file that is fully functional, assuming that the sample files are used for all the servers in the local installation. Instructions on configuring these manually can be found in the Github repository of each server.

Installing Webmaker Core

Webmaker's core apps are the Login, MakeAPI and Webmaker.org servers. These three must be run together in order to function properly with each other, and the other apps that make up Webmaker. You can find installation instructions in the readme file in each server's Github repository:

• Login - https://github.com/mozilla/login.webmaker.org/blob/master/README.md
• MakeAPI - https://github.com/mozilla/MakeAPI/blob/master/README.md
• Webmaker.org - https://github.com/mozilla/webmaker.org/blob/master/README.md

Installing Webmaker Tools

Webmaker's tools (Thimble, Goggles and Popcorn Maker) follow similar procedures for installation and configuration. Keep in mind that they won't function properly without the Webmaker Core services running. You can find installation instructions in the readme file in each server's Github repository:

• Popcorn Maker - https://github.com/mozilla/popcorn.webmaker.org/blob/master/README.md
• Thimble - https://github.com/mozilla/thimble.webmaker.org/blob/master/README.md
• Goggles - https://github.com/mozilla/goggles.webmaker.org/blob/master/README.md

Staying Up-to-date

As we improve features (day-to-day), we release them into the wild almost immediately! For this reason, it's important to keep your servers running the latest code.

Webmaker-suite

Follow the update instructions listed in the Webmaker-suite README section on the topic.

Manual installation

Each server will need to be updated separately, though the process is identical.

These commands use git to pull the latest updates from the Mozilla repo, and then reinstall the node modules (or libraries). If you followed each server's installation instructions exactly, [mozilla upstream remote] should be replaced with origin. Otherwise, replace it with whatever you labeled the Mozilla git remote.

• Dirty update (try this first): git checkout master && git pull [mozilla upstream remote] master && npm install
• Clean update (time intensive) git checkout master && git pull [mozilla upstream remote] master && rm -rf node_modules && npm cache clear && npm install

Option 2: Installing Webmaker on Heroku

TODO

Option 3: Installing Webmaker with Vagrant

NOTE: This is a work in progress, and may have bugs. See https://github.com/mozilla/webmaker-suite for more info on using webmaker-suite itself. You will need a 64bit OS for vagrant to run.

1. Download and install Virtualbox and Vagrant.
2. Download the Vagrantfile from github into a folder where you want to run your webmaker-suite VM from (ie, /Users/myuser/vagrant/webmaker-suite).
3. Open a terminal, cd into that folder, and issue the command vagrant up. This will download the current version box (an Ubuntu-12.04-x86_64 1.7gb image) and may take a while.
4. Once the building is complete, issue the command vagrant ssh to ssh in your new vagrant box.
5. Now in vagrant, issue the command ./webmaker-suite.sh. This will start mongo, elasticsearch, webmaker, popcorn, thimble, makeapi, goggles, etc.

After it completes, point your web browser to:

• http://127.0.0.1:3000 - login.webmaker.org
• http://127.0.0.1:3500 - thimble.webmaker.org
• http://127.0.0.1:5000 - makeapi
• http://127.0.0.1:5050 - htmlsanitizer.org
• http://127.0.0.1:7777 - webmaker.org
• http://127.0.0.1:8888 - popcorn.webmaker.org
• http://127.0.0.1:12416 - goggles.webmaker.org

More detailed information available here.

Example Output:

localmac-23:24:58-imagetest# vagrant up
Bringing machine 'default' up with 'virtualbox' provider...
[default] Box 'webmakersuitev3test' was not found. Fetching box from specified URL for
the provider 'virtualbox'. Note that if the URL does not have
a box for this provider, you should interrupt Vagrant now and add
the box yourself. Otherwise Vagrant will attempt to download the
full box prior to discovering this error.
Downloading or copying the box...
<1.7gb of downloading occurs here>
Successfully added box 'webmakersuitev3test' with provider 'virtualbox'!
[default] Importing base box 'webmakersuitev3test'...
[default] Matching MAC address for NAT networking...
[default] Setting the name of the VM...
[default] Clearing any previously set forwarded ports...
[default] Creating shared folders metadata...
[default] Clearing any previously set network interfaces...
[default] Preparing network interfaces based on configuration...
[default] Forwarding ports...
[default] -- 22 => 2222 (adapter 1)
[default] -- 3000 => 3000 (adapter 1)
[default] -- 3500 => 3500 (adapter 1)
[default] -- 5000 => 5000 (adapter 1)
[default] -- 5050 => 5050 (adapter 1)
[default] -- 7777 => 7777 (adapter 1)
[default] -- 8888 => 8888 (adapter 1)
[default] -- 12416 => 12416 (adapter 1)
[default] Booting VM...
[default] Waiting for machine to boot. This may take a few minutes...
[default] Machine booted and ready!
[default] Mounting shared folders...
[default] -- /vagrant
localmac-23:42:19-imagetest# vagrant ssh
Welcome to Ubuntu 12.04 LTS (GNU/Linux 3.2.0-23-generic x86_64)
 * Documentation:  https://help.ubuntu.com/
Welcome to your Vagrant-built virtual machine.
Last login: Fri Jan 17 20:52:50 2014 from 10.0.2.2
vagrant@precise64:~$ ./webmaker-suite.sh