User:Edransch/BalrogReport
1.0 INTRODUCTION
The Release Engineering team at Mozilla is currently working on implementing a new piece of software to serve updates to Firefox and Thunderbird users. The aim with the new update server, code named Balrog, is to address the pain points in the current implementation of the update server (AUS2). One such area of weakness for AUS2 is the administration of the server and database to modify the updates that are served to Firefox users. AUS2 lacks a web interface, being controlled only via the command line and requires manual adjustment. Thus, Balrog’s administrative interface is an essential part of the project. The target audience of the administrative interface for Balrog is currently Mozilla’s Release Engineering team, but may be expanded in the future to include the Release Management team. There are many possible use cases for the administrative interface that need to be addressed. With limited resources available to work on the project, the Release Engineering team must evaluate new features carefully and allocate resources accordingly. This report examines some possible use cases and features for Balrog’s administrative interface and analyses their viability for inclusion in the project. The features considered are rollback functionality (single change and previous-state rollback), history view, editing JSON dictionaries (blobs), displaying the difference between (diffing) JSON blobs, help documentation, adding release manager access to Balrog, and visualizing how incoming requests are handled by rules. The main criteria used to analyze the above use cases and features are inclusion in existing goals statements, value of the feature, complexity of design and implementation. In this report, the amount of time required to implement a feature will be approximated by “small” meaning less than one week, “moderate” meaning one to two weeks, and “large” meaning more than two weeks. All estimations in this report assume approximately one developer is working full time on the project.
2.0 ANALYSIS
Since Balrog administrative interface is in early stages of development, several features are potential candidates for inclusion in the administrative interface. The use cases considered have all been raised by Release Engineers in team meetings as possible important use cases for Balrog’s administrative interface. Some of the examined features are critical to Release Engineerings Q2 goals while others are out of the scope of Balrog’s specification. Whether or not a feature falls under the Q2 goal, a feature’s overall usefulness, and the difficulty of design and implementation must all be considered when examining a feature.
2.1 Rollback Improvements
One important feature of the administrative interface is support for rolling back changes to the database. The simplest rollback functionality to implement is the ability to rollback a single change. The server side code for the first part has already been implemented. The client side code should be fairly straightforward to implement, taking a short amount of time. This feature has high priority as it is required for Balrog to have functional equivalence to AUS2. Thus, it falls under the Q2 goals list and should be completed by June. Since part of this feature is already implemented and functional, and the completion of the feature is a small amount of work, the target of Q2 is realistic and achievable. (and I should probably do it ASAP :P ) </p>
2.2 History View
Another potential use case for the Balrog administrative interface is to display a visual representation of the history of the database. As mentioned above, each change to each database table is stored in a separate history table. Currently, these history tables cannot be viewed in the administrative interface. A few distinct use cases are candidates for integration into the History view: viewing information on who made changes, rolling back an entire table to a previous state via the interface, and viewing the state of the database at a specific time. Information on who made changes is very useful when debugging problems with the server. If it seems like the problem is related to a specific change, the person who made the change can be consulted to help fix the problem. Providing the option to roll back to a previous state in the table will require additional database level code, which is a moderate amount of work. Finally, viewing the site at a specific state could help users verify that they have chosen the correct state before executing a rollback to that state. This last feature in particular requires substantial design and implementation. It would take a large amount of time to complete the design and implementation and should not be explicitly included in a goal for this year. However, viewing the page at a specific state could be implemented separately from the rest of the history view. The other features are more time efficient to implement and will likely take a moderate amount of work to implement. These features are of high value to the project, and should be implemented as soon as possible. They should be targeted at Q4 2012 or Q1 2013.
2.3 JSON Blobs
A third important use case for Balrog’s administrative interface is managing the JSON blobs that make up the ‘data’ column in the releases table. These blobs contain information about each platform and locale for a release. Unfortunately, they are difficult to edit by hand, since they are very large text files (approximately 500Kb per release). Additionally, it is nearly impossible to notice the differences between two blobs manually. Adding the ability to edit the blobs in an intelligent manner and to view a diff of the files offers a significant convenience gain for Balrog’s administrative interface. However, these features are both very complex. The ideal situation for these features would be that someone has already implemented them and released them under a licence compatible with Mozilla’s. One such implementation for editing JSON is found at jsoneditoronline.com, with source code released under the Apache2.0 licence at https://github.com/wjosdejong/jsoneditoronline. This tool offers a simple and intuitive tree-based interface for editing JSON. Integrating the tool into Balrog’s current code would be a moderate amount of work. Unfortunately, a web based JSON diff tool of similar polish does not exist. In order for Balrog to move forward, the best solution is to rely on other diff tools for comparing blobs. If an html based diff tool is released in the future (as a Google Summer of Code project, for example), it could be integrated into Balrog. Users can currently download and upload blobs as needed, allowing them to edit blobs in their preferred software on their local machine. Thus, these two features are for convenience only and do not provide any missing functionality. Since neither of these two features is required for the Q2 goal, the integration of the JSON editor should be targeted at Q4 2012, and the JSON diff tool should be monitored and integrated when a suitable tool is available.
2.4 Help Documentation
An additional important feature of the Balrog administrative interface is the inclusion of help documentation in the web page. This feature has very high priority as Mozilla’s operations scale, since it will help new users gain familiarity with the administrative interface. Additionally, if adding documentation is postponed and new features that are added to the interface before help documentation is introduced, upfront cost of adding documentation will be larger. Therefore, there is significant benefit to implementing help documentation as soon as possible. However, help documentation is not required for Balrog to have functional equivalence with AUS2, so it is not necessary to include it in the Q2 goals. There are some design decisions that need to be made here regarding whether to make the help available on a separate page, or via mouseover tooltips, or separate sections of the same page. These decisions will affect the complexity of the implementation of the feature. Integrating help into the web pages themselves will likely be a large amount of work, with mouseover tooltips being the most complex. Given the complexity of the implementation and the high priority of adding help documentation, this feature should be implemented in 2012, either in Q3 or Q4.
2.5 Release Manager Access
Another use case for Balrog’s administrative interface would be to provide limited access to the interface for the Release Management team. The administrative interface is designed to simplify the release process and if the Release Management team is able to use some of the features of the administrative interface, the workload on Release Engineering is reduced. This feature has moderate priority, Balrog already reduces the workload required my Release Engineering dramatically, but it remains important to make as many improvements as possible. Designing this interface requires a meeting with the Release Management team to assess which features they would require access to. It is most likely that we can simply restrict access to some of the features of the administrative interface for Release Managers. A full separate interface is not required. Additionally, this feature relies on having thorough help documentation since it will introduce new users to Balrog’s administrative interface. Since there is interaction and iteration with another team required for this feature, it will require a large amount of time to implement. It should be aimed at early 2013, perhaps at Q1.
2.6 Rules Visualization
A final use case of the Balrog administrative interface is the visualization of the rule set. The rules table defines the set of rules that determine which update to server to each incoming request based on characteristics of the request. When multiple rules apply to an incoming request, they are sorted by priority, with the highest priority rule being applied to the request. If the rules set grows to more than approximately 20 rules, it becomes difficult to see by inspection which rule will apply to an incoming request. A tool to visualize the rules table and its application to requests would be extremely helpful for the Release Engineering team to manage the rules table and make necessary adjustments. One option for visualizing the rules table is to visualize the rules as ‘buckets’ catching incoming requests that travel vertically down the page. This implementation is outside of the scope of the Balrog project. It may be possible to open the problem up to the community and have a member of the community volunteer their time to implement the tool to provide such visualizations. Since visualizing the rules is outside of the scope of Balrog, a simple web page to test incoming requests should be constructed. This page would offer the request parameters as text input fields, and would show in real time, which rule applies to the request as it is filled in by the user. This would be sufficient for testing individual requests, and will be helpful in debugging which rules need to be changed if a certain set of requests is providing incorrect information. This simple rules query functionality would be a short or moderate amount of time to implement and should be targeted at Q4 2012. The full visualization of the rules view is outside of the scope of Balrog and should only be included if it can be implemented by a community member or other party.
3.0 CONCLUSIONS
Some essential features to Balrog’s administrative interface can be implemented as part of the Q2 goal, others cannot be implemented before this deadline.
It is important to consider the scope and purpose of Balrog’s administrative interface in order to prevent scope creep and ensure the project can move forward.
Some functionality for Balrog’s administrative interface would be useful, but are outside the resources of Mozilla’s Release Engineering team. Leveraging community contributors as well as other open source projects would be extremely valuable.
4.0 RECOMMENDATIONS
Adding interface support for rolling back a single change is an essential feature for Balrog’s administrative interface to be functionally equivalent to AUS2 and should be implemented as part of Release Engineering’s Q2 goal. Displaying a history view for Balrog’s tables opens up possibilities for multiple features. Allowing the user to view the site at a specified state of the history table is out of the immediate scope of Balrog’s administrative interface and should not be prioritized. However, a simple change list view and support for rolling back to a specific state in the database are valuable use cases that should be implemented by Q1 2013.
Blob editing and diffing are problems that are out of the scope of Balrog’s administrative interface. However, existing tools such as jsoneditoronline can be integrated in to Balrog to provide the necessary functionality. Integrating jsoneditoronline is major convenience gain for Balrog administrators and should therefore be targeted at Q4 2012.
Adding release manager access is dependent on implementing help documentation. The two features can be developed in parallel and are very important for long term scaling of Mozilla’s release process. However, they are large projects and will require interaction with the Release Management team. The time required to complete them is outside of the time available in 2012. These projects should not be implemented in 2012, but rather should be scheduled for early 2013.
Visualizing the rules in a separate view as a graph is outside of the scope of Balrog’s administrative interface. A simpler view for checking which rules apply to a given incoming request should be implemented. This simpler rules view should be implemented by Q4 2012.