Wikimedia Engineering/2014-15 Goals

Purpose of this document: Goals for the Wikimedia Engineering and Product Development department, fiscal year 2014–15 (July 1, 2014 – June 30, 2015). The goalsetting process owner in each section is the person responsible for coordinating completion of the section, in partnership with the team and relevant stakeholders.

These goals will be iterated on a quarterly basis. Each quarter, the top priorities for the department as a whole will be elucidated and highlighted.

Context: Our top objective for the April-June quarter is to progressively release VisualEditor on the English Wikipedia. This is a major software release with significant risks, and consistent with its importance and expected long term impact, we are declaring this as the only top priority for the quarter. Everyone in the engineering/product organization will be called upon to support it as needed.

Objective

Key result

Dependency

ETA

Status

Improve the editing experience for new and anonymous users on the English Wikipedia by giving them VisualEditor

Maintain VisualEditor production service at our quality criteria:

No loss/corruption of existing article data

No loss/corruption of the user's contribution

No security issues

No regression in performance

No regression in usability

Data Research team

User Research team

Analytics Engineering team

Parsing team

Services team

Community department

Team Practices Group

Communications department

June 2015

YDone

Run a test for new accounts on the English Wikipedia, providing VisualEditor by default to measure the impact.

April 2015

June 2015

YDone

[Contingent on previous result.]

Gradual ramp-up of VisualEditor availability on the English Wikipedia for new users, starting from 5% increasing to 100%.

Timing to follow from previous result.

Next quarter.

NDelayed

[Contingent on previous result.]

VisualEditor default availability on the English Wikipedia for anonymous users.

ToolLabs has at least 99.5% provable, measured uptime for each individual 'service' that ToolLabs provides its users

June 2015

YDone

Interdependencies:

Assist other teams in Q3 on:

Beta Labs improvements (RelEng)

Assist other teams in Q4 on:

Isolated CI instances (RelEng)

Dependencies:

Improvements to the Labs management web interface (MediaWiki OpenStack manager or possibly Horizon) will need involvement from the UX group, and potentially development assistent from a (MediaWiki?) development team.

Upload Wizard - refactoring and unit testing (and possibly stability and UX improvements enabled by that). Bugs encountered while undertaking this effort will be fixed, but might not be as high-profile as the ones we'll handle once we have JS error visibility. Tracking funnel metrics over time instead of only having a rolling snapshot.

JS error logging - getting it working on beta and on a small set of pages (including the Upload Wizard page) on production. The goal for the team is to get visibility on which main errors/bugs are responsible for the Upload Wizard funnel drops.

Platform - investigating a better solution for thumbnail storage.

Apr–Jun 2015

Objective

Key result

Dependency

ETA

Status

Make uploading media an easy, integrated process

Users can upload media to Commons by clicking upload or using drag-and-drop inside VisualEditor and the wikitext editor whilst they edit.

This quarter, the team is focusing on the feature requests coming from communities that have enabled or are planning to enable Flow on their wikis. This includes French WP, Catalan WP, Portuguese WP and Translatewiki, with more language partners coming up.

TheVisualEditorTeam is working to make VisualEditor a great editor for new and experienced editors alike, focusing on improving the performance and usability whilst adding some more features to make VisualEditor more helpful, intuitive and practical for use for every content edit, alongside maintaining, improving, and extending the existing editor software. Below are a set of goals that we hope to achieve over the next financial year (July 2014 – June 2015), with a balance of optimistic and pessimistic assumptions about speed of delivery; not all goals may be achieved in the time period indicated, and some may be updated over time.

On-going work happening every quarter:

Stability and bug fixing, prioritising any bugs that cause wikitext corruption or any other form of disruption for our wikis’ communities (including in the wikitext editor);

UX improvements tracked regularly and reported in a quarterly public user testing narrative;

Ensuring the success of VisualEditor on mobile, with a target of feature equivalence on tablet and at least some features on phone, with responsibility for both VisualEditor and wikitext editing pipelines on tablets and phones as well as desktop transitioning from Mobile to the team; and

Collaborating with volunteers and other Engineering teams like Parsoid, Services, Platform, & Core on related efforts like skin improvements, front-end performance, and other areas.

Quarter

Goals

Jul–Sep 2014

NPostponedDeployment: Engaging with English Wikipedia to discover pain points as part of agreeing the criteria for a gradual ramp-up of VisualEditor availability and usage to default, expected to happen in Q2, subject to community discussions (after this point, ongoing support)

YDoneCore: Internet Explorer 10+ browser support

Service deployed in production but client not createdFeature: Auto-filling citations from ISBN, DOI or URL

This fiscal year, the Mobile Apps team will be focused on getting data on feature usage of the new native mobile applications for iOS and Android in order to set goals and prioritize work on new apps features.

The Mobile Web team will split their time between two main focus areas:

Technical enablement – ensuring that other teams can work with MobileFrontend code, standardizing UI and code across desktop and mobile, and helping other teams design and build features that are responsive and work cross-platform.

Product work – this encompasses both a) porting select desktop experiences to tablets and handsets; and b) creating new features and contribution streams that help raise the number of new mobile users who hit the active editor threshold (5+ edits per month; below, referred to as new mobile active editors), so that we not only acquire but retain a healthy new editor population via mobile apps and web.

Our quantitative targets are based on the Growth team's editor model and focus on acquisition (getting more users to register for accounts via mobile), activation (getting more newly registered users to make 5+ edits in their first month), and retention (retaining new mobile active editors for 2+ months post registration). Note: targets for acquisition may change based on ongoing data collection around total edit numbers on tablets; if we observe a sustained drop in total contributions post tablet redirect, we may prioritize work on anonymous editing on mobile, which may change the number of signups on mobile.

The Mobile Web WikiGrok and Collections teams, including engineering, product, design/UX and support from Research and Analytics

WikiGrok: on track – first reader test live

Collections: renamed "Gather", due to launch a pilot by the end of March; scale expected to be limited to qualitative analysis initially.

Apr–Jun 2015

Objective

Key result

Dependency

ETA

Status

Allow users to create individualized collections of articles and share them with others by launching Gather on English Wikipedia, Mobile web

>10,000 creators/month rate

(subject to correction based on beta results)

Team Practices

Design

User Research

Analytics

Community Liaison

Communications

end of June 2015

Pending

>1000 shares per month rate

(subject to correction based on beta results)

end of June 2015

Pending

Interdependencies:

Editing team: We'll be working with the Editing team throughout the year to create a standardized editing experience (both wikitext and VisualEditor) across devices.

Flow: A MobileFrontend engineer with be embedded with the Flow team throughout Q1 to help build a mobile-first Flow experience.

Research & Data and User experience: We will be doing more in-person qualitative testing of new and existing mobile features to ensure good usability and a high-quality user experience. We will also run controlled tests and gather quantitative data on features to determine their effects on new and active editor numbers.

Analytics: We will need help from Analytics to continue monitoring and analyzing pageview traffic to determine the flow of readership to the desktop and mobile sites of all our projects.

Growth: Some of the mobile-specific contribution experiments planned for this year may dovetail with Growth experiments. Because we will be focusing on the same set of metrics/targets, the Growth and Mobile team should be working closely and sharing product thinking, experimental setup and overall strategy.

The Maps & Geo team will be a new team at the Wikimedia Foundation tasked with scaling our existing mapping resources to enable the world to visualize Wikipedia all around them. They will start the year forming the team, inventorying existing work, and scale our tile services and osm db to be production ready. Next they'll work with the Mobile Web and App team to empower our users to use mapping resources.

The success of this team will be determined by the success of the teams that leverage TPG's services. A guiding belief of TPG is that healthy teams build better products. It is the TPG's assumption that by supporting WMF engineering teams in becoming healthier and more sustainable, WMF engineering will be more successful as a whole. In addition to qualitative feedback, we can measure the TPG's impact by the success achieved by the individual teams engaged with the TPG - as measured by those teams meeting, or coming close to meeting, their goals. Further, the TPG can begin to validate its assumptions about team health impacting sustainability and product quality by first establishing a mechanism to measure and evaluate engineering team health.

Quarter

Goals

Jul–Sep 2014

Hire 1 scrummaster to work with the Mobile Web and Mobile Apps teams YDone

Note: This team was dissolved at the end of April, with its members and tasks being absorbed into other teams. With this qualification for the objective, we will still make an assessment at the end of the quarter on whether it was a success or miss.

Significantly contribute to the Editing team's work in avoiding or fixing problems in VisualEditor, and support communications by e.g. the Community Engagement and Communications teams regarding VisualEditor, including:

Run weekly tally of VE edits, e.g. to provide insight into how the frequency of problematic edits develops

Provide quantitative community analytics on VE users as needed

Provide qualitative analysis of VE usage, characterizing usage by 20 or more accounts, e.g. to support communications about VE

Anonymous editor signup invitations and basic signup workflow improvements released on desktop version of all top 10 Wikipedias or more. Stretch goal: collaboration with Mobile Web and Apps teams on mobile versions.

Oct–Dec 2014

Personalized task suggestions and notifications for new editors released as default on Wikipedias where onboarding (GettingStarted and GuidedTour) is present. Stretch goal: task suggestions and notification aimed at very active editors (100+ edits/month).

Jan–Mar 2015

Product roadmap for Q3 and Q4 to be determined based on mid-year targets data.

Apr–Jun 2015

End of year targets: by June 2015, we aim to have gained at least one of the following...

Acquisition: we will increase (compared to a control) new registrations across Wikimedia projects by 23%.

Activation: we will increase (compared to a control) the rate at which new registrations become active editors (5+ content edits/month) by 23%.

Individually, any of these conversion rate increases will lead to a full reversal of the year-over-year decline in total active editors. If all three targets are addressed, only a 7.6% increase in acquisition and activation are required, along with a 29% increase in retention.

Research & Data and User Experience: Both of these teams are core dependencies for Growth. Currently, the Research and Data embeds a full-time research scientist (Aaron Halfaker), and in order to support continued A/B testing capacity this will continue to be a key need on the team. Additionally, User Experience embeds a primary and secondary design on the team (currently Moiz Syed and Kaity Hammerstein).

Analytics: First and foremost, Growth depends on Analytics development for continued support of EventLogging, Wikimetrics, and analytics slave databases. In addition, Growth plans to use planned data products built by Analytics, such as an interest graph, to better attract new editors.

Community Engagement (Product): The Community Engagement group did not provide support for Growth during the 2013–14 fiscal year. We think providing some form of dedicated community liason will be critical if the team is to tackle more contentious areas of the user experience, such as article creation. Currently Product (Steven Walling) is filling this role on an as-needed basis, which puts the team primarily in a reactive mode, rather than having a proactive outreach plan.

Team Practices Group: The Growth team is small but is relatively inexperienced with Agile practices, and the new team practices group could provide extremely valuable support. Growth has implemented some Scrum practicies, but its capacity for process improvement is limited since the team has never had a dedicated scrum master.

Mobile Web: There is close alignment this coming year with the goals of the Mobile Web team and Growth. Mobile wants to spread its knowledge about mobile-first product development and engineering, while Growth wants to help develop mobile versions of its products.

Mobile Apps: As we introduce login, editing, and other contributory workflows to our native apps, Growth should work closely with the Apps team to make sure there is consistent experience, discover ways to integrate its findings in to the apps, and consider methods for attracting apps users to desktop products.