Talk pages project/Notifications

Wikimedia Foundation projects Talk pages Know when someone is saying something relevant to you. Group: Editing Start: 2021-01-01 Team members: Bartosz Dziewoński, David Chan, David Lynch, Marielle Volz, Rummana Yasmeen, Jess Klein, Sherry Snyder Lead: Ed Sanders Management: Marcella Florence (engineering), Leza Zaman (program), Peter Pelberg (product)

Tracked in Phabricator
Task T2738

This page talks about the Editing Team's work to improve the notifications editors receive for the wikitext talk page conversations they are interested in.

This initiative sits within the Talk pages project. This is the team's larger effort to help contributors work together more effectively. To accomplish this, we are building upon existing community conventions to evolve wikitext talk pages. It is our intention to evolve talk pages in a way that gives experienced contributors more leverage to coordinate their work and connect with other editors, while making communicating on-wiki legible and intuitive for newer contributors.

To participate in and follow this project's development, we recommend adding this page to your watchlist.  We will use this page to:

• Share and invite feedback on designs
• Announce deployment plans
• Share data about how the feature is being used

Status updates

This section contains updates about the project's development.

28 June 2021

Today, 28 June 2021, Manual Topic Subscriptions became available as a Beta Feature to volunteers at the projects listed below. We will spend the next several weeks monitoring the feature to ensure it is working as people expect.

In the meantime, if you would like to see manual topic subscriptions offered as a beta feature at your project, please leave a message on the talk page.

Projects where manual topic subscriptions are now available:

• Wikipedias: Arabic, Czech, French, Hungarian, and Dutch
• Meta
• MediaWiki

3 June 2021

Current appearance

The team recently finished usability testing with Junior and Senior Contributors to identify areas where the Topic Subscription prototype did not meed their needs and expectations.

In short, Junior and Senior Contributor test participants found the Topic Subscription prototype to work as they intuitively thought it would/should. You can review the usability test findings in detail below: Usability testing.

These results are leading the Editing Team to feel confident offering Topic Subscriptions to a wider audience by deploying it as a beta feature at the projects listed below in the coming weeks.

• Wikipedias: Arabic, Czech, French, Hungarian, and Dutch
• Meta
• MediaWiki

23 April 2021

Decisions

A screenshot showing a mockup of the topic subscription affordance.

Over the past couple of weeks, we have made the following decisions about the first version of the topic subscriptions feature:

• The affordance for Subscribing / Unsubscribing to/from topics will styled in the same way as the page's existing [ edit ] links and appear next to them.
• To start, Subscribing / Unsubscribing to/from topics and Watching / Unwatching pages will be unrelated. The thinking for how we arrived at this initial decision can be found in the "Open questions" section of T279498's task description.

Looking ahead

• In the next couple of weeks, we will run a series of usability tests with Junior Contributors using usertesting.com to learn whether they can confidently and intuitively use the topic subscription prototype to become aware of new comments posted in conversations they are interested in.

1 April 2021

Topic subscription prototype is ready

A screenshot showing the topic subscription prototype.

An in-progress prototype of the new topic subscription feature is ready for you to try.

This prototype enables you to elect to receive a notification via Echo when someone posts a new comment in any conversation you have decided to "subscribe" to.

To try the prototype, please follow the instructions on the talk page: Topic subscription prototype instructions.

26 March 2021

Topic subscription prototype

Next week, the team will have a technical prototype of the new topic subscription feature ready for testing.

This prototype will enable you to elect to receive a notification via Echo when someone posts a new comment in any conversation you have decided to subscribe to.

We will post on the talk page when the prototype is ready for testing.

Special thanks to people like Enterprisey, GhostInTheMachine, Jack who built the house, Matěj Suchánek, Pelagic Sdkb, Timeshifter, and Vriullop whose feedback informed the decisions we made in building this initial version.

5 February 2021

First intervention

The first intervention we will be working on to increase the likelihood Junior and Senior Contributors receive timely and relevant responses to the things they say will be a feature that enables people to subscribe to be notified about new comments in specific talk page topics.

The team has started the design and technical research that will be needed to offer this functionality.  Next week, you can expect another update where we will share the requirements that will guide the first iteration of the topic subscriptions feature.

In the meantime, you can learn more about why we are prioritizing work on this particular intervention in the Background section below.

Note: we appreciate being able to more easily follow activity in specific talk page sections has been on peoples' minds for some time. If there are conversations, gadgets, research, etc. that you think would be valuable us to consider as part of this work, please boldly add it to the History section below.

20 November 2020

Project start

The Editing Team is currently in the planning and technical exploration phase of this project. We plan to focus on the user-facing components of the implementation during the first three months of 2021.

In the time between now and then, you can follow the research we are doing for this project in Phabricator here: phab:T233447.

Objectives

This work is intended to increase the likelihood Junior and Senior Contributors receive timely and relevant responses to the things they say.

Where "the things they say" in this context means, the comments they post and conversations they start on wikitext talk pages, regardless of the tool used to publish these comments and conversations.

Once defined, you will be able to read about the key metrics the team will use to evaluate the extent to which the enhancements made as part of this project have been successful in achieving this impact.

Evaluating impact

This section will contain the methods and data we will use to evaluate the impact of this feature.

Background

The Editing Team is committed to evolving wikitext talk pages in a way that gives experienced contributors more leverage to coordinate their work and connect with other editors, while making the same communication and coordination practices and capabilities legible and intuitive for newer contributors.

A key part of these "communication and coordination practices" is people knowing when others are saying something relevant to you. Where "relevant" in this context could mean someone is talking to you directly or saying something in a conversation you have expressed an interest in.

The trouble is, people across experience levels find the existing notification system not as effective as they and we think it could be at making them aware when someone says something relevant to them.

Specifically, usability testing, the Talk Page Consultation, Community Wishlist Surveys (2015, 2021) and academic research has demonstrated people, across experience levels, find wikitext talk pages do not offer them a broad enough range of activity they can elect to be notified about. People also mention finding it difficult to understand how and where existing notifications are delivered.

"...pertinent changes to sections that interest me get superseded by subsequent changes to other sections."

—2015 Community Wishlist Survey

"Sometimes the movement of the terrace is very large, and times when only one or two topics account for almost all editions on the terrace. When this happens editions in other topics may go unnoticed among others, then we don't even see that there are new comments, sometimes we don't even see that there is a new topic."

—Machine translated from English Wikipedia and Portuguese Wikipedia discussion

In offering people a greater range of talk page activity they can subscribe to and improving the ways in which notifications about this activity are delivered to people, we are striving to increase the likelihood Junior and Senior Contributors know when someone is talking to them and for the person who is talking to them to receive timely and relevant responses to what they are saying.

Design

This section will contains information about the design strategy.

Deployment

This section will contain information about how and where the Notification enhancements are deployed.

Usability testing

This section contains information about the usability tests the team has run to identify ways in which the Topic Subscription feature could be improved to meet Junior and Senior Contributors' needs and expectations.

Test goals

The usability test we ran with Junior Contributors had two primary goals:

1. Evaluate their ability to intuitively know what to do to receive notifications about new comments posted in conversations they were interested in
2. Identify areas within the user experience where Junior Contributors' needs and expectations were not met

The usability test we ran with Senior Contributors had two primary goals:

1. Verify Topic Subscription tool complimented Senior Contributors' existing workflows
2. Identify areas within the user experience where Senior Contributors' needs and expectations were not met

Testing Conclusions

Both Junior and Senior Contributor test participants found the Topic Subscription prototype to work as they intuitively thought it would/should.

While test participants identified areas where the experience could be enhanced (more in "Detailed findings below"), nothing surfaced that caused the team to question the prototype in fundamental ways.

The majority of test participants were able to:

• Subscribe to a conversation
• Locate and understand the notifications they received when a new comment was posted in a section they had subscribed to
• Unsubscribe from a conversation they no longer wanted to be notified about

Testing Methods

The usability test we ran with Junior Contributors was run on user on usertesting.com (see test protocol) and included people who had used Wikipedia before and who have, at a minimum, edited once before. Test participants were asked to complete a set of pre-defined tasks with the prototype and to narrate what they thought and felt as they completed said tasks.

The usability test we ran with Senior Contributors was run on mediawiki.org, Arabic Wikipedia, and Czech Wikipedia. The test was open to anyone who saw the announcements made at these three wikis. Test participants were asked to experiment with the feature as they saw fit and to share their experiences in writing, on-wiki.

Detailed findings

Senior contributors noted the following potential areas to improve (T275232):

• Revise the appearance of the Subscribe affordance so topics are easier to distinguish: T279149.
• Introduce the ability to subscribe to all conversations on a talk page: T284795
• Introduce the ability for people to customize the kinds of edits they are notified about: T284794
• Make it easier to identify a new comment you have been notified about, on the talk page where said comment had been posted: T282029.
• Clarify the distinction between the two places – "Alerts" and "Notices" – where people can receive notifications (Junior Contributors noted this as well): T142981.

Junior Contributors noted the following potential areas to improve (T281438):

• Changing how comments are visually presented within conversations so they are easier to read, relate, and understand: T282269.
• Revise the appearance of the Subscribe affordance so it is easier to identify on the page: T282261.
• Introduce a way to view and manage all of the topics you are subscribed to: T273342.

History

Many projects, conversations and research have, and are, working to improve contributors' experiences with talk pages. This project is better off for their existence. Some of the projects the team continues to learn from are listed on the main project page and below. If there is a project you think we should be aware of, please boldly add it here.

• User:Enterprisey's Section Watchlists
• pt:English Wikipedia and Portuguese Wikipedia discussion forums (1set2012)
• en:Wikipedia:Village pump (proposals)/Archive 93#Support or oppose a test Village Pump based on Portuguese VP
• Community Wishlist Survey 2015/Watchlists#Section watchlists
• Community Wishlist Survey 2016/Categories/Watchlists#CW2016-R043
• Convenient Discussions
• reply-link
• Flow satisfaction survey
• Moderated Testing, November 2014: talk pages and Flow
• Talk pages consultation
• Wikipedia:Global, cross-wiki, integrated or stacked watchlists. - Interwiki talk page notification.

Glossary

The Talk pages project glossary is intended to help us all communicate about talk pages more effectively by making sure we have a shared understanding about the words we use in our discussions and documentation throughout the project.