Topic on Talk:Talk pages project

Forked thread of other discussion.

Link: Prototype of first version.
Note: Visual Editor is not part of the first production version. The second version is supposed to have some WYSIWYG editor, that's faster and more accurate than the current VisualEditor.

Flow blew up for fundamentally the same reason the the 2017WikitextEditor project blew up, and the same reason the SingleEditTab project went boom. They all went boom because of a problematic obsession with a mostly-insignificant secondary editor that almost no editors want use. They all tried to force the wiki over to VE in various ways.

@PPelberg (WMF), can we please please please agree on the following points?

1. Please confirm the default editor will be wikitext. There may be an option to switch to VE as a secondary editor. (I believe this has already been resolved on Phabricator, but I include it for completeness.)
2. Please confirm the team will not try to use VE's "wikitext mode", also known as 2017WikitextEditor, as the Wikitext editor. The community overwhelmingly rejected deploying VE's wikitext mode as a wikitext editor for article space, and we don't want this jeopardizing the talk project.
3. Please confirm the team will not try to use VE or VE's parser (parsoid) for previews. Previews must use the genuine wikitext parser. If you check the link above, this is one of the reasons the 2017WikitextEditor project blew up.
4. Please close Phab task T238218, which proposes running everything through VE's parser.

Previous discussion with different, justified POVs: Topic:Vbkgcy6e8cuz07eq

Please use a less click-bait title and a more neutral approach to discuss your concerns. "Flow blew up", ... "went boom". If something blew up, it was the community. Flow was rejected is the non-dramatic wording.

Please think about feasible solutions and take into consideration the arguments that differ from your perception of VE instead of creating drama. There are less and less long-time editors each day; giving importance only to their attachment to the wikitext editor would not help with the editor decline, nor to achieve the movement's mid-term targets.

The primary request from new users is an easy-to-use wysiwyg editor. Wikitext is the opposite of that. If VE is too slow, then the solution is to create a new editor from scratch. Reddit managed to create such editor, so it is possible.

From a UX perspective: new users won't be looking for the switch to turn on the wysiwyg editor, whereas experienced users will find with a few clicks the setting to turn it off. That setting won't endanger a project. Unnecessary drama, again.

There is a confusing statement in the topic opening comment. Please clarify this before reposting again: How phab:T238218 (Control whitespace in injected wikitext for multi-line comments) "proposes running everything through VE's parser"?

There are some other problems I have with this comment, notably that you are under the mis-apprehension that the old wikitext parser will continue to exist past a certain date.

It will not. You will need to come to grips to that. That fact is part of the reason that Extension:Linter exists. Continuing to build for the old parser is more or less not a reasonable ask.

2017 WTE did not blow up. There were reasonable problems discussed and voiced in that RFC. It is faster now since then, and I imagine with Parsoid/PHP will be faster still on the server side. (Ssastry has put the status at 30-50% faster parsing there.) I use it today. (Consider that Flow uses it now--you're not writing in the old WTE when you're here on MW.org talk spaces.)

As I said then in that discussion, previews differing is an issue, but it is an issue that needs to be resolved rather than complained about. If you have specific problems with the previews generated in 2017 WTE, you need to file tasks for resolution.

There were other notable objections to this original post in the other discussion, but I won't repeat them here.

I agree the title is click-bait, but VE and Flow could endanger the project.

1. Flow (used here) is not suitable for discussions of Wikipedia edits (without the option of at least 2 WikiText sections per post.) It's also a reason I am not commenting here often.
2. VE is not really suitable for editing talk pages with discussions of how to correctly edit Wikipedia articles. We (or, at least, I) cannot explain how to edit correctly using VE. Most of the time I've added links using VE, I've made a mistake. In en.Wikipedia, when describing Wikitext, I can include the sections in error using < nowiki> tags, so that anyone (even using VE) can see the differences, even if they cannot generate them.
3. I have no objection to changing the Wikitext editor to 2017 WTE, after the bugs are fixed and (most) pages which have too many errors to be edited with the 2017 WTE are fixed. However, what is important is that the actual editable source be in Wikitext (even if, sometimes, the Wikitext saved is different than the Wikitext entered); the same changes to Wikitext produce the same changes in the display, etc. It would be nice, actually, if the saved Wikitext was corrected if formatting errors occured; however '''Bold''BI'''Italics'' probably renders "correctly", but cannot be done in straight, clean Wikitext. Actually, that's a good job: '''Bold''BI'''''< /nowiki>''Italics''?

[Edited to correct my misunderstanding of Parsoid, and to modify point 3.]

[Edited an additional time to show how my example actually got translated into acceptable Wikitext.]

Ping @PPelberg (WMF) again. It's been a week, can I get info on the product plan?

1. Will the initial/default be wikitext or VE?
2. Is the wikitext mode going to be provided via VE's "wikitext mode", known as the 2017WikitextEditor?
3. Are previews going to be provided via VE's parser, parsoid?
4. Is it going to use VE's parser, parsoid, for saving as indicated in Phab task T238218?
5. Aside from perhaps offering access to VE as a secondary editor, will the product be built on VE or VE's parser in any other way?

@Alsee, responses to your comments below. For context, I assume you are asking these questions in the context of the workflow for Replying to specific comments.

In version 1.0 of the replying workflow, contributors will be presented with a plain text input to compose their replies. [1]  For some of the reasons @Aron Manning mentions here (thank you, Aron), in Version 2.0 we plan to test having a rich-text editing mode available as the first experience contributors encounter when drafting a reply using this new replying workflow.

As to the other questions you raised...

The team has not yet come to firm decisions on these questions yet. I understand they come from a place of wanting to make sure the tools we build work in ways contributors expect. So if there are particular annoyances/peculiarities you experience that get in the way of your talk page contributions (I appreciate you raising this point in your comment,@Izno), please let us know so we can address them.

---

1.  We do not currently have plans to affect "full" talk page editing.

@PPelberg (WMF) You referred to understanding the place I am coming from. I would like to define the place I am coming from.

I've been happy and impressed with the new approach the Foundation took in the Talk page consultation. It started with a clean slate, and the Foundation really worked at listening-to and understanding the community. I also watched a video (maybe wikimania?) where you were one of the speakers. It also came across very positive, that you were really trying to work with the community and give us a product we wanted.

Where I am coming from is dismay that this project appears to be falling back into some familiar old patterns. Where I am coming from is dismay that this may be yet another failed project. I'd like to avoid that.

I asked whether the initial/default be wikitext or VE, and whether that would be VE's wikitext mode. You responded "rich-text editing mode [] as the first experience"'. I believe that was an awkward alternate way to say "yes you intend to make VE or VE's-wikitext mode the default". Please let me know if I misunderstood you.

Instead of getting into an argument or debate, I am going to attempt to list things I hope(?) we can agree on.

• For close to a decade, the general Foundation position has been saying it is beneficial to put new users into VE by default.
• For close to a decade, community consensus has been saying you're wrong, that it's harmful to put new users into VE by default.
• Continuing that debate on this page is a waste of time.
• The Foundation thought it would be beneficial to make VE the default, even for wikitext editing. (2017wikitext editor)
• Community consensus thought that was harmful, for reasons including but not necessarily limited to those listed in the RFC.
• The Foundation thought it would be beneficial to default new users into VE, which was being done during the attempted SingleEditTab rollout.
• Multiple communities, including EnWiki, thought that was harmful for new users. They wrote hacks for the sitewide javascript to remove or reverse that VE-default.
• The community was acutely aware that hacking the sitewide javascript risked repeating or escalating beyond the Superprotect crisis, and the community did not hesitate to go there if that's what it took to protect our new users.
• The Superprotect crisis ultimately resulted in the community replacing every elected member of the Board of Trustees, the elected members then replacing every appointed member of the Board, and the new board then replacing the Executive Director.

And now the key point I'm hoping we can agree on:

• The community will consider a VE-design or VE-default for this project to be harmful for new users, the community will passionately defend the best interests of new users (as understood by the community), escalating up to and beyond the Superprotect crisis if necessary.

What I'm hoping for is acknowledgement that you basically have no choice but to choose from the following list of options:

1. The Present Your Case strategy: Attempt to convince the community that a VE-based-design or VE-default is a good idea, acknowledging that the VE-approach is conditional on actually getting people on board. Reminder: Debating that issue on this page is a waste of time.
2. The Flow strategy: Even before the prototype is built you know the community isn't on board with your plan, ignore the issue, sink massive time and money into the project, and act surprised when things go up in flames later.
3. The Build What The Customer Wants strategy: Accept that trying to forcibly impose VE as part of this project would place this project in serious jeopardy of failure, and simply don't take that approach. Let everyone peacefully co-exist with an easy click to VE as a secondary editor. VE can be made the default if and when VE actually becomes the majority choice for editing.

@PPelberg (WMF) I just tested the initial prototype.

I am begging you, please let me help you. Please.

On one hand any random yahoo can make an account, and if you think that random yahoo is spouting fringe garbage then it is perfectly appropriate for staff to blow them off. The magic words are "show me a consensus".

On the other hand I am one of those "community leaders" the Foundation likes to refer to in the abstract. As you saw, I ran the RFC on the 2017editor. I also personally initiated the deletion of Flow pages on EnWiki. I personally negotiated the deinstallation of Flow from EnWiki. I didn't open the Meta RFC on Flow, but I shaped and oversaw the resolution. And when the Foundation dismissed the rejection of Flow by EnWiki and Meta, when the Foundation disregarded the results of it's own biased Flow-survey, I personally opened the RFC to uninstall Flow from Commons. They all got consensus. To avoid a posting a wall of text, I'll just say I've been a key "community leader" regarding other projects as well.

It will needlessly escalate drama if the only way for me to help your team is to come back with an official consensus. One area where the Talk consultation failed is that the Foundation never really understood or accepted why Flow failed. If you don't know why Flow failed, you aren't going to recognize when you are repeating the same mistakes. When I show the community the prototype results, when I show that your project is repeating the exact same catastrophic design flaws as Flow, the "Flow clone" label will stick. It will stench. They won't be interested in extending you good faith, even if you do try to fix it.

@Alsee:

If you don't know why Flow failed, you aren't going to recognize when you are repeating the same mistakes.

There are different explanations of why Flow failed. Why did it fail in your view and what were the mistakes to be avoided this time?

when I show that your project is repeating the exact same catastrophic design flaws as Flow, the "Flow clone" label will stick.

You've seen the prototype. It's a fundamentally different design as Flow. So far it loads fast, does not have VisualEditor and works on a standard wiki markup page, not a proprietary database format. Those were the primary criteria not met by Flow, afaict. Could you elaborate, what are the "exact same catastrophic design flaws" that you see?

It will needlessly escalate drama

It would be more helpful if you would clarify your concerns and suggest solutions before creating unnecessary drama.

On one hand any random yahoo can make an account, and if you think that random yahoo is spouting fringe garbage then it is perfectly appropriate for staff to blow them off. The magic words are "show me a consensus".

Would you translate this to English?

>On one hand any random yahoo can make an account, and if you think that random yahoo is spouting fringe garbage then it is perfectly appropriate for staff to blow them off. The magic words are "show me a consensus".

Would you translate this to English?

If an individual is making arguments or requests or demands and a team disagrees, it is appropriate for the team to disregard that individual. The Foundation tends to deal with this situation with extremely-polite responses that unintentionally encourage endless and futile attempts to make progress on the issue. Both sides may get annoyed or frustrated.

The polite, effective, and firm way to end an unproductive discussion is for staff to say the issue will be considered if/when the user comes back demonstrating a community consensus for the position being argued.

This response clearly tells the individual that the discussion is effectively over.

This response offers the individual a constructive path forwards, and the Foundation doesn't get blamed rejecting the user's idea.

If we are dealing with a rogue individual, the community will shoot down their dumb idea.

If the individual does come back with a consensus, then the team gains important information. This is a real issue that must to be dealt with, and the earlier the team gets this information the better. The team can either accept the consensus, or they can open a dialog with the community trying to sort out the matter together.

Why did [Flow] fail in your view

"My view" basically amounts to reading the community reaction to Flow. If you check the EN:WT:Flow archives, you'll find that Flow's tombstone is already being written at Archive page 2 (and it just gets worse in later pages). Editors are specifying "mandatory" requirements they know Flow will not meet, and there's already an outright declaration that the product "will not be rolled out!".

Flow's cause of death crystal clear in those discussions. Flow died because it was built on top of the VE, and because the community considers complete and accurate wikitext support to be a non-negotiable requirement. Flow was as good as dead as soon as editors discovered that Flow was built on VE and that it would have (at best) broken/partial wikitext support.

You've seen the prototype. It's a fundamentally different design as Flow. So far it loads fast, does not have VisualEditor and Could you elaborate, what are the "exact same catastrophic design flaws" that you see?

You don't see VE up front, but it is is running some of the same code as Flow behind the scenes. It is running Visual Editor's back-end code (Parsoid) to create and/or save the comments. Note that as a user, there is no way in hell that I shouldn't be able to tell what code is running behind the scenes! I know, because the product has the same bugs Flow has! The product has the same design flaw as Flow. That design flaw results in a range of bugs, up to and including content corruption.

The Flow Talk archives make it clear that proper wikitext support is non-negotiable. The design needs to be fixed. The community will get hostile and ban the product if they see it has the same content corruption as Flow.

If you check the EN:WT:Flow archives

I've been looking at those discussions before and now again. I don't see myself having the time to read all the 15 pages of archives. It would help if you could cite and link a few highlights, especially the "mandatory" requirements. On archive 2 I've found only "wikitext is mandatory". The prototype meets this requirement with wikitext editing and a live preview.

It would be even more helpful if you could compile a list of test cases (sample wikitexts) that was corrupted by the 2013 Flow. Flow (and Parsoid) has improved a lot since then. It's unclear to me what wikitext breaks Flow these days, as the biggest trouble I had with Flow was the inability to paste content and this was fixed sometime in the summer, after I reported it.

Also Parsoid was recently rewritten in PHP. Based on the latency of the prototype's preview I guess it uses the new server side parsoid, however the developers could tell exactly how it works now. In any case, all the wikitext that broke the original Flow needs to be tested for corruption with the current prototype, so the developers can fix any corruptions. A full list of breaking wikitext would be great help for that.

Note that using only Parsoid without the VisualEditor does not incur the loading times of VE and its many modules.

Ps. I don't want to take away the chance of PPelberg to answer, these are just my questions for clarification.

A full list of breaking wikitext would be great help for that.

The Flow team pushed that line for the last six years. It is both impossible, and futile.

Any sane software should be able to copy and save a basic text string without corruption. This software is not sane. There is an obvious and fundamental design flaw. That design flaw results in an apparently endless array of symptoms. The Flow team insisted we file bug reports on individual symptoms, and insisted that they would bring the product to acceptability by addressing those symptoms.

That approach failed for the last six years. That approach will fail for next six years. That approach will fail for the next sixteen years. I do not wish to participate any further playing that unconstrutive game. I don't plan to waste more years filing endless new and pointless bug reports on symptoms.

The Talk Consultation clearly established that the original design was fundamentally flawed, that we have to start from scratch with a better design. I'm trying to help the Foundation get the design right this time. The bug report is the design flaw, not the symptoms.

It is very very simple: I'm not editing with VE, I don't want to edit with VE, don't run the text back and forth through VisualEditor's software stack.

You seem to have ignored me entirely above when I said:

There are some other problems I have with this comment, notably that you are under the mis-apprehension that the old wikitext parser will continue to exist past a certain date.

It will not. You will need to come to grips to that. That fact is part of the reason that Extension:Linter exists. Continuing to build for the old parser is more or less not a reasonable ask.

Please feel free to take that as you will, but continuing to say "I won't produce the problem children." is not going to result in a product that you like.

As for That approach failed for the last six years. That approach will fail for next six years. That's because Flow died a quicker death than you seem to appreciate, and went into maintenance mode almost immediately after the software was "released" due to its death. That's not evidence that the approach is problematic, and I am pretty sure it's not evidence of anything of interest.

@Izno the discussion was already running off in too many messy directions. I suggest that topic be left for somewhere and somewhen else. Also, I'm not familiar with your producing-problem-children idiom or metaphor. If it helps, my position is that my personal views are irrelevant. My position is that the Foundation runs into trouble if it tries to implement any product or strategy contrary to community consensus, and I try to help avoid that situation.

...my personal views are irrelevant. My position is that the Foundation runs into trouble if it tries to implement any product or strategy contrary to community consensus, and I try to help avoid that situation.

The sense of your words here contradicts the sense of your earlier words, where you basically ordered the WMF to stop the current line of work and continue in some particular fashion more sympathetic to what you perceive to be the community consensus.

In the general case, you might reconsider your mode of engagement if indeed you wish to be considered an innocent messenger.

...

In this specific case, I'd say your message has probably been received, such as it will be, by now, given the multiple thanks I've gotten on this thread alone. Time to drop the stick and move on to something more productive.

I'd say your message has probably been received, such as it will be, by now, given the multiple thanks I've gotten on this thread alone. Time to drop the stick and move on to something more productive.

Allow me to clarify how I see the situation.

1. I pinged the project manager, trying to warn that there was a problem here.
2. I waited one week, and I pinged him again.
3. I waited one week, and just as we were hitting the two-week mark I got a response.
4. I replied, again with a ping to the project manager, trying to find out how he planned to proceed. When I found and tested the prototype I quickly followed up literally begging the project manager for a response.
5. We are now at another 5 days without a response.

I consider everything else to be disruptive noise clogging up the thread.

I am trying very hard to reach out, and there is exactly one (WMF) post in this thread. Izno, however much we disagree on issues X Y or Z, I hope you can acknowledge that there is a serious problem with Foundation engagement when giant red-flag warnings are ignored for weeks on end.

You say it's time for me to move on to something more productive. Perhaps my biggest mistake over the years is that I extend too much AGF and I spend too long desperately begging and hoping the Foundation will finally engage constructively. With the 2017editor project I begged and pleaded and tried to reach out for four months, before I opened the RFC against it. I didn't want to go that route, I tried too long and too hard trying to find some shred of collaboration from the Foundation. But you're right - I should have opened that RFC much sooner.

If I have to "move on to something more productive", I will. But I'm willing to wait here a few more days before giving up my thin-but-persistent-hope for a collaborative approach. I am not eager to open an RFC against this project. I am not eager to cite Foundation-non-responsiveness as the #1 reason the RFC is needed.

@Alsee: Consider that how you describe the problem you perceive is confusing, chaotic and dramatic. If you want a response, try to be exact - by that I mean provide exact inputs that cause errors. Focus on simple, verifiable facts, examples and present it concisely.

• Differentiate between Flow, VE and Parsoid, cause you talk about the 3 intermixed. A few parts are common between these, but the whole packages are very different, just like the bugs.
• Don't generalize and talk about a common fundamental design flaw in different software, it makes no sense.
• Realize that how you present the 6 year old community discussions reflects your opinion: It is very very simple: I'm not editing with VE, I don't want to edit with VE, don't run the text back and forth through VisualEditor's software stack.. It would be more authentic if you would simply write about your opinion, without dressing it up as community consensus. The community's response was much more complex and differentiated than you represent it.

I consider everything else to be disruptive noise clogging up the thread.

Your dramatic approach is also disruptive. A bit more effort - more testing, examples and a better understanding of the stack - is necessary to be an effective help.

I am not eager to open an RFC against this project. I am not eager to cite Foundation-non-responsiveness as the #1 reason the RFC is needed.

There will be an RfC for enabling the final product. Any kind of RfC you would open before that would just torpedo the project and spread this confused understanding of the software in the community. That's not how you help.

Just an opinion... please excuse my directness: I see your previous RfCs as torpedoes that focused on blocking (stonewalling) a development instead of finding solutions to make it better. I would not be proud of that kind of community influencing. Imho the prevalence of this kind of negativity caused the community to be stuck at a social and technological development level that's a decade old. Free software communities - these are similar in fundamental principles to wikipedia -, are generally more advanced technically and in regards of social structures. A solution oriented approach and some positivity is necessary to progress.

Exact and concise:

Any content sent on a round trip through Parsoid is liable to corruption.

Don't do it.

@Alsee:
> A full list of breaking wikitext would be great help for that.
The Flow team pushed that line for the last six years. It is both impossible, and futile.

It is a basic, fundamental step in software development. Not futile, nor impossible. If you want to help, collecting test cases is one way to do so. Not necessary to collect all, but the more you collect, the better the coverage and the resulting software.

should be able to copy and save a basic text string without corruption. The copy-paste issue was the most frustrating I've met in Flow (although you might be talking about another copy issue...). I was pleasantly surprised that it was fixed soon after my report.

However, you've responded to the current prototype (that has nothing to do with Flow) with "I am begging you, please let me help you." and now criticize Flow's old bugs that might not be fixed already, instead of helping by collecting test cases.

That approach failed for the last six years. That approach will fail for next six years.

Now we are talking about Flow and the new talk pages - two fundamentally different products - at the same time, confusingly mixing assertions about either this or that in the same sentence... It makes no sense and it is kind of unhelpful.

I should not that we are conducting this discussion in the current Flow version, without issues. While the Flow "approach failed" to replace talk pages, it is a usable product, even if not as fast and convenient as reddit, which has threaded discussions and visual editor (wysiwyg) that can be switched to source editor (markdown), so basically the same feature set.

that we have to start from scratch with a better design.

The prototype started from scratch and Parsoid was recently rewritten in PHP.

I'm trying to help the Foundation get the design right this time.

Sorry to say this: the way you try to do that is confused, focusing on dramatic words instead of accurate terminology and exact use-cases.

It is very very simple: I'm not editing with VE, I don't want to edit with VE, don't run the text back and forth through VisualEditor's software stack.

To help first clarify to yourself what is the subject at hand: it's not Flow and not VE anymore. I think you are confusing Parsoid - a small part of all VE's modules - with VE.

For example when someone edits one part of the page and the software corrupts the content on an unrelated part of the page. I'm saying that it is not some random bug. Bugs aren't supposed to be predictable. When I tested the prototype I was able to predict and immediately confirm that it was broken in this manner.

The examples you tested would be very helpful if you were to copy those into your answer.

I should not that we are conducting this discussion in the current Flow version, without issues.

This is a tangent topic, but I once came to the Foundation attempting to file a bug report regarding a different product. However the Foundation converted all of the reporting-pages to Flow. I typed in the bug report, and of course Flow corrupted the bug-report-examples. I tried several times to type or paste the content into Flow in various ways. No matter what I tried, Flow kept corrupting the bug report. Eventually I gave up. Flow is so screwed up that I was unable to file the bug report! That is a circular level of insanity. If I could to dig up the content I was trying to post, the new product would likely also corrupt it.

There are many community members who refuse to post here, because of constant infuriating issues with Flow. I have endless infuriating issues with Flow, but we'd all be screwed if we don't keep open the bridge of communication between the community and the Foundation.

@Alsee: The place to report bugs is phabricator. Does not use Flow.

I'm sorry you are so infuriated. Calm down and you will find the solution. As a last measure you can upload the problematic wikitext to pastebin and link to it. At least links work, don't they?

@Aron Manning you misunderstood me. I'll attempt clarify.

we are talking about Flow and the new talk pages - two fundamentally different products - at the same time, confusingly mixing assertions about either this or that in the same sentence... It makes no sense and it is kind of unhelpful.

It didn't make sense because you missed that I was talking about a single thing.

There aren't two sets of problems, a single problem exists in both products.

There was a design flaw in Flow. The new team cloned that flaw into the new product.

You are asking me to report the same issues that the Flow team asked us to report for the last six years. Just like you are saying now, their position was that this is "a basic, fundamental step in software development". You are asking me to run the same fool's errand they asked us to run. The Flow team failed to solve the issue because they refused to fix the code causing the issue.

There's no need to collect test cases. If you fix the code causing the problem you fix ALL of the test cases in a single stroke. I'm not going to waste time reporting unnecessary and useless test cases.

This design flaw means that the software is unable to INTERNALLY move or save text reliably. When the software translates and re-translates the text it can delete, mangle, corrupt that text in various ways.

The problem can't be fixed if the Foundation refuses to fix the code causing the problem.

I believe the software can't be deployed if the Foundation refuses to fix the code causing the problem.

@Alsee: it is unclear what issues you mention. You say you have tested these cases with the current prototype and made reports 6 years ago for Flow. I understand you don't want to repeat typing the same reports. Would you please link to at least a few of those reports? Without exact cases we are just talking about some imaginary Flow-flaw that I don't see being present.

This design flaw means that the software is unable to INTERNALLY move or save text reliably.

What do you mean by moving text internally? This sounds like refactoring. Why would a reply prototype do that?? I haven't even heard about Flow doing that.

or save text reliably. When the software translates and re-translates the text it can delete, mangle, corrupt that text in various ways.

Any kind of text I've tried was reliably saved. The editing is done in wikitext, thus there is only one translation - for the preview - there is no re-translation. I have the impression that you are talking about something else, not the current prototype.

The problem can't be fixed if the Foundation refuses to fix the code causing the problem.

I don't see the basis of that assertion. Citation needed. Please link to exact issues that were refused.

I believe the software can't be deployed if the Foundation refuses to fix the code causing the problem.

It's still only a prototype and you are racing forward to undermine the final product... Please be aware that this is the same as stonewalling, a very common and ill practice in the communities, that's part of the reason that the UX design of the wikipedia platform is generally 10 years outdated. It's also very disturbing. We seem to be talking about fictitious flaws. Please focus on expressing exactly the issues you experience, with examples. If those examples are 6 years old bug reports, that's fine just link to them.

For what it's worth, I also have been considered an influencer at en.Wikipedia, and I believe the Foundation's statement that something like Parsoid will be required for better handling of the invalid HTML 4 (and maybe 3) that is currently generated on certain pages. The "corruption" Alsee is talking about is, almost always, the result of Parsoid making different translations of invalid WikiCode into valid HTML 5 than the previous (unnamed) renderer was making into valid HTML 4.

Although I don't agree with all of Alsee's statements, it does not appear that the Foundation is taking his statements seriously. The Foundation should determine whether Alsee reflects consensus at en.Wikipedia before they consider rejecting Alsee's statements. [I eliminated all pronouns from this statement, because it was not clear which pronoun referenced which noun.]

The "corruption" Alsee is talking about is...

@Arthur Rubin I'm not talking about preview errors here (although that is another issue). How it's rendered into HTML is irrelevant to my point. I'm talking about flat-out content corruption. For example when someone edits one part of the page and the software corrupts the content on an unrelated part of the page.

I'm saying that it is not some random bug. Bugs aren't supposed to be predictable. When I tested the prototype I was able to predict and immediately confirm that it was broken in this manner. That's not a bug. That is defective by design.

Thank you all for expressing your concerns. And thank you Alsee for checking in with me about whether the words I used came across to you in the way I intended them to (more on this in a soon-to-be-posted response).

This is what I'm hearing in this thread.  Please tell me if you think these points should be revised:

1. Wikitext support needs to be the top-most priority for this project and all new tools and "improvements" need to honor this requirement
2. All enhancements need to work in ways that improve contributors  workflows/experiences, regardless of their experience levels.

Assuming I've understood the concerns above correctly, then I can say we share these concerns. Although, with hindsight, I wonder if not providing details about the current state of our thinking around the questions raised in this thread's original post signaled the opposite.

Right now, I'm finishing gathering a few details so I can provide answers to the questions posed here and should have something to post before tomorrow (Wednesday) is over.

...thank you for being patient with me on this.

@PPelberg (WMF) thanks. I understand you're busy, although I am concerned that this official page for the project has an average response time from the team of nearly two weeks. (Two replies on Project Talk in the last 25.5 days, in case anyone objects to rounding errors.)

Regarding "supporting wikitext" the issue is, what does that phrase actually mean?

Wikitext is used for the overwhelming majority of edits. Wikitext is de facto the primary tool on the wiki. That is how the community views it. The wiki is literally made of wikitext. In the past, the Foundation's use of the phrase "wikitext support" has fallen badly short of the community's expectations. I don't want to burden or blame you for past issues, but the current project also appears to be treating wikitext as a second class citizen.

• You explicitly said you wanted to demote wikitext to second class status, referring to the "first experience contributors encounter". The last time the Foundation changed the first experience contributors encounter, multiple communities (including EnWiki) responded by writing hacks for the sitewide javascript.
  New users should be able to comment without knowing wikitext, but don't seek to hide or hinder the fact that we are a wiki. Don't hide or hinder wikitext.
• The current project corrupts wikitext. That was the poster-child for killing Flow. The team should have treated that report as a fire alarm. The wiki *is* wikitext. Corrupting the wiki is not some insignificant matter to trade-off in pursuit of other goals. Sending the content on a round-trip through parsoid is unnecessary and harmful.
• The current project has inaccurate previews. I understand it's a prototype, but it seems the cause of the issue is essentially disdain for wikitext. I think(?) the problem is that you're not actually generating and using the wikitext for the preview.

I'm sorry for not responding more quickly here.

In an effort to make sure we address these questions, I'd like to come back to the discussion about what we mean when we say, "Wikitext support needs to be the top-most priority for this project..."

To this end, here is a snapshot of our current thinking.

(In an effort to make the content more readable, I am going to post responses in a series of posts)

Will the initial/default text input be wikitext or VE?

The default text input for replying to specific comments and starting a new discussion will, for Version 1.0, be a wikitext text input. In later versions of each feature (see /Replying/Versions), we have plans to develop a rich text editing input and subsequently test which text input method is a "better" default. More on what "better" could mean across different contexts and how we might test and measure "better" below.

Note: we are planning to deploy early iterations of all features on a limited number of wikis first to make sure the tools are behaving in ways contributors expect and find useful before they are deployed more broadly.

Is the wikitext mode going to be provided via VE's "wikitext mode", known as the 2017WikitextEditor?

The wikitext text input mode for replying to specific comments and starting new discussions is not currently using VE's "wikitext mode" aka the 2017 wikitext editor.

We have talked about, and explored, using the 2017 wikitext editor as the editing interface for the wikitext text input mode in the future. Although, we have not made any decisions about this yet and before doing so, we would need to, at a minimum, talk about it on-wiki and conduct user testing. And by "explored" I mean we have started documenting and investigating issues with the 2017Wikitext Editor with the intention of figuring out whether it is a tool we could all rely on in the future.

Are previews going to be provided via VE's parser, parsoid?

The new replying workflow (prototype included) uses the old parser to generate previews. It sounds like you (Alsee) encountered a case where the preview differed from what was saved on the page [1]. Let's continue talking about this in the thread where you first surfaced this issue to try to get to the bottom of it. See here: Usability test: Version 1.0 prototype.

What parser is going to be used for saving content created using these new workflows?

Parsoid is currently being used for saving. More specifically, Parsoid is being used to make sure a contributor's comment is inserted in the right place on the page. Right now, Parsoid is not interacting with/affecting the content of a reply.

In the rich-text mode proposed for version 2.0 of the new replying workflow, we are planning to use Parsoid to convert the rich-text to wikitext.

With the above said, you mentioned there are instances where saving content to the page is causing corruption. This goes against what I understand to be our shared concern for wikitext support.

I see you followed up with more details in this thread (thank you); let's continue to talk about this concern there.

Why are two different parsers being used: the old parser for previewing replies and the new parser, Parsoid, for saving replies?

The primary reasons we are using two different parsers at the moment stems from the following:

• Previewing: in order to get a prototype working as quickly as possible, we used the old parser to implement previews. Further testing and usage will tell whether this is a decision we should revise.
• Saving: A key part of the replying workflow is the software being able to take the comment a contributor composes and insert in the proper position within the existing page. While both parsers – old parser and Parsoid – can handle common comment insertions reasonably well, there are edge cases Parsoid can better support (e.g. supporting transcluded discussion pages).

Two notes:

1. The gadgets we examined [1][2] were split between the two parsing techniques. While inserting a comment in its proper position was a key consideration of the team's when deciding which parser to use for saving, we also had the following thoughts in mind when weighing the different options:

• Future features: Parsoid will be required for saving replies composed using a rich text input. Also, if in the future, the software is to enable contributors to edit individual comments, Parsoid would be required to extract and make individual comments editable.
• Shift to Parsoid: in the future, as part of the parser unification project, Parsoid will also be generating the regular page HTML, so its function as a preview will be further improved.

2. On the topic of the relationship between the reply tool, Parsoid, and the visual editor, the current prototype also depends on VE's API for saving the edit to the page.

Some other themes came up in the process of drafting this reply that seem relevant to mention. They follow in this, and the comments that follow...

Measuring which text input method is "better" to show newer contributors by default...

This seems deserving of its own conversation. I was thinking I would start a separate thread where we can start thinking about a plan for how we go about testing how to evaluate "better." I think having this conversation in January - February makes sense since that is around the time the team will be focused on testing.

"'yes you intend to make VE or VE's-wikitext mode the default'. Please let me know if I misunderstood you." [1]

My reason for using the word "test" in the first comment I posted in this thread [1] was borne out of my attempt to communicate our interest and openness to learning whether rich-text editing being the default input method offers newer contributors a better experience.

The team does not have a particular bias to push out a particular technology (e.g. editing surface, parser, etc.). Although, as mentioned above, there is another conversation to be had on the topic of what kind of experience increases the likelihood newer people will grow into active and productive contributors.

"...repeating the same mistakes [as Flow]." [1]

I am hearing this comment as having a couple components.

One component seems to be the collection of decisions that prevented more experienced contributors' from using talk pages in ways that they needed to (e.g. no support for full-page wikitext editing).

The second, perhaps underlying, component seems to be the sense that experienced contributors' feedback was not heard. We meant what we said in the Talk Page Consultation [4] and at Wikimania [5] about wanting to behave differently with this project and I hope this exchange as well as the work we are doing to expose our assumptions, listen to input and make our progress easily to follow demonstrates this intention.

I think that speaks to the questions raised here, although please let me know if I've missed something. And again, thank you for being patient with me here...I'm sorry it's taken this long for me to post these responses.

I'm sorry for the long trip through background topics, but I'm hoping to communicate a broadly-important "why".

please let me know if I've missed something

The Foundation in general has been missing something for nearly a decade, including why Flow died. The reasons you cite for Flow's death were likely insurmountable, but those were merely the explanations we managed to communicate to the Foundation after the Foundation couldn't/wouldn't hear the much earlier and much simpler reason Flow died.

Flow effectively died before the prototype was even built. Flow was dead when it became clear that the product would have incomplete/defective wikitext support. That's when various editors started posting absolutist declarations like this:

Dear Jorm, let's get this absolutely straight. Unless Flow will be able to render every single aspect identically to every valid editor of the article name space, it must not and will not be rolled out! Talk pages are not independent social websites, but they have only one purpose: to discuss articles. For this we need to be able to copy every kind of content, format and structure between the article name space and the talk pages. Your job is to support us in writing Wikipedia, your job is not to build a flashy social website. The identical rendering of every possible content must be your paramount design goal. Everything else takes second place. rgds --h-stt !? 09:25, 24 July 2013 (UTC).

The initial unspoken assumption by the community was that anything on a wiki inherently comes with complete and flawless wikitext built in. It was unspoken because editors thought it blindingly-obvious. That's the #1 mandatory requirement: Full wikitext, flawless wikitext. Don't screw up wikitext. The wiki is literally made of wikitext. Wikitext is de facto the primary tool for almost all editors. Wikitext will de facto continue to be the primary tool for almost all new users though the foreseeable future. (New users should be able to comment without knowing wikitext yet, but the new tool should not hinder new users by trying to hide wikitext.) The community reacts strongly against anything damaging to the wiki, and that includes damage to wikitext itself.

The Foundation's 2011 strategy explicitly declared a plan to deprecate wikitext. The lead developer for Flow outright declared his desire to kill off wikitext. Flow failed because the Foundation thought it was a good idea to make trade offs damaging to wikitext, in service of the Visual strategy. 2017editor failed because it was designed purely in service of a Visual agenda, to the serious-detriment of the product for actual wikitext editing. The Foundation has repeatedly considered wikitext unimportant at best, and has considered it adequate to provide partial or damaged wikitext because (according to some staff) no one should be using wikitext anyway.

Imagine I propose replacing your compiler. Then you learn the new compiler will have an incomplete or defective implementation of the language. At that point, nothing else matters. A partial or defective implementation of the language is an absolute non-starter. Now imagine I ignore your objections and I proceed to build the new compiler anyway. Not only does the new compiler have a defective implementation of the language... you discover that the compiler sometimes causes corruption to the source files. That is not a sane compiler design. No sane compiler design should ever modify the source files. That basically summarizes Flow. The Flow design was not sane because it was built as part of the 2011 strategy to deprecate wikitext... Flow is literally incapable of saving wikitext... Flow has an unholy-hack to create an illusion that it handles wikitext. It's such an unholy-hack that it even breaks the revert button: Flow can't save wikitext so the revert button has to make up some fictional wikitext version to revert back to, then Flow pretends to save that fictional wikitext. That is insane. Unsurprisingly, a revert button that works in that insane fashion does not always generate a correct revert result.

Now let's consider the current prototype. The core task is to insert one text-string (the new comment) into another text-string (the page). Any normal code for that core task is common and trivial.

• It would be a reasonable bug for it to fail completely, possibly with complete garbage.
• It would be a reasonable bug to get the copy-endpoints wrong, where the copying starts or ends too soon or too late.
• It is not a reasonable bug to mangle the middle of a string being copied. No sane software design should do that, not even as a bug. It's nearly impossible for any sane software design to screw up the middle of a string being copied. Flow did this, the new prototype does this, and the prototype apparently does it for basically the same reason Flow did it. It's not a bug, it's a design flaw. The design flaw is a result of the general Foundation strategy to prioritize the Visual system to the detriment of our primary tool, wikitext.

I believe the prototype is sending the content on a round trip through Parsoid. I believe that round trip creates a double-translation-damaged-reconstruction of the content. I believe the prototype is copying the damaged reconstruction instead of copying the actual content. That crazy complicated code path is approximately the only way a simple string-copy could mangle the middle of the string. The question is, do you consider wikitext corruption important enough to fix, even if a full fix would mean avoiding sending content on a round trip through Parsoid?

@Alsee, the background you shared is helping us to better understand the place you are coming from.

Now, to the points you raised...it seems like there are four that directly relate to the replying feature and the project as a whole.

I'm going to post those four points as separate comments to make it a bit easier read/engage with (as I did above).

Point #1

Flow was dead when it became clear that the product would have incomplete/defective wikitext support

In what ways do you see the approach we have taken thus far as hindering contributors' ability to edit talk pages using wikitext? I ask because one of the core principles of this project is to maintain contributors' ability to edit talk pages using full-page wikitext editing, and I don't think anything we have done or have planned is preventing that.

From the perspective I currently hold, we have been and are continuing to live up to that commitment by ensuring the new tools we build (e.g., reply tool) still enable contributors to use full page wikitext editing to do things like, as user:H-stt puts it, "...copy every kind of content, format and structure between the article name space and the talk page."

If we're seeing this differently, please let me know.

In what ways do you see the approach we have taken thus far as hindering contributors' ability to edit talk pages using wikitext?

I did not intend to suggest any concerns in this area. Aside from the indirect issue of corrupting existing content, the product appears to be excellent in preserving existing functionality.

Point #2

"...you discover that the compiler sometimes causes corruption to the source files."

I think we agree here: it is unacceptable for the tools we build to corrupt the content of the existing talk pages. The approach we are currently taking to uphold this commitment is as follows (the below is copied from T241388#5779747):

• Short-term: investigate (e.g. the point you raised here) where the content of a reply can corrupt the rest of the talk page. I should say: for each case, we will need to understand the issue and assign it an appropriate level of priority before committing to resolving it.
• Longer-term: introduce new wikitext syntax to encapsulate comments and ensure their contents do not impact the rest of the talk page. See: T230683 + T246960

With the above said, beyond this example (see the outcome of our investigation here), are there other instances you have come across where talk pages are being corrupted? If so, are you able to share the content of the reply that is causing the page corruption so we can reproduce the issue?

One issue we're currently working on that may fit what you have in mind is a bug related to broken/incomplete table syntax: T246481.

Point #3

It is not a reasonable bug to mangle the middle of a string being copied

Assuming an example of what you are describing is the scenario where a reply posted using the Replying feature could cause corruption on talk pages that contain broken or incomplete syntax, then we agree, this is an issue that needs to be addressed and one that we are currently working to fix: T246481.

Point #4

I believe the prototype is sending the content on a round trip through Parsoid. I believe that round trip creates a double-translation-damaged-reconstruction of the content. I believe the prototype is copying the damaged reconstruction instead of copying the actual content...The question is, do you consider wikitext corruption important enough to fix, even if a full fix would mean avoiding sending content on a round trip through Parsoid?

What you describe is correct...

In order to save a new reply to the page, the prototype is sending the content on a round trip through Parsoid.

Why is the software sending talk pages through Parsoid?

By sending talk page content through Parsoid, the software is converting the contents of the talk page into HTML. This makes it possible for the software to work out the structure of the page and thus more reliably insert replies in the correct position on the page. Secondarily, by sending talk page content through Parsoid, it enables the reply tool to work on transcluded pages.

I suspect the above are things you already know. I'm sharing them here so others have a clearer sense for how and why we've arrived at the current approach.

What does this approach say about our commitment to resolving issues where wikitext is being corrupted?

On one hand, by using the new parser, we are acknowledging it becomes possible for a comment posted using the new replying workflow to corrupt other parts of the page (e.g., when a table is not properly closed).

With this said, we are actively working to minimize the risk markup errors contained with talk page comments posted using DiscussionTools could "leak out" and disrupt content on the rest of the talk page. [1]

We are also committed to investigating all issues of content corruption and fixing those cases where the severity and frequency of the issue warrants it, even if that "fix" requires us using an approach that is different from the one we are currently using.

---

1. RFC to introduce new syntax for multi-line list items/talk page comments: https://phabricator.wikimedia.org/T246960

@PPelberg (WMF) it appears that we're in full agreement and understanding in how and why the product is broken. Your plans for addressing the issue are unhelpful. I suggest/request that you cease wasting developer time pursing them.

The product has fundamentally the same problem Flow had, for fundamentally the same reason. You're giving fundamentally the same answers that the Flow team gave.

Your product causes content corruption both to existing page content and to the new-comment content, for the exact same reason. You're translating the content through an insanely complex code path, then you're translating it back and attempting to reconstruct the original content, and it's failing. Of course it's failing. It's a miracle that it doesn't fail more often. And just like the Flow team, your answer is say you'll play whackamole on individual fail-cases. No, please, please stop flushing money down the toilet on an endless series of pointless phab tasks chasing individual fail-cases.

The design is fundamentally broken. You chose a design that has inherent content corruption problems. I understand why. In part I sympathize, you're a victim of the last decade of Foundation strategy. You're using the most convenient design and tools at hand.

There's no need to deal with content corruption if you stop running the page-content or comment-content on a round trip through Parsoid. It is possible. You can choose to fix the design.

Use any means you like to find the right byte-location to insert the comment into the page.

Give me an edit box. Pre-populate that edit box with the appropriate indentation-markup (possibly with text styling, possibly click-to-edit). When I hit enter, prepopulate the new line with the indentation again. When I'm done, append the four tilde signature. You now have the exact string a user would have typed in standard edit mode.

The comment is raw-text string X. The page is raw-text string Y. Insert string X into string Y, at the identified location.

There is no chance of content corruption, short of a hardware failure. The user can enter any and all wikitext, and it will work flawlessly. The user can modify or delete the indentation markup, and it will work flawlessly. Just give us a semi-automated system to help us insert raw-text into the page at the right spot.

The only other thing you need is a similarly flawless approach for preview.

I want to add an additional note. I plan to explain this content corruption issue to the community, either in relation to this product or in a discussion of the Foundation's broader VE strategy. When I explain this issue, I'm going to explain why this issue exists. I am going to trace this issue directly back to a strategy document the Foundation published in 2011. This issue exists as a direct result of a Foundation strategy to exterminate wikitext. Your product is doing something insane, that insane thing is causing wikitext content corruption, and it's because the Foundation deliberately decided to sabotage and exterminate wikitext. The community is going to have a shitfit. They will ban deployment of this product.

@Alsee, we agree with you: continuing to invest in a technical approach that causes more issues than it creates does not make sense.

Here are the instances of content being affected in unexpected ways that we are currently aware of:

• T242466: Why does posting a new comment delete `</span>` from a previous one?
• T246481: Prevent page corruption in DiscussionTools caused by broken and incomplete table syntax

Based on our investigations of the above and the feedback others have shared so far, we think the approach we've taken so far (using Parsoid) works well. With this said, we are committed to revising this approach if we find critical instances of corruption that we cannot resolve.

It's with this in mind that we continue to be eager [1][2] to hear about other instances of unexpected behavior the Replying tool could be causing.

---

1. See 12-March update here: Talk_pages_project/replying#12_March_2020

2. See discussion topic here: Call for unexpected behavior

@PPelberg (WMF) wow. After everything I've said I am flabbergasted that your response is to say that you don't care about the problems.

I organized consensus across three wikis to kill Flow. I made every possible effort to warn you that this design is fatally flawed. I told you that I'm in the process of installing a new EnWiki Village Pump page - specifically for the purpose of dealing with the Foundation's polite-brick-wall pattern of ignoring feedback that there's problem. I told you that I intend to make your product a top item of business on the new page once it's deployed. I said I expect the community to ban deployment of the product. And your response is that you don't care about the problems.

You're doing the same thing the Flow team did. You're responding the same way the Flow team responded. They thought the loop-through-Parsoid approach worked well. They knew they were breaking wikitext, and they considered breaking wikitext to be unimportant. Hint hint, the community is rather hostile to people who attempt to deliberately sabotage our most important tool.

The new Pump page should be in place soon and I'll promptly work on bringing you a consensus on the matter. I can go multi-wiki if needed.

@Alsee Parsoid has significantly changed since that time that you're talking about. The issues you claim to have existed, no longer exist. I'm not sure that those issues even existed, as you've refused to show the bug reports that you claim to have made, or to show evidence (examples) of the issue still existing. You've been repeating this claim for 4 months in this extremely long thread and other threads, never presenting a bit of evidence. Please understand that this kind of feedback is unusable, just creates drama. I've tried to explain this to you before: this is not how you can help the project.

I'm also "flabbergasted" to read another threat from You of undermining this project ("ban deployment of the product"), in bold. The current project is taking great shape, there are no "fatal flaws" as you claim, nor can it be compared to Flow in which you see so many flaws, despite that there are projects that use it without major issues. Don't get me wrong, I not a fan of Flow either, but it's nothing like how you describe it. Please excuse my words, but your repetitive "fatally flawed", "breaking wikitext", "sabotage" comments by now just sound like plain drama-mongering.

The wikitext format is a mess, that results in edge cases that cause issues. This does not make the software "fatally flawed". Those issues - if there are - need to be reported and fixed individually. I've asked you previously to report such issues that you are aware of, but you refused. If you aren't willing to spend time with presenting examples - what would help - then please at least don't create unnecessary drama.

Regarding your repeated intent to undermine and stonewall this project: I try to but cannot understand why you think that would benefit the community. For more than a decade one of the most requested features was a discussion tool that's easier to use and doesn't require months of learning to get right. This project will benefit the whole movement. Please understand how important this is. From an engineering perspective it is always important to balance the pros and cons with due weight. Decide for yourself: is your perception of some unexplained "fatal flaw" more important than the fact that this tool is usable for the most basic communication needs in the most cases, by most editors, without a long learning process.

Please change the way you "contribute" to this project fundamentally: focus on factual, helpful contributions and stop creating drama. Understand: a community discussion initiated with this biased perception could misinform the community and create weeks or months of confusion and unnecessary drama, potentially undermining this project. Try to avoid this. If you wish to discuss the lack of communication and cooperation between volunteers and wmf staffers, please do so without damaging this project, and try to be at least a little bit neutral: after experiencing volunteering in Mediawiki development I can attest that the talk pages team is quite responsive compared to other teams and does a pretty good job in incorporating volunteer feedback.

@Aron Manning I'm trying to get the product fixed. I have a long history of accurately serving the community in this area. Your stonewalling against fixing the product is unhelpful. I find it rather strange that anyone would actively argue against fixing content-corruption and other fixable problems.

I suggest there is no point in us debating this any further. Either I am going to return with consensus that the product MUST be fixed before it can be deployed, or I'm not. That will de facto resolve the debate.

@Alsee, I've been trying to understand what flaws you perceive. You have written a lot in this thread, but not one "accurate" description of your issue. I have the impression that you perceive some issue that's either not there or not effecting negatively the current product - which is taking great shape - and you make a mountain out of a molehill.

I'm trying to get the product fixed.

Sometimes look back and evaluate whether your actions have achieved what you are trying to do. You have listed here quite a few projects where you lobbied for the rejection of those projects. How many did you manage to "fix"?

You wrote: "Your stonewalling against fixing the product is unhelpful."

This makes no sense. I've explained to you previously how to make effective, actionable bug reports. You've refused to do that and haven't made any bug reports in the last 4 months.

I have a long history of accurately serving the community in this area.

Not at all. How accurate you are is well demonstrated by your above false statement and this whole drama-thread.

To be accurate: you have a long history of being a self-appointed community spokesperson, who gives undue weight to negative opinions, lacks accuracy and the technical understanding of how product development works. The problem is you don't seem to realize the damage this approach causes.

In my opinion what you are doing is populism, making superficial claims without evidence. You've been asked by a few people - going back years - to adjust your approach and you've even been blocked previously for this reason.

Either I am going to return with consensus that the product MUST be fixed before it can be deployed, or I'm not.

It would be simpler if you would just create clear bug reports for the issues that you perceive, but you refused that. It seems you'd rather turn this into a political lobbying game where you can continue with these dramatic, unclear and - to be honest - manipulative statements you've been making.

If you wish to take that path then keep in mind:

• The projects you've called "fatally flawed" in the past had no opt-out. In this project if you aren't satisfied, you don't have to use it, the source editor is there for you.
• RfCs require neutrality. You have ignored all the positives of this project, giving undue weight to one perceived concern. That's extremely biased.
• RfCs require evidence. So far you haven't shown any.

Tl;dr: I'm trying to understand why you are dead set on blocking this project, but you haven't made a reasonable argument. This has gone too far now.

User-friendly discussions have been a top requested feature for more than a decade, that will have wide-spread positive effect on the whole Movement. Your intent to block it could cause months or years of further delay and monetary damages (wasted developer time). Is that really what you want?

@Aron Manning as I said in my last post, I would seek "consensus that the product MUST be fixed before it can be deployed", and elsewhere I told you that was "to get the product fixed". If you are going to persist in having an argument with an imaginary villain who is dead set on blocking this project, then perhaps I should bow out and let your imaginary villain write the replies to your posts.

The only way this project would actually be blocked is if

1. the community says it must be fixed prior to deployment; AND
2. the Foundation refuses to fix it.

I do not anticipate #2 will happen, and if #2 did happen I reject any blame for it.

You didn't find relevant Phab tasks filed by me because I discussed the issues directly with PPelberg. He filed Phab items for random example cases of the broader systematic problem. (As I'll explain, the details of those examples are not important.) He has acknowledged the problem is real. He simply doesn't want to bother fixing it.

You say you don't understand the issue, so I'll recap. The product translates all of the content through Parsoid, then it attempts to reconstruct the original content by retranslating it back through again Parsoid. The final result is then built based on the reconstructed content instead of the original content. Unsurprisingly, the extremely complex round-trip translation can and does fail in a variety of cases. Sometimes it results in content corruption, sometimes the product fails with an error.

Important point: It fails in a wide variety of cases. A round-trip through Parsoid is so complex that it is effectively impossible to identify all cases where a round trip will fail. Fiddling with individual symptom-cases is pointless. The Flow team demanded that we go on a wild-goose-chase submitting bug reports on each symptom-case. It was a waste of time. The individual symptom-cases generally aren't very fixable, and even if you could fix the individual cases it is futile game of whack-a-mole. An endless stream of new cases keep being found. It isn't a series of bugs - it is a single design flaw with an intractable array of symptoms.

The fix is pretty simple and pretty obvious. Take the user entered comment, add the appropriate indentation characters, then string_insert() the comment at the right spot in the talk page string. Problem solved. All of the symptoms vanish if you stop trying to use round-trip-reconstructed strings where you should be copying clean original strings.

Yes, but how to determine "the right spot in the talk page string"?

I'm not familiar enough with the code to give the best answer, but I can give an almost trivial method just to prove it can be done easily. Factoid: A reply will always inserted as a new line directly after a recognized signature (including a timestamp).

1. Copy the page source, scan it for anything that looks like a timestamp (false positives are ok), and alter the timestamp-digits to show the current source-line-number instead.
2. Use the current code to add a reply with unique content or a unique timestamp.
3. Scan the output for the new comment, then back up one line and read the source-line-number out of the fake-timestamp. That tells you what line to insert-after in the actual page source. Done.

I'm sure our devs can come up with a better approach.

@Alsee: so far, we're aware of three cases you've reported where the new replying tool seems to be causing content corruption.

We've evaluated two of those cases here and here and resolved the third here.

Do you have other examples of corruption you think the Replying tool is causing that we can investigate?

Alsee, Propose better solution of insertion process.