[Wikimedia-l] Subject lines for WMF fundraising emails

[Wikimedia-l] Subject lines for WMF fundraising emails

There is a report from a user on Jimbo's talk page on ENWP regarding
receiving fundraising a fundraising email with the subject line "[NAME] -
Deleting Wikipedia?"

In previous years I've disagreed with some of WMF's fundraising choices,
and it would be unfortunate if in the era of "fake news" becoming
mainstream WMF chooses to continue to be a part of the problem. If this is
happening then I request that WMF put a stop to it. Regardless of how
effective it is to send misleading fundraising appeals and that WMF has
received minimal repercussions for doing so over the years, it's wrong and
it should stop.

Re: [Wikimedia-l] Subject lines for WMF fundraising emails

"Deleting Wikipedia?" was the subject line of the e-mail I received as
well. It also, as usual, included the claim that if enough donations were
received the campaign would end early. That hasn't been the case in the
past when campaign goals are met.

Re: [Wikimedia-l] Subject lines for WMF fundraising emails

On the contrary, Nathan, every single time I have asked them to hold
the campaigns open after the fundraising goal has been met (most if
not almost all of the past ten years) in order to, for example, fund
the Endowment or save more money before the Endowment existed, they
have refused to do so. Is there some document I am missing?

Re: [Wikimedia-l] Subject lines for WMF fundraising emails

For some time we have been trying to find an alternative subject line to
-/This is a little awkward/-. That line works and works very well but we
have found it very difficult to effectively translate and adapt into other
languages, and despite our best efforts have struggled to find an
alternative.

We first tested -/Deleting Wikipedia?/- as a subject line a couple of weeks
ago and it was the only winning variant in hundreds of tests. We retested
in case it was a statistical fluke and continued to see it perform well.
The effectiveness of this subject line for the most part does not come from
its apparent clickbaiting. The change in the number of people opening the
emails was relatively small and unsubscribes remained extremely low. The
big driver in terms of its success was from a significant increase in those
people who opened AND read our email appeal. We posed a question and donors
were motivated to donate when presented with the idea of imagining a world
without Wikipedia.

Our motivation behind this sort of subject line is the fact that in three
countries today it is already as if Wikipedia does not exist. The risk that
this could happen in more countries is greater now that it ever has been.
Censorship, impediments to free speech and over regulation of copyright are
threats that Wikipedia, Wikimedia and its communities face every single day
and it is with that context that we want to lead.

Any email that included this subject line came with at least some context
to flesh out the idea, i.e., “If Wikipedia were deleted, it would be a
great loss to the world,”, but going forward it is our full intention to
make even clearer that we intend for the donor to imagine a world without
Wikipedia and the threats it faces every day, not threaten that it is going
away.

Our plan is to continue to testing on this theme, exploring censorship and
copyright restrictions as well as our increasing role as the backbone of
knowledge on the internet, and help donors see that knowledge can and is
threatened all the time. We are definitely and eagerly open to any
feedback, suggestions and ideas you might have.

We managed to reduce this again through last years fundraising and actively
curtailed the number and frequency of banners that our readers saw.

With regards to email we only send three fundraising asks to any one
individual as it stands which is a fraction of what most charities send to
their previous donors. In the case of last year we actually held off
sending the third fundraising email to a significant portion of individuals
precisely because we were reaching our campaign goal.

So I would argue this is actually the case and that if we can, we will.

Re: [Wikimedia-l] Subject lines for WMF fundraising emails

I believe that the subject line was a moral and leadership failure.
Effective or not, I would respectfully suggest that you stop using it
immediately and begin further discussions with the community about the
organization's goals and purpose.

> Hey Pine,
>
> I appreciate and understand your feedback about this subject line.
>
> For some time we have been trying to find an alternative subject line to
> -/This is a little awkward/-. That line works and works very well but we
> have found it very difficult to effectively translate and adapt into other
> languages, and despite our best efforts have struggled to find an
> alternative.
>
> We first tested -/Deleting Wikipedia?/- as a subject line a couple of weeks
> ago and it was the only winning variant in hundreds of tests. We retested
> in case it was a statistical fluke and continued to see it perform well.
> The effectiveness of this subject line for the most part does not come from
> its apparent clickbaiting. The change in the number of people opening the
> emails was relatively small and unsubscribes remained extremely low. The
> big driver in terms of its success was from a significant increase in those
> people who opened AND read our email appeal. We posed a question and donors
> were motivated to donate when presented with the idea of imagining a world
> without Wikipedia.
>
> Our motivation behind this sort of subject line is the fact that in three
> countries today it is already as if Wikipedia does not exist. The risk that
> this could happen in more countries is greater now that it ever has been.
> Censorship, impediments to free speech and over regulation of copyright are
> threats that Wikipedia, Wikimedia and its communities face every single day
> and it is with that context that we want to lead.
>
> Any email that included this subject line came with at least some context
> to flesh out the idea, i.e., “If Wikipedia were deleted, it would be a
> great loss to the world,”, but going forward it is our full intention to
> make even clearer that we intend for the donor to imagine a world without
> Wikipedia and the threats it faces every day, not threaten that it is going
> away.
>
> Our plan is to continue to testing on this theme, exploring censorship and
> copyright restrictions as well as our increasing role as the backbone of
> knowledge on the internet, and help donors see that knowledge can and is
> threatened all the time. We are definitely and eagerly open to any
> feedback, suggestions and ideas you might have.
>
> Best,
> Seddon
>
> On Tue, Nov 13, 2018 at 11:04 PM Pine W <[hidden email]> wrote:
>
> > There is a report from a user on Jimbo's talk page on ENWP regarding
> > receiving fundraising a fundraising email with the subject line "[NAME] -
> > Deleting Wikipedia?"
> >
> > In previous years I've disagreed with some of WMF's fundraising choices,
> > and it would be unfortunate if in the era of "fake news" becoming
> > mainstream WMF chooses to continue to be a part of the problem. If this
> is
> > happening then I request that WMF put a stop to it. Regardless of how
> > effective it is to send misleading fundraising appeals and that WMF has
> > received minimal repercussions for doing so over the years, it's wrong
> and
> > it should stop.
> >
> > Pine
> > ( https://meta.wikimedia.org/wiki/User:Pine )
> > _______________________________________________
> > Wikimedia-l mailing list, guidelines at:
> > https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and
> > https://meta.wikimedia.org/wiki/Wikimedia-l> > New messages to: [hidden email]> > Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l,
> > <mailto:[hidden email]?subject=unsubscribe>
>
>
>
> --
> Seddon
>
> *Community and Audience Engagement Associate*
> *Advancement (Fundraising), Wikimedia Foundation*
> _______________________________________________
> Wikimedia-l mailing list, guidelines at:
> https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and
> https://meta.wikimedia.org/wiki/Wikimedia-l> New messages to: [hidden email]> Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l,
> <mailto:[hidden email]?subject=unsubscribe>

Re: [Wikimedia-l] Subject lines for WMF fundraising emails

Hi Seddon,

While the fundraising appeal may be successful, the problem that I have
with this subject line is that it can mislead readers into thinking that
someone with the ability to do so is seriously considering, or making an
effort to, delete Wikipedia in entirety. I think that a subject line of
"Block Wikipedia?" might be okay, and I am supportive of encouraging people
not to take Wikipedia for granted. But regarding "Delete Wikipedia?", as
far as I know that generally misrepresents the current situation. I believe
that using "Delete Wikipedia?" as a subject line is inconsistent with
Wikipedia's goals of providing neutral, verifiable, and reliable
information.

I am starting to think that if WMF wants to use the Wikipedia brand name
for WMF fundraising then WMF should first publicly discuss its proposed
uses of the Wikipedia brand name with Wikipedians.

On a related issue, I don't know if it's happening this year, but in the
past another concern that I've had is the conflation of donating to
Wikipedia with donating to WMF. Wikipedia and WMF are related but there is
not a 1:1 relationship, and I hope that WMF makes that clear in its
fundraising. The use of "Delete Wikipedia?" reminds me of these concerns.

I would prefer to avoid diverting the community's limited time into
reviewing WMF's choices, but unfortunately the issues are too significant
to ignore. I don't know how many community members want to volunteer their
time to review fundraising appeals before they go into production, but I
think that it would be good for WMF to ask.

Re: [Wikimedia-l] Subject lines for WMF fundraising emails

Hi all,

Thank you for sharing your concerns. We hear them and we take them
seriously. *As of today, we have pulled this subject line from our testing
rotation.*

On the Fundraising team, we pride ourselves on making data-driven
decisions, and there are many types of data inputs we process outside of
dollar amount raised. For example, how many people choose to unsubscribe
from our list or submit an abuse complaint when we send an email? Does a
certain subject line get very high opens but a low rate of donations per
open--indicating that it is more clickbait than effective content? How much
and what kind of feedback is our Donor Services team getting?

We watched these inputs closely while sending this subject line to donors.
Our unsubscribe and abuse rates were low, the donation per open rate was
even higher than usual, and while our Donor Services team flagged some
negative responses from donors, they determined these comments were not in
a significant volume.

That said, there is a final input which is harder to measure on a per-test
basis: how do we, our colleagues, and volunteers feel about our messaging?
This team cares deeply about Wikipedia and the Wikimedia Foundation, and
about the mission we all work to achieve. We want to represent it
faithfully, and do so in a way our readers and donors can engage with and
understand. This balance can be really hard to strike and it will always be
an ongoing challenge in our work.

We are grateful to be presented with this challenge and with the joy of
telling millions of people about this movement. Thank you for caring so
deeply, for all your contributions, and for keeping this feedback loop
alive.

> Hi Seddon,
>
> While the fundraising appeal may be successful, the problem that I have
> with this subject line is that it can mislead readers into thinking that
> someone with the ability to do so is seriously considering, or making an
> effort to, delete Wikipedia in entirety. I think that a subject line of
> "Block Wikipedia?" might be okay, and I am supportive of encouraging people
> not to take Wikipedia for granted. But regarding "Delete Wikipedia?", as
> far as I know that generally misrepresents the current situation. I believe
> that using "Delete Wikipedia?" as a subject line is inconsistent with
> Wikipedia's goals of providing neutral, verifiable, and reliable
> information.
>
> I am starting to think that if WMF wants to use the Wikipedia brand name
> for WMF fundraising then WMF should first publicly discuss its proposed
> uses of the Wikipedia brand name with Wikipedians.
>
> On a related issue, I don't know if it's happening this year, but in the
> past another concern that I've had is the conflation of donating to
> Wikipedia with donating to WMF. Wikipedia and WMF are related but there is
> not a 1:1 relationship, and I hope that WMF makes that clear in its
> fundraising. The use of "Delete Wikipedia?" reminds me of these concerns.
>
> I would prefer to avoid diverting the community's limited time into
> reviewing WMF's choices, but unfortunately the issues are too significant
> to ignore. I don't know how many community members want to volunteer their
> time to review fundraising appeals before they go into production, but I
> think that it would be good for WMF to ask.
>
> Pine
> ( https://meta.wikimedia.org/wiki/User:Pine )
> _______________________________________________
> Wikimedia-l mailing list, guidelines at:
> https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and
> https://meta.wikimedia.org/wiki/Wikimedia-l> New messages to: [hidden email]> Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l,
> <mailto:[hidden email]?subject=unsubscribe>

Re: [Wikimedia-l] Subject lines for WMF fundraising emails

Thank you very much for this update, Caitlin.

Yesterday I was thinking more about this issue, and today I was planning to
append my earlier comments by saying that I realize that a fundraising
appeal has some differences from an encyclopedia article in terms of
writing style. Also, I realize that sometimes what seems good from one
perspective is problematic from a different perspective.

Perhaps at a time when the Fundraising team is less busy, maybe in January,
there could be an opportunity for a public discussion such as an IRC office
hour, Hangouts meeting, and/or talk page discussion about how to
incorporate community review of Fundraising messages prior to them going
into production.

> Hi all,
>
> Thank you for sharing your concerns. We hear them and we take them
> seriously. *As of today, we have pulled this subject line from our testing
> rotation.*
>
> On the Fundraising team, we pride ourselves on making data-driven
> decisions, and there are many types of data inputs we process outside of
> dollar amount raised. For example, how many people choose to unsubscribe
> from our list or submit an abuse complaint when we send an email? Does a
> certain subject line get very high opens but a low rate of donations per
> open--indicating that it is more clickbait than effective content? How much
> and what kind of feedback is our Donor Services team getting?
>
> We watched these inputs closely while sending this subject line to donors.
> Our unsubscribe and abuse rates were low, the donation per open rate was
> even higher than usual, and while our Donor Services team flagged some
> negative responses from donors, they determined these comments were not in
> a significant volume.
>
> That said, there is a final input which is harder to measure on a per-test
> basis: how do we, our colleagues, and volunteers feel about our messaging?
> This team cares deeply about Wikipedia and the Wikimedia Foundation, and
> about the mission we all work to achieve. We want to represent it
> faithfully, and do so in a way our readers and donors can engage with and
> understand. This balance can be really hard to strike and it will always be
> an ongoing challenge in our work.
>
> We are grateful to be presented with this challenge and with the joy of
> telling millions of people about this movement. Thank you for caring so
> deeply, for all your contributions, and for keeping this feedback loop
> alive.
>
> Sincerely,
> Caitlin
>
> On Wed, Nov 14, 2018 at 1:47 PM Pine W <[hidden email]> wrote:
>
> > Hi Seddon,
> >
> > While the fundraising appeal may be successful, the problem that I have
> > with this subject line is that it can mislead readers into thinking that
> > someone with the ability to do so is seriously considering, or making an
> > effort to, delete Wikipedia in entirety. I think that a subject line of
> > "Block Wikipedia?" might be okay, and I am supportive of encouraging
> people
> > not to take Wikipedia for granted. But regarding "Delete Wikipedia?", as
> > far as I know that generally misrepresents the current situation. I
> believe
> > that using "Delete Wikipedia?" as a subject line is inconsistent with
> > Wikipedia's goals of providing neutral, verifiable, and reliable
> > information.
> >
> > I am starting to think that if WMF wants to use the Wikipedia brand name
> > for WMF fundraising then WMF should first publicly discuss its proposed
> > uses of the Wikipedia brand name with Wikipedians.
> >
> > On a related issue, I don't know if it's happening this year, but in the
> > past another concern that I've had is the conflation of donating to
> > Wikipedia with donating to WMF. Wikipedia and WMF are related but there
> is
> > not a 1:1 relationship, and I hope that WMF makes that clear in its
> > fundraising. The use of "Delete Wikipedia?" reminds me of these concerns.
> >
> > I would prefer to avoid diverting the community's limited time into
> > reviewing WMF's choices, but unfortunately the issues are too significant
> > to ignore. I don't know how many community members want to volunteer
> their
> > time to review fundraising appeals before they go into production, but I
> > think that it would be good for WMF to ask.
> >
> > Pine
> > ( https://meta.wikimedia.org/wiki/User:Pine )
> > _______________________________________________
> > Wikimedia-l mailing list, guidelines at:
> > https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and
> > https://meta.wikimedia.org/wiki/Wikimedia-l> > New messages to: [hidden email]> > Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l,
> > <mailto:[hidden email]?subject=unsubscribe>
>
>
>
> --
> Caitlin Cogdill
> Senior Fundraising Email Manager
> Wikimedia Foundation
>
> Imagine a world in which every single human being can freely share in
> the sum of all knowledge. Help us make it a reality!
>
> *https://donate.wikimedia.org <https://donate.wikimedia.org/>*
> _______________________________________________
> Wikimedia-l mailing list, guidelines at:
> https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and
> https://meta.wikimedia.org/wiki/Wikimedia-l> New messages to: [hidden email]> Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l,
> <mailto:[hidden email]?subject=unsubscribe>

Interestingly a thread was also launched this week on Wikimedia France
on the same topic of communication within fund-raising campaign (the one
for Wikimedia France)[1]. The main difference with this thread is that
it was launched by Amélie Cabon, the chapter fundraising officer, asking
feedback on the communication material she produced so far. Similar
discomforts were raised on punchline wordings that, citing how a
misleading statement was evoking fakenews. I don't know how the email
"Delete Wikipedia?" redaction was realized, with or without community
feedback, if feedback was requested and given, it would be interesting
to give references.

Personnaly, I think that "Deleting Wikipedia?" is fine as a punchline,
if it does the job of catching and keeping attention toward a detailed
explanation of what is the campaign about and make clear and and
truthful statements of the expected outcomes depending on the campaign
results (and possibly to make situation even clearer, things that are
not depending on campaign results). So to my mind, as long as the
punchline let open an honest development without contradiction, what
matter is the full text.

> Hey Pine,
>
> I appreciate and understand your feedback about this subject line.
>
> For some time we have been trying to find an alternative subject line to
> -/This is a little awkward/-. That line works and works very well but we
> have found it very difficult to effectively translate and adapt into other
> languages, and despite our best efforts have struggled to find an
> alternative.
>
> We first tested -/Deleting Wikipedia?/- as a subject line a couple of weeks
> ago and it was the only winning variant in hundreds of tests. We retested
> in case it was a statistical fluke and continued to see it perform well.
> The effectiveness of this subject line for the most part does not come from
> its apparent clickbaiting. The change in the number of people opening the
> emails was relatively small and unsubscribes remained extremely low. The
> big driver in terms of its success was from a significant increase in those
> people who opened AND read our email appeal. We posed a question and donors
> were motivated to donate when presented with the idea of imagining a world
> without Wikipedia.
>
> Our motivation behind this sort of subject line is the fact that in three
> countries today it is already as if Wikipedia does not exist. The risk that
> this could happen in more countries is greater now that it ever has been.
> Censorship, impediments to free speech and over regulation of copyright are
> threats that Wikipedia, Wikimedia and its communities face every single day
> and it is with that context that we want to lead.
>
> Any email that included this subject line came with at least some context
> to flesh out the idea, i.e., “If Wikipedia were deleted, it would be a
> great loss to the world,”, but going forward it is our full intention to
> make even clearer that we intend for the donor to imagine a world without
> Wikipedia and the threats it faces every day, not threaten that it is going
> away.
>
> Our plan is to continue to testing on this theme, exploring censorship and
> copyright restrictions as well as our increasing role as the backbone of
> knowledge on the internet, and help donors see that knowledge can and is
> threatened all the time. We are definitely and eagerly open to any
> feedback, suggestions and ideas you might have.
>
> Best,
> Seddon
>
> On Tue, Nov 13, 2018 at 11:04 PM Pine W <[hidden email]> wrote:
>
>> There is a report from a user on Jimbo's talk page on ENWP regarding
>> receiving fundraising a fundraising email with the subject line "[NAME] -
>> Deleting Wikipedia?"
>>
>> In previous years I've disagreed with some of WMF's fundraising choices,
>> and it would be unfortunate if in the era of "fake news" becoming
>> mainstream WMF chooses to continue to be a part of the problem. If this is
>> happening then I request that WMF put a stop to it. Regardless of how
>> effective it is to send misleading fundraising appeals and that WMF has
>> received minimal repercussions for doing so over the years, it's wrong and
>> it should stop.
>>
>> Pine
>> ( https://meta.wikimedia.org/wiki/User:Pine )
>> _______________________________________________
>> Wikimedia-l mailing list, guidelines at:
>> https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and
>> https://meta.wikimedia.org/wiki/Wikimedia-l>> New messages to: [hidden email]>> Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l,
>> <mailto:[hidden email]?subject=unsubscribe>
>
>

Re: [Wikimedia-l] Subject lines for WMF fundraising emails

Thank you for your feedback. Is there a comprehensive list of data
inputs which are processed, and maybe even more extensive information on
your processes? That might come as an interesting source of inspiration
for other stakeholders in the movement, whether to provide more
feedback or feed their own processes.

Cheers

Le 16/11/2018 à 18:16, Caitlin Cogdill a écrit :

> Hi all,
>
> Thank you for sharing your concerns. We hear them and we take them
> seriously. *As of today, we have pulled this subject line from our testing
> rotation.*
>
> On the Fundraising team, we pride ourselves on making data-driven
> decisions, and there are many types of data inputs we process outside of
> dollar amount raised. For example, how many people choose to unsubscribe
> from our list or submit an abuse complaint when we send an email? Does a
> certain subject line get very high opens but a low rate of donations per
> open--indicating that it is more clickbait than effective content? How much
> and what kind of feedback is our Donor Services team getting?
>
> We watched these inputs closely while sending this subject line to donors.
> Our unsubscribe and abuse rates were low, the donation per open rate was
> even higher than usual, and while our Donor Services team flagged some
> negative responses from donors, they determined these comments were not in
> a significant volume.
>
> That said, there is a final input which is harder to measure on a per-test
> basis: how do we, our colleagues, and volunteers feel about our messaging?
> This team cares deeply about Wikipedia and the Wikimedia Foundation, and
> about the mission we all work to achieve. We want to represent it
> faithfully, and do so in a way our readers and donors can engage with and
> understand. This balance can be really hard to strike and it will always be
> an ongoing challenge in our work.
>
> We are grateful to be presented with this challenge and with the joy of
> telling millions of people about this movement. Thank you for caring so
> deeply, for all your contributions, and for keeping this feedback loop
> alive.
>
> Sincerely,
> Caitlin
>
> On Wed, Nov 14, 2018 at 1:47 PM Pine W <[hidden email]> wrote:
>
>> Hi Seddon,
>>
>> While the fundraising appeal may be successful, the problem that I have
>> with this subject line is that it can mislead readers into thinking that
>> someone with the ability to do so is seriously considering, or making an
>> effort to, delete Wikipedia in entirety. I think that a subject line of
>> "Block Wikipedia?" might be okay, and I am supportive of encouraging people
>> not to take Wikipedia for granted. But regarding "Delete Wikipedia?", as
>> far as I know that generally misrepresents the current situation. I believe
>> that using "Delete Wikipedia?" as a subject line is inconsistent with
>> Wikipedia's goals of providing neutral, verifiable, and reliable
>> information.
>>
>> I am starting to think that if WMF wants to use the Wikipedia brand name
>> for WMF fundraising then WMF should first publicly discuss its proposed
>> uses of the Wikipedia brand name with Wikipedians.
>>
>> On a related issue, I don't know if it's happening this year, but in the
>> past another concern that I've had is the conflation of donating to
>> Wikipedia with donating to WMF. Wikipedia and WMF are related but there is
>> not a 1:1 relationship, and I hope that WMF makes that clear in its
>> fundraising. The use of "Delete Wikipedia?" reminds me of these concerns.
>>
>> I would prefer to avoid diverting the community's limited time into
>> reviewing WMF's choices, but unfortunately the issues are too significant
>> to ignore. I don't know how many community members want to volunteer their
>> time to review fundraising appeals before they go into production, but I
>> think that it would be good for WMF to ask.
>>
>> Pine
>> ( https://meta.wikimedia.org/wiki/User:Pine )
>> _______________________________________________
>> Wikimedia-l mailing list, guidelines at:
>> https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines and
>> https://meta.wikimedia.org/wiki/Wikimedia-l>> New messages to: [hidden email]>> Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l,
>> <mailto:[hidden email]?subject=unsubscribe>
>
>

Re: [Wikimedia-l] Subject lines for WMF fundraising emails

> That said, there is a final input which is harder to measure on a per-test
> basis: how do we, our colleagues, and volunteers feel about our messaging?
> This team cares deeply about Wikipedia and the Wikimedia Foundation, and
> about the mission we all work to achieve. We want to represent it
> faithfully, and do so in a way our readers and donors can engage with and
> understand. This balance can be really hard to strike and it will always be
> an ongoing challenge in our work.

Caitlin, thank you for caring about the concerns that were raised and
taking action to address them. I can imagine that it is difficult to know
how volunteers feel about some action or message. It would be nice if in
case of doubt there would be some invitation to participate assessing the
lines or suggesting alternatives. I'm sure that in this list there are some
people who would be willing to offer their perspective on how the message
comes across. As Pine has suggested a public discussion on how to
incorporate community review would be interesting.