First thing we did after initial talks with the newsroom executives was to deploy surveys (before and shortly after the moderation procedure began) dedicated to all journalists in the newsroom in order to understand their relationship with comments and attitudes towards them.
Although the questionnaires we used went beyond the scope of just informing us on the views about comments and moderation in the newsroom, one could do with just a couple of big themes related to the subject, using free online tools to devise and implement a small survey and get journalists involved in the process.
We expand on a couple of examples of possible questions and main themes:
Some questions could test their views on engaging with the readers and how they view their online audience (on a scale from 0 to 10, where 0=completely untrue and 10=completely true):
- How true it is that in the newsroom we think interaction with readers is just as important as the other journalistic activities?
- How true it is that for journalists today interaction with readers should be just as important as writing articles?
Other questions could interrogate their habit of writing and reading comments:
- How often do you read the comments to your own articles?(With the options: several times a day, all my articles, majority of my articles, half of my articles, a few times a year, never).
- How often do you read the comments for the other articles on our website/on other mass media websites? (With the answer options: several times a day, every day, every week, several times a year, never).
- Is reading the comments one of the first things you do in the newsroom? / Is it one of the most frequent things you do? (Yes/No).
- Do you read comments as frequently as you read the news? / Is reading comments as necessary as reading the news? (Yes/No).
Further on, other questions could shed light on how the journalists perceive comments and their utility (on a scale from 0 to 10, where 0=not at all important and 10=very important):
- How important are online comments:
- as a way to evaluate the value of an article?
- to generate ideas for other articles?
- as feedback mechanism if you made an error in the article?
- as an indication of public opinion on a topic?
- to hear people’s opinions?
- to get useful and interesting information from the readers?
- Have you ever thought of a subject for an article and then let it go because you anticipated disapproving comments? (Yes/No)
- Has it ever happened to you that an editor changed something in your article to avoid problems in the online comments?(Yes/No)
- Have you ever discovered a perspective that you had not thought about before with the help of online comments?(Yes/No)
- Have you ever got some new information from the comments that you could use in an article?(Yes/No)
- Have you ever got a new idea for an article from the online comments?(Yes/No)
Another batch of questions could focus on their perceived effects of comments on the whole news ecosystem, inside and outside the newsroom and also serve as further evidence to back up decision to introduce online moderation and engagement procedures (on a scale from 0 to 10, where 0=completely untrue and 10=completely true):
- How true is it that the comments distract attention from the content of the article?
- How true is it that comments can change the reader’s perception/interpretation of the article?
- How true it is that those who give interviews are preoccupied by the comments and they are influenced by the comments in their answers?
Yet other questions could focus on their beliefs about comments and how to deal with them, if at all:
- How true it is that comments are invariably flooded with racist, violent/brutal and intolerant language?(on a scale from 0 to 10, where 0=completely untrue and 10=completely true)
- Readers are more willing to comment in a civil manner if a journalist is involved in the discussion.(on a scale from 0 to 10, where 0=completely disagree and 10=completely agree)
- Online comments should be (choose the option closer to the respondents opinion):
- not allowed at all
- allowed but moderated
- moved to social media
- allowed with no restrictions
And finally one could test the journalists’ views on moderation (on a scale from 0 to 10, where 0=completely disagree and 10=completely agree):
- We would have more readers and more commenters if we did not tolerate hate speech and intolerance in online comments
- Moderation would be against our principles and freedom of expression
- Effective moderation would be prohibitively expensive
- Even if cost was not an obstacle, we still couldn’t with certainty separate hate speech and prejudice from other aggressive comments
- Online comments should not be anonymous
*The stats mentioned for the possible questions one might ask in a journalist survey, reflect the actual data we gathered from the first survey we implemented on the journalists in the GSP newsroom (before the actual start of the moderation procedure), one of the media partners in the Less Hate, More Speech project. 37 GSP journalists took the survey, which had a 100% response rate, and it was implemented between December 2014 and January 2015. Where questions from the first survey were repeated in the third or last survey of the newsroom we mention the comparative data. The third and last survey took place between February – March 2017, two years from the initial one.
One could repeat these questions or add further ones (for example related to whether they have seen/felt changes since the moderation was implemented) some time down the line, when it feels necessary. And it is important to take into consideration the key role the moderators’ team plays and engage them as well. See how in the next guide. —>
Start with the goals and principles for moderating the comments produced by the online community you cater for