Wikipedia:Wikipedia Signpost/2018-02-20/Recent research
This is a draft of a potential Signpost article, and should not be interpreted as a finished piece. Its content is subject to review by the editorial team and ultimately by JPxG, the editor in chief. Please do not link to this draft as it is unfinished and the URL will change upon publication. If you would like to contribute and are familiar with the requirements of a Signpost article, feel free to be bold in making improvements!
|
A monthly overview of recent academic research about Wikipedia and other Wikimedia projects, also published as the Wikimedia Research Newsletter.
Political diverse editors write better articles, but central topics are less likely to achieve featured article quality
Politically diverse editors and article quality
While politics in the United States appears to be increasingly polarized around extremes in political discourse, it was unclear how this affected the open, collective production of knowledge that is Wikipedia.
The researchers used a data dump of English Wikipedia from 12/1/16, including all edits made since its start within the domains of politics, social issues, and science. They focused on the “American liberalism” and “American conservatism” categories and sub-categories as delimiters, with breakdowns in social issues and science down four levels from the root. The researchers reached out to the Wikipedia community, Wikimedia staff, and those who directly inquired on the page they created through Meta-Wiki, with 118 responses overall for their survey. The researchers then analyzed user edits to determine political alignment based on contributions to conservative or liberal articles.
The researchers found that “articles attracting more attention tend to have more balanced engagement from editors along the conservative-liberal spectrum” (p. 4). They then measured the quality of articles using a tool developed by Wikimedia research staff (ORES), and determined that higher political polarization was associated with higher article quality. All this fed into their study goals of exploring the relationship between diversity of political alignment and article quality and bias. Through their statistical analysis, they determined that the quality of articles in Wikipedia improves when editors on both sides of politically polarized issues work together to seek collaborative consensus on topics. While this research was directly focused on politically-related topics, it surfaced both a need for political diversity and for motivated contributors.
(Cf. related earlier coverage: Being Wikipedian is more important than the political affiliation", Cross-language study of conflict on Wikipedia")
The study of controversy
- "Computing controversy: Formal model and algorithms for detecting controversy on Wikipedia and in search queries"[2]
- Reviewed by Barbara Page and Tilman Bayer
This paper presents a "method for automatic detection of controversial articles and categories in Wikipedia", based on three data sources:
- Ratings submitted by readers via the Article Feedback Tool (AFT) in 2011 and 2012
- The list at Wikipedia:List of controversial issues (manually maintained by Wikipedia editors)
- A sample of 512 sections drawn randomly from the talk pages of articles on that list ("Surprisingly, only 19.5% of the sections turned out to be controversial").
The researchers argue that applying a mathematical model to Wikipedia talk page controversies has the potential of incorporating a 'controversy' metric in web-searches. This should give those searching for information on a topic to quickly assess controversial topics. Wikipedia provides researchers with accessible and historical controversial discussions. The authors further describe their work: "[Assessing] the controversy should offer [readers] a chance to see the 'wider picture' rather than letting [them] obtain one-sided views." The authors' conclusions were: "Our approach can be also applied in Wikipedia or other knowledge bases for supporting the detection of controversy and content maintenance. Finally, we believe that our results could be useful for...understanding the complex nature of controversy..."
Students edit but still doubt the value of Wikipedia
- "Wikipedia in higher education: Changes in perceived value through content contribution"[3]
- Reviewed by Barbara Page
Students are a convenient group to study, especially if being studied is part of the syllabus. The 240 students in this study readily admitted to using Wikipedia as a resource even though they did not be consider it to be 'reliable and trustworthy'. Using Wikipedia as a resource does not necessarily encourage content contributions by students. In addition, when the students in this study actually added content, their perceptions of the reliability and usefulness of Wikipedia did not change.
Researching the research
- "Excavating the mother lode of human-generated text: A systematic review of research that uses the Wikipedia corpus"[4]
- Reviewed by Barbara Page
The amount of research that uses Wikipedia as a source of data continues to grow and enough scholarly content now exists that systematic reviews are available. Computer science has especially been quick to see the potential of this 'mother lode' and how it can be used to study information retrieval, natural language processing, and ontology building. The reference section in this article itself makes interesting reading if only to appreciate the collection of data sets and other research that that exists and continues to expand.
Sneaky editing and masking bias
- "Persistent Bias on Wikipedia: Methods and Responses"[5]
- Reviewed by Barbara Page
Apparently, Wikipedia editors are not the only ones who have observed biased editing. The author of this research article (already mentioned in a previous issue) used his own article as a case study and example of biased editing. It is no surprise that an editor can 'nominally' follow editing guidelines to maintain their bias. Here is the 'how to' on such behavior:
- deleting positive material
- adding negative material
- using a one-sided selection of sources
- exaggerating the significance of references and topics
Those who are biased sometimes support their editing even in 'the face of resistance'. This is done by:
- reverting edits
- selectively invoking Wikipedia rules
- overruling (bullying?) resistant editors
When bias is challenged by other editors, the strategies for dealing with it is making complaints, 'mobilizing counterediting', and exposing the bias. The authors' stinging conclusion speaks for itself: "It is worthwhile becoming aware of persistent bias and developing ways to counter it in order for Wikipedia to move closer to its goal of providing accurate and balanced information."[6]
Seeking credibility
- "Information Fortification: An Online Citation Behavior"[7]
- Reviewed by Barbara Page
This study is a rebuttal to another position paper by Forte and Bruckman related to the use of citations on Wikipedia. Citing sources is associated with other issues of bias and identifies the patterns used to in citing sources to encourage and even fabricate controversy. This study was limited to non-scientific topics and used data derived from edit logs, interviews and text analysis. "[I]nformation fortification [is] a concept that explains online citation activity that arises from both naturally occurring and manufactured forms of controversy."
Anti-vandalism on Wikidata
- Reviewed by Barbara Page
Vandalism of Wikidata can have significant disruptions in the use of the data leading to flaws in the analysis of such data. Collaborative efforts continue to address these concerns and included some friendly 'competitions'. Strategies for 'fighting' vandalism at this time include manual review, community feedback, and analyzing reverting patterns. Other 'vandalism' fighting tools are being developed. Interesting is the discussion about the effort to use "psychologically motivated features capturing a user’s personality and state of mind..." [8]
Articles receiving the most attention (by editors) overall lack the depth of quality found in featured articles
- "Knowledge categorization affects popularity and quality of Wikipedia articles"[9]
- Reviewed by FULBERT
This empirical research paper explored how knowledge categorization, common in classification systems within the information sciences, works as a scientific and social process in how Wikipedia articles are attended to by editors. Categorization leads to nesting of information under major topics, and the further down a hierarchy, the less editing attention articles appear to garner. Articles higher in the hierarchy are referred to as coarse-grained, and while these receive the most attention, their levels of quality have not been the focus of previous studies.
The researchers analyzed a database dump of the English-language Wikipedia from October 20, 2016, considering all articles that were members of at least one category (n=5,006,601). They defined granularity as the length of the shortest path from the root (main category), which averaged 7.59 across all articles, which they then compared to the number of article edits (which related to preception of higher quality articles), the number of articles as rated by importance (done individually by WikiProjects), perceptions of quality (based on being classified as a featured article), and the notion of return on effort (quality of an article relative to the amount of work done on it by editors). They conducted non-parametric and parametric statistical analyses using numerous variables based on the many article records through their data dump.
There were many levels of findings, with the main one being that coarse-grained categories (those nearest the top of the hierarchies) received the most number of edits and attention from editors, though they were least likely to be featured (highest quality) articles. This seemed to surprise the authors, as it means that those articles that receive the most attention (by editors) overall lack the depth of quality found in featured articles, most of which are further down the hierarchy.
Conferences and events
See the research events page on Meta-wiki for upcoming conferences and events, including submission deadlines.
Other recent publications
Other recent publications that could not be covered in time for this issue include the items listed below. contributions are always welcome for reviewing or summarizing newly published research.:Compiled by [[User:|User:]]
- "Time-focused analysis of connectivity and popularity of historical persons in Wikipedia"[10] From the abstract: "Our study sheds new light on the characteristics of information about historical people recorded in the English Wikipedia and quantifies user interest in such data. We propose a novel style of analysis in which we use signals derived from the hyperlink structure of Wikipedia as well as from article view logs, and we overlay them over temporal dimension to understand relations between time periods, link structure and article popularity."
- "Patients are content with Dr. Google"[11] (Article in German, translated title: "Health information: "He who searches, will find - Patients are content with Dr. Google") 72% of patients in Germany consult "Wikipedia and other online encyclopedias" for health information. 54% find Wikipedia "trustworthy".
- "Usage of Wikipedia by health science and social sciences & humanities undergraduates of University of Peradeniya and SouthEastern University of Sri Lanka"[12] From the paper: "[Survey] participants were given five options to indicate how they use information services and technologies in searching for their study requirements. The study show that 79% of total students accessWikipedia and 78% use Google to find their information followed by Google scholar (60%), library (57%), and other databases 43% . 69% of FAC and 67% of FAHS students mentioned Wikipedia as their first source of information while embarking into searching for information. 79% students of both faculties use Wikipedia as the first source, while 69% use it as the only source. [...]. According to the outcome of the study, Wikipedia was crowned as the first source and most common source when undergraduates seeking for information."
- "Emo, Love, and God: Making Sense of Urban Dictionary, a Crowd-Sourced Online Dictionary [Wiktionary]"<ref>Nguyen, Dong; McGillivray, Barbara; Yasseri, Taha (2017-12-22). "Emo, Love, and God: Making Sense of Urban Dictionary, a Crowd-Sourced Online Dictionary". arXiv:1712.08647 [cs]. Conclusions: "The lexical content of UD [Urban Dictionary] is radically different from that of Wiktionary, another crowd-sourced, but more highly moderated dictionary. In general, we can say that the overlap between the two dictionaries is small. Considering all unique UD headwords that are not found in Wiktionary, we found that this number is almost three times the number of headwords that uniquely occur in Wiktionary. However, if we exclude words with only one definition in UD (which tend to be infrequent or idiosyncratic words), we found the opposite pattern, with Wiktionary-only headwords amounting to almost three times the UD-only headwords."
References
- ^ Shi, Feng; Teplitskiy, Misha; Duede, Eamon; Evans, James (2017-11-29). "The Wisdom of Polarized Crowds". arXiv:1712.06414 [cs, stat].
- ^ Zielinski, Kazimierz; Nielek, Radoslaw; Wierzbicki, Adam; Jatowt, Adam (2018). "Computing controversy: Formal model and algorithms for detecting controversy on Wikipedia and in search queries". Information Processing & Management. 54 (1): 14–36. doi:10.1016/j.ipm.2017.08.005.
- ^ Soler-Adillon, Joan; Pavlovic, Dragana; Freixa, Pere (2018). "Wikipedia in higher education: Changes in perceived value through content contribution". Comunicar (in Spanish). 26 (54): 39–48. doi:10.3916/c54-2018-04. ISSN 1134-3478. English version here
- ^ Mehdi, Mohamad; Okoli, Chitu; Mesgari, Mostafa; Nielsen, Finn Årup; Lanamäki, Arto (2017). "Excavating the mother lode of human-generated text: A systematic review of research that uses the wikipedia corpus" (PDF). Information Processing & Management. 53 (2): 505–529. doi:10.1016/j.ipm.2016.07.003.
- ^ Martin, Brian (2017). "Persistent Bias on Wikipedia. Methods and Responses". Social Science Computer Review.
Author's copy
- ^ Martin, Brian (2017). "Persistent Bias on Wikipedia: Methods and Responses" (PDF). Social Science Computer Review.
- ^ Forte, Andrea; Andalibi, Nazanin; Gorichanaz, Tim; Kim, Meen Chul; Park, Thomas; Halfaker, Aaron (2018-01-07). "Information Fortification: An Online Citation Behavior" (PDF). ACM: 83–92. doi:10.1145/3148330.3148347. ISBN 9781450355629.
{{cite journal}}
: Cite journal requires|journal=
(help) - ^ "Overview of the Wikidata Vandalism Detection Task at WSDM Cup 2017" (PDF). 2017; Heindorf S, Potthast M, Engels G, Stein B : Conference paper, the WSDM Cup 2017
{{cite web}}
: CS1 maint: postscript (link) - ^ Lerner, Jürgen; Lomi, Alessandro (2018-01-02). "Knowledge categorization affects popularity and quality of Wikipedia articles". PLOS ONE. 13 (1): 1–22. doi:10.1371/journal.pone.0190674. ISSN 1932-6203.
{{cite journal}}
: CS1 maint: unflagged free DOI (link) - ^ Jatowt, Adam; Kawai, Daisuke; Tanaka, Katsumi (2018-02-08). "Time-focused analysis of connectivity and popularity of historical persons in Wikipedia". International Journal on Digital Libraries. pp. 1–19. doi:10.1007/s00799-018-0231-4. ISSN 1432-1300 1432-5012, 1432-1300.
{{cite web}}
: Check|issn=
value (help) - ^ "Gesundheitsinfos: Wer suchet, der findet – Patienten mit Dr. Google zufrieden". Spotlight Gesundheit, Bertelsmann Stiftung. 2018.
- ^ Dehigama, Kanchana; Jazeel, M. I. M. (2017-12-07). "Usage of Wikipedia by health science and social sciences & humanities undergraduates of University of Peradeniya and SouthEastern University of Sri Lanka".
{{cite journal}}
: Cite journal requires|journal=
(help)
Discuss this story
"The best content is developed through civil collaboration between editors who hold opposing points of view."
-- BullRangifer. From WP:NEUTRALEDITORReferences
{{citation}}
: Italic or bold markup not allowed in:|publisher=
(help){{citation}}
: Italic or bold markup not allowed in:|publisher=
(help){{citation}}
: Italic or bold markup not allowed in:|publisher=
(help)