Wikidata:Item quality campaign

This page is a work in progress, not an article or policy, and may be incomplete and/or unreliable.
Please offer suggestions on the talk page.

One of our challenges in Wikidata is improving our data quality. We always strive to improve our data quality, so that we can make it usable for more use cases.
One way to improve the data quality is applying machine learning to automatically evaluate item quality. We plan to train machine learning algorithms to predict the quality of items automatically. As a result, it could pave the way to diverse use cases which result in the increase of Wikidata data quality. In order to train a machine learning model, we'll need to label a set of items by their quality level. We're using Wiki labels to gather labels for a random sample of items and we need your help.

To get started, add your name to the signup list, then navigate to labels.wmflabs.org/ui/wikidatawiki. Afterwards, click "Request Workset" and click "Open" to start grading item quality. You might need to refer to Wikidata:Item quality in order to determine the quality for each item. The criteria on Wikidata:Item quality was mainly formulated upon Wikidata:Showcase items, in addition to the discussion with the community.

Progress edit

Pilot edit

The pilot campaign has been finished. The analysis result can be found here.

Full campaign edit

We have launched the full campaign to label item quality. There are ~5k items that have to be labeled.

Item quality (5k sample): 100% complete

(get detailed statistics directly from the server)

List of volunteers edit

Add your name here to get pinged with updates.

Glorian WD (talk • contribs • logs)
EpochFail (talk • contribs • logs)
GerardM (talk • contribs • logs)
Abián (talk • contribs • logs)
QZanden (talk • contribs • logs)
Lymantria (talk • contribs • logs)
Jsamwrites (talk • contribs • logs)
Alessandro Piscopo (talk • contribs • logs)
ChristianKl (talk • contribs • logs)
Adelheid Heftberger (talk • contribs • logs)
Mvolz (talk • contribs • logs)
Tubezlob (talk • contribs • logs)
Reem Al-Kashif (talk • contribs • logs)

What will be possible in the future? edit

Once the system is working we can:

see if and how the quality of items on Wikidata is improving over time. This will help us better make the case to Wikipedians for example that we are improving Wikidata's data.
find areas where we are lacking compared to other areas of Wikidata. This will allow us to do dedicated editathons, campaigns and data partnerships.
see where we are particularly good in order to showcase our best areas.
create worklists for wiki projects to help them find the items in their area of expertise that could use most help.
create work groups by quality level focusing effort
assess quality improvement over time
...

Caveats edit

Data quality on Wikidata is a combination of many factors. We have so far always been judged by the number of references to non-Wikimedia sources. That is a good but limited metric. With ORES we are able to expand our understanding of data quality on Wikidata. It will still not give us the full picture however and is not intended to. Additional tools will need to be developed to cover things like the actual verifiability of a data point based on the reference we have. No single tool can give us all metrics. We need to get closer to them tool by tool.