Wikidata:Pywikibot - Python 3 Tutorial
![]() | Warning: pywikibot currently does not support lexical data objects like Lexemes (phab:T189321). Use WikibaseIntegrator or LexData to edit those instead. |
This is an update of the tutorial Wikidata:Creating a bot. But it is also an extension into a fully independent short-course that will show you how to work with Python 3, Wikidata and Pywikibot to handle large datasets.
Most of the text is rewritten to be more expressive and accommodating to people with little or no programming experience. The reader's interest in Wikidata might just be the reason that they want to learn programming.
In order to follow the course, you should have a few days or ideally a few weeks of Python 3 experience. The goal is to reach a level where you can run a bot to gather and edit information and transfer the information between other tools of your workflow.
Using Python and Pywikibot will allow the whole tutorial to be in one programming language. A language that is not only easy to learn, but also happens to be one of the most-used languages in the data-sciences. Focusing on Python 3 (emphasis on the version) will allow the tutorial a much more native approach to the multi-lingual nature of Wikidata.
Anyone can edit this tutorial or ask questions on the different talk-pages.
Installation
editYou can develop bots either online or offline.
Online, in your web browser (PAWS)
editThis is the recommended option for most users:
- https://hub-paws.wmcloud.org/ - A quick overview and setup of PAWS can be found on the MediaWiki website, while full documentation is on wikitech.
Offline, installed on your computer
edit- Setting up Shop: Installation, setup
Introduction
editThis learning block focuses on setting up pywikibot, querying data, — using functions, iteration and generators, — and writing data to a file or database.
- Data Harvest: Gathering data from one Wikidata-item
- Winter Storage: Writing data into a file or database
- Big Data: Iterating over large sets of items
After finishing the first three chapters award yourself the {{User Pywikibot basic}} badge. Congratulations!
This user has basic understanding of the Pywikibot software library. |
Editing Wikidata
editThis learning block focuses on writing to Wikidata and comparing values between Wikipedia and Wikidata.
- Information about bot editing
- Labels: Setting labels, descriptions and aliases.
- Setting sitelinks
- Setting statements
- Changing Items: Correcting claims that link to a wrong item.
- Quantities and Units: Add a quantity claim with uncertainty and units.
- Setting qualifiers
- Setting sources
After finishing the 2nd block award yourself the {{User Pywikibot advanced}} badge. Congratulations!
This user has an advanced understanding of the Pywikibot software library. |
Expert topics and examples
editThis block consists of advanced examples that go beyond iterating over Wikidata and getting or setting statements. This includes for example the templates that are used on Wikidata. You can just read and try out the chapters that you are interested in.
- Templates: Examples how to look for certain templates, statements within them, and editing them.
- Templates, Generators, Tables: Iterate over template usage, and automatically create wikitables
- Talk Page Messages: Learn how to place messages on talk pages.
- Gathering data from Arabic-Wikipedia: Learn how to get and compare data from wikis that are written from right to left.
- Iterate over a SPARQL query: This lesson shows how to iterate over a SPARQL query.
- Page views: Get the page views from items and linked pages
- Create Items: Bots can create missing items
- Merging items
- Get the sum of pages in a category or namespace
- Upload images and write pages on Commons
- Structured Data on Commons (SDC)
- Lexeme
- EntitySchema
- Machine learning with Wikidata
- More ideas needed
- ...
- ..
- .
After finishing the 3rd block, or if you are already experienced with running bots on Wikidata, you can award yourself the {{User Pywikibot professional}} badge. Congratulations!
This user is a Pywikibot expert. |
Helpful Resources
edit- Wikidata:SPARQL query service: resources about Wikidata Query Service (Q20950365)
- https://se.wikimedia.org/wiki/Projekt:Botacademy_2015: A bot workshop in Sweden (28-29 November 2015)
- Wikidata:Creating a bot: An introduction to pywikibot and other bots.
- Attende code examples from session on pywikibot on PAWS at Wikimania 2016
- https://wikitech.wikimedia.org/wiki/PAWS