Wikidata:Pywikibot - Python 3 Tutorial/Templates

Tracking template usage is an important part of keeping Wikidata organized. This chapter will look at how a page can be scanned for templates. This is for example useful for a bot that wants to add or remove templates or track template usage.

Counting constraint templates usage

In this example we will try to iterate over the talk pages of Wikidata properties in order to count the usage of different constraint templates. Again we could again think of a way to preselect certain properties, but for the sake of simplicity we will just use a for-loop with a start and end property ID.

In the for-loop we will call pywikibot.Page(site, "P{}".format(i), 121) which will return the talk page of the property with the ID "i". Next we check if the page does not exist. If that is the case we ouput "Not found" and continue with the next ID in the for-loop. In the other case we get the page content as a string and use the textlib.extract_templates_and_params() function to regex for all the templates in the string. This will return a list of all templates with each entry in the list being a touple of the template-name and an OrderedDict that stores the details of the template.

Now that you know how the for-loop works, try to understand what the count_constraint_templates() function does and then run the whole example:

import pywikibot
from pywikibot import textlib

site = pywikibot.Site('wikidata','wikidata')
repo = site.data_repository()

start = 27
end = 27

def count_constraint_templates(tmpl_list):
    """
    Takes a list of templates, sums usage in dictionary, returns

    This function takes a list of templates, iterates over them
    and counts the various constraint templates. Returns the usage
    in a dictionary where the key is the name of the constraint
    template and the value is an integer of the number of usages.
    """
    tmpl_use = {}

    for tmpl in tmpl_list:
        name = tmpl[0]

        if name.startswith("Constraint") == True:
            if name in tmpl_use.keys():
                tmpl_use[name] = tmpl_use[name] + 1
            else:
                tmpl_use[name] = 1

    return tmpl_use

for i in range(start, end + 1):
    # line below will call the talk page of the respective property
    prop_page = pywikibot.Page(site, "P{}".format(i), 121)
    pywikibot.output("--> {}".format(prop_page.title()))

    if prop_page.exists() == False:
        pywikibot.output("Not found.\n")
        continue

    page_str = prop_page.get()
    tmpl_list = textlib.extract_templates_and_params(page_str)
    tmpl_use = count_constraint_templates(tmpl_list)
    print(tmpl_use)

As you probably figured out, our custom function iterates over the list of templates and only executes additional code when the string of the template-name starts with "Constraint". Then the name of the template is used as the key of the dictionary and the value is set to 1. If the key already exists 1 is added to the current value of the key. Finally the function returns the dictionary and the result is printed at the end of the for-loop. The final output will be (we just iterate over property country of citizenship (P27):

--> Property talk:P27
{'Constraint:Qualifiers': 1, 'Constraint:Item': 3, 'Constraint:Type': 1, 'Constraint:One of': 1, 'Constraint:Value type': 1}

Checking whether a constraint statement is present

Another common task for a bot, is to check whether a template has a certain statement. Instead of looking at the name of the template (in the previous example tmpl[0]) we need to look at the data that is inside the template (similar to the previous example tmpl[1]). This will return a dictionary with keys and values that are both strings. So we have to see what kind of keys the constraint template uses and then we can look for strings in the respective values.

The following example iterates over feast day (P841) and looks for the "Item" constraint. In this template it will check if the statement instance of (P31) = human (Q5) exists:

import pywikibot
from pywikibot import textlib

site = pywikibot.Site('wikidata','wikidata')
repo = site.data_repository()

start = 841
end = 841
cnstrnt = "Item"
prop = "P31" # Instance of
target = "Q5" # Human

def constraint_statement_present(tmpl_list, cnstrnt, prop, target):
    """
    Returns True or False if a template contains a certain statment

    The function takes a template list, a constraint to look for (String),
    a property (String, e.g. "P31") and a target (String, e.g. "Q5").
    It iterates over the list to find the statement and returns True if
    the statement is found at least once.
    """
    prop_exists = False
    target_exists = False
    stmnt_exists = False

    for tmpl in tmpl_list:
        name = tmpl[0]
        dct = tmpl[1]

        if name.endswith(cnstrnt) == True:
            for key in dct:
                if key == "property":
                    prop_exists = prop in dct[key]
                if key == "item":
                    target_exists = target in dct[key]
                if prop_exists and target_exists:
                    stmnt_exists = True

    return stmnt_exists

for i in range(start, end + 1):
    # line below will call the talk page of the respective property
    prop_page = pywikibot.Page(site, "P{}".format(i), 121)
    pywikibot.output("--> {}".format(prop_page.title()))

    if prop_page.exists() == False:
        pywikibot.output("Not found.\n")
        continue

    page_str = prop_page.get()
    tmpl_list = textlib.extract_templates_and_params(page_str)
    stmnt_exists = constraint_statement_present(tmpl_list, cnstrnt, prop, target)
    print("Looking for statement:")
    print("Constraint {}: {} --> {}".format(cnstrnt, prop, target))
    print("Statement on page: {}".format(stmnt_exists))

Running the code will show the result as:

--> Property talk:P841
Looking for statement:
Constraint Item: P31 --> Q5
Statement on page: True

And when we check Property_talk:P841, we can see that indeed that the constraint we are looking for is on that page.