Wikidata

Wikidata:Requests for permissions/Bot/Soweego bot 2

< Wikidata:Requests for permissions‎ | Bot

The following discussion is closed. Please do not modify it. Subsequent comments should be made in a new section. A summary of the conclusions reached follows.

 Approved --Lymantria (talk) 17:42, 14 November 2018 (UTC)[reply]

Soweego bot, task 2 edit

Soweego bot (talk • contribs • new items • new lexemes • SUL • Block log • User rights log • User rights • xtools)
Operator: Hjfocs (talk • contribs • logs)

Task/s: validation of existing identifier statements, as part of the m:Grants:Project/Hjfocs/soweego Wikimedia project grant. This bot partners with Reinheitsgebot, operated by Magnus_Manske for mix'n'match.

Code: User:Hjfocs/soweego_bot.py, from [1]

Function details: this is the second soweego task. For each target catalog as per m:Grants:Project/Hjfocs/soweego/Timeline#August_2018:_big_fishes_selection, run link-based validation over identifier statements that already exist in Wikidata and perform 3 types of edit:

  1. add or reference identifier statements based on the links available in the target catalog entry;
  2. add or reference described at URL (P973) statements based on the links available in the target catalog entry;
  3. deprecate identifier statements that do not share any link with the target catalog entry.

Identifier addition/referencing example:

URL addition/referencing example:

Deprecation example:

soweego partners with Magnus_Manske's work on large catalogs and complements the mix'n'match tool, which focuses on small catalogs.

100 test edits: see [2]

Estimated total edits: will depend on the available links of the target catalog. For instance, we count 9,832 statements to be added/referenced and 21,475 identifier statements to be deprecated for Discogs (Q504063).

Suggestions for a more detailed reference node welcome.

--Hjfocs (talk) 18:09, 5 November 2018 (UTC)[reply]

I am ready to approve this task in a couple of days, provided that no objections are raised. Lymantria (talk) 14:36, 12 November 2018 (UTC)[reply]
The above discussion is preserved as an archive. Please do not modify it. Subsequent comments should be made in a new section.

Feedback edit

@Jura1: thank you very much for your feedback. I have posted a reply here to avoid further modification of this archived page: User_talk:Jura1#Thanks_for_your_feedback_on_User:Soweego_bot_task_2


Dear Jura1,

In Wikidata:Requests_for_permissions/Bot/Soweego_bot_2, you mentioned 2 important points. Let me explain.

  1. IMDb: the bot sees raw URLs from target sources like MusicBrainz (Q14005) and tries its best to convert them to known external identifiers. To achieve so, it attempts to match each given input URL against all formatter URL (P1630) of external identifier properties, and to extract the correct identifier through format as a regular expression (P1793). This is done via SPARQL queries. Unfortunately, IMDb ID (P345) seems to have an exotic formatter URL marked as preferred statement: https://tools.wmflabs.org/wikidata-externalid-url/?p=345&url_prefix=https://www.imdb.com/&id=$1. It will never match an IMDb input URL, hence the reason why you are seeing them added as is. Do you have any suggestions to avoid this? Of course, consider that building a custom rule for each exception is not a sustainable solution;
  2. official website (P856) VS described at URL (P973): I totally understand this point and will implement an extra check for official website (P856) values.

Thanks again for your precious comments.

Cheers,
Hjfocs (talk) 09:46, 19 November 2018 (UTC)[reply]

Sounds good. BTW, I added the commment above as well as it might be easier to find for other people. Further, I included the recommended "new section" header. --- Jura 06:49, 20 November 2018 (UTC)[reply]
Retrieved from "https://www.wikidata.org/w/index.php?title=Wikidata:Requests_for_permissions/Bot/Soweego_bot_2&oldid=837674127"