Wikidata:Lav en bot
Denne side forklarer hvordan man laver en bot til Wikidata. Overvej gerne at dele din kode og tilføje nye eksempler og de forbedringer, du ønsker.
Krav
For at skabe botter har du brug for:
- Visse programmeringsfærdigheder (Python, Perl, PHP...)
- En programmeringsplatform (en af platformene nedenfor) og noget programmeringskode, som kan køres for at fuldføre en bestemt opgave
- En botkonto (og en tilladelse til at køre botten)
- En kildekodeeditor (Notepad++, Geany, vi, emacs)
Anbefaling
- Join a Wikidata telegram channel and participate in the discussions (and ask for help if you get stuck programming).
Pywikibot
Advarsel: This bot framework has incomplete support of lexemes as of June 2022. See other libraries below for full support. |
I de næste afsnit lærer du at installere, konfigurere og logge ind ved hjælp af pywikipediabot. Du behøver kun at udføre de første tre trin én gang. I det følgende er der også nogle simple eksempler til at lære det mest grundlæggende om programmering af botter.
Installation
- For flere detaljer om installation af pywikibot, se mw:Manual:Pywikibot/Installation og Wikidata:Pywikibot - Python 3 Tutorial/Setting up Shop
- For at anvende pywikibot uden installation, se mw:Manual:Pywikibot/PAWS
For at installere pywikipediabot:
- Download pywikibot:
- Som en zip-fil
- Eller anvend git-depotet: mw:Manual:Pywikibot/Gerrit
Konfiguration
- :For flere detaljer om konfiguration af pywikibot, se mw:Manual:Pywikibot/user-config.py.
Du skal konfigurere user-config.py-filen med bot-brugernavnet, overprojekt og sprog. For Wikidata har både overprojekt og sprog de samme parametre, wikidata
.
Extended content |
---|
mylang = "wikidata"
family = "wikidata"
usernames["wikidata"]["wikidata"] = 'MyBotName'
|
Du kan formindske forsinkelsen mellem redigeringer ved at tilføje: put_throttle = 1 put_throttle = 1
Log ind
Efter du har konfigureret user-config.py-filen, logger du ind på denne måde:
$ python login.py
Den vil spørge dig om kodeordet til botten, indtast det og tryk enter. Hvis du gjorde det korrekt, skulle du være logget ind nu.
Eksempel 1: Hent data
Dette eksempel henter data for siden, som handler om Douglas Adams. Gem den følgende kildekode i en fil og kør den med python example1.py
Extended content |
---|
import pywikibot
site = pywikibot.Site("en", "wikipedia")
page = pywikibot.Page(site, 'Douglas Adams')
item = pywikibot.ItemPage.fromPage(page)
dictionary = item.get()
print(dictionary)
print(dictionary.keys())
print(item)
|
item.get()
forbinder til Wikidata og henter data. Resultatet er (genformateret for klarhedens skyld):
{ 'claims': { 'P646': [<pywikibot.page.Claim instance at 0x7f1880188b48>], 'P800': [<pywikibot.page.Claim instance at 0x7f1880188488>, <pywikibot.page.Claim instance at 0x7f1880188368>] ... } 'labels': { 'gu': '\u0aa1\u0a97\u0acd\u0ab2\u0abe\u0ab8 \u0a8f\u0aa1\u0aae\u0acd\u0ab8', 'scn': 'Douglas Adams', ... } 'sitelinks': { 'fiwiki': 'Douglas Adams', 'fawiki': '\u062f\u0627\u06af\u0644\u0627\u0633 \u0622\u062f\u0627\u0645\u0632', 'elwikiquote': '\u039d\u03c4\u03ac\u03b3\u03ba\u03bb\u03b1\u03c2 \u0386\u03bd\u03c4\u03b1\u03bc\u03c2', ... } 'descriptions': { 'eo': 'angla a\u016dtoro de sciencfikcio-romanoj kaj humoristo', 'en': 'English writer and humorist', }, 'aliases': { 'ru': ['\u0410\u0434\u0430\u043c\u0441, \u0414\u0443\u0433\u043b\u0430\u0441'], 'fr': ['Douglas Noel Adams', 'Douglas No\xebl Adams'], ... } } ['claims', 'labels', 'sitelinks', 'descriptions', 'aliases'] [[wikidata:Q42]]
It prints a dictionary with keys for
- the set of claims in the page: Property:P646 is the Freebase identifier, Property:P800 is "notable work", etc.
- the label of the item in many languages
- the sitelinks for the item, not just Wikipedias in many languages, but also Wikiquote in many languages
- the item description in many languages
- the aliases for the item in many languages
Then a list with all the keys for the key-values pairs in the dictionary. Finally, you can see that the Wikidata item about Douglas Adams is Q42.
Alternativer
Eksemplet ovenfor henter ItemPage ved brug af artiklen på den engelsksprogede Wikipedia. Alternativt kan man hente ItemPage direkte:
Extended content |
---|
import pywikibot
site = pywikibot.Site("wikidata", "wikidata")
repo = site.data_repository()
item = pywikibot.ItemPage(repo, 'Q42')
|
Eksempel 3: Hent sproglinks
Efter item.get() kan for eksempel webstedlinks hentes. Disse links er links til alle Wikipedia-versioner, som har den artikel.
Extended content |
---|
import pywikibot
site = pywikibot.Site("wikidata", "wikidata")
repo = site.data_repository()
item = pywikibot.ItemPage(repo, 'Q42')
item.get()
print(",".join(item.sitelinks))
|
Resultatet er:
{'fiwiki': 'Douglas Adams', 'eowiki': 'Douglas Adams', 'dewiki': 'Douglas Adams', ...}
With item.iterlinks(), an iterator over all these sitelinks is returned, where each article is given not as plain text as above but already as a Page object for further treatment (e.g., edit the text in the corresponding Wikipedia articles).
Eksempel 4: Angiv en beskrivelse
Dette eksempel angiver en engelsksproget og en tysksproget beskrivelse for emnet om Douglas Adams.
Extended content |
---|
import pywikibot
site = pywikibot.Site("wikidata", "wikidata")
repo = site.data_repository()
item = pywikibot.ItemPage(repo, 'Q42')
item.get()
mydescriptions = {'en': 'English writer and humorist', 'de': 'Keine Panik!'}
item.editDescriptions(mydescriptions, summary='Setting/updating descriptions.')
|
Angivelse af etiketter og aliasser virker på samme måde.
Eksempel 6: Angiv et webstedlink
For at angive et webstedlink kan man enten oprette en tilsvarende tekst, som tilsvarer eksempel 4 eller anvende sideobjekter:
Extended content |
---|
import pywikibot
site = pywikibot.Site("en", "wikipedia")
repo = site.data_repository()
item = pywikibot.ItemPage(repo, 'Q42')
page = pywikibot.Page(site, 'Douglas Adams')
item.setSitelink(page, summary='Setting (/updating?) sitelink.')
|
Eksempel 7: Angiv et udsagn
Udsagn angives ved brug af klassen Påstand. I det følgende angives for Douglas Adams place of birth (P19): Cambridge (Q350).
Extended content |
---|
import pywikibot
site = pywikibot.Site("wikidata", "wikidata")
repo = site.data_repository()
item = pywikibot.ItemPage(repo, 'Q42')
claim = pywikibot.Claim(repo, 'P19')
target = pywikibot.ItemPage(repo, 'Q350')
claim.setTarget(target)
item.addClaim(claim, summary='Adding claim')
|
For andre datatyper virker det på samme måde. I det følgende tilføjes påstande med datatyperne streng (IMDb ID (P345)) og koordinat (coordinate location (P625)) (URL er det samme som streng):
Extended content |
---|
import pywikibot
site = pywikibot.Site("wikidata", "wikidata")
repo = site.data_repository()
item = pywikibot.ItemPage(repo, 'Q42')
stringclaim = pywikibot.Claim(repo, 'P345')
stringclaim.setTarget('nm0010930')
item.addClaim(stringclaim, summary='Adding string claim')
coordinateclaim = pywikibot.Claim(repo, 'P625')
coordinate = pywikibot.Coordinate(lat=52.208, lon=0.1225, precision=0.001, site=site)
coordinateclaim.setTarget(coordinate)
item.addClaim(coordinateclaim, summary='Adding coordinate claim')
|
Eksempel 8: Tilføj en kvalifikator
Qualifiers are also represented by the Claim class. In the following, we add the qualifier incertae sedis (P678): family (Q35409) to the Claim "claim". Make sure you add the item before adding the qualifier.
Extended content |
---|
qualifier = pywikibot.Claim(repo, 'P678')
target = pywikibot.ItemPage(repo, "Q35409")
qualifier.setTarget(target)
claim.addQualifier(qualifier, summary='Adding a qualifier.')
|
Eksempel 8: Tilføj en kilde
Also, sources are represented by the Claim class. Unlike for qualifiers, a source may contain more than one Claim. In the following, we add stated in (P248): Integrated Taxonomic Information System (Q82575) with retrieved (P813) March 20, 2014 as source to the Claim "claim". The claim has to be either retrieved from Wikidata or added to an itempage beforehand.
Extended content |
---|
statedin = pywikibot.Claim(repo, 'P248')
itis = pywikibot.ItemPage(repo, "Q82575")
statedin.setTarget(itis)
retrieved = pywikibot.Claim(repo, 'P813')
date = pywikibot.WbTime(year=2014, month=3, day=20)
retrieved.setTarget(date)
claim.addSources([statedin, retrieved], summary='Adding sources.')
|
Eksempel 10: Sidegeneratorer
TODO
Eksempel 11: Hent underegenskabers værdier
I det følgende hentes underegenskabers værdier fra described by source (P1343) -> Great Soviet Encyclopedia (1969–1978) (Q17378135) -> egenskaber reference URL (P854) og title (P1476).
Extended content |
---|
import pywikibot
site = pywikibot.Site("wikidata", "wikidata")
repo = site.data_repository()
item = pywikibot.ItemPage(repo, 'Q13515')
item.get()
sourcesid = 'P1343'
sourceid = 'Q17378135'
urlid = 'P854'
nameid = 'P1476'
# item.claims['P1343'][1].qualifiers.items(): # This are direct way to get list qualifiers. But '[1]' is hard link to index of list, it will break over time.
if sourcesid in item.claims:
for source in item.claims[sourcesid]:
if source.target.id == sourceid:
s = source.qualifiers
if urlid in s: url = s.get(urlid)[0].target
if nameid in s: name = s.get(nameid)[0].target['text']
print (url, name)
|
Flere eksempler
Nogle brugere deler deres kildekode. Lær mere i de næste links:
- User:RobotMichiel1972/wikidata lowercase.py - pywikipedia-eksempel på hvordan man kan rette etiketten til små bogstaver ved at anvende små/store bogstaver fra den engelske etiket som 'reference' (her implementeret i hård kode kun for nlwiki) til at gennemløbe et udvalg af sider på ens egen wikipedia.
- File:Bots hackathon 2013.pdf viser "claimit.py" og "template_harvest.py", inkluderet i kerneversionen (tidligere omskrevet)
Wikidata Integrator
WikidataIntegrator is a library for reading and writing to Wikidata/Wikibase. We created it for populating Wikidata with content from authoritative resources on Genes, Proteins, Diseases, Drugs and others. Details on the different tasks can be found on the bot's Wikidata page.
Pywikibot is an existing framework for interacting with the MediaWiki API. The reason why we came up with our own solution is that we need a high integration with the Wikidata SPARQL endpoint in order to ensure data consistency (duplicate checks, consistency checks, correct item selection, etc.). Compared to Pywikibot, WikidataIntegrator currently is not a full Python wrapper for the MediaWiki API but is solely focused on providing an easy means to generate Python-based Wikidata bots.
For more information, documentation, download & installation instructions, see here: https://github.com/SuLab/WikidataIntegrator/
Example Notebook
An example notebook demonstrating an example bot to add therapeutic areas to drug items, including using fastrun mode, checking references, and removing old statements:
http://public-paws.wmcloud.org/46883698/example%20ema%20bot.ipynb
WikibaseIntegrator
Forked from Wikidata Integrator by User:Myst in 2020 and has seen several improvements to the API that makes it even easier to create bots using the library.
For more information, documentation, download & installation instructions, see here: https://github.com/LeMyst/WikibaseIntegrator
Example semi-automatic script
LexUse semi-automatic tool for finding and adding usage examples to lexemes. It's free software written using Python 3 in 2020 Wikidata:LexUse.
Wikibase.NET (Forældet)
Wikibase.NET er API'et, som erstatter den nu forældede DotNetDataBot. De er ikke kompatible, da Wikibase.NET ikke længere behøver at anvende DotNetWikiBots framework.
Download og installation
Du kan downloade Wikibase.NET fra GitHub. Du skal bare følge instruktionerne på siden.
Kendte problemer
Eksempler
Kommer ikke snart...
DotNetDataBot (forældet)
Installation
- Download: DotNetDataBot
Konfiguration
After unpacking the package you can see a file called DotNetDataBot.dll and one called DotNetDataBot.xml. The xml document is only for documentation. To use it you have to create a new refer in your project. Then you can write using DotNetDataBot;
to import the framework.
Log ind
For at logge ind skal du oprette et nyt Site
-objekt med wikiens URL, din bots brugernavn og dens kodeord.
Extended content |
---|
C# using DotNetDataBot;
public static void Main()
{
Site wikidata = new Site("http://www.wikidata.org", "User", "Password");
}
|
Eksempel 1:Hent id ved brug af wiki-side
You can access the id of an item by searching for using the site and the title of the connected page.
Extended content |
---|
C# using DotNetDataBot;
public static void Main()
{
Site site = new Site("http://www.wikidata.org", "User", "Password");
Item item = new Item(site);
if (item.itemExists("it", "George Lucas")) // Check if exist on Wikidata
{
Console.Write("Q" + item.GetIdBySitelink("it", "George Lucas"));
}
else
{
Console.Write("Doesn't exist");
}
}
|
Eksempel 2: Hent sproglinks
You can get the interwiki links of an item by loading the content and accessing the links
field of the object.
Extended content |
---|
C# using DotNetDataBot;
public static void Main()
{
Site site = new Site("http://www.wikidata.org", "User", "Password");
Item item = new Item(site);
item.id = item.GetIdBySitelink("it", "George Lucas");
item.Load();
foreach(KeyValuePair<string, string> link in item.links)
{
Console.Write(link.Key); // lang (eg. en or it)
Console.Write(link.Value); // page (eg. George_Lucas)
}
}
|
Eksempel 3: Angiv en beskrivelse
For at angive en beskrivelse, skal du køre funktionen setDescription
.
Extended content |
---|
C# using DotNetDataBot;
public static void Main()
{
Site site = new Site("http://www.wikidata.org", "User", "Password");
Item item = new Item(site, "Q4115189");
item.Load();
if (item.descriptions.ContainsKey("it")) // if alredy exist description in italian
{
// Nothing to do
}
else
{
item.setDescription("it", "description in italian", "Bot: Add italian description");
}
}
|
Eksempel 4: Angiv en etiket
Virker på samme måde ved angivelse af etiket. Du skal bare køre setLabel
.
Extended content |
---|
C# using DotNetDataBot;
public static void Main()
{
Site site = new Site("http://www.wikidata.org", "User", "Password");
Item item = new Item(site, "Q4115189");
item.Load();
if (item.labels.ContainsKey("it")) // if alredy exist label in italian
{
// Nothing to do
}
else
{
item.setLabel("it", "label in italian", "Bot: Add italian label");
}
}
|
Eksempel 5: Hent sproglinks for 100 sider
Denne funktion understøttes ikke. Løb listen igennem endnu en gang.
Wikibase-api til PHP
Dette er en API-klient til Wikibase, skrevet i PHP. Den kan downloades her.
Eksempel 1: Simpelt eksempel
Kik på kildekommentarerne for at forstå, hvordan det virker.
Extended content |
---|
<?php
/**
* Basic example for the use of the libary with some small edits
*/
require_once( __DIR__ . "/vendor/autoload.php" );
// Creates some useful objects and logs into the api
$api = new \Mediawiki\Api\MediawikiApi( "http://www.wikidata.org/w/api.php" );
$api->login( new \Mediawiki\Api\ApiUser( 'username', 'password' ) );
$dataValueClasses = array(
'unknown' => 'DataValues\UnknownValue',
'string' => 'DataValues\StringValue',
);
$wikidata = new \Wikibase\Api\WikibaseFactory(
$api,
new DataValues\Deserializers\DataValueDeserializer( $dataValueClasses ),
new DataValues\Serializers\DataValueSerializer()
);
// Gets the current revision for item Q777
$revision = $wikidata->newRevisionGetter()->getFromId( 'Q777' );
$item = $revision->getContent()->getData();
// Outputs the current sitelink for enwiki
var_dump( $item->getSiteLink( 'enwiki' ) );
// Sets the de description to 'Foobar'
$item->getFingerprint()->setDescription( 'de', 'Foobar' );
// Saves the item
$wikidata->newRevisionSaver()->save( $revision );
//Log out
$api->logout();
|
Eksempel 2: Oprettelse af påstande
Kik på kildekommentarerne for at forstå, hvordan det virker.
Extended content |
---|
<?php
/**
* Basic example for the use of the library with some small edits.
*/
require_once( __DIR__ . "/vendor/autoload.php" );
// Creates some useful objects and logs into the api
$api = new \Mediawiki\Api\MediawikiApi( "https://www.wikidata.org/w/api.php" );
$api->login( new \Mediawiki\Api\ApiUser( 'username', 'password' ) );
$dataValueClasses = array(
'unknown' => 'DataValues\UnknownValue',
'string' => 'DataValues\StringValue',
);
$services = new \Wikibase\Api\WikibaseFactory(
$api,
new DataValues\Deserializers\DataValueDeserializer( $dataValueClasses ),
new DataValues\Serializers\DataValueSerializer()
);
$revision = $services->newRevisionGetter()->getFromId( 'Q777' );
$item = $revision->getContent()->getData();
$statementList = $item->getStatements();
if( $statementList->getByPropertyId( \Wikibase\DataModel\Entity\PropertyId::newFromNumber( 1320 ) )->isEmpty() ) {
$services->newStatementCreator()->create(
new \Wikibase\DataModel\Snak\PropertyValueSnak(
\Wikibase\DataModel\Entity\PropertyId::newFromNumber( 1320 ),
new \DataValues\StringValue( 'New String Value' )
),
'Q777'
);
}
// Log out
$api->logout();
|
VBot (ingen opdateringer siden 2017)
Framework for Wikidata and Wikipedia. Read and write on Wikidata and other Wikimedia project and have a useful list generator to generate list of Wikipedia page and Wikidata entity. Can read also JSON dump of Wikidata.
Oversigt
Bot til at læse og redigere Wikidata og Wikipedia.
- License: CC0 1.0
- Language C#
- Can read and write entities with all datatype on Wikidata
- Can read and write pages on all Wiki project
- Can read parameter from template on wiki pages
- Can read JSON dump
- Can create lists using:
- Wikidata query
- Catscan 2
- Quick intersection
- What Links Here on Wikidata
- Tested with Visual Studio Express 2013 for Windows Desktop.
- Is necessary to have Newtonsoft.Json. You can install it with NuGet inside Visual Studio
- Is necessary to add manually a reference to System.Web for "HttpUtility.UrlEncode"
Download
Frameworket kan downloades fra GitHub her.
Instruktion
- Wiki (delvis)
- User talk:ValterVB :)
Eksempel 1
Opdatér en etiket til alle emner med instance of (P31): short film (Q24862), som hardirector (P57), og som har publication date (P577) i 1908. (Anvendelse af Wikidata-forespørgsel)
Extended content |
---|
private void CompleteExample()
{
//Wikidata query
string strWDQ = "CLAIM[31:24862] AND CLAIM[57] AND BETWEEN[577,+00000001908-00-00T00:00:00Z,+00000001908-12-31T00:00:00Z]";
ListGenerator lg = new ListGenerator();
List<string> chunks = lg.WDQ(strWDQ, 50);
//Connection to Wikipedia
WikimediaAPI WP = new WikimediaAPI("https://it.wikipedia.org", User, Password);
Pages PageList = new Pages();
//Connection to Wikidata
WikimediaAPI WD = new WikimediaAPI("https://www.wikidata.org", User, Password);
Entities EntityList = new Entities();
Dictionary<string, string> Labels = new Dictionary<string, string>();
foreach (string list in chunks)
{
// Load all entity of the chunk
string strJson = WD.LoadWD(list);
EntityList = new Entities();
EntityList = JsonConvert.DeserializeObject<Entities>(strJson, new DatavalueConverter());
foreach (KeyValuePair<string, Entity> entity in EntityList.entities)
{
if (entity.Value.sitelinks.ContainsKey("itwiki"))
{
// Load Wikipage
string Pages = WP.LoadWP(entity.Value.sitelinks["itwiki"].title);
PageList = JsonConvert.DeserializeObject<Pages>(Pages, new DatavalueConverter());
//Director from template
string director = Utility.GetTemplateParameter(PageList.query.FirstPageText, "film","Regista").Replace("[","").Replace("]", "");
Labels = new Dictionary<string, string>();
if (director=="")
{
Labels.Add("en", "1908 short movie");
}
else
{
Labels.Add("en", "1908 short movie directed by " + director);
}
// Update Wikidata
WD.EditEntity(entity.Value.id, null, Labels, null, null, null, "BOT: Update en label");
}
}
}
}
|
LexData (Python; for Lexicographical data)
LexData is an easy to use python libary to create and edit Lexemes, Senses and Forms.
Tips
The documentation of LexData is still a bit lacking so look at existing implementations in MachtSinn or Wikdata Lexeme Forms for ideas how to use it.
If you only want to add statements to Lexemes (not forms or senses) WikibaseIntegrator might be a better choice, as it is more versatile and support a lot of data types.
Installation
You can install LexData via pip:
$ pip install LexData
Login
For all operations you need a WikidataSession
. You can create it with your credentials, a bot password or an Edit Token (for example to edit via OAUTH):
Extended content |
---|
repo = LexData.WikidataSession("YourUsername", "YourPassword")
|
Retrieve a Lexeme
You can open existing Lexemes and read their content.
Extended content |
---|
L2 = LexData.Lexeme(repo, "L2")
print(L2.claims)
print(L2.forms)
print(L2.senses)
sense1 = L2.senses[0]
print(sense1.claims)
|
Searching and creating Lexemes
If you don't know the L-Id of a lexeme you can search for it. And if it doesn't exist you can create it.
Extended content |
---|
# Find an existing Lexeme by lemma, language and grammatical form
L2 = LexData.search_lexemes(repo, "first", en, "Q1084")
# Create a new Lexeme
L2 = LexData.create_lexeme(repo, "first", en, "Q1084")
# Find or create a Lexeme
L2 = LexData.get_or_create_lexeme(repo, "first", en, "Q1084")
|
Tilføje oplysninger
You can easily create forms or senses, with or without additional claims:
Extended content |
---|
if len(L2.forms) == 0:
L2.createForm("firsts", ["Q146786"])
if len(L2.senses) == 0:
L2.createSense(
{
"en": "Element in an ordered list which comes before all others according to the ordering",
"de": "einer Ordnung folgend das Element vor allen anderen",
},
claims={"P5137": ["Q19269277"]},
)
|
Direkte anvendelse af Wikidatas API
I dette afsnit lærer du, hvordan der foretages forespørgsler direkte til API'et. API-forespørgsler er nødvendige, hvis du ønsker at udvikle din egen programmeringsplatform eller hvis du ønsker at kode en ikke-understøttet funktion til en programmeringsplatform.
Til disse API-eksempler behøver du ikke at installere yderlige software, kun sprogoversætteren (Python, Perl, etc.)
Eksempel 1: Hent Q-nummer
Dette eksempel henter Q-nummeret for den engelsksprogede artikel om Andromedagalaksen. Wikibase API'ets overordnede "arbejdshestmodul" action=wbgetentities
tilvejebringer denne information. HTTP-anmodningen (ved anvendelse af jsonfm
-formatet for et JSON output, der er læsbart for mennesker) er bare
Try following the link. This requests no additional information about the entity; remove &props=
from the URL to see much more information about it.
See the generated help for wbgetentities
for more parameters you can specify.
Python
Extended content |
---|
#!/usr/bin/python3
from requests import get
def get_qnumber(wikiarticle, wikisite):
resp = get('https://www.wikidata.org/w/api.php', {
'action': 'wbgetentities',
'titles': wikiarticle,
'sites': wikisite,
'props': '',
'format': 'json'
}).json()
return list(resp['entities'])[0]
print(get_qnumber(wikiarticle="Andromeda Galaxy", wikisite="enwiki"))
|
Resultatet er:
Q2469
Example 2: Get list of items without particular interwiki
...venligst bidrag hvis du ved hvordan...
Se også
- mw:Wikidata Toolkit Java framework
- Wikidata:Bots