Wikidata

Wikidata:GLAM/Metropolitan Museum of Art/Tools/Met Constituent ID

< Wikidata:GLAM‎ | Metropolitan Museum of Art‎ | Tools

The Met tracks constituents (artist, institution, entity, etc.) in its database using a well maintained unique ID. How might these be useful additions to Wikidata, and how might we go about adding them?

Overview edit

It would be desirable to link Met constituent IDs to Wikidata items in order to more easily link artworks and other metadata. As of 2021, about 13,997 of the roughly 55,922 constituents in the Met constituent database have been reconciled to Wikidata items. However, there are some challenges to using this data set:

 
Met API call returns constituent ID in object records, but has no specific constituent lookup function other than a CSV database dump.
  • The Met does not have an API that returns information about its constituent IDs, and only has them available in a full collections data CSV dump.
  • However, in the Met API, constituent IDs are returned in the JSON output. Example: Portrait of Madame X, by John Singer Sargent (link)
  • This means creating a new Wikidata external identifier through a property proposal may have the shortcoming that the formatter URL will not have a useful link.
 
Lookup of Met constituent ID using Datasette Lite, pulling the CSV from the Github repository.

There are some alternatives in the meantime:

  1. The data has been published to Github as a CSV file. (Link to Github repo | Link to CSV.) There is also an on-wiki table that can be found at Wikidata:GLAM/Metropolitan Museum of Art/CID-ULAN-Wikidata report.
  2. Using powerful tools like Datasette Lite, you can browse and slice the data interactively or by URL parameters. (Link to Datasette Lite session)
  3. Since Datasette Lite is a full fledged database tool, using URL parameters you can select out an exact Met constituent ID, making it act like an API lookup (pictured, right). (Link to Met constituent selecting.)

Interim solutions edit

Pending an external identifier being approved via property proposal (see proposal here), we can add Met constituent ID to Wikidata items using a statement and qualifiers in this manner. After an external identifier is approved, we can use a bot to convert these to the new external ID.

Example edit

Gerhardt Wilhelm von Reutern (German painter (1794-1865))

described by source
  Metropolitan Museum of Art Constituent Database edit
subject named as Gerhardt Wilhelm von Reutern
inventory number 100014
0 references
add reference


add value

Challenges edit

Adding these statements via the Quickstatements utility is a popular solution, but in practice it has many problems. Quickstatements often fails in mysterious and nonverbose ways, so that a work queue simply hangs. There is no explanation for why this is, and restarting the queue in the exact same place where it failed will result in it succeeding. This points to some odd internal bug to Quickstatements that is impossible for the end user to diagnose. At this time, it makes Quickstatements too unreliable for professional work, and pywikibot scripts are preferred. This is too bad, since this now requires programming expertise using PAWS or a Python execution environment.

For reference, the Quickstatements being used take the form of: 

qid,P1343,qal217,qal1810
Q64337351,Q106729628,"""163593""","""Fortuny"""
Q1649002,Q106729628,"""163594""","""Ubaldo Gandolfi"""
Q976786,Q106729628,"""163595""","""Battista Franco"""

Alternatives to Quickstatements edit

An example of a Python script to do the same work as Quickstatements, but with more reliability and "cleaning" of certain qualifiers can be found at this PAWS script: https://public.paws.wmcloud.org/User:Fuzheado/metmuseum/constituent-bot/constituent-add-statements.ipynb

Query all constituents edit

The following SPARQL query shows all entities that have the Met constituent ID added so far, also adding date of birth and place of birth to show some possibilities of visualizing the results.

# Met constituents search
SELECT ?item ?itemLabel ?metcid ?dob ?coord 
WHERE 
{
  ?item p:P1343 [
          ps:P1343 wd:Q106729628 ; 
          pq:P217 ?metcid 
        ] . 
  OPTIONAL { ?item wdt:P569 ?dob }
  OPTIONAL { ?item wdt:P19 ?pob .
             ?pob wdt:P625 ?coord 
           }
  SERVICE wikibase:label { bd:serviceParam wikibase:language "[AUTO_LANGUAGE],en". }
}
Try it!

Report edit

Top groupings (Minimum 10 items) Top Properties (used at least 0 times per grouping)
Name Count sex or gender (P21) date of birth (P569) date of death (P570) place of birth (P19) occupation (P106) Commons category (P373) Union List of Artist Names ID (P245) English English
human (Q5) 13806 99.8% (13779) 🔍 97.33% (13437) 🔍 87.29% (12051) 🔍 83.88% (11581) 🔍 99.37% (13719) 🔍 77.86% (10749) 🔍 93.13% (12858) 🔍 100.0% (13806) 🔍 97.84% (13508) 🔍
notname (Q1747829) 129 86.82% (112) 🔍 68.22% (88) 🔍 34.88% (45) 🔍 7.75% (10) 🔍 93.8% (121) 🔍 87.6% (113) 🔍 99.22% (128) 🔍 99.22% (128) 🔍 92.25% (119) 🔍
business (Q4830453) 68 0% (0) 🔍 0% (0) 🔍 0% (0) 🔍 0% (0) 🔍 4.41% (3) 🔍 77.94% (53) 🔍 75.0% (51) 🔍 100.0% (68) 🔍 98.53% (67) 🔍
fashion house (Q3661311) 19 0% (0) 🔍 0% (0) 🔍 0% (0) 🔍 0% (0) 🔍 0% (0) 🔍 73.68% (14) 🔍 36.84% (7) 🔍 100.0% (19) 🔍 100.0% (19) 🔍
enterprise (Q6881511) 17 0% (0) 🔍 0% (0) 🔍 0% (0) 🔍 0% (0) 🔍 11.76% (2) 🔍 100.0% (17) 🔍 64.71% (11) 🔍 100.0% (17) 🔍 100.0% (17) 🔍
art museum (Q207694) 15 0% (0) 🔍 0% (0) 🔍 0% (0) 🔍 0% (0) 🔍 0% (0) 🔍 100.0% (15) 🔍 100.0% (15) 🔍 100.0% (15) 🔍 100.0% (15) 🔍
photographic studio (Q672070) 14 0% (0) 🔍 0% (0) 🔍 0% (0) 🔍 7.14% (1) 🔍 14.29% (2) 🔍 78.57% (11) 🔍 92.86% (13) 🔍 100.0% (14) 🔍 100.0% (14) 🔍
publisher (Q2085381) 12 0% (0) 🔍 0% (0) 🔍 0% (0) 🔍 0% (0) 🔍 8.33% (1) 🔍 91.67% (11) 🔍 58.33% (7) 🔍 100.0% (12) 🔍 100.0% (12) 🔍
art group (Q4502119) 10 0% (0) 🔍 0% (0) 🔍 0% (0) 🔍 0% (0) 🔍 30.0% (3) 🔍 40.0% (4) 🔍 90.0% (9) 🔍 90.0% (9) 🔍 90.0% (9) 🔍
Totals (all items) 13994 98.49% (13783) 🔍 96.04% (13440) 🔍 86.14% (12054) 🔍 82.79% (11585) 🔍 98.13% (13733) 🔍 77.88% (10898) 🔍 92.9% (13000) 🔍 99.99% (13993) 🔍 97.82% (13689) 🔍


Interesting queries edit

Retrieved from "https://www.wikidata.org/w/index.php?title=Wikidata:GLAM/Metropolitan_Museum_of_Art/Tools/Met_Constituent_ID&oldid=2000348209"