MediaWiki talk:Robots.txt

Wikidata specific entries

Disallow: /wiki/MediaWiki:SetSiteLink
Disallow: /wiki/MediaWiki%3ASetSiteLink
Disallow: /wiki/MediaWiki:SetAliases
Disallow: /wiki/MediaWiki%3ASetAliases
Disallow: /wiki/MediaWiki:SetLabel
Disallow: /wiki/MediaWiki%3ASetLabel
Disallow: /wiki/MediaWiki:SetDescription
Disallow: /wiki/MediaWiki%3ASetDescription

Please add the above to this page. Somehow tons of pages starting with these links get indexed.

For a sample use of this MediaWiki page, see en:MediaWiki:Robots.txt. --  Docu  at 15:18, 14 August 2013 (UTC)

Odd that they would be showing up in any way, but there definitely isn't any harm in exempting them from search results and the like. Ajraddatz (Talk) 15:25, 15 August 2013 (UTC)

Thanks. I wondered where the links came from too ..

BTW, it should be "Special", not "MediaWiki" above. Sorry about that.

It doesn't show up on //www.wikidata.org/robots.txt either. --  Docu  at 15:39, 15 August 2013 (UTC) (edited)

I left a note on Wikidata:Contact_the_development_team. --  Docu  at 15:45, 15 August 2013 (UTC)

Maybe it's just that <syntaxhighlight lang="robots"> needs to be preceeded by a "#". --  Docu  at 16:01, 15 August 2013 (UTC)

Fixed version:

# <syntaxhighlight lang="robots">
# Lines here will be added to the global robots.txt or at least to http://www.wikidata.org/robots.txt
Disallow: /wiki/Special:SetSiteLink
Disallow: /wiki/Special%3ASetSiteLink
Disallow: /wiki/Special:SetAliases
Disallow: /wiki/Special%3ASetAliases
Disallow: /wiki/Special:SetLabel
Disallow: /wiki/Special%3ASetLabel
Disallow: /wiki/Special:SetDescription
Disallow: /wiki/Special%3ASetDescription
#</syntaxhighlight>

Eventually, "Special:" might need internationalization. --  Docu  at 16:06, 15 August 2013 (UTC)

What an odd way to set up the page. Oh well, page updated - hopefully that works. Ajraddatz (Talk) 16:43, 15 August 2013 (UTC)

Thanks. I think it did. The page updated now. We will see in 1 or 2 months if it works as it should. --  Docu  at 16:56, 15 August 2013 (UTC)

In the meantime, the number of archived "Set.."-pages seems to have increased. Hopefully, eventually it gets flushed.

BTW, there seems to be spare space (" ") at the beginning of the message. --  Docu  at 09:54, 24 August 2013 (UTC)

Special:SetLabelDescriptionAliases

{{Editprotected}} I think this should be added, as users without JS will go to this page when editing labels.--GZWDer (talk) 05:24, 11 April 2015 (UTC)

 Done ·addshore· ^{talk to me!} 09:50, 11 April 2015 (UTC)

Also exclude URLs with question marks

Tracked in Phabricator
Task T144308

{{Edit request}} Please update the page as follows:

Disallow: /wiki/Special%3A
Disallow: /wiki/Special:EntityData/
Allow: /wiki/Special:EntityData/*.
Disallow: /wiki/Special:GoToLinkedPage/
Disallow: /wiki/Special:GoToLinkedPage?
Disallow: /wiki/Special:ItemByTitle/
Disallow: /wiki/Special:ItemByTitle?
Disallow: /wiki/Special:SetAliases/
Disallow: /wiki/Special:SetDescription/
Disallow: /wiki/Special:SetLabel/
Disallow: /wiki/Special:SetLabelDescriptionAliases/
Disallow: /wiki/Special:SetSiteLink/
Disallow: /wiki/Special:SetSiteLink?

(Note: Don't remove the slashes and question marks as described in phab:T144308.) Thank you! --Thiemo Mättig (WMDE) 14:56, 24 October 2016 (UTC)

 Done - Mbch331 (talk) 15:01, 24 October 2016 (UTC)