http://forge.typo3.org/http://forge.typo3.org/themes/typo3_forge/favicon/favicon.png?17058661692019-11-08T14:42:03ZTYPO3 ForgeTYPO3 Core - Bug #89617: XML sitemaps should not be indexed by search engineshttp://forge.typo3.org/issues/89617?journal_id=4092212019-11-08T14:42:03ZGerrit Code Review
<ul><li><strong>Status</strong> changed from <i>New</i> to <i>Under Review</i></li></ul><p>Patch set 1 for branch <strong>master</strong> of project <strong>Packages/TYPO3.CMS</strong> has been pushed to the review server.<br />It is available at <a class="external" href="https://review.typo3.org/c/Packages/TYPO3.CMS/+/62253">https://review.typo3.org/c/Packages/TYPO3.CMS/+/62253</a></p> TYPO3 Core - Bug #89617: XML sitemaps should not be indexed by search engineshttp://forge.typo3.org/issues/89617?journal_id=4092242019-11-08T14:54:09ZGerrit Code Review
<ul></ul><p>Patch set 2 for branch <strong>master</strong> of project <strong>Packages/TYPO3.CMS</strong> has been pushed to the review server.<br />It is available at <a class="external" href="https://review.typo3.org/c/Packages/TYPO3.CMS/+/62253">https://review.typo3.org/c/Packages/TYPO3.CMS/+/62253</a></p> TYPO3 Core - Bug #89617: XML sitemaps should not be indexed by search engineshttp://forge.typo3.org/issues/89617?journal_id=4092492019-11-08T23:42:10ZGerrit Code Review
<ul></ul><p>Patch set 1 for branch <strong>9.5</strong> of project <strong>Packages/TYPO3.CMS</strong> has been pushed to the review server.<br />It is available at <a class="external" href="https://review.typo3.org/c/Packages/TYPO3.CMS/+/62219">https://review.typo3.org/c/Packages/TYPO3.CMS/+/62219</a></p> TYPO3 Core - Bug #89617: XML sitemaps should not be indexed by search engineshttp://forge.typo3.org/issues/89617?journal_id=4092502019-11-09T00:00:04ZChris Müller
<ul><li><strong>Status</strong> changed from <i>Under Review</i> to <i>Resolved</i></li><li><strong>% Done</strong> changed from <i>0</i> to <i>100</i></li></ul><p>Applied in changeset <a class="changeset" title="[BUGFIX] Prevent indexing of XML sitemaps by search engines The content of a XML sitemap should ..." href="http://forge.typo3.org/projects/typo3cms-core/repository/1749/revisions/cdab79385c12d4d4bf4059f184243a93c1d4caf3">cdab79385c12d4d4bf4059f184243a93c1d4caf3</a>.</p> TYPO3 Core - Bug #89617: XML sitemaps should not be indexed by search engineshttp://forge.typo3.org/issues/89617?journal_id=4092522019-11-09T07:47:34ZGerrit Code Review
<ul><li><strong>Status</strong> changed from <i>Resolved</i> to <i>Under Review</i></li></ul><p>Patch set 2 for branch <strong>9.5</strong> of project <strong>Packages/TYPO3.CMS</strong> has been pushed to the review server.<br />It is available at <a class="external" href="https://review.typo3.org/c/Packages/TYPO3.CMS/+/62219">https://review.typo3.org/c/Packages/TYPO3.CMS/+/62219</a></p> TYPO3 Core - Bug #89617: XML sitemaps should not be indexed by search engineshttp://forge.typo3.org/issues/89617?journal_id=4092552019-11-09T08:00:04ZChris Müller
<ul><li><strong>Status</strong> changed from <i>Under Review</i> to <i>Resolved</i></li></ul><p>Applied in changeset <a class="changeset" title="[BUGFIX] Prevent indexing of XML sitemaps by search engines The content of a XML sitemap should ..." href="http://forge.typo3.org/projects/typo3cms-core/repository/1749/revisions/ff6c916fbd63e90d714a89ea58f99ea567ced14a">ff6c916fbd63e90d714a89ea58f99ea567ced14a</a>.</p> TYPO3 Core - Bug #89617: XML sitemaps should not be indexed by search engineshttp://forge.typo3.org/issues/89617?journal_id=4120692019-12-23T22:44:09ZBenni Mackbenni@typo3.org
<ul><li><strong>Status</strong> changed from <i>Resolved</i> to <i>Closed</i></li></ul> TYPO3 Core - Bug #89617: XML sitemaps should not be indexed by search engineshttp://forge.typo3.org/issues/89617?journal_id=4709552022-07-08T16:04:35ZMordamir falk.berger@naturefund.de
<ul></ul><p>Whats the reason behind this decision?</p>
<p>With the x-robots-tag set to noindex, google search console does not accept the sitemap. And a xml sitemap is mostly for crawlers. So why should i exclude the crawlers from using the sitemap?</p>
<p>Maybe this is recent change from google?</p> TYPO3 Core - Bug #89617: XML sitemaps should not be indexed by search engineshttp://forge.typo3.org/issues/89617?journal_id=4717532022-07-19T18:18:45ZChris Müller
<ul></ul><p>"With the x-robots-tag set to noindex, google search console does not accept the sitemap."</p>
<p>I can't reproduce that. Added some sitemaps to GSC which is possible some time ago with this change.</p>
<p>You have to distinguish between crawling and indexing. Crawling means, that the crawler is able and allowed to retrieve the content and follow the links. This is possible for XML sitemaps with the "X-Robots-Tag: noindex" header. This is the same for a meta tag "robots" with "noindex" on a page: The page is crawled (and links followed) but the page isn't indexed (= displayed in the search results).</p>
<p>But: The XML sitemap should not be visible in the results of a search engine. Thus the "X-Robots-Tag: noindex". See also the link in the description of this issue.</p>
<p>And: If you want to allow the indexing of the XML sitemap (which means displaying in the search results) for whatever reason, you can easily achieve this by adding the following line in the TypoScript setup of your site package:</p>
<p>seo_sitemap.config.additionalHeaders.20 ></p> TYPO3 Core - Bug #89617: XML sitemaps should not be indexed by search engineshttp://forge.typo3.org/issues/89617?journal_id=4870652023-02-15T09:53:59ZFlorian Seirerflorian.seirer@tirol.gv.at
<ul></ul><p>Had the same issue as <a class="user active user-mention" href="http://forge.typo3.org/users/4057">@Mordamir </a>: Our XML sitemaps were not crawled by Google for weeks because of the noindex header, according to the report in the Google Search Console.</p>
<p>But simply resubmitting the sitemaps fixed the problem. All URLs from the sitemaps are now recognized.</p>
<p>Maybe this was just a temporary issue on Google's side?</p>