The following is an example of an x-robots-tag header response:
Posted: Sat Feb 08, 2025 10:27 am
If you don't want to put any restrictions on your snippet, you can add the following site-wide in your page header:
<meta name="robots" content="max-snippet:-1, max-image-preview:large, max-video-preview:-1" >
What is an X‑Robots-Tag?
Another way to control how search engines crawl and index your pages is to use the x-robots-tag instead of the meta robots tag.
While implementing a meta robots tag in an HTML page is relatively simple, the x-robots-tag is more complex. If you want to control how non-HTML content is handled, such as PDFs, you need to use the x-robots-tag.
This is an response header, not an HTML tag, and any directive india mobile database that can be used as a meta robots tag can also be used as an x-robots-tag.
x-robots-tag: noindex, nofollow
In order to use x-robots-tag, you need to have access to your site's website header .php, .htaccess or server configuration file. If you don't have access to it, you need to use the meta robots tag to instruct the crawler.
When to use X‑Robots-Tag?
Using the x-robots-tag is not as simple as using the meta robots tag, but allows you to instruct search engines how to index and crawl additional file types.
Use the x-robots-tag when:
You need to control how search engines crawl and index non-HTML file types
You need to provide the directive at a global level (site-wide) rather than at the page level
How to set up the Robots meta tag and X‑Robots-Tag.
<meta name="robots" content="max-snippet:-1, max-image-preview:large, max-video-preview:-1" >
What is an X‑Robots-Tag?
Another way to control how search engines crawl and index your pages is to use the x-robots-tag instead of the meta robots tag.
While implementing a meta robots tag in an HTML page is relatively simple, the x-robots-tag is more complex. If you want to control how non-HTML content is handled, such as PDFs, you need to use the x-robots-tag.
This is an response header, not an HTML tag, and any directive india mobile database that can be used as a meta robots tag can also be used as an x-robots-tag.
x-robots-tag: noindex, nofollow
In order to use x-robots-tag, you need to have access to your site's website header .php, .htaccess or server configuration file. If you don't have access to it, you need to use the meta robots tag to instruct the crawler.
When to use X‑Robots-Tag?
Using the x-robots-tag is not as simple as using the meta robots tag, but allows you to instruct search engines how to index and crawl additional file types.
Use the x-robots-tag when:
You need to control how search engines crawl and index non-HTML file types
You need to provide the directive at a global level (site-wide) rather than at the page level
How to set up the Robots meta tag and X‑Robots-Tag.