Hello TYPO3 community!
I have a quick question about optimizing the robots.txt and sitemap.xml.
Firstly, do we have a gold standard for TYPO3?
For example, regarding robots.txt, I have the following setup:
User-agent: *
Disallow: /typo3/
Disallow: /typo3_src/
Would that be enough? What about the fileadmin directory? Should I also block search engine crawlers from crawling it?
I noticed that the sitemap is very simple for each language. Google recommends using hreflang to indicate the other language versions in a single sitemap instead of using multiple sitemaps.
What would be the recommended approach for TYPO3?