Hi, is there a way to add our own robots.txt to disallow crawling, without it being turned into a post of its own?
Also, if there is, how can I make
User-agent: *
Disallow: /
not be read as metadata
Yes, but it's not possible to accomplish this using your folder directly.
You'll need to fork an editable version of your template first. Then open up your template's robots.txt file and make the desired changes. Here's what the robots.txt looks like for the Diary template.
Answered 3 years ago · Improve this answer