A robots.txt file tells search engine crawlers or spiders which web pages or files they can or can't request access from your site. The robot.txt file is important for your SEO because it avoids overloading your server with Google’s crawlers on pages that are not as important to crawl.
Generate your Robots.txt here and paste it in a text editor like notepad or Word, save as .txt file.
Once complete, save and upload your robots.txt file to the root directory of your site. For example, if your domain is www.mydomain.com, you will place the file at www.mydomain.com/robots.txt.