Rewritten the script to work with a 'robots.txt' template file and we generate the 'robots.txt' from the template when we execute the script. It's found to be a better alternative to editing the 'robots.txt' file in-place.
