To create a robots.txt you will need RobotsTXT module. It creates the file for you, place it in your root directory and allows you to dynamically edit it without using Drupal.

Drupal is an amazing content management system (CMS) that can build pretty much any Web site you want it to -- if you can figure out how. Follow along as I remake my personal Web site, bratgrrl ...

Working with Drupal is not for the timid -- but these 10 free modules could make things easier for developers and administrators.