# See http://www.robotstxt.org/robotstxt.html for documentation on how to use the robots.txt file User-agent: * Disallow: / User-agent: Twitterbot Allow: /images User-agent: facebookexternalhit Allow: /images User-agent: Googlebot Disallow: /users/ Disallow: /participants/ Disallow: /shop Disallow: /assets/ Disallow: /rails/