[vc_row][vc_column width=”2/3″][vc_column_text]
- robots.txt is the robots exclusion standard (robot.txt, robots.txt, or robots exclusion standard) that describes how to inform web robots (such as search engine robots) about which parts of your website should not be processed or scanned.