|
|
|
#2
04-09-2012, 02:52 AM
|
|||
|
|||
Robot.txt files tell your instructions to a search engine robot. it depends on what you what it to do. if you dont have the robot.txt file then when a robot visits your site and if it doesnt find the file, it will start to visit all the pages and contents of your site. these files are like walls which helps in preventing some files like java scripts, some images or any other files from indexing. it is also important to make a note that it is not a method that prevents search engines from crawling your site. so it is important to create a robot.txt file. if you need this file to be created you can get it from XnYnZ.com
|
|
#3
09-17-2018, 04:14 AM
|
|||
|
|||
|
robot.txt
Robots.txt is a text (not html) file you put on your site to tell search robots which pages you would like them not to visit. Robots.txt is by no means mandatory for search engines but generally search engines obey what they are asked not to do.
----------------------------------------------------------------------------------------------------- dedicated server in Vietnam
|
 |
| Thread Tools | |
| Display Modes | |
|
|
Linear Mode
