Announcement

Collapse
No announcement yet.

Robots.txt file?

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • Robots.txt file?

    Hi All,
    Tell me how you might use the Robots.txt file?

    Thanks in advance ....
    website design | Polo Shirt | Australia free classifieds Ads

  • #2
    Re: Robots.txt file?

    If you want to prevent any folder or url for indexing then use "robots.txt".
    Last edited by johnBMitchell; 09-11-2012, 06:22 AM.

    Comment


    • #3
      Re: Robots.txt file?

      Robot.txt is a text file through which we can instruct the search engine which have to be cache or not.

      Comment


      • #4
        Re: Robots.txt file?

        Robot.txt is a simple notepad file through which crawler crawl your website.
        Coffee Machine Supplies | Lease Commercial Coffee Machines | Buy Coffee Beans

        Comment


        • #5
          Re: Robots.txt file?

          hide the unwanted or private data from the web crawler
          google nexus 4 | usa online shopping store

          Comment


          • #6
            Re: Robots.txt file?

            Robots.txt is text file with this you can hide unwanted file from the search engine without crawling

            Comment


            • #7
              Re: Robots.txt file?

              "robots.txt" file is text file. which you can allow for crawling by search engine as per your command.
              Australia Immigration | Immigration Overseas | Canada Immigration | New Zealand Immigration

              Comment


              • #8
                Re: Robots.txt file?

                Well, According to my view, Robots.txt file is generally used for hide your URL or folder which you don't want to give permission to search engine to crawl your backlinks or URL. By this file, you can prevent your data or files.
                Build online store

                Comment


                • #9
                  Re: Robots.txt file?

                  Robot.txt is the file that gives the permission to Google regarding what page google will crawl and which one is not.
                  build online store

                  Comment


                  • #10
                    Robots.txt is the first file which used to crawl by the search spiders. And here we can allow the crawlers which file to crawl of which one is to not.
                    Noida Housing Projects | Gurgaon Housing Projects

                    Comment


                    • #11
                      AS everyone mentioned above that robot file is is for privacy or whenever you want to hide something from google spider you mention path of that file or folder into robot.txt file

                      what as per me robot.txt is a must have file that guide your search engine spiders if what to read what not to
                      i will explain it with a example

                      User-agent: * Disallow: /hello-world Disallow: /wp-admin/
                      this is format of your robot.txt file
                      User-agent: * this mean you are allowing search engine spider all of your website
                      Disallow: /hello-world Disallow: /wp-admin/ This mean you want google boot to ignore these two file and folders


                      Robot.txt file you can find this file in your root directory it is a simple notepad file

                      You can learn more such things from SEO Training Blog
                      Indi IT Academy, a well known and best industrial training institute in Chandigarh, Mohali. Endow 6 months six weeks training in seo, php and web designing.

                      Comment


                      • #12
                        Robots.txt is a text file created by webmasters that instruct web robots how to crawl pages in their website. This file also indicates whether the pages of their website should be crawled or not. The crawl instructions are

                        Comment

                        Working...
                        X