Software Downloads for "Robots Txt"
- Advanced Robots.txt Generator
- License: Shareware

Create your robots.txt files in seconds and maintain them easilly and effortlesly with the unique utility Advanced Robots.txt Generator. Advanced Robots.txt Generator is the most complete and easy to use application that allows you to create, maintain and keep track of your Robots.txt files for all your websites. The following is a list of the features of Advanced Robots.txt Generator. * Support for ~300 Robots and Crawlers * Automatic creation of Robots.txt file * Automatic scanning of local site directories * Comparison and correction with previously saved Robots.
- Platform: Windows
- Publisher: BasiSoft Solutions
- Date: 14-11-2006
- Size: 1577 KB
- Advanced Robots txt generator
- License: Shareware

Advanced Robots.txt Generator is the most complete and easy to use application that allows you to manage your Robots.txt files Advanced Robots.txt Generator is the most complete and easy to use application that allows you to create, maintain and keep track of your Robots.txt files for all your websites. Here are some key features of "Advanced Robots txt generator":Advanced Robots.txt generator offers support for ~300 Robots and CrawlersAutomatic creation of Robots.txt fileAutomatic scanning of local site directoriesComparison and correction with previously saved Robots.
- Platform: WinOther
- Publisher: basisoft.com
- Date: 24-08-2009
- Size: 1576 KB
- Robots.txt Maker
- License: Shareware

Make Search Engine Friendly Robots.txt Files Fast and Easy! Pattern matching. Visit Time. Request Rate. Crawl delay, Sitemaps, and more. Add information quickly and easily. Import exiting existing robots. txt file for editing and save it to your hard drive and more. Drag and Drop or Copy/Paste Upload with built in FTP Access. Alternatively, you can use your own favorite FTP program. Easily notify robots which of your files and directories to include and exclude from indexing. Robots.txt Maker will guide you through how to make an Attention Grabbing, Search Engine Friendly robots.
- Platform: Windows
- Publisher: Oven Fresh
- Date: 7-7-2014
- Size: 3710 KB
- Robots.txt editor
- License: Shareware

This program is a visual editor for Robot Exclusion Files and log analyzer software. It allows the user to quickly and easily create the robots.txt files required to instruct web search engines which parts of a web site are not to be indexed and made searchable by the general web public and then to identify spiders, which do not keep to those instructions. The program does this by providing the user a way to log onto his FTP or local network server and then select the documents and directories which are not to be made searchable.
- Platform: Windows
- Publisher: Robots-txt.NET
- Date: 04-10-2009
- Size: 2867 KB
- Robots.txt
- License: Shareware

With Robots.txt Editor you create robot exclusion files by selecting all robots or a specific user-agent and adding documents and/or directories by entering the path names manually or by selecting them using FTP.With Log Analyzer you can get the full statistics of robots visiting the pages of your website.. Web site promotion | search engine optimization | web site statistics - NetPromoter..
- Platform: Windows
- Publisher: NetPromoter Lab
- Date: 14-10-2009
- Size: 2775 KB
- Oven Fresh Robots.txt Maker
- License: Shareware

Easily Make Your Robots.txt File in Minutes. Help prevent unauthorized indexing of your private directories that if indexed by themselves could frustrate a viewer's search experience and possibly alienate them. For instance, a viewer could become discouraged after clicking on a link that only shows your photos without any way to reach your Website. Just fill in a simple form and the code is generated for you.. Jupiter Web Design | Palm Beach Gardens | North Palm Beach | Palm Beach | Web Design Jupiter Florida Web Design.
- Platform: Windows
- Publisher: Oven Fresh Web Design
- Date: 13-10-2009
- Size: 3788 KB
- robots.txt generator
- License: Shareware

This script lets you browse through your serverins directory structure. If a robots.txt is found, it is read and the appropriate allowed/disallowed folders are checked/unchecked. Just change the checkers and hit the inupdate?button to update the robots.txt. You may also delete the file or edit the comment. Changes are always made to inUser-agent: *? Additionally the script warns, if no index-files or htaccess files were found. Updated: Now PHP 4.2 security conform..
- Platform: PHP, Scripts
- Publisher: bodden.de
- Date: 04-08-2011
- robots.txt Tools And Tutorial
- License: Shareware

Tools to automatically creat and validate robot.txt file and a tool to create a .htaccess file to ban bad robots. Also includes a general robots.txt tutorial..
- Platform: Misc Scripts
- Publisher: clockwatchers.com
- Date: 08-09-2011
- Easy Robots.txt Generator
- License: Freeware

The robots.txt file is the first thing search engine 'spiders' look for when indexing a website. The robots.txt file tells search engine spiders (robots) which files and/or directories they are NOT allowed to index. This helps to prevent incomplete site indexing as well as prevent exposing the files and directories that you don't want 'all over the world wide web'. It's fast, easy and FREE..
- Platform: JavaScript, Scripts
- Publisher: John Kline
- Date: 06-03-2011
- Robots parser
- License: Freeware

Reads and parses /robots.txt files..
- Platform: PHP, Scripts
- Publisher: Mustafa Ulu
- Date: 17-01-2013
- Size: 10 KB
- Linking The Web
- License: Freeware

"Linking The Web" is a free utility that regularly monitors all backlinks to your sites. If some of the links don't exist, contain wrong anchor, are covered by a "nofollow" flag or hidden by robots.txt then you will be immediately notified by email. Also the program can keep track of your purchased links and notify you about their expiration dates. The software is free of charge and works under all versions of Microsoft Windows..
- Platform: Windows
- Publisher: S5 Development
- Date: 27-06-2008
- Size: 780 KB
- RoboGen ROBOTSTXT Editor
- License: Shareware

Creates and manages Robot Exclusion iles, ROBOTS.TXT or your websiteRoboGen is a program, or Windows 2000, XP, and Vista, to help you more easily manage the robots.txt iles or your websites. Instead o the time consuming manual management process, described in the tutorial, you can either edit your robot exclusion iles on your local computer or on a remote FTP site. RoboGen is a visual editor or Robot Exclusion Files. The user is allowed to quickly and easily create the ROBOTS.TXT iles that are required to instruct web search engines which parts o a your web site are not to be indexed or made searchable by the public browsing the web.
- Platform: WinOther
- Publisher: rietta.com
- Date: 24-9-2009
- Size: 1484 KB
- Block Bad Bots
- License: Freeware

This simple robots.txt file will help deter some of the unwanted bots that spider your site for email addresses and things of that nature that result in spam. This works for all websites and it is not just limited to phpBB. This has also been configured to allow Google AdSense bots to spider your site if you use the program..
- Platform: PHP, Scripts
- Publisher: funender.com
- Date: 16-03-2011
- PHPCrawl
- License: Freeware

PHPCrawl is a webcrawler/webspider-library written in PHP. It supports filters, limiters, cookie-handling, robots.txt-handling, multiprocessing and much more..
- Platform: Linux, Mac, Windows
- Publisher: phpcrawl.cuab.de
- Date: 11-07-2012
- Size: 327 KB
- Visual SEO Studio
- License: Freeware

Visual SEO Studio is a desktop SEO Audit Tool, a Visual approach to make SEO tasks quick and easy. The program lets you Crawl one or more sites in parallel with its Interactive Web Spider, Spot on-site issues at a glance, Save Page Screenshot miniatures and see where the fold lays, visually edit robots.txt and XML Sitemaps files, Export to native Excel format... With in depth SEO / HTML / URL reports no other SEO product offers (e.g. Truncated page titles, titles longer than 12 words,...), Visual SEO Studio assists you through all the SEO steps to Optimize Ranking, Click-Through Rate, Bounce Rate, and Conversion Rate.
- Platform: Windows
- Publisher: aStonish Studio
- Date: 08-01-2017
- Size: 10069 KB
- Visual SEO Studio for Mac
- License: Freeware

Visual SEO Studio is a desktop SEO Audit Tool, a Visual approach to make SEO tasks quick and easy. The program lets you Crawl one or more sites in parallel with its Interactive Web Spider, Spot on-site issues at a glance, Save Page Screenshot miniatures and see where the fold lays, visually edit robots.txt and XML Sitemaps files, Export to native Excel format... With in depth SEO / HTML / URL reports no other SEO product offers (e.g. Truncated page titles, titles longer than 12 words,...), Visual SEO Studio assists you through all the SEO steps to Optimize Ranking, Click-Through Rate, Bounce Rate, and Conversion Rate.
- Platform: Mac OS X
- Publisher: aStonish Studio
- Date: 19-03-2019
- Size: 48727 KB
- Search Engine Submission Tools For Website Promotion
- License: Shareware

Offers free search engine optimization tools and tutorials in a web based community environment. Basic membership is free and includes free meta tag tester, link checker, search engine submission, robots.txt generator, tutorials, testing wizard and more..
- Platform: Misc Scripts
- Publisher: sitesubmit.ca
- Date: 02-04-2011
- Robot Control Pro
- License: Shareware

This script will block email harvester robots and file download tools from accessing your site. Includes detailed email notification on search engine visits. No need for SSI or converted email addresses. Includes an advanced admin to deny, allow or block robots and users..
- Platform: CGI and Perl, Scripts
- Publisher: http://www.webcomposing....
- Date: 06-03-2011
- ASP Search Engine
- License: Shareware

Indexing of the title, description, keywords and full content of all HTML, ASP and TXT documents using a recursive call to all pages in a web site and sub folders excluding those folders listed in the robots.txt file. Simple search of the index for your web pages or advanced search for the provided search page. ASP or HTML code to insert into you current web pages to provide the search facility..
- Platform: WinOther
- Publisher: Educe Software
- Date: 10-03-2001
- Size: 97 KB
- Page Promoter
- License: Shareware

Page Promoter is an ultimate URL registration, search engine optimization and promotion tool for submitting Web sites to thousands search engines and directories as well as preliminary site optimization and effectiveness check. The software allows you to enhance sites` exposure on the Internet and draw maximum number of targeted traffic. It contains a large and up-to-date submission database, which is updated on a regular basis. Features include Keyword Tracking and Ranking utilities, Page analyzer, Robots.
- Platform: Windows
- Publisher: NetPromoter
- Date: 07-02-2003
- Size: 7077 KB




