Free Downloads: Crawlers
Manage and track visits by robots, spiders and crawlers to your website.
Manage and track visits by robots, spiders and Crawlers to your website. Identify which spiders visited, when they visited, and what pages they indexed. Direct spiders to your most important pages, and exclude them from non-public pages. Maximizes effectiveness with search engines by avoiding over/under submitting. Excellent to improve search engine placements for users of automatic...
Category: Internet / Tools & Utilities
Publisher: Website Management Tools, License: Shareware, Price: USD $129.95, File Size: 3.3 MB
Platform: Windows
ETagFix is an add-on for Microsoft Internet Information Server (IIS) or any web server that supports ISAPI filters, that cuts off part of the etag that IIS produces so that is does not change every time your computer is rebooted. Etag is an HTTP header that is produced by IIS to allow web Crawlers and user clients to check against to see if a web page has changed. ETagFix fixes the...
Category: Internet / Tools & Utilities
Publisher: ISAPILabs, License: Freeware, Price: USD $0.00, File Size: 696.3 KB
Platform: Windows
The web crawler is a program that automatically traverses the web by downloading the pages and following the links from page to page. A general purpose of web crawler is to download any web page that can be accessed through the links.
This process is called web crawling or spidering. Many sites, in particular search engines, use spidering as a means of providing up-to-date data. Web...
Category: Internet / Misc. Plugins
Publisher: VietSpider, License: Freeware, Price: USD $0.00, File Size: 48.8 MB
Platform: Windows, Linux
Freeware
Free Large Android Icons picture various space creatures, such as robots, Crawlers and androids. Not only does the set present the creatures themselves, it also shows their moods and actions. All the images are created by professional artists and demonstrate bright colors and well-finished edges. This free icon set is a perfect choice for your personal desktop customization. Each...
Category: Desktop Enhancements / Icon Tools
Publisher: large-icons.com, License: Freeware, Price: USD $0.00, File Size: 13.7 MB
Platform: Windows, Mac, Linux,
Generates XML Sitemap, REQUIRED for search engine optimization for Google, MSN, Yahoo - all at once!
Generates XML Sitemap, REQUIRED for search engine optimization for Google, MSN, Yahoo - all at once! Search engine rivals Google, Yahoo and Microsoft are teaming up to make it easier for Web site owners to make sure their sites get included in the Web indexes, the companies are expected to announce Thursday, Oct 16, 2006. Now Web site owners will be able to go to one place for alerting all...
Category: Web Authoring / Site Management
Publisher: MagneticOne, License: Commercial, Price: USD $39.00, File Size: 29.0 KB
Platform: Windows, Linux, Unix
Generates XML Sitemap, REQUIRED for search engine optimization for Google, MSN, Yahoo - all at once!
Generates XML Sitemap, REQUIRED for search engine optimization for Google, MSN, Yahoo - all at once!
Search engine rivals Google, Yahoo and Microsoft are teaming up to make it easier for Web site owners to make sure their sites get included in the Web indexes, the companies are expected to announce Thursday, Oct 16, 2006.
Now Web site owners will be able to go to one place for...
Category: Web Authoring / Web Design Utilities
Publisher: MagneticOne, License: Commercial, Price: USD $39.00, File Size: 30.0 KB
Platform: Windows, Linux
Build a custom web spider / web crawler using web data extraction / screen scraping technology. Use the web extract for web data mining of contact lists, product catalogs, government databases, real estate listings, or build a custom email extractor. Creating your own web grabber that can screen scrape the data to a database or Excel has never been easier. The website spider /...
Category: Utilities
Publisher: Velocityscape, License: Shareware, Price: USD $419.00, File Size: 51.3 MB
Platform: Windows
SearchBlox is an out-of-the-box Enterprise Search Solution built on top of Apache Lucene.
SearchBlox is an out-of-the-box Enterprise Search Solution built on top of Apache Lucene. It is fast to deploy, easy to manage and available for both on-premise and cloud deployment.
Best of all, it is free. No limitations. No restrictions.
SearchBlox provides Crawlers for Filesystems, HTTP/HTTPS and RSS Feeds that work out-of-the-box. The HTTP/HTTPS crawler can index...
Category: Internet / Web Search Utilities
Publisher: SearchBlox Software, Inc., License: Shareware, Price: USD $0.00, File Size: 0
Platform: Windows
This program lets you to substitute URI for symbol codes of the ASCII table.
This program lets you to substitute URI for symbol codes of the ASCII table. It's very useful to prevent scanning of web pages by the Crawlers gathering e-mail addresses for broadcasting spam messages on the next step. Using this software you can protect your mailbox from tons of junk mail, and will not be afraid of publishing contacts on your home page or a company's web site....
Category: Security & Privacy / Encrypting
Publisher: Loonies Software, License: Freeware, Price: USD $0.00, File Size: 63.3 KB
Platform: Windows
Files Crawler is a file search application.
Files Crawler is a file search application. It lets you search tons of files and download music, videos, images, documents and software. Files Crawler is actually performing its search at Google. As Google allows us to search for more than just Web pages, due to its special indexing method of directory listing. Directory listing of Websites, is actually what allows Google to see all sort of...
Category: Internet / Tools & Utilities
Publisher: Speedapps Inc, License: Freeware, Price: USD $0.00, File Size: 747.5 KB
Platform: Windows
Super Arcade Machine is an ultimate 25 game pack including arkanoid and breakout style games, pacman style games and frantic space shooters. Incredible music, sound and graphics with many configurable options in each of the titles. Most games include many features.
This package of games is fun for the whole family and features 25 neat and addicting games including Animal Army,...
Category: Games / Simulations
Publisher: Super Arcade Machine, License: Shareware, Price: USD $19.99, File Size: 103.6 MB
Platform: Windows
Create your robots.txt files in seconds and maintain them easilly and effortlesly with the unique utility Advanced Robots.txt Generator. Advanced Robots.txt Generator is the most complete and easy to use application that allows you to create, maintain and keep track of your Robots.txt files for all your websites. The following is a list of the features of Advanced Robots.txt Generator. *...
Category: Web Authoring / HTML Editors
Publisher: BasiSoft Software Solutions, License: Shareware, Price: USD $35.00, File Size: 2.1 MB
Platform: Windows
Online Web-based applications are increasingly at risk from professional hackers who target such applications in order to commit data theft or fraud.
The D-Guard Web Application Firewall is a powerful security tool for Web applications and Web sites.
D-Guard Web Application Firewall is designed to combat all attack types that have been categorized as significant threats,...
Category: Security & Privacy / Access Control
Publisher: D-Guard Technology Co., Ltd., License: Shareware, Price: USD $399.00, File Size: 4.6 MB
Platform: Windows
Online Web-based applications are increasingly at risk from professional hackers who target such applications in order to commit data theft or fraud.
The D-Guard Web Application Firewall is a powerful security tool for Web applications and Web sites.
D-Guard Web Application Firewall is designed to combat all attack types that have been categorized as significant threats,...
Category: Security & Privacy / Access Control
Publisher: D-Guard Technology Co., Ltd., License: Shareware, Price: USD $6999.00, File Size: 4.6 MB
Platform: Windows
Create your robots.txt files in seconds and maintain them easilly and effortlesly with the unique utility Advanced Robots.txt Generator. Advanced Robots.txt Generator is the most complete and easy to use application that allows you to create, maintain and keep track of your Robots.txt files for all your websites. The following is a list of the features of Advanced Robots.txt Generator. *...
Category: Software Development
Publisher: BasiSoft Solutions, License: Shareware, Price: USD $65.00, File Size: 1.5 MB
Platform: Windows
The InnerPeace WebMaster Toolkit is a collection of essensial tools assembeled and made by the Webmasters of InnerPeace. Some of these essensial tools are not available anywhere else. Includes a fast, easy 4 hexillion color combo picker, PageJumper web page forwarder, EScrambler to scrambler your email addresses and hide them from spamcrawlers, plus some excellent articles. Colors on web pages...
Category: Utilities / Misc. Utilities
Publisher: innerpeace.org, License: Freeware, Price: USD $0.00, File Size: 370.0 KB
Platform: Windows, Linux, Mac, Palm
Cable Player is a Entry Level digital video player for cable TV operators.
Cable Player is a Entry Level digital video player for cable TV operators. With Cable Player you need not worry about faulty VCRs’, VHS tapes, VCD Players or even the staff running your Cable Channel. All you have got to do is load movies, songs, ads, promos for days on the computer, set the sequence and let Cable Player do the rest. Cable Player will play your movies, songs along with...
Category: Multimedia & Design / Video
Publisher: Trinity Technologies, License: Freeware, Price: USD $0.00, File Size: 4.1 MB
Platform: Windows
Make a web of links to your posts and you will get a lot of new readers. Blogspreading is a small free utility, which automatically appends a link to the latest post in your blog to your e-mail signature. This will easily catch the recipients' interest and convert them to the readers of your blog. All e-mail clients allow automatic adding of a signature to outgoing message. If you use an...
Category: Audio / MP3 Search Tools
Publisher: Softvoile, License: Freeware, Price: USD $0.00, File Size: 578.8 KB
Platform: Windows
The main features:
Automatically generated sitemaps, With yahoo, google and other large-scale search engine standards.
Simulation crawling web Crawlers,can check out the incorrect link.
Set up a free multi-threaded, to generate extremely fast.
Manual support Web site editor, artificial setting parameters such as frequency of updates.
To support the import and export...
Category: Internet / Communications
Publisher: obdkit Technology, License: Shareware, Price: USD $0.00, File Size: 1.6 MB
Platform: Windows
The main features:
Automatically generated sitemaps, With yahoo, google and other large-scale search engine standards.
Simulation crawling web Crawlers,can check out the incorrect link.
Set up a free multi-threaded, to generate extremely fast.
Manual support Web site editor, artificial setting parameters such as frequency of updates.
To support the import and export...
Category: Internet / Communications
Publisher: Changsha aeke Information Technology Co., License: Shareware, Price: USD $0.00, File Size: 1.6 MB
Platform: Windows
The main features:
Automatically generated sitemaps, With yahoo, google and other large-scale search engine standards.
Simulation crawling web Crawlers,can check out the incorrect link.
Set up a free multi-threaded, to generate extremely fast.
Manual support Web site editor, artificial setting parameters such as frequency of updates.
To support the import and export...
Category: Internet / Download Managers
Publisher: tradesirTechnology, License: Shareware, Price: USD $0.00, File Size: 1.6 MB
Platform: Windows
The main features:
Automatically generated sitemaps, With yahoo, google and other large-scale search engine standards.
Simulation crawling web Crawlers,can check out the incorrect link.
Set up a free multi-threaded, to generate extremely fast.
Manual support Web site editor, artificial setting parameters such as frequency of updates.
To support the import and export...
Category: Web Authoring / Site Management
Publisher: Changsha Bowu Information Technology Co., License: Shareware, Price: USD $9.00, File Size: 1.6 MB
Platform: Windows
Eliminate cut and paste! Web spider / web crawler using web data extraction / screen scraping technology. Use the web extract for web data mining of contact lists, product catalogs, govt. databases, real estate listings. Our easiest product yet! Creating your own web grabber that can screen scrape the web to a database or Excel has never been easier. Web Scraper Lite delivers the data you...
Category: Internet
Publisher: Velocityscape, License: Shareware, Price: USD $89.00, File Size: 50.9 MB
Platform: Windows
Shareware
Sitemaps are a great SEO tool. They give you a way to stand up and say, "Hey, search engines! Over here! Check out my site." And sitemaps are also a great usability tool. They give your visitors a secondary navigation option, not to mention a peek at how your website is structured. Yep, sitemaps form a near-perfect intersection of SEO and usability. Unfortunately, they're pretty tedious...
Category: Web Authoring / HTML Editors
Publisher: CoffeeCup Software, License: Shareware, Price: USD $34.00, File Size: 6.4 MB
Platform: Windows
Shareware
XFileSharing is an advanced solution for creating professional file sharing services. It can be installed on any virtual/shared or dedicated Linux hosting. You can customize look and feel of your site with basic knowledge of HTML and place banners, adsense or other type advertisement on its pages. You can quickly monetize your service selling premium accounts. Our upload system allow...
Category: Internet / File Sharing
Publisher: SibSoft Ltd., License: Shareware, Price: USD $120.00, File Size: 227.7 KB
Platform: Windows, Mac, Linux, Unix, Source
 |