Free Downloads: Crawlers
License: All | Free
Manage and track visits by robots, spiders and crawlers to your website.
Manage and track visits by robots, spiders and Crawlers to your website. Identify which spiders visited, when they visited, and what pages they indexed. Direct spiders to your most important pages, and exclude them from non-public pages. Maximizes effectiveness with search engines by avoiding over/under submitting. Excellent to improve search engine placements for users of automatic...
Category: Internet / Tools & Utilities
Publisher: Website Management Tools, License: Shareware, Price: USD $129.95, File Size: 3.3 MB
Platform: Windows
ETagFix is an add-on for Microsoft Internet Information Server (IIS) or any web server that supports ISAPI filters, that cuts off part of the etag that IIS produces so that is does not change every time your computer is rebooted. Etag is an HTTP header that is produced by IIS to allow web Crawlers and user clients to check against to see if a web page has changed. ETagFix fixes the...
Category: Internet / Tools & Utilities
Publisher: ISAPILabs, License: Freeware, Price: USD $0.00, File Size: 696.3 KB
Platform: Windows
The web crawler is a program that automatically traverses the web by downloading the pages and following the links from page to page. A general purpose of web crawler is to download any web page that can be accessed through the links.
This process is called web crawling or spidering. Many sites, in particular search engines, use spidering as a means of providing up-to-date data. Web...
Category: Internet / Misc. Plugins
Publisher: VietSpider, License: Freeware, Price: USD $0.00, File Size: 48.8 MB
Platform: Windows, Linux
Freeware
Free Large Android Icons picture various space creatures, such as robots, Crawlers and androids. Not only does the set present the creatures themselves, it also shows their moods and actions. All the images are created by professional artists and demonstrate bright colors and well-finished edges. This free icon set is a perfect choice for your personal desktop customization. Each...
Category: Desktop Enhancements / Icon Tools
Publisher: large-icons.com, License: Freeware, Price: USD $0.00, File Size: 13.7 MB
Platform: Windows, Mac, Linux,
Generates XML Sitemap, REQUIRED for search engine optimization for Google, MSN, Yahoo - all at once!
Generates XML Sitemap, REQUIRED for search engine optimization for Google, MSN, Yahoo - all at once! Search engine rivals Google, Yahoo and Microsoft are teaming up to make it easier for Web site owners to make sure their sites get included in the Web indexes, the companies are expected to announce Thursday, Oct 16, 2006. Now Web site owners will be able to go to one place for alerting all...
Category: Web Authoring / Site Management
Publisher: MagneticOne, License: Commercial, Price: USD $39.00, File Size: 29.0 KB
Platform: Windows, Linux, Unix
Generates XML Sitemap, REQUIRED for search engine optimization for Google, MSN, Yahoo - all at once!
Generates XML Sitemap, REQUIRED for search engine optimization for Google, MSN, Yahoo - all at once!
Search engine rivals Google, Yahoo and Microsoft are teaming up to make it easier for Web site owners to make sure their sites get included in the Web indexes, the companies are expected to announce Thursday, Oct 16, 2006.
Now Web site owners will be able to go to one place for...
Category: Web Authoring / Web Design Utilities
Publisher: MagneticOne, License: Commercial, Price: USD $39.00, File Size: 30.0 KB
Platform: Windows, Linux
Freeware
KeyWord generator is free powerfull tool to generate keywords and save them. Using KeyWord generator lite you can generate more than 100 keywords and use them in your meta tags, titles or content of your web page. Keywords are really powerfull way to make search engines remember about you. Meta keywords can be extremely effective if they feat content properly. But beware of spam keywording....
Category: Multimedia & Design / Button Creators
Publisher: Oksima Plus s.r.o., License: Freeware, Price: USD $0.00, File Size: 141.0 KB
Platform: Windows
Why do you need Fake Referrals? Have you ever seen sites with 'last 10 referrers' area? Just go to Yahoo! and input this query string 'link:http://cialis-online-buy.surf.to'. Then click search. Almost each site from results contains a small area with referrals. Those areas are indexed by search engines too. You found results using search engine? Crawlers find same results too. You do...
Category: Internet
Publisher: Oksima team, License: Shareware, Price: USD $35.00, File Size: 1.2 MB
Platform: Windows
Shareware
Why do you need Log Advertiser? Have you ever seen sites with 'last 10 referrers' area? Just go to Yahoo! and input this query string. Then click search. Almost each site from results contains a small area with referrals. Those areas are indexed by search engines too. You found results using search engine? Crawlers find same results too. You do not need any expensive advertising to...
Category: Multimedia & Design / Button Creators
Publisher: Oksima Plus s.r.o., License: Shareware, Price: USD $30.00, File Size: 5.5 MB
Platform: Windows
Build a custom web spider / web crawler using web data extraction / screen scraping technology. Use the web extract for web data mining of contact lists, product catalogs, government databases, real estate listings, or build a custom email extractor. Creating your own web grabber that can screen scrape the data to a database or Excel has never been easier. The website spider /...
Category: Utilities
Publisher: Velocityscape, License: Shareware, Price: USD $419.00, File Size: 51.3 MB
Platform: Windows
SearchBlox is an out-of-the-box Enterprise Search Solution built on top of Apache Lucene.
SearchBlox is an out-of-the-box Enterprise Search Solution built on top of Apache Lucene. It is fast to deploy, easy to manage and available for both on-premise and cloud deployment.
Best of all, it is free. No limitations. No restrictions.
SearchBlox provides Crawlers for Filesystems, HTTP/HTTPS and RSS Feeds that work out-of-the-box. The HTTP/HTTPS crawler can index...
Category: Internet / Web Search Utilities
Publisher: SearchBlox Software, Inc., License: Shareware, Price: USD $0.00, File Size: 0
Platform: Windows
This program lets you to substitute URI for symbol codes of the ASCII table.
This program lets you to substitute URI for symbol codes of the ASCII table. It's very useful to prevent scanning of web pages by the Crawlers gathering e-mail addresses for broadcasting spam messages on the next step. Using this software you can protect your mailbox from tons of junk mail, and will not be afraid of publishing contacts on your home page or a company's web site....
Category: Security & Privacy / Encrypting
Publisher: Loonies Software, License: Freeware, Price: USD $0.00, File Size: 63.3 KB
Platform: Windows
Files Crawler is a file search application.
Files Crawler is a file search application. It lets you search tons of files and download music, videos, images, documents and software. Files Crawler is actually performing its search at Google. As Google allows us to search for more than just Web pages, due to its special indexing method of directory listing. Directory listing of Websites, is actually what allows Google to see all sort of...
Category: Internet / Tools & Utilities
Publisher: Speedapps Inc, License: Freeware, Price: USD $0.00, File Size: 747.5 KB
Platform: Windows
Super Arcade Machine is an ultimate 25 game pack including arkanoid and breakout style games, pacman style games and frantic space shooters. Incredible music, sound and graphics with many configurable options in each of the titles. Most games include many features.
This package of games is fun for the whole family and features 25 neat and addicting games including Animal Army,...
Category: Games / Simulations
Publisher: Super Arcade Machine, License: Shareware, Price: USD $19.99, File Size: 103.6 MB
Platform: Windows
Create your robots.txt files in seconds and maintain them easilly and effortlesly with the unique utility Advanced Robots.txt Generator. Advanced Robots.txt Generator is the most complete and easy to use application that allows you to create, maintain and keep track of your Robots.txt files for all your websites. The following is a list of the features of Advanced Robots.txt Generator. *...
Category: Web Authoring / HTML Editors
Publisher: BasiSoft Software Solutions, License: Shareware, Price: USD $35.00, File Size: 2.1 MB
Platform: Windows
Online Web-based applications are increasingly at risk from professional hackers who target such applications in order to commit data theft or fraud.
The D-Guard Web Application Firewall is a powerful security tool for Web applications and Web sites.
D-Guard Web Application Firewall is designed to combat all attack types that have been categorized as significant threats,...
Category: Security & Privacy / Access Control
Publisher: D-Guard Technology Co., Ltd., License: Shareware, Price: USD $6999.00, File Size: 4.6 MB
Platform: Windows
Online Web-based applications are increasingly at risk from professional hackers who target such applications in order to commit data theft or fraud.
The D-Guard Web Application Firewall is a powerful security tool for Web applications and Web sites.
D-Guard Web Application Firewall is designed to combat all attack types that have been categorized as significant threats,...
Category: Security & Privacy / Access Control
Publisher: D-Guard Technology Co., Ltd., License: Shareware, Price: USD $399.00, File Size: 4.6 MB
Platform: Windows
Create your robots.txt files in seconds and maintain them easilly and effortlesly with the unique utility Advanced Robots.txt Generator. Advanced Robots.txt Generator is the most complete and easy to use application that allows you to create, maintain and keep track of your Robots.txt files for all your websites. The following is a list of the features of Advanced Robots.txt Generator. *...
Category: Software Development
Publisher: BasiSoft Solutions, License: Shareware, Price: USD $65.00, File Size: 1.5 MB
Platform: Windows
The InnerPeace WebMaster Toolkit is a collection of essensial tools assembeled and made by the Webmasters of InnerPeace. Some of these essensial tools are not available anywhere else. Includes a fast, easy 4 hexillion color combo picker, PageJumper web page forwarder, EScrambler to scrambler your email addresses and hide them from spamcrawlers, plus some excellent articles. Colors on web pages...
Category: Utilities / Misc. Utilities
Publisher: innerpeace.org, License: Freeware, Price: USD $0.00, File Size: 370.0 KB
Platform: Windows, Linux, Mac, Palm
Make a web of links to your posts and you will get a lot of new readers. Blogspreading is a small free utility, which automatically appends a link to the latest post in your blog to your e-mail signature. This will easily catch the recipients' interest and convert them to the readers of your blog. All e-mail clients allow automatic adding of a signature to outgoing message. If you use an...
Category: Internet
Publisher: Softvoile, License: Freeware, Price: USD $0.00, File Size: 578.8 KB
Platform: Windows
Cable Player is a Entry Level digital video player for cable TV operators.
Cable Player is a Entry Level digital video player for cable TV operators. With Cable Player you need not worry about faulty VCRs’, VHS tapes, VCD Players or even the staff running your Cable Channel. All you have got to do is load movies, songs, ads, promos for days on the computer, set the sequence and let Cable Player do the rest. Cable Player will play your movies, songs along with...
Category: Multimedia & Design / Video
Publisher: Trinity Technologies, License: Freeware, Price: USD $0.00, File Size: 4.1 MB
Platform: Windows
The main features:
Automatically generated sitemaps, With yahoo, google and other large-scale search engine standards.
Simulation crawling web Crawlers,can check out the incorrect link.
Set up a free multi-threaded, to generate extremely fast.
Manual support Web site editor, artificial setting parameters such as frequency of updates.
To support the import and export...
Category: Internet / Communications
Publisher: Changsha aeke Information Technology Co., License: Shareware, Price: USD $0.00, File Size: 1.6 MB
Platform: Windows
The main features:
Automatically generated sitemaps, With yahoo, google and other large-scale search engine standards.
Simulation crawling web Crawlers,can check out the incorrect link.
Set up a free multi-threaded, to generate extremely fast.
Manual support Web site editor, artificial setting parameters such as frequency of updates.
To support the import and export...
Category: Internet / Communications
Publisher: obdkit Technology, License: Shareware, Price: USD $0.00, File Size: 1.6 MB
Platform: Windows
The main features:
Automatically generated sitemaps, With yahoo, google and other large-scale search engine standards.
Simulation crawling web Crawlers,can check out the incorrect link.
Set up a free multi-threaded, to generate extremely fast.
Manual support Web site editor, artificial setting parameters such as frequency of updates.
To support the import and export...
Category: Internet / Download Managers
Publisher: tradesirTechnology, License: Shareware, Price: USD $0.00, File Size: 1.6 MB
Platform: Windows
The main features:
Automatically generated sitemaps, With yahoo, google and other large-scale search engine standards.
Simulation crawling web Crawlers,can check out the incorrect link.
Set up a free multi-threaded, to generate extremely fast.
Manual support Web site editor, artificial setting parameters such as frequency of updates.
To support the import and export...
Category: Web Authoring / Site Management
Publisher: Changsha Bowu Information Technology Co., License: Shareware, Price: USD $9.00, File Size: 1.6 MB
Platform: Windows
 |