Software Listing of Author : "Vietspider"

The web crawler is a program that automatically traverses the web by downloading the pages and following the links from page to page. A general purpose of web crawler is to download any web page that can be accessed through the links.

This process is called web crawling or spidering. Many sites, in particular search engines, use spidering as a means of providing up-to-date data. Web crawlers are mainly used to create a copy of all the visited pages for later processing by a search engine that will index the downloaded pages to provide fast searches. Crawlers can also be used for automating maintenance tasks on a website, such as checking links or validating HTML code. Also, crawlers can be used to gather specific types of information from Web pages, such as harvest ing e-mail addresses (usually for spam).

A web crawler is...

  • Platform: Linux, Win2000, Windows 7, WinOther, WinVista
New Reviews

Autodesk Smoke

Autodesk is the company well known for making some of the most feature rich and versatile tools for 3D modeling, animation and engineering design related work. Its Smoke is a Mac OS specific app that is aimed at professionals involved in ...


Nearly all PC users need to use various utilities to enhance performance of their laptop or desktop from time to time. Windows inbuilt utilities can serve the purpose but when you can find more capable third party apps at low or zero cost, why ...

FileStream TurboBackup

Nowadays, data security and backup have become commonplace activity for nearly all PC users. You need to backup precious data either in external media or cloud storage services to evade circumstances like hard disk failure or malware invasion. There ...


There are so many image editors out there but when it comes to powerful and versatile RAW file editors, the choices are somewhat limited. With Adobe switching to Cloud-based subscriptions for most of its apps, semiprofessional users including ...

McAfee AntiVirus Plus

In PC security sector, McAfee is a big name. Along with Symantec, this is one of the contenders catering to the PC users for a long time. However, with time, several new contenders have appeared on horizon and some of them are giving the old players ...

Norman Antivirus

There is hardly any computer user who does not require protection from various types of malware and internet based threats these days. Instances of malware creators and hackers breaking into top corporate firms and celebrity websites can send a ...

IDrive Classic

For every type of computer user, backing up data has become near compulsory. Apart from data loss caused by hardware snags, you cannot rule out the risk of data loss caused by malware attacks. That explains why millions of PC users back up their ...

New Downloads


ProbeIT is a comparison tool
to help you diagnose the
issues related to missing
setup, wrong version of
program files or ...

Free File Encrypter

Free file encryptor is
designed to encrypt files of
any types like audio, video,
text, images, EXE programs,
etc. in few ...

Fax Server Pro

Fax Server Pro is a client and
server system designed to
integrates fax communication
with your IT infrastructure.

DTM Data Modeler

DTM Data Modeler is a CASE
tool for database developers
that supports both forward and
reverse engineering. It is an

Nevron Gauge for SharePoint

Nevron Gauge for SharePoint is
an advanced web part, which
delivers a complete set of
radial and linear gauges,
numeric ...

Exe Icon Changer

The tool can replace the icon
in the executable file! Even
the executable file compressed
or the size of the icon is ...

EximiousSoft Banner Maker

EximiousSoft Banner Maker is a
powerful yet easily software
for ceating banner ads.web
buttons and headers etc. web

dbForge Data Generator for
SQL Server

dbForge Data Generator for SQL
Server is a GUI tool for a
fast generation of large
volumes of SQL Server test
table data. ...

Sounding Keyboard and Mouse

It Lets you hear magic sound
when you press keys or click
mouse. The program allows you
to hear the sound or the words


StarMule is a new file sharing
program that includes all new
p2p optimizations, enabling
users to search and download