A web crawler is essential for search engines.

搜索引擎需要网络爬虫。

Site-based Web Crawler is crawlers using site as its basic unit.

基于站点的采集器是一类有针对性的采集器，它以站点作为采集的单位。

Web crawler is an important component for search engine and Web information retrieval system.

Web爬虫是搜索引擎和Web信息检索系统的重要基础.

If using a web crawler, this could be done by publishing a web page on a local site running on IBM HTTP Server.

如果使用的是 web 爬行器，那么可以通过在 IBM HTTP Server 上运行的本地站点中发布一个 web 页面实现。

A MapReduce application or a web crawler application fits perfectly with this model.

典型的如MapReduce框架，或者一个web crawler应用都很适合这个模型。

A new Web Crawler including sophisticated searching process especialized by language!

一个新的网络爬行器,包含了最受欢迎的搜索进程.