网络 Crawler software
Thread count: the number of concurrent working threads in the crawler.
线程数: 在爬虫软件并发工作线程的数量。
—— 给力词典精选The crawler works with multiple threads to fetch URLs from the crawler queue.
爬虫软件使用多个线程从队列爬虫获取url。
—— 给力词典精选My simple crawler contains three views that can follow the crawling process, check the details, and view the crawling errors.
我的简单爬虫软件包含三个视图可以爬行的过程,检查细节,并查看抓取错误。
—— 给力词典精选Thread sleep time between two connections: the time that each thread sleeps after handling any request, which is a very important value to prevent hosts from blocking the crawler due to heavy loads.
两个连接之间的线程睡眠时间:每个线程的时间睡在处理任何请求,这是一个非常重要的价值,以防止主机挡住了爬虫软件由于沉重的负荷。
—— 给力词典精选