nagooglesearch
v8.0
不是另一个Google搜索库。只是开玩笑 - 是的。
出于教育目的而设计。我希望它会有所帮助!
未来计划:
pip3 install nagooglesearch
pip3 install --upgrade nagooglesearch运行以下命令:
git clone https://github.com/ivan-sincek/nagooglesearch && cd nagooglesearch
python3 -m pip install --upgrade build
python3 -m build
python3 -m pip install dist/nagooglesearch-8.0-py3-none-any.whl默认值:
nagooglesearch . GoogleClient (
tld = "com" ,
homepage_parameters = {
"btnK" : "Google+Search" ,
"source" : "hp"
},
search_parameters = {
},
user_agent = "" ,
proxy = "" ,
max_results = 100 ,
min_sleep = 8 ,
max_sleep = 18 ,
debug = False
)只有没有它们的关键字google而不是以关键字goo.gl结尾的域才被接受为有效结果。最终输出是URL的唯一且分类的列表。
示例,标准:
import nagooglesearch
# the following query string parameters are set only if 'start' query string parameter is not set or is equal to zero
# simulate a homepage search
homepage_parameters = {
"btnK" : "Google+Search" ,
"source" : "hp"
}
# search the internet for additional query string parameters
search_parameters = {
"q" : "site:*.example.com intext:password" , # search query
"tbs" : "li:1" , # specify 'li:1' for verbatim search (no alternate spellings, etc.)
"hl" : "en" ,
"lr" : "lang_en" ,
"cr" : "countryUS" ,
"filter" : "0" , # specify '0' to display hidden results
"safe" : "images" , # specify 'images' to turn off safe search, or specify 'active' to turn on safe search
"num" : "80" # number of results per page
}
client = nagooglesearch . GoogleClient (
tld = "com" , # top level domain, e.g., www.google.com or www.google.hr
homepage_parameters = homepage_parameters , # 'search_parameters' will override 'homepage_parameters'
search_parameters = search_parameters ,
user_agent = "curl/3.30.1" , # a random user agent will be set if none is provided
proxy = "socks5://127.0.0.1:9050" , # one of the supported URL schemes are 'http[s]', 'socks4[h]', and 'socks5[h]'
max_results = 200 , # maximum unique URLs to return
min_sleep = 15 , # minimum sleep between page requests
max_sleep = 30 , # maximum sleep between page requests
debug = True # enable debug output
)
urls = client . search ()
if client . get_error () == "REQUESTS_EXCEPTION" :
print ( "[ Requests Exception ]" )
# do something
elif client . get_error () == "429_TOO_MANY_REQUESTS" :
print ( "[ HTTP 429 Too Many Requests ]" )
# do something
for url in urls :
print ( url )
# do something如果将max_results设置为,例如200 ,例如80 , num可以返回的最大唯一URL实际上可以达到240 。
在此处查看用户代理列表。对于更多用户代理,请检查scrapeops.io。
例如,最短:
import nagooglesearch
urls = nagooglesearch . GoogleClient ( search_parameters = { "q" : "site:*.example.com intext:password" }). search ()
# do something例如,不要显示超过6个月的结果:
import nagooglesearch , dateutil . relativedelta as relativedelta
def get_tbs ( months ):
today = datetime . datetime . today ()
return nagooglesearch . get_tbs ( today , today - relativedelta . relativedelta ( months = months ))
search_parameters = {
"tbs" : get_tbs ( 6 )
}
# do something例如,获取所有用户代理:
import nagooglesearch
user_agents = nagooglesearch . get_all_user_agents ()
print ( user_agents )
# do something例如,获取一个随机用户代理:
import nagooglesearch
user_agent = nagooglesearch . get_random_user_agent ()
print ( user_agent )
# do something