nagooglesearch
v8.0
不是另一個Google搜索庫。只是開玩笑 - 是的。
出於教育目的而設計。我希望它會有所幫助!
未來計劃:
pip3 install nagooglesearch
pip3 install --upgrade nagooglesearch運行以下命令:
git clone https://github.com/ivan-sincek/nagooglesearch && cd nagooglesearch
python3 -m pip install --upgrade build
python3 -m build
python3 -m pip install dist/nagooglesearch-8.0-py3-none-any.whl預設值:
nagooglesearch . GoogleClient (
tld = "com" ,
homepage_parameters = {
"btnK" : "Google+Search" ,
"source" : "hp"
},
search_parameters = {
},
user_agent = "" ,
proxy = "" ,
max_results = 100 ,
min_sleep = 8 ,
max_sleep = 18 ,
debug = False
)只有沒有它們的關鍵字google而不是以關鍵字goo.gl結尾的域才被接受為有效結果。最終輸出是URL的唯一且分類的列表。
示例,標準:
import nagooglesearch
# the following query string parameters are set only if 'start' query string parameter is not set or is equal to zero
# simulate a homepage search
homepage_parameters = {
"btnK" : "Google+Search" ,
"source" : "hp"
}
# search the internet for additional query string parameters
search_parameters = {
"q" : "site:*.example.com intext:password" , # search query
"tbs" : "li:1" , # specify 'li:1' for verbatim search (no alternate spellings, etc.)
"hl" : "en" ,
"lr" : "lang_en" ,
"cr" : "countryUS" ,
"filter" : "0" , # specify '0' to display hidden results
"safe" : "images" , # specify 'images' to turn off safe search, or specify 'active' to turn on safe search
"num" : "80" # number of results per page
}
client = nagooglesearch . GoogleClient (
tld = "com" , # top level domain, e.g., www.google.com or www.google.hr
homepage_parameters = homepage_parameters , # 'search_parameters' will override 'homepage_parameters'
search_parameters = search_parameters ,
user_agent = "curl/3.30.1" , # a random user agent will be set if none is provided
proxy = "socks5://127.0.0.1:9050" , # one of the supported URL schemes are 'http[s]', 'socks4[h]', and 'socks5[h]'
max_results = 200 , # maximum unique URLs to return
min_sleep = 15 , # minimum sleep between page requests
max_sleep = 30 , # maximum sleep between page requests
debug = True # enable debug output
)
urls = client . search ()
if client . get_error () == "REQUESTS_EXCEPTION" :
print ( "[ Requests Exception ]" )
# do something
elif client . get_error () == "429_TOO_MANY_REQUESTS" :
print ( "[ HTTP 429 Too Many Requests ]" )
# do something
for url in urls :
print ( url )
# do something如果將max_results設置為,例如200 ,例如80 , num可以返回的最大唯一URL實際上可以達到240 。
在此處查看用戶代理列表。對於更多用戶代理,請檢查scrapeops.io。
例如,最短:
import nagooglesearch
urls = nagooglesearch . GoogleClient ( search_parameters = { "q" : "site:*.example.com intext:password" }). search ()
# do something例如,不要顯示超過6個月的結果:
import nagooglesearch , dateutil . relativedelta as relativedelta
def get_tbs ( months ):
today = datetime . datetime . today ()
return nagooglesearch . get_tbs ( today , today - relativedelta . relativedelta ( months = months ))
search_parameters = {
"tbs" : get_tbs ( 6 )
}
# do something例如,獲取所有用戶代理:
import nagooglesearch
user_agents = nagooglesearch . get_all_user_agents ()
print ( user_agents )
# do something例如,獲取一個隨機用戶代理:
import nagooglesearch
user_agent = nagooglesearch . get_random_user_agent ()
print ( user_agent )
# do something