Python Selenium automation without login (cookie and token)

2024-07-12

Table of contents

Using login can reduce the repeated operations of each login, directly operate the menu page after the system login, and can also reduce the security verification login, such as image verification login operations. Note: Cookies and tokens have validity periods.

Get cookies directly from the developer tools and add them. The figure below shows the cookies of multiple sites on the web page. Select the ones you need and add them.


from selenium import webdriver
from selenium.webdriver.edge.options import Options
 
# 一般只需要name和value
cookie = {'name': 'ZY44', 'value': 'tLonhTkz50iHzxjhIsaaaafferr:C'}
 
options = Options()
# options.add_argument('--headless')
wd = webdriver.Edge(options=options)
 
wd.add_cookie(cookie_dict=cookie)
 
#for c in cookies:   # 如果是多个cookie要添加，cookies存储为列表是，使用循环添加
#   wd.add_cookie(c)
 
wd.refresh()  # 刷新页面
 
wd.get(URL)
 
wd.quit()

raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.InvalidCookieDomainException: Message: invalid cookie domain
(Session info: MicrosoftEdge=126.0.2592.87)

If the above error occurs, add a line of wd.get(URL) before wd.add_cookie(cookie_dict=cookie), as follows:


wd.get(URL)
wd.add_cookie(cookie_dict=cookie)
wd.get(URL)

Get cookies through the interface

After obtaining the cookie data through the interface, add the cookie to selenium for use


def get_cookies():
    headers = {
        'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36',
    }
    host = 'https://baike.baidu.com'
    req = requests.get(host, headers=headers)
    cookie_data = req.cookies.get_dict()
    cookies = []
    for key, value in cookie_data.items():
        cookies.append(dict(name=key, value=value))
    return cookies

Enable browser to bypass login

Type in Google or Edge browser:chrome://version/ 或 edge://version/ 查看配置文件夹路径，Remove the Default at the end, and then add --user-data-dir= before the path to splice out the path we want.

profile_directory = r'--user-data-dir=C:UsersxxxAppDataLocalMicrosoftEdgeUser Data'


# 这里使用模糊匹配，把edge开头的进程都杀掉
if platform.system() == "Windows":
    os.system("taskkill -im msedge* -f")
else:
    os.system("killall -9 msedge*")
 
user_data = r'C:UsersxxxAppDataLocalMicrosoftEdgeUser Data'
profile_directory = rf'--user-data-dir={user_data}'
 
options = Options()
# options.add_argument('--headless')
options.add_argument(profile_directory)
wd = webdriver.Edge(options=options)
 
wd.maximize_window()
wd.get(URL)
 
wd.quit()

Note: This method requires closing the corresponding browser program when used, otherwise an error will be reported, so the corresponding browser process needs to be killed before execution. The above code uses fuzzy matching query to kill the process, and the following is a full match.


returnCode=os.system('taskkill /F /iM chrome.exe')    # 谷歌
returnCode=os.system('taskkill /F /iM iexplore.exe')  # IE
returnCode=os.system('taskkill /F /iM firefox.exe')  # 火狐
returnCode=os.system('taskkill /F /iM msedge.exe')  # edge
assert returnCode==0 #判断浏览器进程是否杀完

Adding a token


    token = "my_token"
 
    options = Options()
    options.add_argument('--headless')
    wd = webdriver.Edge(options=options)
 
    wd.execute_script("window.localStorage.setItem('token', '%s');" % token)  # 使用selenium执行js的操作添加token
 
    wd.maximize_window()
    wd.get(url)
    wd.quit()

Technology Sharing