Technology sharing

Python crawler consequat PARS VI - sessionem utens ad petitionem inchoandam

2024-07-12

한어Русский языкEnglishFrançaisIndonesianSanskrit日本語DeutschPortuguêsΕλληνικάespañolItalianoSuomalainenLatina

Quare uti session?

Antea induximus quomodo reqesuta utendi ad petitionem inchoandam. Hodie inducemus quomodo sessionem utendi ad petitionem inchoandam. Simplex intellegentia sessionis mechanismum est sessionis. Postquam in navigatro aperimus, non opus est ut iterum aperi cum opera data postea petant parametri crustulum portare. Si reqeusts.request uteris, manually crustulum parametris singulis diebus portare debes. Reqeuest.Session() sessionis obiectum permittit ut certos ambitus per petitiones persistere. Etiam crustulum inter omnes petitiones ab eadem Sessione factas instantia servabit, quare opus est ut crustulum statum singulis diebus manually tractare.

Documenta relatio:
Documenta officialis

quam uti?

Usus modus sessionis reqeust methodo reapse similis est, ac etiam sessionem.get(), session.post(), session.request() aliosque modos sustinet.

s = requests.Session()

s.get('https://httpbin.org/cookies/set/sessioncookie/123456789')
r = s.get('https://httpbin.org/cookies')

print(r.text)
# '{"cookies": {"sessioncookie": "123456789"}}'
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7

Best Practices Quisque Reuse

Permissionum verificationis login requiritur cum per sessionem nonnulla pagina expedienda peti possunt. Crustula consecuta post colligationem in salvari potest, ut quotienscumque postea inire debes, protinus crustulum servatum utere et hoc crustulum uti potes sessionem et petitionem inchoare ne saepe logins. Apta ad concurrentia notitiarum reptilium a pluribus machinis.

Quisque reuse usu:

import json
import traceback

import requests.utils

from executor.page_executor import PageExecutor
from file_path import get_absolute_path


cookie_path = get_absolute_path('data/cookie.txt')
request_session: requests.Session = None

def __load_cookie():
    '''
    加载本地cookie,如果存在加载,如果不存在就返回空
    :param session:
    :return:
    '''
    try:
        with open(cookie_path, "r") as f:
            load_cookie = json.load(f)
            return requests.utils.cookiejar_from_dict(load_cookie)
    except Exception as e:
        traceback.print_exc()
        return None

def get_session():
    global request_session
    if request_session is not None:
        return request_session
    else:
        request_session = requests.Session()
        exist_cookies = __load_cookie()
        if exist_cookies is not None:
            request_session.cookies.update(exist_cookies)

        return request_session

def save_cookie():
    # 登录成功, session里的cookie是最全的,response返回的cookie不全
    cookiejar = requests.utils.dict_from_cookiejar(request_session.cookies)
    with open(cookie_path, "w") as f:
        json.dump(cookiejar, f, indent=True)
    whv_logger.info('cookies saved to ./data/cookie.txt')


def update_cookie():
    '''
    为什么需要一个新的session
    # 走到这一步,说明session已经过期,重新获取session,需要重新处理下session
    # 1. 但是因为携带有旧的session,导致携带旧的__RequestVerificationToken和新的__RequestVerificationToken一起请求,登录失败
    # 2. 所以需要重新处理下session,主要是处理__RequestVerificationToken
    :return:
    '''
    error_cookie_jar = requests.utils.dict_from_cookiejar(request_session.cookies)

    new_cookie_jar = {'__RequestVerificationToken': error_cookie_jar['__RequestVerificationToken']}
    new_cookie = requests.utils.cookiejar_from_dict(new_cookie_jar)

    # 清空旧的cookie
    request_session.cookies.clear_session_cookies()
    # 填充新的cookie
    request_session.cookies.update(new_cookie)
  • 1
  • 2
  • 3
  • 4
  • 5
  • 6
  • 7
  • 8
  • 9
  • 10
  • 11
  • 12
  • 13
  • 14
  • 15
  • 16
  • 17
  • 18
  • 19
  • 20
  • 21
  • 22
  • 23
  • 24
  • 25
  • 26
  • 27
  • 28
  • 29
  • 30
  • 31
  • 32
  • 33
  • 34
  • 35
  • 36
  • 37
  • 38
  • 39
  • 40
  • 41
  • 42
  • 43
  • 44
  • 45
  • 46
  • 47
  • 48
  • 49
  • 50
  • 51
  • 52
  • 53
  • 54
  • 55
  • 56
  • 57
  • 58
  • 59
  • 60
  • 61
  • 62
  • 63