python-requests 모듈의 모든 요청을 기록합니다.

python-requests 모듈의 모든 요청을 기록합니다.

---

나는 파이썬 요청을 사용하고 있습니다. 일부 OAuth활동 을 디버그해야하며 이를 위해 수행중인 모든 요청을 기록하고 싶습니다. 나는이 정보를 얻을 수 ngrep있지만, (에 필요한 그렙 HTTPS 연결 할 수 없습니다 불행히도 OAuth)

Requests액세스 하는 모든 URL (+ 매개 변수)의 로깅을 활성화하려면 어떻게해야 합니까?

---

기본 urllib3라이브러리는 본문이 아닌 logging모듈을 사용하여 모든 새 연결 및 URL을 기록합니다 POST. 대한 GET요청이 충분해야한다 :

import logging

logging.basicConfig(level=logging.DEBUG)

가장 자세한 로깅 옵션을 제공합니다. 로깅 수준 및 대상을 구성하는 방법에 대한 자세한 내용 은 logging HOWTO 를 참조하십시오.

짧은 데모 :

>>> import requests
>>> import logging
>>> logging.basicConfig(level=logging.DEBUG)
>>> r = requests.get('http://httpbin.org/get?foo=bar&baz=python')
INFO:requests.packages.urllib3.connectionpool:Starting new HTTP connection (1): httpbin.org
DEBUG:requests.packages.urllib3.connectionpool:"GET /get?foo=bar&baz=python HTTP/1.1" 200 353

다음 메시지가 기록됩니다.

• INFO: 새 연결 (HTTP 또는 HTTPS)
• INFO: 끊어진 연결
• INFO: 리디렉션
• WARN: 연결 풀이 가득 참 (자주 발생하는 경우 연결 풀 크기를 늘림)
• WARN: 연결 재시도
• DEBUG: 연결 세부 정보 : 방법, 경로, HTTP 버전, 상태 코드 및 응답 길이

---

httplib레벨 ( requests→ urllib3→ httplib) 에서 디버깅을 활성화해야합니다 .

다음은 토글 ( ..._on()및 ..._off())을 전환 하거나 일시적으로 활성화하는 몇 가지 기능 입니다.

import logging
import contextlib
try:
    from http.client import HTTPConnection # py3
except ImportError:
    from httplib import HTTPConnection # py2

def debug_requests_on():
    '''Switches on logging of the requests module.'''
    HTTPConnection.debuglevel = 1

    logging.basicConfig()
    logging.getLogger().setLevel(logging.DEBUG)
    requests_log = logging.getLogger("requests.packages.urllib3")
    requests_log.setLevel(logging.DEBUG)
    requests_log.propagate = True

def debug_requests_off():
    '''Switches off logging of the requests module, might be some side-effects'''
    HTTPConnection.debuglevel = 0

    root_logger = logging.getLogger()
    root_logger.setLevel(logging.WARNING)
    root_logger.handlers = []
    requests_log = logging.getLogger("requests.packages.urllib3")
    requests_log.setLevel(logging.WARNING)
    requests_log.propagate = False

@contextlib.contextmanager
def debug_requests():
    '''Use with 'with'!'''
    debug_requests_on()
    yield
    debug_requests_off()

데모 사용 :

>>> requests.get('http://httpbin.org/')
<Response [200]>

>>> debug_requests_on()
>>> requests.get('http://httpbin.org/')
INFO:requests.packages.urllib3.connectionpool:Starting new HTTP connection (1): httpbin.org
DEBUG:requests.packages.urllib3.connectionpool:"GET / HTTP/1.1" 200 12150
send: 'GET / HTTP/1.1\r\nHost: httpbin.org\r\nConnection: keep-alive\r\nAccept-
Encoding: gzip, deflate\r\nAccept: */*\r\nUser-Agent: python-requests/2.11.1\r\n\r\n'
reply: 'HTTP/1.1 200 OK\r\n'
header: Server: nginx
...
<Response [200]>

>>> debug_requests_off()
>>> requests.get('http://httpbin.org/')
<Response [200]>

>>> with debug_requests():
...     requests.get('http://httpbin.org/')
INFO:requests.packages.urllib3.connectionpool:Starting new HTTP connection (1): httpbin.org
...
<Response [200]>

HEADERS 및 DATA를 포함한 REQUEST 및 HEADERS에 대한 응답을 볼 수 있지만 DATA는 없습니다. 빠진 유일한 것은 기록되지 않은 response.body입니다.

출처

---

Python 3 이상을 사용하는 경우

import requests
import logging
import http.client

http.client.HTTPConnection.debuglevel = 1

logging.basicConfig()
logging.getLogger().setLevel(logging.DEBUG)
requests_log = logging.getLogger("requests.packages.urllib3")
requests_log.setLevel(logging.DEBUG)
requests_log.propagate = True

---

Python 로깅 시스템 ( import logging)이 낮은 수준의 디버그 로그 메시지를 내보내 도록 시도 할 때 다음과 같은 사실을 발견하는 것이 놀랍습니다.

requests --> urllib3 --> http.client.HTTPConnection

that only urlb3 actually uses the Python logging system:

• requests no
• http.client.HTTPConnection no
• urllib3 yes

Sure, you can extract debug messages from HTTPConnection by setting:

HTTPConnection.debuglevel = 1

but these outputs are merely emitted via the print statement. To prove this, simply grep the Python 3.7 client.py source code and view the print statements yourself (thanks @Yohann):

curl https://raw.githubusercontent.com/python/cpython/3.7/Lib/http/client.py |grep -A1 debuglevel` 

Presumably redirecting stdout in some way might work to shoe-horn stdout into the logging system and potentionally capture to e.g. a log file.

Choose the 'urlib3' logger not 'requests.packages.urllib3'

To capture urlib3 debug information through the Python 3 logging system, contrary to much advice on the internet, and as @MikeSmith points out, you won’t have much luck intercepting:

log = logging.getLogger('requests.packages.urllib3')

instead you need to:

log = logging.getLogger('urllib3')

Debugging url3lib to a log file

Here is some code which logs url3lib workings to a log file using the Python logging system:

import requests
import logging
from http.client import HTTPConnection  # py3

# log = logging.getLogger('requests.packages.urllib3')  # useless
log = logging.getLogger('urllib3')  # works

log.setLevel(logging.DEBUG)  # needed
fh = logging.FileHandler("requests.log")
log.addHandler(fh)

requests.get('http://httpbin.org/')

the result:

Starting new HTTP connection (1): httpbin.org:80
http://httpbin.org:80 "GET / HTTP/1.1" 200 3168

Enabling the HTTPConnection.debuglevel print() statements

If you set HTTPConnection.debuglevel = 1

from http.client import HTTPConnection  # py3
HTTPConnection.debuglevel = 1
requests.get('http://httpbin.org/')

you'll get the print statement output of additional juicy low level info:

send: b'GET / HTTP/1.1\r\nHost: httpbin.org\r\nUser-Agent: python- 
requests/2.22.0\r\nAccept-Encoding: gzip, deflate\r\nAccept: */*\r\nConnection: keep-alive\r\n\r\n'
reply: 'HTTP/1.1 200 OK\r\n'
header: Access-Control-Allow-Credentials header: Access-Control-Allow-Origin 
header: Content-Encoding header: Content-Type header: Date header: ...

Remember this output uses print and not the Python logging system, and thus cannot be captured using a traditional logging stream or file handler (though it may be possible to capture output to a file by redirecting stdout).

Combine the two above - maximise all possible logging to console

To maximise all possible logging, you must settle for console/stdout output with this:

import requests
import logging
from http.client import HTTPConnection  # py3

log = logging.getLogger('urllib3')
log.setLevel(logging.DEBUG)

# logging from urllib3 to console
ch = logging.StreamHandler()
ch.setLevel(logging.DEBUG)
log.addHandler(ch)

# print statements from `http.client.HTTPConnection` to console/stdout
HTTPConnection.debuglevel = 1

requests.get('http://httpbin.org/')

giving the full range of output:

Starting new HTTP connection (1): httpbin.org:80
send: b'GET / HTTP/1.1\r\nHost: httpbin.org\r\nUser-Agent: python-requests/2.22.0\r\nAccept-Encoding: gzip, deflate\r\nAccept: */*\r\nConnection: keep-alive\r\n\r\n'
reply: 'HTTP/1.1 200 OK\r\n'
http://httpbin.org:80 "GET / HTTP/1.1" 200 3168
header: Access-Control-Allow-Credentials header: Access-Control-Allow-Origin 
header: Content-Encoding header: ...

---

I'm using python 3.4, requests 2.19.1:

'urllib3' is the logger to get now (no longer 'requests.packages.urllib3'). Basic logging will still happen without setting http.client.HTTPConnection.debuglevel

참고URL : https://stackoverflow.com/questions/16337511/log-all-requests-from-the-python-requests-module