Python: Use urllib.request to do http request

In this article, I will show you how to execute HTTP requests with urllib.request.

Environment

• MacOS 11.2
• Python3.9

Workflow

• Create a request by urllib.request.Request method.
• Assign the request as a parameter of urllib.request.urlopen method to get response.
• use json module to parse the response.

urllib.request.Request

urllib.request.Request(reqUrl, reqData, headers, method)

• reqUrl: url of request
• reqData: post data
• headers: request's header
• method: The default is 'GET' if reqData is None or 'POST' otherwise.

urllib.request.urlopen

urllib.request.urlopen(req)

If the response is in json format, use json.load to parse the response.

Here is an example.

with urllib.request.urlopen(req) as res:
    body = json.load(res)
    ... # Do something you want.

Sample

The following example tries an HTTP request using an unauthenticated API:https://randomuser.me/api/.
https://randomuser.me/api/ is easy to use, and it returns fake user information.

import urllib.request
import json
reqUrl = 'https://randomuser.me/api/'
reqData = None
headers = {
    'Content-Type': 'application/json'
}
method = 'GET'
req = urllib.request.Request(reqUrl, reqData, headers, method=method)
with urllib.request.urlopen(req) as res:
  body = json.load(res)
  print(body)
  print(f"gender: {body['results'][0]['gender']}")
  print(f"name: {body['results'][0]['name']['first']} {body['results'][0]['name']['last']}")
  print(f"country: {body['results'][0]['location']['country']}")

Reference

urllib.request — Extensible library for opening URLs
Free API – Top 15 APIs You Can Use For Free (No Key Needed)

This is a simple way to execute HTTP requests with urllib.request.