Unix Domain Sockets with urllib

Convenient Python HTTP clients such as requests, aiohttp, or urllib3 and derivatives are ubiquitous but still need to be installed as external dependencies. Without additional requirements, one has to stick with the standard but more restricted urllib, though.

HTTP endpoints are not always provided by “ordinary” TCP, but can also base on Unix Domain Sockets. Amongst others, the Docker daemon API is exposed on such a local UDS per default. However, unix connections as underlying transport are not readily provided, and extending the library in this regard seems to be poorly documented. The following gist extends HTTPConnection to add handling of local socket filenames.

import urllib.request
import http.client
import socket

class UnixHTTPConnection(http.client.HTTPConnection):
    def __init__(self, *args, **kwargs) -> None:
        self._unix_path: str = kwargs.pop("unix_path")
        super().__init__(*args, **kwargs)

    def connect(self) -> None:
        self.sock = socket.socket(family=socket.AF_UNIX, type=socket.SOCK_STREAM)
        if self.timeout is not socket._GLOBAL_DEFAULT_TIMEOUT:
            self.sock.settimeout(self.timeout)
        self.sock.connect(self._unix_path)
        if self._tunnel_host:
            self._tunnel()

class UnixHTTPHandler(urllib.request.AbstractHTTPHandler):
    def __init__(self, unix_path: str) -> None:
        self._unix_path: str = unix_path
        super().__init__()

    def unix_open(self, request: urllib.request.Request) -> http.client.HTTPResponse:
        return self.do_open(UnixHTTPConnection, request, unix_path=self._unix_path)

    def unix_request(self, request: urllib.request.Request) -> urllib.request.Request:
        return self.do_request_(request)

The HTTP handler for Unix Domain Sockets can be registered using the OpenerDirector interface. By this, all requests to URLs with the unix:// protocol will get routed to a certain socket file. For example, directly listing all Docker containers with via /var/run/docker.sock:

if __name__ == "__main__":
    opener: urllib.request.OpenerDirector = urllib.request.build_opener(
        UnixHTTPHandler("/var/run/docker.sock")
    )
    # urllib.request.install_opener(opener)  # set as global default for urlopen() or use directly:
    with opener.open("unix://localhost/containers/json", timeout=10.0) as response:
        print(json.loads(response.read().decode()))