REST client¶

The HTTP client sources fetch with, its authentication schemes and its paginators.

RESTClient ¶

RESTClient(
    base_url: str,
    auth: Auth | None = None,
    timeout: float | None = None,
    headers: dict[str, str] | None = None,
    params: dict[str, str] | None = None,
    **kwargs: Any,
)

Bases: httpx2.Client

A REST client that extends httpx2.Client with pagination support.

Initialize the REST client.

Parameters:

Name	Type	Description	Default
`base_url`	`str`	The base URL of the API.	required
`auth`	`Auth \| None`	The authentication method (httpx2.Auth instance).	`None`
`timeout`	`float \| None`	The timeout for requests.	`None`
`headers`	`dict[str, str] \| None`	The headers to include in requests.	`None`
`params`	`dict[str, str] \| None`	The parameters to include in requests.	`None`
`**kwargs`	`Any`	Additional keyword arguments to pass to httpx2.Client.	required

paginate ¶

paginate(
    path: str,
    paginator: BasePaginator,
    *,
    params: dict[str, Any] | None = None,
    headers: dict[str, str] | None = None,
    data_selector: DataSelector = None,
) -> Iterator[Any]

Walk a paginated resource, yielding one page of selected data at a time.

Sequential by nature; each next request is derived from the previous response per paginator. (The async client additionally fans out RangePaginator pages concurrently.)

Parameters:

Name	Type	Description	Default
`path`	`str`	The resource path.	required
`paginator`	`BasePaginator`	The pagination strategy.	required
`params`	`dict[str, Any] \| None`	Static query params applied to every page request.	`None`
`headers`	`dict[str, str] \| None`	Extra headers applied to every page request.	`None`
`data_selector`	`DataSelector`	How to pull each page's records from the response.	`None`

Yields:

Type	Description
`Any`	Each page's selected data.

AsyncRESTClient ¶

AsyncRESTClient(
    base_url: str,
    auth: Auth | None = None,
    timeout: float | None = None,
    headers: dict[str, str] | None = None,
    params: dict[str, str] | None = None,
    **kwargs: Any,
)

Bases: httpx2.AsyncClient

Async counterpart to RESTClient for IO-bound extraction.

Same construction surface as RESTClient; a connection exposes it as client and uses it from an async def data() to overlap independent requests (paginated pages via paginate, per-entity calls via interloper.utils.bounded_gather).

Initialize the async REST client.

Parameters:

Name	Type	Description	Default
`base_url`	`str`	The base URL of the API.	required
`auth`	`Auth \| None`	The authentication method (httpx2.Auth instance).	`None`
`timeout`	`float \| None`	The timeout for requests.	`None`
`headers`	`dict[str, str] \| None`	The headers to include in requests.	`None`
`params`	`dict[str, str] \| None`	The parameters to include in requests.	`None`
`**kwargs`	`Any`	Additional keyword arguments to pass to httpx2.AsyncClient.	required

paginate ¶

paginate(
    path: str,
    paginator: BasePaginator,
    *,
    params: dict[str, Any] | None = None,
    headers: dict[str, str] | None = None,
    data_selector: DataSelector = None,
    concurrency: int = 8,
) -> AsyncIterator[Any]

Walk a paginated resource, yielding one page of selected data at a time.

For a RangePaginator whose total is known after the first response, pages 2..N are fetched concurrently (bounded by concurrency). Otherwise — cursor/link paginators, or an unknown total — it walks sequentially like the sync client.

Parameters:

Name	Type	Description	Default
`path`	`str`	The resource path.	required
`paginator`	`BasePaginator`	The pagination strategy.	required
`params`	`dict[str, Any] \| None`	Static query params applied to every page request.	`None`
`headers`	`dict[str, str] \| None`	Extra headers applied to every page request.	`None`
`data_selector`	`DataSelector`	How to pull each page's records from the response.	`None`
`concurrency`	`int`	Max concurrent page requests for range paginators.	`8`

Yields:

Type	Description
`AsyncIterator[Any]`	Each page's selected data (first page first; the rest in page order).

HTTPBearerAuth ¶

HTTPBearerAuth(token: str)

Bases: httpx2.Auth

HTTP Bearer authentication.

Initialize the HTTP Bearer authentication.

Parameters:

Name	Type	Description	Default
`token`	`str`	The bearer token.	required

auth_flow ¶

auth_flow(
    request: Request,
) -> Generator[Request, Response, None]

Authenticate the request with a Bearer token.

Defining auth_flow (rather than sync_auth_flow) lets httpx2 drive the same generator for RESTClient and AsyncRESTClient; a sync-only flow leaves the async client's requests unauthenticated.

Parameters:

Name	Type	Description	Default
`request`	`Request`	The request to authenticate.	required

Yields:

Type	Description
`Request`	The authenticated request.

OAuth2Auth ¶

OAuth2Auth(
    base_url: str,
    client_id: str,
    client_secret: str,
    refresh_token: str | None = None,
    scope: str | None = None,
    token_endpoint: str = "/oauth2/token",
    access_token: str | None = None,
)

Bases: httpx2.Auth

OAuth2 authentication with automatic token acquisition and 401 refresh.

Async-native: the token request is yielded into the active client's flow rather than performed inline, so a single auth_flow drives both RESTClient and AsyncRESTClient — the token exchange runs sync on one and async on the other, with no blocking I/O on the event loop.

Initialize the OAuth2 authentication.

Parameters:

Name	Type	Description	Default
`base_url`	`str`	The base URL of the API (token endpoint is resolved against it).	required
`client_id`	`str`	The client ID.	required
`client_secret`	`str`	The client secret.	required
`refresh_token`	`str \| None`	The refresh token (optional).	`None`
`scope`	`str \| None`	The scope (optional).	`None`
`token_endpoint`	`str`	The token endpoint path.	`'/oauth2/token'`
`access_token`	`str \| None`	Pre-existing access token (optional).	`None`

access_token ¶

access_token

The access token.

Raises:

Type	Description
`AuthenticationError`	If no access token is available.

auth_data ¶

auth_data

The authentication data.

auth_headers ¶

auth_headers

The authentication headers for token requests.

grant_type ¶

grant_type

The grant type.

refresh_token ¶

refresh_token

The refresh token.

requires_response_body ¶

requires_response_body = True

bool(x) -> bool

Returns True when the argument x is true, False otherwise. The builtins True and False are the only two instances of the class bool. The class bool is a subclass of the class int, and cannot be subclassed.

auth_flow ¶

auth_flow(
    request: Request,
) -> Generator[Request, Response, None]

Attach a Bearer token, acquiring/refreshing it via yielded requests.

Defining auth_flow (rather than sync_auth_flow) lets httpx2 drive the same generator for sync and async clients: the yielded token request is executed by whichever client is active.

Parameters:

Name	Type	Description	Default
`request`	`Request`	The request to authenticate; its `Authorization` header is set in place, and reset after a 401-triggered refresh.	required

Yields:

Type	Description
`Request`	The token request (when needed) and the authenticated request.

clear_token ¶

clear_token() -> None

Clear the cached access token (forces re-acquisition on the next request).

OAuth2ClientCredentialsAuth ¶

OAuth2ClientCredentialsAuth(
    base_url: str,
    client_id: str,
    client_secret: str,
    refresh_token: str | None = None,
    scope: str | None = None,
    token_endpoint: str = "/oauth2/token",
    access_token: str | None = None,
)

Bases: interloper.rest.auth.OAuth2Auth

OAuth2 client credentials authentication.

grant_type ¶

grant_type

The grant type.

OAuth2RefreshTokenAuth ¶

OAuth2RefreshTokenAuth(
    base_url: str,
    client_id: str,
    client_secret: str,
    refresh_token: str,
    scope: str | None = None,
    token_endpoint: str = "/oauth2/token",
)

Bases: interloper.rest.auth.OAuth2Auth

OAuth2 refresh token authentication.

Initialize the OAuth2 refresh token authentication.

Parameters:

Name	Type	Description	Default
`base_url`	`str`	The base URL of the API.	required
`client_id`	`str`	The client ID.	required
`client_secret`	`str`	The client secret.	required
`refresh_token`	`str`	The refresh token (required).	required
`scope`	`str \| None`	The scope (optional).	`None`
`token_endpoint`	`str`	The token endpoint path.	`'/oauth2/token'`

auth_data ¶

auth_data

The authentication data.

Raises:

Type	Description
`AuthenticationError`	If no refresh token is available.

grant_type ¶

grant_type

The grant type.

BasePaginator ¶

Bases: abc.ABC

Sequential paginator: advance one request per page off each response.

The client calls init_request once, then loops: send → update_state → (has_next?) → update_request.

has_next ¶

has_next

Whether another page should be fetched after the last response.

init_request ¶

init_request(request: Request) -> None

Set first-page params on the initial request (default: no-op).

Parameters:

Name	Type	Description	Default
`request`	`Request`	The initial request, mutated in place.	required

update_request ¶

update_request(request: Request) -> None

Mutate request in place to fetch the next page.

Parameters:

Name	Type	Description	Default
`request`	`Request`	The request to advance, mutated in place.	required

update_state ¶

update_state(response: Response) -> None

Read next-page info (cursor / link / total) from response and set state.

Parameters:

Name	Type	Description	Default
`response`	`Response`	The response just received for the current page.	required

SinglePagePaginator ¶

Bases: interloper.rest.paginator.BasePaginator

No pagination — a single request is the whole result.

has_next ¶

has_next

Always False.

update_request ¶

update_request(request: Request) -> None

No-op; there is never a next page.

Parameters:

Name	Type	Description	Default
`request`	`Request`	The single page's request, left unchanged.	required

update_state ¶

update_state(response: Response) -> None

No-op; there is never a next page.

Parameters:

Name	Type	Description	Default
`response`	`Response`	The single page's response, ignored.	required

OffsetPaginator ¶

OffsetPaginator(
    *,
    limit: int,
    offset_param: str = "offset",
    limit_param: str = "limit",
    base_offset: int = 0,
    total_path: str | None = None,
    maximum_offset: int | None = None,
    stop_on_empty: bool = True,
    data_selector: DataSelector = None,
)

Bases: interloper.rest.paginator.RangePaginator

Offset/limit pagination (?offset=0&limit=100, ?offset=100&limit=100, …).

When the response carries a total record count (total_path) or a maximum_offset is set, the async client fans out the remaining offsets concurrently; otherwise it walks sequentially until an empty page.

Configure offset/limit pagination.

Parameters:

Name	Type	Description	Default
`limit`	`int`	Number of records requested per page; also the offset step.	required
`offset_param`	`str`	Query parameter carrying the record offset.	`'offset'`
`limit_param`	`str`	Query parameter carrying the page size.	`'limit'`
`base_offset`	`int`	Offset of the first page, normally `0`.	`0`
`total_path`	`str \| None`	Dotted JSON path to the total record count in each response, or `None` when the API reports no total.	`None`
`maximum_offset`	`int \| None`	Hard cap on the offset, used when `total_path` yields nothing.	`None`
`stop_on_empty`	`bool`	Whether an empty page ends the walk when no total is known.	`True`
`data_selector`	`DataSelector`	How to pull each page's records from the response, used by the `stop_on_empty` check.	`None`

has_next ¶

has_next

Whether another page remains.

init_request ¶

init_request(request: Request) -> None

Reset state and set the first offset + limit params.

Parameters:

Name	Type	Description	Default
`request`	`Request`	The initial request, mutated in place.	required

remaining_requests ¶

remaining_requests(
    base_request: Request, first_response: Response
) -> list[Request] | None

Build requests for offsets limit, 2*limit, … < total when the total is known.

Parameters:

Name	Type	Description	Default
`base_request`	`Request`	The first page's request, used as the template to clone from.	required
`first_response`	`Response`	The first page's response, read for the total.	required

Returns:

Type	Description
`list[Request] \| None`	One request per remaining offset, or `None` if the total is unknown.

update_request ¶

update_request(request: Request) -> None

Advance to the next offset.

Parameters:

Name	Type	Description	Default
`request`	`Request`	The request to advance, mutated in place.	required

update_state ¶

update_state(response: Response) -> None

Stop once the next offset would reach the total, or on an empty page.

Parameters:

Name	Type	Description	Default
`response`	`Response`	The response just received for the current page.	required

PageNumberPaginator ¶

PageNumberPaginator(
    *,
    page_param: str = "page",
    base_page: int = 1,
    total_path: str | None = None,
    maximum_page: int | None = None,
    stop_on_empty: bool = True,
    data_selector: DataSelector = None,
)

Bases: interloper.rest.paginator.RangePaginator

Page-number pagination (?page=1, ?page=2, …).

When the response carries a total (total_path) or a maximum_page is set, the page set is bounded and the async client fans out pages 2..N concurrently; otherwise it walks sequentially until an empty page.

Configure page-number pagination.

Parameters:

Name	Type	Description	Default
`page_param`	`str`	Query parameter carrying the page number.	`'page'`
`base_page`	`int`	Page number of the first page (`1` for 1-indexed APIs, `0` for 0-indexed).	`1`
`total_path`	`str \| None`	Dotted JSON path to the last page number in each response, or `None` when the API reports no total.	`None`
`maximum_page`	`int \| None`	Hard cap on the last page, used when `total_path` yields nothing.	`None`
`stop_on_empty`	`bool`	Whether an empty page ends the walk when no total is known.	`True`
`data_selector`	`DataSelector`	How to pull each page's records from the response, used by the `stop_on_empty` check.	`None`

has_next ¶

has_next

Whether another page remains.

init_request ¶

init_request(request: Request) -> None

Reset state and set the first page parameter.

Parameters:

Name	Type	Description	Default
`request`	`Request`	The initial request, mutated in place.	required

remaining_requests ¶

remaining_requests(
    base_request: Request, first_response: Response
) -> list[Request] | None

Build requests for pages base_page+1..last when the total is known.

Parameters:

Name	Type	Description	Default
`base_request`	`Request`	The first page's request, used as the template to clone from.	required
`first_response`	`Response`	The first page's response, read for the total.	required

Returns:

Type	Description
`list[Request] \| None`	One request per remaining page, or `None` if the total is unknown.

update_request ¶

update_request(request: Request) -> None

Advance to the next page number.

Parameters:

Name	Type	Description	Default
`request`	`Request`	The request to advance, mutated in place.	required

update_state ¶

update_state(response: Response) -> None

Stop at the known last page, or on an empty page when no total is known.

Parameters:

Name	Type	Description	Default
`response`	`Response`	The response just received for the current page.	required

RangePaginator ¶

Bases: interloper.rest.paginator.BasePaginator

A paginator whose complete page set is known after the first response.

Adds the seam the async client uses to fetch pages 2..N concurrently. When the total can't be determined up front, remaining_requests returns None and the client falls back to the sequential walk.

remaining_requests ¶

remaining_requests(
    base_request: Request, first_response: Response
) -> list[Request] | None

Requests for every page after the first, or None if the total is unknown.

Parameters:

Name	Type	Description	Default
`base_request`	`Request`	The first page's request, used as the template to clone from.	required
`first_response`	`Response`	The first page's response, carrying the total when the API reports one.	required

HeaderLinkPaginator ¶

HeaderLinkPaginator(*, rel: str = 'next')

Bases: interloper.rest.paginator.BasePaginator

Follow the Link header's rel=next URL (RFC 5988). Sequential.

Follow the Link header rel.

Parameters:

Name	Type	Description	Default
`rel`	`str`	The `Link` header relation naming the next page.	`'next'`

has_next ¶

has_next

Whether the response carried a next link.

update_request ¶

update_request(request: Request) -> None

Point the request at the next link URL.

Parameters:

Name	Type	Description	Default
`request`	`Request`	The request to advance, mutated in place.	required

update_state ¶

update_state(response: Response) -> None

Read the rel=next link from the response Link header.

Parameters:

Name	Type	Description	Default
`response`	`Response`	The response just received for the current page.	required

JSONLinkPaginator ¶

JSONLinkPaginator(*, next_url_path: str)

Bases: interloper.rest.paginator.BasePaginator

Follow a next-page URL found at a JSON path in the response. Sequential.

Follow a next-page URL at a JSON path.

Parameters:

Name	Type	Description	Default
`next_url_path`	`str`	Dotted JSON path to the next-page URL in each response.	required

has_next ¶

has_next

Whether the response carried a next URL.

update_request ¶

update_request(request: Request) -> None

Point the request at the next URL (absolute or base-relative).

Parameters:

Name	Type	Description	Default
`request`	`Request`	The request to advance, mutated in place.	required

update_state ¶

update_state(response: Response) -> None

Read the next-page URL from the response JSON.

Parameters:

Name	Type	Description	Default
`response`	`Response`	The response just received for the current page.	required

JSONCursorPaginator ¶

JSONCursorPaginator(*, cursor_path: str, cursor_param: str)

Bases: interloper.rest.paginator.BasePaginator

Carry a cursor value from the response JSON into the next request parameter. Sequential.

Carry a JSON cursor into the next request.

Parameters:

Name	Type	Description	Default
`cursor_path`	`str`	Dotted JSON path to the next cursor in each response.	required
`cursor_param`	`str`	Query parameter the cursor is sent back in.	required

has_next ¶

has_next

Whether the response carried a next cursor.

update_request ¶

update_request(request: Request) -> None

Set the cursor query parameter for the next page.

Parameters:

Name	Type	Description	Default
`request`	`Request`	The request to advance, mutated in place.	required

update_state ¶

update_state(response: Response) -> None

Read the next cursor from the response JSON.

Parameters:

Name	Type	Description	Default
`response`	`Response`	The response just received for the current page.	required