biopb

biopb

Top-level BioPB Python package metadata.

Also the client for a biopb control plane (:mod:biopb._control, private): every name below except __version__ is that package's content, re-exported here rather than at biopb.control so that dotted path is never mistaken for biopb-control, the separate control-plane server distribution this is a client of. Stdlib only, so importing bare biopb stays cheap even though biopb.tensor/biopb.image (pyarrow, dask) are not imported here.

DataPlaneEndpoint dataclass

DataPlaneEndpoint(
    url: str,
    token: Optional[str] = None,
    tls_fingerprint: Optional[str] = None,
    origin: str = "default",
    tls_ca_pem: Optional[bytes] = None,
)

A resolved data-plane dial: where, with what credential, on what anchor.

origin_note property

origin_note: str

Human phrase for :attr:origin, for error messages.

LocalTrustError

Bases: RuntimeError

The local data plane's TLS certificate could not be used as a trust anchor.

A distinct type because the layers above classify connect failures by substring, and the most likely cause here — an unreadable cert file — stringifies as [Errno 13] Permission denied. That matches an authentication marker, so a file-permission problem would be reported as "the server needs a token" and send the reader after a credential that has nothing to do with it. Matching on the type is exact, and no errno wording can break it.

algorithm_logs

algorithm_logs(
    name: str, lines: int = 200, timeout: float = 10.0
) -> list[str]

The tail of a script entry's log: its installs and its server's output. ValueError for a url entry.

Source code in src/main/python/biopb/_control/_algorithms.py

def algorithm_logs(name: str, lines: int = 200, timeout: float = 10.0) -> list[str]:
    """The tail of a script entry's log: its installs and its server's output.
    ``ValueError`` for a url entry."""
    return _verb("GET", "logs", name, timeout, lines=lines)["lines"]

algorithms

algorithms(timeout: float = 10.0) -> Optional[list[dict]]

Every registry entry with its state and cached ops, or None when no control answers. A url entry is probed, so this can take a probe's time.

Source code in src/main/python/biopb/_control/_algorithms.py

def algorithms(timeout: float = 10.0) -> Optional[list[dict]]:
    """Every registry entry with its state and cached ops, or ``None`` when no
    control answers. A url entry is probed, so this can take a probe's time."""
    return _listing("GET", "/api/algorithms", timeout)

base_url

base_url() -> str

Where the control listens, e.g. http://127.0.0.1:8813.

Source code in src/main/python/biopb/_control/_client.py

def base_url() -> str:
    """Where the control listens, e.g. ``http://127.0.0.1:8813``."""
    return control_base_url()

control_grpc_url

control_grpc_url(timeout: float = 1.0) -> Optional[str]

The data-plane URL the control publishes, or None if no control answers.

GETs the control's bare, unauthenticated /health and reads data_plane.grpc_url from the supervisor snapshot — the single source of truth for where the plane lives (biopb/biopb#413), because the control is what chose the bind, the port, and the scheme. Best-effort: an absent, slow, or malformed control is "no answer", never an exception, so the caller falls through to the default rather than failing to resolve anything at all.

Source code in src/main/python/biopb/_control/_data_plane.py

def control_grpc_url(timeout: float = 1.0) -> Optional[str]:
    """The data-plane URL the control publishes, or ``None`` if no control answers.

    GETs the control's bare, unauthenticated ``/health`` and reads
    ``data_plane.grpc_url`` from the supervisor snapshot — the single source of
    truth for where the plane lives (biopb/biopb#413), because the control is what
    chose the bind, the port, and the scheme. Best-effort: an absent, slow, or
    malformed control is "no answer", never an exception, so the caller falls
    through to the default rather than failing to resolve anything at all.
    """
    try:
        with urllib.request.urlopen(
            f"{control_base_url()}/health", timeout=timeout
        ) as resp:
            if resp.status != 200:
                return None
            payload = json.loads(resp.read().decode())
    except Exception:  # noqa: BLE001 - best-effort discovery; the caller falls back
        return None
    data_plane = payload.get("data_plane") if isinstance(payload, dict) else None
    if isinstance(data_plane, dict):
        url = data_plane.get("grpc_url")
        if isinstance(url, str) and url:
            return url
    return None

default_data_plane_url

default_data_plane_url() -> str

The endpoint a default deployment puts the data plane on (grpc://…:8815).

Source code in src/main/python/biopb/_control/_data_plane.py

def default_data_plane_url() -> str:
    """The endpoint a default deployment puts the data plane on (``grpc://…:8815``)."""
    return f"grpc://127.0.0.1:{flight_port_for(BASE_DEFAULT_PORT)}"

ensure_algorithm

ensure_algorithm(name: str, timeout: float = 600.0) -> dict

Bring a script entry up, installing it first if its file changed, and answer its row; a url entry is probed. Waits under timeout: a row still installing or starting means ask again.

Raises LookupError for an unknown name.

Source code in src/main/python/biopb/_control/_algorithms.py

def ensure_algorithm(name: str, timeout: float = 600.0) -> dict:
    """Bring a script entry up, installing it first if its file changed, and
    answer its row; a url entry is probed. Waits under *timeout*: a row still
    ``installing`` or ``starting`` means ask again.

    Raises ``LookupError`` for an unknown name.
    """
    return _verb("POST", "ensure", name, timeout, client_timeout=timeout)["server"]

ensure_data_plane

ensure_data_plane(timeout: float = 60.0) -> Optional[dict]

Have the control bring its plane up; {"url", "token"} or None.

POST /api/data_plane/ensure, idempotent on the control's side. timeout is both this call's HTTP timeout and the client_timeout the control keeps its own wait under, so a slow start comes back as a verdict rather than as a timeout that looks like no control at all. None when no control answers or it could not bring the plane up.

Source code in src/main/python/biopb/_control/_client.py

def ensure_data_plane(timeout: float = 60.0) -> Optional[dict]:
    """Have the control bring its plane up; ``{"url", "token"}`` or ``None``.

    ``POST /api/data_plane/ensure``, idempotent on the control's side.
    *timeout* is both this call's HTTP timeout and the ``client_timeout`` the
    control keeps its own wait under, so a slow start comes back as a verdict
    rather than as a timeout that looks like no control at all. ``None`` when
    no control answers or it could not bring the plane up.
    """
    try:
        payload = _request(
            "POST", "/api/data_plane/ensure", {"client_timeout": timeout}, timeout
        )
    except Exception as exc:  # noqa: BLE001 - no answer is None, not an error
        logger.info("control ensure_data_plane failed: %s", exc)
        return None
    snapshot = payload.get("data_plane") if isinstance(payload, dict) else None
    url = snapshot.get("grpc_url") if isinstance(snapshot, dict) else None
    if not url:
        logger.warning("control answered ensure without a data-plane url")
    return _answer(url)

find_data_plane

find_data_plane(timeout: float = 1.0) -> Optional[dict]

The plane the control names, {"url", "token"}, or None.

A plain read of GET /health: None when no control answers or it names no plane. It does not start the plane; :func:ensure_data_plane does.

Source code in src/main/python/biopb/_control/_client.py

def find_data_plane(timeout: float = 1.0) -> Optional[dict]:
    """The plane the control names, ``{"url", "token"}``, or ``None``.

    A plain read of ``GET /health``: ``None`` when no control answers or it
    names no plane. It does not start the plane; :func:`ensure_data_plane`
    does.
    """
    return _answer(_data_plane.control_grpc_url(timeout=timeout))

is_local_url

is_local_url(url: str) -> bool

Whether url points at this machine.

Source code in src/main/python/biopb/_control/_data_plane.py

def is_local_url(url: str) -> bool:
    """Whether *url* points at this machine."""
    try:
        host = urlparse(url).hostname
    except ValueError:
        return False
    return host is None or host in _LOCAL_HOSTS

local_data_plane_fingerprint

local_data_plane_fingerprint(url: str) -> Optional[str]

Identity of the certificate a local plane serves, as a SHA-256 digest.

A loopback grpcs:// plane is this machine's own, so what it serves is knowable here rather than something to accept on first sight (TOFU). The digest is checked against the certificate the server actually presents on every connect, which is strictly stronger than a pin learned from the wire and keeps the client out of the shared pin store — where an operator's cert init --force would otherwise strand it.

A fingerprint rather than the PEM itself, deliberately: handing the client a PEM resolves trust entirely offline, which also skips the hostname-override probe, and a local client dials loopback. A certificate carrying only the host's public name — the ordinary shape of an operator's own cert — then fails hostname verification on every connect (biopb/biopb#916).

Two sources, in order:

• what the plane published for this port (:mod:biopb._tls_record), which is the only thing that knows about a --tls-cert the plane was handed;
• failing that, the certificate the plane would have minted (state/biopb/tls/server-cert.pem), which is what a plane too old to publish anything serves.

None — leaving TOFU in charge — for a plaintext endpoint or a remote one, whose certificate is not on this disk and cannot be. Raises :class:LocalTrustError when a local plane is TLS and neither source answers: silently falling back to TOFU there would trade a verified identity for an unverified one exactly where the strong option was meant to apply.

Known edge: a loopback grpcs:// URL that is really an ssh -L tunnel to a remote plane is indistinguishable from a local one by host alone, so it is checked against the local plane's identity and fails. Loud and fixable (dial the plane directly, or tunnel to a non-loopback alias), not silent.

Source code in src/main/python/biopb/_control/_data_plane.py

def local_data_plane_fingerprint(url: str) -> Optional[str]:
    """Identity of the certificate a *local* plane serves, as a SHA-256 digest.

    A loopback ``grpcs://`` plane is this machine's own, so what it serves is
    knowable here rather than something to accept on first sight (TOFU). The
    digest is checked against the certificate the server actually presents on
    every connect, which is strictly stronger than a pin learned from the wire
    and keeps the client out of the shared pin store — where an operator's
    ``cert init --force`` would otherwise strand it.

    A fingerprint rather than the PEM itself, deliberately: handing the client a
    PEM resolves trust entirely offline, which also skips the hostname-override
    probe, and a local client dials loopback. A certificate carrying only the
    host's public name — the ordinary shape of an operator's own cert — then
    fails hostname verification on every connect (biopb/biopb#916).

    Two sources, in order:

    - what the plane **published** for this port (:mod:`biopb._tls_record`),
      which is the only thing that knows about a ``--tls-cert`` the plane was
      handed;
    - failing that, the certificate the plane would have **minted**
      (``state/biopb/tls/server-cert.pem``), which is what a plane too old to
      publish anything serves.

    ``None`` — leaving TOFU in charge — for a plaintext endpoint or a remote one,
    whose certificate is not on this disk and cannot be. Raises
    :class:`LocalTrustError` when a local plane is TLS and neither source
    answers: silently falling back to TOFU there would trade a verified identity
    for an unverified one exactly where the strong option was meant to apply.

    Known edge: a loopback ``grpcs://`` URL that is really an ``ssh -L`` tunnel to
    a *remote* plane is indistinguishable from a local one by host alone, so it is
    checked against the local plane's identity and fails. Loud and fixable (dial
    the plane directly, or tunnel to a non-loopback alias), not silent.
    """
    if not url.lower().startswith("grpcs://") or not is_local_url(url):
        return None

    from .. import _tls_material, _tls_record
    from .._locations import tls_served_certs, tls_server_cert

    port = urlparse(url).port
    if port is not None:
        published = _tls_record.lookup(port)
        if published:
            return published

    cert_path = tls_server_cert()
    try:
        pem = cert_path.read_bytes()
    except OSError as exc:
        raise LocalTrustError(
            f"The local data plane at {url} serves TLS, but nothing on this "
            f"machine says which certificate: no record for port {port} in "
            f"{tls_served_certs()}, and no certificate at {cert_path} ({exc}). A "
            "local plane is verified against what it serves, not pinned from the "
            "wire, so this is not retried as trust-on-first-use. Check the state "
            "dir is the one the server writes to (BIOPB_STATE_HOME); a plane "
            "publishes its record at startup, so restart one that predates this "
            "build, or mint the certificate with `biopb-tensor-server cert init`."
        ) from exc
    if not pem.strip():
        raise LocalTrustError(
            f"The local data plane's TLS certificate at {cert_path} is empty."
        )
    try:
        return _tls_material.fingerprint(_tls_material.leaf_pem(pem))
    except ValueError as exc:
        raise LocalTrustError(
            f"The local data plane's TLS certificate at {cert_path} is not "
            f"readable as PEM ({exc})."
        ) from exc

probe_data_plane_scheme

probe_data_plane_scheme(
    host: str, port: int, timeout: float = 0.5
) -> Optional[str]

"grpcs" / "grpc" by asking the listener, or None if nothing is there.

The scheme is the one thing a directly-launched plane still tells you for free: a TLS listener completes a handshake and a plaintext one does not. So ask it, rather than inferring from the presence of a cert on disk — a cert minted once by cert init says nothing about whether the running plane was started with --tls, and guessing wrong in either direction produces the same "server unreachable" that #615 was filed for.

Certificate validation is deliberately off: this asks a yes/no question about the wire protocol, and the answer decides which scheme to dial. Trust is established afterwards, on the real connection, by :func:local_data_plane_fingerprint or TOFU.

Source code in src/main/python/biopb/_control/_data_plane.py

def probe_data_plane_scheme(
    host: str, port: int, timeout: float = 0.5
) -> Optional[str]:
    """``"grpcs"`` / ``"grpc"`` by asking the listener, or ``None`` if nothing is there.

    The scheme is the one thing a directly-launched plane still tells you for
    free: a TLS listener completes a handshake and a plaintext one does not. So
    ask it, rather than inferring from the presence of a cert on disk — a cert
    minted once by ``cert init`` says nothing about whether the running plane was
    started with ``--tls``, and guessing wrong in *either* direction produces the
    same "server unreachable" that #615 was filed for.

    Certificate validation is deliberately off: this asks a yes/no question about
    the wire protocol, and the answer decides which scheme to dial. Trust is
    established afterwards, on the real connection, by :func:`local_data_plane_fingerprint`
    or TOFU.
    """
    import socket
    import ssl

    ctx = ssl.SSLContext(ssl.PROTOCOL_TLS_CLIENT)
    ctx.check_hostname = False
    ctx.verify_mode = ssl.CERT_NONE
    try:
        with socket.create_connection((host, port), timeout=timeout) as sock:
            try:
                sock.settimeout(timeout)
                with ctx.wrap_socket(sock, server_hostname=host):
                    return "grpcs"
            except (OSError, ValueError):
                # A plaintext HTTP/2 listener answers a ClientHello with garbage
                # (or a reset). It is listening; it just isn't TLS.
                return "grpc"
    except OSError:
        return None

refresh_algorithms

refresh_algorithms(
    timeout: float = 10.0,
) -> Optional[list[dict]]

Have the control install and describe new or edited script entries; answers the rows at once, with those entries installing.

Source code in src/main/python/biopb/_control/_algorithms.py

def refresh_algorithms(timeout: float = 10.0) -> Optional[list[dict]]:
    """Have the control install and describe new or edited script entries;
    answers the rows at once, with those entries ``installing``."""
    return _listing("POST", "/api/algorithms/refresh", timeout)

resolve_data_plane

resolve_data_plane(
    override: Optional[str] = None,
    token: Optional[str] = None,
    *,
    timeout: float = 1.0,
    probe: bool = True
) -> DataPlaneEndpoint

Resolve the data-plane endpoint: override -> env -> control -> default.

override is an explicit --server-style address and wins over everything; it is the escape hatch for a plane nothing records — one launched directly on a custom port. token is an explicit --token and likewise wins over the environment and over the control's credential file.

The credential file is read only for an endpoint the control named (see the module docstring): an address that bypassed the control is dialed with an explicit token or with none.

Set probe=False to skip the socket scheme probe on the default fallback (a caller that only wants to name the endpoint, not dial it).

Raises :class:LocalTrustError when the resolved plane is local TLS but nothing on this machine says what it serves — see :func:local_data_plane_fingerprint. The TLS anchor is :func:data_plane_trust's.

Source code in src/main/python/biopb/_control/_data_plane.py

def resolve_data_plane(
    override: Optional[str] = None,
    token: Optional[str] = None,
    *,
    timeout: float = 1.0,
    probe: bool = True,
) -> DataPlaneEndpoint:
    """Resolve the data-plane endpoint: override -> env -> control -> default.

    *override* is an explicit ``--server``-style address and wins over everything;
    it is the escape hatch for a plane nothing records — one launched directly on
    a custom port. *token* is an explicit ``--token`` and likewise wins over the
    environment and over the control's credential file.

    The credential file is read only for an endpoint the control named (see the
    module docstring): an address that bypassed the control is dialed with an
    explicit token or with none.

    Set ``probe=False`` to skip the socket scheme probe on the default fallback
    (a caller that only wants to *name* the endpoint, not dial it).

    Raises :class:`LocalTrustError` when the resolved plane is local TLS but
    nothing on this machine says what it serves — see :func:`local_data_plane_fingerprint`.
    The TLS anchor is :func:`data_plane_trust`'s.
    """
    url, origin = _resolve_url(override, timeout=timeout, probe=probe)
    trust = data_plane_trust(url, origin)
    return DataPlaneEndpoint(
        url=url,
        # The credential file is the control's handoff for the plane IT owns, so
        # it travels only with an endpoint the control named. An address given on
        # the command line or in the environment routed around the control on
        # purpose -- possibly to somebody else's server -- and attaching this
        # machine's token to that dial would hand a local credential to a host the
        # user never authorized it for. Those endpoints authenticate explicitly or
        # not at all.
        token=resolve_data_plane_token(
            token, allow_credential_file=origin == "control"
        ),
        tls_fingerprint=trust.fingerprint,
        tls_ca_pem=trust.ca_pem,
        origin=origin,
    )

resolve_data_plane_token

resolve_data_plane_token(
    explicit: Optional[str] = None,
    *,
    allow_credential_file: bool = True
) -> Optional[str]

The data-plane token: explicit -> BIOPB_TENSOR_TOKEN -> credential file.

The credential file is what closes the gap for a local plane behind a token (biopb/biopb#470): the control writes the resolved token to an owner-only file in the user's state dir, so a client that never inherited the control's environment can still authenticate. The core CLI read only the env var until #615, which is why a token-gated local plane reported itself as unreachable.

allow_credential_file=False drops that last step, for a caller dialing an endpoint the control did not name: the file holds this machine's credential for the control's own plane, and it has no business being sent to an address the user pointed elsewhere. The two explicit sources still apply — someone naming a server can also name its token.

None — unauthenticated, correct for a tokenless local plane — when nothing yields one. A blank value is None, never "": an empty string would be sent as an empty Bearer header rather than omitted.

Source code in src/main/python/biopb/_control/_data_plane.py

def resolve_data_plane_token(
    explicit: Optional[str] = None, *, allow_credential_file: bool = True
) -> Optional[str]:
    """The data-plane token: explicit -> ``BIOPB_TENSOR_TOKEN`` -> credential file.

    The credential file is what closes the gap for a *local plane behind a token*
    (biopb/biopb#470): the control writes the resolved token to an owner-only file
    in the user's state dir, so a client that never inherited the control's
    environment can still authenticate. The core CLI read only the env var
    until #615, which is why a token-gated local plane reported itself as
    unreachable.

    ``allow_credential_file=False`` drops that last step, for a caller dialing an
    endpoint the control did not name: the file holds *this machine's* credential
    for the control's own plane, and it has no business being sent to an address
    the user pointed elsewhere. The two explicit sources still apply — someone
    naming a server can also name its token.

    ``None`` — unauthenticated, correct for a tokenless local plane — when nothing
    yields one. A blank value is ``None``, never ``""``: an empty string would be
    sent as an empty ``Bearer`` header rather than omitted.
    """
    given = (explicit or "").strip() or os.environ.get(ENV_TENSOR_TOKEN, "").strip()
    if given:
        return given
    if not allow_credential_file:
        return None

    from .._credentials import read_credential

    return read_credential()

restart_algorithm

restart_algorithm(
    name: str, timeout: float = 600.0
) -> dict

Stop a script entry and ensure it again. ValueError for a url entry.

Source code in src/main/python/biopb/_control/_algorithms.py

def restart_algorithm(name: str, timeout: float = 600.0) -> dict:
    """Stop a script entry and ensure it again. ``ValueError`` for a url entry."""
    return _verb("POST", "restart", name, timeout, client_timeout=timeout)["server"]

stop_algorithm

stop_algorithm(name: str, timeout: float = 30.0) -> dict

Stop a script entry's server. ValueError for a url entry.

Source code in src/main/python/biopb/_control/_algorithms.py

def stop_algorithm(name: str, timeout: float = 30.0) -> dict:
    """Stop a script entry's server. ``ValueError`` for a url entry."""
    return _verb("POST", "stop", name, timeout)["server"]

user_base_url

user_base_url() -> str

Where the user's browser reaches the control, for a link handed to them.

:func:control_base_url is where this machine connects, and behind a reverse proxy (an Open OnDemand /node/<host>/<port> route) that is a loopback address the user's browser cannot reach. A control told its public form (--url-prefix, --public-origin) publishes it in its record: origin plus prefix, or the bare path when only the prefix is set (a path on whatever site the user opened the session from). Otherwise the two are the same.

Source code in src/main/python/biopb/_control/_endpoints.py

def user_base_url() -> str:
    """Where the *user's browser* reaches the control, for a link handed to them.

    :func:`control_base_url` is where this machine connects, and behind a reverse
    proxy (an Open OnDemand ``/node/<host>/<port>`` route) that is a loopback
    address the user's browser cannot reach. A control told its public form
    (``--url-prefix``, ``--public-origin``) publishes it in its record: origin plus
    prefix, or the bare path when only the prefix is set (a path on whatever site
    the user opened the session from). Otherwise the two are the same.
    """
    published = _runtime_record().get("user_url")
    if isinstance(published, str) and published.strip():
        return published.strip().rstrip("/")
    return control_base_url()