Skip to content

biopb.tensor

biopb.tensor

Tensor storage framework on Arrow Flight.

This package provides TensorStore-like framework built on Apache Arrow Flight for efficient multi-dimensional array storage and retrieval.

Key components: - TensorFlightClient: Client for accessing tensors from a TensorFlightServer - Connection: the data plane this machine's control names, dialed and shared - Proto messages: TensorTicket, ChunkBounds, TensorDescriptor, SliceHint - query / resolve hand back sources catalog rows; what you decode them into is yours (descriptors_from_rows is the deprecated proto form) - CLI diagnostics: biopb tensor command for inspecting sources and tensors

The CLI module provides the biopb tensor command with four subcommands: - query: List sources and tensors from a running server - metadata: Inspect source metadata and tensor descriptors - get: Download tensor data to file or stdout - stats: Compute min/max/mean statistics for a tensor

Note: Server components have been moved to the biopb-tensor-server package.

LabelAddress

Bases: NamedTuple

A label set's array_id, taken apart.

descriptor_from_row 

descriptor_from_row(
    row: Mapping[str, Any],
) -> DataSourceDescriptor

One sources row -> DataSourceDescriptor.

.. deprecated:: Use the row. See the module docstring.

Source code in src/main/python/biopb/tensor/_catalog_rows.py 
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
def descriptor_from_row(row: Mapping[str, Any]) -> DataSourceDescriptor:
    """One ``sources`` row -> ``DataSourceDescriptor``.

    .. deprecated::
        Use the row. See the module docstring.
    """
    warnings.warn(
        _DEPRECATION.format(name="descriptor_from_row"),
        DeprecationWarning,
        stacklevel=2,
    )
    return _descriptor_from_row(row)

descriptors_from_rows 

descriptors_from_rows(
    rows: Iterable[Mapping[str, Any]],
) -> List[DataSourceDescriptor]

sources rows -> DataSourceDescriptors.

.. deprecated:: Use the rows. See the module docstring.

Source code in src/main/python/biopb/tensor/_catalog_rows.py 
106
107
108
109
110
111
112
113
114
115
116
117
118
119
def descriptors_from_rows(
    rows: Iterable[Mapping[str, Any]],
) -> List[DataSourceDescriptor]:
    """``sources`` rows -> ``DataSourceDescriptor``s.

    .. deprecated::
        Use the rows. See the module docstring.
    """
    warnings.warn(
        _DEPRECATION.format(name="descriptors_from_rows"),
        DeprecationWarning,
        stacklevel=2,
    )
    return [_descriptor_from_row(r) for r in rows]

is_reserved_label_name 

is_reserved_label_name(name: str) -> bool

Whether name is a set the server owns -- read-only to every client.

Source code in src/main/python/biopb/tensor/_labels.py 
78
79
80
def is_reserved_label_name(name: str) -> bool:
    """Whether *name* is a set the server owns -- read-only to every client."""
    return name.startswith(RESERVED_LABEL_PREFIX)

label_image_axes 

label_image_axes(
    label_desc: Any, image_desc: Any
) -> Optional[List[int]]

For each axis of a set, the index of the image axis it indexes.

[0, 2, 3, 4] for a T Z Y X set of a T C Z Y X image: a set spans the image's non-channel extent, so every axis after the image's c sits one place to the left in the set. A client that instead matched axes by position reads frame 0 of a timelapse where frame 40 was asked for, which is a picture rather than an error.

Read, not derived, when the server says. biopb.labels.image_axes in the set's metadata_json is the server's own statement of the mapping; a descriptor fetched without metadata, or from a server that predates the field, has none, and the extent rule is re-derived here instead -- the same answer, from the one place that still has to know the rule.

None when the set does not span the image at all, which leaves the caller nothing to align and is the server's own answer in that case too.

Source code in src/main/python/biopb/tensor/_labels.py 
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
def label_image_axes(label_desc: Any, image_desc: Any) -> Optional[List[int]]:
    """For each axis of a set, the index of the image axis it indexes.

    ``[0, 2, 3, 4]`` for a ``T Z Y X`` set of a ``T C Z Y X`` image: a set spans
    the image's **non-channel** extent, so every axis after the image's ``c``
    sits one place to the left in the set. A client that instead matched axes by
    position reads frame 0 of a timelapse where frame 40 was asked for, which is
    a picture rather than an error.

    **Read, not derived, when the server says.** ``biopb.labels.image_axes`` in
    the set's ``metadata_json`` is the server's own statement of the mapping;
    a descriptor fetched without metadata, or from a server that predates the
    field, has none, and the extent rule is re-derived here instead -- the same
    answer, from the one place that still has to know the rule.

    ``None`` when the set does not span the image at all, which leaves the
    caller nothing to align and is the server's own answer in that case too.
    """
    stated = _stated_image_axes(label_desc)
    if stated is not None and len(stated) == len(label_desc.shape):
        return stated
    non_channel = [
        i
        for i, label in enumerate(image_desc.dim_labels)
        if str(label).lower() not in _CHANNEL_LABELS
    ]
    # Ranks are the check. An image whose labels went missing derives an empty
    # or over-long list, which cannot match a real set and so answers None --
    # the same "nothing to align" the server gives.
    return non_channel if len(non_channel) == len(label_desc.shape) else None

split_label_array_id 

split_label_array_id(
    array_id: str,
) -> Optional[LabelAddress]

Take a label set's array_id apart, or None if it names no set.

"src0/@labels/nuclei" -> image "src0", name "nuclei". Pass a stable id: a content-pinned id@token is not one.

Source code in src/main/python/biopb/tensor/_labels.py 
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
def split_label_array_id(array_id: str) -> Optional[LabelAddress]:
    """Take a label set's ``array_id`` apart, or ``None`` if it names no set.

    ``"src0/@labels/nuclei"`` -> image ``"src0"``, name ``"nuclei"``. Pass a
    *stable* id: a content-pinned ``id@token`` is not one.
    """
    parts = array_id.split("/")
    # From 1: ``parts[0]`` is the source_id, which is slash-free by the identity
    # policy and so cannot be the ``labels`` segment of a field. A bare source
    # called "@labels" is a source.
    for i in range(len(parts) - 2, 0, -1):
        if parts[i] != LABELS_SEGMENT or not parts[i + 1]:
            continue
        return LabelAddress(
            image_array_id="/".join(parts[:i]),
            name=parts[i + 1],
            level="/".join(parts[i + 2 :]) or None,
        )
    return None