Package biopb.tensor

Class TensorFlightClient

java.lang.Object
biopb.tensor.TensorFlightClient
All Implemented Interfaces:
AutoCloseable
public class TensorFlightClient extends Object implements AutoCloseable
Client for accessing tensors from a TensorFlightServer. This client uses Apache Arrow Flight to discover data sources, request logical read plans, and fetch chunk payloads from a TensorFlightServer. It supports multifield acquisitions where tensors within a source have different shapes. The Java client returns lazy cell-backed images when the logical Flight endpoint layout matches the descriptor chunk grid. In that case, imglib2's internal cell cache is the primary cache for repeated reads.

A tensor is identified by its globally-unique array_id (the tensor identity policy; see the top of proto/biopb/tensor/descriptor.proto): either source_id for a single-tensor source or source_id/field for a multi-tensor one. The array_id-first methods (getTensor(String), getDescriptor(String), getPhysicalScale(String)) take that one identifier; there is no (sourceId, tensorId) form. Usage: 

 TensorFlightClient client = new TensorFlightClient("localhost:8815");

 // Browse the catalog (SQL over the server's DuckDB)
 VectorSchemaRoot rows = client.query("SELECT * FROM sources");

 // Access a tensor by its array_id ("source_id" or "source_id/field")
 RandomAccessibleInterval<UnsignedByteType> arr = client.getTensor("my-source/tensor-0");
 long[] pos = { 10, 20, 30 };
 UnsignedByteType pixel = arr.getAt(pos);
 client.close();

  • Constructor Details

    • TensorFlightClient

      public TensorFlightClient(String host, int port)
      Create a new TensorFlightClient.
      Parameters:
      host - Server host
      port - Server port

    • TensorFlightClient

      public TensorFlightClient(String host, int port, long cacheBytes)
      Create a new TensorFlightClient with custom cache size.
      Parameters:
      host - Server host
      port - Server port
      cacheBytes - Maximum cache size in bytes

    • TensorFlightClient

      public TensorFlightClient(String host, int port, long cacheBytes, String token)
      Create a new TensorFlightClient with authentication token.
      Parameters:
      host - Server host
      port - Server port
      cacheBytes - Maximum cache size in bytes
      token - Bearer token for authentication (null disables auth)

    • TensorFlightClient

      public TensorFlightClient(org.apache.arrow.flight.Location location)
      Create a new TensorFlightClient for an Arrow Flight location.
      Parameters:
      location - Flight server location

    • TensorFlightClient

      public TensorFlightClient(org.apache.arrow.flight.Location location, long cacheBytes)
      Create a new TensorFlightClient for an Arrow Flight location.
      Parameters:
      location - Flight server location
      cacheBytes - Maximum cache size in bytes

    • TensorFlightClient

      public TensorFlightClient(org.apache.arrow.flight.Location location, long cacheBytes, String token)
      Create a new TensorFlightClient for an Arrow Flight location with authentication.
      Parameters:
      location - Flight server location
      cacheBytes - Maximum cache size in bytes
      token - Bearer token for authentication (null disables auth)

  • Method Details

    • getLocation

      public org.apache.arrow.flight.Location getLocation()
      Get the Flight server location.
      Returns:
      Location used for this client

    • getToken

      public String getToken()
      Get the authentication token.
      Returns:
      Bearer token (null if no authentication)

    • getCacheBytes

      public long getCacheBytes()
      Get the cache size in bytes.
      Returns:
      Maximum cache size for cell images

    • query

      public org.apache.arrow.vector.VectorSchemaRoot query(String sql) throws IOException
      Execute SQL query against server's source metadata database. Returns Arrow VectorSchemaRoot with query results. Schema metadata may contain "total_sources" key if result was truncated. The server-side metadata database is mandatory (biopb/biopb#225), so any standard tensor-server supports this. Only an embedded server explicitly constructed without a metadata database rejects the query.
      Parameters:
      sql - SQL query (e.g., "SELECT source_id FROM sources WHERE source_url LIKE '%plate%'")
      Returns:
      VectorSchemaRoot with query results (caller must close)
      Throws:
      IOException - If query fails or the server has no metadata database attached Example: 
                           VectorSchemaRoot result = client.query("SELECT source_id, source_type FROM sources");
                     System.out.println("Found " + result.getRowCount() + " sources");
                     result.close();

    • getSourceMetadata

      public Map<String,Object> getSourceMetadata(String sourceId) throws IOException
      Get source-level OME/vendor metadata as a map.

      Source-scoped, and read from the source's own catalog row: this is the metadata the format carries for the whole container. A field's own extras (an OME-Zarr HCS field's OME block, an EMD signal's original_metadata) are per-tensor and are not merged in here.

      Parameters:
      sourceId - Source identifier
      Returns:
      The source's metadata map, or an empty map if it carries none
      Throws:
      IllegalArgumentException - if the source is unknown
      IllegalStateException - if the source is unresolved (cloud / synced-folder) -- call resolveSource(java.lang.String) first
      IOException

    • resolveSource

      public org.apache.arrow.vector.VectorSchemaRoot resolveSource(String sourceId) throws IOException
      Resolve an unresolved source and return its sources catalog row.

      An unresolved source is catalogued by URL only -- its shape/dtype/field list are unknown until first access (it lists with is_resolved false and an empty tensors). The canonical case is a cloud / synced-folder ("Files-On-Demand") source.

      Resolving asks the server to hydrate it. For a dehydrated placeholder this downloads the whole file -- a recall that can take minutes, consume local disk, and fail when offline -- then reads its real shape, dtype, and field list. This is the heavyweight, consenting operation that catalog browsing (query(java.lang.String)) deliberately avoids; call it only when you intend to read the data. Afterwards getTensor(java.lang.String) and friends work normally. Idempotent.

      Parameters:
      sourceId - The source to resolve (e.g. "onedrive_a3f2")
      Returns:
      The source's sources row, with every tensor enumerated -- one VectorSchemaRoot, the same type query(java.lang.String) returns, which the caller must close. A row, not a type this SDK picked: every client can already decode one, and what you decode it into stays yours (biopb/biopb#1032).

      Unlike warmSource(java.lang.String), which returns a status because residency is not a durable catalog fact and its file counts exist nowhere else, this returns the result: resolving is defined by what it writes to the row.

      Throws:
      IOException - If the action fails or the server returns no row

    • resolveSource

      public org.apache.arrow.vector.VectorSchemaRoot resolveSource(String sourceId, Consumer<ResolveProgress> onProgress, BooleanSupplier shouldCancel) throws IOException
      Resolve an unresolved source with optional progress and cancellation hooks.

      Both callbacks run on the calling thread after each streamed action message. Returning true from shouldCancel stops consuming the action stream; the server may finish its recall independently and cache the result for a later call.

      Parameters:
      sourceId - source to resolve
      onProgress - receives server progress heartbeats, or null
      shouldCancel - polled once per action message, or null
      Returns:
      the terminal catalog row; caller must close it
      Throws:
      IOException

    • warmSource

      public WarmProgress warmSource(String sourceId) throws IOException
      Hydrate-ahead: ask the server to recall all of a resolved multi-file source's member files, so later reads are warm and never stall.

      resolveSource(java.lang.String) populates a source's metadata but, for a multi-file cloud source (zarr / ome-zarr / ndtiff / tiff-sequence / micromanager), leaves the bulk pixel data dehydrated -- each member file then recalls one-at-a-time, slowly, the first time a read touches it. This walks the source directory server-side and reads every file to force the sync engine's recall; no pixels cross the wire, only progress. It is idempotent (already-resident files are cheap local reads) and a no-op for a single-file source (resolveSource already recalled it). A remote-url source -- an object store, or a grpc:// mirror -- fails instead: nothing on the serving machine can be made resident (biopb/biopb#1035).

      Parameters:
      sourceId - The (already-resolved) source to warm.
      Returns:
      The terminal WarmProgress snapshot (files/bytes made resident). filesTotal == 0 means the source was local and had nothing to warm, i.e. single-file; "not applicable" raises.
      Throws:
      IOException - If the action fails or it returns no terminal status
      UnsupportedOperationException - If the server predates the warm action

    • warmSource

      public WarmProgress warmSource(String sourceId, Consumer<WarmProgress> onProgress, BooleanSupplier shouldCancel) throws IOException
      Warm a source with optional progress and cancellation hooks.
      Parameters:
      sourceId - source to warm
      onProgress - receives non-terminal warm progress, or null
      shouldCancel - polled once per action message, or null
      Returns:
      the terminal progress snapshot
      Throws:
      IOException

    • registerLocalPath

      public AddSourceResult registerLocalPath(String url) throws IOException
      Register a local path on the SERVER as a served source at runtime.

      Hands the server a filesystem path (or directory) that it interprets on its own filesystem, and the server routes it through the same claim -> adapter -> catalog pipeline the directory watcher uses. A directory that is not itself a dataset is walked recursively and may register several sources, so this reports a tally rather than one source.

      Parameters:
      url - Absolute path (or directory) on the server's filesystem
      Returns:
      the terminal AddSourceResult
      Throws:
      IOException - If the action fails, the server is too old to support the add_source action, or it returns no terminal result

    • registerLocalPath

      public AddSourceResult registerLocalPath(String url, String sourceType, Consumer<AddSourceProgress> onProgress, BooleanSupplier shouldCancel) throws IOException
      Register a path on the server, with progress and cancellation hooks.

      Because a dropped directory's walk has no known size up front, there is no percentage -- progress is a running count of sources registered so far.

      Cancelling cancels the RPC, which the server observes and stops discovery on; sources already registered stay registered, and this returns an empty tally rather than raising -- the cancel was intentional.

      Parameters:
      url - Absolute path (or directory) on the server's filesystem
      sourceType - Explicit adapter type ("zarr", "ome-zarr", ...); empty means auto-detect via the adapters' claim protocol
      onProgress - receives one AddSourceProgress per source as it registers, or null
      shouldCancel - polled once per action message, or null
      Returns:
      the terminal AddSourceResult: added / alreadyPresent / refreshed / removed source_ids and failed (path, reason) pairs. Re-adding a registered path REBUILDS it against the file as it is now -- that is what refreshed reports, and it is how a source picks up an in-place edit. Registration wrote each source's catalog row, so anything beyond the ids is one query(java.lang.String) away.
      Throws:
      IOException

    • deregisterLocalPath

      public RemoveSourceResult deregisterLocalPath(String rootUrl) throws IOException
      Deregister a drag-dropped source branch on the SERVER at runtime.

      The narrow counterpart to registerLocalPath(java.lang.String): it removes ONLY drag-dropped sources, which the server identifies by the dnd:// origin scheme on their catalog source_url. Every source at or under rootUrl goes as a unit; a non-dnd:// root is refused by the server.

      Parameters:
      rootUrl - the dnd:// branch root to remove
      Returns:
      removed source_ids, and failed entries whose path carries the source_id
      Throws:
      IOException

    • getLabelSets

      public List<String> getLabelSets(String imageArrayId) throws IOException
      The array_ids of the label sets served under an image.

      A label set is an ordinary tensor of its image, named <image array_id>/@labels/<name>, so this is a catalog query over the path and nothing more -- getTensor(java.lang.String) / getDescriptor(java.lang.String) read one like any other tensor.

      Parameters:
      imageArrayId - the image's array_id
      Returns:
      the sets' array_ids, sorted; empty when the image has none
      Throws:
      IOException

    • listRois

      public RoiListResult listRois(String arrayId) throws IOException
      Fetch a tensor's ROI annotations.

      There is no plane or bbox filter: a client hit-tests and re-renders from the resident set. Annotations are private data, gated by the tensor's source like its pixels, so they are not on the SQL surface.

      Parameters:
      arrayId - unversioned array_id of the tensor
      Returns:
      the annotations, a truncated flag, and sets -- every set on the tensor with its stored row count, whatever rois covers
      Throws:
      IOException

    • listRois

      public RoiListResult listRois(String arrayId, String setName) throws IOException
      Fetch one layer of a tensor's ROI annotations.
      Parameters:
      arrayId - unversioned array_id of the tensor
      setName - restrict to one layer, and the only way to read a reserved (@) set; empty means the client-owned sets
      Returns:
      the annotations, a truncated flag, and the tensor's sets
      Throws:
      IOException

    • putRois

      public RoiPutResult putRois(String arrayId, List<RoiAnnotation> rois) throws IOException
      Create or update ROI annotations on a tensor, as one batch.

      Geometry is biopb.image.ROI in LEVEL-0 pixel coordinates -- a shape drawn on a downsampled level must be scaled up by the caller. Only the 2-D vector arms are accepted (point / rectangle / ellipse / polygon / polyline); a mask or mesh is refused, because instance segmentation belongs in a label tensor.

      An annotation with an empty roi_id is created (the server mints a uuid4); one that names an existing id is updated. The batch is applied in a single transaction, last writer wins.

      Parameters:
      arrayId - unversioned array_id every annotation belongs to
      rois - the annotations to store
      Returns:
      stored (with server-assigned roi_id / rev / timestamps) and conflicts
      Throws:
      IOException

    • putRois

      public RoiPutResult putRois(String arrayId, List<RoiAnnotation> rois, boolean checkRev) throws IOException
      Create or update ROI annotations, optionally conditional on rev.
      Parameters:
      arrayId - unversioned array_id every annotation belongs to
      rois - the annotations to store
      checkRev - make each write conditional on rev matching what is stored; mismatches come back in conflicts and are not applied, and the rest of the batch still lands
      Returns:
      stored and conflicts
      Throws:
      IOException

    • deleteRois

      public RoiDeleteResult deleteRois(String arrayId, List<String> roiIds, String setName) throws IOException
      Delete ROI annotations.

      With roiIds, deletes exactly those. Without, deletes every annotation on the tensor -- narrowed to setName when given, which is how a whole layer is dropped.

      Parameters:
      arrayId - unversioned array_id of the tensor
      roiIds - the ids to delete, or empty for all
      setName - narrow a delete-all to one layer, or empty
      Returns:
      the ids actually removed
      Throws:
      IOException

    • pruneRois

      public RoiPruneResult pruneRois(int unseenDays, boolean apply) throws IOException
      Report, and with apply delete, annotations whose image is gone.

      An annotation is unseen when the catalog has not held its source for unseenDays (a row whose source never appeared counts from its creation). Reserved, server-owned sets are never pruned. Requires the server-wide token: orphans have no source to authorize against.

      Parameters:
      unseenDays - how long a source must have been absent to count
      apply - false reports only; true deletes
      Returns:
      the unseen annotations grouped per tensor, and the row count deleted (0 on a report)
      Throws:
      IOException

    • getDescriptor

      public TensorDescriptor getDescriptor(String arrayId)
      Fetch one tensor's TensorDescriptor by its globally-unique array_id.

      A tensor is identified by its array_id alone (see the tensor identity policy at the top of proto/biopb/tensor/descriptor.proto), so this takes that one identifier. Works even when the source is beyond the server's query row cap. One GetFlightInfo per call -- nothing is cached, because a descriptor is what the server says now. A bare source_id (single-tensor source, or to anchor on a multi-tensor source's default/first tensor) is accepted. To enumerate ALL tensors/scenes of a source, read its catalog row's tensors column -- NOT this method.

      This is a cheap probe -- it does NOT resolve. On an unresolved (cloud / synced-folder) source it raises an error pointing at resolveSource(java.lang.String).

      Parameters:
      arrayId - Globally-unique tensor id, e.g. "zarr_a3f2" or "aics_7f3/Image:0"
      Returns:
      The TensorDescriptor for that tensor

    • getPhysicalScale

      public TensorFlightClient.PhysicalScale getPhysicalScale(String arrayId)
      Per-dimension physical pixel size + unit for a tensor.

      Returns a TensorFlightClient.PhysicalScale whose scale and unit arrays are aligned with the tensor's dim_labels (source axis order), or null when no physical sizes are known (an older server, or a format that carries none).

      physical_scale/physical_unit are TensorDescriptor fields the server fills on every GetFlightInfo (issue #31), so this is one describe -- the cheap projection, which never requests the opt-in metadata_json field or the O(chunks) endpoint plan. (Contrast getSourceMetadata(java.lang.String), which ships the whole OME tree; do not dig physical sizes out of that -- this is the compact projection meant for display scale.)

      Parameters:
      arrayId - Globally-unique tensor id (source_id or source_id/field). A bare source id anchors on the source's default (first) tensor.
      Returns:
      A PhysicalScale, or null if no physical scale is known

    • getTensor

      public <T extends net.imglib2.type.NativeType<T> & net.imglib2.type.numeric.RealType<T>> net.imglib2.RandomAccessibleInterval<T> getTensor(String arrayId)
      Get a RandomAccessibleInterval for a tensor by its globally-unique array_id.
      Type Parameters:
      T - The pixel type
      Parameters:
      arrayId - Globally-unique tensor id (source_id or source_id/field)
      Returns:
      RandomAccessibleInterval containing the requested tensor

    • getTensor

      public <T extends net.imglib2.type.NativeType<T> & net.imglib2.type.numeric.RealType<T>> net.imglib2.RandomAccessibleInterval<T> getTensor(String arrayId, SliceHint sliceHint)
      Get a RandomAccessibleInterval for a tensor (by array_id) with a slice hint.
      Type Parameters:
      T - The pixel type
      Parameters:
      arrayId - Globally-unique tensor id (source_id or source_id/field)
      sliceHint - Optional slice hint
      Returns:
      RandomAccessibleInterval containing the requested tensor

    • getTensor

      public <T extends net.imglib2.type.NativeType<T> & net.imglib2.type.numeric.RealType<T>> net.imglib2.RandomAccessibleInterval<T> getTensor(String arrayId, long[] scaleHint, String reductionMethod)
      Get a RandomAccessibleInterval for a tensor (by array_id) with scaled reads.
      Type Parameters:
      T - The pixel type
      Parameters:
      arrayId - Globally-unique tensor id (source_id or source_id/field)
      scaleHint - Per-dimension scale factors
      reductionMethod - Requested reduction method
      Returns:
      RandomAccessibleInterval containing the requested tensor

    • getTensor

      public <T extends net.imglib2.type.NativeType<T> & net.imglib2.type.numeric.RealType<T>> net.imglib2.RandomAccessibleInterval<T> getTensor(String arrayId, SliceHint sliceHint, long[] scaleHint, String reductionMethod)
      Get a RandomAccessibleInterval for a tensor (by array_id) with all options.
      Type Parameters:
      T - The pixel type
      Parameters:
      arrayId - Globally-unique tensor id (source_id or source_id/field)
      sliceHint - Optional slice hint
      scaleHint - Per-dimension scale factors
      reductionMethod - Requested reduction method
      Returns:
      lazy RandomAccessibleInterval containing the requested tensor

    • getTensorAsPb

      public SerializedTensor getTensorAsPb(String arrayId)
      Get a SerializedTensor protobuf for a whole tensor.
      Parameters:
      arrayId - Globally-unique tensor id (source_id or source_id/field)
      Returns:
      SerializedTensor protobuf object

    • getTensorAsPb

      public SerializedTensor getTensorAsPb(String arrayId, SliceHint sliceHint, long[] scaleHint, String reductionMethod)
      Get a SerializedTensor protobuf for cross-process transfer. Returns a protobuf containing connection info and chunk tickets for lazy reconstruction. The protobuf can be serialized to bytes and broadcast to worker processes (e.g., Spark), where each worker can call tensorFromPb() to reconstruct a lazy imglib2 array.
      Parameters:
      arrayId - Globally-unique tensor id (source_id or source_id/field)
      sliceHint - Optional slice hint
      scaleHint - Per-dimension scale factors
      reductionMethod - Requested reduction method
      Returns:
      SerializedTensor protobuf object

    • flightInfoOf

      public static org.apache.arrow.flight.FlightInfo flightInfoOf(SerializedTensor pb)
      The plan a SerializedTensor carries: its serialized Arrow FlightInfo.

    • descriptorOf

      public static TensorDescriptor descriptorOf(SerializedTensor pb)
      The resolved descriptor a SerializedTensor's plan names.

    • tensorFromPb

      public static <T extends net.imglib2.type.NativeType<T> & net.imglib2.type.numeric.RealType<T>> net.imglib2.RandomAccessibleInterval<T> tensorFromPb(SerializedTensor pb, long cacheBytes)
      The lazy imglib2 array a SerializedTensor describes. The one consumer-side helper. The handle is a FlightInfo plus where and as whom to read it. Its plan is consumed directly on first access; only the documented endpoint-less progressive-discovery plan is refreshed.
      Type Parameters:
      T - The pixel type
      Parameters:
      pb - SerializedTensor protobuf object
      cacheBytes - Maximum cache size in bytes
      Returns:
      RandomAccessibleInterval with lazy chunk loading

    • close

      public void close()
      Specified by:
      close in interface AutoCloseable

    • healthCheck

      public Map<String,Object> healthCheck() throws IOException
      Check server health status via Flight action. Returns a map with health status information including: - status: "SERVING" or other status string - source_count: number of registered sources - metadata_db_enabled: whether the server offers a catalog (false means it serves sources by id alone and every catalog surface refuses) - writable: whether server accepts uploads - uptime_seconds: server uptime in seconds
      Returns:
      Map containing health status
      Throws:
      IOException - If action fails

    • getUploadStatus

      public Map<String,Object> getUploadStatus(String arrayId) throws IOException
      Get upload status for a writable tensor.
      Parameters:
      arrayId - the array_id setupArrayUpload() returned
      Returns:
      Map containing source_id (the array_id passed in -- the key name mirrors the server's own status map shape), state, expected_chunks, uploaded_chunks and reason
      Throws:
      IOException - If the action fails

    • setupArrayUpload

      public TensorDescriptor setupArrayUpload(String arrayId, long[] shape, String dtype, long[] chunkShape, List<String> dimLabels, String omeMetadataJson)
      Declare a tensor to fill: the first half of an upload.

      Experimental. The upload API (tensor creation, chunk upload, and upload-status polling) may change.

      An upload adds a tensor to a source that already exists and never creates one. A result that belongs to no source of yours goes on the server's scratch source, at the fixed id "scratch" -- every writable server serves one, so there is nothing to ask for first. Declare, then fill: the returned descriptor is the server's echo -- array_id, shape, dtype, chunk_shape, dim_labels -- and is what uploadArray(biopb.tensor.TensorDescriptor, net.imglib2.RandomAccessibleInterval<T>), uploadChunk(biopb.tensor.TensorDescriptor, biopb.tensor.ChunkBounds, net.imglib2.RandomAccessibleInterval<T>) and setUploadStatus(java.lang.String, biopb.tensor.UploadStatus.State, java.lang.String) take. setUploadStatus(java.lang.String, biopb.tensor.UploadStatus.State, java.lang.String) is what publishes the tensor and marks it complete.

      A field is taken while its tensor is served: a second add under it -- pending, published or discarded -- is refused. Only the server's reclaim sweep frees one, after a discarded upload's upload_ttl.

      Parameters:
      arrayId - "<scheme>://<source_id>/@fields/<name>", where scheme is the store format -- zarr for an OME-Zarr image group, cache for the chunks as uploaded -- and source_id is a source the server already serves -- one it discovered, or "scratch". The @fields segment is not optional: an uploaded tensor keeps its own store beside its source, and the mark is what stops its id colliding with one of the file's own tensors, so a bare "<source_id>/<field>" is a native tensor id and is refused here. The one other form is "zarr://<image array_id>/@labels/<name>", a label set of an image the server already serves. A set is unsigned-integer, spans its image's non-channel axes at full length, and its all-zero chunks are skipped by uploadArray(biopb.tensor.TensorDescriptor, net.imglib2.RandomAccessibleInterval<T>). The scheme names the store format and nothing else: the answered id carries none
      shape - the tensor's shape
      dtype - the numpy dtype string to store it as (e.g. "<u2")
      chunkShape - the upload grid; null or empty means one chunk. A request, not a promise: the server plans on its own grid and answers with it
      dimLabels - optional dimension labels
      omeMetadataJson - ignored except for a label set's image-label block. Metadata is source-scoped: a tensor inherits its source's, and the scratch source has none
      Returns:
      the new tensor's descriptor, under the id it keeps

    • setupArrayUpload

      public TensorDescriptor setupArrayUpload(String arrayId, long[] shape, String dtype, long[] chunkShape, List<String> dimLabels, String omeMetadataJson, Integer ttlSeconds)
      setupArrayUpload(String, long[], String, long[], List, String) with a lifetime.

      How long the result is worth keeping is the producer's to say, and null asks for no deadline. A source may cap it -- a scratch source caps every upload on it, an unset request included -- so the answer's own ttl_seconds is what was granted, which may be shorter. Past it the tensor is discarded as if you had discarded it: the store goes and the id reads DISCARDED.

      Parameters:
      ttlSeconds - seconds to keep the tensor, or null for no deadline; must be positive, since zero is not a lifetime
      Returns:
      the new tensor's descriptor, its ttl_seconds the lifetime granted

    • setupArrayUpload

      public <T extends net.imglib2.type.NativeType<T> & net.imglib2.type.numeric.RealType<T>> TensorDescriptor setupArrayUpload(String arrayId, net.imglib2.RandomAccessibleInterval<T> template, long[] chunkShape, List<String> dimLabels, String omeMetadataJson)
      Declare a tensor shaped like an array you already hold.

      Experimental, with the rest of the upload API. The template's shape and pixel type stand in for the explicit shape/dtype of setupArrayUpload(String, long[], String, long[], List, String); it is the array about to be uploaded, or one shaped like it.

      Type Parameters:
      T - the pixel type
      Parameters:
      arrayId - as in setupArrayUpload(String, long[], String, long[], List, String)
      template - the array to be uploaded, or one shaped like it
      chunkShape - the upload grid; null or empty means one chunk
      dimLabels - optional dimension labels
      omeMetadataJson - as in setupArrayUpload(String, long[], String, long[], List, String)
      Returns:
      the new tensor's descriptor

    • uploadArray

      public <T extends net.imglib2.type.NativeType<T> & net.imglib2.type.numeric.RealType<T>> Map<String,Object> uploadArray(TensorDescriptor descriptor, net.imglib2.RandomAccessibleInterval<T> array)
      Fill a declared tensor with an array, and seal it.

      Experimental, with the rest of the upload API.

      array must match the descriptor's shape; it is walked on the descriptor's chunk grid, every block is sent as one chunk, and the source is finished. An all-zero block of a label set is not sent at all: the store's fill value already reads as background, so one labelled frame of a thousand costs one frame (biopb/biopb#1059).

      Type Parameters:
      T - the pixel type
      Parameters:
      descriptor - the descriptor setupArrayUpload(java.lang.String, long[], java.lang.String, long[], java.util.List<java.lang.String>, java.lang.String) returned
      array - the array to upload
      Returns:
      the sealed upload status, as getUploadStatus(java.lang.String) reports it
      Throws:
      IllegalArgumentException - if array does not match the declared shape
      UploadRefusedException - if the upload is over -- sealed or discarded

    • uploadChunk

      public <T extends net.imglib2.type.NativeType<T> & net.imglib2.type.numeric.RealType<T>> void uploadChunk(TensorDescriptor descriptor, ChunkBounds bounds, net.imglib2.RandomAccessibleInterval<T> source)
      Upload one chunk of a declared tensor.

      Experimental, with the rest of the upload API.

      The manual half of uploadArray(biopb.tensor.TensorDescriptor, net.imglib2.RandomAccessibleInterval<T>): a caller writing chunks itself calls this per chunk and setUploadStatus(java.lang.String, biopb.tensor.UploadStatus.State, java.lang.String) when done. The chunk's elements are read out of source at bounds -- in that interval's own global coordinates, so the whole array can be passed for every chunk.

      Type Parameters:
      T - the pixel type
      Parameters:
      descriptor - the descriptor setupArrayUpload(java.lang.String, long[], java.lang.String, long[], java.util.List<java.lang.String>, java.lang.String) returned
      bounds - chunk start/stop coordinates
      source - the array to read the chunk out of
      Throws:
      UploadRefusedException - if the upload is over -- sealed or discarded

    • setUploadStatus

      public Map<String,Object> setUploadStatus(String arrayId, UploadStatus.State state, String reason)
      Move an upload along its lifecycle; the only thing that moves one.

      Experimental, with the rest of the upload API.

      The states form a ladder, and a call climbs it or stands still:

      • READY -- publish and seal. The source becomes readable, a chunk that was never uploaded reads back as zeros, and no further chunk is accepted, so what is there is final. This is the state a consumer waiting on a result polls for, and what uploadArray(biopb.tensor.TensorDescriptor, net.imglib2.RandomAccessibleInterval<T>) sets for you.
      • DISCARDED -- give up, from any of the above. Whatever the server minted goes with it: a zarr:// member's store, a label set's sidecar, and the tensor's place in its source's listing. This is how an uploaded tensor is deleted; the field frees after the server's reclaim sweep, like any other discarded upload's.

      Setting the state the upload is already in is a no-op; moving back down the ladder is refused.

      Parameters:
      arrayId - what setupArrayUpload(java.lang.String, long[], java.lang.String, long[], java.util.List<java.lang.String>, java.lang.String) answered with, or a label set's array_id as getLabelSets(java.lang.String) reports it
      state - READY or DISCARDED
      reason - why, for DISCARDED; it is what a poller waiting on this result reads back, so write it for them
      Returns:
      the resulting upload status, as getUploadStatus(java.lang.String) reports it. DISCARDED is total -- an id tracking no upload answers UNKNOWN rather than throwing
      Throws:
      UploadRefusedException - if the upload was discarded, so it cannot be moved