Tandemn-Labs/tensor-iroh

AI Summary: Implement time and latency measurements within the existing Rust unit test for the Iroh P2P protocol. This involves adding code to the test to record the time taken for message transmission between two nodes, even when running on the same machine. The goal is to provide a basic performance benchmark, even in a simplified test environment.

AI Summary: Optimize the peer-to-peer tensor transfer mechanism in a distributed inference system. The current implementation uses a new QUIC connection for each tensor, resulting in high latency. The task involves refactoring the code to use a singleton Endpoint and persistent connections for reusing connections and reducing handshake overhead. This will likely involve modifying both Python (for the Iroh bindings) and potentially Rust (for the Iroh core) code to support persistent connections, multiplexed streams, and 0-RTT resumption.

AI Summary: The task involves improving the performance of a Python-based peer-to-peer tensor processing system by replacing the existing UniFFI bindings with PyO3 bindings. This is to leverage PyO3's faster, more direct interface with Python's C API, thereby reducing latency in data transfer between nodes in the distributed system. The current system uses Iroh for gossip and blob transfer, and the improvement focuses on optimizing the data transfer mechanism between peers.