Open Issues Need Help
View All on GitHubDedup and compress your device mapper devices. Works especially well with thin provisioning.
AI Summary: Rename the Rust-based blk-archive project to avoid confusion with a pre-existing project. The new name should be concise, memorable, and reflect the tool's functionality (deduplication and compression of block devices, potentially extending to files). Consider user suggestions and choose a name before the beta release.
Dedup and compress your device mapper devices. Works especially well with thin provisioning.
AI Summary: Implement a cryptographic hash verification mechanism for blk-archive, allowing integrity checks without needing the original source. This involves adding a new metadata field to store the hash (e.g., BLAKE3 or SHA-256) during the 'pack' phase and modifying the 'verify' command to support standalone verification using this hash.
Dedup and compress your device mapper devices. Works especially well with thin provisioning.
AI Summary: Analyze the I/O performance of the `blk-archive` tool using `strace` to identify and optimize inefficient read patterns, such as excessively small reads. This involves running `strace` on `blk-archive unpack` and `blk-archive verify`, analyzing the output for small reads, and proposing and implementing buffered read strategies to improve performance. Benchmarking before and after the changes is crucial.
Dedup and compress your device mapper devices. Works especially well with thin provisioning.
AI Summary: Investigate and implement the use of `fallocate()` in the `FileUnpackDest` component of the blk-archive project to efficiently handle sparse data during unpacking. This involves evaluating current zero/unmapped region handling, implementing `fallocate()` with appropriate flags, ensuring cross-filesystem compatibility, adding benchmarks and tests, and documenting any filesystem-specific behavior.
Dedup and compress your device mapper devices. Works especially well with thin provisioning.
AI Summary: Implement automatic detection and regeneration of offset files for slab files in the blk-archive tool. This involves comparing timestamps to detect inconsistencies, rebuilding offset files from slab data if necessary, adding CLI commands for manual rebuilds, integrating checks into existing verification and restoration processes, and writing comprehensive test cases.
Dedup and compress your device mapper devices. Works especially well with thin provisioning.
AI Summary: Implement a `migrate` subcommand for the `blk-archive` tool to enable moving streams between archives. This is crucial for garbage collection and reclaiming space by copying only live data to a new archive, leaving behind obsolete data. The subcommand needs to handle selective stream copying, avoid redundant data duplication, and maintain archive consistency. Thorough testing and documentation are also required.
Dedup and compress your device mapper devices. Works especially well with thin provisioning.
AI Summary: The task is to gather feedback from alpha users on the need for remote repository support in the blk-archive tool. This involves finalizing the design for remote repository configuration and access, documenting supported protocols, and determining the implementation timeline based on user feedback. The design should consider bandwidth efficiency, authentication, error handling, and read-only vs. read-write capabilities.
Dedup and compress your device mapper devices. Works especially well with thin provisioning.
AI Summary: The task involves enhancing the robustness of the `blk-archive` tool by implementing comprehensive handling of archive corruption. This includes designing a fault model, adding integrity metadata (checksums, etc.) to slab files, creating tools to simulate and detect damage, and enabling partial restoration in degraded modes. The goal is to improve the reliability and trustworthiness of the archive format, especially for long-term storage.
Dedup and compress your device mapper devices. Works especially well with thin provisioning.
AI Summary: Parallelize the slab decompression and verification processes within the blk-archive tool to improve the performance of the `unpack` and `verify` commands. This involves profiling the current single-threaded bottleneck, refactoring code to utilize a thread pool or parallel iterator, ensuring deterministic output and handling potential memory usage and output stream issues.
Dedup and compress your device mapper devices. Works especially well with thin provisioning.
AI Summary: Optimize the Content Defined Chunking (CDC) algorithm within the blk-archive tool to improve its performance, particularly when dealing with large datasets. This involves profiling the current CDC implementation, identifying bottlenecks, benchmarking alternative algorithms or parameter adjustments, and potentially exploring parallel processing or GPU acceleration while maintaining compatibility with existing archive formats.
Dedup and compress your device mapper devices. Works especially well with thin provisioning.
AI Summary: Enhance the blk-archive tool by adding versioning and descriptive metadata to the slab file header. This involves defining a new header structure, updating code to read and write the new format, adding validation and compatibility checks, and updating documentation. The goal is to improve forward compatibility, error handling, and debugging capabilities.
Dedup and compress your device mapper devices. Works especially well with thin provisioning.
AI Summary: Investigate and implement a fallback mechanism in the blk-archive tool to write zeroes to unallocated regions of block devices when discard operations (TRIM) are unavailable. This involves auditing existing code, potentially adding new logic for zero-writing, and updating documentation and tests to handle scenarios with disabled discards.
Dedup and compress your device mapper devices. Works especially well with thin provisioning.
AI Summary: Verify that the `blk-archive create` command correctly reads and uses all relevant environment variables, including handling of missing or invalid values. This involves identifying all environment variables used (documented and undocumented), auditing the code for their usage, ensuring appropriate fallback behavior, and documenting the variables and their effects.
Dedup and compress your device mapper devices. Works especially well with thin provisioning.
AI Summary: Create a man page for the blk-archive command-line tool, documenting its usage, options, examples, and other relevant information for users to easily understand and utilize the tool.
Dedup and compress your device mapper devices. Works especially well with thin provisioning.
AI Summary: Improve the documentation for the `blk-archive` tool by expanding the existing use cases document (`doc/Use Cases.md`). This involves reorganizing the document, converting bullet points into descriptive scenarios, adding concrete command-line examples, clarifying terminology, and potentially splitting the content into multiple files for better readability. The goal is to create clear, user-friendly examples that demonstrate how `blk-archive` can be used in real-world scenarios.
Dedup and compress your device mapper devices. Works especially well with thin provisioning.
AI Summary: The task is to fix a bug in the `blk-archive` tool where interrupting a `pack` operation causes the process to hang. The solution involves improving the signal handling and thread management within the Rust codebase to ensure clean termination upon interruption, potentially including cleanup of partial states. The goal is to allow for graceful exits and potentially future support for resuming interrupted operations.
Dedup and compress your device mapper devices. Works especially well with thin provisioning.