Open Issues Need Help
View All on GitHubLichtFeld Studio: Where reality and the digital world blend.
LichtFeld Studio: Where reality and the digital world blend.
AI Summary: This issue aims to implement proper splat transformations within the CUDA-based inference rasterizer. Currently, splats rely on a placeholder view matrix transform, and the task is to replace this with a correct and performant transformation directly in the CUDA rasterization code.
LichtFeld Studio: Where reality and the digital world blend.
LichtFeld Studio: Where reality and the digital world blend.
LichtFeld Studio: Where reality and the digital world blend.
LichtFeld Studio: Where reality and the digital world blend.
LichtFeld Studio: Where reality and the digital world blend.
LichtFeld Studio: Where reality and the digital world blend.
AI Summary: Evaluate the performance and suitability of integrating the DeGauss.io 3D reconstruction tool into the existing 3D Gaussian Splatting project. Consider its performance, licensing (GPLv3), and potential benefits/drawbacks compared to the current implementation. A recommendation on whether to integrate it should be provided.
LichtFeld Studio: Where reality and the digital world blend.
AI Summary: Implement the Histogram of Oriented Gradients (HOG) feature descriptor from the provided paper into an existing C++/CUDA 3D Gaussian Splatting renderer, supporting both Markov Chain Monte Carlo (MCMC) and Adaptive Density Control (ADC) methods. This involves integrating the HOG implementation into the existing codebase, ensuring compatibility with the current rendering pipeline, and potentially optimizing for performance on GPUs.
LichtFeld Studio: Where reality and the digital world blend.
LichtFeld Studio: Where reality and the digital world blend.
AI Summary: The task is to debug and fix a UI issue in a C++/CUDA application for 3D Gaussian Splatting. Dragging rendering setting windows causes unintended mouse events, and the fix requires preventing other events while dragging. This likely involves modifying event handling within the application's UI framework.
LichtFeld Studio: Where reality and the digital world blend.
LichtFeld Studio: Where reality and the digital world blend.
AI Summary: The task involves upgrading the C++ codebase of a high-performance 3D Gaussian Splatting renderer from C++17 to C++23. This will enable the use of modern C++ features like ranges, coroutines, and newer STL components for potential performance improvements and code simplification.
LichtFeld Studio: Where reality and the digital world blend.
AI Summary: Port a sparsification technique from a specific branch of the gsplat repository (https://github.com/MrNeRF/gsplat/tree/sparsity) into the main Gaussian Splatting CUDA project. This technique, demonstrated in a Sutro Tower example (https://vincentwoo.com/3d/sutro_tower/), aims to reduce the number of splats after training, improving splat fidelity and potentially speeding up rendering. The goal is to integrate this functionality to allow users to apply the sparsification after training their own models.
LichtFeld Studio: Where reality and the digital world blend.
AI Summary: Verify the functionality of batched image processing within the existing 3D Gaussian Splatting C++/CUDA implementation. This involves modifying the training loop to handle multiple images simultaneously (batch_size > 1) and ensuring correct functionality and performance improvements. Testing should confirm that batched processing improves geometry and doesn't introduce errors.
LichtFeld Studio: Where reality and the digital world blend.
AI Summary: Integrate the fisheye undistortion fix from the linked gsplat pull request (#736) into the existing 3D Gaussian Splatting C++ and CUDA implementation. This likely involves reviewing the changes in the pull request, adapting them to the current codebase, testing the integration, and potentially resolving any conflicts or bugs that arise.
LichtFeld Studio: Where reality and the digital world blend.
AI Summary: Implement a default densification strategy for 3D Gaussian Splatting in C++ and CUDA, porting functionality from a Python implementation in the gsplat repository. This involves understanding the original densification algorithm (adc) and translating it into a performant C++/CUDA version.
LichtFeld Studio: Where reality and the digital world blend.
AI Summary: The task involves modifying the C++/CUDA code of a 3D Gaussian Splatting renderer to compute the backward pass after each loss calculation instead of accumulating losses before backpropagation. This change aims to improve memory efficiency and requires benchmarking and memory consumption verification. The modification should be made around line 220 of `trainer.cpp`.
LichtFeld Studio: Where reality and the digital world blend.
AI Summary: The task requires enabling optional pose optimization in a C++ and CUDA implementation of 3D Gaussian Splatting. This involves connecting existing, but unused, gradient computations for camera extrinsics (view matrices) in the rasterizer's backward pass, likely requiring modifications to the C++ code and potentially the CUDA kernels to correctly propagate gradients. Reference implementation details are provided from a Python example.
LichtFeld Studio: Where reality and the digital world blend.
AI Summary: Remove the unnecessary `abs()` function calls from the scale and opacity regularization terms within the `trainer.cpp` file of the 3D Gaussian Splatting C++/CUDA project. This involves modifying two lines of code to improve efficiency.
LichtFeld Studio: Where reality and the digital world blend.
AI Summary: Implement a flexible cluster-based level-of-detail (LOD) system for real-time rendering of composed scenes within the existing 3D Gaussian Splatting C++ and CUDA framework. This involves designing and integrating a system that dynamically adjusts the detail level of rendered scenes based on factors like distance from the camera and available resources, potentially leveraging the existing Gaussian splatting structure for efficient LOD management. The goal is to improve rendering performance, particularly for complex scenes, while maintaining visual fidelity.
LichtFeld Studio: Where reality and the digital world blend.
AI Summary: Integrate RmlUi into an existing C++/CUDA project (3D Gaussian Splatting renderer) to create a Blender-style user interface, alongside maintaining ImGui for debugging. This involves designing and implementing the UI elements (tool panels, property editors, etc.) using RmlUi, and integrating it seamlessly with the existing rendering pipeline.
LichtFeld Studio: Where reality and the digital world blend.
AI Summary: Implement Visual SLAM (Simultaneous Localization and Mapping) to replace the Colmap dependency in the 3D Gaussian Splatting project. This involves creating a system to estimate camera poses and build a 3D map without relying on pre-existing structure from motion data. The initial goal is to generate pose priors for global bundle adjustment.
LichtFeld Studio: Where reality and the digital world blend.
AI Summary: Implement a visualization tool for the 3D Gaussian Splatting project to display COLMAP camera positions and dataset images before training. This involves creating a 3D viewer showing camera frustums, integrating an image browser for dataset images, and adding GUI controls for navigation and validation. The goal is to improve scene understanding and allow users to verify dataset integrity before starting the computationally expensive training process.
LichtFeld Studio: Where reality and the digital world blend.
AI Summary: Implement key editing features from the Splatshop project (brush selection, deletion, basic transformations, undo/redo) into the existing 3D Gaussian Splatting C++/CUDA renderer. Future phases involve adding a layer system and advanced features like painting and asset management.
LichtFeld Studio: Where reality and the digital world blend.
AI Summary: Implement a viewer-only mode for the 3D Gaussian Splatting project, allowing users to load and visualize pre-trained models without the training pipeline. This involves loading splat data from a .ply file, setting up a camera (either from COLMAP data or a default), integrating the viewer with existing rendering capabilities, and ensuring efficient performance.
LichtFeld Studio: Where reality and the digital world blend.
AI Summary: Refactor the C++/CUDA 3D Gaussian Splatting trainer to run in a separate thread when a GUI is enabled, allowing the GUI to be the main entry point. Implement GUI controls (stop/resume, save, exit) to manage the training process, ensuring thread-safe communication and graceful handling of training state.
LichtFeld Studio: Where reality and the digital world blend.
AI Summary: Implement enhanced camera controls for a 3D Gaussian Splatting viewer, including orbit and first-person modes, and a GUI control widget to switch between them. This involves modifying existing viewer code to add new camera movement logic and a user interface element.
LichtFeld Studio: Where reality and the digital world blend.
AI Summary: Integrate the existing 3DGUT kernels and the implemented MCMC strategy into the project's trainer to enable 3DGUT functionality. This involves connecting the kernels and strategy to the existing training pipeline.
LichtFeld Studio: Where reality and the digital world blend.