Visualizing Repo-Level Dedupe

This visualization demonstrates block-level deduplication across all models in bartowski/gemma-2-9b-it-GGUF.

Each row represents a file in the repository grouped into blocks of up to 64MB. The color of each block represents the deduplication ratio for the block, which is a function of how often the chunks in the block are shared between files. The darker the color, the more frequently content is shared, the better the overall upload and download times for a given file! You can read more about chunks, blocks, and the nitty gritty details of how we make this all work in our accompanying blog post.

To explore the visualization:

• Hover over a block in an individual file to highlight it and see where else it appears in the repository.
• Click any block in a file to see all other files that share blocks.
• Double-click anywhere on any file to reset and continue exploring.