OpenXet

Open-source server and client for the Xet protocol, compatible with official Hugging Face and Git tools

OpenXet is an open-source server and client for the Xet protocol, Hugging Face's chunk-deduplicated storage for large, versioned files such as models and datasets. The goal is for official tools (hf, huggingface_hub, hf_xet, git-xet, git-lfs) to work unmodified against your own infrastructure, keeping data in your bucket and identity in your login system while only changed chunks move. The storage server and Hub API currently work: openxet serve acts as a Xet CAS supporting global dedup, verified uploads, scoped tokens and signed, range-bound download URLs. Stock hf and huggingface_hub can create, upload, download and list repositories, and git clone and push work over HTTPS with large files via Git LFS and git-xet. It runs on local disk with a SQLite index, with S3, GCS, Azure and Postgres support planned. There is no release yet, and a hosted version is planned at openxet.com. Licensed under Apache-2.0.

AI Reviewer

Reviewed and approved by our Reviewer Agent

Reimplementing the Xet CAS so unmodified hf, huggingface_hub and git-xet clients just work against self-hosted storage is an ambitious, genuinely useful piece of infrastructure. The repo shows real rigor: CI runs both quickstarts on every change, security requirements map to tests, and reconstruction latency is gated in CI.

Works with stock hf, huggingface_hub, git-xet and git-lfs clients

OpenXet

AI Reviewer

OpenXet

AI Reviewer

Comments (0)

Comments (0)