How can I permanently delete a file stored in Git?
Categories:
Permanently Delete a File from Git History
Learn how to completely remove sensitive files from your Git repository's history, ensuring they are no longer traceable in any commit.
Accidentally committed a sensitive file, a large binary, or credentials to your Git repository? Simply deleting the file and committing the change only removes it from the latest commit, but it remains accessible in the repository's history. To truly remove a file permanently from all commits, you need to rewrite your repository's history. This article will guide you through the process using git filter-repo and BFG Repo-Cleaner, two powerful tools for this task.
Understanding the Need for History Rewriting
When you commit a file to Git, it becomes part of the repository's history. Even if you later delete the file in a subsequent commit, its previous versions are still stored within earlier commits. This means anyone with access to your repository can traverse the commit history and retrieve the 'deleted' file. For sensitive data like API keys, passwords, or large files that bloat your repository size, this is unacceptable. Rewriting history effectively creates a new history where the specified file never existed, making it inaccessible.
Method 1: Using git filter-repo (Recommended)
git filter-repo is the successor to git filter-branch and is significantly faster and safer for rewriting repository history. It requires Python 3.5 or newer. This tool is ideal for removing files, directories, or even specific content from your entire commit history.
1. Install git filter-repo
First, ensure you have git filter-repo installed. If you don't, you can install it via pip:
2. Clone a fresh copy of your repository
It's crucial to work on a fresh clone to avoid accidentally corrupting your local working copy or other branches. Do not perform this operation on your original clone.
3. Navigate into the new repository directory
Change your current directory to the newly cloned repository.
4. Run git filter-repo to remove the file
Execute the command to remove the specific file from all commits. Replace path/to/your/sensitive_file.txt with the actual path to the file you want to delete.
5. Force push the rewritten history
After git filter-repo completes, your local repository's history will be rewritten. You must then force push these changes to your remote repository. This will overwrite the old history. Inform your collaborators about this change.
pip install git-filter-repo
git clone --mirror git@github.com:your-org/your-repo.git
cd your-repo.git
git filter-repo --path path/to/your/sensitive_file.txt --invert-paths
git push origin --force --all
git push origin --force --tags
Commands to install, clone, filter, and force push using git filter-repo.
--path path/to/your/sensitive_directory/ --invert-paths. To remove multiple files/directories, you can specify --path-glob 'path/to/file1' --path-glob 'path/to/dir2/*' --invert-paths.Method 2: Using BFG Repo-Cleaner
BFG Repo-Cleaner is a faster, simpler alternative to git filter-branch for removing large files or sensitive data. It's written in Scala and runs on the JVM. It's particularly good at removing large files that are no longer needed.
1. Download BFG Repo-Cleaner
Download the latest bfg.jar from the official BFG Repo-Cleaner website or its GitHub releases page.
2. Clone a fresh bare repository
Similar to git filter-repo, work on a fresh clone. A bare clone is recommended for BFG.
3. Run BFG to clean the repository
Execute BFG with the appropriate command to remove your file. Replace your-repo.git with your repository name and sensitive_file.txt with the file you want to delete.
4. Navigate into the repository and clean up
BFG creates a new history. You need to navigate into the repository and run git reflog expire --expire=now --all && git gc --prune=now --aggressive to clean up old references and garbage collect the repository.
5. Force push the rewritten history
Finally, force push the cleaned history to your remote repository. This will overwrite the old history.
wget https://repo1.maven.org/maven2/com/madgag/bfg/1.14.0/bfg-1.14.0.jar -O bfg.jar
git clone --mirror git@github.com:your-org/your-repo.git
java -jar bfg.jar --delete-files sensitive_file.txt your-repo.git
cd your-repo.git
git reflog expire --expire=now --all && git gc --prune=now --aggressive
git push origin --force --all
git push origin --force --tags
Commands to download, clone, clean with BFG, and force push.
--strip-blobs-bigger-than 100M or replace text using --replace-text passwords.txt.Post-Deletion Workflow and Collaboration
After rewriting history and force pushing, all collaborators must re-clone the repository or perform a specific set of commands to update their local copies. Simply pulling will not work correctly because the history has diverged.
sequenceDiagram
participant DevA as Developer A
participant DevB as Developer B
participant Remote as Remote Repository
DevA->>Remote: Original Push (with sensitive file)
DevA->>DevA: Realizes sensitive file is present
DevA->>DevA: Rewrites history (git filter-repo/BFG)
DevA->>Remote: Force Push (new history)
Remote-->>DevA: History Updated
DevA->>DevB: "Hey, I force pushed, please re-clone or reset!"
DevB->>DevB: git fetch origin
DevB->>DevB: git reset --hard origin/main (or relevant branch)
DevB->>DevB: git clean -dfx (to remove untracked files)
DevB-->>DevB: Local history updated to match remoteSequence diagram illustrating the post-deletion workflow for collaborators.
git reset --hard origin/main will discard those changes. They should stash or commit their work on a temporary branch before resetting, then reapply it to the new history.