Detaching Subdirectories into Separate Git Repositories Using Subtree and Filter-Branch

Introduction

In modern software development, as project scales expand and modularization requirements increase, there is often a need to separate specific subdirectories from large codebases into independent Git repositories. This refactoring operation not only facilitates modular code management but also enhances team collaboration efficiency. However, how to completely preserve subdirectory commit history during separation while ensuring the independence of new repositories presents a challenging technical problem.

Problem Context and Requirements Analysis

Assume we have a Git repository named XYZ with the following directory structure:

XYZ/
    .git/
    XY1/
    ABC/
    XY2/

Where the ABC subdirectory is functionally independent from other directories and needs to be separated into an independent repository. The target structure becomes:

XYZ/
    .git/
    XY1/
    XY2/
ABC/
    .git/
    ABC/

Key requirements include: completely preserving ABC directory commit history, ensuring new repository independence, and safely removing the separated subdirectory from the original repository.

Git Subtree Method Detailed Explanation

Git subtree is an official tool introduced in Git version 1.7.11, specifically designed for subdirectory separation and merging operations. This method is simple to operate with lower risk, making it the currently recommended primary solution.

Basic Operational Steps

First execute subtree splitting in the original repository:

cd /path/to/XYZ
git subtree split -P ABC -b abc-only

Here the -P parameter specifies the subdirectory path to separate, while -b parameter creates the new branch name. It's important to note that path parameters should not contain leading or trailing slashes. For deep paths in Windows systems, Unix-style separators must be used.

Next create the new repository and import separated content:

mkdir /path/to/ABC && cd /path/to/ABC
git init
git pull /path/to/XYZ abc-only

Configure remote repository and push code:

git remote add origin git@github.com:user/abc-repo.git
git push -u origin master

Cleaning Original Repository

After separation completes, remove the subdirectory from the original repository:

cd /path/to/XYZ
git rm -rf ABC
git commit -m "Remove ABC directory, separated to independent repository"

Filter-Branch Method In-Depth Analysis

git filter-branch is a more traditional solution that provides finer-grained control but involves relatively complex operations. This method achieves subdirectory separation by rewriting Git history.

Complete Operation Process

First clone the original repository:

git clone /XYZ /ABC

Git uses hard links to optimize the cloning process, which doesn't affect subsequent operation safety.

Backup and clean remote references:

cd /ABC
for branch in $(git branch -r | sed "s/.*origin\///"); do
    git branch -t $branch origin/$branch
done
git remote rm origin

Execute filter-branch operation:

git filter-branch --tag-name-filter cat --prune-empty --subdirectory-filter ABC -- --all

Key parameter explanations: --subdirectory-filter ABC specifies retaining only ABC directory content; --prune-empty automatically removes empty commits generated by filtering; -- --all ensures all branches and tags are processed.

Space Optimization and Cleaning

After filter-branch operations, thorough cleaning is required to reclaim disk space:

git reset --hard
git for-each-ref --format="%(refname)" refs/original/ | xargs -n 1 git update-ref -d
git reflog expire --expire=now --all
git gc --aggressive --prune=now

Method Comparison and Selection Recommendations

Both methods have their advantages and disadvantages: git subtree offers simple operations with lower risk, suitable for most scenarios; git filter-branch provides finer control but involves complex operations requiring more technical experience. For new projects or teams less familiar with Git, the git subtree method is recommended.

Advanced Topics: History Cleaning and Security Considerations

In certain sensitive scenarios, complete removal of subdirectory historical traces may be necessary. This typically involves security requirements such as password leaks or intellectual property protection.

Complete History Deletion

Using filter-branch to thoroughly remove all history of specific directories:

git filter-branch --prune-empty --tree-filter 'rm -rf ABC' HEAD

Verification of Cleaning Effectiveness

Confirm complete removal from history:

git log -- ABC

This command should return no results, proving the ABC directory has been completely removed from history.

Practical Application Case

Consider a Node.js browser compatibility library project:

node-browser-compat
├── ArrayBuffer
├── Audio
├── Blob
├── FormData
├── atob
├── btoa
├── location
└── navigator

Need to separate btoa functionality into an independent repository:

cd ~/node-browser-compat/
git subtree split -P btoa -b btoa-only
mkdir ~/btoa/ && cd ~/btoa/
git init
git pull ~/node-browser-compat btoa-only

Best Practices and Important Notes

Before executing separation operations, always create complete repository backups. For team projects, coordinate all members to pause commits during operations. After separation completes, promptly update relevant build scripts and documentation.

Pay special attention to path handling: avoid using relative paths or pathnames containing special characters. For subprojects containing multiple directory levels, ensure correct path separators are used.

Conclusion

Through both git subtree and git filter-branch methods, developers can effectively separate subdirectories from Git repositories into independent codebases while completely preserving historical records. Choosing the appropriate method depends on specific requirements, team experience levels, and security considerations. Proper implementation of these technologies can significantly improve code maintainability and team development efficiency.