Add repository migration plan and cleanup script (#400)

This commit is contained in:
Peter Steinberger 2025-07-18 08:08:27 +02:00 committed by GitHub
parent 44946f1006
commit 1f75177208
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
3 changed files with 683 additions and 0 deletions

196
docs/org-migrate.md Normal file
View file

@ -0,0 +1,196 @@
# GitHub Organization Migration Plan
This document outlines the migration process for moving the VibeTunnel repository from `amantus-ai/vibetunnel` to `vibetunnel/vibetunnel`.
## 1. Fast-lane Transfer (GitHub Built-in Method)
The simplest approach using GitHub's native transfer feature:
1. Navigate to **Settings → General → Danger Zone → Transfer**
2. Enter the new owner: `vibetunnel`
3. Type the repository name to confirm
4. Accept the invite from the destination org
5. Done! ✅
### What Transfers Automatically
✅ **Code & History**
- All branches and commit history
- Git tags and annotated tags
✅ **Project Management**
- Issues and pull requests (with all comments)
- Projects (classic and new)
- Releases and release assets
- Milestones and labels
✅ **Community Features**
- Stars and watchers
- Wiki content
- Fork relationships
✅ **Security & Integration**
- Webhooks configurations
- Deploy keys
- Repository-level secrets
- GitHub Actions workflows
- Git LFS objects (copied in background)
### What Needs Manual Updates
⚠️ **Organization-level Settings**
- Branch protection rules (inherits new org defaults - review carefully)
- Organization-level secrets (must recreate in new org)
- Environment-level secrets (if used outside repo scope)
- Team permissions (reassign in new org structure)
⚠️ **External Integrations**
- CI/CD systems with hardcoded URLs
- Documentation with repository links
- Package registries (npm, etc.)
- External webhooks
- Status badges in README
## 2. Zero-Downtime Migration Checklist
### Pre-Migration (1-2 days before)
- [ ] **Prepare Target Organization**
- Create `vibetunnel` organization if not exists
- Set up teams and permissions structure
- Configure organization-level settings
- Review default branch protection rules
- [ ] **Audit Current Setup**
- Document all webhooks and integrations
- List organization/environment secrets
- Note branch protection rules
- Identify external services using the repo
- [ ] **Notify Stakeholders**
- Team members about the migration
- Users via issue/discussion if needed
- Update any public documentation
### Migration Day
- [ ] **Pause Activity (Optional but Recommended)**
- Merge or close active PRs
- Create a migration tag: `pre-migration-snapshot`
- Announce brief maintenance window
- [ ] **Execute Transfer**
- Perform the GitHub transfer
- Accept invitation immediately
- Verify all content transferred
- [ ] **Immediate Post-Transfer**
- Push a test commit to verify write access
- Check Actions are running
- Verify webhooks are firing
### Post-Migration Updates
- [ ] **Update Git Remotes**
```bash
# For all local clones
git remote set-url origin git@github.com:vibetunnel/vibetunnel.git
# Verify the change
git remote -v
```
- [ ] **Update Documentation**
- README badges and links
- Installation instructions
- Contributing guidelines
- API documentation
- [ ] **Update External Services**
- CI/CD configurations
- Package registry URLs
- Monitoring services
- Documentation sites
- [ ] **Update Package Configurations**
```json
// package.json
{
"repository": {
"type": "git",
"url": "git+https://github.com/vibetunnel/vibetunnel.git"
},
"bugs": {
"url": "https://github.com/vibetunnel/vibetunnel/issues"
},
"homepage": "https://github.com/vibetunnel/vibetunnel#readme"
}
```
## 3. Redirect Behavior
GitHub automatically sets up redirects:
- `https://github.com/amantus-ai/vibetunnel``https://github.com/vibetunnel/vibetunnel`
- Git operations: `git clone git@github.com:amantus-ai/vibetunnel.git` still works
- API calls to old URL redirect automatically
⚠️ **Redirect Limitations**:
- Redirects break if someone creates a new repo at `amantus-ai/vibetunnel`
- Some tools may not follow redirects properly
- Best practice: Update all references ASAP
## 4. Specific VibeTunnel Updates
### Code Updates
- [ ] Update `GITHUB_URL` in `mac/VibeTunnel/version.xcconfig`
- [ ] Update repository URLs in `package.json` files
- [ ] Update any hardcoded GitHub URLs in documentation
- [ ] Update CLAUDE.md references
### Build & Release
- [ ] Update GitHub Actions secrets if needed
- [ ] Verify macOS notarization still works
- [ ] Test release workflow with new repo URL
- [ ] Update Sparkle appcast URLs if applicable
### npm Package
- [ ] Update package.json repository field
- [ ] Consider publishing a patch version with updated URLs
- [ ] Update npm package description if needed
## 5. Rollback Plan
If issues arise:
1. GitHub Support can reverse transfers within a short window
2. Keep the migration tag for reference
3. Document any issues for future reference
## 6. Timeline
**Day 1**: Preparation
- Set up new organization
- Audit current configuration
- Notify team
**Day 2**: Migration
- Morning: Final preparations
- Midday: Execute transfer
- Afternoon: Update configurations
**Day 3**: Verification
- Test all integrations
- Monitor for issues
- Complete documentation updates
## Notes
- GitHub's transfer process is well-tested and reliable
- The automatic redirects provide good backward compatibility
- Most disruption comes from external tools, not GitHub itself
- Consider doing this during a low-activity period
## References
- [GitHub Docs: Transferring a repository](https://docs.github.com/en/repositories/creating-and-managing-repositories/transferring-a-repository)
- [GitHub Docs: About repository transfers](https://docs.github.com/en/repositories/creating-and-managing-repositories/transferring-a-repository#about-repository-transfers)
- [GitHub Docs: Repository redirects](https://docs.github.com/en/repositories/creating-and-managing-repositories/transferring-a-repository#redirects-and-git-operations)

278
web/docs/org-migrate.md Normal file
View file

@ -0,0 +1,278 @@
# GitHub Organization Migration Plan with Repository Cleanup
This document outlines the migration process for moving VibeTunnel to `vibetunnel/vibetunnel` with a clean history.
## Overview
Since `https://github.com/vibetunnel/vibetunnel` already exists, we'll perform a clean migration that:
1. Removes large files from history
2. Cleans up accidental commits
3. Preserves important history
4. Maintains all issues, PRs, and project management features
## Migration Strategy
### Option 1: Clean History Migration (Recommended)
Use BFG Repo-Cleaner or git-filter-repo to create a cleaned version of the repository, then push to the new location.
### Option 2: Fresh Start with Preserved History
Create a new repository with cleaned history while maintaining a reference to the old repository for historical purposes.
## Migration Script
Save this as `migrate-clean.sh`:
```bash
#!/bin/bash
set -euo pipefail
# Configuration
OLD_REPO="git@github.com:amantus-ai/vibetunnel.git"
NEW_REPO="git@github.com:vibetunnel/vibetunnel.git"
TEMP_DIR="vibetunnel-migration-$(date +%Y%m%d-%H%M%S)"
SIZE_THRESHOLD="10M" # Files larger than this will be removed
echo "🚀 Starting VibeTunnel repository migration with cleanup..."
# Create temporary directory
mkdir -p "$TEMP_DIR"
cd "$TEMP_DIR"
# Clone the repository (all branches and tags)
echo "📥 Cloning repository with all history..."
git clone --mirror "$OLD_REPO" vibetunnel-mirror
cd vibetunnel-mirror
# Create a backup first
echo "💾 Creating backup..."
cp -r . ../vibetunnel-backup
# Analyze repository for large files
echo "🔍 Analyzing repository for large files..."
git rev-list --objects --all | \
git cat-file --batch-check='%(objecttype) %(objectname) %(objectsize) %(rest)' | \
awk '/^blob/ {print substr($0,6)}' | \
sort --numeric-sort --key=2 --reverse | \
awk '$2 >= 10485760 {print $1, $2, $3}' > ../large-files.txt
echo "📊 Large files found:"
cat ../large-files.txt | while read hash size path; do
echo " - $path ($(numfmt --to=iec $size))"
done
# Download BFG Repo-Cleaner if not available
if ! command -v bfg &> /dev/null && [ ! -f ../bfg.jar ]; then
echo "📦 Downloading BFG Repo-Cleaner..."
curl -L -o ../bfg.jar https://repo1.maven.org/maven2/com/madgag/bfg/1.14.0/bfg-1.14.0.jar
fi
# Clean large files using BFG
echo "🧹 Removing large files from history..."
java -jar ../bfg.jar --strip-blobs-bigger-than "$SIZE_THRESHOLD" --no-blob-protection
# Clean specific file patterns (customize as needed)
echo "🗑️ Removing unwanted file patterns..."
java -jar ../bfg.jar --delete-files '*.{log,tmp,cache}' --no-blob-protection
java -jar ../bfg.jar --delete-folders '{node_modules,dist,build}' --no-blob-protection
# Remove sensitive data patterns (customize as needed)
echo "🔒 Removing potentially sensitive patterns..."
# Example: Remove files with specific names
# java -jar ../bfg.jar --delete-files 'secrets.json' --no-blob-protection
# Clean up the repository
echo "♻️ Cleaning up repository..."
git reflog expire --expire=now --all
git gc --prune=now --aggressive
# Show size comparison
echo "📏 Size comparison:"
cd ..
ORIGINAL_SIZE=$(du -sh vibetunnel-backup | cut -f1)
CLEANED_SIZE=$(du -sh vibetunnel-mirror | cut -f1)
echo " Original: $ORIGINAL_SIZE"
echo " Cleaned: $CLEANED_SIZE"
# Prepare for push
cd vibetunnel-mirror
# Update remote URL
echo "🔄 Updating remote URL..."
git remote set-url origin "$NEW_REPO"
# Create a migration notes file
cat > MIGRATION_NOTES.md << EOF
# Repository Migration Notes
This repository was migrated from amantus-ai/vibetunnel on $(date +%Y-%m-%d).
## Changes during migration:
- Removed files larger than $SIZE_THRESHOLD from history
- Cleaned up temporary and build files
- Preserved all code, issues, and pull requests
## Original repository:
- https://github.com/amantus-ai/vibetunnel
## Large files removed:
$(cat ../large-files.txt | while read hash size path; do echo "- $path ($(numfmt --to=iec $size))"; done)
EOF
# Interactive confirmation
echo "⚠️ Ready to push to $NEW_REPO"
echo "This will:"
echo " - Push all cleaned branches and tags"
echo " - Permanently remove large files from history"
echo ""
read -p "Continue? (y/N) " -n 1 -r
echo
if [[ ! $REPLY =~ ^[Yy]$ ]]; then
echo "❌ Aborted"
exit 1
fi
# Push to new repository
echo "📤 Pushing to new repository..."
git push --mirror "$NEW_REPO"
echo "✅ Migration complete!"
echo ""
echo "Next steps:"
echo "1. Check the new repository: https://github.com/vibetunnel/vibetunnel"
echo "2. Update all local clones to use the new URL"
echo "3. Update CI/CD configurations"
echo "4. Archive or delete the old repository"
echo ""
echo "Migration backup saved in: $(pwd)/../vibetunnel-backup"
```
## Alternative: Using git-filter-repo (More Powerful)
```bash
#!/bin/bash
# migrate-with-filter-repo.sh
# Install git-filter-repo first:
# brew install git-filter-repo
set -euo pipefail
OLD_REPO="git@github.com:amantus-ai/vibetunnel.git"
NEW_REPO="git@github.com:vibetunnel/vibetunnel.git"
TEMP_DIR="vibetunnel-clean-$(date +%Y%m%d-%H%M%S)"
# Clone repository
git clone "$OLD_REPO" "$TEMP_DIR"
cd "$TEMP_DIR"
# Analyze repository
git filter-repo --analyze
echo "Check .git/filter-repo/analysis for detailed reports"
# Remove large files (size in bytes, 10MB = 10485760)
git filter-repo --strip-blobs-bigger-than 10M
# Remove specific paths
git filter-repo --path node_modules/ --invert-paths
git filter-repo --path dist/ --invert-paths
git filter-repo --path build/ --invert-paths
git filter-repo --path "*.log" --invert-paths
git filter-repo --path "*.tmp" --invert-paths
# Remove specific commits by message pattern (optional)
# git filter-repo --message-callback '
# if b"WIP" in message or b"temp" in message.lower():
# message = b"[CLEANED] " + message
# return message
# '
# Add new remote and push
git remote add new-origin "$NEW_REPO"
git push new-origin --all
git push new-origin --tags
echo "✅ Migration with cleanup complete!"
```
## Pre-Migration Checklist
- [ ] **Identify files to remove**
```bash
# Find large files in history
git rev-list --objects --all | \
git cat-file --batch-check='%(objecttype) %(objectname) %(objectsize) %(rest)' | \
awk '/^blob/ && $3 > 10485760 {print $3, $4}' | \
sort -rn | \
numfmt --field=1 --to=iec
```
- [ ] **List accidental commits**
- Large binary files
- Generated files (dist/, build/)
- Dependencies (node_modules/)
- Log files
- Temporary files
- Any sensitive data
- [ ] **Backup current repository**
```bash
git clone --mirror git@github.com:amantus-ai/vibetunnel.git vibetunnel-backup
```
## What Gets Preserved
✅ **Preserved:**
- All source code
- Commit messages and authors
- Branch structure
- Tags and releases
- Important history
❌ **Removed:**
- Large binary files
- Generated/built files
- Accidentally committed dependencies
- Log and temporary files
- Specified sensitive data
## Post-Migration Steps
1. **Update all references**
- Package.json repository URLs
- Documentation links
- CI/CD configurations
- Local git remotes
2. **Verify the migration**
- Check file sizes
- Verify important history
- Test clone and build
- Ensure CI/CD works
3. **Communicate changes**
- Notify all contributors
- Update README with new URL
- Add note about the migration
## Important Notes
- This process rewrites history - all commit SHAs will change
- Contributors will need to re-clone or rebase their work
- The old repository should be archived, not deleted immediately
- Consider keeping the cleaned backup for a few weeks
## Size Reduction Examples
Common size reductions from cleaning:
- Removing accidentally committed node_modules: 50-200MB
- Removing build artifacts: 10-50MB
- Removing large media files: Variable
- Removing package-lock.json history: 5-20MB
## References
- [BFG Repo-Cleaner](https://rtyley.github.io/bfg-repo-cleaner/)
- [git-filter-repo](https://github.com/newren/git-filter-repo)
- [GitHub: Removing sensitive data](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/removing-sensitive-data-from-a-repository)

View file

@ -0,0 +1,209 @@
#!/bin/bash
# migrate-aggressive-clean-v2.sh - Aggressive cleanup script for VibeTunnel repository (preserves assets)
set -euo pipefail
# Color output
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
BLUE='\033[0;34m'
NC='\033[0m' # No Color
# Configuration
OLD_REPO="git@github.com:amantus-ai/vibetunnel.git"
NEW_REPO="git@github.com:vibetunnel/vibetunnel.git"
SIZE_THRESHOLD="5M" # More aggressive - remove files larger than 5MB
BFG_VERSION="1.14.0"
echo -e "${BLUE}🚀 VibeTunnel Repository AGGRESSIVE Cleanup (Assets Preserved)${NC}"
echo -e "${BLUE}=========================================================${NC}"
echo "Old repo: $OLD_REPO"
echo "New repo: $NEW_REPO"
echo "Size threshold: $SIZE_THRESHOLD"
echo ""
# Clone the repository (all branches and tags)
echo -e "${YELLOW}📥 Cloning repository with all history...${NC}"
git clone --mirror "$OLD_REPO" vibetunnel-mirror
cd vibetunnel-mirror
# Create a backup first
echo -e "${YELLOW}💾 Creating backup...${NC}"
cd ..
cp -r vibetunnel-mirror vibetunnel-backup
cd vibetunnel-mirror
# Get initial size
ORIGINAL_SIZE=$(du -sh . | cut -f1)
echo -e "${YELLOW}📏 Original size: ${RED}$ORIGINAL_SIZE${NC}"
# Download BFG Repo-Cleaner if not available
if ! command -v bfg &> /dev/null && [ ! -f ../bfg.jar ]; then
echo -e "${YELLOW}📦 Downloading BFG Repo-Cleaner...${NC}"
curl -L -o ../bfg.jar "https://repo1.maven.org/maven2/com/madgag/bfg/${BFG_VERSION}/bfg-${BFG_VERSION}.jar"
fi
# Determine BFG command
if command -v bfg &> /dev/null; then
BFG_CMD="bfg"
else
BFG_CMD="java -jar ../bfg.jar"
fi
echo -e "${YELLOW}🧹 Starting aggressive cleanup...${NC}"
# Remove large files EXCEPT those in assets/
echo -e "${YELLOW}🗑️ Removing all files larger than $SIZE_THRESHOLD (except assets/)...${NC}"
# First, get a list of large files NOT in assets/
git rev-list --objects --all | \
git cat-file --batch-check='%(objecttype) %(objectname) %(objectsize) %(rest)' | \
awk '/^blob/ {print substr($0,6)}' | \
awk '$2 >= 5242880 && $3 !~ /^assets\// {print $1}' > ../large-files-to-remove.txt
if [ -s ../large-files-to-remove.txt ]; then
echo "Found $(wc -l < ../large-files-to-remove.txt) large files to remove (excluding assets/)"
# Use BFG to remove specific blobs
while read -r blob_id; do
$BFG_CMD --strip-blobs-with-ids <(echo "$blob_id") --no-blob-protection .
done < ../large-files-to-remove.txt
fi
# Remove specific large directories and files
echo -e "${YELLOW}🗑️ Removing specific large files and directories...${NC}"
# Remove BunPrebuilts
$BFG_CMD --delete-folders 'BunPrebuilts' --no-blob-protection .
# Remove all node_modules everywhere
$BFG_CMD --delete-folders 'node_modules' --no-blob-protection .
# Remove all target directories (Rust builds)
$BFG_CMD --delete-folders 'target' --no-blob-protection .
# Remove electron binaries
$BFG_CMD --delete-folders 'electron' --no-blob-protection .
# Remove build artifacts
$BFG_CMD --delete-folders '{dist,build,out,.next,coverage,.nyc_output}' --no-blob-protection .
# Remove specific file patterns
echo -e "${YELLOW}🗑️ Removing unwanted file patterns...${NC}"
$BFG_CMD --delete-files '*.{log,tmp,cache,swp,swo,zip,tar,gz,dmg,pkg,exe,msi,deb,rpm,AppImage}' --no-blob-protection .
$BFG_CMD --delete-files '.DS_Store' --no-blob-protection .
$BFG_CMD --delete-files 'Thumbs.db' --no-blob-protection .
# Remove binaries
$BFG_CMD --delete-files '*.{dylib,so,dll,node}' --no-blob-protection . # Remove native binaries
$BFG_CMD --delete-files '*.rlib' --no-blob-protection . # Remove Rust libraries
# Remove data directory
$BFG_CMD --delete-folders 'data' --no-blob-protection .
# Remove linux binaries
$BFG_CMD --delete-files 'vibetunnel' --no-blob-protection .
$BFG_CMD --delete-files 'vibetunnel-tls' --no-blob-protection .
# Clean up temporary files
rm -f ../large-files-to-remove.txt
# Clean up the repository
echo -e "${YELLOW}♻️ Optimizing repository...${NC}"
git reflog expire --expire=now --all
git gc --prune=now --aggressive
# Show size comparison
echo -e "${BLUE}📏 Size comparison:${NC}"
cd ..
ORIGINAL_BYTES=$(du -sb vibetunnel-backup | cut -f1)
CLEANED_SIZE=$(du -sh vibetunnel-mirror | cut -f1)
CLEANED_BYTES=$(du -sb vibetunnel-mirror | cut -f1)
REDUCTION=$((100 - (CLEANED_BYTES * 100 / ORIGINAL_BYTES)))
echo -e " Original: ${RED}$ORIGINAL_SIZE${NC}"
echo -e " Cleaned: ${GREEN}$CLEANED_SIZE${NC}"
echo -e " Reduction: ${GREEN}${REDUCTION}%${NC}"
# Prepare for push
cd vibetunnel-mirror
# Update remote URL
echo -e "${YELLOW}🔄 Updating remote URL...${NC}"
git remote set-url origin "$NEW_REPO"
# Create a migration report
cat > ../MIGRATION_REPORT.md << EOF
# Repository Migration Report - AGGRESSIVE CLEANUP
**Migration Date:** $(date +"%Y-%m-%d %H:%M:%S")
**Original Repository:** https://github.com/amantus-ai/vibetunnel
**New Repository:** https://github.com/vibetunnel/vibetunnel
## Size Reduction Summary
- **Original Size:** $ORIGINAL_SIZE
- **Cleaned Size:** $CLEANED_SIZE
- **Reduction:** ${REDUCTION}% 🎉
## Aggressive Cleanup Performed
- ✅ All files larger than 5MB removed (except assets/)
- ✅ All node_modules directories removed
- ✅ All Rust target directories removed
- ✅ All BunPrebuilts removed (57MB + 53MB)
- ✅ All electron binaries removed
- ✅ All build artifacts removed (dist, build, out, .next)
- ✅ All archives removed (zip, tar, gz)
- ✅ All binary files removed (dylib, so, dll, node, rlib)
- ✅ All package files removed (dmg, pkg, exe, msi, deb, rpm, AppImage)
- ✅ All data directories removed
- ✅ Linux binaries removed
## What's Preserved
- ✅ All source code (TypeScript, Swift, JavaScript)
- ✅ All documentation
- ✅ All configuration files
- ✅ All tracked assets in assets/ directory (logos, icons, banners)
- ✅ Complete commit history
- ✅ All branches and tags
- ✅ Author information
## Important Notes
- **All commit SHAs have changed** due to history rewriting
- Contributors must re-clone the repository
- The old repository should be archived for reference
- Some CI/CD processes may need adjustment to rebuild removed artifacts
## Next Steps
1. Push to new repository:
\`\`\`bash
cd vibetunnel-mirror
git push --mirror git@github.com:vibetunnel/vibetunnel.git
\`\`\`
2. Update all local clones:
\`\`\`bash
git remote set-url origin git@github.com:vibetunnel/vibetunnel.git
\`\`\`
3. Archive the old repository on GitHub
4. Update all references in:
- CI/CD configurations
- package.json files
- Documentation
- Any external services
## Backup Location
The original repository backup is saved at:
\`/Users/steipete/Projects/vibetunnel/web/vibetunnel-backup/\`
EOF
echo -e "${GREEN}✅ Aggressive cleanup complete!${NC}"
echo ""
echo -e "${YELLOW}Repository is ready to push.${NC}"
echo -e "Cleaned size: ${GREEN}$CLEANED_SIZE${NC} (${GREEN}${REDUCTION}% reduction${NC})"
echo ""
echo "To push to the new repository, run:"
echo -e "${BLUE}cd $(pwd)${NC}"
echo -e "${BLUE}git push --mirror $NEW_REPO${NC}"
echo ""
echo "Backup saved in: $(dirname $(pwd))/vibetunnel-backup"
echo "Migration report: $(dirname $(pwd))/MIGRATION_REPORT.md"