From 1f751772088198cd6cd11b00fc155a99304b393f Mon Sep 17 00:00:00 2001 From: Peter Steinberger Date: Fri, 18 Jul 2025 08:08:27 +0200 Subject: [PATCH] Add repository migration plan and cleanup script (#400) --- docs/org-migrate.md | 196 +++++++++++++++++ web/docs/org-migrate.md | 278 ++++++++++++++++++++++++ web/scripts/migrate-aggressive-clean.sh | 209 ++++++++++++++++++ 3 files changed, 683 insertions(+) create mode 100644 docs/org-migrate.md create mode 100644 web/docs/org-migrate.md create mode 100755 web/scripts/migrate-aggressive-clean.sh diff --git a/docs/org-migrate.md b/docs/org-migrate.md new file mode 100644 index 00000000..d10630de --- /dev/null +++ b/docs/org-migrate.md @@ -0,0 +1,196 @@ +# GitHub Organization Migration Plan + +This document outlines the migration process for moving the VibeTunnel repository from `amantus-ai/vibetunnel` to `vibetunnel/vibetunnel`. + +## 1. Fast-lane Transfer (GitHub Built-in Method) + +The simplest approach using GitHub's native transfer feature: + +1. Navigate to **Settings → General → Danger Zone → Transfer** +2. Enter the new owner: `vibetunnel` +3. Type the repository name to confirm +4. Accept the invite from the destination org +5. Done! ✅ + +### What Transfers Automatically + +✅ **Code & History** +- All branches and commit history +- Git tags and annotated tags + +✅ **Project Management** +- Issues and pull requests (with all comments) +- Projects (classic and new) +- Releases and release assets +- Milestones and labels + +✅ **Community Features** +- Stars and watchers +- Wiki content +- Fork relationships + +✅ **Security & Integration** +- Webhooks configurations +- Deploy keys +- Repository-level secrets +- GitHub Actions workflows +- Git LFS objects (copied in background) + +### What Needs Manual Updates + +⚠️ **Organization-level Settings** +- Branch protection rules (inherits new org defaults - review carefully) +- Organization-level secrets (must recreate in new org) +- Environment-level secrets (if used outside repo scope) +- Team permissions (reassign in new org structure) + +⚠️ **External Integrations** +- CI/CD systems with hardcoded URLs +- Documentation with repository links +- Package registries (npm, etc.) +- External webhooks +- Status badges in README + +## 2. Zero-Downtime Migration Checklist + +### Pre-Migration (1-2 days before) + +- [ ] **Prepare Target Organization** + - Create `vibetunnel` organization if not exists + - Set up teams and permissions structure + - Configure organization-level settings + - Review default branch protection rules + +- [ ] **Audit Current Setup** + - Document all webhooks and integrations + - List organization/environment secrets + - Note branch protection rules + - Identify external services using the repo + +- [ ] **Notify Stakeholders** + - Team members about the migration + - Users via issue/discussion if needed + - Update any public documentation + +### Migration Day + +- [ ] **Pause Activity (Optional but Recommended)** + - Merge or close active PRs + - Create a migration tag: `pre-migration-snapshot` + - Announce brief maintenance window + +- [ ] **Execute Transfer** + - Perform the GitHub transfer + - Accept invitation immediately + - Verify all content transferred + +- [ ] **Immediate Post-Transfer** + - Push a test commit to verify write access + - Check Actions are running + - Verify webhooks are firing + +### Post-Migration Updates + +- [ ] **Update Git Remotes** + ```bash + # For all local clones + git remote set-url origin git@github.com:vibetunnel/vibetunnel.git + + # Verify the change + git remote -v + ``` + +- [ ] **Update Documentation** + - README badges and links + - Installation instructions + - Contributing guidelines + - API documentation + +- [ ] **Update External Services** + - CI/CD configurations + - Package registry URLs + - Monitoring services + - Documentation sites + +- [ ] **Update Package Configurations** + ```json + // package.json + { + "repository": { + "type": "git", + "url": "git+https://github.com/vibetunnel/vibetunnel.git" + }, + "bugs": { + "url": "https://github.com/vibetunnel/vibetunnel/issues" + }, + "homepage": "https://github.com/vibetunnel/vibetunnel#readme" + } + ``` + +## 3. Redirect Behavior + +GitHub automatically sets up redirects: +- `https://github.com/amantus-ai/vibetunnel` → `https://github.com/vibetunnel/vibetunnel` +- Git operations: `git clone git@github.com:amantus-ai/vibetunnel.git` still works +- API calls to old URL redirect automatically + +⚠️ **Redirect Limitations**: +- Redirects break if someone creates a new repo at `amantus-ai/vibetunnel` +- Some tools may not follow redirects properly +- Best practice: Update all references ASAP + +## 4. Specific VibeTunnel Updates + +### Code Updates +- [ ] Update `GITHUB_URL` in `mac/VibeTunnel/version.xcconfig` +- [ ] Update repository URLs in `package.json` files +- [ ] Update any hardcoded GitHub URLs in documentation +- [ ] Update CLAUDE.md references + +### Build & Release +- [ ] Update GitHub Actions secrets if needed +- [ ] Verify macOS notarization still works +- [ ] Test release workflow with new repo URL +- [ ] Update Sparkle appcast URLs if applicable + +### npm Package +- [ ] Update package.json repository field +- [ ] Consider publishing a patch version with updated URLs +- [ ] Update npm package description if needed + +## 5. Rollback Plan + +If issues arise: +1. GitHub Support can reverse transfers within a short window +2. Keep the migration tag for reference +3. Document any issues for future reference + +## 6. Timeline + +**Day 1**: Preparation +- Set up new organization +- Audit current configuration +- Notify team + +**Day 2**: Migration +- Morning: Final preparations +- Midday: Execute transfer +- Afternoon: Update configurations + +**Day 3**: Verification +- Test all integrations +- Monitor for issues +- Complete documentation updates + +## Notes + +- GitHub's transfer process is well-tested and reliable +- The automatic redirects provide good backward compatibility +- Most disruption comes from external tools, not GitHub itself +- Consider doing this during a low-activity period + +## References + +- [GitHub Docs: Transferring a repository](https://docs.github.com/en/repositories/creating-and-managing-repositories/transferring-a-repository) +- [GitHub Docs: About repository transfers](https://docs.github.com/en/repositories/creating-and-managing-repositories/transferring-a-repository#about-repository-transfers) +- [GitHub Docs: Repository redirects](https://docs.github.com/en/repositories/creating-and-managing-repositories/transferring-a-repository#redirects-and-git-operations) \ No newline at end of file diff --git a/web/docs/org-migrate.md b/web/docs/org-migrate.md new file mode 100644 index 00000000..c151f210 --- /dev/null +++ b/web/docs/org-migrate.md @@ -0,0 +1,278 @@ +# GitHub Organization Migration Plan with Repository Cleanup + +This document outlines the migration process for moving VibeTunnel to `vibetunnel/vibetunnel` with a clean history. + +## Overview + +Since `https://github.com/vibetunnel/vibetunnel` already exists, we'll perform a clean migration that: +1. Removes large files from history +2. Cleans up accidental commits +3. Preserves important history +4. Maintains all issues, PRs, and project management features + +## Migration Strategy + +### Option 1: Clean History Migration (Recommended) + +Use BFG Repo-Cleaner or git-filter-repo to create a cleaned version of the repository, then push to the new location. + +### Option 2: Fresh Start with Preserved History + +Create a new repository with cleaned history while maintaining a reference to the old repository for historical purposes. + +## Migration Script + +Save this as `migrate-clean.sh`: + +```bash +#!/bin/bash +set -euo pipefail + +# Configuration +OLD_REPO="git@github.com:amantus-ai/vibetunnel.git" +NEW_REPO="git@github.com:vibetunnel/vibetunnel.git" +TEMP_DIR="vibetunnel-migration-$(date +%Y%m%d-%H%M%S)" +SIZE_THRESHOLD="10M" # Files larger than this will be removed + +echo "🚀 Starting VibeTunnel repository migration with cleanup..." + +# Create temporary directory +mkdir -p "$TEMP_DIR" +cd "$TEMP_DIR" + +# Clone the repository (all branches and tags) +echo "📥 Cloning repository with all history..." +git clone --mirror "$OLD_REPO" vibetunnel-mirror +cd vibetunnel-mirror + +# Create a backup first +echo "💾 Creating backup..." +cp -r . ../vibetunnel-backup + +# Analyze repository for large files +echo "🔍 Analyzing repository for large files..." +git rev-list --objects --all | \ + git cat-file --batch-check='%(objecttype) %(objectname) %(objectsize) %(rest)' | \ + awk '/^blob/ {print substr($0,6)}' | \ + sort --numeric-sort --key=2 --reverse | \ + awk '$2 >= 10485760 {print $1, $2, $3}' > ../large-files.txt + +echo "📊 Large files found:" +cat ../large-files.txt | while read hash size path; do + echo " - $path ($(numfmt --to=iec $size))" +done + +# Download BFG Repo-Cleaner if not available +if ! command -v bfg &> /dev/null && [ ! -f ../bfg.jar ]; then + echo "📦 Downloading BFG Repo-Cleaner..." + curl -L -o ../bfg.jar https://repo1.maven.org/maven2/com/madgag/bfg/1.14.0/bfg-1.14.0.jar +fi + +# Clean large files using BFG +echo "🧹 Removing large files from history..." +java -jar ../bfg.jar --strip-blobs-bigger-than "$SIZE_THRESHOLD" --no-blob-protection + +# Clean specific file patterns (customize as needed) +echo "🗑️ Removing unwanted file patterns..." +java -jar ../bfg.jar --delete-files '*.{log,tmp,cache}' --no-blob-protection +java -jar ../bfg.jar --delete-folders '{node_modules,dist,build}' --no-blob-protection + +# Remove sensitive data patterns (customize as needed) +echo "🔒 Removing potentially sensitive patterns..." +# Example: Remove files with specific names +# java -jar ../bfg.jar --delete-files 'secrets.json' --no-blob-protection + +# Clean up the repository +echo "♻️ Cleaning up repository..." +git reflog expire --expire=now --all +git gc --prune=now --aggressive + +# Show size comparison +echo "📏 Size comparison:" +cd .. +ORIGINAL_SIZE=$(du -sh vibetunnel-backup | cut -f1) +CLEANED_SIZE=$(du -sh vibetunnel-mirror | cut -f1) +echo " Original: $ORIGINAL_SIZE" +echo " Cleaned: $CLEANED_SIZE" + +# Prepare for push +cd vibetunnel-mirror + +# Update remote URL +echo "🔄 Updating remote URL..." +git remote set-url origin "$NEW_REPO" + +# Create a migration notes file +cat > MIGRATION_NOTES.md << EOF +# Repository Migration Notes + +This repository was migrated from amantus-ai/vibetunnel on $(date +%Y-%m-%d). + +## Changes during migration: +- Removed files larger than $SIZE_THRESHOLD from history +- Cleaned up temporary and build files +- Preserved all code, issues, and pull requests + +## Original repository: +- https://github.com/amantus-ai/vibetunnel + +## Large files removed: +$(cat ../large-files.txt | while read hash size path; do echo "- $path ($(numfmt --to=iec $size))"; done) +EOF + +# Interactive confirmation +echo "⚠️ Ready to push to $NEW_REPO" +echo "This will:" +echo " - Push all cleaned branches and tags" +echo " - Permanently remove large files from history" +echo "" +read -p "Continue? (y/N) " -n 1 -r +echo +if [[ ! $REPLY =~ ^[Yy]$ ]]; then + echo "❌ Aborted" + exit 1 +fi + +# Push to new repository +echo "📤 Pushing to new repository..." +git push --mirror "$NEW_REPO" + +echo "✅ Migration complete!" +echo "" +echo "Next steps:" +echo "1. Check the new repository: https://github.com/vibetunnel/vibetunnel" +echo "2. Update all local clones to use the new URL" +echo "3. Update CI/CD configurations" +echo "4. Archive or delete the old repository" +echo "" +echo "Migration backup saved in: $(pwd)/../vibetunnel-backup" +``` + +## Alternative: Using git-filter-repo (More Powerful) + +```bash +#!/bin/bash +# migrate-with-filter-repo.sh + +# Install git-filter-repo first: +# brew install git-filter-repo + +set -euo pipefail + +OLD_REPO="git@github.com:amantus-ai/vibetunnel.git" +NEW_REPO="git@github.com:vibetunnel/vibetunnel.git" +TEMP_DIR="vibetunnel-clean-$(date +%Y%m%d-%H%M%S)" + +# Clone repository +git clone "$OLD_REPO" "$TEMP_DIR" +cd "$TEMP_DIR" + +# Analyze repository +git filter-repo --analyze +echo "Check .git/filter-repo/analysis for detailed reports" + +# Remove large files (size in bytes, 10MB = 10485760) +git filter-repo --strip-blobs-bigger-than 10M + +# Remove specific paths +git filter-repo --path node_modules/ --invert-paths +git filter-repo --path dist/ --invert-paths +git filter-repo --path build/ --invert-paths +git filter-repo --path "*.log" --invert-paths +git filter-repo --path "*.tmp" --invert-paths + +# Remove specific commits by message pattern (optional) +# git filter-repo --message-callback ' +# if b"WIP" in message or b"temp" in message.lower(): +# message = b"[CLEANED] " + message +# return message +# ' + +# Add new remote and push +git remote add new-origin "$NEW_REPO" +git push new-origin --all +git push new-origin --tags + +echo "✅ Migration with cleanup complete!" +``` + +## Pre-Migration Checklist + +- [ ] **Identify files to remove** + ```bash + # Find large files in history + git rev-list --objects --all | \ + git cat-file --batch-check='%(objecttype) %(objectname) %(objectsize) %(rest)' | \ + awk '/^blob/ && $3 > 10485760 {print $3, $4}' | \ + sort -rn | \ + numfmt --field=1 --to=iec + ``` + +- [ ] **List accidental commits** + - Large binary files + - Generated files (dist/, build/) + - Dependencies (node_modules/) + - Log files + - Temporary files + - Any sensitive data + +- [ ] **Backup current repository** + ```bash + git clone --mirror git@github.com:amantus-ai/vibetunnel.git vibetunnel-backup + ``` + +## What Gets Preserved + +✅ **Preserved:** +- All source code +- Commit messages and authors +- Branch structure +- Tags and releases +- Important history + +❌ **Removed:** +- Large binary files +- Generated/built files +- Accidentally committed dependencies +- Log and temporary files +- Specified sensitive data + +## Post-Migration Steps + +1. **Update all references** + - Package.json repository URLs + - Documentation links + - CI/CD configurations + - Local git remotes + +2. **Verify the migration** + - Check file sizes + - Verify important history + - Test clone and build + - Ensure CI/CD works + +3. **Communicate changes** + - Notify all contributors + - Update README with new URL + - Add note about the migration + +## Important Notes + +- This process rewrites history - all commit SHAs will change +- Contributors will need to re-clone or rebase their work +- The old repository should be archived, not deleted immediately +- Consider keeping the cleaned backup for a few weeks + +## Size Reduction Examples + +Common size reductions from cleaning: +- Removing accidentally committed node_modules: 50-200MB +- Removing build artifacts: 10-50MB +- Removing large media files: Variable +- Removing package-lock.json history: 5-20MB + +## References + +- [BFG Repo-Cleaner](https://rtyley.github.io/bfg-repo-cleaner/) +- [git-filter-repo](https://github.com/newren/git-filter-repo) +- [GitHub: Removing sensitive data](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/removing-sensitive-data-from-a-repository) \ No newline at end of file diff --git a/web/scripts/migrate-aggressive-clean.sh b/web/scripts/migrate-aggressive-clean.sh new file mode 100755 index 00000000..0b0b2c4b --- /dev/null +++ b/web/scripts/migrate-aggressive-clean.sh @@ -0,0 +1,209 @@ +#!/bin/bash +# migrate-aggressive-clean-v2.sh - Aggressive cleanup script for VibeTunnel repository (preserves assets) +set -euo pipefail + +# Color output +RED='\033[0;31m' +GREEN='\033[0;32m' +YELLOW='\033[1;33m' +BLUE='\033[0;34m' +NC='\033[0m' # No Color + +# Configuration +OLD_REPO="git@github.com:amantus-ai/vibetunnel.git" +NEW_REPO="git@github.com:vibetunnel/vibetunnel.git" +SIZE_THRESHOLD="5M" # More aggressive - remove files larger than 5MB +BFG_VERSION="1.14.0" + +echo -e "${BLUE}🚀 VibeTunnel Repository AGGRESSIVE Cleanup (Assets Preserved)${NC}" +echo -e "${BLUE}=========================================================${NC}" +echo "Old repo: $OLD_REPO" +echo "New repo: $NEW_REPO" +echo "Size threshold: $SIZE_THRESHOLD" +echo "" + +# Clone the repository (all branches and tags) +echo -e "${YELLOW}📥 Cloning repository with all history...${NC}" +git clone --mirror "$OLD_REPO" vibetunnel-mirror +cd vibetunnel-mirror + +# Create a backup first +echo -e "${YELLOW}💾 Creating backup...${NC}" +cd .. +cp -r vibetunnel-mirror vibetunnel-backup +cd vibetunnel-mirror + +# Get initial size +ORIGINAL_SIZE=$(du -sh . | cut -f1) +echo -e "${YELLOW}📏 Original size: ${RED}$ORIGINAL_SIZE${NC}" + +# Download BFG Repo-Cleaner if not available +if ! command -v bfg &> /dev/null && [ ! -f ../bfg.jar ]; then + echo -e "${YELLOW}📦 Downloading BFG Repo-Cleaner...${NC}" + curl -L -o ../bfg.jar "https://repo1.maven.org/maven2/com/madgag/bfg/${BFG_VERSION}/bfg-${BFG_VERSION}.jar" +fi + +# Determine BFG command +if command -v bfg &> /dev/null; then + BFG_CMD="bfg" +else + BFG_CMD="java -jar ../bfg.jar" +fi + +echo -e "${YELLOW}🧹 Starting aggressive cleanup...${NC}" + +# Remove large files EXCEPT those in assets/ +echo -e "${YELLOW}🗑️ Removing all files larger than $SIZE_THRESHOLD (except assets/)...${NC}" +# First, get a list of large files NOT in assets/ +git rev-list --objects --all | \ + git cat-file --batch-check='%(objecttype) %(objectname) %(objectsize) %(rest)' | \ + awk '/^blob/ {print substr($0,6)}' | \ + awk '$2 >= 5242880 && $3 !~ /^assets\// {print $1}' > ../large-files-to-remove.txt + +if [ -s ../large-files-to-remove.txt ]; then + echo "Found $(wc -l < ../large-files-to-remove.txt) large files to remove (excluding assets/)" + # Use BFG to remove specific blobs + while read -r blob_id; do + $BFG_CMD --strip-blobs-with-ids <(echo "$blob_id") --no-blob-protection . + done < ../large-files-to-remove.txt +fi + +# Remove specific large directories and files +echo -e "${YELLOW}🗑️ Removing specific large files and directories...${NC}" + +# Remove BunPrebuilts +$BFG_CMD --delete-folders 'BunPrebuilts' --no-blob-protection . + +# Remove all node_modules everywhere +$BFG_CMD --delete-folders 'node_modules' --no-blob-protection . + +# Remove all target directories (Rust builds) +$BFG_CMD --delete-folders 'target' --no-blob-protection . + +# Remove electron binaries +$BFG_CMD --delete-folders 'electron' --no-blob-protection . + +# Remove build artifacts +$BFG_CMD --delete-folders '{dist,build,out,.next,coverage,.nyc_output}' --no-blob-protection . + +# Remove specific file patterns +echo -e "${YELLOW}🗑️ Removing unwanted file patterns...${NC}" +$BFG_CMD --delete-files '*.{log,tmp,cache,swp,swo,zip,tar,gz,dmg,pkg,exe,msi,deb,rpm,AppImage}' --no-blob-protection . +$BFG_CMD --delete-files '.DS_Store' --no-blob-protection . +$BFG_CMD --delete-files 'Thumbs.db' --no-blob-protection . + +# Remove binaries +$BFG_CMD --delete-files '*.{dylib,so,dll,node}' --no-blob-protection . # Remove native binaries +$BFG_CMD --delete-files '*.rlib' --no-blob-protection . # Remove Rust libraries + +# Remove data directory +$BFG_CMD --delete-folders 'data' --no-blob-protection . + +# Remove linux binaries +$BFG_CMD --delete-files 'vibetunnel' --no-blob-protection . +$BFG_CMD --delete-files 'vibetunnel-tls' --no-blob-protection . + +# Clean up temporary files +rm -f ../large-files-to-remove.txt + +# Clean up the repository +echo -e "${YELLOW}♻️ Optimizing repository...${NC}" +git reflog expire --expire=now --all +git gc --prune=now --aggressive + +# Show size comparison +echo -e "${BLUE}📏 Size comparison:${NC}" +cd .. +ORIGINAL_BYTES=$(du -sb vibetunnel-backup | cut -f1) +CLEANED_SIZE=$(du -sh vibetunnel-mirror | cut -f1) +CLEANED_BYTES=$(du -sb vibetunnel-mirror | cut -f1) +REDUCTION=$((100 - (CLEANED_BYTES * 100 / ORIGINAL_BYTES))) + +echo -e " Original: ${RED}$ORIGINAL_SIZE${NC}" +echo -e " Cleaned: ${GREEN}$CLEANED_SIZE${NC}" +echo -e " Reduction: ${GREEN}${REDUCTION}%${NC}" + +# Prepare for push +cd vibetunnel-mirror + +# Update remote URL +echo -e "${YELLOW}🔄 Updating remote URL...${NC}" +git remote set-url origin "$NEW_REPO" + +# Create a migration report +cat > ../MIGRATION_REPORT.md << EOF +# Repository Migration Report - AGGRESSIVE CLEANUP + +**Migration Date:** $(date +"%Y-%m-%d %H:%M:%S") +**Original Repository:** https://github.com/amantus-ai/vibetunnel +**New Repository:** https://github.com/vibetunnel/vibetunnel + +## Size Reduction Summary +- **Original Size:** $ORIGINAL_SIZE +- **Cleaned Size:** $CLEANED_SIZE +- **Reduction:** ${REDUCTION}% 🎉 + +## Aggressive Cleanup Performed +- ✅ All files larger than 5MB removed (except assets/) +- ✅ All node_modules directories removed +- ✅ All Rust target directories removed +- ✅ All BunPrebuilts removed (57MB + 53MB) +- ✅ All electron binaries removed +- ✅ All build artifacts removed (dist, build, out, .next) +- ✅ All archives removed (zip, tar, gz) +- ✅ All binary files removed (dylib, so, dll, node, rlib) +- ✅ All package files removed (dmg, pkg, exe, msi, deb, rpm, AppImage) +- ✅ All data directories removed +- ✅ Linux binaries removed + +## What's Preserved +- ✅ All source code (TypeScript, Swift, JavaScript) +- ✅ All documentation +- ✅ All configuration files +- ✅ All tracked assets in assets/ directory (logos, icons, banners) +- ✅ Complete commit history +- ✅ All branches and tags +- ✅ Author information + +## Important Notes +- **All commit SHAs have changed** due to history rewriting +- Contributors must re-clone the repository +- The old repository should be archived for reference +- Some CI/CD processes may need adjustment to rebuild removed artifacts + +## Next Steps +1. Push to new repository: + \`\`\`bash + cd vibetunnel-mirror + git push --mirror git@github.com:vibetunnel/vibetunnel.git + \`\`\` + +2. Update all local clones: + \`\`\`bash + git remote set-url origin git@github.com:vibetunnel/vibetunnel.git + \`\`\` + +3. Archive the old repository on GitHub + +4. Update all references in: + - CI/CD configurations + - package.json files + - Documentation + - Any external services + +## Backup Location +The original repository backup is saved at: +\`/Users/steipete/Projects/vibetunnel/web/vibetunnel-backup/\` +EOF + +echo -e "${GREEN}✅ Aggressive cleanup complete!${NC}" +echo "" +echo -e "${YELLOW}Repository is ready to push.${NC}" +echo -e "Cleaned size: ${GREEN}$CLEANED_SIZE${NC} (${GREEN}${REDUCTION}% reduction${NC})" +echo "" +echo "To push to the new repository, run:" +echo -e "${BLUE}cd $(pwd)${NC}" +echo -e "${BLUE}git push --mirror $NEW_REPO${NC}" +echo "" +echo "Backup saved in: $(dirname $(pwd))/vibetunnel-backup" +echo "Migration report: $(dirname $(pwd))/MIGRATION_REPORT.md" \ No newline at end of file