Fix npm package to handle authenticate-pam as optional dependency (#390)

2026-04-26 15:07:39 +00:00 · 2025-07-17 09:30:28 +02:00 · 2025-07-17 09:30:28 +02:00 · 693565d9ea
commit 693565d9ea
parent fab0647cbe
10 changed files with 544 additions and 20 deletions
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@ -1,5 +1,35 @@
 # Changelog

+## [1.0.0-beta.12] - 2025-01-17
+
+### 🚀 Major Improvements
+
+#### **NPM Package Fixes**
+- Fixed critical npm installation issues on Linux systems
+- `authenticate-pam` is now correctly an optional dependency
+- Linux users can install without PAM development headers
+- Proper package.npm.json configuration is now used for npm releases
+
+### 🐛 Bug Fixes
+
+- Fixed PAM module and npm_config_prefix issues on Ubuntu (#380)
+- Fixed SSH key generation errors on non-localhost HTTP connections (#384)
+- Fixed npm package documentation and unified READMEs (#381)
+- Removed verbose output from vt script (#383)
+
+### 📚 Documentation
+
+- Added comprehensive npm publishing guide (npm.md) with installation instructions
+- Added Discord member count badge with correct server ID
+- Added Node.js version, Discord, and Homebrew installation badges
+- Improved npm package installation documentation
+- Added Docker test script for npm package validation
+
+### 🧹 Maintenance
+
+- Cleaned up old npm package versions (11.2, 11.3) due to installation issues
+- Added test-npm-docker.sh script for automated npm package testing
+
 ## [1.0.0-beta.11] - 2025-01-16

 #### **Better Settings Organization**
--- a/mac/VibeTunnel/version.xcconfig
+++ b/mac/VibeTunnel/version.xcconfig
@ -1,8 +1,8 @@
 // VibeTunnel Version Configuration
 // This file contains the version and build number for the app

-MARKETING_VERSION = 1.0.0-beta.11
-CURRENT_PROJECT_VERSION = 200
+MARKETING_VERSION = 1.0.0-beta.12
+CURRENT_PROJECT_VERSION = 201

 // Domain and GitHub configuration
 APP_DOMAIN = vibetunnel.sh
--- a/web/.gitignore
+++ b/web/.gitignore
@ -156,3 +156,9 @@ test.cast

 # Temporary build directories
 temp-spawn-helper/
+
+# Package directory (generated by build)
+package/
+
+# Docker test files (moved to scripts/)
+test-vibetunnel*.dockerfile
--- a/web/npm.md
+++ b/web/npm.md
@ -0,0 +1,305 @@
+# NPM Publishing Guide for VibeTunnel
+
+## Installation Guide
+
+### Installing VibeTunnel from NPM
+
+VibeTunnel is published as an npm package that works on macOS and Linux. The package includes prebuilt binaries for common platforms to avoid compilation.
+
+#### Basic Installation
+
+```bash
+# Install globally (recommended)
+npm install -g vibetunnel
+
+# Or install locally in a project
+npm install vibetunnel
+```
+
+#### Platform-Specific Notes
+
+**macOS**: 
+- Works out of the box
+- PAM authentication supported natively
+
+**Linux**:
+- Works without additional dependencies
+- PAM authentication is optional - installs only if PAM headers are available
+- If you need PAM authentication, install development headers first:
+  ```bash
+  # Ubuntu/Debian
+  sudo apt-get install libpam0g-dev
+  
+  # RHEL/CentOS/Fedora
+  sudo yum install pam-devel
+  ```
+
+#### Verifying Installation
+
+```bash
+# Check version
+vibetunnel --version
+
+# Run the server
+vibetunnel
+
+# The server will start on http://localhost:4020
+```
+
+#### Docker Installation
+
+For containerized environments:
+
+```dockerfile
+FROM node:20-slim
+
+# Optional: Install PAM headers for authentication support
+# RUN apt-get update && apt-get install -y libpam0g-dev
+
+# Install VibeTunnel
+RUN npm install -g vibetunnel
+
+# Expose the default port
+EXPOSE 4020
+
+# Run VibeTunnel
+CMD ["vibetunnel"]
+```
+
+#### Troubleshooting Installation
+
+1. **"Cannot find module '../build/Release/pty.node'"**
+   - The package includes prebuilds, this shouldn't happen
+   - Try reinstalling: `npm uninstall -g vibetunnel && npm install -g vibetunnel`
+
+2. **PAM authentication not working on Linux**
+   - Install PAM headers: `sudo apt-get install libpam0g-dev`
+   - Reinstall VibeTunnel to compile the PAM module
+
+3. **Permission errors during installation**
+   - Use a Node.js version manager (nvm, fnm) instead of system Node.js
+   - Or fix npm permissions: https://docs.npmjs.com/resolving-eacces-permissions-errors
+
+## Quick Release Checklist
+
+1. **Update versions** in all 3 files:
+   - `package.json`
+   - `package.npm.json` 
+   - `../mac/VibeTunnel/version.xcconfig`
+
+2. **Build**: `pnpm run build:npm`
+
+3. **Verify**: 
+   ```bash
+   tar -xf vibetunnel-*.tgz package/package.json
+   grep optionalDependencies package/package.json  # Must show authenticate-pam
+   rm -rf package/
+   ```
+
+4. **Publish tarball**:
+   ```bash
+   npm publish vibetunnel-*.tgz --tag beta
+   npm dist-tag add vibetunnel@VERSION latest
+   ```
+
+⚠️ **NEVER** use `npm publish` without the tarball filename!
+
+## Critical Issue History: Wrong package.json Used in Releases
+
+### The Problem
+
+We've repeatedly published npm packages with the wrong configuration:
+- **Version 11.2**: Used main `package.json` instead of `package.npm.json`
+- **Version 11.3**: Also used main `package.json` despite having the correct `package.npm.json`
+- **Both versions had to be unpublished** due to Linux installation failures
+
+This causes installation failures on Linux systems because:
+- Main `package.json` has `authenticate-pam` as a **regular dependency**
+- `package.npm.json` has `authenticate-pam` as an **optional dependency**
+
+When `authenticate-pam` is a regular dependency, npm fails the entire installation if PAM headers (libpam0g-dev) aren't available.
+
+### Root Cause
+
+The build script (`scripts/build-npm.js`) checks for `package.npm.json` and uses it if available, BUT:
+- During `npm publish`, npm runs the `prepublishOnly` script which triggers `build:npm`
+- This rebuilds the package, potentially overwriting the correct configuration
+- The timing and execution context can cause the wrong package.json to be used
+
+### The Solution
+
+**NEVER use `npm publish` directly!** Instead:
+
+1. Build the package explicitly:
+   ```bash
+   pnpm run build:npm
+   ```
+
+2. Verify the package has the correct configuration:
+   ```bash
+   # Extract and check package.json from the tarball
+   tar -xf vibetunnel-*.tgz package/package.json
+   cat package/package.json | grep -A5 -B5 authenticate-pam
+   ```
+
+3. Ensure `authenticate-pam` is under `optionalDependencies`:
+   ```json
+   "optionalDependencies": {
+     "authenticate-pam": "^1.0.5"
+   }
+   ```
+
+4. Publish the pre-built tarball:
+   ```bash
+   npm publish vibetunnel-*.tgz --tag beta
+   npm dist-tag add vibetunnel@VERSION latest  # if needed
+   ```
+
+## Correct Release Process
+
+### 1. Update Version Numbers
+```bash
+# Update all three version files - MUST keep in sync!
+vim package.json          # Update version
+vim package.npm.json      # Update version to match
+vim ../mac/VibeTunnel/version.xcconfig  # Update MARKETING_VERSION
+```
+
+### 2. Build the Package
+```bash
+pnpm run build:npm
+```
+
+### 3. Verify the Build
+```bash
+# Check the tarball exists (look in parent directory!)
+ls -la *.tgz
+
+# Extract and verify authenticate-pam is optional
+tar -xf vibetunnel-*.tgz package/package.json
+cat package/package.json | grep -A5 -B5 authenticate-pam
+
+# Should show:
+#   "optionalDependencies": {
+#     "authenticate-pam": "^1.0.5"
+#   }
+
+# Clean up
+rm -rf package/
+```
+
+### 4. Test Installation Locally
+```bash
+# Test on a system without PAM headers
+docker run --rm -it node:20 bash
+npm install /path/to/vibetunnel-*.tgz
+# Should succeed even without libpam0g-dev
+```
+
+### 5. Publish
+```bash
+# Publish the pre-built tarball with beta tag
+npm publish vibetunnel-*.tgz --tag beta
+
+# Also tag as latest if stable
+npm dist-tag add vibetunnel@VERSION latest
+```
+
+## Package Configuration Files
+
+### package.json (Main Development)
+- Used for development environment
+- Has ALL dependencies including devDependencies
+- `authenticate-pam` is a regular dependency (for development)
+- **DO NOT USE FOR NPM PUBLISHING**
+
+### package.npm.json (NPM Distribution)
+- Used for npm package distribution
+- Has only runtime dependencies
+- `authenticate-pam` is an **optional dependency**
+- **ALWAYS USE THIS FOR NPM PUBLISHING**
+
+## Common Mistakes
+
+1. **Running `npm publish` without arguments**
+   - This triggers rebuild and may use wrong package.json
+   - Always publish pre-built tarball
+
+2. **Not verifying the package before publishing**
+   - Always check that authenticate-pam is optional
+   - Test installation on Linux without PAM headers
+
+3. **Version mismatch**
+   - Keep package.json, package.npm.json, and version.xcconfig in sync
+
+## Testing npm Package
+
+### Quick Docker Test
+```bash
+# Test on Ubuntu without PAM headers
+docker run --rm -it ubuntu:22.04 bash
+apt update && apt install -y nodejs npm
+npm install vibetunnel@VERSION
+# Should succeed without libpam0g-dev
+
+# Test with PAM headers
+apt install -y libpam0g-dev
+npm install vibetunnel@VERSION
+# Should also succeed and include authenticate-pam
+```
+
+### Verify Installation
+```bash
+# Check if vibetunnel works
+npx vibetunnel --version
+
+# On Linux, check if PAM module loaded (optional)
+node -e "try { require('authenticate-pam'); console.log('PAM available'); } catch { console.log('PAM not available'); }"
+```
+
+## Emergency Fixes
+
+If you accidentally published with wrong configuration:
+
+1. **Unpublish if within 72 hours** (not recommended):
+   ```bash
+   npm unpublish vibetunnel@VERSION
+   ```
+
+2. **Publish a fix version**:
+   - Increment version (e.g., 11.3 → 11.4)
+   - Follow correct process above
+   - Deprecate the broken version:
+   ```bash
+   npm deprecate vibetunnel@BROKEN_VERSION "Has installation issues on Linux. Please use VERSION or later."
+   ```
+
+## Release History & Lessons Learned
+
+### Version 11.1 (Good)
+- Used `package.npm.json` correctly
+- `authenticate-pam` was an optional dependency
+- Linux installations worked without PAM headers
+
+### Version 11.2 (Bad - Unpublished)
+- Accidentally used main `package.json`
+- `authenticate-pam` was a required dependency
+- Failed on Linux without libpam0g-dev
+- **Issue**: Wrong package.json configuration
+
+### Version 11.3 (Bad - Unpublished)
+- Also used main `package.json` despite fix attempts
+- Same Linux installation failures
+- **Issue**: npm publish process overwrote correct configuration
+
+### Version 11.4 (Good)
+- Built with explicit `pnpm run build:npm`
+- Published pre-built tarball
+- `authenticate-pam` correctly optional
+- Linux installations work properly
+
+## Summary
+
+The critical lesson: **package.npm.json must be used for npm distribution**, not package.json. The build script supports this, but you must publish the pre-built tarball, not rely on npm's prepublish hooks.
+
+**Golden Rule**: Always build first, verify the package configuration, then publish the tarball. Never use `npm publish` without arguments.
--- a/web/package.json
+++ b/web/package.json
@ -1,6 +1,6 @@
 {
  "name": "vibetunnel",
-  "version": "1.0.0-beta.11",
+  "version": "1.0.0-beta.12",
  "description": "Terminal sharing server with web interface - supports macOS, Linux, and headless environments",
  "main": "dist/server/server.js",
  "bin": {
--- a/web/package.npm.json
+++ b/web/package.npm.json
@ -1,6 +1,6 @@
 {
  "name": "vibetunnel",
-  "version": "1.0.0-beta.11",
+  "version": "1.0.0-beta.12",
  "description": "Terminal sharing server with web interface - supports macOS, Linux, and headless environments",
  "main": "lib/cli.js",
  "bin": {
--- a/web/scripts/postinstall-npm.js
+++ b/web/scripts/postinstall-npm.js
@ -79,7 +79,7 @@ const tryPrebuildInstall = (moduleName, moduleDir) => {
 };

 // Function to manually extract prebuild
-const extractPrebuild = (name, version, targetDir) => {
+const extractPrebuild = (name, version, targetDir, skipDirCheck = false) => {
  const prebuildFile = path.join(__dirname, '..', 'prebuilds', 
    `${name}-v${version}-node-v${nodeABI}-${platform}-${normalizedArch}.tar.gz`);
  
@ -88,13 +88,20 @@ const extractPrebuild = (name, version, targetDir) => {
    return false;
  }

+  // For optional dependencies like authenticate-pam, check if the module exists
+  // If not, extract to a different location
+  let extractDir = targetDir;
+  if (skipDirCheck && name === 'authenticate-pam' && !fs.existsSync(targetDir)) {
+    // Extract to a controlled location since node_modules/authenticate-pam doesn't exist
+    extractDir = path.join(__dirname, '..', 'optional-modules', name);
+  }
+
  // Create the parent directory
-  const buildParentDir = path.join(targetDir);
-  fs.mkdirSync(buildParentDir, { recursive: true });
+  fs.mkdirSync(extractDir, { recursive: true });

  try {
    // Extract directly into the module directory - the tar already contains build/Release structure
-    execSync(`tar -xzf "${prebuildFile}" -C "${buildParentDir}"`, { stdio: 'inherit' });
+    execSync(`tar -xzf "${prebuildFile}" -C "${extractDir}"`, { stdio: 'inherit' });
    console.log(`✓ ${name} prebuilt binary extracted`);
    return true;
  } catch (error) {
@ -145,7 +152,8 @@ const modules = [
    dir: path.join(__dirname, '..', 'node_modules', 'authenticate-pam'),
    build: path.join(__dirname, '..', 'node_modules', 'authenticate-pam', 'build', 'Release', 'authenticate_pam.node'),
    essential: false, // Optional - falls back to other auth methods
-    platforms: ['linux', 'darwin'] // Needed on Linux and macOS
+    platforms: ['linux', 'darwin'], // Needed on Linux and macOS
+    skipDirCheck: true // Don't check if dir exists since it's optional
  }
 ];

@ -162,15 +170,27 @@ for (const module of modules) {

  // Check if module directory exists
  if (!fs.existsSync(module.dir)) {
-    console.warn(`  Warning: ${module.name} directory not found at ${module.dir}`);
-    if (module.essential) {
-      hasErrors = true;
+    if (module.skipDirCheck) {
+      // For optional modules, we'll try to extract the prebuild anyway
+      console.log(`  ${module.name} not installed via npm (optional dependency), will extract prebuild`);
+    } else {
+      console.warn(`  Warning: ${module.name} directory not found at ${module.dir}`);
+      if (module.essential) {
+        hasErrors = true;
+      }
+      continue;
    }
-    continue;
  }

  // Check if already built
-  if (fs.existsSync(module.build)) {
+  // For optional modules, also check the alternative location
+  let buildPath = module.build;
+  if (module.skipDirCheck && module.name === 'authenticate-pam' && !fs.existsSync(module.dir)) {
+    // Check the optional-modules location instead
+    buildPath = path.join(__dirname, '..', 'optional-modules', module.name, 'build', 'Release', 'authenticate_pam.node');
+  }
+  
+  if (fs.existsSync(buildPath)) {
    console.log(`✓ ${module.name} already available`);
    continue;
  }
@ -178,16 +198,18 @@ for (const module of modules) {
  // Try installation methods in order
  let success = false;

-  // Method 1: Try prebuild-install (preferred)
-  success = tryPrebuildInstall(module.name, module.dir);
+  // Method 1: Try prebuild-install (preferred) - skip if directory doesn't exist
+  if (fs.existsSync(module.dir)) {
+    success = tryPrebuildInstall(module.name, module.dir);
+  }

  // Method 2: Manual prebuild extraction
  if (!success) {
-    success = extractPrebuild(module.name, module.version, module.dir);
+    success = extractPrebuild(module.name, module.version, module.dir, module.skipDirCheck);
  }

-  // Method 3: Compile from source
-  if (!success && fs.existsSync(path.join(module.dir, 'binding.gyp'))) {
+  // Method 3: Compile from source (skip if directory doesn't exist)
+  if (!success && fs.existsSync(module.dir) && fs.existsSync(path.join(module.dir, 'binding.gyp'))) {
    success = compileFromSource(module.name, module.dir);
  }

--- a/web/scripts/test-npm-package.dockerfile
+++ b/web/scripts/test-npm-package.dockerfile
@ -0,0 +1,93 @@
+# Test VibeTunnel npm package installation and functionality
+ARG NODE_VERSION=22
+FROM node:${NODE_VERSION}-slim
+
+# Install dependencies for terminal functionality and building native modules
+RUN apt-get update && apt-get install -y \
+    curl \
+    procps \
+    python3 \
+    build-essential \
+    libpam0g-dev \
+    && rm -rf /var/lib/apt/lists/*
+
+# Accept package version as build arg (defaults to latest)
+ARG PACKAGE_VERSION=latest
+
+# Install vibetunnel globally as root
+RUN npm install -g vibetunnel@${PACKAGE_VERSION}
+
+# Create a test user
+RUN useradd -m -s /bin/bash testuser
+
+# Switch to test user
+USER testuser
+WORKDIR /home/testuser
+
+# Create a comprehensive test script
+RUN echo '#!/bin/bash\n\
+set -e\n\
+\n\
+echo "=== VibeTunnel npm Package Test ==="\n\
+echo "Node version: $(node --version)"\n\
+echo "npm version: $(npm --version)"\n\
+echo ""\n\
+\n\
+echo "=== Installation Check ==="\n\
+which vibetunnel && echo "✅ vibetunnel command found" || echo "❌ vibetunnel command not found"\n\
+which vt && echo "✅ vt command found" || echo "❌ vt command not found"\n\
+echo ""\n\
+\n\
+echo "=== Version Check ==="\n\
+vibetunnel --version || echo "Note: Version check failed"\n\
+echo ""\n\
+\n\
+echo "=== Native Module Check ==="\n\
+echo "Checking node-pty installation..."\n\
+ls -la /usr/local/lib/node_modules/vibetunnel/node-pty/build/Release/pty.node 2>/dev/null && \\\n\
+    echo "✅ node-pty native module found" || echo "❌ node-pty native module not found"\n\
+echo ""\n\
+echo "Checking authenticate-pam installation..."\n\
+if [ -f /usr/local/lib/node_modules/vibetunnel/optional-modules/authenticate-pam/build/Release/authenticate_pam.node ]; then\n\
+    echo "✅ authenticate-pam found in optional-modules"\n\
+elif [ -f /usr/local/lib/node_modules/vibetunnel/node_modules/authenticate-pam/build/Release/authenticate_pam.node ]; then\n\
+    echo "✅ authenticate-pam found in node_modules"\n\
+else\n\
+    echo "⚠️  authenticate-pam not found (optional dependency)"\n\
+fi\n\
+echo ""\n\
+\n\
+echo "=== Server Start Test ==="\n\
+echo "Starting VibeTunnel server on port 4021..."\n\
+timeout 10 vibetunnel --port 4021 --no-auth &\n\
+SERVER_PID=$!\n\
+sleep 3\n\
+\n\
+echo "Testing if server is running..."\n\
+if curl -s http://localhost:4021 > /dev/null; then\n\
+    echo "✅ Server is responding on port 4021"\n\
+    \n\
+    # Test API endpoint\n\
+    echo ""\n\
+    echo "=== API Test ==="\n\
+    STATUS=$(curl -s -o /dev/null -w "%{http_code}" http://localhost:4021/api/status)\n\
+    if [ "$STATUS" = "200" ]; then\n\
+        echo "✅ API status endpoint returned 200"\n\
+    else\n\
+        echo "❌ API status endpoint returned $STATUS"\n\
+    fi\n\
+else\n\
+    echo "❌ Server not responding"\n\
+fi\n\
+\n\
+echo ""\n\
+echo "Stopping server..."\n\
+kill $SERVER_PID 2>/dev/null || true\n\
+wait $SERVER_PID 2>/dev/null || true\n\
+\n\
+echo ""\n\
+echo "=== Test Summary ==="\n\
+echo "All tests completed. Check results above for any failures."\n\
+' > test.sh && chmod +x test.sh
+
+CMD ["./test.sh"]
--- a/web/scripts/test-npm-package.sh
+++ b/web/scripts/test-npm-package.sh
@ -0,0 +1,34 @@
+#!/bin/bash
+
+# Test VibeTunnel npm package using Docker
+# Usage: ./test-npm-package.sh [version] [node_version]
+# Examples:
+#   ./test-npm-package.sh                  # Test latest with Node.js 22
+#   ./test-npm-package.sh beta.12          # Test specific version
+#   ./test-npm-package.sh latest 20        # Test with Node.js 20
+#   ./test-npm-package.sh beta.12 24       # Test beta.12 with Node.js 24
+
+PACKAGE_VERSION=${1:-latest}
+NODE_VERSION=${2:-22}
+
+echo "Testing VibeTunnel npm package"
+echo "Package version: $PACKAGE_VERSION"
+echo "Node.js version: $NODE_VERSION"
+echo "================================================"
+
+# Build Docker image
+docker build \
+    --build-arg NODE_VERSION=$NODE_VERSION \
+    --build-arg PACKAGE_VERSION=$PACKAGE_VERSION \
+    -t vibetunnel-npm-test:$PACKAGE_VERSION-node$NODE_VERSION \
+    -f "$(dirname "$0")/test-npm-package.dockerfile" \
+    "$(dirname "$0")"
+
+# Run the test
+docker run --rm vibetunnel-npm-test:$PACKAGE_VERSION-node$NODE_VERSION
+
+# Cleanup
+docker rmi vibetunnel-npm-test:$PACKAGE_VERSION-node$NODE_VERSION 2>/dev/null || true
+
+echo ""
+echo "✅ Test complete!"
--- a/web/src/server/services/authenticate-pam-loader.ts
+++ b/web/src/server/services/authenticate-pam-loader.ts
@ -63,12 +63,46 @@ if (fs.existsSync(seaPamPath) || fs.existsSync(seaNativePamPath)) {
  }
 } else {
  // Development mode - use regular require
+  let loaded = false;
+
+  // First, try the normal require path
  try {
    const pamModule = require('authenticate-pam');
    // Handle both direct export and default export cases
    authenticate = pamModule.authenticate || pamModule.default || pamModule;
+    loaded = true;
  } catch (_error) {
-    // In development mode but module not found
+    // Module not found via normal require
+  }
+
+  // If normal require failed, try the optional-modules location
+  if (!loaded) {
+    const optionalModulePath = path.join(
+      __dirname,
+      '..',
+      '..',
+      '..',
+      'optional-modules',
+      'authenticate-pam',
+      'build',
+      'Release',
+      'authenticate_pam.node'
+    );
+    if (fs.existsSync(optionalModulePath)) {
+      try {
+        const nativeModule = loadNativeModule(optionalModulePath);
+        if (nativeModule.authenticate) {
+          authenticate = nativeModule.authenticate;
+          loaded = true;
+          console.log('Loaded authenticate-pam from optional-modules location');
+        }
+      } catch (_loadError) {
+        // Continue to stub
+      }
+    }
+  }
+
+  if (!loaded) {
    console.warn(
      'Warning: authenticate-pam native module not found. PAM authentication will not work.'
    );