Fix npm package to handle authenticate-pam as optional dependency (#390)

CHANGELOG.md

@@ -1,5 +1,35 @@
 # Changelog
 
+## [1.0.0-beta.12] - 2025-01-17
+
+### 🚀 Major Improvements
+
+#### **NPM Package Fixes**
+- Fixed critical npm installation issues on Linux systems
+- `authenticate-pam` is now correctly an optional dependency
+- Linux users can install without PAM development headers
+- Proper package.npm.json configuration is now used for npm releases
+
+### 🐛 Bug Fixes
+
+- Fixed PAM module and npm_config_prefix issues on Ubuntu (#380)
+- Fixed SSH key generation errors on non-localhost HTTP connections (#384)
+- Fixed npm package documentation and unified READMEs (#381)
+- Removed verbose output from vt script (#383)
+
+### 📚 Documentation
+
+- Added comprehensive npm publishing guide (npm.md) with installation instructions
+- Added Discord member count badge with correct server ID
+- Added Node.js version, Discord, and Homebrew installation badges
+- Improved npm package installation documentation
+- Added Docker test script for npm package validation
+
+### 🧹 Maintenance
+
+- Cleaned up old npm package versions (11.2, 11.3) due to installation issues
+- Added test-npm-docker.sh script for automated npm package testing
+
 ## [1.0.0-beta.11] - 2025-01-16
 
 #### **Better Settings Organization**

mac/VibeTunnel/version.xcconfig

@@ -1,8 +1,8 @@
 // VibeTunnel Version Configuration
 // This file contains the version and build number for the app
 
-MARKETING_VERSION = 1.0.0-beta.11
+MARKETING_VERSION = 1.0.0-beta.12
-CURRENT_PROJECT_VERSION = 200
+CURRENT_PROJECT_VERSION = 201
 
 // Domain and GitHub configuration
 APP_DOMAIN = vibetunnel.sh

web/.gitignore

@@ -156,3 +156,9 @@ test.cast
 
 # Temporary build directories
 temp-spawn-helper/
+
+# Package directory (generated by build)
+package/
+
+# Docker test files (moved to scripts/)
+test-vibetunnel*.dockerfile

web/npm.md

@@ -0,0 +1,305 @@
+# NPM Publishing Guide for VibeTunnel
+
+## Installation Guide
+
+### Installing VibeTunnel from NPM
+
+VibeTunnel is published as an npm package that works on macOS and Linux. The package includes prebuilt binaries for common platforms to avoid compilation.
+
+#### Basic Installation
+
+```bash
+# Install globally (recommended)
+npm install -g vibetunnel
+
+# Or install locally in a project
+npm install vibetunnel
+```
+
+#### Platform-Specific Notes
+
+**macOS**: 
+- Works out of the box
+- PAM authentication supported natively
+
+**Linux**:
+- Works without additional dependencies
+- PAM authentication is optional - installs only if PAM headers are available
+- If you need PAM authentication, install development headers first:
+  ```bash
+  # Ubuntu/Debian
+  sudo apt-get install libpam0g-dev
+  
+  # RHEL/CentOS/Fedora
+  sudo yum install pam-devel
+  ```
+
+#### Verifying Installation
+
+```bash
+# Check version
+vibetunnel --version
+
+# Run the server
+vibetunnel
+
+# The server will start on http://localhost:4020
+```
+
+#### Docker Installation
+
+For containerized environments:
+
+```dockerfile
+FROM node:20-slim
+
+# Optional: Install PAM headers for authentication support
+# RUN apt-get update && apt-get install -y libpam0g-dev
+
+# Install VibeTunnel
+RUN npm install -g vibetunnel
+
+# Expose the default port
+EXPOSE 4020
+
+# Run VibeTunnel
+CMD ["vibetunnel"]
+```
+
+#### Troubleshooting Installation
+
+1. **"Cannot find module '../build/Release/pty.node'"**
+   - The package includes prebuilds, this shouldn't happen
+   - Try reinstalling: `npm uninstall -g vibetunnel && npm install -g vibetunnel`
+
+2. **PAM authentication not working on Linux**
+   - Install PAM headers: `sudo apt-get install libpam0g-dev`
+   - Reinstall VibeTunnel to compile the PAM module
+
+3. **Permission errors during installation**
+   - Use a Node.js version manager (nvm, fnm) instead of system Node.js
+   - Or fix npm permissions: https://docs.npmjs.com/resolving-eacces-permissions-errors
+
+## Quick Release Checklist
+
+1. **Update versions** in all 3 files:
+   - `package.json`
+   - `package.npm.json` 
+   - `../mac/VibeTunnel/version.xcconfig`
+
+2. **Build**: `pnpm run build:npm`
+
+3. **Verify**: 
+   ```bash
+   tar -xf vibetunnel-*.tgz package/package.json
+   grep optionalDependencies package/package.json  # Must show authenticate-pam
+   rm -rf package/
+   ```
+
+4. **Publish tarball**:
+   ```bash
+   npm publish vibetunnel-*.tgz --tag beta
+   npm dist-tag add vibetunnel@VERSION latest
+   ```
+
+⚠️ **NEVER** use `npm publish` without the tarball filename!
+
+## Critical Issue History: Wrong package.json Used in Releases
+
+### The Problem
+
+We've repeatedly published npm packages with the wrong configuration:
+- **Version 11.2**: Used main `package.json` instead of `package.npm.json`
+- **Version 11.3**: Also used main `package.json` despite having the correct `package.npm.json`
+- **Both versions had to be unpublished** due to Linux installation failures
+
+This causes installation failures on Linux systems because:
+- Main `package.json` has `authenticate-pam` as a **regular dependency**
+- `package.npm.json` has `authenticate-pam` as an **optional dependency**
+
+When `authenticate-pam` is a regular dependency, npm fails the entire installation if PAM headers (libpam0g-dev) aren't available.
+
+### Root Cause
+
+The build script (`scripts/build-npm.js`) checks for `package.npm.json` and uses it if available, BUT:
+- During `npm publish`, npm runs the `prepublishOnly` script which triggers `build:npm`
+- This rebuilds the package, potentially overwriting the correct configuration
+- The timing and execution context can cause the wrong package.json to be used
+
+### The Solution
+
+**NEVER use `npm publish` directly!** Instead:
+
+1. Build the package explicitly:
+   ```bash
+   pnpm run build:npm
+   ```
+
+2. Verify the package has the correct configuration:
+   ```bash
+   # Extract and check package.json from the tarball
+   tar -xf vibetunnel-*.tgz package/package.json
+   cat package/package.json | grep -A5 -B5 authenticate-pam
+   ```
+
+3. Ensure `authenticate-pam` is under `optionalDependencies`:
+   ```json
+   "optionalDependencies": {
+     "authenticate-pam": "^1.0.5"
+   }
+   ```
+
+4. Publish the pre-built tarball:
+   ```bash
+   npm publish vibetunnel-*.tgz --tag beta
+   npm dist-tag add vibetunnel@VERSION latest  # if needed
+   ```
+
+## Correct Release Process
+
+### 1. Update Version Numbers
+```bash
+# Update all three version files - MUST keep in sync!
+vim package.json          # Update version
+vim package.npm.json      # Update version to match
+vim ../mac/VibeTunnel/version.xcconfig  # Update MARKETING_VERSION
+```
+
+### 2. Build the Package
+```bash
+pnpm run build:npm
+```
+
+### 3. Verify the Build
+```bash
+# Check the tarball exists (look in parent directory!)
+ls -la *.tgz
+
+# Extract and verify authenticate-pam is optional
+tar -xf vibetunnel-*.tgz package/package.json
+cat package/package.json | grep -A5 -B5 authenticate-pam
+
+# Should show:
+#   "optionalDependencies": {
+#     "authenticate-pam": "^1.0.5"
+#   }
+
+# Clean up
+rm -rf package/
+```
+
+### 4. Test Installation Locally
+```bash
+# Test on a system without PAM headers
+docker run --rm -it node:20 bash
+npm install /path/to/vibetunnel-*.tgz
+# Should succeed even without libpam0g-dev
+```
+
+### 5. Publish
+```bash
+# Publish the pre-built tarball with beta tag
+npm publish vibetunnel-*.tgz --tag beta
+
+# Also tag as latest if stable
+npm dist-tag add vibetunnel@VERSION latest
+```
+
+## Package Configuration Files
+
+### package.json (Main Development)
+- Used for development environment
+- Has ALL dependencies including devDependencies
+- `authenticate-pam` is a regular dependency (for development)
+- **DO NOT USE FOR NPM PUBLISHING**
+
+### package.npm.json (NPM Distribution)
+- Used for npm package distribution
+- Has only runtime dependencies
+- `authenticate-pam` is an **optional dependency**
+- **ALWAYS USE THIS FOR NPM PUBLISHING**
+
+## Common Mistakes
+
+1. **Running `npm publish` without arguments**
+   - This triggers rebuild and may use wrong package.json
+   - Always publish pre-built tarball
+
+2. **Not verifying the package before publishing**
+   - Always check that authenticate-pam is optional
+   - Test installation on Linux without PAM headers
+
+3. **Version mismatch**
+   - Keep package.json, package.npm.json, and version.xcconfig in sync
+
+## Testing npm Package
+
+### Quick Docker Test
+```bash
+# Test on Ubuntu without PAM headers
+docker run --rm -it ubuntu:22.04 bash
+apt update && apt install -y nodejs npm
+npm install vibetunnel@VERSION
+# Should succeed without libpam0g-dev
+
+# Test with PAM headers
+apt install -y libpam0g-dev
+npm install vibetunnel@VERSION
+# Should also succeed and include authenticate-pam
+```
+
+### Verify Installation
+```bash
+# Check if vibetunnel works
+npx vibetunnel --version
+
+# On Linux, check if PAM module loaded (optional)
+node -e "try { require('authenticate-pam'); console.log('PAM available'); } catch { console.log('PAM not available'); }"
+```
+
+## Emergency Fixes
+
+If you accidentally published with wrong configuration:
+
+1. **Unpublish if within 72 hours** (not recommended):
+   ```bash
+   npm unpublish vibetunnel@VERSION
+   ```
+
+2. **Publish a fix version**:
+   - Increment version (e.g., 11.3 → 11.4)
+   - Follow correct process above
+   - Deprecate the broken version:
+   ```bash
+   npm deprecate vibetunnel@BROKEN_VERSION "Has installation issues on Linux. Please use VERSION or later."
+   ```
+
+## Release History & Lessons Learned
+
+### Version 11.1 (Good)
+- Used `package.npm.json` correctly
+- `authenticate-pam` was an optional dependency
+- Linux installations worked without PAM headers
+
+### Version 11.2 (Bad - Unpublished)
+- Accidentally used main `package.json`
+- `authenticate-pam` was a required dependency
+- Failed on Linux without libpam0g-dev
+- **Issue**: Wrong package.json configuration
+
+### Version 11.3 (Bad - Unpublished)
+- Also used main `package.json` despite fix attempts
+- Same Linux installation failures
+- **Issue**: npm publish process overwrote correct configuration
+
+### Version 11.4 (Good)
+- Built with explicit `pnpm run build:npm`
+- Published pre-built tarball
+- `authenticate-pam` correctly optional
+- Linux installations work properly
+
+## Summary
+
+The critical lesson: **package.npm.json must be used for npm distribution**, not package.json. The build script supports this, but you must publish the pre-built tarball, not rely on npm's prepublish hooks.
+
+**Golden Rule**: Always build first, verify the package configuration, then publish the tarball. Never use `npm publish` without arguments.

web/package.json

@@ -1,6 +1,6 @@
 {
   "name": "vibetunnel",
-  "version": "1.0.0-beta.11",
+  "version": "1.0.0-beta.12",
   "description": "Terminal sharing server with web interface - supports macOS, Linux, and headless environments",
   "main": "dist/server/server.js",
   "bin": {

web/package.npm.json

@@ -1,6 +1,6 @@
 {
   "name": "vibetunnel",
-  "version": "1.0.0-beta.11",
+  "version": "1.0.0-beta.12",
   "description": "Terminal sharing server with web interface - supports macOS, Linux, and headless environments",
   "main": "lib/cli.js",
   "bin": {

web/scripts/postinstall-npm.js

@@ -79,7 +79,7 @@ const tryPrebuildInstall = (moduleName, moduleDir) => {
 };
 
 // Function to manually extract prebuild
-const extractPrebuild = (name, version, targetDir) => {
+const extractPrebuild = (name, version, targetDir, skipDirCheck = false) => {
   const prebuildFile = path.join(__dirname, '..', 'prebuilds', 
     `${name}-v${version}-node-v${nodeABI}-${platform}-${normalizedArch}.tar.gz`);
   
@@ -88,13 +88,20 @@ const extractPrebuild = (name, version, targetDir) => {
     return false;
   }
 
+  // For optional dependencies like authenticate-pam, check if the module exists
+  // If not, extract to a different location
+  let extractDir = targetDir;
+  if (skipDirCheck && name === 'authenticate-pam' && !fs.existsSync(targetDir)) {
+    // Extract to a controlled location since node_modules/authenticate-pam doesn't exist
+    extractDir = path.join(__dirname, '..', 'optional-modules', name);
+  }
+
   // Create the parent directory
-  const buildParentDir = path.join(targetDir);
+  fs.mkdirSync(extractDir, { recursive: true });
-  fs.mkdirSync(buildParentDir, { recursive: true });
 
   try {
     // Extract directly into the module directory - the tar already contains build/Release structure
-    execSync(`tar -xzf "${prebuildFile}" -C "${buildParentDir}"`, { stdio: 'inherit' });
+    execSync(`tar -xzf "${prebuildFile}" -C "${extractDir}"`, { stdio: 'inherit' });
     console.log(`✓ ${name} prebuilt binary extracted`);
     return true;
   } catch (error) {
@@ -145,7 +152,8 @@ const modules = [
     dir: path.join(__dirname, '..', 'node_modules', 'authenticate-pam'),
     build: path.join(__dirname, '..', 'node_modules', 'authenticate-pam', 'build', 'Release', 'authenticate_pam.node'),
     essential: false, // Optional - falls back to other auth methods
-    platforms: ['linux', 'darwin'] // Needed on Linux and macOS
+    platforms: ['linux', 'darwin'], // Needed on Linux and macOS
+    skipDirCheck: true // Don't check if dir exists since it's optional
   }
 ];
 
@@ -162,15 +170,27 @@ for (const module of modules) {
 
   // Check if module directory exists
   if (!fs.existsSync(module.dir)) {
+    if (module.skipDirCheck) {
+      // For optional modules, we'll try to extract the prebuild anyway
+      console.log(`  ${module.name} not installed via npm (optional dependency), will extract prebuild`);
+    } else {
       console.warn(`  Warning: ${module.name} directory not found at ${module.dir}`);
       if (module.essential) {
         hasErrors = true;
       }
       continue;
     }
+  }
 
   // Check if already built
-  if (fs.existsSync(module.build)) {
+  // For optional modules, also check the alternative location
+  let buildPath = module.build;
+  if (module.skipDirCheck && module.name === 'authenticate-pam' && !fs.existsSync(module.dir)) {
+    // Check the optional-modules location instead
+    buildPath = path.join(__dirname, '..', 'optional-modules', module.name, 'build', 'Release', 'authenticate_pam.node');
+  }
+  
+  if (fs.existsSync(buildPath)) {
     console.log(`✓ ${module.name} already available`);
     continue;
   }
@@ -178,16 +198,18 @@ for (const module of modules) {
   // Try installation methods in order
   let success = false;
 
-  // Method 1: Try prebuild-install (preferred)
+  // Method 1: Try prebuild-install (preferred) - skip if directory doesn't exist
+  if (fs.existsSync(module.dir)) {
     success = tryPrebuildInstall(module.name, module.dir);
+  }
 
   // Method 2: Manual prebuild extraction
   if (!success) {
-    success = extractPrebuild(module.name, module.version, module.dir);
+    success = extractPrebuild(module.name, module.version, module.dir, module.skipDirCheck);
   }
 
-  // Method 3: Compile from source
+  // Method 3: Compile from source (skip if directory doesn't exist)
-  if (!success && fs.existsSync(path.join(module.dir, 'binding.gyp'))) {
+  if (!success && fs.existsSync(module.dir) && fs.existsSync(path.join(module.dir, 'binding.gyp'))) {
     success = compileFromSource(module.name, module.dir);
   }
 

web/scripts/test-npm-package.dockerfile

@@ -0,0 +1,93 @@
+# Test VibeTunnel npm package installation and functionality
+ARG NODE_VERSION=22
+FROM node:${NODE_VERSION}-slim
+
+# Install dependencies for terminal functionality and building native modules
+RUN apt-get update && apt-get install -y \
+    curl \
+    procps \
+    python3 \
+    build-essential \
+    libpam0g-dev \
+    && rm -rf /var/lib/apt/lists/*
+
+# Accept package version as build arg (defaults to latest)
+ARG PACKAGE_VERSION=latest
+
+# Install vibetunnel globally as root
+RUN npm install -g vibetunnel@${PACKAGE_VERSION}
+
+# Create a test user
+RUN useradd -m -s /bin/bash testuser
+
+# Switch to test user
+USER testuser
+WORKDIR /home/testuser
+
+# Create a comprehensive test script
+RUN echo '#!/bin/bash\n\
+set -e\n\
+\n\
+echo "=== VibeTunnel npm Package Test ==="\n\
+echo "Node version: $(node --version)"\n\
+echo "npm version: $(npm --version)"\n\
+echo ""\n\
+\n\
+echo "=== Installation Check ==="\n\
+which vibetunnel && echo "✅ vibetunnel command found" || echo "❌ vibetunnel command not found"\n\
+which vt && echo "✅ vt command found" || echo "❌ vt command not found"\n\
+echo ""\n\
+\n\
+echo "=== Version Check ==="\n\
+vibetunnel --version || echo "Note: Version check failed"\n\
+echo ""\n\
+\n\
+echo "=== Native Module Check ==="\n\
+echo "Checking node-pty installation..."\n\
+ls -la /usr/local/lib/node_modules/vibetunnel/node-pty/build/Release/pty.node 2>/dev/null && \\\n\
+    echo "✅ node-pty native module found" || echo "❌ node-pty native module not found"\n\
+echo ""\n\
+echo "Checking authenticate-pam installation..."\n\
+if [ -f /usr/local/lib/node_modules/vibetunnel/optional-modules/authenticate-pam/build/Release/authenticate_pam.node ]; then\n\
+    echo "✅ authenticate-pam found in optional-modules"\n\
+elif [ -f /usr/local/lib/node_modules/vibetunnel/node_modules/authenticate-pam/build/Release/authenticate_pam.node ]; then\n\
+    echo "✅ authenticate-pam found in node_modules"\n\
+else\n\
+    echo "⚠️  authenticate-pam not found (optional dependency)"\n\
+fi\n\
+echo ""\n\
+\n\
+echo "=== Server Start Test ==="\n\
+echo "Starting VibeTunnel server on port 4021..."\n\
+timeout 10 vibetunnel --port 4021 --no-auth &\n\
+SERVER_PID=$!\n\
+sleep 3\n\
+\n\
+echo "Testing if server is running..."\n\
+if curl -s http://localhost:4021 > /dev/null; then\n\
+    echo "✅ Server is responding on port 4021"\n\
+    \n\
+    # Test API endpoint\n\
+    echo ""\n\
+    echo "=== API Test ==="\n\
+    STATUS=$(curl -s -o /dev/null -w "%{http_code}" http://localhost:4021/api/status)\n\
+    if [ "$STATUS" = "200" ]; then\n\
+        echo "✅ API status endpoint returned 200"\n\
+    else\n\
+        echo "❌ API status endpoint returned $STATUS"\n\
+    fi\n\
+else\n\
+    echo "❌ Server not responding"\n\
+fi\n\
+\n\
+echo ""\n\
+echo "Stopping server..."\n\
+kill $SERVER_PID 2>/dev/null || true\n\
+wait $SERVER_PID 2>/dev/null || true\n\
+\n\
+echo ""\n\
+echo "=== Test Summary ==="\n\
+echo "All tests completed. Check results above for any failures."\n\
+' > test.sh && chmod +x test.sh
+
+CMD ["./test.sh"]

web/scripts/test-npm-package.sh

@@ -0,0 +1,34 @@
+#!/bin/bash
+
+# Test VibeTunnel npm package using Docker
+# Usage: ./test-npm-package.sh [version] [node_version]
+# Examples:
+#   ./test-npm-package.sh                  # Test latest with Node.js 22
+#   ./test-npm-package.sh beta.12          # Test specific version
+#   ./test-npm-package.sh latest 20        # Test with Node.js 20
+#   ./test-npm-package.sh beta.12 24       # Test beta.12 with Node.js 24
+
+PACKAGE_VERSION=${1:-latest}
+NODE_VERSION=${2:-22}
+
+echo "Testing VibeTunnel npm package"
+echo "Package version: $PACKAGE_VERSION"
+echo "Node.js version: $NODE_VERSION"
+echo "================================================"
+
+# Build Docker image
+docker build \
+    --build-arg NODE_VERSION=$NODE_VERSION \
+    --build-arg PACKAGE_VERSION=$PACKAGE_VERSION \
+    -t vibetunnel-npm-test:$PACKAGE_VERSION-node$NODE_VERSION \
+    -f "$(dirname "$0")/test-npm-package.dockerfile" \
+    "$(dirname "$0")"
+
+# Run the test
+docker run --rm vibetunnel-npm-test:$PACKAGE_VERSION-node$NODE_VERSION
+
+# Cleanup
+docker rmi vibetunnel-npm-test:$PACKAGE_VERSION-node$NODE_VERSION 2>/dev/null || true
+
+echo ""
+echo "✅ Test complete!"

web/src/server/services/authenticate-pam-loader.ts

@@ -63,12 +63,46 @@ if (fs.existsSync(seaPamPath) || fs.existsSync(seaNativePamPath)) {
   }
 } else {
   // Development mode - use regular require
+  let loaded = false;
+
+  // First, try the normal require path
   try {
     const pamModule = require('authenticate-pam');
     // Handle both direct export and default export cases
     authenticate = pamModule.authenticate || pamModule.default || pamModule;
+    loaded = true;
   } catch (_error) {
-    // In development mode but module not found
+    // Module not found via normal require
+  }
+
+  // If normal require failed, try the optional-modules location
+  if (!loaded) {
+    const optionalModulePath = path.join(
+      __dirname,
+      '..',
+      '..',
+      '..',
+      'optional-modules',
+      'authenticate-pam',
+      'build',
+      'Release',
+      'authenticate_pam.node'
+    );
+    if (fs.existsSync(optionalModulePath)) {
+      try {
+        const nativeModule = loadNativeModule(optionalModulePath);
+        if (nativeModule.authenticate) {
+          authenticate = nativeModule.authenticate;
+          loaded = true;
+          console.log('Loaded authenticate-pam from optional-modules location');
+        }
+      } catch (_loadError) {
+        // Continue to stub
+      }
+    }
+  }
+
+  if (!loaded) {
     console.warn(
       'Warning: authenticate-pam native module not found. PAM authentication will not work.'
     );