scripts/netbird-install-update/README.md

# Netbird Install/Update Script

A comprehensive bash script for installing, updating, and managing Netbird across Linux systems. Supports local execution, single-host remote deployment, and multi-host fleet management.

## Features

- **Smart Installation**: Automatically detects and handles different Netbird installation methods
- **Package Manager Replacement**: Converts package manager installations to binary installs for better control
- **Connection-Safe Updates**: Background update process survives SSH disconnections during Netbird service restarts
- **Auto-Update Timer**: Systemd timer for daily automatic updates with persistent scheduling
- **Remote Deployment**: Deploy to single hosts or entire fleets via SSH with timeout handling
- **Detailed Summary**: Multi-host deployments show which hosts succeeded/failed and what action was performed
- **SELinux Compatible**: Automatically fixes SELinux contexts on RHEL/Fedora systems
- **No Artifacts**: Clean execution with no leftover files or logs

## Requirements

- Linux operating system
- Root/sudo access
- curl
- SSH client (for remote deployment)
- Internet connection to download Netbird

## Installation

Make the script executable:

```bash
chmod +x netbird-install-update.sh
```

## Usage

### Local Execution

Install or update Netbird on the local machine:

```bash
./netbird-install-update.sh
```

Install with automatic daily updates:

```bash
./netbird-install-update.sh --timer
```

Install with custom update schedule:

```bash
./netbird-install-update.sh --timer --time "03:00"
```

### Single Host Remote Deployment

Deploy to a remote host via SSH:

```bash
./netbird-install-update.sh --ssh user@hostname
```

Deploy with sudo password (non-interactive):

```bash
./netbird-install-update.sh --ssh user@hostname --password "your_sudo_password"
```

Deploy with auto-update timer enabled:

```bash
./netbird-install-update.sh --ssh user@hostname --timer
```

Deploy with custom update schedule:

```bash
./netbird-install-update.sh --ssh user@hostname --timer --time "03:00"
```

### Multi-Host Fleet Deployment

Deploy to multiple hosts using a hosts file:

```bash
./netbird-install-update.sh --file hosts.txt
```

Deploy with default sudo password for all hosts:

```bash
./netbird-install-update.sh --file hosts.txt --password "your_sudo_password"
```

Deploy with auto-update timer to all hosts:

```bash
./netbird-install-update.sh --file hosts.txt --timer
```

Deploy with custom update schedule to all hosts:

```bash
./netbird-install-update.sh --file hosts.txt --timer --time "03:00"
```

#### Hosts File Format

Create a text file with one hostname or IP address per line. Empty lines and comments (starting with `#`) are ignored.

**Optional: Include sudo password**

You can optionally include the sudo password after the hostname (separated by space). Host-specific passwords override the default password provided via `--password`.

```bash
# Production servers (with host-specific sudo passwords)
prod-web-01.example.com MySudoPassword123
prod-web-02.example.com MySudoPassword456
prod-db-01.example.com MySudoPassword789

# Development servers (will use default password from --password flag)
dev-app-01.example.com
dev-app-02.example.com

# IP addresses also work
192.168.1.100
10.0.0.50 MyPassword
```

**Password Precedence**:
1. Host-specific password in hosts file (highest priority)
2. Default password from `--password` flag
3. Interactive sudo prompt (if no password provided)

**Security Warning**: Storing passwords in plain text is not recommended for production environments. Consider using SSH key authentication with passwordless sudo instead.

**Requirements**:
- If a password is provided (via hosts file or `--password` flag), `sshpass` must be installed on the local machine
- If no password is provided, you'll be prompted to enter the sudo password interactively for each host

See `hosts.example` for a template.

## Command-Line Options

| Option | Description |
|--------|-------------|
| `--ssh <host>` | Deploy to a single remote host via SSH |
| `--file <path>` | Deploy to multiple hosts listed in a file |
| `--hosts-file <path>` | Alias for `--file` |
| `--timer` | Install systemd timer for auto-updates |
| `--time <schedule>` | Set update schedule (default: "daily") |
| `--password, -p <password>` | Default sudo password for remote hosts |

The `--time` option accepts systemd calendar event format. Examples:
- `daily` - Run once per day at midnight (default)
- `03:00` - Run at 3:00 AM daily
- `Mon,Fri 02:30` - Run at 2:30 AM on Monday and Friday
- `hourly` - Run every hour
- `weekly` - Run once per week

The `--password` option provides a default sudo password for remote deployments. This can be overridden per-host in the hosts file.

## How It Works

### Installation Detection

The script detects three installation states:

1. **Not Installed**: Netbird binary not found in PATH
2. **Package Manager Install**: Installed via apt, yum, dnf, zypper, pacman, or rpm-ostree
3. **Binary Install**: Installed via the official curl-based installer

Detection is performed by:
- Checking `/etc/netbird/install.conf` (authoritative source)
- Querying package managers (dpkg, rpm)
- Falling back to binary install detection

### Installation Flow

**Fresh Installation (Not Installed)**:
1. Downloads official Netbird install script
2. Runs binary-only installation (no GUI, no package manager)
3. Ensures proper ownership and permissions
4. Fixes SELinux contexts if applicable

**Package Manager Replacement**:
1. Downloads official Netbird install script
2. Schedules background replacement (3-second delay)
3. Stops Netbird service
4. Removes package manager installation
5. Runs binary installation
6. Ensures proper ownership, permissions, and SELinux contexts

**Binary Update**:
1. Compares current version with latest release
2. If outdated, schedules background update (3-second delay)
3. Runs official update process
4. Ensures proper ownership, permissions, and SELinux contexts

### Connection-Safe Updates

When updating over SSH, the script uses a detached background process to avoid disconnection issues:

```bash
setsid bash -c 'sleep 3 && [update commands]' </dev/null >/var/log/netbird-update.log 2>&1 &
```

The 3-second delay allows the SSH session to complete cleanly before the Netbird service is stopped and restarted. The update continues even if your connection drops.

### Auto-Update Timer

When `--timer` is specified, the script creates a systemd timer that:

- Runs on a configurable schedule (default: daily at midnight with random 0-10 minute delay)
- Uses `Persistent=true` to catch up on missed runs after system boot
- Logs output to `/var/log/netbird-update.log`
- Automatically updates existing timers if already installed

The schedule can be customized with the `--time` option. Examples:
- `--time "daily"` - Run once per day at midnight (default)
- `--time "03:00"` - Run at 3:00 AM daily
- `--time "Mon,Fri 02:30"` - Run at 2:30 AM on Monday and Friday
- `--time "hourly"` - Run every hour
- `--time "weekly"` - Run once per week

The timer consists of two systemd units:

**netbird-update.service**:
```ini
[Service]
Type=oneshot
ExecStart=/bin/bash -c 'curl -fsSL https://pkgs.netbird.io/install.sh | UPDATE_NETBIRD=true sh && chown root:root /usr/bin/netbird 2>/dev/null || true && chmod +x /usr/bin/netbird 2>/dev/null || true && restorecon -v /usr/bin/netbird 2>/dev/null || true'
```

**netbird-update.timer**:
```ini
[Timer]
OnCalendar=daily
Persistent=true
RandomizedDelaySec=600

[Install]
WantedBy=timers.target
```

## Logging

All background operations log to `/var/log/netbird-update.log`:

- Package manager replacement operations
- In-place updates
- Systemd timer executions

Check the log to monitor update progress or troubleshoot issues:

```bash
sudo tail -f /var/log/netbird-update.log
```

## Troubleshooting

### Netbird Service Fails to Start

If the Netbird service fails with "Permission denied" errors:

```bash
# Check binary permissions
ls -la /usr/bin/netbird

# Fix ownership and permissions
sudo chown root:root /usr/bin/netbird
sudo chmod +x /usr/bin/netbird

# Fix SELinux context (RHEL/Fedora)
sudo restorecon -v /usr/bin/netbird

# Restart service
sudo systemctl restart netbird
```

### Check Update Status

```bash
# Check current version
netbird version

# Check service status
sudo systemctl status netbird

# View update logs
sudo cat /var/log/netbird-update.log

# Check timer status (if installed)
systemctl status netbird-update.timer
```

### Manual Update

If you need to manually trigger an update:

```bash
./netbird-install-update.sh
```

Or use the official Netbird update method:

```bash
curl -fsSL https://pkgs.netbird.io/install.sh | sudo UPDATE_NETBIRD=true sh
```

### Timer Management

```bash
# Check timer status
systemctl status netbird-update.timer

# View timer schedule
systemctl list-timers netbird-update.timer

# Manually trigger timer
sudo systemctl start netbird-update.service

# Disable timer
sudo systemctl disable --now netbird-update.timer

# Remove timer completely
sudo systemctl disable --now netbird-update.timer
sudo rm /etc/systemd/system/netbird-update.service
sudo rm /etc/systemd/system/netbird-update.timer
sudo systemctl daemon-reload
```

## Why Binary Install?

The script converts package manager installations to binary installs because:

1. **Better Control**: Direct binary management allows precise version control
2. **Simpler Updates**: Binary updates don't require package manager configuration
3. **Consistency**: Same installation method across all Linux distributions
4. **Official Support**: Binary install is the officially recommended method by Netbird

## Remote Deployment Details

When deploying remotely:

1. Script copies itself to the remote host via `scp`
2. Executes remotely with `sudo` via `ssh -t` (terminal allocation for sudo password prompt)
3. Passes through additional flags (like `--timer`)
4. Cleans up the remote copy after execution

The remote deployment continues even if the local script is interrupted.

**Important Requirements**:
- **SSH Authentication**: Remote deployment requires SSH key authentication. Password-based SSH authentication is not supported.
- **Sudo Access**: The script will prompt for the sudo password on each remote host. For fully automated deployments, configure passwordless sudo on the remote hosts (e.g., add a sudoers rule like `username ALL=(ALL) NOPASSWD: /bin/bash`).

### SSH Timeout and Error Handling

The script includes a 30-second timeout for SSH connections to prevent hanging on unreachable hosts. If a host is unreachable or the connection times out, the script will:

- Mark the host as failed in the deployment summary
- Continue to the next host in the list
- Report the failure at the end

### Deployment Summary

After multi-host deployment, the script provides a detailed summary showing:

- **Total successful and failed deployments**
- **List of successful hosts** with the action performed:
  - `fresh install` - Netbird was not installed and has been installed
  - `pkg replacement` - Netbird was installed via package manager and converted to binary install
  - `updated` - Netbird was updated to a newer version
  - `up to date` - Netbird was already at the latest version
- **List of failed hosts** that could not be reached or had errors

Example output:

```
╔═══════════════════════════════════════════════════════════════╗
║ ✓ Deployment Summary
╠═══════════════════════════════════════════════════════════════╣
║  ✓ Successful: 3
║  ✗ Failed:     1
╚═══════════════════════════════════════════════════════════════╝

Successful Hosts:
  ✓ host-01.example.com (updated)
  ✓ host-02.example.com (fresh install)
  ✓ host-03.example.com (up to date)

Failed Hosts:
  ✗ host-04.example.com
```

## Exit Codes

- `0`: Success
- `1`: Error (file not found, SSH failure, etc.)

## Compatibility

### Tested Package Managers

- apt (Debian/Ubuntu)
- yum (RHEL/CentOS 7)
- dnf (Fedora/RHEL 8+/CentOS 8+)
- zypper (openSUSE)
- pacman (Arch Linux)
- rpm-ostree (Fedora Silverblue)

### SELinux Support

Automatically handles SELinux contexts on systems with SELinux enabled (RHEL, Fedora, CentOS). The `restorecon` command is safely ignored on systems without SELinux.

## Security Considerations

- Script requires root/sudo access
- Downloads official Netbird installer from `https://pkgs.netbird.io/install.sh`
- All operations are logged to `/var/log/netbird-update.log`
- No credentials are stored or transmitted
- Temporary files are cleaned up automatically

## Limitations

- Linux only (no macOS or Windows support)
- Requires SSH key authentication for remote deployment (password authentication is not supported)
- Background updates cannot be cancelled once started

## License

This script is provided as-is for managing Netbird installations. Netbird itself is subject to its own license terms.

## Contributing

Contributions are welcome! Please ensure:
- Script syntax is valid (`bash -n script.sh`)
- Changes are tested on multiple Linux distributions
- SELinux compatibility is maintained
- No artifacts are left behind after execution

## Support

For Netbird-specific issues, refer to the [official Netbird documentation](https://docs.netbird.io).

For script-specific issues, check the troubleshooting section or review the logs at `/var/log/netbird-update.log`.