Compare commits

..

35 Commits

Author SHA1 Message Date
dependabot[bot]
4bc0211920 build(deps-dev): Bump prettier from 3.8.0 to 3.8.1
Bumps [prettier](https://github.com/prettier/prettier) from 3.8.0 to 3.8.1.
- [Release notes](https://github.com/prettier/prettier/releases)
- [Changelog](https://github.com/prettier/prettier/blob/main/CHANGELOG.md)
- [Commits](https://github.com/prettier/prettier/compare/3.8.0...3.8.1)

---
updated-dependencies:
- dependency-name: prettier
  dependency-version: 3.8.1
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-01-30 19:56:25 +00:00
github-actions[bot]
351ba09f4e chore: bump version to 0.5.6 (VERSION + package.json) (#482)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2026-01-29 15:08:41 +00:00
Michel Roegl-Brunner
580986abfa Merge pull request #481 from community-scripts/fix/466
fix: resolve server from DB for SSH when client sends no ssh_key_path (fixes #466)
2026-01-29 16:03:08 +01:00
Michel Rögl-Brunner
e1d270d52c fix: resolve server from DB for SSH when client sends no ssh_key_path (fixes #466)
- Add resolveServerForSSH() to load full server (including ssh_key_path) from DB
  when WebSocket server has id but key auth without valid ssh_key_path
- Call resolver in handleMessage for all start flows (clone, backup, update,
  shell, script) so Shell and Update over SSH work with key auth
- Extend ServerInfo typedef with auth_type, ssh_key_path for TypeScript
2026-01-29 15:59:58 +01:00
Michel Roegl-Brunner
20dbcae42a Merge pull request #480 from community-scripts/fix/405
fix: delete local JSON files when removed from remote repo (fixes #405)
2026-01-29 15:47:31 +01:00
Michel Rögl-Brunner
8e8c724392 fix: delete local JSON files when removed from remote repo (fixes #405)
- Add deleteLocalFilesRemovedFromRepo() to remove local script JSON files
  that belong to the synced repo but are no longer in the remote list
- Call it in syncJsonFilesForRepo() before find/sync so stale scripts
  no longer appear and download attempts don't 404
- Extend sync return types with deletedFiles; aggregate in syncJsonFiles()
  and include removed count in success message
2026-01-29 15:44:45 +01:00
Michel Roegl-Brunner
201b33ec84 Merge pull request #479 from community-scripts/fix/464
fix: use node-specific Proxmox config paths for VM vs LXC (fixes #464)
2026-01-29 15:31:57 +01:00
Michel Rögl-Brunner
6d2df9929c fix: use node-specific Proxmox config paths for VM vs LXC detection
- isVM(): check /etc/pve/nodes/<server.name>/qemu-server and lxc first, fallback to /etc/pve/qemu-server and lxc for single-node
- checkConfigAndExtractInfo, config-existence checks, getContainerHostname, addClonedContainerToDatabase: use node-specific paths
- syncLXCConfig/updateLXCConfig: use node-specific LXC config path
- server.js clone flow: use node-specific config path

Fixes #464
2026-01-29 15:29:35 +01:00
Michel Roegl-Brunner
f33504baf5 Merge pull request #478 from community-scripts/fix/312
fix: handle special characters in SSH password/passphrase (Fixes #312)
2026-01-29 15:20:44 +01:00
Michel Rögl-Brunner
4bc5f4d6ad fix: handle special characters in SSH password/passphrase (Fixes #312)
- Use sshpass -f with temp file in transferScriptsFolder so password/passphrase
  never go through shell; safe for {, $, ", etc.
- Pass password via SSH_PASSWORD env in testWithExpect instead of embedding
  in script
- Add ServerForm hint: SSH key recommended; special chars supported
2026-01-29 15:18:41 +01:00
Michel Rögl-Brunner
a52a897346 chore: update publish_release workflow to bump package.json version too 2026-01-29 14:44:53 +01:00
Michel Rögl-Brunner
1d585d4d3f Unf**k deps 2026-01-29 14:43:56 +01:00
Michel Rögl-Brunner
d4b8ceb581 Merge fix/362: chore deps and overrides (next >=16.1.5, hono >=4.11.7, lodash >=4.17.23) 2026-01-29 14:29:46 +01:00
Michel Rögl-Brunner
7079c236ab chore: bump deps and overrides (next >=16.1.5, hono >=4.11.7, lodash >=4.17.23) 2026-01-29 14:27:56 +01:00
Michel Roegl-Brunner
0678aba911 Merge pull request #463 from community-scripts/dependabot/npm_and_yarn/npm_and_yarn-eb4f97c0ca
build(deps): Bump hono from 4.10.6 to 4.11.4 in the npm_and_yarn group across 1 directory
2026-01-29 14:16:56 +01:00
Michel Roegl-Brunner
ffdd742aa0 Merge pull request #476 from community-scripts/fix/362
Fix #362: auto-detect race, VM shell path, UI hints
2026-01-29 14:15:21 +01:00
Michel Roegl-Brunner
f4de214a83 Merge pull request #461 from community-scripts/dependabot/npm_and_yarn/testing-library/react-16.3.2
build(deps-dev): Bump @testing-library/react from 16.3.1 to 16.3.2
2026-01-29 14:14:26 +01:00
Michel Roegl-Brunner
3b0da19cd1 Merge pull request #460 from community-scripts/dependabot/npm_and_yarn/tanstack/react-query-5.90.19
build(deps): Bump @tanstack/react-query from 5.90.18 to 5.90.19
2026-01-29 14:14:18 +01:00
Michel Roegl-Brunner
08bc4ab37b Merge pull request #459 from community-scripts/dependabot/npm_and_yarn/typescript-eslint-8.53.1
build(deps-dev): Bump typescript-eslint from 8.53.0 to 8.53.1
2026-01-29 14:14:08 +01:00
Michel Roegl-Brunner
d2e7477898 Merge pull request #458 from community-scripts/dependabot/npm_and_yarn/better-sqlite3-12.6.2
build(deps): Bump better-sqlite3 from 12.6.0 to 12.6.2
2026-01-29 14:13:59 +01:00
Michel Rögl-Brunner
b5c6beafff Fix #362: auto-detect race, VM shell path, UI hints
- Defer resolve in autoDetectLXCContainers (pct/qm list) so stdout is complete
- Pass containerType when opening shell; use qm terminal for VMs, pct enter for LXC
- Add UI hint for VM shell (serial console, Ctrl+O, serial port requirement)
- Rename auto-detect to Containers & VMs and update help text

Fixes #362
2026-01-29 14:12:49 +01:00
Michel Roegl-Brunner
a34566651a Merge pull request #475 from community-scripts/fix/438
Fix PBS certificate validation (Fixes #438)
2026-01-29 13:57:51 +01:00
Michel Rögl-Brunner
4628e67e5c Fix PBS certificate validation: pass PBS_FINGERPRINT, optional fingerprint for trusted CA
- Pass stored pbs_fingerprint as PBS_FINGERPRINT in login, snapshot list, and restore
- Allow empty fingerprint so trusted-CA PBS works without entering one
- Make fingerprint field optional in PBSCredentialsModal with updated helper text

Fixes #438
2026-01-29 13:55:53 +01:00
Michel Roegl-Brunner
578fa28461 Merge pull request #474 from community-scripts/fix/404
fix: allow domain names for APT Cacher in container creation UI
2026-01-29 13:42:31 +01:00
Michel Rögl-Brunner
9e6154b0de fix: allow domain names for APT Cacher in container creation UI
- Add validateHostname and validateAptCacherAddress (IPv4 or hostname)
- Use new validator for var_apt_cacher_ip; error message: Invalid IPv4 or hostname
- Label: APT Cacher host or IP; placeholder shows IP or hostname example

Fixes #404
2026-01-29 13:40:19 +01:00
Michel Roegl-Brunner
d29f71a92f Merge pull request #473 from community-scripts/fix/365
fix: detect app slug from LXC /usr/bin/update for port lookup
2026-01-29 13:28:36 +01:00
Michel Rögl-Brunner
aea14cda7e fix: detect app slug from LXC /usr/bin/update for port lookup
Resolve interface_port from community-scripts update file when hostname
differs from JSON slug (e.g. lxcpeanut vs peanut). Primary: slug parsed
from pct exec ... cat /usr/bin/update; fallback: hostname/suffix match.

Fixes #365
2026-01-29 13:26:29 +01:00
Michel Roegl-Brunner
4893ccda6e Merge pull request #472 from community-scripts/feat/406
feat: private/custom git repos - GitHub, GitLab, Bitbucket, custom
2026-01-29 13:11:54 +01:00
Michel Rögl-Brunner
a56c625b4f feat: private/custom git repos - GitHub, GitLab, Bitbucket, custom
- Add repository URL validation for GitHub, GitLab, Bitbucket, and custom hosts
- Add git provider layer (listDirectory, downloadRawFile) for all providers
- Wire githubJsonService and scriptDownloader to use provider; sync/download from any supported source
- Update GeneralSettingsModal placeholder and help text; .env.example and env schema for GITLAB_TOKEN, BITBUCKET_APP_PASSWORD

Closes #406
2026-01-29 13:08:28 +01:00
Michel Roegl-Brunner
54b2187f98 Merge pull request #471 from community-scripts/feat/419
feat: add TUN/TAP (VPN) option to container features in web GUI
2026-01-29 11:37:51 +01:00
dependabot[bot]
c06b8e6731 build(deps-dev): Bump typescript-eslint from 8.53.0 to 8.53.1
Bumps [typescript-eslint](https://github.com/typescript-eslint/typescript-eslint/tree/HEAD/packages/typescript-eslint) from 8.53.0 to 8.53.1.
- [Release notes](https://github.com/typescript-eslint/typescript-eslint/releases)
- [Changelog](https://github.com/typescript-eslint/typescript-eslint/blob/main/packages/typescript-eslint/CHANGELOG.md)
- [Commits](https://github.com/typescript-eslint/typescript-eslint/commits/v8.53.1/packages/typescript-eslint)

---
updated-dependencies:
- dependency-name: typescript-eslint
  dependency-version: 8.53.1
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-01-29 09:12:03 +00:00
dependabot[bot]
14e01513e3 build(deps): Bump @tanstack/react-query from 5.90.18 to 5.90.19
Bumps [@tanstack/react-query](https://github.com/TanStack/query/tree/HEAD/packages/react-query) from 5.90.18 to 5.90.19.
- [Release notes](https://github.com/TanStack/query/releases)
- [Changelog](https://github.com/TanStack/query/blob/main/packages/react-query/CHANGELOG.md)
- [Commits](https://github.com/TanStack/query/commits/@tanstack/react-query@5.90.19/packages/react-query)

---
updated-dependencies:
- dependency-name: "@tanstack/react-query"
  dependency-version: 5.90.19
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-01-29 09:11:59 +00:00
dependabot[bot]
2e4634ca25 build(deps): Bump hono in the npm_and_yarn group across 1 directory
Bumps the npm_and_yarn group with 1 update in the / directory: [hono](https://github.com/honojs/hono).


Updates `hono` from 4.10.6 to 4.11.4
- [Release notes](https://github.com/honojs/hono/releases)
- [Commits](https://github.com/honojs/hono/compare/v4.10.6...v4.11.4)

---
updated-dependencies:
- dependency-name: hono
  dependency-version: 4.11.4
  dependency-type: indirect
  dependency-group: npm_and_yarn
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-01-23 22:50:38 +00:00
dependabot[bot]
a82bc02b15 build(deps-dev): Bump @testing-library/react from 16.3.1 to 16.3.2
Bumps [@testing-library/react](https://github.com/testing-library/react-testing-library) from 16.3.1 to 16.3.2.
- [Release notes](https://github.com/testing-library/react-testing-library/releases)
- [Changelog](https://github.com/testing-library/react-testing-library/blob/main/CHANGELOG.md)
- [Commits](https://github.com/testing-library/react-testing-library/compare/v16.3.1...v16.3.2)

---
updated-dependencies:
- dependency-name: "@testing-library/react"
  dependency-version: 16.3.2
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-01-19 21:19:12 +00:00
dependabot[bot]
2ea44e6b24 build(deps): Bump better-sqlite3 from 12.6.0 to 12.6.2
Bumps [better-sqlite3](https://github.com/WiseLibs/better-sqlite3) from 12.6.0 to 12.6.2.
- [Release notes](https://github.com/WiseLibs/better-sqlite3/releases)
- [Commits](https://github.com/WiseLibs/better-sqlite3/compare/v12.6.0...v12.6.2)

---
updated-dependencies:
- dependency-name: better-sqlite3
  dependency-version: 12.6.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-01-19 21:18:19 +00:00
31 changed files with 1222 additions and 722 deletions

View File

@@ -18,7 +18,12 @@ ALLOWED_SCRIPT_PATHS="scripts/"
WEBSOCKET_PORT="3001"
# User settings
# Optional tokens for private repos: GITHUB_TOKEN (GitHub), GITLAB_TOKEN (GitLab),
# BITBUCKET_APP_PASSWORD or BITBUCKET_TOKEN (Bitbucket). REPO_URL and added repos
# can be GitHub, GitLab, Bitbucket, or custom Git servers.
GITHUB_TOKEN=
GITLAB_TOKEN=
BITBUCKET_APP_PASSWORD=
SAVE_FILTER=false
FILTERS=
AUTH_USERNAME=

View File

@@ -31,20 +31,24 @@ jobs:
echo "Found draft version: ${{ steps.draft.outputs.tag_name }}"
- name: Create branch and commit VERSION
- name: Create branch and commit VERSION and package.json
run: |
branch="update-version-${{ steps.draft.outputs.tag_name }}"
# Delete remote branch if exists
git push origin --delete "$branch" || echo "No remote branch to delete"
git fetch origin main
git checkout -b "$branch" origin/main
# Write VERSION file and timestamp to ensure a diff
# Version without 'v' prefix (e.g. v1.2.3 -> 1.2.3)
version="${{ steps.draft.outputs.tag_name }}"
echo "$version" | sed 's/^v//' > VERSION
git add VERSION
version_plain=$(echo "$version" | sed 's/^v//')
# Write VERSION file
echo "$version_plain" > VERSION
# Update package.json version
jq --arg v "$version_plain" '.version = $v' package.json > package.json.tmp && mv package.json.tmp package.json
git add VERSION package.json
git config user.name "github-actions[bot]"
git config user.email "github-actions[bot]@users.noreply.github.com"
git commit -m "chore: add VERSION $version" --allow-empty
git commit -m "chore: bump version to $version_plain (VERSION + package.json)" --allow-empty
- name: Push changes
run: |
@@ -57,8 +61,8 @@ jobs:
pr_url=$(gh pr create \
--base main \
--head update-version-${{ steps.draft.outputs.tag_name }} \
--title "chore: add VERSION ${{ steps.draft.outputs.tag_name }}" \
--body "Adds VERSION file for release ${{ steps.draft.outputs.tag_name }}" \
--title "chore: bump version to ${{ steps.draft.outputs.tag_name }} (VERSION + package.json)" \
--body "Updates VERSION file and package.json version for release ${{ steps.draft.outputs.tag_name }}" \
--label automated)
pr_number=$(echo "$pr_url" | awk -F/ '{print $NF}')

View File

@@ -1 +1 @@
0.5.5
0.5.6

686
package-lock.json generated

File diff suppressed because it is too large Load Diff

View File

@@ -1,6 +1,6 @@
{
"name": "pve-scripts-local",
"version": "0.1.0",
"version": "0.5.6",
"private": true,
"type": "module",
"scripts": {
@@ -25,13 +25,13 @@
"typecheck": "tsc --noEmit"
},
"dependencies": {
"@prisma/adapter-better-sqlite3": "^7.2.0",
"@prisma/client": "^7.2.0",
"@prisma/adapter-better-sqlite3": "^7.3.0",
"@prisma/client": "^7.3.0",
"@radix-ui/react-dropdown-menu": "^2.1.16",
"@radix-ui/react-slot": "^1.2.4",
"@t3-oss/env-nextjs": "^0.13.10",
"@tailwindcss/typography": "^0.5.19",
"@tanstack/react-query": "^5.90.18",
"@tanstack/react-query": "^5.90.20",
"@trpc/client": "^11.8.1",
"@trpc/react-query": "^11.8.1",
"@trpc/server": "^11.8.1",
@@ -42,14 +42,14 @@
"@xterm/xterm": "^6.0.0",
"axios": "^1.13.2",
"bcryptjs": "^3.0.3",
"better-sqlite3": "^12.6.0",
"better-sqlite3": "^12.6.2",
"class-variance-authority": "^0.7.1",
"clsx": "^2.1.1",
"cron-validator": "^1.4.0",
"dotenv": "^17.2.3",
"jsonwebtoken": "^9.0.3",
"lucide-react": "^0.562.0",
"next": "^16.1.3",
"next": ">=16.1.5",
"node-cron": "^4.2.1",
"node-pty": "^1.1.0",
"react": "^19.2.3",
@@ -66,9 +66,10 @@
"zod": "^4.3.5"
},
"devDependencies": {
"next": ">=16.1.5",
"@tailwindcss/postcss": "^4.1.18",
"@testing-library/jest-dom": "^6.9.1",
"@testing-library/react": "^16.3.1",
"@testing-library/react": "^16.3.2",
"@testing-library/user-event": "^14.6.1",
"@types/bcryptjs": "^3.0.0",
"@types/better-sqlite3": "^7.6.13",
@@ -85,13 +86,13 @@
"eslint-config-next": "^16.1.3",
"jsdom": "^27.4.0",
"postcss": "^8.5.6",
"prettier": "^3.8.0",
"prettier": "^3.8.1",
"prettier-plugin-tailwindcss": "^0.7.2",
"prisma": "^7.2.0",
"prisma": "^7.3.0",
"tailwindcss": "^4.1.18",
"tsx": "^4.21.0",
"typescript": "^5.9.3",
"typescript-eslint": "^8.53.0",
"typescript-eslint": "^8.54.0",
"vitest": "^4.0.17"
},
"ct3aMetadata": {
@@ -102,6 +103,7 @@
"node": ">=24.0.0"
},
"overrides": {
"prismjs": "^1.30.0"
"prismjs": "^1.30.0",
"hono": ">=4.11.7"
}
}
}

View File

@@ -3,6 +3,7 @@ import { parse } from 'url';
import next from 'next';
import { WebSocketServer } from 'ws';
import { spawn } from 'child_process';
import { existsSync } from 'fs';
import { join, resolve } from 'path';
import stripAnsi from 'strip-ansi';
import { spawn as ptySpawn } from 'node-pty';
@@ -56,6 +57,8 @@ const handle = app.getRequestHandler();
* @property {string} user
* @property {string} password
* @property {number} [id]
* @property {string} [auth_type]
* @property {string} [ssh_key_path]
*/
/**
@@ -295,6 +298,20 @@ class ScriptExecutionHandler {
});
}
/**
* Resolve full server from DB when client sends server with id but no ssh_key_path (e.g. for Shell/Update over SSH).
* @param {ServerInfo|null} server - Server from WebSocket message
* @returns {Promise<ServerInfo|null>} Same server or full server from DB
*/
async resolveServerForSSH(server) {
if (!server?.id) return server;
if (server.auth_type === 'key' && (!server.ssh_key_path || !existsSync(server.ssh_key_path))) {
const full = await this.db.getServerById(server.id);
return /** @type {ServerInfo|null} */ (full ?? server);
}
return server;
}
/**
* @param {ExtendedWebSocket} ws
* @param {WebSocketMessage} message
@@ -305,16 +322,21 @@ class ScriptExecutionHandler {
switch (action) {
case 'start':
if (scriptPath && executionId) {
let serverToUse = server;
if (serverToUse?.id) {
serverToUse = await this.resolveServerForSSH(serverToUse) ?? serverToUse;
}
const resolved = serverToUse ?? server;
if (isClone && containerId && storage && server && cloneCount && hostnames && containerType) {
await this.startSSHCloneExecution(ws, containerId, executionId, storage, server, containerType, cloneCount, hostnames);
await this.startSSHCloneExecution(ws, containerId, executionId, storage, /** @type {ServerInfo} */ (resolved), containerType, cloneCount, hostnames);
} else if (isBackup && containerId && storage) {
await this.startBackupExecution(ws, containerId, executionId, storage, mode, server);
await this.startBackupExecution(ws, containerId, executionId, storage, mode, resolved);
} else if (isUpdate && containerId) {
await this.startUpdateExecution(ws, containerId, executionId, mode, server, backupStorage);
await this.startUpdateExecution(ws, containerId, executionId, mode, resolved, backupStorage);
} else if (isShell && containerId) {
await this.startShellExecution(ws, containerId, executionId, mode, server);
await this.startShellExecution(ws, containerId, executionId, mode, resolved, containerType);
} else {
await this.startScriptExecution(ws, scriptPath, executionId, mode, server, envVars);
await this.startScriptExecution(ws, scriptPath, executionId, mode, resolved, envVars);
}
} else {
this.sendMessage(ws, {
@@ -1153,10 +1175,11 @@ class ScriptExecutionHandler {
const hostname = hostnames[i];
try {
// Read config file to get hostname/name
// Read config file to get hostname/name (node-specific path)
const nodeName = server.name;
const configPath = containerType === 'lxc'
? `/etc/pve/lxc/${nextId}.conf`
: `/etc/pve/qemu-server/${nextId}.conf`;
? `/etc/pve/nodes/${nodeName}/lxc/${nextId}.conf`
: `/etc/pve/nodes/${nodeName}/qemu-server/${nextId}.conf`;
let configContent = '';
await new Promise(/** @type {(resolve: (value?: void) => void) => void} */ ((resolve) => {
@@ -1474,21 +1497,21 @@ class ScriptExecutionHandler {
* @param {string} executionId
* @param {string} mode
* @param {ServerInfo|null} server
* @param {'lxc'|'vm'} [containerType='lxc']
*/
async startShellExecution(ws, containerId, executionId, mode = 'local', server = null) {
async startShellExecution(ws, containerId, executionId, mode = 'local', server = null, containerType = 'lxc') {
try {
// Send start message
const typeLabel = containerType === 'vm' ? 'VM' : 'container';
this.sendMessage(ws, {
type: 'start',
data: `Starting shell session for container ${containerId}...`,
data: `Starting shell session for ${typeLabel} ${containerId}...`,
timestamp: Date.now()
});
if (mode === 'ssh' && server) {
await this.startSSHShellExecution(ws, containerId, executionId, server);
await this.startSSHShellExecution(ws, containerId, executionId, server, containerType);
} else {
await this.startLocalShellExecution(ws, containerId, executionId);
await this.startLocalShellExecution(ws, containerId, executionId, containerType);
}
} catch (error) {
@@ -1505,12 +1528,12 @@ class ScriptExecutionHandler {
* @param {ExtendedWebSocket} ws
* @param {string} containerId
* @param {string} executionId
* @param {'lxc'|'vm'} [containerType='lxc']
*/
async startLocalShellExecution(ws, containerId, executionId) {
async startLocalShellExecution(ws, containerId, executionId, containerType = 'lxc') {
const { spawn } = await import('node-pty');
// Create a shell process that will run pct enter
const childProcess = spawn('bash', ['-c', `pct enter ${containerId}`], {
const shellCommand = containerType === 'vm' ? `qm terminal ${containerId}` : `pct enter ${containerId}`;
const childProcess = spawn('bash', ['-c', shellCommand], {
name: 'xterm-color',
cols: 80,
rows: 24,
@@ -1553,14 +1576,15 @@ class ScriptExecutionHandler {
* @param {string} containerId
* @param {string} executionId
* @param {ServerInfo} server
* @param {'lxc'|'vm'} [containerType='lxc']
*/
async startSSHShellExecution(ws, containerId, executionId, server) {
async startSSHShellExecution(ws, containerId, executionId, server, containerType = 'lxc') {
const sshService = getSSHExecutionService();
const shellCommand = containerType === 'vm' ? `qm terminal ${containerId}` : `pct enter ${containerId}`;
try {
const execution = await sshService.executeCommand(
server,
`pct enter ${containerId}`,
shellCommand,
/** @param {string} data */
(data) => {
this.sendMessage(ws, {

View File

@@ -199,6 +199,17 @@ export function ConfigurationModal({
return !isNaN(num) && num > 0;
};
const validateHostname = (hostname: string): boolean => {
if (!hostname || hostname.length > 253) return false;
const label = /^[a-zA-Z0-9]([a-zA-Z0-9-]*[a-zA-Z0-9])?$/;
const labels = hostname.split('.');
return labels.length >= 1 && labels.every(l => l.length >= 1 && l.length <= 63 && label.test(l));
};
const validateAptCacherAddress = (value: string): boolean => {
return validateIPv4(value) || validateHostname(value);
};
const validateForm = (): boolean => {
const newErrors: Record<string, string> = {};
@@ -216,8 +227,8 @@ export function ConfigurationModal({
if (advancedVars.var_ns && !validateIPv4(advancedVars.var_ns as string)) {
newErrors.var_ns = 'Invalid IPv4 address';
}
if (advancedVars.var_apt_cacher_ip && !validateIPv4(advancedVars.var_apt_cacher_ip as string)) {
newErrors.var_apt_cacher_ip = 'Invalid IPv4 address';
if (advancedVars.var_apt_cacher_ip && !validateAptCacherAddress(advancedVars.var_apt_cacher_ip as string)) {
newErrors.var_apt_cacher_ip = 'Invalid IPv4 address or hostname';
}
// Validate IPv4 CIDR if network mode is static
const netValue = advancedVars.var_net;
@@ -904,13 +915,13 @@ export function ConfigurationModal({
</div>
<div>
<label className="block text-sm font-medium text-foreground mb-2">
APT Cacher IP
APT Cacher host or IP
</label>
<Input
type="text"
value={typeof advancedVars.var_apt_cacher_ip === 'boolean' ? '' : String(advancedVars.var_apt_cacher_ip ?? '')}
onChange={(e) => updateAdvancedVar('var_apt_cacher_ip', e.target.value)}
placeholder="192.168.1.10"
placeholder="192.168.1.10 or apt-cacher.internal"
className={errors.var_apt_cacher_ip ? 'border-destructive' : ''}
/>
{errors.var_apt_cacher_ip && (

View File

@@ -1617,7 +1617,7 @@ export function GeneralSettingsModal({
<Input
id="new-repo-url"
type="url"
placeholder="https://github.com/owner/repo"
placeholder="https://github.com/owner/repo or https://git.example.com/owner/repo"
value={newRepoUrl}
onChange={(e: React.ChangeEvent<HTMLInputElement>) =>
setNewRepoUrl(e.target.value)
@@ -1626,8 +1626,9 @@ export function GeneralSettingsModal({
className="w-full"
/>
<p className="text-muted-foreground mt-1 text-xs">
Enter a GitHub repository URL (e.g.,
https://github.com/owner/repo)
Supported: GitHub, GitLab, Bitbucket, or custom Git
servers (e.g. https://github.com/owner/repo,
https://gitlab.com/owner/repo)
</p>
</div>
<div className="border-border flex items-center justify-between gap-3 rounded-lg border p-3">

View File

@@ -80,6 +80,7 @@ export function InstalledScriptsTab() {
id: number;
containerId: string;
server?: any;
containerType?: 'lxc' | 'vm';
} | null>(null);
const [showBackupPrompt, setShowBackupPrompt] = useState(false);
const [showStorageSelection, setShowStorageSelection] = useState(false);
@@ -1167,6 +1168,7 @@ export function InstalledScriptsTab() {
id: script.id,
containerId: script.container_id,
server: server,
containerType: script.is_vm ? 'vm' : 'lxc',
});
};
@@ -1452,6 +1454,13 @@ export function InstalledScriptsTab() {
{/* Shell Terminal */}
{openingShell && (
<div className="mb-8" data-terminal="shell">
{openingShell.containerType === 'vm' && (
<p className="text-muted-foreground mb-2 text-sm">
VM shell uses the Proxmox serial console. The VM must have a
serial port configured (e.g. <code className="bg-muted rounded px-1">qm set {openingShell.containerId} -serial0 socket</code>).
Detach with <kbd className="bg-muted rounded px-1">Ctrl+O</kbd>.
</p>
)}
<Terminal
scriptPath={`shell-${openingShell.containerId}`}
onClose={handleCloseShellTerminal}
@@ -1459,6 +1468,7 @@ export function InstalledScriptsTab() {
server={openingShell.server}
isShell={true}
containerId={openingShell.containerId}
containerType={openingShell.containerType}
/>
</div>
)}
@@ -1538,7 +1548,7 @@ export function InstalledScriptsTab() {
>
{showAutoDetectForm
? "Cancel Auto-Detect"
: '🔍 Auto-Detect LXC Containers (Must contain a tag with "community-script")'}
: '🔍 Auto-Detect Containers & VMs (tag: community-script)'}
</Button>
<Button
onClick={() => {
@@ -1764,12 +1774,11 @@ export function InstalledScriptsTab() {
</div>
)}
{/* Auto-Detect LXC Containers Form */}
{/* Auto-Detect Containers & VMs Form */}
{showAutoDetectForm && (
<div className="bg-card border-border mb-6 rounded-lg border p-4 shadow-sm sm:p-6">
<h3 className="text-foreground mb-4 text-lg font-semibold sm:mb-6">
Auto-Detect LXC Containers (Must contain a tag with
&quot;community-script&quot;)
Auto-Detect Containers &amp; VMs (tag: community-script)
</h3>
<div className="space-y-4 sm:space-y-6">
<div className="bg-muted/30 border-muted rounded-lg border p-4">
@@ -1795,12 +1804,12 @@ export function InstalledScriptsTab() {
<p>This feature will:</p>
<ul className="mt-1 list-inside list-disc space-y-1">
<li>Connect to the selected server via SSH</li>
<li>Scan all LXC config files in /etc/pve/lxc/</li>
<li>Scan LXC configs in /etc/pve/lxc/ and VM configs in /etc/pve/qemu-server/</li>
<li>
Find containers with &quot;community-script&quot; in
Find containers and VMs with &quot;community-script&quot; in
their tags
</li>
<li>Extract the container ID and hostname</li>
<li>Extract the container/VM ID and hostname or name</li>
<li>Add them as installed script entries</li>
</ul>
</div>
@@ -2302,6 +2311,11 @@ export function InstalledScriptsTab() {
"stopped"
}
className="text-muted-foreground hover:text-foreground hover:bg-muted/20 focus:bg-muted/20"
title={
script.is_vm
? "VM serial console (requires serial port; detach with Ctrl+O)"
: undefined
}
>
Shell
</DropdownMenuItem>

View File

@@ -270,22 +270,21 @@ export function PBSCredentialsModal({
htmlFor="pbs-fingerprint"
className="text-foreground mb-1 block text-sm font-medium"
>
Fingerprint <span className="text-error">*</span>
Fingerprint
</label>
<input
type="text"
id="pbs-fingerprint"
value={pbsFingerprint}
onChange={(e) => setPbsFingerprint(e.target.value)}
required
disabled={isLoading}
className="bg-card text-foreground placeholder-muted-foreground focus:ring-ring focus:border-ring border-border w-full rounded-md border px-3 py-2 shadow-sm focus:ring-2 focus:outline-none"
placeholder="e.g., 7b:e5:87:38:5e:16:05:d1:12:22:7f:73:d2:e2:d0:cf:8c:cb:28:e2:74:0c:78:91:1a:71:74:2e:79:20:5a:02"
/>
<p className="text-muted-foreground mt-1 text-xs">
Server fingerprint for auto-acceptance. You can find this on
your PBS dashboard by clicking the &quot;Show Fingerprint&quot;
button.
Leave empty if PBS uses a trusted CA (e.g. Let&apos;s Encrypt).
For self-signed certificates, enter the server fingerprint from
the PBS dashboard (&quot;Show Fingerprint&quot;).
</p>
</div>

View File

@@ -438,6 +438,11 @@ export function ServerForm({
{errors.password && (
<p className="text-destructive mt-1 text-sm">{errors.password}</p>
)}
<p className="text-muted-foreground mt-1 text-xs">
SSH key is recommended when possible. Special characters (e.g.{" "}
<code className="rounded bg-muted px-0.5">{"{ } $ \" '"}</code>) are
supported.
</p>
</div>
)}

View File

@@ -23,8 +23,11 @@ export const env = createEnv({
ALLOWED_SCRIPT_PATHS: z.string().default("scripts/"),
// WebSocket Configuration
WEBSOCKET_PORT: z.string().default("3001"),
// GitHub Configuration
// Git provider tokens (optional, for private repos)
GITHUB_TOKEN: z.string().optional(),
GITLAB_TOKEN: z.string().optional(),
BITBUCKET_APP_PASSWORD: z.string().optional(),
BITBUCKET_TOKEN: z.string().optional(),
// Authentication Configuration
AUTH_USERNAME: z.string().optional(),
AUTH_PASSWORD_HASH: z.string().optional(),
@@ -62,8 +65,10 @@ export const env = createEnv({
ALLOWED_SCRIPT_PATHS: process.env.ALLOWED_SCRIPT_PATHS,
// WebSocket Configuration
WEBSOCKET_PORT: process.env.WEBSOCKET_PORT,
// GitHub Configuration
GITHUB_TOKEN: process.env.GITHUB_TOKEN,
GITLAB_TOKEN: process.env.GITLAB_TOKEN,
BITBUCKET_APP_PASSWORD: process.env.BITBUCKET_APP_PASSWORD,
BITBUCKET_TOKEN: process.env.BITBUCKET_TOKEN,
// Authentication Configuration
AUTH_USERNAME: process.env.AUTH_USERNAME,
AUTH_PASSWORD_HASH: process.env.AUTH_PASSWORD_HASH,

View File

@@ -418,44 +418,46 @@ async function isVM(scriptId: number, containerId: string, serverId: number | nu
return false; // Default to LXC if SSH fails
}
// Check both config file paths
const vmConfigPath = `/etc/pve/qemu-server/${containerId}.conf`;
const lxcConfigPath = `/etc/pve/lxc/${containerId}.conf`;
// Check VM config file
let vmConfigExists = false;
await new Promise<void>((resolve) => {
void sshExecutionService.executeCommand(
server as Server,
`test -f "${vmConfigPath}" && echo "exists" || echo "not_exists"`,
(data: string) => {
if (data.includes('exists')) {
vmConfigExists = true;
}
},
() => resolve(),
() => resolve()
);
});
if (vmConfigExists) {
return true; // VM config file exists
}
// Check LXC config file (not needed for return value, but check for completeness)
await new Promise<void>((resolve) => {
void sshExecutionService.executeCommand(
server as Server,
`test -f "${lxcConfigPath}" && echo "exists" || echo "not_exists"`,
(_data: string) => {
// Data handler not needed - just checking if file exists
},
() => resolve(),
() => resolve()
);
});
// Node-specific paths (multi-node Proxmox: /etc/pve/nodes/NODENAME/...)
const nodeName = (server as Server).name;
const vmConfigPathNode = `/etc/pve/nodes/${nodeName}/qemu-server/${containerId}.conf`;
const lxcConfigPathNode = `/etc/pve/nodes/${nodeName}/lxc/${containerId}.conf`;
// Fallback for single-node or when server.name is not the Proxmox node name
const vmConfigPathFallback = `/etc/pve/qemu-server/${containerId}.conf`;
const lxcConfigPathFallback = `/etc/pve/lxc/${containerId}.conf`;
return false; // Always LXC since VM config doesn't exist
const checkPathExists = (path: string): Promise<boolean> =>
new Promise<boolean>((resolve) => {
let exists = false;
void sshExecutionService.executeCommand(
server as Server,
`test -f "${path}" && echo "exists" || echo "not_exists"`,
(data: string) => {
if (data.includes('exists')) exists = true;
},
() => resolve(exists),
() => resolve(exists)
);
});
// Prefer node-specific paths first
const vmConfigExistsNode = await checkPathExists(vmConfigPathNode);
if (vmConfigExistsNode) {
return true; // VM config file exists on node
}
const lxcConfigExistsNode = await checkPathExists(lxcConfigPathNode);
if (lxcConfigExistsNode) {
return false; // LXC config file exists on node
}
// Fallback: single-node or server.name not matching Proxmox node name
const vmConfigExistsFallback = await checkPathExists(vmConfigPathFallback);
if (vmConfigExistsFallback) {
return true;
}
return false; // LXC (or neither path exists)
} catch (error) {
console.error('Error determining container type:', error);
return false; // Default to LXC on error
@@ -971,10 +973,11 @@ export const installedScriptsRouter = createTRPCRouter({
};
// Helper function to check config file for community-script tag and extract hostname/name
const nodeName = (server as Server).name;
const checkConfigAndExtractInfo = async (id: string, isVM: boolean): Promise<any> => {
const configPath = isVM
? `/etc/pve/qemu-server/${id}.conf`
: `/etc/pve/lxc/${id}.conf`;
? `/etc/pve/nodes/${nodeName}/qemu-server/${id}.conf`
: `/etc/pve/nodes/${nodeName}/lxc/${id}.conf`;
const readCommand = `cat "${configPath}" 2>/dev/null`;
@@ -1060,7 +1063,7 @@ export const installedScriptsRouter = createTRPCRouter({
reject(new Error(`pct list failed: ${error}`));
},
(_exitCode: number) => {
resolve();
setImmediate(() => resolve());
}
);
});
@@ -1079,7 +1082,7 @@ export const installedScriptsRouter = createTRPCRouter({
reject(new Error(`qm list failed: ${error}`));
},
(_exitCode: number) => {
resolve();
setImmediate(() => resolve());
}
);
});
@@ -1318,10 +1321,10 @@ export const installedScriptsRouter = createTRPCRouter({
// Check if ID exists in either pct list (containers) or qm list (VMs)
if (!existingIds.has(containerId)) {
// Also verify config file doesn't exist as a double-check
// Check both container and VM config paths
const checkContainerCommand = `test -f "/etc/pve/lxc/${containerId}.conf" && echo "exists" || echo "not_found"`;
const checkVMCommand = `test -f "/etc/pve/qemu-server/${containerId}.conf" && echo "exists" || echo "not_found"`;
// Also verify config file doesn't exist as a double-check (node-specific paths)
const nodeName = (server as Server).name;
const checkContainerCommand = `test -f "/etc/pve/nodes/${nodeName}/lxc/${containerId}.conf" && echo "exists" || echo "not_found"`;
const checkVMCommand = `test -f "/etc/pve/nodes/${nodeName}/qemu-server/${containerId}.conf" && echo "exists" || echo "not_found"`;
const configExists = await new Promise<boolean>((resolve) => {
let combinedOutput = '';
@@ -2068,32 +2071,72 @@ export const installedScriptsRouter = createTRPCRouter({
};
}
// Get the script's interface_port from metadata (prioritize metadata over existing database values)
let detectedPort = 80; // Default fallback
// Resolve app slug from /usr/bin/update (community-scripts) when available; else from hostname/suffix.
let slugFromUpdate: string | null = null;
try {
const updateCommand = `pct exec ${scriptData.container_id} -- cat /usr/bin/update 2>/dev/null`;
let updateOutput = '';
await new Promise<void>((resolve) => {
void sshExecutionService.executeCommand(
server as Server,
updateCommand,
(data: string) => { updateOutput += data; },
() => {},
() => resolve()
);
});
const ctSlugMatch = /ct\/([a-zA-Z0-9_.-]+)\.sh/.exec(updateOutput);
if (ctSlugMatch?.[1]) {
slugFromUpdate = ctSlugMatch[1].trim().toLowerCase();
console.log('🔍 Slug from /usr/bin/update:', slugFromUpdate);
}
} catch {
// Container may not be from community-scripts; use hostname fallback
}
// Get the script's interface_port from metadata. Primary: slug from /usr/bin/update; fallback: hostname/suffix.
let detectedPort = 80; // Default fallback
try {
// Import localScriptsService to get script metadata
const { localScriptsService } = await import('~/server/services/localScripts');
// Get all scripts and find the one matching our script name
const allScripts = await localScriptsService.getAllScripts();
// Extract script slug from script_name (remove .sh extension)
const scriptSlug = scriptData.script_name.replace(/\.sh$/, '');
console.log('🔍 Looking for script with slug:', scriptSlug);
const scriptMetadata = allScripts.find(script => script.slug === scriptSlug);
const nameFromHostname = scriptData.script_name.replace(/\.sh$/, '').toLowerCase();
// Primary: slug from /usr/bin/update (community-scripts)
let scriptMetadata =
slugFromUpdate != null
? allScripts.find((s) => s.slug === slugFromUpdate)
: undefined;
if (scriptMetadata) {
console.log('🔍 Using slug from /usr/bin/update for metadata:', scriptMetadata.slug);
}
// Fallback: exact hostname then hostname ends with slug (longest wins)
if (!scriptMetadata) {
scriptMetadata = allScripts.find((script) => script.slug === nameFromHostname);
if (!scriptMetadata) {
const suffixMatches = allScripts.filter((script) => nameFromHostname.endsWith(script.slug));
scriptMetadata =
suffixMatches.length > 0
? suffixMatches.reduce((a, b) => (a.slug.length >= b.slug.length ? a : b))
: undefined;
if (scriptMetadata) {
console.log('🔍 Matched metadata by slug suffix in hostname:', scriptMetadata.slug);
}
}
}
if (scriptMetadata?.interface_port) {
detectedPort = scriptMetadata.interface_port;
console.log('📋 Found interface_port in metadata:', detectedPort);
} else {
console.log('📋 No interface_port found in metadata, using default port 80');
detectedPort = 80; // Default to port 80 if no metadata port found
detectedPort = 80;
}
} catch (error) {
console.log('⚠️ Error getting script metadata, using default port 80:', error);
detectedPort = 80; // Default to port 80 if metadata lookup fails
detectedPort = 80;
}
console.log('🎯 Final detected port:', detectedPort);
@@ -2197,8 +2240,9 @@ export const installedScriptsRouter = createTRPCRouter({
};
}
// Read config file
const configPath = `/etc/pve/lxc/${script.container_id}.conf`;
// Read config file (node-specific path)
const nodeName = (server as Server).name;
const configPath = `/etc/pve/nodes/${nodeName}/lxc/${script.container_id}.conf`;
const readCommand = `cat "${configPath}" 2>/dev/null`;
let rawConfig = '';
@@ -2328,8 +2372,9 @@ export const installedScriptsRouter = createTRPCRouter({
};
}
// Write config file using heredoc for safe escaping
const configPath = `/etc/pve/lxc/${script.container_id}.conf`;
// Write config file using heredoc for safe escaping (node-specific path)
const nodeName = (server as Server).name;
const configPath = `/etc/pve/nodes/${nodeName}/lxc/${script.container_id}.conf`;
const writeCommand = `cat > "${configPath}" << 'EOFCONFIG'
${rawConfig}
EOFCONFIG`;
@@ -2737,9 +2782,10 @@ EOFCONFIG`;
const { getSSHExecutionService } = await import('~/server/ssh-execution-service');
const sshExecutionService = getSSHExecutionService();
const nodeName = (server as Server).name;
const configPath = input.containerType === 'lxc'
? `/etc/pve/lxc/${input.containerId}.conf`
: `/etc/pve/qemu-server/${input.containerId}.conf`;
? `/etc/pve/nodes/${nodeName}/lxc/${input.containerId}.conf`
: `/etc/pve/nodes/${nodeName}/qemu-server/${input.containerId}.conf`;
let configContent = '';
await new Promise<void>((resolve) => {
@@ -3131,10 +3177,11 @@ EOFCONFIG`;
const { getSSHExecutionService } = await import('~/server/ssh-execution-service');
const sshExecutionService = getSSHExecutionService();
// Read config file to get hostname/name
// Read config file to get hostname/name (node-specific path)
const nodeName = (server as Server).name;
const configPath = input.containerType === 'lxc'
? `/etc/pve/lxc/${input.containerId}.conf`
: `/etc/pve/qemu-server/${input.containerId}.conf`;
? `/etc/pve/nodes/${nodeName}/lxc/${input.containerId}.conf`
: `/etc/pve/nodes/${nodeName}/qemu-server/${input.containerId}.conf`;
let configContent = '';
await new Promise<void>((resolve) => {

View File

@@ -0,0 +1,55 @@
import type { DirEntry, GitProvider } from './types';
import { parseRepoUrl } from '../repositoryUrlValidation';
export class BitbucketProvider implements GitProvider {
async listDirectory(repoUrl: string, path: string, branch: string): Promise<DirEntry[]> {
const { owner, repo } = parseRepoUrl(repoUrl);
const listUrl = `https://api.bitbucket.org/2.0/repositories/${owner}/${repo}/src/${encodeURIComponent(branch)}/${path}`;
const headers: Record<string, string> = {
'User-Agent': 'PVEScripts-Local/1.0',
};
const token = process.env.BITBUCKET_APP_PASSWORD ?? process.env.BITBUCKET_TOKEN;
if (token) {
const auth = Buffer.from(`:${token}`).toString('base64');
headers.Authorization = `Basic ${auth}`;
}
const response = await fetch(listUrl, { headers });
if (!response.ok) {
throw new Error(`Bitbucket API error: ${response.status} ${response.statusText}`);
}
const body = (await response.json()) as { values?: { path: string; type: string }[] };
const data = body.values ?? (Array.isArray(body) ? body : []);
if (!Array.isArray(data)) {
throw new Error('Bitbucket API returned unexpected response');
}
return data.map((item: { path: string; type: string }) => {
const name = item.path.split('/').pop() ?? item.path;
return {
name,
path: item.path,
type: item.type === 'commit_directory' ? ('dir' as const) : ('file' as const),
};
});
}
async downloadRawFile(repoUrl: string, filePath: string, branch: string): Promise<string> {
const { owner, repo } = parseRepoUrl(repoUrl);
const rawUrl = `https://api.bitbucket.org/2.0/repositories/${owner}/${repo}/src/${encodeURIComponent(branch)}/${filePath}`;
const headers: Record<string, string> = {
'User-Agent': 'PVEScripts-Local/1.0',
};
const token = process.env.BITBUCKET_APP_PASSWORD ?? process.env.BITBUCKET_TOKEN;
if (token) {
const auth = Buffer.from(`:${token}`).toString('base64');
headers.Authorization = `Basic ${auth}`;
}
const response = await fetch(rawUrl, { headers });
if (!response.ok) {
throw new Error(`Failed to download ${filePath}: ${response.status} ${response.statusText}`);
}
return response.text();
}
}

View File

@@ -0,0 +1,44 @@
import type { DirEntry, GitProvider } from "./types";
import { parseRepoUrl } from "../repositoryUrlValidation";
export class CustomProvider implements GitProvider {
async listDirectory(repoUrl: string, path: string, branch: string): Promise<DirEntry[]> {
const { origin, owner, repo } = parseRepoUrl(repoUrl);
const apiUrl = `${origin}/api/v1/repos/${owner}/${repo}/contents/${path}?ref=${encodeURIComponent(branch)}`;
const headers: Record<string, string> = { "User-Agent": "PVEScripts-Local/1.0" };
const token = process.env.GITEA_TOKEN ?? process.env.GIT_TOKEN;
if (token) headers.Authorization = `token ${token}`;
const response = await fetch(apiUrl, { headers });
if (!response.ok) {
throw new Error(`Custom Git server: list directory failed (${response.status}).`);
}
const data = (await response.json()) as { type: string; name: string; path: string }[];
if (!Array.isArray(data)) {
const single = data as unknown as { type?: string; name?: string; path?: string };
if (single?.name) {
return [{ name: single.name, path: single.path ?? path, type: single.type === "dir" ? "dir" : "file" }];
}
throw new Error("Custom Git server returned unexpected response");
}
return data.map((item) => ({
name: item.name,
path: item.path,
type: item.type === "dir" ? ("dir" as const) : ("file" as const),
}));
}
async downloadRawFile(repoUrl: string, filePath: string, branch: string): Promise<string> {
const { origin, owner, repo } = parseRepoUrl(repoUrl);
const rawUrl = `${origin}/${owner}/${repo}/raw/${encodeURIComponent(branch)}/${filePath}`;
const headers: Record<string, string> = { "User-Agent": "PVEScripts-Local/1.0" };
const token = process.env.GITEA_TOKEN ?? process.env.GIT_TOKEN;
if (token) headers.Authorization = `token ${token}`;
const response = await fetch(rawUrl, { headers });
if (!response.ok) {
throw new Error(`Failed to download ${filePath} from custom Git server (${response.status}).`);
}
return response.text();
}
}

View File

@@ -0,0 +1,60 @@
import type { DirEntry, GitProvider } from './types';
import { parseRepoUrl } from '../repositoryUrlValidation';
export class GitHubProvider implements GitProvider {
async listDirectory(repoUrl: string, path: string, branch: string): Promise<DirEntry[]> {
const { owner, repo } = parseRepoUrl(repoUrl);
const apiUrl = `https://api.github.com/repos/${owner}/${repo}/contents/${path}?ref=${encodeURIComponent(branch)}`;
const headers: Record<string, string> = {
Accept: 'application/vnd.github.v3+json',
'User-Agent': 'PVEScripts-Local/1.0',
};
const token = process.env.GITHUB_TOKEN;
if (token) headers.Authorization = `token ${token}`;
const response = await fetch(apiUrl, { headers });
if (!response.ok) {
if (response.status === 403) {
const err = new Error(
`GitHub API rate limit exceeded. Consider setting GITHUB_TOKEN. Status: ${response.status} ${response.statusText}`
);
(err as Error & { name: string }).name = 'RateLimitError';
throw err;
}
throw new Error(`GitHub API error: ${response.status} ${response.statusText}`);
}
const data = (await response.json()) as { type: string; name: string; path: string }[];
if (!Array.isArray(data)) {
throw new Error('GitHub API returned unexpected response');
}
return data.map((item) => ({
name: item.name,
path: item.path,
type: item.type === 'dir' ? ('dir' as const) : ('file' as const),
}));
}
async downloadRawFile(repoUrl: string, filePath: string, branch: string): Promise<string> {
const { owner, repo } = parseRepoUrl(repoUrl);
const rawUrl = `https://raw.githubusercontent.com/${owner}/${repo}/${encodeURIComponent(branch)}/${filePath}`;
const headers: Record<string, string> = {
'User-Agent': 'PVEScripts-Local/1.0',
};
const token = process.env.GITHUB_TOKEN;
if (token) headers.Authorization = `token ${token}`;
const response = await fetch(rawUrl, { headers });
if (!response.ok) {
if (response.status === 403) {
const err = new Error(
`GitHub rate limit exceeded while downloading ${filePath}. Consider setting GITHUB_TOKEN.`
);
(err as Error & { name: string }).name = 'RateLimitError';
throw err;
}
throw new Error(`Failed to download ${filePath}: ${response.status} ${response.statusText}`);
}
return response.text();
}
}

View File

@@ -0,0 +1,58 @@
import type { DirEntry, GitProvider } from './types';
import { parseRepoUrl } from '../repositoryUrlValidation';
export class GitLabProvider implements GitProvider {
private getBaseUrl(repoUrl: string): string {
const { origin } = parseRepoUrl(repoUrl);
return origin;
}
private getProjectId(repoUrl: string): string {
const { owner, repo } = parseRepoUrl(repoUrl);
return encodeURIComponent(`${owner}/${repo}`);
}
async listDirectory(repoUrl: string, path: string, branch: string): Promise<DirEntry[]> {
const baseUrl = this.getBaseUrl(repoUrl);
const projectId = this.getProjectId(repoUrl);
const apiUrl = `${baseUrl}/api/v4/projects/${projectId}/repository/tree?path=${encodeURIComponent(path)}&ref=${encodeURIComponent(branch)}&per_page=100`;
const headers: Record<string, string> = {
'User-Agent': 'PVEScripts-Local/1.0',
};
const token = process.env.GITLAB_TOKEN;
if (token) headers['PRIVATE-TOKEN'] = token;
const response = await fetch(apiUrl, { headers });
if (!response.ok) {
throw new Error(`GitLab API error: ${response.status} ${response.statusText}`);
}
const data = (await response.json()) as { type: string; name: string; path: string }[];
if (!Array.isArray(data)) {
throw new Error('GitLab API returned unexpected response');
}
return data.map((item) => ({
name: item.name,
path: item.path,
type: item.type === 'tree' ? ('dir' as const) : ('file' as const),
}));
}
async downloadRawFile(repoUrl: string, filePath: string, branch: string): Promise<string> {
const baseUrl = this.getBaseUrl(repoUrl);
const projectId = this.getProjectId(repoUrl);
const encodedPath = encodeURIComponent(filePath);
const rawUrl = `${baseUrl}/api/v4/projects/${projectId}/repository/files/${encodedPath}/raw?ref=${encodeURIComponent(branch)}`;
const headers: Record<string, string> = {
'User-Agent': 'PVEScripts-Local/1.0',
};
const token = process.env.GITLAB_TOKEN;
if (token) headers['PRIVATE-TOKEN'] = token;
const response = await fetch(rawUrl, { headers });
if (!response.ok) {
throw new Error(`Failed to download ${filePath}: ${response.status} ${response.statusText}`);
}
return response.text();
}
}

View File

@@ -0,0 +1 @@
export { listDirectory, downloadRawFile, getRepoProvider } from "./index.ts";

View File

@@ -0,0 +1,28 @@
import type { DirEntry, GitProvider } from "./types";
import { getRepoProvider } from "../repositoryUrlValidation";
import { GitHubProvider } from "./github";
import { GitLabProvider } from "./gitlab";
import { BitbucketProvider } from "./bitbucket";
import { CustomProvider } from "./custom";
const providers: Record<string, GitProvider> = {
github: new GitHubProvider(),
gitlab: new GitLabProvider(),
bitbucket: new BitbucketProvider(),
custom: new CustomProvider(),
};
export type { DirEntry, GitProvider };
export { getRepoProvider };
export function getGitProvider(repoUrl: string): GitProvider {
return providers[getRepoProvider(repoUrl)]!;
}
export async function listDirectory(repoUrl: string, path: string, branch: string): Promise<DirEntry[]> {
return getGitProvider(repoUrl).listDirectory(repoUrl, path, branch);
}
export async function downloadRawFile(repoUrl: string, filePath: string, branch: string): Promise<string> {
return getGitProvider(repoUrl).downloadRawFile(repoUrl, filePath, branch);
}

View File

@@ -0,0 +1,14 @@
/**
* Git provider interface for listing and downloading repository files.
*/
export type DirEntry = {
name: string;
path: string;
type: 'file' | 'dir';
};
export interface GitProvider {
listDirectory(repoUrl: string, path: string, branch: string): Promise<DirEntry[]>;
downloadRawFile(repoUrl: string, filePath: string, branch: string): Promise<string>;
}

View File

@@ -0,0 +1,37 @@
/**
* Repository URL validation (JS mirror for server.js).
*/
const VALID_REPO_URL =
/^(https?:\/\/)(github\.com|gitlab\.com|bitbucket\.org|[^/]+)\/[^/]+\/[^/]+$/;
export const REPO_URL_ERROR_MESSAGE =
'Invalid repository URL. Supported: GitHub, GitLab, Bitbucket, and custom Git servers (e.g. https://host/owner/repo).';
export function isValidRepositoryUrl(url) {
if (typeof url !== 'string' || !url.trim()) return false;
return VALID_REPO_URL.test(url.trim());
}
export function getRepoProvider(url) {
if (!isValidRepositoryUrl(url)) throw new Error(REPO_URL_ERROR_MESSAGE);
const normalized = url.trim().toLowerCase();
if (normalized.includes('github.com')) return 'github';
if (normalized.includes('gitlab.com')) return 'gitlab';
if (normalized.includes('bitbucket.org')) return 'bitbucket';
return 'custom';
}
export function parseRepoUrl(url) {
if (!isValidRepositoryUrl(url)) throw new Error(REPO_URL_ERROR_MESSAGE);
try {
const u = new URL(url.trim());
const pathParts = u.pathname.replace(/^\/+/, '').replace(/\.git\/?$/, '').split('/');
return {
origin: u.origin,
owner: pathParts[0] ?? '',
repo: pathParts[1] ?? '',
};
} catch {
throw new Error(REPO_URL_ERROR_MESSAGE);
}
}

View File

@@ -0,0 +1,57 @@
/**
* Repository URL validation and provider detection.
* Supports GitHub, GitLab, Bitbucket, and custom Git servers.
*/
const VALID_REPO_URL =
/^(https?:\/\/)(github\.com|gitlab\.com|bitbucket\.org|[^/]+)\/[^/]+\/[^/]+$/;
export const REPO_URL_ERROR_MESSAGE =
'Invalid repository URL. Supported: GitHub, GitLab, Bitbucket, and custom Git servers (e.g. https://host/owner/repo).';
export type RepoProvider = 'github' | 'gitlab' | 'bitbucket' | 'custom';
/**
* Check if a string is a valid repository URL (format only).
*/
export function isValidRepositoryUrl(url: string): boolean {
if (typeof url !== 'string' || !url.trim()) return false;
return VALID_REPO_URL.test(url.trim());
}
/**
* Detect the Git provider from a repository URL.
*/
export function getRepoProvider(url: string): RepoProvider {
if (!isValidRepositoryUrl(url)) {
throw new Error(REPO_URL_ERROR_MESSAGE);
}
const normalized = url.trim().toLowerCase();
if (normalized.includes('github.com')) return 'github';
if (normalized.includes('gitlab.com')) return 'gitlab';
if (normalized.includes('bitbucket.org')) return 'bitbucket';
return 'custom';
}
/**
* Parse owner and repo from a repository URL (path segments).
* Works for GitHub, GitLab, Bitbucket, and custom (host/owner/repo).
*/
export function parseRepoUrl(url: string): { origin: string; owner: string; repo: string } {
if (!isValidRepositoryUrl(url)) {
throw new Error(REPO_URL_ERROR_MESSAGE);
}
try {
const u = new URL(url.trim());
const pathParts = u.pathname.replace(/^\/+/, '').replace(/\.git\/?$/, '').split('/');
const owner = pathParts[0] ?? '';
const repo = pathParts[1] ?? '';
return {
origin: u.origin,
owner,
repo,
};
} catch {
throw new Error(REPO_URL_ERROR_MESSAGE);
}
}

View File

@@ -327,13 +327,16 @@ class BackupService {
// PBS supports PBS_PASSWORD and PBS_REPOSITORY environment variables for non-interactive login
const repository = `root@pam@${pbsIp}:${pbsDatastore}`;
// Escape password for shell safety (single quotes)
// Escape password and fingerprint for shell safety (single quotes)
const escapedPassword = credential.pbs_password.replace(/'/g, "'\\''");
// Use PBS_PASSWORD environment variable for non-interactive authentication
// Auto-accept fingerprint by piping "y" to stdin
// PBS will use PBS_PASSWORD env var if available, avoiding interactive prompt
const fullCommand = `echo "y" | PBS_PASSWORD='${escapedPassword}' PBS_REPOSITORY='${repository}' timeout 10 proxmox-backup-client login --repository ${repository} 2>&1`;
const fingerprint = credential.pbs_fingerprint?.trim() ?? '';
const escapedFingerprint = fingerprint ? fingerprint.replace(/'/g, "'\\''") : '';
const envParts = [`PBS_PASSWORD='${escapedPassword}'`, `PBS_REPOSITORY='${repository}'`];
if (escapedFingerprint) {
envParts.push(`PBS_FINGERPRINT='${escapedFingerprint}'`);
}
const envStr = envParts.join(' ');
const fullCommand = `${envStr} timeout 10 proxmox-backup-client login --repository ${repository} 2>&1`;
console.log(`[BackupService] Logging into PBS: ${repository}`);
@@ -419,9 +422,12 @@ class BackupService {
// Build full repository string: root@pam@<IP>:<DATASTORE>
const repository = `root@pam@${pbsIp}:${pbsDatastore}`;
const fingerprint = credential.pbs_fingerprint?.trim() ?? '';
const escapedFingerprint = fingerprint ? fingerprint.replace(/'/g, "'\\''") : '';
const snapshotEnvParts = escapedFingerprint ? [`PBS_FINGERPRINT='${escapedFingerprint}'`] : [];
const snapshotEnvStr = snapshotEnvParts.length ? snapshotEnvParts.join(' ') + ' ' : '';
// Use correct command: snapshot list ct/<CT_ID> --repository <full_repo_string>
const command = `timeout 30 proxmox-backup-client snapshot list ct/${ctId} --repository ${repository} 2>&1 || echo "PBS_ERROR"`;
const command = `${snapshotEnvStr}timeout 30 proxmox-backup-client snapshot list ct/${ctId} --repository ${repository} 2>&1 || echo "PBS_ERROR"`;
let output = '';
console.log(`[BackupService] Discovering PBS backups for CT ${ctId} on repository ${repository}`);

View File

@@ -1,7 +1,8 @@
// JavaScript wrapper for githubJsonService (for use with node server.js)
import { writeFile, mkdir, readdir, readFile } from 'fs/promises';
import { writeFile, mkdir, readdir, readFile, unlink } from 'fs/promises';
import { join } from 'path';
import { repositoryService } from './repositoryService.js';
import { listDirectory, downloadRawFile } from '../lib/gitProvider/index.js';
// Get environment variables
const getEnv = () => ({
@@ -28,76 +29,9 @@ class GitHubJsonService {
}
}
getBaseUrl(repoUrl) {
const urlMatch = /github\.com\/([^\/]+)\/([^\/]+)/.exec(repoUrl);
if (!urlMatch) {
throw new Error(`Invalid GitHub repository URL: ${repoUrl}`);
}
const [, owner, repo] = urlMatch;
return `https://api.github.com/repos/${owner}/${repo}`;
}
extractRepoPath(repoUrl) {
const match = /github\.com\/([^\/]+)\/([^\/]+)/.exec(repoUrl);
if (!match) {
throw new Error('Invalid GitHub repository URL');
}
return `${match[1]}/${match[2]}`;
}
async fetchFromGitHub(repoUrl, endpoint) {
const baseUrl = this.getBaseUrl(repoUrl);
const env = getEnv();
const headers = {
'Accept': 'application/vnd.github.v3+json',
'User-Agent': 'PVEScripts-Local/1.0',
};
if (env.GITHUB_TOKEN) {
headers.Authorization = `token ${env.GITHUB_TOKEN}`;
}
const response = await fetch(`${baseUrl}${endpoint}`, { headers });
if (!response.ok) {
if (response.status === 403) {
const error = new Error(`GitHub API rate limit exceeded. Consider setting GITHUB_TOKEN for higher limits. Status: ${response.status} ${response.statusText}`);
error.name = 'RateLimitError';
throw error;
}
throw new Error(`GitHub API error: ${response.status} ${response.statusText}`);
}
return response.json();
}
async downloadJsonFile(repoUrl, filePath) {
this.initializeConfig();
const repoPath = this.extractRepoPath(repoUrl);
const rawUrl = `https://raw.githubusercontent.com/${repoPath}/${this.branch}/${filePath}`;
const env = getEnv();
const headers = {
'User-Agent': 'PVEScripts-Local/1.0',
};
if (env.GITHUB_TOKEN) {
headers.Authorization = `token ${env.GITHUB_TOKEN}`;
}
const response = await fetch(rawUrl, { headers });
if (!response.ok) {
if (response.status === 403) {
const error = new Error(`GitHub rate limit exceeded while downloading ${filePath}. Consider setting GITHUB_TOKEN for higher limits.`);
error.name = 'RateLimitError';
throw error;
}
throw new Error(`Failed to download ${filePath}: ${response.status} ${response.statusText}`);
}
const content = await response.text();
const content = await downloadRawFile(repoUrl, filePath, this.branch);
const script = JSON.parse(content);
script.repository_url = repoUrl;
return script;
@@ -105,16 +39,13 @@ class GitHubJsonService {
async getJsonFiles(repoUrl) {
this.initializeConfig();
try {
const files = await this.fetchFromGitHub(
repoUrl,
`/contents/${this.jsonFolder}?ref=${this.branch}`
);
return files.filter(file => file.name.endsWith('.json'));
const entries = await listDirectory(repoUrl, this.jsonFolder, this.branch);
return entries
.filter((e) => e.type === 'file' && e.name.endsWith('.json'))
.map((e) => ({ name: e.name, path: e.path }));
} catch (error) {
console.error(`Error fetching JSON files from GitHub (${repoUrl}):`, error);
console.error(`Error fetching JSON files from repository (${repoUrl}):`, error);
throw new Error(`Failed to fetch script files from repository: ${repoUrl}`);
}
}
@@ -232,25 +163,42 @@ class GitHubJsonService {
const localFiles = await this.getLocalJsonFiles();
console.log(`Found ${localFiles.length} local JSON files`);
// Delete local JSON files that belong to this repo but are no longer in the remote
const remoteFilenames = new Set(githubFiles.map((f) => f.name));
const deletedFiles = await this.deleteLocalFilesRemovedFromRepo(repoUrl, remoteFilenames);
if (deletedFiles.length > 0) {
console.log(`Removed ${deletedFiles.length} obsolete JSON file(s) no longer in ${repoUrl}`);
}
const filesToSync = await this.findFilesToSyncForRepo(repoUrl, githubFiles, localFiles);
console.log(`Found ${filesToSync.length} files that need syncing from ${repoUrl}`);
if (filesToSync.length === 0) {
const msg =
deletedFiles.length > 0
? `All JSON files are up to date for repository: ${repoUrl}. Removed ${deletedFiles.length} obsolete file(s).`
: `All JSON files are up to date for repository: ${repoUrl}`;
return {
success: true,
message: `All JSON files are up to date for repository: ${repoUrl}`,
message: msg,
count: 0,
syncedFiles: []
syncedFiles: [],
deletedFiles
};
}
const syncedFiles = await this.syncSpecificFiles(repoUrl, filesToSync);
const msg =
deletedFiles.length > 0
? `Successfully synced ${syncedFiles.length} JSON files from ${repoUrl}, removed ${deletedFiles.length} obsolete file(s).`
: `Successfully synced ${syncedFiles.length} JSON files from ${repoUrl}`;
return {
success: true,
message: `Successfully synced ${syncedFiles.length} JSON files from ${repoUrl}`,
message: msg,
count: syncedFiles.length,
syncedFiles
syncedFiles,
deletedFiles
};
} catch (error) {
console.error(`JSON sync failed for ${repoUrl}:`, error);
@@ -258,7 +206,8 @@ class GitHubJsonService {
success: false,
message: `Failed to sync JSON files from ${repoUrl}: ${error instanceof Error ? error.message : 'Unknown error'}`,
count: 0,
syncedFiles: []
syncedFiles: [],
deletedFiles: []
};
}
}
@@ -274,13 +223,15 @@ class GitHubJsonService {
success: false,
message: 'No enabled repositories found',
count: 0,
syncedFiles: []
syncedFiles: [],
deletedFiles: []
};
}
console.log(`Found ${enabledRepos.length} enabled repositories`);
const allSyncedFiles = [];
const allDeletedFiles = [];
const processedSlugs = new Set();
let totalSynced = 0;
@@ -291,6 +242,7 @@ class GitHubJsonService {
const result = await this.syncJsonFilesForRepo(repo.url);
if (result.success) {
allDeletedFiles.push(...(result.deletedFiles ?? []));
const newFiles = result.syncedFiles.filter(file => {
const slug = file.replace('.json', '');
if (processedSlugs.has(slug)) {
@@ -312,11 +264,16 @@ class GitHubJsonService {
await this.updateExistingFilesWithRepositoryUrl();
const msg =
allDeletedFiles.length > 0
? `Successfully synced ${totalSynced} JSON files from ${enabledRepos.length} repositories, removed ${allDeletedFiles.length} obsolete file(s).`
: `Successfully synced ${totalSynced} JSON files from ${enabledRepos.length} repositories`;
return {
success: true,
message: `Successfully synced ${totalSynced} JSON files from ${enabledRepos.length} repositories`,
message: msg,
count: totalSynced,
syncedFiles: allSyncedFiles
syncedFiles: allSyncedFiles,
deletedFiles: allDeletedFiles
};
} catch (error) {
console.error('Multi-repository JSON sync failed:', error);
@@ -324,7 +281,8 @@ class GitHubJsonService {
success: false,
message: `Failed to sync JSON files: ${error instanceof Error ? error.message : 'Unknown error'}`,
count: 0,
syncedFiles: []
syncedFiles: [],
deletedFiles: []
};
}
}
@@ -366,6 +324,32 @@ class GitHubJsonService {
}
}
async deleteLocalFilesRemovedFromRepo(repoUrl, remoteFilenames) {
this.initializeConfig();
const localFiles = await this.getLocalJsonFiles();
const deletedFiles = [];
for (const file of localFiles) {
try {
const filePath = join(this.localJsonDirectory, file);
const content = await readFile(filePath, 'utf-8');
const script = JSON.parse(content);
if (script.repository_url === repoUrl && !remoteFilenames.has(file)) {
await unlink(filePath);
const slug = file.replace(/\.json$/, '');
this.scriptCache.delete(slug);
deletedFiles.push(file);
console.log(`Removed obsolete script JSON: ${file} (no longer in ${repoUrl})`);
}
} catch {
// If we can't read or parse the file, skip (do not delete)
}
}
return deletedFiles;
}
async findFilesToSyncForRepo(repoUrl, githubFiles, localFiles) {
const filesToSync = [];

View File

@@ -1,8 +1,9 @@
import { writeFile, mkdir, readdir, readFile } from 'fs/promises';
import { writeFile, mkdir, readdir, readFile, unlink } from 'fs/promises';
import { join } from 'path';
import { env } from '../../env.js';
import type { Script, ScriptCard, GitHubFile } from '../../types/script';
import { repositoryService } from './repositoryService';
import { listDirectory, downloadRawFile } from '~/server/lib/gitProvider';
export class GitHubJsonService {
private branch: string | null = null;
@@ -22,96 +23,24 @@ export class GitHubJsonService {
}
}
private getBaseUrl(repoUrl: string): string {
const urlMatch = /github\.com\/([^\/]+)\/([^\/]+)/.exec(repoUrl);
if (!urlMatch) {
throw new Error(`Invalid GitHub repository URL: ${repoUrl}`);
}
const [, owner, repo] = urlMatch;
return `https://api.github.com/repos/${owner}/${repo}`;
}
private extractRepoPath(repoUrl: string): string {
const match = /github\.com\/([^\/]+)\/([^\/]+)/.exec(repoUrl);
if (!match) {
throw new Error('Invalid GitHub repository URL');
}
return `${match[1]}/${match[2]}`;
}
private async fetchFromGitHub<T>(repoUrl: string, endpoint: string): Promise<T> {
const baseUrl = this.getBaseUrl(repoUrl);
const headers: HeadersInit = {
'Accept': 'application/vnd.github.v3+json',
'User-Agent': 'PVEScripts-Local/1.0',
};
// Add GitHub token authentication if available
if (env.GITHUB_TOKEN) {
headers.Authorization = `token ${env.GITHUB_TOKEN}`;
}
const response = await fetch(`${baseUrl}${endpoint}`, { headers });
if (!response.ok) {
if (response.status === 403) {
const error = new Error(`GitHub API rate limit exceeded. Consider setting GITHUB_TOKEN for higher limits. Status: ${response.status} ${response.statusText}`);
error.name = 'RateLimitError';
throw error;
}
throw new Error(`GitHub API error: ${response.status} ${response.statusText}`);
}
const data = await response.json();
return data as T;
}
private async downloadJsonFile(repoUrl: string, filePath: string): Promise<Script> {
this.initializeConfig();
const repoPath = this.extractRepoPath(repoUrl);
const rawUrl = `https://raw.githubusercontent.com/${repoPath}/${this.branch!}/${filePath}`;
const headers: HeadersInit = {
'User-Agent': 'PVEScripts-Local/1.0',
};
// Add GitHub token authentication if available
if (env.GITHUB_TOKEN) {
headers.Authorization = `token ${env.GITHUB_TOKEN}`;
}
const response = await fetch(rawUrl, { headers });
if (!response.ok) {
if (response.status === 403) {
const error = new Error(`GitHub rate limit exceeded while downloading ${filePath}. Consider setting GITHUB_TOKEN for higher limits. Status: ${response.status} ${response.statusText}`);
error.name = 'RateLimitError';
throw error;
}
throw new Error(`Failed to download ${filePath}: ${response.status} ${response.statusText}`);
}
const content = await response.text();
const content = await downloadRawFile(repoUrl, filePath, this.branch!);
const script = JSON.parse(content) as Script;
// Add repository_url to script
script.repository_url = repoUrl;
return script;
}
async getJsonFiles(repoUrl: string): Promise<GitHubFile[]> {
this.initializeConfig();
try {
const files = await this.fetchFromGitHub<GitHubFile[]>(
repoUrl,
`/contents/${this.jsonFolder!}?ref=${this.branch!}`
);
// Filter for JSON files only
return files.filter(file => file.name.endsWith('.json'));
const entries = await listDirectory(repoUrl, this.jsonFolder!, this.branch!);
const files: GitHubFile[] = entries
.filter((e) => e.type === 'file' && e.name.endsWith('.json'))
.map((e) => ({ name: e.name, path: e.path } as GitHubFile));
return files;
} catch (error) {
console.error(`Error fetching JSON files from GitHub (${repoUrl}):`, error);
console.error(`Error fetching JSON files from repository (${repoUrl}):`, error);
throw new Error(`Failed to fetch script files from repository: ${repoUrl}`);
}
}
@@ -229,12 +158,11 @@ export class GitHubJsonService {
/**
* Sync JSON files from a specific repository
*/
async syncJsonFilesForRepo(repoUrl: string): Promise<{ success: boolean; message: string; count: number; syncedFiles: string[] }> {
async syncJsonFilesForRepo(repoUrl: string): Promise<{ success: boolean; message: string; count: number; syncedFiles: string[]; deletedFiles: string[] }> {
try {
console.log(`Starting JSON sync from repository: ${repoUrl}`);
// Get file list from GitHub
console.log(`Fetching file list from GitHub (${repoUrl})...`);
console.log(`Fetching file list from repository (${repoUrl})...`);
const githubFiles = await this.getJsonFiles(repoUrl);
console.log(`Found ${githubFiles.length} JSON files in repository ${repoUrl}`);
@@ -242,28 +170,45 @@ export class GitHubJsonService {
const localFiles = await this.getLocalJsonFiles();
console.log(`Found ${localFiles.length} local JSON files`);
// Delete local JSON files that belong to this repo but are no longer in the remote
const remoteFilenames = new Set(githubFiles.map((f) => f.name));
const deletedFiles = await this.deleteLocalFilesRemovedFromRepo(repoUrl, remoteFilenames);
if (deletedFiles.length > 0) {
console.log(`Removed ${deletedFiles.length} obsolete JSON file(s) no longer in ${repoUrl}`);
}
// Compare and find files that need syncing
// For multi-repo support, we need to check if file exists AND if it's from this repo
const filesToSync = await this.findFilesToSyncForRepo(repoUrl, githubFiles, localFiles);
console.log(`Found ${filesToSync.length} files that need syncing from ${repoUrl}`);
if (filesToSync.length === 0) {
const msg =
deletedFiles.length > 0
? `All JSON files are up to date for repository: ${repoUrl}. Removed ${deletedFiles.length} obsolete file(s).`
: `All JSON files are up to date for repository: ${repoUrl}`;
return {
success: true,
message: `All JSON files are up to date for repository: ${repoUrl}`,
message: msg,
count: 0,
syncedFiles: []
syncedFiles: [],
deletedFiles
};
}
// Download and save only the files that need syncing
const syncedFiles = await this.syncSpecificFiles(repoUrl, filesToSync);
const msg =
deletedFiles.length > 0
? `Successfully synced ${syncedFiles.length} JSON files from ${repoUrl}, removed ${deletedFiles.length} obsolete file(s).`
: `Successfully synced ${syncedFiles.length} JSON files from ${repoUrl}`;
return {
success: true,
message: `Successfully synced ${syncedFiles.length} JSON files from ${repoUrl}`,
message: msg,
count: syncedFiles.length,
syncedFiles
syncedFiles,
deletedFiles
};
} catch (error) {
console.error(`JSON sync failed for ${repoUrl}:`, error);
@@ -271,7 +216,8 @@ export class GitHubJsonService {
success: false,
message: `Failed to sync JSON files from ${repoUrl}: ${error instanceof Error ? error.message : 'Unknown error'}`,
count: 0,
syncedFiles: []
syncedFiles: [],
deletedFiles: []
};
}
}
@@ -279,7 +225,7 @@ export class GitHubJsonService {
/**
* Sync JSON files from all enabled repositories (main repo has priority)
*/
async syncJsonFiles(): Promise<{ success: boolean; message: string; count: number; syncedFiles: string[] }> {
async syncJsonFiles(): Promise<{ success: boolean; message: string; count: number; syncedFiles: string[]; deletedFiles: string[] }> {
try {
console.log('Starting multi-repository JSON sync...');
@@ -290,13 +236,15 @@ export class GitHubJsonService {
success: false,
message: 'No enabled repositories found',
count: 0,
syncedFiles: []
syncedFiles: [],
deletedFiles: []
};
}
console.log(`Found ${enabledRepos.length} enabled repositories`);
const allSyncedFiles: string[] = [];
const allDeletedFiles: string[] = [];
const processedSlugs = new Set<string>(); // Track slugs we've already processed
let totalSynced = 0;
@@ -308,6 +256,7 @@ export class GitHubJsonService {
const result = await this.syncJsonFilesForRepo(repo.url);
if (result.success) {
allDeletedFiles.push(...(result.deletedFiles ?? []));
// Only count files that weren't already processed from a higher priority repo
const newFiles = result.syncedFiles.filter(file => {
const slug = file.replace('.json', '');
@@ -331,11 +280,16 @@ export class GitHubJsonService {
// Also update existing files that don't have repository_url set (backward compatibility)
await this.updateExistingFilesWithRepositoryUrl();
const msg =
allDeletedFiles.length > 0
? `Successfully synced ${totalSynced} JSON files from ${enabledRepos.length} repositories, removed ${allDeletedFiles.length} obsolete file(s).`
: `Successfully synced ${totalSynced} JSON files from ${enabledRepos.length} repositories`;
return {
success: true,
message: `Successfully synced ${totalSynced} JSON files from ${enabledRepos.length} repositories`,
message: msg,
count: totalSynced,
syncedFiles: allSyncedFiles
syncedFiles: allSyncedFiles,
deletedFiles: allDeletedFiles
};
} catch (error) {
console.error('Multi-repository JSON sync failed:', error);
@@ -343,7 +297,8 @@ export class GitHubJsonService {
success: false,
message: `Failed to sync JSON files: ${error instanceof Error ? error.message : 'Unknown error'}`,
count: 0,
syncedFiles: []
syncedFiles: [],
deletedFiles: []
};
}
}
@@ -388,6 +343,36 @@ export class GitHubJsonService {
}
}
/**
* Delete local JSON files that belong to this repo but are no longer in the remote list.
* Returns the list of deleted filenames.
*/
private async deleteLocalFilesRemovedFromRepo(repoUrl: string, remoteFilenames: Set<string>): Promise<string[]> {
this.initializeConfig();
const localFiles = await this.getLocalJsonFiles();
const deletedFiles: string[] = [];
for (const file of localFiles) {
try {
const filePath = join(this.localJsonDirectory!, file);
const content = await readFile(filePath, 'utf-8');
const script = JSON.parse(content) as Script;
if (script.repository_url === repoUrl && !remoteFilenames.has(file)) {
await unlink(filePath);
const slug = file.replace(/\.json$/, '');
this.scriptCache.delete(slug);
deletedFiles.push(file);
console.log(`Removed obsolete script JSON: ${file} (no longer in ${repoUrl})`);
}
} catch {
// If we can't read or parse the file, skip (do not delete)
}
}
return deletedFiles;
}
/**
* Find files that need syncing for a specific repository
* This checks if file exists locally AND if it's from the same repository

View File

@@ -1,5 +1,6 @@
// JavaScript wrapper for repositoryService (for use with node server.js)
import { prisma } from '../db.js';
import { isValidRepositoryUrl, REPO_URL_ERROR_MESSAGE } from '../lib/repositoryUrlValidation.js';
class RepositoryService {
/**
@@ -89,9 +90,8 @@ class RepositoryService {
* Create a new repository
*/
async createRepository(data) {
// Validate GitHub URL
if (!data.url.match(/^https:\/\/github\.com\/[^\/]+\/[^\/]+$/)) {
throw new Error('Invalid GitHub repository URL. Format: https://github.com/owner/repo');
if (!isValidRepositoryUrl(data.url)) {
throw new Error(REPO_URL_ERROR_MESSAGE);
}
// Check for duplicates
@@ -122,10 +122,9 @@ class RepositoryService {
* Update repository
*/
async updateRepository(id, data) {
// If updating URL, validate it
if (data.url) {
if (!data.url.match(/^https:\/\/github\.com\/[^\/]+\/[^\/]+$/)) {
throw new Error('Invalid GitHub repository URL. Format: https://github.com/owner/repo');
if (!isValidRepositoryUrl(data.url)) {
throw new Error(REPO_URL_ERROR_MESSAGE);
}
// Check for duplicates (excluding current repo)

View File

@@ -1,5 +1,5 @@
/* eslint-disable @typescript-eslint/prefer-regexp-exec */
import { prisma } from '../db';
import { isValidRepositoryUrl, REPO_URL_ERROR_MESSAGE } from '../lib/repositoryUrlValidation';
export class RepositoryService {
/**
@@ -93,9 +93,8 @@ export class RepositoryService {
enabled?: boolean;
priority?: number;
}) {
// Validate GitHub URL
if (!data.url.match(/^https:\/\/github\.com\/[^\/]+\/[^\/]+$/)) {
throw new Error('Invalid GitHub repository URL. Format: https://github.com/owner/repo');
if (!isValidRepositoryUrl(data.url)) {
throw new Error(REPO_URL_ERROR_MESSAGE);
}
// Check for duplicates
@@ -130,10 +129,9 @@ export class RepositoryService {
url?: string;
priority?: number;
}) {
// If updating URL, validate it
if (data.url) {
if (!data.url.match(/^https:\/\/github\.com\/[^\/]+\/[^\/]+$/)) {
throw new Error('Invalid GitHub repository URL. Format: https://github.com/owner/repo');
if (!isValidRepositoryUrl(data.url)) {
throw new Error(REPO_URL_ERROR_MESSAGE);
}
// Check for duplicates (excluding current repo)

View File

@@ -250,9 +250,16 @@ class RestoreService {
const targetFolder = `/var/lib/vz/dump/vzdump-lxc-${ctId}-${snapshotNameForPath}`;
const targetTar = `${targetFolder}.tar`;
// Use PBS_PASSWORD env var and add timeout for long downloads
// Use PBS_PASSWORD env var and add timeout for long downloads; PBS_FINGERPRINT when set for cert validation
const escapedPassword = credential.pbs_password.replace(/'/g, "'\\''");
const restoreCommand = `PBS_PASSWORD='${escapedPassword}' PBS_REPOSITORY='${repository}' timeout 300 proxmox-backup-client restore "${snapshotPath}" root.pxar "${targetFolder}" --repository '${repository}' 2>&1`;
const fingerprint = credential.pbs_fingerprint?.trim() ?? '';
const escapedFingerprint = fingerprint ? fingerprint.replace(/'/g, "'\\''") : '';
const restoreEnvParts = [`PBS_PASSWORD='${escapedPassword}'`, `PBS_REPOSITORY='${repository}'`];
if (escapedFingerprint) {
restoreEnvParts.push(`PBS_FINGERPRINT='${escapedFingerprint}'`);
}
const restoreEnvStr = restoreEnvParts.join(' ');
const restoreCommand = `${restoreEnvStr} timeout 300 proxmox-backup-client restore "${snapshotPath}" root.pxar "${targetFolder}" --repository '${repository}' 2>&1`;
let output = '';
let exitCode = 0;

View File

@@ -1,6 +1,7 @@
// Real JavaScript implementation for script downloading
import { join } from 'path';
import { writeFile, mkdir, access, readFile, unlink } from 'fs/promises';
import { downloadRawFile } from '../lib/gitProvider/index.js';
export class ScriptDownloaderService {
constructor() {
@@ -82,51 +83,18 @@ export class ScriptDownloaderService {
}
/**
* Extract repository path from GitHub URL
* @param {string} repoUrl - The GitHub repository URL
* @returns {string}
*/
extractRepoPath(repoUrl) {
const match = /github\.com\/([^\/]+)\/([^\/]+)/.exec(repoUrl);
if (!match) {
throw new Error(`Invalid GitHub repository URL: ${repoUrl}`);
}
return `${match[1]}/${match[2]}`;
}
/**
* Download a file from GitHub
* @param {string} repoUrl - The GitHub repository URL
* Download a file from the repository (GitHub, GitLab, Bitbucket, or custom)
* @param {string} repoUrl - The repository URL
* @param {string} filePath - The file path within the repository
* @param {string} [branch] - The branch to download from
* @returns {Promise<string>}
*/
async downloadFileFromGitHub(repoUrl, filePath, branch = 'main') {
this.initializeConfig();
async downloadFileFromRepo(repoUrl, filePath, branch = 'main') {
if (!repoUrl) {
throw new Error('Repository URL is not set');
}
const repoPath = this.extractRepoPath(repoUrl);
const url = `https://raw.githubusercontent.com/${repoPath}/${branch}/${filePath}`;
/** @type {Record<string, string>} */
const headers = {
'User-Agent': 'PVEScripts-Local/1.0',
};
// Add GitHub token authentication if available
if (process.env.GITHUB_TOKEN) {
headers.Authorization = `token ${process.env.GITHUB_TOKEN}`;
}
console.log(`Downloading from GitHub: ${url}`);
const response = await fetch(url, { headers });
if (!response.ok) {
throw new Error(`Failed to download ${filePath} from ${repoUrl}: ${response.status} ${response.statusText}`);
}
return response.text();
console.log(`Downloading from repository: ${repoUrl} (${filePath})`);
return downloadRawFile(repoUrl, filePath, branch);
}
/**
@@ -184,9 +152,8 @@ export class ScriptDownloaderService {
const fileName = scriptPath.split('/').pop();
if (fileName) {
// Download from GitHub using the script's repository URL
console.log(`Downloading script file: ${scriptPath} from ${repoUrl}`);
const content = await this.downloadFileFromGitHub(repoUrl, scriptPath, branch);
const content = await this.downloadFileFromRepo(repoUrl, scriptPath, branch);
// Determine target directory based on script path
let targetDir;
@@ -250,7 +217,7 @@ export class ScriptDownloaderService {
const installScriptName = `${script.slug}-install.sh`;
try {
console.log(`Downloading install script: install/${installScriptName} from ${repoUrl}`);
const installContent = await this.downloadFileFromGitHub(repoUrl, `install/${installScriptName}`, branch);
const installContent = await this.downloadFileFromRepo(repoUrl, `install/${installScriptName}`, branch);
const localInstallPath = join(this.scriptsDirectory, 'install', installScriptName);
await writeFile(localInstallPath, installContent, 'utf-8');
files.push(`install/${installScriptName}`);
@@ -274,7 +241,7 @@ export class ScriptDownloaderService {
const alpineInstallScriptName = `alpine-${script.slug}-install.sh`;
try {
console.log(`[${script.slug}] Downloading alpine install script: install/${alpineInstallScriptName} from ${repoUrl}`);
const alpineInstallContent = await this.downloadFileFromGitHub(repoUrl, `install/${alpineInstallScriptName}`, branch);
const alpineInstallContent = await this.downloadFileFromRepo(repoUrl, `install/${alpineInstallScriptName}`, branch);
const localAlpineInstallPath = join(this.scriptsDirectory, 'install', alpineInstallScriptName);
await writeFile(localAlpineInstallPath, alpineInstallContent, 'utf-8');
files.push(`install/${alpineInstallScriptName}`);
@@ -681,7 +648,7 @@ export class ScriptDownloaderService {
console.log(`[Comparison] Local file size: ${localContent.length} bytes`);
// Download remote content from the script's repository
const remoteContent = await this.downloadFileFromGitHub(repoUrl, remotePath, branch);
const remoteContent = await this.downloadFileFromRepo(repoUrl, remotePath, branch);
console.log(`[Comparison] Remote file size: ${remoteContent.length} bytes`);
// Apply modification only for CT scripts, not for other script types
@@ -739,7 +706,7 @@ export class ScriptDownloaderService {
// Find the corresponding script path in install_methods
const method = script.install_methods?.find(m => m.script === filePath);
if (method?.script) {
const downloadedContent = await this.downloadFileFromGitHub(repoUrl, method.script, branch);
const downloadedContent = await this.downloadFileFromRepo(repoUrl, method.script, branch);
remoteContent = this.modifyScriptContent(downloadedContent);
}
} catch {
@@ -756,7 +723,7 @@ export class ScriptDownloaderService {
}
try {
remoteContent = await this.downloadFileFromGitHub(repoUrl, filePath, branch);
remoteContent = await this.downloadFileFromRepo(repoUrl, filePath, branch);
} catch {
// Error downloading remote install script
}

View File

@@ -1,6 +1,8 @@
import { spawn } from 'child_process';
import { spawn as ptySpawn } from 'node-pty';
import { existsSync } from 'fs';
import { existsSync, writeFileSync, chmodSync, unlinkSync } from 'fs';
import { join } from 'path';
import { tmpdir } from 'os';
/**
@@ -194,26 +196,45 @@ class SSHExecutionService {
*/
async transferScriptsFolder(server, onData, onError) {
const { ip, user, password, auth_type = 'password', ssh_key_passphrase, ssh_key_path, ssh_port = 22 } = server;
const cleanupTempFile = (/** @type {string | null} */ tempPath) => {
if (tempPath) {
try {
unlinkSync(tempPath);
} catch (_) {
// ignore
}
}
};
return new Promise((resolve, reject) => {
/** @type {string | null} */
let tempPath = null;
try {
// Build rsync command based on authentication type
// Build rsync command based on authentication type.
// Use sshpass -f with a temp file so password/passphrase never go through the shell (safe for special chars like {, $, ").
let rshCommand;
if (auth_type === 'key') {
if (!ssh_key_path || !existsSync(ssh_key_path)) {
throw new Error('SSH key file not found');
}
if (ssh_key_passphrase) {
rshCommand = `sshpass -P passphrase -p ${ssh_key_passphrase} ssh -i ${ssh_key_path} -p ${ssh_port} -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null`;
tempPath = join(tmpdir(), `sshpass-${process.pid}-${Date.now()}.tmp`);
writeFileSync(tempPath, ssh_key_passphrase);
chmodSync(tempPath, 0o600);
rshCommand = `sshpass -P passphrase -f ${tempPath} ssh -i ${ssh_key_path} -p ${ssh_port} -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null`;
} else {
rshCommand = `ssh -i ${ssh_key_path} -p ${ssh_port} -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null`;
}
} else {
// Password authentication
rshCommand = `sshpass -p ${password} ssh -p ${ssh_port} -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null`;
tempPath = join(tmpdir(), `sshpass-${process.pid}-${Date.now()}.tmp`);
writeFileSync(tempPath, password ?? '');
chmodSync(tempPath, 0o600);
rshCommand = `sshpass -f ${tempPath} ssh -p ${ssh_port} -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null`;
}
const rsyncCommand = spawn('rsync', [
'-avz',
'--delete',
@@ -226,31 +247,31 @@ class SSHExecutionService {
stdio: ['pipe', 'pipe', 'pipe']
});
rsyncCommand.stdout.on('data', (/** @type {Buffer} */ data) => {
// Ensure proper UTF-8 encoding for ANSI colors
const output = data.toString('utf8');
onData(output);
});
rsyncCommand.stdout.on('data', (/** @type {Buffer} */ data) => {
const output = data.toString('utf8');
onData(output);
});
rsyncCommand.stderr.on('data', (/** @type {Buffer} */ data) => {
// Ensure proper UTF-8 encoding for ANSI colors
const output = data.toString('utf8');
onError(output);
});
rsyncCommand.stderr.on('data', (/** @type {Buffer} */ data) => {
const output = data.toString('utf8');
onError(output);
});
rsyncCommand.on('close', (code) => {
if (code === 0) {
resolve();
} else {
reject(new Error(`rsync failed with code ${code}`));
}
});
rsyncCommand.on('close', (code) => {
cleanupTempFile(tempPath);
if (code === 0) {
resolve();
} else {
reject(new Error(`rsync failed with code ${code}`));
}
});
rsyncCommand.on('error', (error) => {
reject(error);
});
rsyncCommand.on('error', (error) => {
cleanupTempFile(tempPath);
reject(error);
});
} catch (error) {
cleanupTempFile(tempPath);
reject(error);
}
});

View File

@@ -169,16 +169,17 @@ class SSHService {
const timeout = 10000;
let resolved = false;
// Pass password via env so it is not embedded in the script (safe for special chars like {, $, ").
const expectScript = `#!/usr/bin/expect -f
set timeout 10
spawn ssh -p ${ssh_port} -o ConnectTimeout=10 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=ERROR -o PasswordAuthentication=yes -o PubkeyAuthentication=no ${user}@${ip} "echo SSH_LOGIN_SUCCESS"
expect {
"password:" {
send "${password}\r"
send "$env(SSH_PASSWORD)\\r"
exp_continue
}
"Password:" {
send "${password}\r"
send "$env(SSH_PASSWORD)\\r"
exp_continue
}
"SSH_LOGIN_SUCCESS" {
@@ -193,7 +194,8 @@ expect {
}`;
const expectCommand = spawn('expect', ['-c', expectScript], {
stdio: ['pipe', 'pipe', 'pipe']
stdio: ['pipe', 'pipe', 'pipe'],
env: { ...process.env, SSH_PASSWORD: password ?? '' }
});
const timer = setTimeout(() => {