S1: Einrichtungs-Assistent, Twitch-Auth, Datenbank und Chat-Aufnahme
Zugangsdaten kommen jetzt aus der App statt aus einer .env: ein Einrichtungs-Assistent auf http://localhost:4792/setup nimmt Client-ID, Secret und OBS-Passwort entgegen, der OAuth-Rueckruf laeuft auf denselben Server. Gespeichert wird verschlueselt (AES-256-GCM) in einem Volume, bewusst nicht in der Datenbank - die wird auf die Windows-Seite gesichert. Siehe ADR-0005. Technikbasis vereinfacht: Node 24 bringt node:sqlite, WebSocket, fetch und die Ausfuehrung von TypeScript mit. Das Projekt hat damit null Laufzeitabhaengigkeiten und keinen Build-Schritt; pnpm-Workspaces und die Paketmanifeste sind entfallen, Importe laufen ueber die imports-Map der Wurzel-package.json. erasableSyntaxOnly verhindert Syntax, die das Type Stripping stillschweigend entfernen wuerde. Siehe Nachtrag 2 in ADR-0001. Enthalten: - packages/core: Config, Layout, Logger mit Secret-Maskierung, Retry, YAML - packages/db: Migrationen, Repositories, alle SQL-Anweisungen an einem Ort - packages/twitch: Secret-Speicher, OAuth, Helix mit Token-Bucket, IRC - apps/ingest: Daemon, Einrichtungsseite, Spike-Erkennung - 20 Tests (IRC-Parser, Spike-Detektor, Datenbankschicht) Der Chat verbindet sich anonym, solange nichts hinterlegt ist - damit laesst sich die Datenaufnahme pruefen, bevor die Twitch-Apps existieren. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013VTGjd92LsDDWF3frd9FTB
This commit is contained in:
1 parent
4ceb61f095
commit
d07c18441b
82 files changed
+2775
-838
No files matched your search
+10
-38
@@ -1,41 +1,13 @@
|
|||||||
# Kopie als .env anlegen. Diese Datei enthaelt Secrets und gehoert nie ins Repo.
|
# Diese Datei ist OPTIONAL. Ohne sie startet alles.
|
||||||
|
#
|
||||||
|
# Zugangsdaten (Twitch, OBS, Discord, Bluesky) werden NICHT hier eingetragen,
|
||||||
|
# sondern im Einrichtungs-Assistenten unter http://localhost:4792/setup
|
||||||
|
# Begruendung: docs/decisions/ADR-0005-zugangsdaten.md
|
||||||
|
|
||||||
# Twitch-App fuer den Broadcaster-Account (abdulive_)
|
# Windows-Pfad zum OBS-Wiedergabepuffer-Ordner (fuer den Bind-Mount).
|
||||||
TWITCH_CLIENT_ID=
|
|
||||||
TWITCH_CLIENT_SECRET=
|
|
||||||
|
|
||||||
# Twitch-App fuer das Bot-Konto
|
|
||||||
TWITCH_BOT_CLIENT_ID=
|
|
||||||
TWITCH_BOT_CLIENT_SECRET=
|
|
||||||
|
|
||||||
# OAuth-Redirect (muss in beiden Apps eingetragen sein)
|
|
||||||
TWITCH_REDIRECT_URI=http://localhost:4780/callback
|
|
||||||
|
|
||||||
# obs-websocket v5
|
|
||||||
OBS_WS_URL=ws://127.0.0.1:4455
|
|
||||||
OBS_WS_PASSWORD=
|
|
||||||
|
|
||||||
# Discord-Webhooks
|
|
||||||
DISCORD_WEBHOOK_GOLIVE=
|
|
||||||
DISCORD_WEBHOOK_RECAP=
|
|
||||||
DISCORD_WEBHOOK_INTERNAL=
|
|
||||||
|
|
||||||
# Bluesky
|
|
||||||
BLUESKY_HANDLE=
|
|
||||||
BLUESKY_APP_PASSWORD=
|
|
||||||
|
|
||||||
# Mastodon (optional)
|
|
||||||
MASTODON_INSTANCE=
|
|
||||||
MASTODON_TOKEN=
|
|
||||||
|
|
||||||
# Push-Benachrichtigung aufs Handy (optional, ntfy.sh)
|
|
||||||
NTFY_TOPIC=
|
|
||||||
|
|
||||||
# Schluessel fuer die Token-Verschluesselung in data/db/tokens.json
|
|
||||||
TOKEN_ENCRYPTION_KEY=
|
|
||||||
|
|
||||||
LOG_LEVEL=info
|
|
||||||
|
|
||||||
# Windows-Pfad zum OBS-Wiedergabepuffer-Ordner (fuer den Bind-Mount)
|
|
||||||
# Forward-Slashes verwenden, z. B. C:/Users/abdu/Videos/replay
|
# Forward-Slashes verwenden, z. B. C:/Users/abdu/Videos/replay
|
||||||
|
# Ohne Angabe wird ./data/replay benutzt.
|
||||||
OBS_REPLAY_DIR=
|
OBS_REPLAY_DIR=
|
||||||
|
|
||||||
|
# debug | info | warn | error
|
||||||
|
LOG_LEVEL=info
|
||||||
@@ -4,6 +4,10 @@ Werkzeugkasten für den Twitch-Kanal **abdulive_**. Ein Monorepo mit mehreren
|
|||||||
kleinen Diensten, die sich einen gemeinsamen Kern teilen: Twitch-Auth,
|
kleinen Diensten, die sich einen gemeinsamen Kern teilen: Twitch-Auth,
|
||||||
Helix-Client, EventSub/IRC-Ingest und eine SQLite-Datenbank.
|
Helix-Client, EventSub/IRC-Ingest und eine SQLite-Datenbank.
|
||||||
|
|
||||||
|
TypeScript auf Node 24, **ohne Laufzeitabhängigkeiten und ohne Build-Schritt** —
|
||||||
|
Node führt die `.ts`-Dateien direkt aus, SQLite und WebSocket sind eingebaut.
|
||||||
|
Betrieb über Docker Compose.
|
||||||
|
|
||||||
Zielgröße des Kanals: ~5–30 gleichzeitige Zuschauer. Alles hier ist auf diese
|
Zielgröße des Kanals: ~5–30 gleichzeitige Zuschauer. Alles hier ist auf diese
|
||||||
Phase optimiert — nicht auf Skalierung, sondern auf **Wachstum und Bindung**.
|
Phase optimiert — nicht auf Skalierung, sondern auf **Wachstum und Bindung**.
|
||||||
|
|
||||||
@@ -23,7 +27,7 @@ Phase optimiert — nicht auf Skalierung, sondern auf **Wachstum und Bindung**.
|
|||||||
|
|
||||||
Dazu der Dienst, der unter allen anderen liegt:
|
Dazu der Dienst, der unter allen anderen liegt:
|
||||||
|
|
||||||
| — | **Ingest-Daemon** | `apps/ingest` | EventSub + IRC → Event-Bus + DB | **zuerst** |
|
| — | **Ingest-Daemon** | `apps/ingest` | IRC + Helix → DB, Einrichtungsseite | **läuft** |
|
||||||
|
|
||||||
## Doku
|
## Doku
|
||||||
|
|
||||||
@@ -45,13 +49,26 @@ gebraucht, ffmpeg und whisper.cpp stecken im Image.
|
|||||||
Siehe [`docs/05-setup.md`](docs/05-setup.md).
|
Siehe [`docs/05-setup.md`](docs/05-setup.md).
|
||||||
|
|
||||||
```powershell
|
```powershell
|
||||||
Copy-Item .env.example .env
|
|
||||||
Copy-Item config\config.example.yaml config\config.yaml
|
Copy-Item config\config.example.yaml config\config.yaml
|
||||||
docker compose build
|
docker compose build
|
||||||
docker compose run --rm toolkit check-env
|
docker compose run --rm toolkit check-env
|
||||||
|
docker compose run --rm toolkit fetch-model
|
||||||
|
docker compose up -d
|
||||||
```
|
```
|
||||||
|
|
||||||
|
Dann **http://localhost:4792/setup** öffnen und die Twitch-Zugangsdaten
|
||||||
|
eintragen. Sie werden verschlüsselt im Docker-Volume abgelegt, nicht in einer
|
||||||
|
Datei im Projekt ([ADR-0005](docs/decisions/ADR-0005-zugangsdaten.md)).
|
||||||
|
|
||||||
## Aktueller Stand
|
## Aktueller Stand
|
||||||
|
|
||||||
Struktur und Dokumentation stehen, Code noch nicht. Als Nächstes:
|
Scheiben **S0 und S1** des MVP „Clip-Knopf" sind fertig: Container-Toolchain,
|
||||||
**MVP „Clip-Knopf"** nach [`docs/07-mvp-plan.md`](docs/07-mvp-plan.md).
|
Einrichtung im Browser, Twitch-Auth, Datenbank und Chat-Aufnahme. Als Nächstes
|
||||||
|
S2 — OBS-Anbindung und der `!clip`-Trigger. Fahrplan:
|
||||||
|
[`docs/07-mvp-plan.md`](docs/07-mvp-plan.md).
|
||||||
|
|
||||||
|
## Tests
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
docker run --rm -v "${PWD}:/app" -w /app node:24-bookworm-slim node --test "packages/**/*.test.ts" "apps/**/*.test.ts"
|
||||||
|
```
|
||||||
@@ -1,12 +0,0 @@
|
|||||||
{
|
|
||||||
"name": "@abdulive/broadcaster",
|
|
||||||
"version": "0.1.0",
|
|
||||||
"private": true,
|
|
||||||
"type": "module",
|
|
||||||
"main": "dist/index.js",
|
|
||||||
"scripts": {
|
|
||||||
"build": "tsc -p tsconfig.json",
|
|
||||||
"typecheck": "tsc -p tsconfig.json --noEmit",
|
|
||||||
"test": "node --test"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,2 +0,0 @@
|
|||||||
// @abdulive/broadcaster — siehe docs/projects/go-live-broadcaster.md
|
|
||||||
export {};
|
|
||||||
@@ -1,5 +0,0 @@
|
|||||||
{
|
|
||||||
"extends": "../../tsconfig.base.json",
|
|
||||||
"compilerOptions": { "outDir": "dist", "rootDir": "src" },
|
|
||||||
"include": ["src"]
|
|
||||||
}
|
|
||||||
@@ -1,12 +0,0 @@
|
|||||||
{
|
|
||||||
"name": "@abdulive/clipper",
|
|
||||||
"version": "0.1.0",
|
|
||||||
"private": true,
|
|
||||||
"type": "module",
|
|
||||||
"main": "dist/index.js",
|
|
||||||
"scripts": {
|
|
||||||
"build": "tsc -p tsconfig.json",
|
|
||||||
"typecheck": "tsc -p tsconfig.json --noEmit",
|
|
||||||
"test": "node --test"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,2 +0,0 @@
|
|||||||
// @abdulive/clipper — siehe docs/projects/clip-shorts.md
|
|
||||||
export {};
|
|
||||||
@@ -1,5 +0,0 @@
|
|||||||
{
|
|
||||||
"extends": "../../tsconfig.base.json",
|
|
||||||
"compilerOptions": { "outDir": "dist", "rootDir": "src" },
|
|
||||||
"include": ["src"]
|
|
||||||
}
|
|
||||||
@@ -1,12 +0,0 @@
|
|||||||
{
|
|
||||||
"name": "@abdulive/community",
|
|
||||||
"version": "0.1.0",
|
|
||||||
"private": true,
|
|
||||||
"type": "module",
|
|
||||||
"main": "dist/index.js",
|
|
||||||
"scripts": {
|
|
||||||
"build": "tsc -p tsconfig.json",
|
|
||||||
"typecheck": "tsc -p tsconfig.json --noEmit",
|
|
||||||
"test": "node --test"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,2 +0,0 @@
|
|||||||
// @abdulive/community — siehe docs/projects/community-memory.md
|
|
||||||
export {};
|
|
||||||
@@ -1,5 +0,0 @@
|
|||||||
{
|
|
||||||
"extends": "../../tsconfig.base.json",
|
|
||||||
"compilerOptions": { "outDir": "dist", "rootDir": "src" },
|
|
||||||
"include": ["src"]
|
|
||||||
}
|
|
||||||
@@ -1,12 +0,0 @@
|
|||||||
{
|
|
||||||
"name": "@abdulive/dashboard",
|
|
||||||
"version": "0.1.0",
|
|
||||||
"private": true,
|
|
||||||
"type": "module",
|
|
||||||
"main": "dist/index.js",
|
|
||||||
"scripts": {
|
|
||||||
"build": "tsc -p tsconfig.json",
|
|
||||||
"typecheck": "tsc -p tsconfig.json --noEmit",
|
|
||||||
"test": "node --test"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,2 +0,0 @@
|
|||||||
// @abdulive/dashboard — siehe docs/projects/analytics.md
|
|
||||||
export {};
|
|
||||||
@@ -1,5 +0,0 @@
|
|||||||
{
|
|
||||||
"extends": "../../tsconfig.base.json",
|
|
||||||
"compilerOptions": { "outDir": "dist", "rootDir": "src" },
|
|
||||||
"include": ["src"]
|
|
||||||
}
|
|
||||||
@@ -1,12 +0,0 @@
|
|||||||
{
|
|
||||||
"name": "@abdulive/ingest",
|
|
||||||
"version": "0.1.0",
|
|
||||||
"private": true,
|
|
||||||
"type": "module",
|
|
||||||
"main": "dist/index.js",
|
|
||||||
"scripts": {
|
|
||||||
"build": "tsc -p tsconfig.json",
|
|
||||||
"typecheck": "tsc -p tsconfig.json --noEmit",
|
|
||||||
"test": "node --test"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
+176
-2
@@ -1,2 +1,176 @@
|
|||||||
// @abdulive/ingest — siehe docs/projects/00-ingest.md
|
// Ingest-Daemon. Liest Chat und Zuschauerzahlen, schreibt beides in die
|
||||||
export {};
|
// Datenbank und haelt die Einrichtungsseite bereit.
|
||||||
|
//
|
||||||
|
// Stand S1: IRC + Helix-Polling. EventSub und OBS kommen in S2.
|
||||||
|
// Siehe docs/projects/00-ingest.md.
|
||||||
|
|
||||||
|
import { loadConfig } from '#core/config.ts';
|
||||||
|
import { logger } from '#core/logger.ts';
|
||||||
|
import { sleep } from '#core/retry.ts';
|
||||||
|
import { openDb, closeDb } from '#db/index.ts';
|
||||||
|
import { repos } from '#db/repos.ts';
|
||||||
|
import { helix } from '#twitch/helix.ts';
|
||||||
|
import { createIrc } from '#twitch/irc.ts';
|
||||||
|
import { readSecrets } from '#twitch/secrets.ts';
|
||||||
|
import { createSpikeDetector } from './spike.ts';
|
||||||
|
import { startServer } from './server.ts';
|
||||||
|
|
||||||
|
const log = logger('ingest');
|
||||||
|
|
||||||
|
export async function runIngest(): Promise<void> {
|
||||||
|
const cfg = loadConfig();
|
||||||
|
const db = openDb();
|
||||||
|
const repo = repos(db);
|
||||||
|
const spikes = createSpikeDetector(cfg.ingest.spike);
|
||||||
|
|
||||||
|
const state = {
|
||||||
|
startedAt: new Date().toISOString(),
|
||||||
|
streamId: null as string | null,
|
||||||
|
lastSeenLive: 0,
|
||||||
|
viewers: 0,
|
||||||
|
chatConnected: false,
|
||||||
|
chatAnonymous: true,
|
||||||
|
messages: 0,
|
||||||
|
spikeCandidates: 0,
|
||||||
|
lastError: null as string | null,
|
||||||
|
};
|
||||||
|
|
||||||
|
const abort = new AbortController();
|
||||||
|
const stop = (signal: string) => {
|
||||||
|
log.info(`${signal} empfangen, fahre herunter`);
|
||||||
|
abort.abort();
|
||||||
|
setTimeout(() => { closeDb(); process.exit(0); }, 500);
|
||||||
|
};
|
||||||
|
process.on('SIGINT', () => stop('SIGINT'));
|
||||||
|
process.on('SIGTERM', () => stop('SIGTERM'));
|
||||||
|
|
||||||
|
startServer({
|
||||||
|
channel: cfg.channel.login,
|
||||||
|
port: cfg.dashboard.port,
|
||||||
|
health: () => ({
|
||||||
|
...state,
|
||||||
|
db: repo.stats(),
|
||||||
|
spikeArmed: spikes.isArmed(),
|
||||||
|
note: state.chatAnonymous
|
||||||
|
? 'Chat wird anonym mitgelesen. Zum Senden und fuer Helix: Einrichtung unter /setup.'
|
||||||
|
: undefined,
|
||||||
|
}),
|
||||||
|
});
|
||||||
|
|
||||||
|
// ---- Chat
|
||||||
|
const irc = createIrc(cfg.channel.login, {
|
||||||
|
onConnected: (anonymous) => {
|
||||||
|
state.chatConnected = true;
|
||||||
|
state.chatAnonymous = anonymous;
|
||||||
|
},
|
||||||
|
onMessage: (msg) => {
|
||||||
|
try {
|
||||||
|
const { streamsChatted } = repo.recordMessage(msg, state.streamId);
|
||||||
|
state.messages++;
|
||||||
|
repo.event('chat.message', {
|
||||||
|
text: msg.text, login: msg.login, isFirst: msg.isFirst, bits: msg.bits,
|
||||||
|
}, state.streamId, msg.userId, msg.ts);
|
||||||
|
|
||||||
|
if (msg.isFirst) {
|
||||||
|
repo.event('chat.first_message', { login: msg.login }, state.streamId, msg.userId, msg.ts);
|
||||||
|
log.info(`Erstchatter: ${msg.displayName}`);
|
||||||
|
} else if (streamsChatted === 1) {
|
||||||
|
log.debug(`Erster Stream fuer ${msg.displayName}`);
|
||||||
|
}
|
||||||
|
|
||||||
|
const spike = spikes.push(msg);
|
||||||
|
if (spike) {
|
||||||
|
state.spikeCandidates++;
|
||||||
|
repo.event('chat.spike', spike, state.streamId, null, msg.ts);
|
||||||
|
log.info(`Spitze im Chat: Punktzahl ${spike.score} ` +
|
||||||
|
`(${spike.rate} Nachrichten von ${spike.chatters} Leuten, Basislinie ${spike.baseline})` +
|
||||||
|
(spike.overThreshold ? '' : ' - nur protokolliert, Ausloesung ist aus'));
|
||||||
|
}
|
||||||
|
} catch (err) {
|
||||||
|
state.lastError = (err as Error).message;
|
||||||
|
log.error(`Nachricht nicht verarbeitet: ${(err as Error).message}`);
|
||||||
|
}
|
||||||
|
},
|
||||||
|
});
|
||||||
|
void irc.run(abort.signal);
|
||||||
|
|
||||||
|
// ---- Zuschauerzahlen und Stream-Sessions
|
||||||
|
void pollStream(cfg, repo, state, abort.signal);
|
||||||
|
|
||||||
|
log.info(`Ingest laeuft fuer #${cfg.channel.login}`);
|
||||||
|
await new Promise<void>((resolve) => abort.signal.addEventListener('abort', () => resolve()));
|
||||||
|
}
|
||||||
|
|
||||||
|
type State = {
|
||||||
|
streamId: string | null; lastSeenLive: number; viewers: number; lastError: string | null;
|
||||||
|
};
|
||||||
|
|
||||||
|
async function pollStream(
|
||||||
|
cfg: ReturnType<typeof loadConfig>,
|
||||||
|
repo: ReturnType<typeof repos>,
|
||||||
|
state: State,
|
||||||
|
signal: AbortSignal
|
||||||
|
): Promise<void> {
|
||||||
|
const gapMs = cfg.ingest.sessionGapMinutes * 60_000;
|
||||||
|
let warnedMissingAuth = false;
|
||||||
|
|
||||||
|
// Ein laufender Stream wird beim Start uebernommen, damit ein Neustart
|
||||||
|
// mitten im Stream keine zweite Sitzung anlegt.
|
||||||
|
const open = repo.latestOpenStream();
|
||||||
|
if (open) {
|
||||||
|
state.streamId = open.id;
|
||||||
|
state.lastSeenLive = Date.now();
|
||||||
|
log.info(`Laufende Sitzung uebernommen: ${open.id} seit ${open.started_at}`);
|
||||||
|
}
|
||||||
|
|
||||||
|
while (!signal.aborted) {
|
||||||
|
try {
|
||||||
|
if (!readSecrets().tokens.broadcaster) {
|
||||||
|
if (!warnedMissingAuth) {
|
||||||
|
log.warn('Kein Broadcaster-Token - Zuschauerzahlen und Stream-Sitzungen bleiben aus. ' +
|
||||||
|
'Einrichtung unter /setup.');
|
||||||
|
warnedMissingAuth = true;
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
warnedMissingAuth = false;
|
||||||
|
const userId = cfg.channel.userId ||
|
||||||
|
(await helix.userByLogin(cfg.channel.login))?.id || '';
|
||||||
|
const live = userId ? await helix.streamByUserId(userId) : null;
|
||||||
|
const now = Date.now();
|
||||||
|
|
||||||
|
if (live) {
|
||||||
|
if (state.streamId !== live.id) {
|
||||||
|
// Kurze Unterbrechungen gelten als dieselbe Sitzung, sonst
|
||||||
|
// zerfaellt ein Stream bei jedem Aussetzer in Fragmente.
|
||||||
|
const wasRecent = now - state.lastSeenLive < gapMs;
|
||||||
|
if (state.streamId && wasRecent) {
|
||||||
|
log.info(`Neue Twitch-Stream-ID ${live.id}, behandle als Fortsetzung von ${state.streamId}`);
|
||||||
|
} else {
|
||||||
|
state.streamId = live.id;
|
||||||
|
repo.openStream(live.id, live.started_at);
|
||||||
|
repo.event('stream.online',
|
||||||
|
{ title: live.title, category: live.game_name }, live.id);
|
||||||
|
log.info(`Stream online: "${live.title}" (${live.game_name})`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
state.lastSeenLive = now;
|
||||||
|
state.viewers = live.viewer_count;
|
||||||
|
if (state.streamId) {
|
||||||
|
repo.sample(state.streamId, new Date().toISOString(), live.viewer_count);
|
||||||
|
}
|
||||||
|
} else if (state.streamId && now - state.lastSeenLive > gapMs) {
|
||||||
|
const ended = new Date().toISOString();
|
||||||
|
repo.closeStream(state.streamId, ended);
|
||||||
|
repo.event('stream.offline', {}, state.streamId);
|
||||||
|
log.info(`Stream offline: ${state.streamId}`);
|
||||||
|
state.streamId = null;
|
||||||
|
state.viewers = 0;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} catch (err) {
|
||||||
|
state.lastError = (err as Error).message;
|
||||||
|
log.error(`Abfrage fehlgeschlagen: ${(err as Error).message}`);
|
||||||
|
}
|
||||||
|
try { await sleep(cfg.ingest.viewerPollSeconds * 1000, signal); } catch { return; }
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,152 @@
|
|||||||
|
// Ein HTTP-Server fuer Einrichtung, OAuth-Rueckruf und Statusabfrage.
|
||||||
|
// Bindet ausschliesslich auf 127.0.0.1 - siehe ADR-0005.
|
||||||
|
// Ab S5 kommt hier die Sichtungsseite fuer Clip-Kandidaten dazu.
|
||||||
|
|
||||||
|
import http from 'node:http';
|
||||||
|
import { logger } from '#core/logger.ts';
|
||||||
|
import { authorizeUrl, handleCallback, redirectUri } from '#twitch/oauth.ts';
|
||||||
|
import { secretStatus, updateSecrets, type Role } from '#twitch/secrets.ts';
|
||||||
|
import { setupPage } from './setup-page.ts';
|
||||||
|
|
||||||
|
const log = logger('web');
|
||||||
|
|
||||||
|
export type ServerDeps = {
|
||||||
|
channel: string;
|
||||||
|
port: number;
|
||||||
|
health: () => Record<string, unknown>;
|
||||||
|
};
|
||||||
|
|
||||||
|
function readBody(req: http.IncomingMessage): Promise<URLSearchParams> {
|
||||||
|
return new Promise((resolve, reject) => {
|
||||||
|
const chunks: Buffer[] = [];
|
||||||
|
let size = 0;
|
||||||
|
req.on('data', (c: Buffer) => {
|
||||||
|
size += c.length;
|
||||||
|
if (size > 64 * 1024) { reject(new Error('Anfrage zu gross')); req.destroy(); return; }
|
||||||
|
chunks.push(c);
|
||||||
|
});
|
||||||
|
req.on('end', () => resolve(new URLSearchParams(Buffer.concat(chunks).toString('utf8'))));
|
||||||
|
req.on('error', reject);
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
function isRole(v: string | null): v is Role {
|
||||||
|
return v === 'broadcaster' || v === 'bot';
|
||||||
|
}
|
||||||
|
|
||||||
|
export function startServer(deps: ServerDeps): http.Server {
|
||||||
|
const server = http.createServer((req, res) => {
|
||||||
|
void handle(req, res, deps).catch((err: Error) => {
|
||||||
|
log.error(`Anfrage fehlgeschlagen: ${err.message}`);
|
||||||
|
if (!res.headersSent) res.writeHead(500, { 'content-type': 'text/plain; charset=utf-8' });
|
||||||
|
res.end('Interner Fehler');
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
// Vorgabe ist die Loopback-Adresse. Im Container muss das 0.0.0.0 sein -
|
||||||
|
// dort ist 127.0.0.1 die Container-Loopback, die Dockers Portweiterleitung
|
||||||
|
// nie erreicht; die Abschottung kommt dann daher, dass Compose den Port
|
||||||
|
// ausschliesslich auf 127.0.0.1 des Hosts veroeffentlicht. Das Image setzt
|
||||||
|
// BIND_HOST entsprechend. Siehe ADR-0005.
|
||||||
|
const host = process.env.BIND_HOST || '127.0.0.1';
|
||||||
|
server.listen(deps.port, host, () => {
|
||||||
|
log.info(`Einrichtung und Status: http://localhost:${deps.port}/setup (gebunden an ${host})`);
|
||||||
|
});
|
||||||
|
return server;
|
||||||
|
}
|
||||||
|
|
||||||
|
async function handle(
|
||||||
|
req: http.IncomingMessage, res: http.ServerResponse, deps: ServerDeps
|
||||||
|
): Promise<void> {
|
||||||
|
const url = new URL(req.url ?? '/', `http://localhost:${deps.port}`);
|
||||||
|
const html = (body: string, code = 200) => {
|
||||||
|
res.writeHead(code, { 'content-type': 'text/html; charset=utf-8',
|
||||||
|
'cache-control': 'no-store', 'referrer-policy': 'no-referrer' });
|
||||||
|
res.end(body);
|
||||||
|
};
|
||||||
|
const back = (kind: 'ok' | 'err', text: string) => {
|
||||||
|
res.writeHead(303, { location: `/setup?${kind}=${encodeURIComponent(text)}` });
|
||||||
|
res.end();
|
||||||
|
};
|
||||||
|
|
||||||
|
// ---- Einrichtungsseite
|
||||||
|
if (url.pathname === '/' || url.pathname === '/setup') {
|
||||||
|
const ok = url.searchParams.get('ok');
|
||||||
|
const err = url.searchParams.get('err');
|
||||||
|
return html(setupPage(secretStatus(), {
|
||||||
|
channel: deps.channel,
|
||||||
|
redirectUri: redirectUri(deps.port),
|
||||||
|
...(ok ? { notice: { kind: 'ok' as const, text: ok } }
|
||||||
|
: err ? { notice: { kind: 'err' as const, text: err } } : {}),
|
||||||
|
}));
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---- Client-ID und Secret speichern
|
||||||
|
if (url.pathname === '/setup/app' && req.method === 'POST') {
|
||||||
|
const form = await readBody(req);
|
||||||
|
const role = form.get('role');
|
||||||
|
const clientId = (form.get('clientId') ?? '').trim();
|
||||||
|
const clientSecret = (form.get('clientSecret') ?? '').trim();
|
||||||
|
if (!isRole(role)) return back('err', 'Unbekannte Rolle');
|
||||||
|
if (!clientId) return back('err', 'Client-ID fehlt');
|
||||||
|
|
||||||
|
updateSecrets((s) => {
|
||||||
|
const existing = s.apps[role];
|
||||||
|
// Leeres Secret-Feld heisst "unveraendert lassen", nicht "loeschen"
|
||||||
|
const secret = clientSecret || existing?.clientSecret;
|
||||||
|
if (!secret) throw new Error('Client-Secret fehlt');
|
||||||
|
s.apps[role] = { clientId, clientSecret: secret };
|
||||||
|
// Bei gewechselter App sind die alten Tokens wertlos
|
||||||
|
if (existing && existing.clientId !== clientId) delete s.tokens[role];
|
||||||
|
});
|
||||||
|
log.info(`Client-Daten fuer ${role} gespeichert`);
|
||||||
|
return back('ok', 'Gespeichert.');
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---- OBS-Passwort
|
||||||
|
if (url.pathname === '/setup/obs' && req.method === 'POST') {
|
||||||
|
const password = ((await readBody(req)).get('password') ?? '').trim();
|
||||||
|
if (!password) return back('err', 'Kein Passwort eingegeben - unveraendert gelassen.');
|
||||||
|
updateSecrets((s) => { s.obs = { password }; });
|
||||||
|
log.info('OBS-Passwort gespeichert');
|
||||||
|
return back('ok', 'OBS-Passwort gespeichert.');
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---- OAuth starten
|
||||||
|
if (url.pathname === '/setup/connect') {
|
||||||
|
const role = url.searchParams.get('role');
|
||||||
|
if (!isRole(role)) return back('err', 'Unbekannte Rolle');
|
||||||
|
try {
|
||||||
|
res.writeHead(303, { location: authorizeUrl(role, deps.port) });
|
||||||
|
res.end();
|
||||||
|
return;
|
||||||
|
} catch (err) {
|
||||||
|
return back('err', (err as Error).message);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---- OAuth-Rueckruf
|
||||||
|
if (url.pathname === '/callback') {
|
||||||
|
const error = url.searchParams.get('error_description') ?? url.searchParams.get('error');
|
||||||
|
if (error) return back('err', `Twitch hat abgelehnt: ${error}`);
|
||||||
|
const code = url.searchParams.get('code');
|
||||||
|
const state = url.searchParams.get('state');
|
||||||
|
if (!code || !state) return back('err', 'Antwort von Twitch war unvollstaendig');
|
||||||
|
try {
|
||||||
|
const role = await handleCallback(code, state, deps.port);
|
||||||
|
return back('ok', `${role === 'bot' ? 'Bot' : 'Broadcaster'}-Konto verbunden.`);
|
||||||
|
} catch (err) {
|
||||||
|
return back('err', (err as Error).message);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---- Status fuer Skripte und Dashboard
|
||||||
|
if (url.pathname === '/health') {
|
||||||
|
res.writeHead(200, { 'content-type': 'application/json; charset=utf-8' });
|
||||||
|
res.end(JSON.stringify(deps.health(), null, 2));
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
res.writeHead(404, { 'content-type': 'text/plain; charset=utf-8' });
|
||||||
|
res.end('Nicht gefunden');
|
||||||
|
}
|
||||||
@@ -0,0 +1,165 @@
|
|||||||
|
// Einrichtungsseite. Bewusst eine einzelne HTML-Datei ohne Framework und ohne
|
||||||
|
// Build - sie wird ein paar Mal im Leben des Projekts benutzt.
|
||||||
|
//
|
||||||
|
// Regeln aus ADR-0005:
|
||||||
|
// - gespeicherte Geheimnisse werden nie zurueckgegeben, nur "gesetzt"
|
||||||
|
// - oben steht eine Warnung, die Seite nicht im Stream zu oeffnen
|
||||||
|
// - der Server bindet nur auf 127.0.0.1
|
||||||
|
|
||||||
|
import type { SecretStatus } from '#twitch/secrets.ts';
|
||||||
|
|
||||||
|
function esc(s: string): string {
|
||||||
|
return s.replace(/[&<>"']/g, (c) =>
|
||||||
|
({ '&': '&', '<': '<', '>': '>', '"': '"', "'": ''' }[c]!));
|
||||||
|
}
|
||||||
|
|
||||||
|
const STYLE = `
|
||||||
|
:root{--bg:#14151a;--card:#1d1f27;--line:#2e3140;--fg:#e7e9ee;--dim:#9aa0b0;
|
||||||
|
--ok:#4ec98a;--warn:#e3b341;--accent:#9147ff;--danger:#f0616d}
|
||||||
|
*{box-sizing:border-box}
|
||||||
|
body{margin:0;background:var(--bg);color:var(--fg);
|
||||||
|
font:15px/1.55 system-ui,-apple-system,"Segoe UI",sans-serif}
|
||||||
|
.wrap{max-width:760px;margin:0 auto;padding:32px 20px 80px}
|
||||||
|
h1{font-size:22px;margin:0 0 4px}
|
||||||
|
h2{font-size:16px;margin:0 0 14px;font-weight:600}
|
||||||
|
p.lead{color:var(--dim);margin:0 0 28px}
|
||||||
|
.warn{background:#3a2418;border:1px solid #6b3d1f;border-radius:10px;
|
||||||
|
padding:12px 14px;margin:0 0 26px;color:#f5c99b}
|
||||||
|
.warn b{color:#ffd7ae}
|
||||||
|
.card{background:var(--card);border:1px solid var(--line);border-radius:12px;
|
||||||
|
padding:18px 20px;margin:0 0 16px}
|
||||||
|
.row{display:flex;align-items:center;justify-content:space-between;gap:16px;
|
||||||
|
padding:9px 0;border-top:1px solid var(--line)}
|
||||||
|
.row:first-of-type{border-top:0}
|
||||||
|
label{display:block;color:var(--dim);font-size:13px;margin:12px 0 5px}
|
||||||
|
input{width:100%;padding:9px 11px;border-radius:8px;border:1px solid var(--line);
|
||||||
|
background:#111219;color:var(--fg);font:inherit;font-size:14px}
|
||||||
|
input:focus{outline:2px solid var(--accent);outline-offset:-1px}
|
||||||
|
button{border:0;border-radius:8px;padding:9px 16px;font:inherit;font-weight:600;
|
||||||
|
cursor:pointer;background:var(--accent);color:#fff}
|
||||||
|
button.ghost{background:#2b2e3a;color:var(--fg)}
|
||||||
|
a.btn{display:inline-block;text-decoration:none;border-radius:8px;padding:9px 16px;
|
||||||
|
font-weight:600;background:var(--accent);color:#fff}
|
||||||
|
a.btn.ghost{background:#2b2e3a;color:var(--fg)}
|
||||||
|
.pill{font-size:12px;padding:3px 9px;border-radius:99px;white-space:nowrap}
|
||||||
|
.pill.on{background:#14331f;color:var(--ok)}
|
||||||
|
.pill.off{background:#332020;color:var(--danger)}
|
||||||
|
.pill.wait{background:#33290f;color:var(--warn)}
|
||||||
|
code{background:#111219;padding:2px 6px;border-radius:5px;font-size:13px}
|
||||||
|
.hint{color:var(--dim);font-size:13px;margin:8px 0 0}
|
||||||
|
.done{color:var(--ok)}
|
||||||
|
.msg{border-radius:10px;padding:11px 14px;margin:0 0 20px}
|
||||||
|
.msg.ok{background:#14331f;color:#8ce0b0}
|
||||||
|
.msg.err{background:#33191c;color:#ffb3ba}
|
||||||
|
form{margin:0}
|
||||||
|
.grid{display:grid;grid-template-columns:1fr 1fr;gap:0 14px}
|
||||||
|
@media(max-width:560px){.grid{grid-template-columns:1fr}}
|
||||||
|
`;
|
||||||
|
|
||||||
|
export function setupPage(
|
||||||
|
status: SecretStatus,
|
||||||
|
opts: { channel: string; redirectUri: string; notice?: { kind: 'ok' | 'err'; text: string } }
|
||||||
|
): string {
|
||||||
|
const app = (role: 'broadcaster' | 'bot', title: string, note: string) => {
|
||||||
|
const a = status.apps[role];
|
||||||
|
const t = status.tokens[role];
|
||||||
|
return `
|
||||||
|
<div class="card">
|
||||||
|
<h2>${esc(title)}</h2>
|
||||||
|
<p class="hint" style="margin-top:-8px">${note}</p>
|
||||||
|
|
||||||
|
<form method="post" action="/setup/app">
|
||||||
|
<input type="hidden" name="role" value="${role}">
|
||||||
|
<div class="grid">
|
||||||
|
<div>
|
||||||
|
<label for="${role}-id">Client-ID</label>
|
||||||
|
<input id="${role}-id" name="clientId" value="${esc(a.clientId)}"
|
||||||
|
placeholder="z. B. gp762nuuoqcoxypju8c569th9wz7q5" autocomplete="off" required>
|
||||||
|
</div>
|
||||||
|
<div>
|
||||||
|
<label for="${role}-secret">Client-Secret ${a.hasSecret ? '<span class="done">gespeichert</span>' : ''}</label>
|
||||||
|
<input id="${role}-secret" name="clientSecret" type="password"
|
||||||
|
placeholder="${a.hasSecret ? '••••••••• leer lassen = unveraendert' : 'aus der Twitch-Konsole'}"
|
||||||
|
autocomplete="new-password" ${a.hasSecret ? '' : 'required'}>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
<div style="margin-top:14px"><button type="submit">Speichern</button></div>
|
||||||
|
</form>
|
||||||
|
|
||||||
|
<div class="row" style="margin-top:16px">
|
||||||
|
<div>
|
||||||
|
<strong>Twitch-Konto</strong><br>
|
||||||
|
<span class="hint">${t.connected
|
||||||
|
? `verbunden als <code>${esc(t.login ?? '')}</code>, ${t.scopes} Scopes`
|
||||||
|
: 'noch nicht verbunden'}</span>
|
||||||
|
</div>
|
||||||
|
<div style="display:flex;gap:8px;align-items:center">
|
||||||
|
<span class="pill ${t.connected ? 'on' : 'off'}">${t.connected ? 'verbunden' : 'offen'}</span>
|
||||||
|
${a.clientId && a.hasSecret
|
||||||
|
? `<a class="btn ${t.connected ? 'ghost' : ''}" href="/setup/connect?role=${role}">${
|
||||||
|
t.connected ? 'Neu verbinden' : 'Mit Twitch verbinden'}</a>`
|
||||||
|
: '<span class="hint">erst Client-Daten speichern</span>'}
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</div>`;
|
||||||
|
};
|
||||||
|
|
||||||
|
return `<!doctype html>
|
||||||
|
<html lang="de"><head><meta charset="utf-8">
|
||||||
|
<meta name="viewport" content="width=device-width,initial-scale=1">
|
||||||
|
<title>Einrichtung - abdulive-toolkit</title><style>${STYLE}</style></head>
|
||||||
|
<body><div class="wrap">
|
||||||
|
|
||||||
|
<h1>Einrichtung</h1>
|
||||||
|
<p class="lead">Kanal <code>${esc(opts.channel)}</code></p>
|
||||||
|
|
||||||
|
<div class="warn">
|
||||||
|
<b>Diese Seite zeigt Zugangsdaten.</b> Nicht oeffnen, waehrend gesendet wird.
|
||||||
|
Gespeicherte Geheimnisse werden nie wieder angezeigt, aber was hier
|
||||||
|
eingetippt wird, steht im Klartext im Eingabefeld.
|
||||||
|
</div>
|
||||||
|
|
||||||
|
${opts.notice ? `<div class="msg ${opts.notice.kind}">${esc(opts.notice.text)}</div>` : ''}
|
||||||
|
|
||||||
|
<div class="card">
|
||||||
|
<h2>Bevor es losgeht</h2>
|
||||||
|
<p class="hint" style="margin-top:-6px">
|
||||||
|
Zwei Anwendungen anlegen unter
|
||||||
|
<a href="https://dev.twitch.tv/console/apps" target="_blank" rel="noreferrer"
|
||||||
|
style="color:#b18cff">dev.twitch.tv/console/apps</a>.
|
||||||
|
Bei beiden als OAuth-Redirect-URL exakt eintragen:
|
||||||
|
</p>
|
||||||
|
<p style="margin:10px 0 0"><code>${esc(opts.redirectUri)}</code></p>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
${app('broadcaster', '1. Broadcaster-Konto',
|
||||||
|
'Fuer Clips, Titel, Raids und Zuschauerzahlen. Das ist dein eigener Kanal.')}
|
||||||
|
${app('bot', '2. Bot-Konto',
|
||||||
|
'Nur fuer Chat-Nachrichten. Ein eigenes Twitch-Konto, damit Bot-Antworten als Bot erkennbar sind. ' +
|
||||||
|
'Ohne das liest das Toolkit den Chat trotzdem mit - es kann nur nicht antworten.')}
|
||||||
|
|
||||||
|
<div class="card">
|
||||||
|
<h2>3. OBS</h2>
|
||||||
|
<p class="hint" style="margin-top:-8px">
|
||||||
|
Werkzeuge → WebSocket-Servereinstellungen → Server aktivieren, Port 4455.
|
||||||
|
</p>
|
||||||
|
<form method="post" action="/setup/obs">
|
||||||
|
<label for="obs">Passwort ${status.obs ? '<span class="done">gespeichert</span>' : ''}</label>
|
||||||
|
<input id="obs" name="password" type="password"
|
||||||
|
placeholder="${status.obs ? '••••••••• leer lassen = unveraendert' : 'aus den OBS-Einstellungen'}"
|
||||||
|
autocomplete="new-password">
|
||||||
|
<div style="margin-top:14px"><button type="submit" class="ghost">Speichern</button></div>
|
||||||
|
</form>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div class="card">
|
||||||
|
<div class="row">
|
||||||
|
<div><strong>Status</strong><br><span class="hint">${status.ready
|
||||||
|
? 'Alles Noetige ist hinterlegt. Der Ingest laeuft.'
|
||||||
|
: 'Solange das Broadcaster-Konto nicht verbunden ist, liest das Toolkit den Chat anonym mit.'}</span></div>
|
||||||
|
<span class="pill ${status.ready ? 'on' : 'wait'}">${status.ready ? 'bereit' : 'unvollstaendig'}</span>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
</div></body></html>`;
|
||||||
|
}
|
||||||
@@ -0,0 +1,84 @@
|
|||||||
|
import { test } from 'node:test';
|
||||||
|
import assert from 'node:assert/strict';
|
||||||
|
import type { ChatMessage } from '#shared/events.ts';
|
||||||
|
import { createSpikeDetector, type SpikeConfig } from './spike.ts';
|
||||||
|
|
||||||
|
const CFG: SpikeConfig = {
|
||||||
|
windowSeconds: 15,
|
||||||
|
baselineMinutes: 10,
|
||||||
|
factor: 3,
|
||||||
|
minDelta: 4,
|
||||||
|
cooldownSeconds: 60,
|
||||||
|
armed: false,
|
||||||
|
laughEmotes: ['KEKW', 'LUL'],
|
||||||
|
};
|
||||||
|
|
||||||
|
let clock = Date.parse('2026-09-09T20:00:00.000Z');
|
||||||
|
|
||||||
|
function msg(text: string, userId: string, advanceMs = 0): ChatMessage {
|
||||||
|
clock += advanceMs;
|
||||||
|
return {
|
||||||
|
id: `m${clock}-${userId}`, ts: new Date(clock).toISOString(), userId,
|
||||||
|
login: `u${userId}`, displayName: `U${userId}`, text, emotes: [],
|
||||||
|
isFirst: false, isReturning: false, isCommand: text.startsWith('!'),
|
||||||
|
isMod: false, isSub: false, isBroadcaster: false, bits: 0, color: null,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
test('ruhiger Chat erzeugt keinen Kandidaten', () => {
|
||||||
|
clock = Date.parse('2026-09-09T20:00:00.000Z');
|
||||||
|
const d = createSpikeDetector(CFG);
|
||||||
|
let hits = 0;
|
||||||
|
// Eine Nachricht alle 20 s - das Fenster ist nie voller als eine
|
||||||
|
for (let i = 0; i < 30; i++) if (d.push(msg('hi', String(i % 4), 20_000))) hits++;
|
||||||
|
assert.equal(hits, 0);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('ploetzlicher Ausbruch von mehreren Leuten wird erkannt', () => {
|
||||||
|
clock = Date.parse('2026-09-09T20:00:00.000Z');
|
||||||
|
const d = createSpikeDetector(CFG);
|
||||||
|
for (let i = 0; i < 30; i++) d.push(msg('ruhig', String(i % 3), 20_000));
|
||||||
|
|
||||||
|
let spike = null;
|
||||||
|
for (let i = 0; i < 8 && !spike; i++) spike = d.push(msg('KEKW', `x${i}`, 300));
|
||||||
|
assert.ok(spike, 'Ausbruch haette einen Kandidaten liefern muessen');
|
||||||
|
assert.ok(spike.chatters >= 4, `zu wenig verschiedene Chatter: ${spike.chatters}`);
|
||||||
|
assert.ok(spike.emoteHits > 0, 'Lach-Emotes wurden nicht gezaehlt');
|
||||||
|
assert.equal(spike.overThreshold, false, 'armed=false darf nicht ausloesen');
|
||||||
|
});
|
||||||
|
|
||||||
|
test('Vielschreiber allein erzeugt weniger Punkte als eine Gruppe', () => {
|
||||||
|
const run = (users: string[]) => {
|
||||||
|
clock = Date.parse('2026-09-09T20:00:00.000Z');
|
||||||
|
const d = createSpikeDetector(CFG);
|
||||||
|
for (let i = 0; i < 30; i++) d.push(msg('ruhig', 'base', 20_000));
|
||||||
|
let spike = null;
|
||||||
|
for (let i = 0; i < 10 && !spike; i++) spike = d.push(msg('hahaha', users[i % users.length]!, 300));
|
||||||
|
return spike;
|
||||||
|
};
|
||||||
|
const solo = run(['einer']);
|
||||||
|
const gruppe = run(['a', 'b', 'c', 'd', 'e']);
|
||||||
|
assert.ok(solo && gruppe);
|
||||||
|
assert.ok(gruppe.score > solo.score,
|
||||||
|
`Gruppe (${gruppe.score}) muss hoeher liegen als Einzelperson (${solo.score})`);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('das Wort clip zaehlt stark', () => {
|
||||||
|
clock = Date.parse('2026-09-09T20:00:00.000Z');
|
||||||
|
const d = createSpikeDetector(CFG);
|
||||||
|
for (let i = 0; i < 30; i++) d.push(msg('ruhig', String(i % 3), 20_000));
|
||||||
|
let spike = null;
|
||||||
|
for (let i = 0; i < 8 && !spike; i++) spike = d.push(msg('das war ein CLIP wert', `y${i}`, 300));
|
||||||
|
assert.ok(spike);
|
||||||
|
assert.ok(spike.clipMentions >= 4, `clip-Erwaehnungen: ${spike.clipMentions}`);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('Abklingzeit verhindert Kandidaten im Sekundentakt', () => {
|
||||||
|
clock = Date.parse('2026-09-09T20:00:00.000Z');
|
||||||
|
const d = createSpikeDetector(CFG);
|
||||||
|
for (let i = 0; i < 30; i++) d.push(msg('ruhig', String(i % 3), 20_000));
|
||||||
|
let hits = 0;
|
||||||
|
for (let i = 0; i < 60; i++) if (d.push(msg('KEKW', `z${i}`, 500))) hits++;
|
||||||
|
// 60 Nachrichten in 30 s: ohne Abklingzeit waeren das viele Kandidaten
|
||||||
|
assert.ok(hits <= 1, `Abklingzeit greift nicht, ${hits} Kandidaten`);
|
||||||
|
});
|
||||||
@@ -0,0 +1,109 @@
|
|||||||
|
// Erkennung von Chat-Spitzen.
|
||||||
|
//
|
||||||
|
// Wichtig: bei zehn Zuschauern ist die Nachrichtenrate so niedrig, dass drei
|
||||||
|
// gleichzeitige "lol" bereits eine Verdreifachung sind. Der Detektor laeuft
|
||||||
|
// deshalb von Anfang an mit und protokolliert JEDEN Kandidaten samt Punktzahl,
|
||||||
|
// loest aber nichts aus, solange `armed` false ist. Die Schwelle wird spaeter
|
||||||
|
// an echtem Material kalibriert, statt sie zu raten.
|
||||||
|
// Siehe docs/projects/00-ingest.md.
|
||||||
|
|
||||||
|
import type { ChatMessage, SpikeEvent } from '#shared/events.ts';
|
||||||
|
|
||||||
|
export type SpikeConfig = {
|
||||||
|
windowSeconds: number;
|
||||||
|
baselineMinutes: number;
|
||||||
|
factor: number;
|
||||||
|
minDelta: number;
|
||||||
|
cooldownSeconds: number;
|
||||||
|
armed: boolean;
|
||||||
|
laughEmotes: string[];
|
||||||
|
};
|
||||||
|
|
||||||
|
type Entry = { ts: number; userId: string; emoteHits: number; clipMention: number; shout: number };
|
||||||
|
|
||||||
|
export function createSpikeDetector(cfg: SpikeConfig) {
|
||||||
|
const window: Entry[] = [];
|
||||||
|
const rateHistory: Array<{ ts: number; rate: number }> = [];
|
||||||
|
const laugh = new Set(cfg.laughEmotes.map((e) => e.toLowerCase()));
|
||||||
|
let lastSpike = 0;
|
||||||
|
|
||||||
|
function isShout(text: string): number {
|
||||||
|
const letters = text.replace(/[^\p{L}]/gu, '');
|
||||||
|
if (letters.length < 4) return 0;
|
||||||
|
const upper = (text.match(/\p{Lu}/gu) ?? []).length;
|
||||||
|
const excl = /[!?]{2,}/.test(text) ? 1 : 0;
|
||||||
|
return (upper / letters.length > 0.6 ? 1 : 0) + excl;
|
||||||
|
}
|
||||||
|
|
||||||
|
function score(msg: ChatMessage): Entry {
|
||||||
|
let emoteHits = msg.emotes.reduce(
|
||||||
|
(n, e) => n + (laugh.has(e.name.toLowerCase()) ? e.count : 0), 0);
|
||||||
|
// Emotes ohne Twitch-Tag (BTTV/7TV) stehen nur als Text in der Nachricht
|
||||||
|
for (const word of msg.text.split(/\s+/)) {
|
||||||
|
if (laugh.has(word.toLowerCase())) emoteHits++;
|
||||||
|
}
|
||||||
|
return {
|
||||||
|
ts: Date.parse(msg.ts),
|
||||||
|
userId: msg.userId,
|
||||||
|
emoteHits,
|
||||||
|
clipMention: /\bclip\b/i.test(msg.text) ? 1 : 0,
|
||||||
|
shout: isShout(msg.text),
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Gibt einen Kandidaten zurueck, wenn das Fenster auffaellig ist - sonst null. */
|
||||||
|
function push(msg: ChatMessage): SpikeEvent | null {
|
||||||
|
const entry = score(msg);
|
||||||
|
window.push(entry);
|
||||||
|
|
||||||
|
const now = entry.ts;
|
||||||
|
const windowStart = now - cfg.windowSeconds * 1000;
|
||||||
|
while (window.length && window[0]!.ts < windowStart) window.shift();
|
||||||
|
|
||||||
|
const rate = window.length;
|
||||||
|
rateHistory.push({ ts: now, rate });
|
||||||
|
const baseStart = now - cfg.baselineMinutes * 60_000;
|
||||||
|
while (rateHistory.length && rateHistory[0]!.ts < baseStart) rateHistory.shift();
|
||||||
|
|
||||||
|
// Median statt Mittelwert: ein einzelner Ausbruch soll die Basislinie
|
||||||
|
// nicht mit anheben und damit den naechsten Ausbruch verstecken.
|
||||||
|
const sorted = rateHistory.map((r) => r.rate).sort((a, b) => a - b);
|
||||||
|
const baseline = sorted.length
|
||||||
|
? sorted[Math.floor(sorted.length / 2)]!
|
||||||
|
: 0;
|
||||||
|
|
||||||
|
const threshold = Math.max(baseline * cfg.factor, baseline + cfg.minDelta);
|
||||||
|
const chatters = new Set(window.map((e) => e.userId)).size;
|
||||||
|
const emoteHits = window.reduce((n, e) => n + e.emoteHits, 0);
|
||||||
|
const clipMentions = window.reduce((n, e) => n + e.clipMention, 0);
|
||||||
|
const shouts = window.reduce((n, e) => n + e.shout, 0);
|
||||||
|
|
||||||
|
const overThreshold = rate >= threshold && rate >= cfg.minDelta;
|
||||||
|
const cooled = now - lastSpike > cfg.cooldownSeconds * 1000;
|
||||||
|
if (!overThreshold || !cooled) return null;
|
||||||
|
lastSpike = now;
|
||||||
|
|
||||||
|
// Mehrere Personen wiegen schwerer als viele Nachrichten einer Person -
|
||||||
|
// sonst erzeugt ein einzelner Vielschreiber laufend Kandidaten.
|
||||||
|
const value =
|
||||||
|
(baseline > 0 ? rate / baseline : rate) * 1.0 +
|
||||||
|
chatters * 1.5 +
|
||||||
|
emoteHits * 0.8 +
|
||||||
|
shouts * 0.5 +
|
||||||
|
clipMentions * 3.0;
|
||||||
|
|
||||||
|
return {
|
||||||
|
score: Math.round(value * 100) / 100,
|
||||||
|
rate,
|
||||||
|
baseline,
|
||||||
|
chatters,
|
||||||
|
emoteHits,
|
||||||
|
clipMentions,
|
||||||
|
windowFrom: new Date(windowStart).toISOString(),
|
||||||
|
windowTo: new Date(now).toISOString(),
|
||||||
|
overThreshold: cfg.armed,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
return { push, isArmed: () => cfg.armed };
|
||||||
|
}
|
||||||
@@ -1,5 +0,0 @@
|
|||||||
{
|
|
||||||
"extends": "../../tsconfig.base.json",
|
|
||||||
"compilerOptions": { "outDir": "dist", "rootDir": "src" },
|
|
||||||
"include": ["src"]
|
|
||||||
}
|
|
||||||
@@ -1,12 +0,0 @@
|
|||||||
{
|
|
||||||
"name": "@abdulive/loyalty",
|
|
||||||
"version": "0.1.0",
|
|
||||||
"private": true,
|
|
||||||
"type": "module",
|
|
||||||
"main": "dist/index.js",
|
|
||||||
"scripts": {
|
|
||||||
"build": "tsc -p tsconfig.json",
|
|
||||||
"typecheck": "tsc -p tsconfig.json --noEmit",
|
|
||||||
"test": "node --test"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,2 +0,0 @@
|
|||||||
// @abdulive/loyalty — siehe docs/projects/loyalty-shop.md
|
|
||||||
export {};
|
|
||||||
@@ -1,5 +0,0 @@
|
|||||||
{
|
|
||||||
"extends": "../../tsconfig.base.json",
|
|
||||||
"compilerOptions": { "outDir": "dist", "rootDir": "src" },
|
|
||||||
"include": ["src"]
|
|
||||||
}
|
|
||||||
@@ -1,12 +0,0 @@
|
|||||||
{
|
|
||||||
"name": "@abdulive/overlay-server",
|
|
||||||
"version": "0.1.0",
|
|
||||||
"private": true,
|
|
||||||
"type": "module",
|
|
||||||
"main": "dist/index.js",
|
|
||||||
"scripts": {
|
|
||||||
"build": "tsc -p tsconfig.json",
|
|
||||||
"typecheck": "tsc -p tsconfig.json --noEmit",
|
|
||||||
"test": "node --test"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,2 +0,0 @@
|
|||||||
// @abdulive/overlay-server — siehe docs/projects/chat-control.md
|
|
||||||
export {};
|
|
||||||
@@ -1,5 +0,0 @@
|
|||||||
{
|
|
||||||
"extends": "../../tsconfig.base.json",
|
|
||||||
"compilerOptions": { "outDir": "dist", "rootDir": "src" },
|
|
||||||
"include": ["src"]
|
|
||||||
}
|
|
||||||
@@ -1,12 +0,0 @@
|
|||||||
{
|
|
||||||
"name": "@abdulive/overlay-web",
|
|
||||||
"version": "0.1.0",
|
|
||||||
"private": true,
|
|
||||||
"type": "module",
|
|
||||||
"main": "dist/index.js",
|
|
||||||
"scripts": {
|
|
||||||
"build": "tsc -p tsconfig.json",
|
|
||||||
"typecheck": "tsc -p tsconfig.json --noEmit",
|
|
||||||
"test": "node --test"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,2 +0,0 @@
|
|||||||
// @abdulive/overlay-web — siehe docs/projects/chat-control.md
|
|
||||||
export {};
|
|
||||||
@@ -1,5 +0,0 @@
|
|||||||
{
|
|
||||||
"extends": "../../tsconfig.base.json",
|
|
||||||
"compilerOptions": { "outDir": "dist", "rootDir": "src" },
|
|
||||||
"include": ["src"]
|
|
||||||
}
|
|
||||||
@@ -1,12 +0,0 @@
|
|||||||
{
|
|
||||||
"name": "@abdulive/raid-scout",
|
|
||||||
"version": "0.1.0",
|
|
||||||
"private": true,
|
|
||||||
"type": "module",
|
|
||||||
"main": "dist/index.js",
|
|
||||||
"scripts": {
|
|
||||||
"build": "tsc -p tsconfig.json",
|
|
||||||
"typecheck": "tsc -p tsconfig.json --noEmit",
|
|
||||||
"test": "node --test"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,2 +0,0 @@
|
|||||||
// @abdulive/raid-scout — siehe docs/projects/raid-network.md
|
|
||||||
export {};
|
|
||||||
@@ -1,5 +0,0 @@
|
|||||||
{
|
|
||||||
"extends": "../../tsconfig.base.json",
|
|
||||||
"compilerOptions": { "outDir": "dist", "rootDir": "src" },
|
|
||||||
"include": ["src"]
|
|
||||||
}
|
|
||||||
@@ -1,12 +0,0 @@
|
|||||||
{
|
|
||||||
"name": "@abdulive/recap",
|
|
||||||
"version": "0.1.0",
|
|
||||||
"private": true,
|
|
||||||
"type": "module",
|
|
||||||
"main": "dist/index.js",
|
|
||||||
"scripts": {
|
|
||||||
"build": "tsc -p tsconfig.json",
|
|
||||||
"typecheck": "tsc -p tsconfig.json --noEmit",
|
|
||||||
"test": "node --test"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,2 +0,0 @@
|
|||||||
// @abdulive/recap — siehe docs/projects/stream-recap.md
|
|
||||||
export {};
|
|
||||||
@@ -1,5 +0,0 @@
|
|||||||
{
|
|
||||||
"extends": "../../tsconfig.base.json",
|
|
||||||
"compilerOptions": { "outDir": "dist", "rootDir": "src" },
|
|
||||||
"include": ["src"]
|
|
||||||
}
|
|
||||||
@@ -1,12 +0,0 @@
|
|||||||
{
|
|
||||||
"name": "@abdulive/vod-chapters",
|
|
||||||
"version": "0.1.0",
|
|
||||||
"private": true,
|
|
||||||
"type": "module",
|
|
||||||
"main": "dist/index.js",
|
|
||||||
"scripts": {
|
|
||||||
"build": "tsc -p tsconfig.json",
|
|
||||||
"typecheck": "tsc -p tsconfig.json --noEmit",
|
|
||||||
"test": "node --test"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,2 +0,0 @@
|
|||||||
// @abdulive/vod-chapters — siehe docs/projects/vod-chapters.md
|
|
||||||
export {};
|
|
||||||
@@ -1,5 +0,0 @@
|
|||||||
{
|
|
||||||
"extends": "../../tsconfig.base.json",
|
|
||||||
"compilerOptions": { "outDir": "dist", "rootDir": "src" },
|
|
||||||
"include": ["src"]
|
|
||||||
}
|
|
||||||
@@ -33,6 +33,12 @@ clipper:
|
|||||||
alsoCreateTwitchClip: true
|
alsoCreateTwitchClip: true
|
||||||
maxDurationSeconds: 60
|
maxDurationSeconds: 60
|
||||||
renderWhileLive: false # true = sofort rendern, false = erst nach Streamende
|
renderWhileLive: false # true = sofort rendern, false = erst nach Streamende
|
||||||
|
audio:
|
||||||
|
# OBS nimmt getrennte Spuren auf. Untertitel entstehen NUR aus dem Mikrofon -
|
||||||
|
# Spielton im Transkript macht Whisper unbrauchbar. Die Spur im fertigen
|
||||||
|
# Video bleibt die Mischung.
|
||||||
|
transcribeTrack: 2 # OBS-Spur mit dem Mikrofon
|
||||||
|
outputTrack: 1 # OBS-Spur mit der Mischung (kommt ins Short)
|
||||||
whisper:
|
whisper:
|
||||||
binary: whisper-cli # whisper.cpp, im Image enthalten
|
binary: whisper-cli # whisper.cpp, im Image enthalten
|
||||||
model: ggml-small.bin
|
model: ggml-small.bin
|
||||||
|
|||||||
+16
-6
@@ -1,6 +1,8 @@
|
|||||||
# Stand: Scheibe S0 aus docs/07-mvp-plan.md.
|
# Stand: Scheiben S0 und S1 aus docs/07-mvp-plan.md.
|
||||||
# Es gibt noch keinen Daemon - der Container fuehrt einzelne Befehle aus
|
#
|
||||||
# (check-env, fetch-model, calibrate). Ab S1 wird daraus ein Langlaeufer.
|
# Nach `docker compose up -d` laeuft der Ingest-Daemon und die Einrichtung
|
||||||
|
# ist unter http://localhost:4792/setup erreichbar. Zugangsdaten werden dort
|
||||||
|
# eingegeben, nicht hier - siehe docs/decisions/ADR-0005-zugangsdaten.md.
|
||||||
#
|
#
|
||||||
# OBS_REPLAY_DIR in .env auf den Wiedergabepuffer-Ordner setzen, z. B.
|
# OBS_REPLAY_DIR in .env auf den Wiedergabepuffer-Ordner setzen, z. B.
|
||||||
# OBS_REPLAY_DIR=C:/Users/abdu/Videos/replay
|
# OBS_REPLAY_DIR=C:/Users/abdu/Videos/replay
|
||||||
@@ -15,16 +17,17 @@ services:
|
|||||||
env_file:
|
env_file:
|
||||||
- path: .env
|
- path: .env
|
||||||
required: false
|
required: false
|
||||||
command: ["help"]
|
command: ["ingest"]
|
||||||
|
|
||||||
# OBS laeuft auf dem Host, nicht im Container
|
# OBS laeuft auf dem Host, nicht im Container
|
||||||
extra_hosts:
|
extra_hosts:
|
||||||
- "host.docker.internal:host-gateway"
|
- "host.docker.internal:host-gateway"
|
||||||
|
|
||||||
ports:
|
ports:
|
||||||
- "127.0.0.1:4780:4780" # OAuth-Rueckruf, nur waehrend `auth` (ab S1)
|
# Ein Server fuer Einrichtung, OAuth-Rueckruf, Sichtungsseite und
|
||||||
|
# Dashboard - siehe docs/decisions/ADR-0005-zugangsdaten.md
|
||||||
|
- "127.0.0.1:4792:4792"
|
||||||
- "127.0.0.1:4791:4791" # Overlay-Server (ab M3)
|
- "127.0.0.1:4791:4791" # Overlay-Server (ab M3)
|
||||||
- "127.0.0.1:4792:4792" # Sichtungsseite (ab S5)
|
|
||||||
|
|
||||||
volumes:
|
volumes:
|
||||||
# Konfiguration: nur lesend
|
# Konfiguration: nur lesend
|
||||||
@@ -43,6 +46,13 @@ services:
|
|||||||
cpus: "3.0"
|
cpus: "3.0"
|
||||||
mem_limit: 3g
|
mem_limit: 3g
|
||||||
|
|
||||||
|
healthcheck:
|
||||||
|
test: ["CMD", "curl", "-fsS", "http://127.0.0.1:4792/health"]
|
||||||
|
interval: 30s
|
||||||
|
timeout: 5s
|
||||||
|
retries: 3
|
||||||
|
start_period: 20s
|
||||||
|
|
||||||
volumes:
|
volumes:
|
||||||
abdulive-db:
|
abdulive-db:
|
||||||
abdulive-models:
|
abdulive-models:
|
||||||
|
|||||||
+17
-12
@@ -1,8 +1,9 @@
|
|||||||
# Scheibe S0 aus docs/07-mvp-plan.md: Toolchain-Image.
|
# Scheiben S0 und S1 aus docs/07-mvp-plan.md.
|
||||||
#
|
#
|
||||||
# Bewusst noch ohne `pnpm install` - es gibt noch keine Abhaengigkeiten und
|
# Das Projekt hat KEINE npm-Abhaengigkeiten: Node 24 bringt SQLite
|
||||||
# keine Lockfile. Der Entrypoint kommt mit reinem Node aus. Sobald in S1
|
# (node:sqlite), WebSocket und fetch mit und fuehrt TypeScript direkt aus
|
||||||
# echter Code dazukommt, wird hier eine deps-Stufe ergaenzt.
|
# (Type Stripping). Deshalb gibt es weder eine deps-Stufe noch einen
|
||||||
|
# Build-Schritt. Siehe ADR-0001 und ADR-0004.
|
||||||
|
|
||||||
# ---------- whisper.cpp bauen ----------
|
# ---------- whisper.cpp bauen ----------
|
||||||
FROM debian:bookworm-slim AS whisper
|
FROM debian:bookworm-slim AS whisper
|
||||||
@@ -18,11 +19,10 @@ RUN cmake -B build -DCMAKE_BUILD_TYPE=Release -DWHISPER_BUILD_TESTS=OFF \
|
|||||||
&& cmake --build build --config Release -j"$(nproc)" \
|
&& cmake --build build --config Release -j"$(nproc)" \
|
||||||
&& mkdir -p /out \
|
&& mkdir -p /out \
|
||||||
&& cp build/bin/whisper-cli /out/ \
|
&& cp build/bin/whisper-cli /out/ \
|
||||||
&& (cp build/src/libwhisper.so* /out/ 2>/dev/null || true) \
|
&& find build -name 'libwhisper.so*' -o -name 'libggml*.so' | xargs -r cp -P -t /out/
|
||||||
&& (cp build/ggml/src/libggml*.so /out/ 2>/dev/null || true)
|
|
||||||
|
|
||||||
# ---------- Laufzeit ----------
|
# ---------- Laufzeit ----------
|
||||||
FROM node:22-bookworm-slim
|
FROM node:24-bookworm-slim
|
||||||
|
|
||||||
RUN apt-get update && apt-get install -y --no-install-recommends \
|
RUN apt-get update && apt-get install -y --no-install-recommends \
|
||||||
ffmpeg fonts-dejavu-core ca-certificates curl \
|
ffmpeg fonts-dejavu-core ca-certificates curl \
|
||||||
@@ -34,10 +34,13 @@ RUN ln -s /usr/local/lib/whisper/whisper-cli /usr/local/bin/whisper-cli \
|
|||||||
&& ldconfig
|
&& ldconfig
|
||||||
|
|
||||||
WORKDIR /app
|
WORKDIR /app
|
||||||
COPY docker/ ./docker/
|
# package.json zuerst: sie enthaelt die "imports"-Map, ueber die #core/... aufgeloest wird
|
||||||
COPY package.json ./
|
COPY package.json ./
|
||||||
|
COPY packages/ ./packages/
|
||||||
|
COPY apps/ ./apps/
|
||||||
|
COPY docker/ ./docker/
|
||||||
|
|
||||||
# Die Datenbank und die Sprachmodelle liegen in benannten Volumes, nicht im
|
# Datenbank und Sprachmodelle liegen in benannten Volumes, nicht im
|
||||||
# Bind-Mount - siehe docs/decisions/ADR-0004-docker.md
|
# Bind-Mount - siehe docs/decisions/ADR-0004-docker.md
|
||||||
ENV WHISPER_BIN=/usr/local/bin/whisper-cli \
|
ENV WHISPER_BIN=/usr/local/bin/whisper-cli \
|
||||||
MODEL_DIR=/models \
|
MODEL_DIR=/models \
|
||||||
@@ -46,9 +49,11 @@ ENV WHISPER_BIN=/usr/local/bin/whisper-cli \
|
|||||||
CLIPS_DIR=/app/data/clips \
|
CLIPS_DIR=/app/data/clips \
|
||||||
REPLAY_DIR=/replay \
|
REPLAY_DIR=/replay \
|
||||||
CONFIG_DIR=/app/config \
|
CONFIG_DIR=/app/config \
|
||||||
NODE_ENV=production
|
NODE_ENV=production \
|
||||||
|
NODE_OPTIONS=--disable-warning=ExperimentalWarning \
|
||||||
|
BIND_HOST=0.0.0.0
|
||||||
|
|
||||||
EXPOSE 4780 4790 4791 4792
|
EXPOSE 4790 4791 4792
|
||||||
|
|
||||||
ENTRYPOINT ["node", "/app/docker/entrypoint.js"]
|
ENTRYPOINT ["node", "/app/docker/entrypoint.ts"]
|
||||||
CMD ["help"]
|
CMD ["help"]
|
||||||
@@ -1,353 +0,0 @@
|
|||||||
#!/usr/bin/env node
|
|
||||||
// Einstiegspunkt des Container-Images. Scheibe S0 aus docs/07-mvp-plan.md.
|
|
||||||
//
|
|
||||||
// Bewusst ohne npm-Abhaengigkeiten: dieses Skript muss laufen, bevor
|
|
||||||
// irgendetwas anderes installiert oder gebaut ist.
|
|
||||||
|
|
||||||
import { execFile, execFileSync } from 'node:child_process';
|
|
||||||
import { createWriteStream } from 'node:fs';
|
|
||||||
import fs from 'node:fs/promises';
|
|
||||||
import net from 'node:net';
|
|
||||||
import path from 'node:path';
|
|
||||||
import { pipeline } from 'node:stream/promises';
|
|
||||||
import { parseYaml } from './lib/yaml.js';
|
|
||||||
|
|
||||||
const ENV = {
|
|
||||||
whisper: process.env.WHISPER_BIN || 'whisper-cli',
|
|
||||||
modelDir: process.env.MODEL_DIR || '/models',
|
|
||||||
dbDir: process.env.DB_DIR || '/db',
|
|
||||||
workDir: process.env.WORK_DIR || '/work',
|
|
||||||
clipsDir: process.env.CLIPS_DIR || '/app/data/clips',
|
|
||||||
replayDir: process.env.REPLAY_DIR || '/replay',
|
|
||||||
configDir: process.env.CONFIG_DIR || '/app/config',
|
|
||||||
obsHost: process.env.OBS_WS_HOST || 'host.docker.internal',
|
|
||||||
obsPort: Number(process.env.OBS_WS_PORT || 4455),
|
|
||||||
};
|
|
||||||
|
|
||||||
const MODELS = {
|
|
||||||
'ggml-small.bin': 'https://huggingface.co/ggerganov/whisper.cpp/resolve/main/ggml-small.bin',
|
|
||||||
'ggml-medium.bin': 'https://huggingface.co/ggerganov/whisper.cpp/resolve/main/ggml-medium.bin',
|
|
||||||
'ggml-base.bin': 'https://huggingface.co/ggerganov/whisper.cpp/resolve/main/ggml-base.bin',
|
|
||||||
};
|
|
||||||
|
|
||||||
// ---------------------------------------------------------------- Ausgabe
|
|
||||||
|
|
||||||
const C = process.stdout.isTTY
|
|
||||||
? { ok: '\x1b[32m', warn: '\x1b[33m', bad: '\x1b[31m', dim: '\x1b[90m', off: '\x1b[0m' }
|
|
||||||
: { ok: '', warn: '', bad: '', dim: '', off: '' };
|
|
||||||
|
|
||||||
let hardFailures = 0;
|
|
||||||
let warnings = 0;
|
|
||||||
|
|
||||||
function report(state, label, detail = '') {
|
|
||||||
const mark = { ok: `${C.ok} OK ${C.off}`, warn: `${C.warn} WARN ${C.off}`, bad: `${C.bad} FEHLT${C.off}` }[state];
|
|
||||||
if (state === 'bad') hardFailures++;
|
|
||||||
if (state === 'warn') warnings++;
|
|
||||||
console.log(`${mark} ${label.padEnd(34)} ${C.dim}${detail}${C.off}`);
|
|
||||||
}
|
|
||||||
|
|
||||||
function section(title) {
|
|
||||||
console.log(`\n${title}`);
|
|
||||||
console.log('-'.repeat(Math.max(title.length, 40)));
|
|
||||||
}
|
|
||||||
|
|
||||||
// ---------------------------------------------------------------- Helfer
|
|
||||||
|
|
||||||
function run(cmd, args, opts = {}) {
|
|
||||||
return execFileSync(cmd, args, { encoding: 'utf8', stdio: ['ignore', 'pipe', 'pipe'], ...opts });
|
|
||||||
}
|
|
||||||
|
|
||||||
function runAsync(cmd, args, opts = {}) {
|
|
||||||
return new Promise((resolve, reject) => {
|
|
||||||
execFile(cmd, args, { maxBuffer: 32 * 1024 * 1024, ...opts }, (err, stdout, stderr) =>
|
|
||||||
err ? reject(Object.assign(err, { stdout, stderr })) : resolve({ stdout, stderr })
|
|
||||||
);
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
async function canWrite(dir) {
|
|
||||||
const probe = path.join(dir, `.probe-${process.pid}`);
|
|
||||||
try {
|
|
||||||
await fs.mkdir(dir, { recursive: true });
|
|
||||||
await fs.writeFile(probe, 'x');
|
|
||||||
await fs.unlink(probe);
|
|
||||||
return true;
|
|
||||||
} catch {
|
|
||||||
return false;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
function tcpProbe(host, port, timeout = 2500) {
|
|
||||||
return new Promise((resolve) => {
|
|
||||||
const sock = new net.Socket();
|
|
||||||
const done = (result) => { sock.destroy(); resolve(result); };
|
|
||||||
sock.setTimeout(timeout);
|
|
||||||
sock.once('connect', () => done(true));
|
|
||||||
sock.once('timeout', () => done(false));
|
|
||||||
sock.once('error', () => done(false));
|
|
||||||
sock.connect(port, host);
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
async function readLayout() {
|
|
||||||
const file = path.join(ENV.configDir, 'layout.yaml');
|
|
||||||
const text = await fs.readFile(file, 'utf8');
|
|
||||||
return { file, layout: parseYaml(text) };
|
|
||||||
}
|
|
||||||
|
|
||||||
// ---------------------------------------------------------------- check-env
|
|
||||||
|
|
||||||
async function checkEnv() {
|
|
||||||
console.log('\nabdulive-toolkit - Selbsttest der Umgebung');
|
|
||||||
|
|
||||||
section('Laufzeit');
|
|
||||||
const major = Number(process.versions.node.split('.')[0]);
|
|
||||||
report(major >= 22 ? 'ok' : 'bad', 'Node.js', `v${process.versions.node}`);
|
|
||||||
|
|
||||||
section('Medienwerkzeuge');
|
|
||||||
try {
|
|
||||||
const v = run('ffmpeg', ['-version']).split('\n')[0];
|
|
||||||
report('ok', 'ffmpeg', v.replace('ffmpeg version ', '').slice(0, 40));
|
|
||||||
const cfg = run('ffmpeg', ['-hide_banner', '-buildconf']);
|
|
||||||
report(cfg.includes('--enable-libx264') ? 'ok' : 'bad', ' libx264 (H.264-Encoder)');
|
|
||||||
report(cfg.includes('--enable-libass') ? 'ok' : 'bad', ' libass (Untertitel einbrennen)');
|
|
||||||
} catch {
|
|
||||||
report('bad', 'ffmpeg', 'nicht im PATH');
|
|
||||||
}
|
|
||||||
try {
|
|
||||||
run('ffprobe', ['-version']);
|
|
||||||
report('ok', 'ffprobe');
|
|
||||||
} catch {
|
|
||||||
report('bad', 'ffprobe', 'nicht im PATH');
|
|
||||||
}
|
|
||||||
try {
|
|
||||||
const out = run(ENV.whisper, ['--help'], { stdio: ['ignore', 'pipe', 'pipe'] });
|
|
||||||
report('ok', 'whisper-cli', out.split('\n')[0].slice(0, 40) || ENV.whisper);
|
|
||||||
} catch (err) {
|
|
||||||
// whisper-cli beendet sich bei --help teils mit Code != 0, Ausgabe zaehlt
|
|
||||||
const text = `${err.stdout || ''}${err.stderr || ''}`;
|
|
||||||
report(text.includes('usage') || text.includes('whisper') ? 'ok' : 'bad', 'whisper-cli',
|
|
||||||
text.includes('whisper') ? '' : 'nicht lauffaehig');
|
|
||||||
}
|
|
||||||
|
|
||||||
section('Sprachmodell');
|
|
||||||
let anyModel = false;
|
|
||||||
for (const name of Object.keys(MODELS)) {
|
|
||||||
try {
|
|
||||||
const st = await fs.stat(path.join(ENV.modelDir, name));
|
|
||||||
report('ok', name, `${(st.size / 1024 / 1024).toFixed(0)} MB`);
|
|
||||||
anyModel = true;
|
|
||||||
} catch { /* nicht vorhanden, ist in Ordnung */ }
|
|
||||||
}
|
|
||||||
if (!anyModel) report('warn', 'kein Modell vorhanden', 'holen mit: fetch-model ggml-small.bin');
|
|
||||||
|
|
||||||
section('Verzeichnisse');
|
|
||||||
for (const [label, dir] of [
|
|
||||||
['Datenbank (Volume)', ENV.dbDir],
|
|
||||||
['Arbeitsverzeichnis (Volume)', ENV.workDir],
|
|
||||||
['Sprachmodelle (Volume)', ENV.modelDir],
|
|
||||||
['Clips (Bind-Mount)', ENV.clipsDir],
|
|
||||||
]) {
|
|
||||||
report(await canWrite(dir) ? 'ok' : 'bad', label, `${dir} beschreibbar`);
|
|
||||||
}
|
|
||||||
for (const [label, dir, hard] of [
|
|
||||||
['OBS-Wiedergabepuffer', ENV.replayDir, false],
|
|
||||||
['Konfiguration', ENV.configDir, true],
|
|
||||||
]) {
|
|
||||||
try {
|
|
||||||
const n = (await fs.readdir(dir)).length;
|
|
||||||
report('ok', label, `${dir} lesbar, ${n} Eintraege`);
|
|
||||||
} catch {
|
|
||||||
report(hard ? 'bad' : 'warn', label, `${dir} nicht lesbar - Mount gesetzt?`);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
section('Layout (Facecam-Ausschnitt)');
|
|
||||||
try {
|
|
||||||
const { layout } = await readLayout();
|
|
||||||
const prof = layout.profiles?.default;
|
|
||||||
if (!prof) throw new Error('Profil "default" fehlt');
|
|
||||||
const { cam, game } = prof;
|
|
||||||
const sum = cam.band.h + game.band.h;
|
|
||||||
report(sum === layout.target.height ? 'ok' : 'bad', 'Baender ergeben Zielhoehe',
|
|
||||||
`${cam.band.h} + ${game.band.h} = ${sum} (soll ${layout.target.height})`);
|
|
||||||
for (const [name, c] of [['Facecam', cam.crop], ['Gameplay', game.crop]]) {
|
|
||||||
const inside = c.x >= 0 && c.y >= 0 &&
|
|
||||||
c.x + c.w <= layout.source.width && c.y + c.h <= layout.source.height;
|
|
||||||
report(inside ? 'ok' : 'bad', `${name}-Ausschnitt im Bild`,
|
|
||||||
`${c.w}x${c.h} bei ${c.x},${c.y}`);
|
|
||||||
}
|
|
||||||
for (const [name, c, b] of [['Facecam', cam.crop, cam.band], ['Gameplay', game.crop, game.band]]) {
|
|
||||||
const dev = Math.abs((c.w / c.h) / (b.w / b.h) - 1) * 100;
|
|
||||||
report(dev < 1 ? 'ok' : 'warn', `${name}-Seitenverhaeltnis`,
|
|
||||||
`Abweichung ${dev.toFixed(2)} % (>1 % verzerrt sichtbar)`);
|
|
||||||
}
|
|
||||||
} catch (err) {
|
|
||||||
report('bad', 'config/layout.yaml', err.message);
|
|
||||||
}
|
|
||||||
|
|
||||||
section('OBS');
|
|
||||||
const obsUp = await tcpProbe(ENV.obsHost, ENV.obsPort);
|
|
||||||
report(obsUp ? 'ok' : 'warn', 'obs-websocket erreichbar',
|
|
||||||
`${ENV.obsHost}:${ENV.obsPort}${obsUp ? '' : ' - OBS laeuft nicht oder WebSocket-Server ist aus'}`);
|
|
||||||
|
|
||||||
section('Zugangsdaten (.env)');
|
|
||||||
for (const key of ['TWITCH_CLIENT_ID', 'TWITCH_CLIENT_SECRET', 'OBS_WS_PASSWORD']) {
|
|
||||||
report(process.env[key] ? 'ok' : 'warn', key, process.env[key] ? 'gesetzt' : 'fehlt');
|
|
||||||
}
|
|
||||||
|
|
||||||
console.log('');
|
|
||||||
if (hardFailures > 0) {
|
|
||||||
console.log(`${C.bad}${hardFailures} harte Fehler${C.off}, ${warnings} Warnungen. Das Image ist noch nicht einsatzbereit.`);
|
|
||||||
process.exitCode = 1;
|
|
||||||
} else if (warnings > 0) {
|
|
||||||
console.log(`${C.ok}Keine harten Fehler${C.off}, ${warnings} Warnungen - die sind fuer S0 in Ordnung.`);
|
|
||||||
} else {
|
|
||||||
console.log(`${C.ok}Alles in Ordnung.${C.off}`);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// ---------------------------------------------------------------- fetch-model
|
|
||||||
|
|
||||||
async function fetchModel(name = 'ggml-small.bin') {
|
|
||||||
const url = MODELS[name];
|
|
||||||
if (!url) {
|
|
||||||
console.error(`Unbekanntes Modell "${name}". Bekannt: ${Object.keys(MODELS).join(', ')}`);
|
|
||||||
process.exitCode = 1;
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
await fs.mkdir(ENV.modelDir, { recursive: true });
|
|
||||||
const target = path.join(ENV.modelDir, name);
|
|
||||||
try {
|
|
||||||
const st = await fs.stat(target);
|
|
||||||
console.log(`${name} liegt schon da (${(st.size / 1024 / 1024).toFixed(0)} MB).`);
|
|
||||||
return;
|
|
||||||
} catch { /* nicht da, also laden */ }
|
|
||||||
|
|
||||||
console.log(`Lade ${name} ...`);
|
|
||||||
const res = await fetch(url, { redirect: 'follow' });
|
|
||||||
if (!res.ok) throw new Error(`Download fehlgeschlagen: HTTP ${res.status}`);
|
|
||||||
const total = Number(res.headers.get('content-length') || 0);
|
|
||||||
let seen = 0;
|
|
||||||
let lastPrint = 0;
|
|
||||||
const tmp = `${target}.part`;
|
|
||||||
const out = createWriteStream(tmp);
|
|
||||||
const body = res.body.getReader ? res.body : res.body;
|
|
||||||
await pipeline(
|
|
||||||
(async function* () {
|
|
||||||
for await (const chunk of body) {
|
|
||||||
seen += chunk.length;
|
|
||||||
const now = Date.now();
|
|
||||||
if (now - lastPrint > 1000) {
|
|
||||||
lastPrint = now;
|
|
||||||
const pct = total ? ` (${((seen / total) * 100).toFixed(0)} %)` : '';
|
|
||||||
process.stdout.write(`\r ${(seen / 1024 / 1024).toFixed(0)} MB${pct} `);
|
|
||||||
}
|
|
||||||
yield chunk;
|
|
||||||
}
|
|
||||||
})(),
|
|
||||||
out
|
|
||||||
);
|
|
||||||
await fs.rename(tmp, target);
|
|
||||||
console.log(`\rFertig: ${target} (${(seen / 1024 / 1024).toFixed(0)} MB)`);
|
|
||||||
}
|
|
||||||
|
|
||||||
// ---------------------------------------------------------------- calibrate
|
|
||||||
|
|
||||||
function buildFilter(prof) {
|
|
||||||
const { cam, game } = prof;
|
|
||||||
return [
|
|
||||||
`[0:v]crop=${cam.crop.w}:${cam.crop.h}:${cam.crop.x}:${cam.crop.y},scale=${cam.band.w}:${cam.band.h},setsar=1[cam]`,
|
|
||||||
`[0:v]crop=${game.crop.w}:${game.crop.h}:${game.crop.x}:${game.crop.y},scale=${game.band.w}:${game.band.h},setsar=1[game]`,
|
|
||||||
`[cam][game]vstack=inputs=2[v]`,
|
|
||||||
].join(';');
|
|
||||||
}
|
|
||||||
|
|
||||||
async function calibrate(input, profileName = 'default') {
|
|
||||||
const { layout } = await readLayout();
|
|
||||||
const prof = layout.profiles?.[profileName];
|
|
||||||
if (!prof) throw new Error(`Profil "${profileName}" steht nicht in config/layout.yaml`);
|
|
||||||
|
|
||||||
if (!input) {
|
|
||||||
const files = (await fs.readdir(ENV.replayDir).catch(() => []))
|
|
||||||
.filter((f) => /\.(mp4|mkv|mov|flv|png|jpg)$/i.test(f))
|
|
||||||
.sort();
|
|
||||||
input = files.length ? path.join(ENV.replayDir, files[files.length - 1]) : null;
|
|
||||||
if (!input) throw new Error(`Keine Datei angegeben und nichts in ${ENV.replayDir} gefunden.`);
|
|
||||||
console.log(`Neueste Datei aus ${ENV.replayDir}: ${path.basename(input)}`);
|
|
||||||
}
|
|
||||||
|
|
||||||
const outDir = path.join(ENV.clipsDir, '_calibration');
|
|
||||||
await fs.mkdir(outDir, { recursive: true });
|
|
||||||
|
|
||||||
const isImage = /\.(png|jpe?g)$/i.test(input);
|
|
||||||
const seek = isImage ? [] : ['-ss', '5'];
|
|
||||||
|
|
||||||
// 1. Referenzbild aus der Quelle, unveraendert
|
|
||||||
const frame = path.join(outDir, 'frame.png');
|
|
||||||
await runAsync('ffmpeg', ['-y', '-hide_banner', '-loglevel', 'error',
|
|
||||||
...seek, '-i', input, '-frames:v', '1', frame]);
|
|
||||||
|
|
||||||
// 2. Vorschau der 9:16-Komposition mit den Werten aus layout.yaml
|
|
||||||
const preview = path.join(outDir, `preview-${profileName}.png`);
|
|
||||||
await runAsync('ffmpeg', ['-y', '-hide_banner', '-loglevel', 'error',
|
|
||||||
'-i', frame, '-filter_complex', buildFilter(prof), '-map', '[v]',
|
|
||||||
'-frames:v', '1', preview]);
|
|
||||||
|
|
||||||
// 3. Kontrollbild: Ausschnitte im Originalbild markiert
|
|
||||||
const marked = path.join(outDir, 'ausschnitte.png');
|
|
||||||
const box = (c, color) =>
|
|
||||||
`drawbox=x=${c.x}:y=${c.y}:w=${c.w}:h=${c.h}:color=${color}:t=4`;
|
|
||||||
await runAsync('ffmpeg', ['-y', '-hide_banner', '-loglevel', 'error',
|
|
||||||
'-i', frame, '-vf',
|
|
||||||
`${box(prof.cam.crop, 'yellow')},${box(prof.game.crop, 'cyan')}`,
|
|
||||||
'-frames:v', '1', marked]);
|
|
||||||
|
|
||||||
console.log('\nGeschrieben:');
|
|
||||||
console.log(` ${frame} Einzelbild aus der Quelle`);
|
|
||||||
console.log(` ${marked} Ausschnitte markiert (gelb = Cam, cyan = Gameplay)`);
|
|
||||||
console.log(` ${preview} fertige 9:16-Vorschau`);
|
|
||||||
console.log('\nSitzt der Ausschnitt nicht, die Werte in config/layout.yaml anpassen');
|
|
||||||
console.log('und den Befehl erneut ausfuehren.');
|
|
||||||
}
|
|
||||||
|
|
||||||
// ---------------------------------------------------------------- help
|
|
||||||
|
|
||||||
function help() {
|
|
||||||
console.log(`
|
|
||||||
abdulive-toolkit - Container-Einstiegspunkt
|
|
||||||
|
|
||||||
check-env Umgebung pruefen (Werkzeuge, Volumes, Layout, OBS)
|
|
||||||
fetch-model [name] Whisper-Modell laden (Vorgabe: ggml-small.bin)
|
|
||||||
calibrate [datei] [profil] Facecam-Ausschnitt pruefen, Vorschau rendern
|
|
||||||
help Diese Uebersicht
|
|
||||||
|
|
||||||
Beispiele:
|
|
||||||
docker compose run --rm toolkit check-env
|
|
||||||
docker compose run --rm toolkit fetch-model ggml-small.bin
|
|
||||||
docker compose run --rm toolkit calibrate
|
|
||||||
|
|
||||||
Stand: Scheibe S0 aus docs/07-mvp-plan.md. Ingest, Clipping und Untertitel
|
|
||||||
folgen in S1 bis S4.
|
|
||||||
`);
|
|
||||||
}
|
|
||||||
|
|
||||||
// ---------------------------------------------------------------- main
|
|
||||||
|
|
||||||
const [cmd, ...args] = process.argv.slice(2);
|
|
||||||
try {
|
|
||||||
switch (cmd) {
|
|
||||||
case 'check-env': await checkEnv(); break;
|
|
||||||
case 'fetch-model': await fetchModel(args[0]); break;
|
|
||||||
case 'calibrate': await calibrate(args[0], args[1]); break;
|
|
||||||
case 'help': case undefined: help(); break;
|
|
||||||
default:
|
|
||||||
console.error(`Unbekannter Befehl "${cmd}".`);
|
|
||||||
help();
|
|
||||||
process.exitCode = 1;
|
|
||||||
}
|
|
||||||
} catch (err) {
|
|
||||||
console.error(`\n${C.bad}Fehler:${C.off} ${err.message}`);
|
|
||||||
if (err.stderr) console.error(String(err.stderr).trim().split('\n').slice(-5).join('\n'));
|
|
||||||
process.exitCode = 1;
|
|
||||||
}
|
|
||||||
@@ -0,0 +1,341 @@
|
|||||||
|
#!/usr/bin/env node
|
||||||
|
// Einstiegspunkt des Container-Images.
|
||||||
|
|
||||||
|
import { execFile, execFileSync } from 'node:child_process';
|
||||||
|
import { createWriteStream } from 'node:fs';
|
||||||
|
import fs from 'node:fs/promises';
|
||||||
|
import net from 'node:net';
|
||||||
|
import path from 'node:path';
|
||||||
|
import { pipeline } from 'node:stream/promises';
|
||||||
|
import { loadLayout, paths, type LayoutProfile, type Rect } from '#core/config.ts';
|
||||||
|
import { openDb } from '#db/index.ts';
|
||||||
|
import { secretStatus } from '#twitch/secrets.ts';
|
||||||
|
|
||||||
|
const WHISPER = process.env.WHISPER_BIN || 'whisper-cli';
|
||||||
|
const OBS_HOST = process.env.OBS_WS_HOST || 'host.docker.internal';
|
||||||
|
const OBS_PORT = Number(process.env.OBS_WS_PORT || 4455);
|
||||||
|
|
||||||
|
const MODELS: Record<string, string> = {
|
||||||
|
'ggml-small.bin': 'https://huggingface.co/ggerganov/whisper.cpp/resolve/main/ggml-small.bin',
|
||||||
|
'ggml-medium.bin': 'https://huggingface.co/ggerganov/whisper.cpp/resolve/main/ggml-medium.bin',
|
||||||
|
'ggml-base.bin': 'https://huggingface.co/ggerganov/whisper.cpp/resolve/main/ggml-base.bin',
|
||||||
|
};
|
||||||
|
|
||||||
|
const C = process.stdout.isTTY
|
||||||
|
? { ok: '\x1b[32m', warn: '\x1b[33m', bad: '\x1b[31m', dim: '\x1b[90m', off: '\x1b[0m' }
|
||||||
|
: { ok: '', warn: '', bad: '', dim: '', off: '' };
|
||||||
|
|
||||||
|
let hardFailures = 0;
|
||||||
|
let warnings = 0;
|
||||||
|
|
||||||
|
function report(state: 'ok' | 'warn' | 'bad', label: string, detail = ''): void {
|
||||||
|
const mark = { ok: `${C.ok} OK ${C.off}`, warn: `${C.warn} WARN ${C.off}`, bad: `${C.bad} FEHLT${C.off}` }[state];
|
||||||
|
if (state === 'bad') hardFailures++;
|
||||||
|
if (state === 'warn') warnings++;
|
||||||
|
console.log(`${mark} ${label.padEnd(34)} ${C.dim}${detail}${C.off}`);
|
||||||
|
}
|
||||||
|
|
||||||
|
function section(title: string): void {
|
||||||
|
console.log(`\n${title}`);
|
||||||
|
console.log('-'.repeat(Math.max(title.length, 40)));
|
||||||
|
}
|
||||||
|
|
||||||
|
function run(cmd: string, args: string[]): string {
|
||||||
|
return execFileSync(cmd, args, { encoding: 'utf8', stdio: ['ignore', 'pipe', 'pipe'] });
|
||||||
|
}
|
||||||
|
|
||||||
|
function runAsync(cmd: string, args: string[]): Promise<{ stdout: string; stderr: string }> {
|
||||||
|
return new Promise((resolve, reject) => {
|
||||||
|
execFile(cmd, args, { maxBuffer: 32 * 1024 * 1024 }, (err, stdout, stderr) =>
|
||||||
|
err ? reject(Object.assign(err, { stdout, stderr })) : resolve({ stdout, stderr }));
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
async function canWrite(dir: string): Promise<boolean> {
|
||||||
|
const probe = path.join(dir, `.probe-${process.pid}`);
|
||||||
|
try {
|
||||||
|
await fs.mkdir(dir, { recursive: true });
|
||||||
|
await fs.writeFile(probe, 'x');
|
||||||
|
await fs.unlink(probe);
|
||||||
|
return true;
|
||||||
|
} catch { return false; }
|
||||||
|
}
|
||||||
|
|
||||||
|
function tcpProbe(host: string, port: number, timeout = 2500): Promise<boolean> {
|
||||||
|
return new Promise((resolve) => {
|
||||||
|
const sock = new net.Socket();
|
||||||
|
const done = (r: boolean) => { sock.destroy(); resolve(r); };
|
||||||
|
sock.setTimeout(timeout);
|
||||||
|
sock.once('connect', () => done(true));
|
||||||
|
sock.once('timeout', () => done(false));
|
||||||
|
sock.once('error', () => done(false));
|
||||||
|
sock.connect(port, host);
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---------------------------------------------------------------- check-env
|
||||||
|
|
||||||
|
async function checkEnv(): Promise<void> {
|
||||||
|
console.log('\nabdulive-toolkit - Selbsttest der Umgebung');
|
||||||
|
|
||||||
|
section('Laufzeit');
|
||||||
|
const major = Number(process.versions.node.split('.')[0]);
|
||||||
|
report(major >= 24 ? 'ok' : 'bad', 'Node.js', `v${process.versions.node}`);
|
||||||
|
try {
|
||||||
|
const { DatabaseSync } = await import('node:sqlite');
|
||||||
|
new DatabaseSync(':memory:').close();
|
||||||
|
report('ok', 'node:sqlite (eingebaut)');
|
||||||
|
} catch (err) {
|
||||||
|
report('bad', 'node:sqlite', (err as Error).message.slice(0, 50));
|
||||||
|
}
|
||||||
|
report(typeof WebSocket !== 'undefined' ? 'ok' : 'bad', 'WebSocket (eingebaut)');
|
||||||
|
|
||||||
|
section('Medienwerkzeuge');
|
||||||
|
try {
|
||||||
|
const v = run('ffmpeg', ['-version']).split('\n')[0] ?? '';
|
||||||
|
report('ok', 'ffmpeg', v.replace('ffmpeg version ', '').slice(0, 40));
|
||||||
|
const cfg = run('ffmpeg', ['-hide_banner', '-buildconf']);
|
||||||
|
report(cfg.includes('--enable-libx264') ? 'ok' : 'bad', ' libx264 (H.264-Encoder)');
|
||||||
|
report(cfg.includes('--enable-libass') ? 'ok' : 'bad', ' libass (Untertitel einbrennen)');
|
||||||
|
} catch { report('bad', 'ffmpeg', 'nicht im PATH'); }
|
||||||
|
try { run('ffprobe', ['-version']); report('ok', 'ffprobe'); }
|
||||||
|
catch { report('bad', 'ffprobe', 'nicht im PATH'); }
|
||||||
|
try {
|
||||||
|
run(WHISPER, ['--help']);
|
||||||
|
report('ok', 'whisper-cli', WHISPER);
|
||||||
|
} catch (err) {
|
||||||
|
const text = `${(err as { stdout?: string }).stdout ?? ''}${(err as { stderr?: string }).stderr ?? ''}`;
|
||||||
|
report(text.includes('whisper') || text.includes('usage') ? 'ok' : 'bad', 'whisper-cli',
|
||||||
|
text.includes('whisper') ? WHISPER : 'nicht lauffaehig');
|
||||||
|
}
|
||||||
|
|
||||||
|
section('Sprachmodell');
|
||||||
|
let anyModel = false;
|
||||||
|
for (const name of Object.keys(MODELS)) {
|
||||||
|
try {
|
||||||
|
const st = await fs.stat(path.join(paths.models, name));
|
||||||
|
report('ok', name, `${(st.size / 1024 / 1024).toFixed(0)} MB`);
|
||||||
|
anyModel = true;
|
||||||
|
} catch { /* nicht vorhanden */ }
|
||||||
|
}
|
||||||
|
if (!anyModel) report('warn', 'kein Modell vorhanden', 'holen mit: fetch-model ggml-small.bin');
|
||||||
|
|
||||||
|
section('Verzeichnisse');
|
||||||
|
for (const [label, dir] of [
|
||||||
|
['Datenbank (Volume)', paths.db],
|
||||||
|
['Arbeitsverzeichnis (Volume)', paths.work],
|
||||||
|
['Sprachmodelle (Volume)', paths.models],
|
||||||
|
['Clips (Bind-Mount)', paths.clips],
|
||||||
|
] as const) {
|
||||||
|
report(await canWrite(dir) ? 'ok' : 'bad', label, `${dir} beschreibbar`);
|
||||||
|
}
|
||||||
|
for (const [label, dir, hard] of [
|
||||||
|
['OBS-Wiedergabepuffer', paths.replay, false],
|
||||||
|
['Konfiguration', paths.config, true],
|
||||||
|
] as const) {
|
||||||
|
try {
|
||||||
|
const n = (await fs.readdir(dir)).length;
|
||||||
|
report('ok', label, `${dir} lesbar, ${n} Eintraege`);
|
||||||
|
} catch { report(hard ? 'bad' : 'warn', label, `${dir} nicht lesbar - Mount gesetzt?`); }
|
||||||
|
}
|
||||||
|
|
||||||
|
section('Datenbank');
|
||||||
|
try {
|
||||||
|
const db = openDb();
|
||||||
|
const n = (db.prepare('SELECT COUNT(*) AS n FROM schema_migrations').get() as { n: number }).n;
|
||||||
|
report('ok', 'Schema', `${n} Migration(en) angewendet`);
|
||||||
|
} catch (err) {
|
||||||
|
report('bad', 'Schema', (err as Error).message.slice(0, 60));
|
||||||
|
}
|
||||||
|
|
||||||
|
section('Layout (Facecam-Ausschnitt)');
|
||||||
|
try {
|
||||||
|
const layout = loadLayout();
|
||||||
|
const prof = layout.profiles?.['default'];
|
||||||
|
if (!prof) throw new Error('Profil "default" fehlt');
|
||||||
|
const sum = prof.cam.band.h + prof.game.band.h;
|
||||||
|
report(sum === layout.target.height ? 'ok' : 'bad', 'Baender ergeben Zielhoehe',
|
||||||
|
`${prof.cam.band.h} + ${prof.game.band.h} = ${sum} (soll ${layout.target.height})`);
|
||||||
|
for (const [name, c] of [['Facecam', prof.cam.crop], ['Gameplay', prof.game.crop]] as const) {
|
||||||
|
const inside = c.x >= 0 && c.y >= 0 &&
|
||||||
|
c.x + c.w <= layout.source.width && c.y + c.h <= layout.source.height;
|
||||||
|
report(inside ? 'ok' : 'bad', `${name}-Ausschnitt im Bild`, `${c.w}x${c.h} bei ${c.x},${c.y}`);
|
||||||
|
}
|
||||||
|
for (const [name, c, b] of [
|
||||||
|
['Facecam', prof.cam.crop, prof.cam.band], ['Gameplay', prof.game.crop, prof.game.band],
|
||||||
|
] as const) {
|
||||||
|
const dev = Math.abs((c.w / c.h) / (b.w / b.h) - 1) * 100;
|
||||||
|
report(dev < 1 ? 'ok' : 'warn', `${name}-Seitenverhaeltnis`,
|
||||||
|
`Abweichung ${dev.toFixed(2)} % (>1 % verzerrt sichtbar)`);
|
||||||
|
}
|
||||||
|
} catch (err) {
|
||||||
|
report('bad', 'config/layout.yaml', (err as Error).message);
|
||||||
|
}
|
||||||
|
|
||||||
|
section('OBS');
|
||||||
|
const obsUp = await tcpProbe(OBS_HOST, OBS_PORT);
|
||||||
|
report(obsUp ? 'ok' : 'warn', 'obs-websocket erreichbar',
|
||||||
|
`${OBS_HOST}:${OBS_PORT}${obsUp ? '' : ' - OBS laeuft nicht oder WebSocket-Server ist aus'}`);
|
||||||
|
|
||||||
|
section('Zugangsdaten');
|
||||||
|
// Meldet nur, OB etwas hinterlegt ist - nie was. Siehe ADR-0005.
|
||||||
|
const st = secretStatus();
|
||||||
|
for (const role of ['broadcaster', 'bot'] as const) {
|
||||||
|
const a = st.apps[role];
|
||||||
|
report(a.clientId && a.hasSecret ? 'ok' : 'warn', `${role}: App-Daten`,
|
||||||
|
a.clientId ? (a.hasSecret ? 'Client-ID und Secret hinterlegt' : 'Secret fehlt') : 'nichts hinterlegt');
|
||||||
|
report(st.tokens[role].connected ? 'ok' : 'warn', `${role}: Twitch verbunden`,
|
||||||
|
st.tokens[role].connected ? `als ${st.tokens[role].login}` : 'Einrichtung unter /setup');
|
||||||
|
}
|
||||||
|
report(st.obs ? 'ok' : 'warn', 'OBS-Passwort', st.obs ? 'hinterlegt' : 'Einrichtung unter /setup');
|
||||||
|
|
||||||
|
console.log('');
|
||||||
|
if (hardFailures > 0) {
|
||||||
|
console.log(`${C.bad}${hardFailures} harte Fehler${C.off}, ${warnings} Warnungen. Noch nicht einsatzbereit.`);
|
||||||
|
process.exitCode = 1;
|
||||||
|
} else {
|
||||||
|
console.log(`${C.ok}Keine harten Fehler${C.off}, ${warnings} Warnungen.` +
|
||||||
|
(warnings ? ' Warnungen ohne Einrichtung sind normal.' : ''));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---------------------------------------------------------------- fetch-model
|
||||||
|
|
||||||
|
async function fetchModel(name = 'ggml-small.bin'): Promise<void> {
|
||||||
|
const url = MODELS[name];
|
||||||
|
if (!url) {
|
||||||
|
console.error(`Unbekanntes Modell "${name}". Bekannt: ${Object.keys(MODELS).join(', ')}`);
|
||||||
|
process.exitCode = 1;
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
await fs.mkdir(paths.models, { recursive: true });
|
||||||
|
const target = path.join(paths.models, name);
|
||||||
|
try {
|
||||||
|
const st = await fs.stat(target);
|
||||||
|
console.log(`${name} liegt schon da (${(st.size / 1024 / 1024).toFixed(0)} MB).`);
|
||||||
|
return;
|
||||||
|
} catch { /* laden */ }
|
||||||
|
|
||||||
|
console.log(`Lade ${name} ...`);
|
||||||
|
const res = await fetch(url, { redirect: 'follow' });
|
||||||
|
if (!res.ok || !res.body) throw new Error(`Download fehlgeschlagen: HTTP ${res.status}`);
|
||||||
|
const total = Number(res.headers.get('content-length') || 0);
|
||||||
|
let seen = 0;
|
||||||
|
let lastPrint = 0;
|
||||||
|
const tmp = `${target}.part`;
|
||||||
|
|
||||||
|
await pipeline(
|
||||||
|
(async function* () {
|
||||||
|
for await (const chunk of res.body as unknown as AsyncIterable<Uint8Array>) {
|
||||||
|
seen += chunk.length;
|
||||||
|
const now = Date.now();
|
||||||
|
if (now - lastPrint > 1000) {
|
||||||
|
lastPrint = now;
|
||||||
|
const pct = total ? ` (${((seen / total) * 100).toFixed(0)} %)` : '';
|
||||||
|
process.stdout.write(`\r ${(seen / 1024 / 1024).toFixed(0)} MB${pct} `);
|
||||||
|
}
|
||||||
|
yield chunk;
|
||||||
|
}
|
||||||
|
})(),
|
||||||
|
createWriteStream(tmp)
|
||||||
|
);
|
||||||
|
await fs.rename(tmp, target);
|
||||||
|
console.log(`\rFertig: ${target} (${(seen / 1024 / 1024).toFixed(0)} MB)`);
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---------------------------------------------------------------- calibrate
|
||||||
|
|
||||||
|
function buildFilter(prof: LayoutProfile): string {
|
||||||
|
const c = (r: Rect) => `crop=${r.w}:${r.h}:${r.x}:${r.y}`;
|
||||||
|
return [
|
||||||
|
`[0:v]${c(prof.cam.crop)},scale=${prof.cam.band.w}:${prof.cam.band.h},setsar=1[cam]`,
|
||||||
|
`[0:v]${c(prof.game.crop)},scale=${prof.game.band.w}:${prof.game.band.h},setsar=1[game]`,
|
||||||
|
'[cam][game]vstack=inputs=2[v]',
|
||||||
|
].join(';');
|
||||||
|
}
|
||||||
|
|
||||||
|
async function calibrate(input?: string, profileName = 'default'): Promise<void> {
|
||||||
|
const layout = loadLayout();
|
||||||
|
const prof = layout.profiles?.[profileName];
|
||||||
|
if (!prof) throw new Error(`Profil "${profileName}" steht nicht in config/layout.yaml`);
|
||||||
|
|
||||||
|
if (!input) {
|
||||||
|
const files = (await fs.readdir(paths.replay).catch(() => [] as string[]))
|
||||||
|
.filter((f) => /\.(mp4|mkv|mov|flv|png|jpe?g)$/i.test(f)).sort();
|
||||||
|
const last = files[files.length - 1];
|
||||||
|
if (!last) throw new Error(`Keine Datei angegeben und nichts in ${paths.replay} gefunden.`);
|
||||||
|
input = path.join(paths.replay, last);
|
||||||
|
console.log(`Neueste Datei aus ${paths.replay}: ${last}`);
|
||||||
|
}
|
||||||
|
|
||||||
|
const outDir = path.join(paths.clips, '_calibration');
|
||||||
|
await fs.mkdir(outDir, { recursive: true });
|
||||||
|
const seek = /\.(png|jpe?g)$/i.test(input) ? [] : ['-ss', '5'];
|
||||||
|
const quiet = ['-y', '-hide_banner', '-loglevel', 'error'];
|
||||||
|
|
||||||
|
const frame = path.join(outDir, 'frame.png');
|
||||||
|
await runAsync('ffmpeg', [...quiet, ...seek, '-i', input, '-frames:v', '1', frame]);
|
||||||
|
|
||||||
|
const preview = path.join(outDir, `preview-${profileName}.png`);
|
||||||
|
await runAsync('ffmpeg', [...quiet, '-i', frame, '-filter_complex', buildFilter(prof),
|
||||||
|
'-map', '[v]', '-frames:v', '1', preview]);
|
||||||
|
|
||||||
|
const marked = path.join(outDir, 'ausschnitte.png');
|
||||||
|
const box = (r: Rect, color: string) => `drawbox=x=${r.x}:y=${r.y}:w=${r.w}:h=${r.h}:color=${color}:t=4`;
|
||||||
|
await runAsync('ffmpeg', [...quiet, '-i', frame, '-vf',
|
||||||
|
`${box(prof.cam.crop, 'yellow')},${box(prof.game.crop, 'cyan')}`, '-frames:v', '1', marked]);
|
||||||
|
|
||||||
|
console.log('\nGeschrieben:');
|
||||||
|
console.log(` ${frame} Einzelbild aus der Quelle`);
|
||||||
|
console.log(` ${marked} Ausschnitte markiert (gelb = Cam, cyan = Gameplay)`);
|
||||||
|
console.log(` ${preview} fertige 9:16-Vorschau`);
|
||||||
|
console.log('\nSitzt der Ausschnitt nicht, die Werte in config/layout.yaml anpassen');
|
||||||
|
console.log('und den Befehl erneut ausfuehren.');
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---------------------------------------------------------------- help
|
||||||
|
|
||||||
|
function help(): void {
|
||||||
|
console.log(`
|
||||||
|
abdulive-toolkit - Container-Einstiegspunkt
|
||||||
|
|
||||||
|
ingest Daemon starten (Chat, Zuschauerzahlen, Einrichtungsseite)
|
||||||
|
check-env Umgebung pruefen
|
||||||
|
db:migrate Migrationen anwenden
|
||||||
|
fetch-model [name] Whisper-Modell laden (Vorgabe: ggml-small.bin)
|
||||||
|
calibrate [datei] [profil] Facecam-Ausschnitt pruefen, Vorschau rendern
|
||||||
|
help Diese Uebersicht
|
||||||
|
|
||||||
|
Einrichtung nach dem Start: http://localhost:4792/setup
|
||||||
|
|
||||||
|
Stand: Scheiben S0 und S1 aus docs/07-mvp-plan.md. OBS-Anbindung, Clipping
|
||||||
|
und Untertitel folgen in S2 bis S4.
|
||||||
|
`);
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---------------------------------------------------------------- main
|
||||||
|
|
||||||
|
const [cmd, ...args] = process.argv.slice(2);
|
||||||
|
try {
|
||||||
|
switch (cmd) {
|
||||||
|
case 'ingest': {
|
||||||
|
const { runIngest } = await import('../apps/ingest/src/index.ts');
|
||||||
|
await runIngest();
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
case 'check-env': await checkEnv(); break;
|
||||||
|
case 'db:migrate': openDb(); console.log('Migrationen sind aktuell.'); break;
|
||||||
|
case 'fetch-model': await fetchModel(args[0]); break;
|
||||||
|
case 'calibrate': await calibrate(args[0], args[1]); break;
|
||||||
|
case 'help': case undefined: help(); break;
|
||||||
|
default:
|
||||||
|
console.error(`Unbekannter Befehl "${cmd}".`);
|
||||||
|
help();
|
||||||
|
process.exitCode = 1;
|
||||||
|
}
|
||||||
|
} catch (err) {
|
||||||
|
console.error(`\n${C.bad}Fehler:${C.off} ${(err as Error).message}`);
|
||||||
|
const stderr = (err as { stderr?: string }).stderr;
|
||||||
|
if (stderr) console.error(String(stderr).trim().split('\n').slice(-5).join('\n'));
|
||||||
|
process.exitCode = 1;
|
||||||
|
}
|
||||||
+13
-6
@@ -93,24 +93,31 @@ Vollständige Event-Liste: siehe [`03-data-model.md`](03-data-model.md).
|
|||||||
|
|
||||||
| Paket | Inhalt |
|
| Paket | Inhalt |
|
||||||
|-------|--------|
|
|-------|--------|
|
||||||
| `packages/shared` | Typen, Event-Definitionen, Zod-Schemas |
|
| `packages/shared` | Ereignistypen, ULID |
|
||||||
| `packages/core` | Config-Laden, Logger, Event-Bus, Scheduler, Retry |
|
| `packages/core` | Config, Layout, Logger mit Maskierung, Retry, YAML-Leser |
|
||||||
| `packages/twitch` | OAuth-Token-Store, Helix-Client, EventSub-WS, IRC-Client |
|
| `packages/twitch` | Secret-Speicher, OAuth, Helix-Client, IRC-Client |
|
||||||
| `packages/db` | SQLite-Verbindung, Migrationen, Repositories |
|
| `packages/db` | SQLite-Verbindung, Migrationen, Repositories |
|
||||||
|
|
||||||
Regel: Apps dürfen von Paketen abhängen, Pakete nie von Apps. Apps hängen nicht
|
Regel: Apps dürfen von Paketen abhängen, Pakete nie von Apps. Apps hängen nicht
|
||||||
voneinander ab — Kommunikation läuft über Bus und DB.
|
voneinander ab — Kommunikation läuft über Bus und DB.
|
||||||
|
|
||||||
|
Importiert wird über die `imports`-Map der Wurzel-`package.json`
|
||||||
|
(`#core/config.ts`, `#db/repos.ts`). Das ist eine Node-Funktion, kein
|
||||||
|
Build-Werkzeug — es gibt keinen Bundler und keine Pfad-Aliasse im Compiler.
|
||||||
|
|
||||||
## Technologie
|
## Technologie
|
||||||
|
|
||||||
- **TypeScript / Node 22**, pnpm-Workspaces — siehe
|
- **TypeScript auf Node 24**, ohne Build-Schritt und ohne
|
||||||
|
Laufzeitabhängigkeiten — Node führt die `.ts`-Dateien direkt aus. Siehe
|
||||||
[ADR-0001](decisions/ADR-0001-monorepo-typescript.md)
|
[ADR-0001](decisions/ADR-0001-monorepo-typescript.md)
|
||||||
- **SQLite** über `better-sqlite3` — siehe [ADR-0002](decisions/ADR-0002-sqlite.md)
|
- **SQLite** über das eingebaute `node:sqlite` — siehe
|
||||||
|
[ADR-0002](decisions/ADR-0002-sqlite.md)
|
||||||
- **EventSub über WebSocket**, nicht Webhooks — siehe
|
- **EventSub über WebSocket**, nicht Webhooks — siehe
|
||||||
[ADR-0003](decisions/ADR-0003-eventsub-websocket.md)
|
[ADR-0003](decisions/ADR-0003-eventsub-websocket.md)
|
||||||
- **ffmpeg** (CLI) für Videoverarbeitung
|
- **ffmpeg** (CLI) für Videoverarbeitung
|
||||||
- **whisper.cpp** (CLI) für Untertitel — kein Python, keine GPU
|
- **whisper.cpp** (CLI) für Untertitel — kein Python, keine GPU
|
||||||
- **Vite + Preact** für Dashboard und Overlay-Seiten
|
- **Serverseitiges HTML** ohne Framework für Einrichtung und Sichtungsseite;
|
||||||
|
für Dashboard und Overlays ab M3 ggf. Vite + Preact
|
||||||
- **Docker Compose** für den Betrieb — siehe [ADR-0004](decisions/ADR-0004-docker.md)
|
- **Docker Compose** für den Betrieb — siehe [ADR-0004](decisions/ADR-0004-docker.md)
|
||||||
|
|
||||||
## Fehlerverhalten
|
## Fehlerverhalten
|
||||||
|
|||||||
@@ -11,9 +11,11 @@ Es werden zwei getrennte Twitch-Apps/Tokens gebraucht:
|
|||||||
Trennung, damit Bot-Nachrichten als Bot erkennbar sind und der Broadcaster-
|
Trennung, damit Bot-Nachrichten als Bot erkennbar sind und der Broadcaster-
|
||||||
Token nicht im Chat-Client hängt.
|
Token nicht im Chat-Client hängt.
|
||||||
|
|
||||||
Beide Tokens liegen verschlüsselt in `data/db/tokens.json`, werden über Refresh
|
Client-IDs, Client-Secrets und beide Tokens liegen verschlüsselt in
|
||||||
Tokens automatisch erneuert (`packages/twitch/tokenStore.ts`). Client-Secret
|
`/db/secrets.json` im Docker-Volume und werden über Refresh Tokens automatisch
|
||||||
kommt aus `.env`, niemals aus der DB, niemals ins Repo.
|
erneuert (`packages/twitch/secrets.ts`). Eingegeben werden sie im
|
||||||
|
Einrichtungs-Assistenten unter `http://localhost:4792/setup`, **nicht** in einer
|
||||||
|
`.env` und nie im Repo — siehe [ADR-0005](decisions/ADR-0005-zugangsdaten.md).
|
||||||
|
|
||||||
## Scopes
|
## Scopes
|
||||||
|
|
||||||
@@ -34,8 +36,11 @@ kommt aus `.env`, niemals aus der DB, niemals ins Repo.
|
|||||||
| `analytics:read:games` | Game-Analytics-Report | Broadcaster |
|
| `analytics:read:games` | Game-Analytics-Report | Broadcaster |
|
||||||
| `bits:read` | Bits-Events | Broadcaster |
|
| `bits:read` | Bits-Events | Broadcaster |
|
||||||
|
|
||||||
Der Autorisierungs-Flow läuft einmalig über `docker compose run --rm --service-ports toolkit auth` → lokaler Redirect auf
|
Der Autorisierungs-Flow läuft über den Einrichtungs-Assistenten:
|
||||||
`http://localhost:4780/callback`.
|
`docker compose up -d`, dann `http://localhost:4792/setup` öffnen und je einmal
|
||||||
|
für Broadcaster- und Bot-Konto auf „Mit Twitch verbinden" klicken. Die
|
||||||
|
Redirect-URI, die in beiden Twitch-Apps hinterlegt sein muss, lautet
|
||||||
|
`http://localhost:4792/callback`.
|
||||||
|
|
||||||
## EventSub (WebSocket)
|
## EventSub (WebSocket)
|
||||||
|
|
||||||
|
|||||||
+29
-11
@@ -34,16 +34,18 @@ Danach `wsl --shutdown` und Docker Desktop neu starten.
|
|||||||
|
|
||||||
Zwei Anwendungen unter https://dev.twitch.tv/console/apps:
|
Zwei Anwendungen unter https://dev.twitch.tv/console/apps:
|
||||||
|
|
||||||
1. `abdulive-toolkit` — OAuth Redirect `http://localhost:4780/callback`
|
1. `abdulive-toolkit` — OAuth Redirect `http://localhost:4792/callback`
|
||||||
2. `abdulive-bot` — gleicher Redirect
|
2. `abdulive-bot` — gleicher Redirect
|
||||||
|
|
||||||
Client-ID und Client-Secret beider Apps in `.env` eintragen
|
Client-ID und Client-Secret **nicht** in eine Datei eintragen, sondern später
|
||||||
(Vorlage: `.env.example`).
|
im Einrichtungs-Assistenten einfügen (siehe unten). Begründung:
|
||||||
|
[ADR-0005](decisions/ADR-0005-zugangsdaten.md).
|
||||||
|
|
||||||
## OBS vorbereiten
|
## OBS vorbereiten
|
||||||
|
|
||||||
1. **obs-websocket aktivieren:** Werkzeuge → WebSocket-Servereinstellungen →
|
1. **obs-websocket aktivieren:** Werkzeuge → WebSocket-Servereinstellungen →
|
||||||
Server aktivieren, Port 4455, Passwort setzen und in `.env` eintragen.
|
Server aktivieren, Port 4455, Passwort setzen — eingetragen wird es später
|
||||||
|
im Einrichtungs-Assistenten.
|
||||||
2. **Wiedergabepuffer aktivieren:** Einstellungen → Ausgabe → Wiedergabepuffer,
|
2. **Wiedergabepuffer aktivieren:** Einstellungen → Ausgabe → Wiedergabepuffer,
|
||||||
90 Sekunden. Den Zielordner merken und in `.env` als `OBS_REPLAY_DIR`
|
90 Sekunden. Den Zielordner merken und in `.env` als `OBS_REPLAY_DIR`
|
||||||
eintragen — mit Forward-Slashes, z. B. `C:/Users/abdu/Videos/replay`.
|
eintragen — mit Forward-Slashes, z. B. `C:/Users/abdu/Videos/replay`.
|
||||||
@@ -51,23 +53,39 @@ Client-ID und Client-Secret beider Apps in `.env` eintragen
|
|||||||
Das ist der Clip-Knopf. Ein globaler Windows-Hotkey wird bewusst *nicht*
|
Das ist der Clip-Knopf. Ein globaler Windows-Hotkey wird bewusst *nicht*
|
||||||
gebaut — aus einem Container ist er nicht abfangbar, und OBS meldet den
|
gebaut — aus einem Container ist er nicht abfangbar, und OBS meldet den
|
||||||
gespeicherten Puffer ohnehin über WebSocket.
|
gespeicherten Puffer ohnehin über WebSocket.
|
||||||
4. **Browser-Sources** (erst ab Meilenstein M3 relevant): „Browser-Quelle beim
|
4. **Getrennte Audiospuren** (bereits eingerichtet): Mikrofon auf Spur 2,
|
||||||
|
Spielton auf Spur 3, Mischung auf Spur 1. Die Untertitel werden aus Spur 2
|
||||||
|
erzeugt - das ist bei lauten Spielen der Unterschied zwischen brauchbarem
|
||||||
|
und unbrauchbarem Transkript. Einstellungen -> Ausgabe -> Aufnahme ->
|
||||||
|
Audiospuren, und im Audiomixer je Quelle die Spurzuordnung pruefen.
|
||||||
|
5. **Browser-Sources** (erst ab Meilenstein M3 relevant): „Browser-Quelle beim
|
||||||
Szenenwechsel neu laden" **aus**, sonst brechen laufende Effekte ab.
|
Szenenwechsel neu laden" **aus**, sonst brechen laufende Effekte ab.
|
||||||
|
|
||||||
## Erste Einrichtung
|
## Erste Einrichtung
|
||||||
|
|
||||||
```powershell
|
```powershell
|
||||||
Copy-Item .env.example .env # Secrets und OBS_REPLAY_DIR eintragen
|
|
||||||
Copy-Item config\config.example.yaml config\config.yaml
|
Copy-Item config\config.example.yaml config\config.yaml
|
||||||
|
|
||||||
docker compose build
|
docker compose build
|
||||||
docker compose run --rm toolkit check-env # prueft ffmpeg, whisper, Modell, Volumes
|
docker compose run --rm toolkit check-env # ffmpeg, whisper, Modell, Volumes, Layout
|
||||||
docker compose run --rm --service-ports toolkit auth # Browser oeffnet sich, zweimal bestaetigen
|
docker compose run --rm toolkit fetch-model # Sprachmodell, ca. 465 MB
|
||||||
docker compose run --rm toolkit db:migrate
|
docker compose up -d
|
||||||
```
|
```
|
||||||
|
|
||||||
Der `auth`-Lauf braucht `--service-ports`, damit Port 4780 für den
|
Dann **http://localhost:4792/setup** öffnen. Dort werden nacheinander
|
||||||
OAuth-Rücksprung erreichbar ist.
|
eingetragen:
|
||||||
|
|
||||||
|
1. Client-ID und Client-Secret der beiden Twitch-Apps
|
||||||
|
2. „Mit Twitch verbinden" für Broadcaster- und Bot-Konto (OAuth)
|
||||||
|
3. obs-websocket-Passwort
|
||||||
|
4. später: Discord-Webhook, Bluesky-Zugang
|
||||||
|
|
||||||
|
Alles landet verschlüsselt im Docker-Volume, nicht in einer Datei im Projekt.
|
||||||
|
|
||||||
|
> **Diese Seite zeigt Zugangsdaten. Nicht öffnen, während gesendet wird.**
|
||||||
|
|
||||||
|
Eine `.env` ist optional und enthält nur noch den Pfad zum Wiedergabepuffer
|
||||||
|
und das Log-Level (Vorlage: `.env.example`).
|
||||||
|
|
||||||
## Facecam einmessen
|
## Facecam einmessen
|
||||||
|
|
||||||
|
|||||||
+57
-31
@@ -36,14 +36,34 @@ gibt. Vorher ist sie nur Ballast.
|
|||||||
|---|---|
|
|---|---|
|
||||||
| S0 Container + Toolchain | **fertig** — Image 258 MB, `check-env` ohne harte Fehler |
|
| S0 Container + Toolchain | **fertig** — Image 258 MB, `check-env` ohne harte Fehler |
|
||||||
| Layout eingemessen | **fertig** — `config/layout.yaml`, vorgezogen aus S3 |
|
| Layout eingemessen | **fertig** — `config/layout.yaml`, vorgezogen aus S3 |
|
||||||
| S1 Auth + DB + IRC-Ingest | als Nächstes |
|
| S1 Einrichtung + Auth + DB + IRC-Ingest | **fertig** — 20 Tests grün, Daemon läuft |
|
||||||
| S2–S6 | offen |
|
| S2–S6 | offen |
|
||||||
|
|
||||||
|
**S1 abgenommen am 2026-09-09.** Der Daemon läuft, hängt am Chat, die
|
||||||
|
Einrichtung ist unter `http://localhost:4792/setup` erreichbar.
|
||||||
|
|
||||||
|
Beim Bauen hat sich die Technikbasis vereinfacht: Node 24 bringt SQLite,
|
||||||
|
WebSocket, `fetch` und die Ausführung von TypeScript mit, deshalb hat das
|
||||||
|
Projekt **null Laufzeitabhängigkeiten und keinen Build-Schritt**. pnpm-Workspaces
|
||||||
|
und die Paketmanifeste sind entfallen. Siehe Nachtrag 2 in
|
||||||
|
[ADR-0001](decisions/ADR-0001-monorepo-typescript.md).
|
||||||
|
|
||||||
|
Nachgewiesen:
|
||||||
|
|
||||||
|
- 20 Tests grün (IRC-Parser, Spike-Detektor, Datenbankschicht)
|
||||||
|
- Chat verbindet sich **anonym**, solange keine Zugangsdaten hinterlegt sind —
|
||||||
|
damit lässt sich die Datenaufnahme prüfen, bevor die Twitch-Apps existieren
|
||||||
|
- Zugangsdaten überstehen die Runde durch die Einrichtungsseite, liegen
|
||||||
|
verschlüsselt mit Rechten `0600` im Volume, tauchen weder im HTML noch in der
|
||||||
|
Datenbank noch in einer Volltextsuche im Klartext auf
|
||||||
|
- Was noch aussteht: die Strecke IRC → Datenbank im Livebetrieb. Die Teile sind
|
||||||
|
einzeln getestet, die Verdrahtung wird beim ersten echten Stream sichtbar.
|
||||||
|
|
||||||
**S0 abgenommen am 2026-09-09.** Im Image: Node 22.23, ffmpeg 5.1.9 mit
|
**S0 abgenommen am 2026-09-09.** Im Image: Node 22.23, ffmpeg 5.1.9 mit
|
||||||
libx264 und libass, whisper.cpp v1.7.4. `check-env` prüft Werkzeuge,
|
libx264 und libass, whisper.cpp v1.7.4, Sprachmodell `ggml-small` geladen.
|
||||||
Volumes, Layout-Geometrie, OBS-Erreichbarkeit und Zugangsdaten; die fünf
|
`check-env` prüft Werkzeuge, Volumes, Layout-Geometrie, OBS-Erreichbarkeit und
|
||||||
verbleibenden Warnungen (Sprachmodell, OBS, drei `.env`-Schlüssel) sind auf
|
Zugangsdaten; die verbleibenden Warnungen (OBS nicht erreichbar, noch keine
|
||||||
dieser Entwicklungsmaschine erwartbar.
|
Zugangsdaten hinterlegt) sind auf dieser Entwicklungsmaschine erwartbar.
|
||||||
|
|
||||||
`calibrate` wurde mit einem echten 1920×1080-Bild durchgespielt: 1080×1920
|
`calibrate` wurde mit einem echten 1920×1080-Bild durchgespielt: 1080×1920
|
||||||
heraus, Ausschnitte an der richtigen Stelle. Damit ist der Filtergraph für S3
|
heraus, Ausschnitte an der richtigen Stelle. Damit ist der Filtergraph für S3
|
||||||
@@ -56,13 +76,6 @@ und **die Facecam ist 4:3, nicht 16:9**. Beides ist in
|
|||||||
[`projects/clip-shorts.md`](projects/clip-shorts.md) dokumentiert und in
|
[`projects/clip-shorts.md`](projects/clip-shorts.md) dokumentiert und in
|
||||||
`config/layout.yaml` aufgelöst.
|
`config/layout.yaml` aufgelöst.
|
||||||
|
|
||||||
Das Einmessen der Facecam war eigentlich Teil von S3, ließ sich aber aus dem
|
|
||||||
gelieferten Screenshot sofort erledigen. Dabei kamen zwei Dinge heraus, die das
|
|
||||||
geplante Layout verändert haben — **der Chat ist ins Streambild eingebrannt**
|
|
||||||
und **die Facecam ist 4:3, nicht 16:9**. Beides ist in
|
|
||||||
[`projects/clip-shorts.md`](projects/clip-shorts.md) dokumentiert und in
|
|
||||||
`config/layout.yaml` aufgelöst.
|
|
||||||
|
|
||||||
## Umfang
|
## Umfang
|
||||||
|
|
||||||
### Drin
|
### Drin
|
||||||
@@ -101,13 +114,19 @@ in ein eigenes Volume, `scripts/check-env` als Selbsttest im Container.
|
|||||||
**Fertig, wenn:** `docker compose run --rm toolkit check-env` meldet ffmpeg,
|
**Fertig, wenn:** `docker compose run --rm toolkit check-env` meldet ffmpeg,
|
||||||
whisper.cpp, Modell und die Schreibrechte auf allen Volumes als OK.
|
whisper.cpp, Modell und die Schreibrechte auf allen Volumes als OK.
|
||||||
|
|
||||||
### S1 — Auth und Datenaufnahme · ~2 Tage
|
### S1 — Einrichtung, Auth und Datenaufnahme · ~3 Tage
|
||||||
|
|
||||||
OAuth-Flow (`auth`-Kommando, Callback auf Port 4780), verschlüsselter
|
**Einrichtungs-Assistent** auf `http://localhost:4792/setup`: Client-ID und
|
||||||
Token-Store mit automatischem Refresh, Helix-Client mit Rate-Limit-Bucket,
|
-Secret werden dort eingegeben, nicht in eine Datei. Verschlüsselter
|
||||||
DB-Migrationen für `streams`, `users`, `user_streams`, `chat_messages`,
|
Secret-Speicher im Volume, OAuth-Flow mit Rücksprung auf denselben Port,
|
||||||
`viewer_samples`, `clips`. IRC-Client, der jede Nachricht schreibt.
|
Token-Refresh. Siehe [ADR-0005](decisions/ADR-0005-zugangsdaten.md).
|
||||||
Stream-Sessions über Helix-Polling im Minutentakt.
|
|
||||||
|
Dazu Helix-Client mit Rate-Limit-Bucket, DB-Migrationen für `streams`, `users`,
|
||||||
|
`user_streams`, `chat_messages`, `viewer_samples`, `clips`, ein IRC-Client, der
|
||||||
|
jede Nachricht schreibt, und Stream-Sessions über Helix-Polling im Minutentakt.
|
||||||
|
|
||||||
|
> Der Assistent war ursprünglich nicht eingeplant und kostet etwa einen Tag
|
||||||
|
> mehr. Dafür entfällt der `.env`-Schritt aus der Inbetriebnahme komplett.
|
||||||
|
|
||||||
**Fertig, wenn:** Nach einem echten Stream stehen alle Nachrichten in
|
**Fertig, wenn:** Nach einem echten Stream stehen alle Nachrichten in
|
||||||
`chat_messages` und eine korrekte Zeile in `streams`.
|
`chat_messages` und eine korrekte Zeile in `streams`.
|
||||||
@@ -140,10 +159,12 @@ korrekt sitzender Facecam oben und Gameplay unten.
|
|||||||
|
|
||||||
### S4 — Untertitel · ~1,5 Tage
|
### S4 — Untertitel · ~1,5 Tage
|
||||||
|
|
||||||
Audiospur extrahieren (16 kHz Mono WAV), whisper.cpp mit `ggml-small` und
|
**Nur die Mikrofonspur** extrahieren (OBS nimmt getrennt auf: Mikro Spur 2,
|
||||||
`--language de`, Ausgabe als SRT mit Wort-Zeitstempeln, Umwandlung in ASS mit
|
Spielton Spur 3, Mischung Spur 1) als 16 kHz Mono WAV, whisper.cpp mit
|
||||||
wortweisem Highlight, Stil aus `config/subtitle-style.ass`, Einbrennen.
|
`ggml-small` und `--language de`, Ausgabe als SRT mit Wort-Zeitstempeln,
|
||||||
Sicherheitsabstand nach unten mindestens 320 px.
|
Umwandlung in ASS mit wortweisem Highlight, Stil aus
|
||||||
|
`config/subtitle-style.ass`, Einbrennen. Sicherheitsabstand nach unten
|
||||||
|
mindestens 320 px. Ins Video kommt weiterhin die Mischung von Spur 1.
|
||||||
|
|
||||||
**Fertig, wenn:** `final.mp4` hat lesbare, zeitlich passende deutsche
|
**Fertig, wenn:** `final.mp4` hat lesbare, zeitlich passende deutsche
|
||||||
Untertitel, und der Text wird auf dem Handy von der TikTok-Oberfläche nicht
|
Untertitel, und der Text wird auf dem Handy von der TikTok-Oberfläche nicht
|
||||||
@@ -174,12 +195,13 @@ merken — deshalb ist sie trotzdem drin.
|
|||||||
|
|
||||||
| Woche | Scheiben | Ergebnis |
|
| Woche | Scheiben | Ergebnis |
|
||||||
|---|---|---|
|
|---|---|---|
|
||||||
| 1 | S0, S1 | Läuft im Container, schreibt jeden Stream mit |
|
| 1 | ~~S0~~, S1 | Einrichtung im Browser, schreibt jeden Stream mit |
|
||||||
| 2 | S2, S3 | Knopfdruck erzeugt ein fertiges 9:16-Video |
|
| 2 | S2, S3 | Knopfdruck erzeugt ein fertiges 9:16-Video |
|
||||||
| 3 | S4, S5, S6 | Untertitel, Sichtungsseite, Begrüßung |
|
| 3 | S4, S5, S6 | Untertitel, Sichtungsseite, Begrüßung |
|
||||||
|
|
||||||
Rund neun Arbeitstage. Realistisch drei Kalenderwochen, wenn nebenher noch
|
Rund zehn Arbeitstage (S0 erledigt, S1 durch den Einrichtungs-Assistenten einen
|
||||||
etwas anderes läuft.
|
Tag teurer). Realistisch drei Kalenderwochen, wenn nebenher noch etwas
|
||||||
|
anderes läuft.
|
||||||
|
|
||||||
## Definition of Done für das gesamte MVP
|
## Definition of Done für das gesamte MVP
|
||||||
|
|
||||||
@@ -200,7 +222,7 @@ Ein echter Stream, ohne Eingriff von außen:
|
|||||||
| Risiko | Wahrscheinlichkeit | Gegenmaßnahme |
|
| Risiko | Wahrscheinlichkeit | Gegenmaßnahme |
|
||||||
|---|---|---|
|
|---|---|---|
|
||||||
| Encoding kostet Frames im Stream | mittel | `-preset veryfast`, CPU-Limit im Compose-File, Schalter `renderWhileLive: false` — dann läuft alles nach Streamende |
|
| Encoding kostet Frames im Stream | mittel | `-preset veryfast`, CPU-Limit im Compose-File, Schalter `renderWhileLive: false` — dann läuft alles nach Streamende |
|
||||||
| whisper.cpp versteht Spielgeräusche statt Sprache | mittel | Falls OBS getrennte Audiospuren aufnimmt, nur die Mikrofonspur transkribieren. Sonst Rauschgatter vorschalten. |
|
| whisper.cpp versteht Spielgeräusche statt Sprache | ~~mittel~~ **gelöst** | OBS nimmt getrennte Spuren auf; transkribiert wird nur das Mikrofon (Spur 2). |
|
||||||
| Twitch ändert die Ableitung der Clip-MP4-URL | niedrig | Betrifft nur den Zweitweg; die Rohdatei kommt aus OBS. Gekapselt in einer Datei. |
|
| Twitch ändert die Ableitung der Clip-MP4-URL | niedrig | Betrifft nur den Zweitweg; die Rohdatei kommt aus OBS. Gekapselt in einer Datei. |
|
||||||
| Replay-Buffer-Pfad-Abbildung Windows ↔ Container | mittel | In S2 als Erstes lösen, mit einem Testlauf vor allem anderen |
|
| Replay-Buffer-Pfad-Abbildung Windows ↔ Container | mittel | In S2 als Erstes lösen, mit einem Testlauf vor allem anderen |
|
||||||
| Docker Desktop belastet den Stream-PC | mittel | `.wslconfig` mit Speicher- und CPU-Grenze; wenn es stört, ist ein Betrieb ohne Container jederzeit möglich, weil nur ffmpeg und whisper.cpp fehlen |
|
| Docker Desktop belastet den Stream-PC | mittel | `.wslconfig` mit Speicher- und CPU-Grenze; wenn es stört, ist ein Betrieb ohne Container jederzeit möglich, weil nur ffmpeg und whisper.cpp fehlen |
|
||||||
@@ -208,13 +230,17 @@ Ein echter Stream, ohne Eingriff von außen:
|
|||||||
## Was Abdu vorher tun muss
|
## Was Abdu vorher tun muss
|
||||||
|
|
||||||
1. Docker Desktop installieren, WSL2-Backend aktivieren
|
1. Docker Desktop installieren, WSL2-Backend aktivieren
|
||||||
2. Zwei Twitch-Apps anlegen (siehe [`05-setup.md`](05-setup.md))
|
2. Zwei Twitch-Apps anlegen, Redirect-URI `http://localhost:4792/callback`
|
||||||
|
(siehe [`05-setup.md`](05-setup.md)). Client-ID und Secret **nicht**
|
||||||
|
irgendwo eintragen — die kommen in den Einrichtungs-Assistenten.
|
||||||
3. In OBS: obs-websocket aktivieren (Port 4455, Passwort), Wiedergabepuffer auf
|
3. In OBS: obs-websocket aktivieren (Port 4455, Passwort), Wiedergabepuffer auf
|
||||||
90 Sekunden, Hotkey auf „Wiedergabepuffer speichern" legen
|
90 Sekunden, Hotkey auf „Wiedergabepuffer speichern" legen
|
||||||
4. **Einen Screenshot der laufenden Szene** schicken — daraus wird der
|
4. ~~Screenshot der laufenden Szene schicken~~ — **erledigt**, Crop ist
|
||||||
Facecam-Crop eingemessen
|
eingemessen und steht in `config/layout.yaml`
|
||||||
5. Klären, ob OBS Mikrofon und Spiel auf getrennte Audiospuren aufnimmt (macht
|
5. ~~Getrennte Audiospuren klären~~ — **erledigt**, OBS nimmt getrennt auf;
|
||||||
die Untertitel deutlich besser)
|
Whisper bekommt nur die Mikrofonspur
|
||||||
|
6. In GTA die spieleigenen Untertitel ausschalten (werden vom Ausschnitt rechts
|
||||||
|
abgeschnitten, und die eigenen liegen ohnehin darüber)
|
||||||
|
|
||||||
## Danach
|
## Danach
|
||||||
|
|
||||||
|
|||||||
@@ -13,12 +13,40 @@ riecht; Teile sind Browser-Overlays, was TypeScript nahelegt.
|
|||||||
Alles in TypeScript auf Node 22, als pnpm-Workspace-Monorepo. ffmpeg und
|
Alles in TypeScript auf Node 22, als pnpm-Workspace-Monorepo. ffmpeg und
|
||||||
whisper.cpp werden als **CLI-Prozesse** aufgerufen, nicht über Sprachbindings.
|
whisper.cpp werden als **CLI-Prozesse** aufgerufen, nicht über Sprachbindings.
|
||||||
|
|
||||||
> **Nachtrag 2026-09-09:** Ursprünglich war faster-whisper (Python) vorgesehen.
|
> **Nachtrag 1 (2026-09-09):** Ursprünglich war faster-whisper (Python)
|
||||||
> Der Check der Zielmaschine ergab Python 3.14, für das `ctranslate2` noch keine
|
> vorgesehen. Der Check der Zielmaschine ergab Python 3.14, für das
|
||||||
> verlässlichen Wheels liefert. Da ohnehin nur ein Kommandozeilenaufruf
|
> `ctranslate2` noch keine verlässlichen Wheels liefert. Da ohnehin nur ein
|
||||||
> gebraucht wird, fällt die Wahl auf **whisper.cpp** — eine einzelne Binary
|
> Kommandozeilenaufruf gebraucht wird, fällt die Wahl auf **whisper.cpp** —
|
||||||
> ohne Laufzeitabhängigkeiten. Damit entfällt Python vollständig aus dem
|
> eine einzelne Binary ohne Laufzeitabhängigkeiten. Damit entfällt Python
|
||||||
> Projekt. Siehe [ADR-0004](ADR-0004-docker.md).
|
> vollständig. Siehe [ADR-0004](ADR-0004-docker.md).
|
||||||
|
|
||||||
|
> **Nachtrag 2 (2026-09-09, beim Bau von S1):** Der Plan sah pnpm-Workspaces
|
||||||
|
> mit einem Paketmanifest je Paket und einen `tsc`-Build vor. Beides ist
|
||||||
|
> weggefallen, weil sich beim Bauen zeigte, dass **Node 24 alles Nötige
|
||||||
|
> mitbringt**:
|
||||||
|
>
|
||||||
|
> | Gebraucht | Statt npm-Paket | Eingebaut |
|
||||||
|
> |---|---|---|
|
||||||
|
> | SQLite | `better-sqlite3` (nativ, node-gyp) | `node:sqlite` |
|
||||||
|
> | WebSocket-Client | `ws` | globales `WebSocket` |
|
||||||
|
> | HTTP-Aufrufe | `undici`/`axios` | globales `fetch` |
|
||||||
|
> | TypeScript ausführen | `tsc`-Build, `tsx` | Type Stripping |
|
||||||
|
> | YAML | `yaml` | eigener Leser, 130 Zeilen |
|
||||||
|
>
|
||||||
|
> Das Projekt hat damit **null Laufzeitabhängigkeiten und keinen
|
||||||
|
> Build-Schritt**. `node docker/entrypoint.ts` läuft direkt. Auflösung der
|
||||||
|
> paketübergreifenden Importe (`#core/…`, `#db/…`) über die `imports`-Map in
|
||||||
|
> der Wurzel-`package.json` — ebenfalls eine Node-Funktion, kein Werkzeug.
|
||||||
|
>
|
||||||
|
> Konsequenz: Die Paketmanifeste und `pnpm-workspace.yaml` sind gelöscht, es
|
||||||
|
> gibt eine `package.json`. `typescript` bleibt als einzige
|
||||||
|
> Entwicklungsabhängigkeit für `tsc --noEmit`.
|
||||||
|
>
|
||||||
|
> **Preis:** Nur *löschbare* TypeScript-Syntax ist erlaubt — keine `enum`,
|
||||||
|
> keine Parameter-Properties, keine `namespace`. Das erzwingt
|
||||||
|
> `erasableSyntaxOnly: true` in der `tsconfig.json`, und es hat beim ersten
|
||||||
|
> Durchlauf sofort einen echten Fund gehabt (`constructor(private capacity)`
|
||||||
|
> im Helix-Client, das zur Laufzeit stillschweigend verschwunden wäre).
|
||||||
|
|
||||||
## Begründung
|
## Begründung
|
||||||
|
|
||||||
|
|||||||
@@ -10,7 +10,8 @@ Viewer-Samples pro Monat — wenige MB.
|
|||||||
|
|
||||||
## Entscheidung
|
## Entscheidung
|
||||||
|
|
||||||
SQLite (`better-sqlite3`) im WAL-Modus, eine Datei unter `data/db/`.
|
SQLite über das eingebaute **`node:sqlite`** im WAL-Modus, eine Datei im
|
||||||
|
Docker-Volume `abdulive-db`.
|
||||||
|
|
||||||
## Begründung
|
## Begründung
|
||||||
|
|
||||||
@@ -18,8 +19,11 @@ SQLite (`better-sqlite3`) im WAL-Modus, eine Datei unter `data/db/`.
|
|||||||
Dateikopie-Vorgang (`VACUUM INTO`).
|
Dateikopie-Vorgang (`VACUUM INTO`).
|
||||||
- WAL erlaubt beliebig viele gleichzeitige Leser neben einem Schreiber. Genau
|
- WAL erlaubt beliebig viele gleichzeitige Leser neben einem Schreiber. Genau
|
||||||
unsere Topologie: Der Ingest schreibt, alle anderen lesen.
|
unsere Topologie: Der Ingest schreibt, alle anderen lesen.
|
||||||
- `better-sqlite3` ist synchron, was den Code im Ingest erheblich vereinfacht —
|
- `node:sqlite` ist synchron (`DatabaseSync`), was den Code im Ingest erheblich
|
||||||
keine Race Conditions zwischen Event-Verarbeitung und Schreibvorgang.
|
vereinfacht - keine Race Conditions zwischen Ereignisverarbeitung und
|
||||||
|
Schreibvorgang.
|
||||||
|
- Es ist **in Node 24 enthalten**: keine native Abhaengigkeit, kein node-gyp,
|
||||||
|
keine Prebuild-Suche im Docker-Build. Siehe Nachtrag 2 in ADR-0001.
|
||||||
- Die Datenmenge ist um Größenordnungen von SQLites Grenzen entfernt.
|
- Die Datenmenge ist um Größenordnungen von SQLites Grenzen entfernt.
|
||||||
|
|
||||||
## Konsequenzen
|
## Konsequenzen
|
||||||
|
|||||||
@@ -0,0 +1,87 @@
|
|||||||
|
# ADR-0005: Zugangsdaten kommen aus der App, nicht aus der `.env`
|
||||||
|
|
||||||
|
**Status:** akzeptiert · **Datum:** 2026-09-09
|
||||||
|
|
||||||
|
## Kontext
|
||||||
|
|
||||||
|
Das Toolkit braucht Client-ID und Client-Secret zweier Twitch-Apps, ein
|
||||||
|
obs-websocket-Passwort und später Discord-/Bluesky-Zugänge. Der erste Entwurf
|
||||||
|
sah dafür eine `.env`-Datei vor, die von Hand befüllt wird.
|
||||||
|
|
||||||
|
Das passt nicht zum Nutzer. Abdu ist Streamer, kein Entwickler. „Kopiere
|
||||||
|
`.env.example` nach `.env` und trage sieben Werte ein" ist die Stelle, an der
|
||||||
|
ein Werkzeug liegen bleibt. Und es kommt ein zweites Problem dazu, das für
|
||||||
|
diesen Anwendungsfall besonders zählt: **Wer eine Klartextdatei mit
|
||||||
|
Zugangsdaten öffnet, während er sendet, hat sie live gezeigt.**
|
||||||
|
|
||||||
|
## Entscheidung
|
||||||
|
|
||||||
|
Zugangsdaten werden über einen **Einrichtungs-Assistenten im Browser**
|
||||||
|
eingegeben, unter `http://localhost:4792/setup`. Gespeichert werden sie
|
||||||
|
verschlüsselt in `/db/secrets.json` im Docker-Volume.
|
||||||
|
|
||||||
|
Die `.env` bleibt bestehen, enthält aber nur noch **optionale** Werte
|
||||||
|
(Log-Level, Pfad zum Wiedergabepuffer). Ohne `.env` startet alles.
|
||||||
|
|
||||||
|
Damit fällt auch Port 4780 weg: Einrichtung, OAuth-Rücksprung und später
|
||||||
|
Sichtungsseite und Dashboard laufen über **einen** Server auf Port 4792. Die
|
||||||
|
Redirect-URI in der Twitch-App lautet entsprechend
|
||||||
|
`http://localhost:4792/callback`.
|
||||||
|
|
||||||
|
## Speicherung
|
||||||
|
|
||||||
|
| | |
|
||||||
|
|---|---|
|
||||||
|
| Ort | `/db/secrets.json`, benanntes Volume `abdulive-db` |
|
||||||
|
| Verfahren | AES-256-GCM, Schlüssel in `/db/.secret-key` (Modus 0600) |
|
||||||
|
| Schlüssel | wird beim ersten Start erzeugt, nie ausgegeben, nie geloggt |
|
||||||
|
| **Nicht** in SQLite | damit das Backup-Skript sie nie mitkopiert |
|
||||||
|
|
||||||
|
Die Secrets liegen bewusst **außerhalb der Datenbank**. `scripts/backup.ps1`
|
||||||
|
zieht die `.db` aus dem Volume nach `data/db/backup/` — also auf die
|
||||||
|
Windows-Seite, wo sie in einer Cloud-Synchronisation landen kann. Was dort
|
||||||
|
nichts zu suchen hat, darf gar nicht erst in der Datenbank stehen.
|
||||||
|
|
||||||
|
## Was diese Verschlüsselung leistet — und was nicht
|
||||||
|
|
||||||
|
Der Schlüssel liegt im selben Volume wie die verschlüsselten Daten. Gegen
|
||||||
|
jemanden, der Zugriff auf das Volume hat, schützt das **nicht**, und es wird
|
||||||
|
hier auch nicht so getan.
|
||||||
|
|
||||||
|
Wogegen es schützt, und das ist der eigentliche Zweck:
|
||||||
|
|
||||||
|
- **Versehentliche Anzeige im Stream.** `cat secrets.json` zeigt Kauderwelsch,
|
||||||
|
keine Client-Secrets.
|
||||||
|
- **Verirrte Kopien.** Ein Volume-Export oder ein weitergegebener Ordner
|
||||||
|
enthält ohne die Schlüsseldatei nichts Verwertbares.
|
||||||
|
- **Grep-Unfälle.** Secrets tauchen in keiner Volltextsuche und in keinem
|
||||||
|
versehentlich geteilten Log auf.
|
||||||
|
|
||||||
|
Gegen eine Kompromittierung der Maschine schützt es nicht. Das ist bei einem
|
||||||
|
lokalen Einzelplatzwerkzeug auch nicht das Bedrohungsmodell — wer dort Zugriff
|
||||||
|
hat, hat auch Zugriff auf OBS, den Browser und die gespeicherten Twitch-Cookies.
|
||||||
|
|
||||||
|
## Konsequenzen für die Oberfläche
|
||||||
|
|
||||||
|
- Der Einrichtungs-Assistent **zeigt gespeicherte Secrets nie wieder an.**
|
||||||
|
Nach dem Speichern steht dort `•••••• gespeichert` und ein Knopf *Ersetzen*.
|
||||||
|
- Über der Seite steht ein Hinweis: **„Diese Seite zeigt Zugangsdaten — nicht
|
||||||
|
im Stream öffnen."**
|
||||||
|
- Der Server ist nur lokal erreichbar. **Nicht** dadurch, dass er im Container
|
||||||
|
auf `127.0.0.1` bindet — das wäre dort die Container-Loopback, die Dockers
|
||||||
|
Portweiterleitung nie erreicht, der Dienst wäre schlicht tot. Die Abschottung
|
||||||
|
kommt aus zwei Teilen: der Container bindet auf `0.0.0.0` **innerhalb seines
|
||||||
|
eigenen Netzwerk-Namensraums**, und Compose veröffentlicht den Port
|
||||||
|
ausschließlich als `127.0.0.1:4792`. Außerhalb von Docker bindet der Server
|
||||||
|
weiterhin auf die Loopback-Adresse; gesteuert über `BIND_HOST`, das nur das
|
||||||
|
Image auf `0.0.0.0` setzt.
|
||||||
|
- Secrets werden nie geloggt, auch nicht bei Fehlern. Der Logger maskiert
|
||||||
|
bekannte Schlüsselnamen.
|
||||||
|
- `check-env` meldet, *ob* Zugangsdaten hinterlegt sind, nie *welche*.
|
||||||
|
|
||||||
|
## Preis
|
||||||
|
|
||||||
|
Der Einrichtungs-Assistent kostet in S1 ungefähr einen zusätzlichen Tag —
|
||||||
|
kleiner HTTP-Server, eine Seite, der Secret-Speicher. Dafür entfällt der
|
||||||
|
gesamte `.env`-Schritt aus der Inbetriebnahme, und das Werkzeug lässt sich
|
||||||
|
später ohne Texteditor umkonfigurieren.
|
||||||
@@ -6,6 +6,7 @@
|
|||||||
| [0002](ADR-0002-sqlite.md) | SQLite statt Postgres | akzeptiert |
|
| [0002](ADR-0002-sqlite.md) | SQLite statt Postgres | akzeptiert |
|
||||||
| [0003](ADR-0003-eventsub-websocket.md) | EventSub ueber WebSocket, Chat ueber IRC | akzeptiert |
|
| [0003](ADR-0003-eventsub-websocket.md) | EventSub ueber WebSocket, Chat ueber IRC | akzeptiert |
|
||||||
| [0004](ADR-0004-docker.md) | Betrieb in Docker, whisper.cpp statt Python | akzeptiert |
|
| [0004](ADR-0004-docker.md) | Betrieb in Docker, whisper.cpp statt Python | akzeptiert |
|
||||||
|
| [0005](ADR-0005-zugangsdaten.md) | Zugangsdaten ueber Einrichtungs-Assistent statt .env | akzeptiert |
|
||||||
|
|
||||||
Neue Entscheidung: Datei `ADR-<nummer>-<kurzname>.md`, Aufbau Kontext /
|
Neue Entscheidung: Datei `ADR-<nummer>-<kurzname>.md`, Aufbau Kontext /
|
||||||
Entscheidung / Begruendung / Konsequenzen.
|
Entscheidung / Begruendung / Konsequenzen.
|
||||||
@@ -122,8 +122,26 @@ Bei Quelle = getrennte Vertical-Szene aus OBS entfällt der ganze Filtergraph.
|
|||||||
## Untertitel
|
## Untertitel
|
||||||
|
|
||||||
`whisper.cpp` (`whisper-cli`), Modell `ggml-small` (deutsch, reicht; `medium`
|
`whisper.cpp` (`whisper-cli`), Modell `ggml-small` (deutsch, reicht; `medium`
|
||||||
bei Bedarf), auf der extrahierten Audiospur (16 kHz Mono WAV). Kein Python,
|
bei Bedarf). Kein Python, keine GPU nötig — auf CPU braucht ein
|
||||||
keine GPU nötig — auf CPU braucht ein 30-Sekunden-Clip unter einer Minute.
|
30-Sekunden-Clip unter einer Minute.
|
||||||
|
|
||||||
|
**Transkribiert wird ausschließlich die Mikrofonspur.** OBS nimmt getrennte
|
||||||
|
Audiospuren auf (Mikro auf Spur 2, Spielton auf Spur 3, Mischung auf Spur 1).
|
||||||
|
Whisper bekommt nur Spur 2 — bei GTA-Motoren, Schüssen und Spielmusik ist das
|
||||||
|
der Unterschied zwischen einem brauchbaren und einem unbrauchbaren Transkript.
|
||||||
|
Ins fertige Short kommt weiterhin die Mischung von Spur 1.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Mikrofonspur herausziehen, 16 kHz Mono, so wie whisper.cpp es erwartet
|
||||||
|
ffmpeg -i raw.mp4 -map 0:a:1 -ac 1 -ar 16000 -c:a pcm_s16le mic.wav
|
||||||
|
whisper-cli -m /models/ggml-small.bin -l de -oj -osrt -f mic.wav
|
||||||
|
```
|
||||||
|
|
||||||
|
`-map 0:a:1` ist der zweite Audiostrom der Datei — welcher das ist, hängt davon
|
||||||
|
ab, wie OBS die Spuren schreibt. Die Zuordnung steht in
|
||||||
|
`config/config.yaml` unter `clipper.audio` und wird beim ersten Clip mit
|
||||||
|
`ffprobe` gegengeprüft.
|
||||||
|
|
||||||
Ausgabe als SRT, Umwandlung in ASS mit Karaoke-Timing
|
Ausgabe als SRT, Umwandlung in ASS mit Karaoke-Timing
|
||||||
(wortweises Highlight — der Stil, der auf Shorts funktioniert), dann einbrennen:
|
(wortweises Highlight — der Stil, der auf Shorts funktioniert), dann einbrennen:
|
||||||
|
|
||||||
|
|||||||
+15
-17
@@ -5,27 +5,25 @@
|
|||||||
"description": "Werkzeugkasten fuer den Twitch-Kanal abdulive_",
|
"description": "Werkzeugkasten fuer den Twitch-Kanal abdulive_",
|
||||||
"type": "module",
|
"type": "module",
|
||||||
"engines": {
|
"engines": {
|
||||||
"node": ">=22"
|
"node": ">=24"
|
||||||
|
},
|
||||||
|
"imports": {
|
||||||
|
"#shared/*": "./packages/shared/src/*",
|
||||||
|
"#core/*": "./packages/core/src/*",
|
||||||
|
"#db/*": "./packages/db/src/*",
|
||||||
|
"#twitch/*": "./packages/twitch/src/*"
|
||||||
},
|
},
|
||||||
"packageManager": "pnpm@9.12.0",
|
|
||||||
"scripts": {
|
"scripts": {
|
||||||
"auth": "pnpm --filter @abdulive/twitch run auth",
|
|
||||||
"db:migrate": "pnpm --filter @abdulive/db run migrate",
|
|
||||||
"db:backup": "powershell -File scripts/backup.ps1",
|
|
||||||
"dev:ingest": "pnpm --filter @abdulive/ingest run dev",
|
|
||||||
"dev:overlay": "pnpm --filter @abdulive/overlay-server run dev",
|
|
||||||
"dev:dashboard": "pnpm --filter @abdulive/dashboard run dev",
|
|
||||||
"clipper:calibrate": "pnpm --filter @abdulive/clipper run calibrate",
|
|
||||||
"clipper:run": "pnpm --filter @abdulive/clipper run once",
|
|
||||||
"raid:scout": "pnpm --filter @abdulive/raid-scout run once",
|
|
||||||
"recap:run": "pnpm --filter @abdulive/recap run once",
|
|
||||||
"build": "pnpm -r run build",
|
|
||||||
"typecheck": "pnpm -r run typecheck",
|
|
||||||
"lint": "eslint .",
|
|
||||||
"test": "pnpm -r run test",
|
|
||||||
"start": "docker compose up -d",
|
"start": "docker compose up -d",
|
||||||
"stop": "docker compose down",
|
"stop": "docker compose down",
|
||||||
"status": "docker compose ps",
|
"status": "docker compose ps",
|
||||||
"logs": "docker compose logs -f"
|
"logs": "docker compose logs -f",
|
||||||
|
"check-env": "node docker/entrypoint.ts check-env",
|
||||||
|
"ingest": "node docker/entrypoint.ts ingest",
|
||||||
|
"typecheck": "tsc --noEmit",
|
||||||
|
"test": "node --test \"packages/**/*.test.ts\" \"apps/**/*.test.ts\""
|
||||||
|
},
|
||||||
|
"devDependencies": {
|
||||||
|
"typescript": "^5.9.0"
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -1,12 +0,0 @@
|
|||||||
{
|
|
||||||
"name": "@abdulive/core",
|
|
||||||
"version": "0.1.0",
|
|
||||||
"private": true,
|
|
||||||
"type": "module",
|
|
||||||
"main": "dist/index.js",
|
|
||||||
"scripts": {
|
|
||||||
"build": "tsc -p tsconfig.json",
|
|
||||||
"typecheck": "tsc -p tsconfig.json --noEmit",
|
|
||||||
"test": "node --test"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -0,0 +1,92 @@
|
|||||||
|
// Konfiguration und Pfade. Alles, was kein Geheimnis ist, steht in
|
||||||
|
// config/config.yaml; Zugangsdaten liegen woanders (siehe #twitch/secrets.ts).
|
||||||
|
|
||||||
|
import fs from 'node:fs';
|
||||||
|
import path from 'node:path';
|
||||||
|
import { parseYaml, type YamlMap } from './yaml.ts';
|
||||||
|
|
||||||
|
export const paths = {
|
||||||
|
config: process.env.CONFIG_DIR || '/app/config',
|
||||||
|
db: process.env.DB_DIR || '/db',
|
||||||
|
work: process.env.WORK_DIR || '/work',
|
||||||
|
clips: process.env.CLIPS_DIR || '/app/data/clips',
|
||||||
|
replay: process.env.REPLAY_DIR || '/replay',
|
||||||
|
models: process.env.MODEL_DIR || '/models',
|
||||||
|
};
|
||||||
|
|
||||||
|
export type Config = {
|
||||||
|
channel: { login: string; userId: string; language: string; timezone: string };
|
||||||
|
bot: { login: string };
|
||||||
|
ingest: {
|
||||||
|
viewerPollSeconds: number;
|
||||||
|
chattersPollSeconds: number;
|
||||||
|
sessionGapMinutes: number;
|
||||||
|
spike: {
|
||||||
|
windowSeconds: number; baselineMinutes: number; factor: number;
|
||||||
|
minDelta: number; cooldownSeconds: number; armed: boolean; laughEmotes: string[];
|
||||||
|
};
|
||||||
|
};
|
||||||
|
obs: { url: string; replayBufferSeconds: number; chapterScenes: string[] };
|
||||||
|
ingestServer: { port: number };
|
||||||
|
dashboard: { port: number };
|
||||||
|
raw: YamlMap;
|
||||||
|
};
|
||||||
|
|
||||||
|
const DEFAULTS = {
|
||||||
|
channel: { login: 'abdulive_', userId: '', language: 'de', timezone: 'Europe/Berlin' },
|
||||||
|
bot: { login: '' },
|
||||||
|
ingest: {
|
||||||
|
viewerPollSeconds: 60,
|
||||||
|
chattersPollSeconds: 300,
|
||||||
|
sessionGapMinutes: 5,
|
||||||
|
spike: {
|
||||||
|
windowSeconds: 15, baselineMinutes: 10, factor: 3, minDelta: 4,
|
||||||
|
cooldownSeconds: 60, armed: false,
|
||||||
|
laughEmotes: ['KEKW', 'LUL', 'OMEGALUL', 'PogChamp', 'Pog'],
|
||||||
|
},
|
||||||
|
},
|
||||||
|
obs: { url: 'ws://host.docker.internal:4455', replayBufferSeconds: 90, chapterScenes: [] },
|
||||||
|
ingestServer: { port: 4790 },
|
||||||
|
dashboard: { port: 4792 },
|
||||||
|
};
|
||||||
|
|
||||||
|
function merge<T>(base: T, over: unknown): T {
|
||||||
|
if (over === null || over === undefined) return base;
|
||||||
|
if (typeof base !== 'object' || base === null || Array.isArray(base)) return over as T;
|
||||||
|
if (typeof over !== 'object' || over === null || Array.isArray(over)) return base;
|
||||||
|
const out: Record<string, unknown> = { ...(base as Record<string, unknown>) };
|
||||||
|
for (const [k, v] of Object.entries(over)) {
|
||||||
|
out[k] = k in out ? merge((base as Record<string, unknown>)[k], v) : v;
|
||||||
|
}
|
||||||
|
return out as T;
|
||||||
|
}
|
||||||
|
|
||||||
|
let cached: Config | null = null;
|
||||||
|
|
||||||
|
export function loadConfig(): Config {
|
||||||
|
if (cached) return cached;
|
||||||
|
const file = path.join(paths.config, 'config.yaml');
|
||||||
|
let raw: YamlMap = {};
|
||||||
|
if (fs.existsSync(file)) {
|
||||||
|
raw = parseYaml(fs.readFileSync(file, 'utf8'));
|
||||||
|
}
|
||||||
|
cached = { ...merge(DEFAULTS, raw), raw } as Config;
|
||||||
|
return cached;
|
||||||
|
}
|
||||||
|
|
||||||
|
export type Rect = { x: number; y: number; w: number; h: number };
|
||||||
|
export type LayoutProfile = {
|
||||||
|
description?: string;
|
||||||
|
cam: { crop: Rect; band: { w: number; h: number } };
|
||||||
|
game: { crop: Rect; band: { w: number; h: number } };
|
||||||
|
};
|
||||||
|
export type Layout = {
|
||||||
|
source: { width: number; height: number };
|
||||||
|
target: { width: number; height: number };
|
||||||
|
profiles: Record<string, LayoutProfile>;
|
||||||
|
};
|
||||||
|
|
||||||
|
export function loadLayout(): Layout {
|
||||||
|
const file = path.join(paths.config, 'layout.yaml');
|
||||||
|
return parseYaml(fs.readFileSync(file, 'utf8')) as unknown as Layout;
|
||||||
|
}
|
||||||
@@ -1,2 +0,0 @@
|
|||||||
// @abdulive/core — siehe docs/01-architecture.md
|
|
||||||
export {};
|
|
||||||
@@ -0,0 +1,63 @@
|
|||||||
|
// Logger mit Maskierung. Zugangsdaten duerfen nie in einem Log landen -
|
||||||
|
// Logs werden weitergegeben, in Issues gepastet und im Zweifel im Stream
|
||||||
|
// gezeigt. Siehe ADR-0005.
|
||||||
|
|
||||||
|
export type Level = 'debug' | 'info' | 'warn' | 'error';
|
||||||
|
|
||||||
|
const ORDER: Record<Level, number> = { debug: 10, info: 20, warn: 30, error: 40 };
|
||||||
|
const MIN = ORDER[(process.env.LOG_LEVEL as Level) || 'info'] ?? ORDER.info;
|
||||||
|
|
||||||
|
// Feldnamen, deren Wert nie ausgegeben wird
|
||||||
|
const SECRET_KEYS =
|
||||||
|
/(secret|token|password|passwort|authorization|client_?id|refresh|api_?key|webhook)/i;
|
||||||
|
|
||||||
|
// Muster, die auch in freiem Text maskiert werden
|
||||||
|
const SECRET_PATTERNS: Array<[RegExp, string]> = [
|
||||||
|
[/oauth:[A-Za-z0-9]+/g, 'oauth:•••'],
|
||||||
|
[/\b[a-z0-9]{30}\b/g, '•••'], // Twitch Client-ID / Token
|
||||||
|
[/\bBearer\s+\S+/gi, 'Bearer •••'],
|
||||||
|
];
|
||||||
|
|
||||||
|
export function mask(value: unknown): unknown {
|
||||||
|
if (typeof value === 'string') {
|
||||||
|
let out = value;
|
||||||
|
for (const [re, replacement] of SECRET_PATTERNS) out = out.replace(re, replacement);
|
||||||
|
return out;
|
||||||
|
}
|
||||||
|
if (Array.isArray(value)) return value.map(mask);
|
||||||
|
if (value && typeof value === 'object') {
|
||||||
|
const out: Record<string, unknown> = {};
|
||||||
|
for (const [k, v] of Object.entries(value)) {
|
||||||
|
out[k] = SECRET_KEYS.test(k) ? (v == null ? v : '•••') : mask(v);
|
||||||
|
}
|
||||||
|
return out;
|
||||||
|
}
|
||||||
|
return value;
|
||||||
|
}
|
||||||
|
|
||||||
|
function emit(level: Level, scope: string, msg: string, extra?: unknown) {
|
||||||
|
if (ORDER[level] < MIN) return;
|
||||||
|
const ts = new Date().toISOString().slice(11, 23);
|
||||||
|
const tag = level.toUpperCase().padEnd(5);
|
||||||
|
let line = `${ts} ${tag} [${scope}] ${String(mask(msg))}`;
|
||||||
|
if (extra !== undefined) line += ` ${JSON.stringify(mask(extra))}`;
|
||||||
|
(level === 'error' || level === 'warn' ? console.error : console.log)(line);
|
||||||
|
}
|
||||||
|
|
||||||
|
export type Logger = {
|
||||||
|
debug(msg: string, extra?: unknown): void;
|
||||||
|
info(msg: string, extra?: unknown): void;
|
||||||
|
warn(msg: string, extra?: unknown): void;
|
||||||
|
error(msg: string, extra?: unknown): void;
|
||||||
|
child(sub: string): Logger;
|
||||||
|
};
|
||||||
|
|
||||||
|
export function logger(scope: string): Logger {
|
||||||
|
return {
|
||||||
|
debug: (m, e) => emit('debug', scope, m, e),
|
||||||
|
info: (m, e) => emit('info', scope, m, e),
|
||||||
|
warn: (m, e) => emit('warn', scope, m, e),
|
||||||
|
error: (m, e) => emit('error', scope, m, e),
|
||||||
|
child: (sub) => logger(`${scope}:${sub}`),
|
||||||
|
};
|
||||||
|
}
|
||||||
@@ -0,0 +1,70 @@
|
|||||||
|
// Wiederholung mit exponentiellem Backoff und Jitter. Jeder Netzwerkaufruf
|
||||||
|
// laeuft hierdurch - siehe docs/01-architecture.md, Abschnitt Fehlerverhalten.
|
||||||
|
|
||||||
|
export type RetryOptions = {
|
||||||
|
attempts?: number;
|
||||||
|
baseMs?: number;
|
||||||
|
maxMs?: number;
|
||||||
|
onRetry?: (err: unknown, attempt: number, waitMs: number) => void;
|
||||||
|
signal?: AbortSignal;
|
||||||
|
};
|
||||||
|
|
||||||
|
export function sleep(ms: number, signal?: AbortSignal): Promise<void> {
|
||||||
|
return new Promise((resolve, reject) => {
|
||||||
|
if (signal?.aborted) return reject(new Error('abgebrochen'));
|
||||||
|
const t = setTimeout(resolve, ms);
|
||||||
|
signal?.addEventListener('abort', () => { clearTimeout(t); reject(new Error('abgebrochen')); },
|
||||||
|
{ once: true });
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function retry<T>(fn: () => Promise<T>, opts: RetryOptions = {}): Promise<T> {
|
||||||
|
const attempts = opts.attempts ?? 5;
|
||||||
|
const base = opts.baseMs ?? 500;
|
||||||
|
const max = opts.maxMs ?? 30_000;
|
||||||
|
let lastErr: unknown;
|
||||||
|
|
||||||
|
for (let i = 0; i < attempts; i++) {
|
||||||
|
try {
|
||||||
|
return await fn();
|
||||||
|
} catch (err) {
|
||||||
|
lastErr = err;
|
||||||
|
if (i === attempts - 1) break;
|
||||||
|
const backoff = Math.min(max, base * 2 ** i);
|
||||||
|
const wait = Math.round(backoff * (0.5 + Math.random() * 0.5));
|
||||||
|
opts.onRetry?.(err, i + 1, wait);
|
||||||
|
await sleep(wait, opts.signal);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
throw lastErr;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Haelt eine Verbindung dauerhaft am Leben: `connect` wird nach jedem Ende
|
||||||
|
* neu aufgerufen, mit wachsender Pause. Der Zaehler wird zurueckgesetzt,
|
||||||
|
* sobald eine Verbindung laenger als `stableMs` gehalten hat - sonst waechst
|
||||||
|
* die Pause nach Tagen Laufzeit ins Absurde.
|
||||||
|
*/
|
||||||
|
export async function forever(
|
||||||
|
connect: (signal: AbortSignal) => Promise<void>,
|
||||||
|
opts: { signal: AbortSignal; baseMs?: number; maxMs?: number; stableMs?: number;
|
||||||
|
onWait?: (ms: number, failures: number) => void }
|
||||||
|
): Promise<void> {
|
||||||
|
const base = opts.baseMs ?? 1000;
|
||||||
|
const max = opts.maxMs ?? 60_000;
|
||||||
|
const stable = opts.stableMs ?? 60_000;
|
||||||
|
let failures = 0;
|
||||||
|
|
||||||
|
while (!opts.signal.aborted) {
|
||||||
|
const started = Date.now();
|
||||||
|
try {
|
||||||
|
await connect(opts.signal);
|
||||||
|
} catch { /* Ursache hat der Aufrufer schon geloggt */ }
|
||||||
|
if (opts.signal.aborted) return;
|
||||||
|
if (Date.now() - started > stable) failures = 0;
|
||||||
|
failures++;
|
||||||
|
const wait = Math.min(max, base * 2 ** (failures - 1));
|
||||||
|
opts.onWait?.(wait, failures);
|
||||||
|
try { await sleep(wait, opts.signal); } catch { return; }
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -2,10 +2,14 @@
|
|||||||
//
|
//
|
||||||
// Deckt genau das ab, was config/*.yaml benutzt: verschachtelte Maps ueber
|
// Deckt genau das ab, was config/*.yaml benutzt: verschachtelte Maps ueber
|
||||||
// Einrueckung, Skalare, Inline-Maps `{a: 1, b: 2}`, Listen aus Skalaren und
|
// Einrueckung, Skalare, Inline-Maps `{a: 1, b: 2}`, Listen aus Skalaren und
|
||||||
// Blockskalare (`>` / `|`). Bewusst kein vollstaendiges YAML - sobald in S1
|
// Blockskalare (`>` / `|`). Bewusst kein vollstaendiges YAML - das Projekt
|
||||||
// echte Abhaengigkeiten dazukommen, wird das durch `yaml` aus npm ersetzt.
|
// kommt ohne npm-Abhaengigkeiten aus, und der Sprachumfang, den wir selbst
|
||||||
|
// schreiben, ist ueberschaubar.
|
||||||
|
|
||||||
function scalar(raw) {
|
export type YamlValue = string | number | boolean | null | YamlValue[] | { [k: string]: YamlValue };
|
||||||
|
export type YamlMap = { [k: string]: YamlValue };
|
||||||
|
|
||||||
|
function scalar(raw: string): YamlValue {
|
||||||
const s = raw.trim();
|
const s = raw.trim();
|
||||||
if (s === '' || s === '~' || s === 'null') return null;
|
if (s === '' || s === '~' || s === 'null') return null;
|
||||||
if (s === 'true') return true;
|
if (s === 'true') return true;
|
||||||
@@ -18,8 +22,8 @@ function scalar(raw) {
|
|||||||
return s;
|
return s;
|
||||||
}
|
}
|
||||||
|
|
||||||
function flowMap(raw) {
|
function flowMap(raw: string): YamlMap {
|
||||||
const out = {};
|
const out: YamlMap = {};
|
||||||
for (const part of raw.slice(1, -1).split(',')) {
|
for (const part of raw.slice(1, -1).split(',')) {
|
||||||
if (!part.trim()) continue;
|
if (!part.trim()) continue;
|
||||||
const i = part.indexOf(':');
|
const i = part.indexOf(':');
|
||||||
@@ -29,43 +33,43 @@ function flowMap(raw) {
|
|||||||
return out;
|
return out;
|
||||||
}
|
}
|
||||||
|
|
||||||
function stripComment(line) {
|
function stripComment(line: string): string {
|
||||||
let quote = null;
|
let quote: string | null = null;
|
||||||
for (let i = 0; i < line.length; i++) {
|
for (let i = 0; i < line.length; i++) {
|
||||||
const c = line[i];
|
const c = line[i]!;
|
||||||
if (quote) {
|
if (quote) {
|
||||||
if (c === quote) quote = null;
|
if (c === quote) quote = null;
|
||||||
} else if (c === '"' || c === "'") {
|
} else if (c === '"' || c === "'") {
|
||||||
quote = c;
|
quote = c;
|
||||||
} else if (c === '#' && (i === 0 || /\s/.test(line[i - 1]))) {
|
} else if (c === '#' && (i === 0 || /\s/.test(line[i - 1]!))) {
|
||||||
return line.slice(0, i);
|
return line.slice(0, i);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
return line;
|
return line;
|
||||||
}
|
}
|
||||||
|
|
||||||
export function parseYaml(text) {
|
type Frame = { indent: number; target: YamlMap; listKey?: string };
|
||||||
|
|
||||||
|
export function parseYaml(text: string): YamlMap {
|
||||||
const lines = text.split(/\r?\n/);
|
const lines = text.split(/\r?\n/);
|
||||||
const root = {};
|
const root: YamlMap = {};
|
||||||
// Stapel aus [Einrueckung, Zielobjekt]
|
const stack: Frame[] = [{ indent: -1, target: root }];
|
||||||
const stack = [[-1, root]];
|
|
||||||
let i = 0;
|
let i = 0;
|
||||||
|
|
||||||
while (i < lines.length) {
|
while (i < lines.length) {
|
||||||
const rawLine = lines[i];
|
const line = stripComment(lines[i]!);
|
||||||
const line = stripComment(rawLine);
|
|
||||||
if (!line.trim()) { i++; continue; }
|
if (!line.trim()) { i++; continue; }
|
||||||
|
|
||||||
const indent = line.length - line.trimStart().length;
|
const indent = line.length - line.trimStart().length;
|
||||||
const body = line.trim();
|
const body = line.trim();
|
||||||
|
|
||||||
while (stack.length > 1 && indent <= stack[stack.length - 1][0]) stack.pop();
|
while (stack.length > 1 && indent <= stack[stack.length - 1]!.indent) stack.pop();
|
||||||
const parent = stack[stack.length - 1][1];
|
const frame = stack[stack.length - 1]!;
|
||||||
|
|
||||||
// Listeneintrag
|
|
||||||
if (body.startsWith('- ')) {
|
if (body.startsWith('- ')) {
|
||||||
const key = stack[stack.length - 1][2];
|
const key = frame.listKey;
|
||||||
if (key && Array.isArray(parent[key])) parent[key].push(scalar(body.slice(2)));
|
const list = key ? frame.target[key] : undefined;
|
||||||
|
if (Array.isArray(list)) list.push(scalar(body.slice(2)));
|
||||||
i++;
|
i++;
|
||||||
continue;
|
continue;
|
||||||
}
|
}
|
||||||
@@ -76,40 +80,40 @@ export function parseYaml(text) {
|
|||||||
const rest = body.slice(colon + 1).trim();
|
const rest = body.slice(colon + 1).trim();
|
||||||
|
|
||||||
if (rest === '') {
|
if (rest === '') {
|
||||||
// Verschachtelte Map, leere Liste oder leerer Wert - anhand der
|
// Verschachtelte Map, Liste oder leerer Wert - die naechste
|
||||||
// naechsten inhaltstragenden Zeile entscheiden
|
// inhaltstragende Zeile entscheidet
|
||||||
let j = i + 1;
|
let j = i + 1;
|
||||||
while (j < lines.length && !stripComment(lines[j]).trim()) j++;
|
while (j < lines.length && !stripComment(lines[j]!).trim()) j++;
|
||||||
const next = j < lines.length ? stripComment(lines[j]) : '';
|
const next = j < lines.length ? stripComment(lines[j]!) : '';
|
||||||
const nextIndent = next ? next.length - next.trimStart().length : -1;
|
const nextIndent = next ? next.length - next.trimStart().length : -1;
|
||||||
if (next && nextIndent > indent && next.trim().startsWith('- ')) {
|
if (next && nextIndent > indent && next.trim().startsWith('- ')) {
|
||||||
parent[key] = [];
|
frame.target[key] = [];
|
||||||
stack.push([indent, parent, key]);
|
stack.push({ indent, target: frame.target, listKey: key });
|
||||||
} else if (next && nextIndent > indent) {
|
} else if (next && nextIndent > indent) {
|
||||||
parent[key] = {};
|
const child: YamlMap = {};
|
||||||
stack.push([indent, parent[key]]);
|
frame.target[key] = child;
|
||||||
|
stack.push({ indent, target: child });
|
||||||
} else {
|
} else {
|
||||||
parent[key] = null;
|
frame.target[key] = null;
|
||||||
}
|
}
|
||||||
} else if (rest === '>' || rest === '|' || rest === '>-' || rest === '|-') {
|
} else if (rest === '>' || rest === '|' || rest === '>-' || rest === '|-') {
|
||||||
// Blockskalar: alle staerker eingerueckten Zeilen einsammeln
|
const parts: string[] = [];
|
||||||
const parts = [];
|
|
||||||
let j = i + 1;
|
let j = i + 1;
|
||||||
for (; j < lines.length; j++) {
|
for (; j < lines.length; j++) {
|
||||||
if (!lines[j].trim()) { parts.push(''); continue; }
|
const raw = lines[j]!;
|
||||||
const ind = lines[j].length - lines[j].trimStart().length;
|
if (!raw.trim()) { parts.push(''); continue; }
|
||||||
if (ind <= indent) break;
|
if (raw.length - raw.trimStart().length <= indent) break;
|
||||||
parts.push(lines[j].trim());
|
parts.push(raw.trim());
|
||||||
}
|
}
|
||||||
parent[key] = rest.startsWith('>') ? parts.join(' ').trim() : parts.join('\n');
|
frame.target[key] = rest.startsWith('>') ? parts.join(' ').trim() : parts.join('\n');
|
||||||
i = j;
|
i = j;
|
||||||
continue;
|
continue;
|
||||||
} else if (rest.startsWith('{')) {
|
} else if (rest.startsWith('{')) {
|
||||||
parent[key] = flowMap(rest);
|
frame.target[key] = flowMap(rest);
|
||||||
} else if (rest.startsWith('[')) {
|
} else if (rest.startsWith('[')) {
|
||||||
parent[key] = rest.slice(1, -1).split(',').map(scalar).filter((v) => v !== null);
|
frame.target[key] = rest.slice(1, -1).split(',').map(scalar).filter((v) => v !== null);
|
||||||
} else {
|
} else {
|
||||||
parent[key] = scalar(rest);
|
frame.target[key] = scalar(rest);
|
||||||
}
|
}
|
||||||
i++;
|
i++;
|
||||||
}
|
}
|
||||||
@@ -1,5 +0,0 @@
|
|||||||
{
|
|
||||||
"extends": "../../tsconfig.base.json",
|
|
||||||
"compilerOptions": { "outDir": "dist", "rootDir": "src" },
|
|
||||||
"include": ["src"]
|
|
||||||
}
|
|
||||||
@@ -0,0 +1,111 @@
|
|||||||
|
-- Grundschema. Siehe docs/03-data-model.md.
|
||||||
|
-- Zugangsdaten stehen bewusst NICHT hier drin (ADR-0005) - diese Datei
|
||||||
|
-- wird per VACUUM INTO auf die Windows-Seite gesichert.
|
||||||
|
|
||||||
|
CREATE TABLE events (
|
||||||
|
id TEXT PRIMARY KEY,
|
||||||
|
ts TEXT NOT NULL,
|
||||||
|
type TEXT NOT NULL,
|
||||||
|
stream_id TEXT,
|
||||||
|
user_id TEXT,
|
||||||
|
payload TEXT NOT NULL
|
||||||
|
);
|
||||||
|
CREATE INDEX idx_events_ts ON events(ts);
|
||||||
|
CREATE INDEX idx_events_type ON events(type, ts);
|
||||||
|
CREATE INDEX idx_events_strm ON events(stream_id, ts);
|
||||||
|
|
||||||
|
CREATE TABLE streams (
|
||||||
|
id TEXT PRIMARY KEY,
|
||||||
|
started_at TEXT NOT NULL,
|
||||||
|
ended_at TEXT,
|
||||||
|
vod_id TEXT,
|
||||||
|
peak_viewers INTEGER NOT NULL DEFAULT 0,
|
||||||
|
avg_viewers REAL,
|
||||||
|
unique_chatters INTEGER NOT NULL DEFAULT 0,
|
||||||
|
new_followers INTEGER NOT NULL DEFAULT 0
|
||||||
|
);
|
||||||
|
|
||||||
|
CREATE TABLE stream_segments (
|
||||||
|
id INTEGER PRIMARY KEY,
|
||||||
|
stream_id TEXT NOT NULL REFERENCES streams(id),
|
||||||
|
started_at TEXT NOT NULL,
|
||||||
|
ended_at TEXT,
|
||||||
|
title TEXT,
|
||||||
|
category_id TEXT,
|
||||||
|
category TEXT,
|
||||||
|
obs_scene TEXT
|
||||||
|
);
|
||||||
|
CREATE INDEX idx_segments_stream ON stream_segments(stream_id, started_at);
|
||||||
|
|
||||||
|
CREATE TABLE viewer_samples (
|
||||||
|
stream_id TEXT NOT NULL REFERENCES streams(id),
|
||||||
|
ts TEXT NOT NULL,
|
||||||
|
viewers INTEGER NOT NULL,
|
||||||
|
PRIMARY KEY (stream_id, ts)
|
||||||
|
);
|
||||||
|
|
||||||
|
CREATE TABLE users (
|
||||||
|
id TEXT PRIMARY KEY,
|
||||||
|
login TEXT NOT NULL,
|
||||||
|
display_name TEXT NOT NULL,
|
||||||
|
first_seen_at TEXT NOT NULL,
|
||||||
|
last_seen_at TEXT NOT NULL,
|
||||||
|
streams_present INTEGER NOT NULL DEFAULT 0,
|
||||||
|
streams_chatted INTEGER NOT NULL DEFAULT 0,
|
||||||
|
message_count INTEGER NOT NULL DEFAULT 0,
|
||||||
|
is_follower INTEGER NOT NULL DEFAULT 0,
|
||||||
|
followed_at TEXT,
|
||||||
|
is_sub INTEGER NOT NULL DEFAULT 0,
|
||||||
|
sub_tier TEXT,
|
||||||
|
points INTEGER NOT NULL DEFAULT 0,
|
||||||
|
watch_minutes INTEGER NOT NULL DEFAULT 0,
|
||||||
|
greeting TEXT,
|
||||||
|
notes TEXT,
|
||||||
|
pronouns TEXT,
|
||||||
|
muted_greeting INTEGER NOT NULL DEFAULT 0
|
||||||
|
);
|
||||||
|
CREATE INDEX idx_users_login ON users(login);
|
||||||
|
|
||||||
|
CREATE TABLE user_streams (
|
||||||
|
user_id TEXT NOT NULL REFERENCES users(id),
|
||||||
|
stream_id TEXT NOT NULL REFERENCES streams(id),
|
||||||
|
first_ts TEXT NOT NULL,
|
||||||
|
last_ts TEXT NOT NULL,
|
||||||
|
messages INTEGER NOT NULL DEFAULT 0,
|
||||||
|
minutes INTEGER NOT NULL DEFAULT 0,
|
||||||
|
PRIMARY KEY (user_id, stream_id)
|
||||||
|
);
|
||||||
|
|
||||||
|
CREATE TABLE chat_messages (
|
||||||
|
id TEXT PRIMARY KEY,
|
||||||
|
stream_id TEXT REFERENCES streams(id),
|
||||||
|
ts TEXT NOT NULL,
|
||||||
|
user_id TEXT NOT NULL REFERENCES users(id),
|
||||||
|
text TEXT NOT NULL,
|
||||||
|
emotes TEXT,
|
||||||
|
is_first INTEGER NOT NULL DEFAULT 0,
|
||||||
|
is_command INTEGER NOT NULL DEFAULT 0,
|
||||||
|
bits INTEGER NOT NULL DEFAULT 0
|
||||||
|
);
|
||||||
|
CREATE INDEX idx_chat_stream_ts ON chat_messages(stream_id, ts);
|
||||||
|
CREATE INDEX idx_chat_user ON chat_messages(user_id, ts);
|
||||||
|
|
||||||
|
CREATE TABLE clips (
|
||||||
|
id TEXT PRIMARY KEY,
|
||||||
|
twitch_clip_id TEXT,
|
||||||
|
stream_id TEXT REFERENCES streams(id),
|
||||||
|
created_at TEXT NOT NULL,
|
||||||
|
trigger TEXT NOT NULL,
|
||||||
|
trigger_score REAL,
|
||||||
|
source TEXT NOT NULL,
|
||||||
|
raw_path TEXT,
|
||||||
|
vertical_path TEXT,
|
||||||
|
subtitle_path TEXT,
|
||||||
|
duration_s REAL,
|
||||||
|
title TEXT,
|
||||||
|
transcript TEXT,
|
||||||
|
status TEXT NOT NULL,
|
||||||
|
score REAL,
|
||||||
|
uploaded_to TEXT
|
||||||
|
);
|
||||||
|
CREATE INDEX idx_clips_stream ON clips(stream_id, created_at);
|
||||||
@@ -1,12 +0,0 @@
|
|||||||
{
|
|
||||||
"name": "@abdulive/db",
|
|
||||||
"version": "0.1.0",
|
|
||||||
"private": true,
|
|
||||||
"type": "module",
|
|
||||||
"main": "dist/index.js",
|
|
||||||
"scripts": {
|
|
||||||
"build": "tsc -p tsconfig.json",
|
|
||||||
"typecheck": "tsc -p tsconfig.json --noEmit",
|
|
||||||
"test": "node --test"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,2 +1,63 @@
|
|||||||
// @abdulive/db — siehe docs/03-data-model.md
|
// SQLite ueber das eingebaute node:sqlite - keine native Abhaengigkeit,
|
||||||
export {};
|
// kein node-gyp, kein Build. Siehe ADR-0002.
|
||||||
|
|
||||||
|
import { DatabaseSync } from 'node:sqlite';
|
||||||
|
import fs from 'node:fs';
|
||||||
|
import path from 'node:path';
|
||||||
|
import { fileURLToPath } from 'node:url';
|
||||||
|
import { paths } from '#core/config.ts';
|
||||||
|
import { logger } from '#core/logger.ts';
|
||||||
|
|
||||||
|
const log = logger('db');
|
||||||
|
const MIGRATIONS = path.join(path.dirname(fileURLToPath(import.meta.url)), '..', 'migrations');
|
||||||
|
|
||||||
|
export type Db = DatabaseSync;
|
||||||
|
|
||||||
|
let db: Db | null = null;
|
||||||
|
|
||||||
|
export function openDb(file = path.join(paths.db, 'abdulive.db')): Db {
|
||||||
|
if (db) return db;
|
||||||
|
fs.mkdirSync(path.dirname(file), { recursive: true });
|
||||||
|
const handle = new DatabaseSync(file);
|
||||||
|
|
||||||
|
// WAL erlaubt beliebig viele Leser neben einem Schreiber - genau unsere
|
||||||
|
// Topologie. busy_timeout faengt die kurzen Ueberschneidungen ab.
|
||||||
|
handle.exec('PRAGMA journal_mode = WAL');
|
||||||
|
handle.exec('PRAGMA busy_timeout = 5000');
|
||||||
|
handle.exec('PRAGMA foreign_keys = ON');
|
||||||
|
handle.exec('PRAGMA synchronous = NORMAL');
|
||||||
|
|
||||||
|
migrate(handle);
|
||||||
|
db = handle;
|
||||||
|
return db;
|
||||||
|
}
|
||||||
|
|
||||||
|
export function migrate(handle: Db): void {
|
||||||
|
handle.exec('CREATE TABLE IF NOT EXISTS schema_migrations (name TEXT PRIMARY KEY, applied_at TEXT NOT NULL)');
|
||||||
|
const applied = new Set(
|
||||||
|
(handle.prepare('SELECT name FROM schema_migrations').all() as Array<{ name: string }>)
|
||||||
|
.map((r) => r.name)
|
||||||
|
);
|
||||||
|
|
||||||
|
const files = fs.readdirSync(MIGRATIONS).filter((f) => f.endsWith('.sql')).sort();
|
||||||
|
for (const name of files) {
|
||||||
|
if (applied.has(name)) continue;
|
||||||
|
const sql = fs.readFileSync(path.join(MIGRATIONS, name), 'utf8');
|
||||||
|
handle.exec('BEGIN');
|
||||||
|
try {
|
||||||
|
handle.exec(sql);
|
||||||
|
handle.prepare('INSERT INTO schema_migrations (name, applied_at) VALUES (?, ?)')
|
||||||
|
.run(name, new Date().toISOString());
|
||||||
|
handle.exec('COMMIT');
|
||||||
|
log.info(`Migration angewendet: ${name}`);
|
||||||
|
} catch (err) {
|
||||||
|
handle.exec('ROLLBACK');
|
||||||
|
throw new Error(`Migration ${name} fehlgeschlagen: ${(err as Error).message}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
export function closeDb(): void {
|
||||||
|
db?.close();
|
||||||
|
db = null;
|
||||||
|
}
|
||||||
@@ -0,0 +1,114 @@
|
|||||||
|
import { test } from 'node:test';
|
||||||
|
import assert from 'node:assert/strict';
|
||||||
|
import fs from 'node:fs';
|
||||||
|
import os from 'node:os';
|
||||||
|
import path from 'node:path';
|
||||||
|
import type { ChatMessage } from '#shared/events.ts';
|
||||||
|
import { migrate } from './index.ts';
|
||||||
|
import { repos } from './repos.ts';
|
||||||
|
import { DatabaseSync } from 'node:sqlite';
|
||||||
|
|
||||||
|
function freshDb() {
|
||||||
|
const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'abdulive-test-'));
|
||||||
|
const db = new DatabaseSync(path.join(dir, 't.db'));
|
||||||
|
db.exec('PRAGMA foreign_keys = ON');
|
||||||
|
migrate(db);
|
||||||
|
return { db, repo: repos(db), dir };
|
||||||
|
}
|
||||||
|
|
||||||
|
function msg(over: Partial<ChatMessage> = {}): ChatMessage {
|
||||||
|
return {
|
||||||
|
id: `m-${Math.random().toString(36).slice(2)}`,
|
||||||
|
ts: '2026-09-09T20:00:00.000Z',
|
||||||
|
userId: '42', login: 'maxi_2k', displayName: 'maxi_2k',
|
||||||
|
text: 'moin', emotes: [], isFirst: false, isReturning: false,
|
||||||
|
isCommand: false, isMod: false, isSub: false, isBroadcaster: false,
|
||||||
|
bits: 0, color: null, ...over,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
test('legt Nutzer an und zaehlt Nachrichten', () => {
|
||||||
|
const { db, repo } = freshDb();
|
||||||
|
repo.openStream('s1', '2026-09-09T19:00:00.000Z');
|
||||||
|
repo.recordMessage(msg(), 's1');
|
||||||
|
repo.recordMessage(msg({ text: 'zweite' }), 's1');
|
||||||
|
|
||||||
|
const user = db.prepare('SELECT * FROM users WHERE id = ?').get('42') as
|
||||||
|
{ message_count: number; login: string; first_seen_at: string };
|
||||||
|
assert.equal(user.message_count, 2);
|
||||||
|
assert.equal(user.login, 'maxi_2k');
|
||||||
|
const n = db.prepare('SELECT COUNT(*) AS n FROM chat_messages').get() as { n: number };
|
||||||
|
assert.equal(n.n, 2);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('Namenswechsel erhaelt die Historie', () => {
|
||||||
|
const { db, repo } = freshDb();
|
||||||
|
repo.openStream('s1', '2026-09-09T19:00:00.000Z');
|
||||||
|
repo.recordMessage(msg(), 's1');
|
||||||
|
// Gleiche user-id, neuer Login - so sieht eine Twitch-Namensaenderung aus
|
||||||
|
repo.recordMessage(msg({ login: 'maxi_neu', displayName: 'Maxi_Neu' }), 's1');
|
||||||
|
|
||||||
|
const rows = db.prepare('SELECT * FROM users').all() as Array<{ login: string; message_count: number }>;
|
||||||
|
assert.equal(rows.length, 1, 'darf keinen zweiten Nutzer angelegt haben');
|
||||||
|
assert.equal(rows[0]!.login, 'maxi_neu');
|
||||||
|
assert.equal(rows[0]!.message_count, 2);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('zaehlt Streams, an denen jemand geschrieben hat', () => {
|
||||||
|
const { repo } = freshDb();
|
||||||
|
repo.openStream('s1', '2026-09-09T19:00:00.000Z');
|
||||||
|
repo.openStream('s2', '2026-09-10T19:00:00.000Z');
|
||||||
|
assert.equal(repo.recordMessage(msg(), 's1').streamsChatted, 1);
|
||||||
|
assert.equal(repo.recordMessage(msg(), 's1').streamsChatted, 1);
|
||||||
|
assert.equal(repo.recordMessage(msg(), 's2').streamsChatted, 2);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('doppelte Nachrichten-IDs werden nicht zweimal gespeichert', () => {
|
||||||
|
const { db, repo } = freshDb();
|
||||||
|
repo.openStream('s1', '2026-09-09T19:00:00.000Z');
|
||||||
|
const m = msg({ id: 'immer-gleich' });
|
||||||
|
repo.recordMessage(m, 's1');
|
||||||
|
repo.recordMessage(m, 's1');
|
||||||
|
const n = db.prepare('SELECT COUNT(*) AS n FROM chat_messages').get() as { n: number };
|
||||||
|
assert.equal(n.n, 1);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('Zuschauerzahlen setzen den Hoechststand', () => {
|
||||||
|
const { repo } = freshDb();
|
||||||
|
repo.openStream('s1', '2026-09-09T19:00:00.000Z');
|
||||||
|
repo.sample('s1', '2026-09-09T19:01:00.000Z', 8);
|
||||||
|
repo.sample('s1', '2026-09-09T19:02:00.000Z', 17);
|
||||||
|
repo.sample('s1', '2026-09-09T19:03:00.000Z', 11);
|
||||||
|
assert.equal(repo.getStream('s1')?.peak_viewers, 17);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('offene Sitzung wird gefunden und geschlossen', () => {
|
||||||
|
const { repo } = freshDb();
|
||||||
|
repo.openStream('s1', '2026-09-09T19:00:00.000Z');
|
||||||
|
assert.equal(repo.latestOpenStream()?.id, 's1');
|
||||||
|
repo.closeStream('s1', '2026-09-09T22:00:00.000Z');
|
||||||
|
assert.equal(repo.latestOpenStream(), undefined);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('erneutes openStream ueberschreibt eine laufende Sitzung nicht', () => {
|
||||||
|
const { repo } = freshDb();
|
||||||
|
repo.openStream('s1', '2026-09-09T19:00:00.000Z');
|
||||||
|
repo.openStream('s1', '2026-09-09T21:00:00.000Z');
|
||||||
|
assert.equal(repo.getStream('s1')?.started_at, '2026-09-09T19:00:00.000Z');
|
||||||
|
});
|
||||||
|
|
||||||
|
test('Nachrichten ohne laufenden Stream gehen trotzdem nicht verloren', () => {
|
||||||
|
const { db, repo } = freshDb();
|
||||||
|
repo.recordMessage(msg(), null);
|
||||||
|
const row = db.prepare('SELECT stream_id FROM chat_messages').get() as { stream_id: string | null };
|
||||||
|
assert.equal(row.stream_id, null);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('unique_chatters zaehlt Personen, nicht Nachrichten', () => {
|
||||||
|
const { repo } = freshDb();
|
||||||
|
repo.openStream('s1', '2026-09-09T19:00:00.000Z');
|
||||||
|
repo.recordMessage(msg({ userId: '1', login: 'a' }), 's1');
|
||||||
|
repo.recordMessage(msg({ userId: '1', login: 'a' }), 's1');
|
||||||
|
repo.recordMessage(msg({ userId: '2', login: 'b' }), 's1');
|
||||||
|
assert.equal(repo.getStream('s1')?.unique_chatters, 2);
|
||||||
|
});
|
||||||
@@ -0,0 +1,125 @@
|
|||||||
|
// Alle SQL-Anweisungen des Projekts stehen hier. Kein anderer Ort schreibt
|
||||||
|
// SQL - das haelt das Schema an einer Stelle nachvollziehbar.
|
||||||
|
|
||||||
|
import type { Db } from './index.ts';
|
||||||
|
import type { ChatMessage, EventType, StreamEvent } from '#shared/events.ts';
|
||||||
|
import { ulid } from '#shared/events.ts';
|
||||||
|
|
||||||
|
export type StreamRow = {
|
||||||
|
id: string; started_at: string; ended_at: string | null;
|
||||||
|
peak_viewers: number; unique_chatters: number;
|
||||||
|
};
|
||||||
|
|
||||||
|
export function repos(db: Db) {
|
||||||
|
const stmt = {
|
||||||
|
insertEvent: db.prepare(
|
||||||
|
'INSERT OR IGNORE INTO events (id, ts, type, stream_id, user_id, payload) VALUES (?,?,?,?,?,?)'
|
||||||
|
),
|
||||||
|
upsertUser: db.prepare(`
|
||||||
|
INSERT INTO users (id, login, display_name, first_seen_at, last_seen_at, message_count)
|
||||||
|
VALUES (?,?,?,?,?,0)
|
||||||
|
ON CONFLICT(id) DO UPDATE SET
|
||||||
|
login = excluded.login,
|
||||||
|
display_name = excluded.display_name,
|
||||||
|
last_seen_at = excluded.last_seen_at
|
||||||
|
`),
|
||||||
|
bumpUserMessage: db.prepare(
|
||||||
|
'UPDATE users SET message_count = message_count + 1, last_seen_at = ? WHERE id = ?'
|
||||||
|
),
|
||||||
|
getUser: db.prepare('SELECT * FROM users WHERE id = ?'),
|
||||||
|
insertChat: db.prepare(`
|
||||||
|
INSERT OR IGNORE INTO chat_messages
|
||||||
|
(id, stream_id, ts, user_id, text, emotes, is_first, is_command, bits)
|
||||||
|
VALUES (?,?,?,?,?,?,?,?,?)
|
||||||
|
`),
|
||||||
|
openStream: db.prepare(
|
||||||
|
'INSERT OR IGNORE INTO streams (id, started_at) VALUES (?,?)'
|
||||||
|
),
|
||||||
|
closeStream: db.prepare('UPDATE streams SET ended_at = ? WHERE id = ? AND ended_at IS NULL'),
|
||||||
|
getStream: db.prepare('SELECT * FROM streams WHERE id = ?'),
|
||||||
|
latestOpenStream: db.prepare(
|
||||||
|
'SELECT * FROM streams WHERE ended_at IS NULL ORDER BY started_at DESC LIMIT 1'
|
||||||
|
),
|
||||||
|
sample: db.prepare(
|
||||||
|
'INSERT OR REPLACE INTO viewer_samples (stream_id, ts, viewers) VALUES (?,?,?)'
|
||||||
|
),
|
||||||
|
bumpPeak: db.prepare('UPDATE streams SET peak_viewers = ? WHERE id = ? AND peak_viewers < ?'),
|
||||||
|
touchUserStream: db.prepare(`
|
||||||
|
INSERT INTO user_streams (user_id, stream_id, first_ts, last_ts, messages)
|
||||||
|
VALUES (?,?,?,?,?)
|
||||||
|
ON CONFLICT(user_id, stream_id) DO UPDATE SET
|
||||||
|
last_ts = excluded.last_ts,
|
||||||
|
messages = user_streams.messages + excluded.messages
|
||||||
|
`),
|
||||||
|
countUserStreams: db.prepare(
|
||||||
|
'SELECT COUNT(*) AS n FROM user_streams WHERE user_id = ?'
|
||||||
|
),
|
||||||
|
recountChatters: db.prepare(`
|
||||||
|
UPDATE streams SET unique_chatters =
|
||||||
|
(SELECT COUNT(*) FROM user_streams WHERE stream_id = ? AND messages > 0)
|
||||||
|
WHERE id = ?
|
||||||
|
`),
|
||||||
|
recentChat: db.prepare(
|
||||||
|
'SELECT ts, text, user_id FROM chat_messages WHERE ts >= ? ORDER BY ts'
|
||||||
|
),
|
||||||
|
stats: db.prepare(`
|
||||||
|
SELECT
|
||||||
|
(SELECT COUNT(*) FROM chat_messages) AS messages,
|
||||||
|
(SELECT COUNT(*) FROM users) AS users,
|
||||||
|
(SELECT COUNT(*) FROM streams) AS streams,
|
||||||
|
(SELECT COUNT(*) FROM events) AS events
|
||||||
|
`),
|
||||||
|
};
|
||||||
|
|
||||||
|
return {
|
||||||
|
event(type: EventType, payload: unknown, streamId: string | null = null,
|
||||||
|
userId: string | null = null, ts = new Date().toISOString()): StreamEvent {
|
||||||
|
const ev: StreamEvent = { id: ulid(), ts, type, streamId, userId, payload };
|
||||||
|
stmt.insertEvent.run(ev.id, ev.ts, ev.type, ev.streamId, ev.userId, JSON.stringify(payload));
|
||||||
|
return ev;
|
||||||
|
},
|
||||||
|
|
||||||
|
/** Schreibt Nutzer, Nachricht und Anwesenheit in einer Transaktion. */
|
||||||
|
recordMessage(msg: ChatMessage, streamId: string | null): { streamsChatted: number } {
|
||||||
|
db.exec('BEGIN');
|
||||||
|
try {
|
||||||
|
stmt.upsertUser.run(msg.userId, msg.login, msg.displayName, msg.ts, msg.ts);
|
||||||
|
stmt.bumpUserMessage.run(msg.ts, msg.userId);
|
||||||
|
stmt.insertChat.run(
|
||||||
|
msg.id, streamId, msg.ts, msg.userId, msg.text,
|
||||||
|
JSON.stringify(msg.emotes), msg.isFirst ? 1 : 0, msg.isCommand ? 1 : 0, msg.bits
|
||||||
|
);
|
||||||
|
if (streamId) {
|
||||||
|
stmt.touchUserStream.run(msg.userId, streamId, msg.ts, msg.ts, 1);
|
||||||
|
stmt.recountChatters.run(streamId, streamId);
|
||||||
|
}
|
||||||
|
db.exec('COMMIT');
|
||||||
|
} catch (err) {
|
||||||
|
db.exec('ROLLBACK');
|
||||||
|
throw err;
|
||||||
|
}
|
||||||
|
const row = stmt.countUserStreams.get(msg.userId) as { n: number } | undefined;
|
||||||
|
return { streamsChatted: row?.n ?? 0 };
|
||||||
|
},
|
||||||
|
|
||||||
|
openStream(id: string, startedAt: string): void { stmt.openStream.run(id, startedAt); },
|
||||||
|
closeStream(id: string, endedAt: string): void { stmt.closeStream.run(endedAt, id); },
|
||||||
|
getStream(id: string) { return stmt.getStream.get(id) as StreamRow | undefined; },
|
||||||
|
latestOpenStream() { return stmt.latestOpenStream.get() as StreamRow | undefined; },
|
||||||
|
|
||||||
|
sample(streamId: string, ts: string, viewers: number): void {
|
||||||
|
stmt.sample.run(streamId, ts, viewers);
|
||||||
|
stmt.bumpPeak.run(viewers, streamId, viewers);
|
||||||
|
},
|
||||||
|
|
||||||
|
recentChat(sinceIso: string) {
|
||||||
|
return stmt.recentChat.all(sinceIso) as Array<{ ts: string; text: string; user_id: string }>;
|
||||||
|
},
|
||||||
|
|
||||||
|
stats() {
|
||||||
|
return stmt.stats.get() as { messages: number; users: number; streams: number; events: number };
|
||||||
|
},
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
export type Repos = ReturnType<typeof repos>;
|
||||||
@@ -1,5 +0,0 @@
|
|||||||
{
|
|
||||||
"extends": "../../tsconfig.base.json",
|
|
||||||
"compilerOptions": { "outDir": "dist", "rootDir": "src" },
|
|
||||||
"include": ["src"]
|
|
||||||
}
|
|
||||||
@@ -1,12 +0,0 @@
|
|||||||
{
|
|
||||||
"name": "@abdulive/shared",
|
|
||||||
"version": "0.1.0",
|
|
||||||
"private": true,
|
|
||||||
"type": "module",
|
|
||||||
"main": "dist/index.js",
|
|
||||||
"scripts": {
|
|
||||||
"build": "tsc -p tsconfig.json",
|
|
||||||
"typecheck": "tsc -p tsconfig.json --noEmit",
|
|
||||||
"test": "node --test"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -0,0 +1,66 @@
|
|||||||
|
// Einheitliches Ereignisformat. Alles, was aus IRC, EventSub, Helix oder OBS
|
||||||
|
// hereinkommt, wird darauf normalisiert und in `events` geschrieben.
|
||||||
|
// Siehe docs/03-data-model.md.
|
||||||
|
|
||||||
|
export type EventType =
|
||||||
|
| 'stream.online' | 'stream.offline' | 'stream.segment_changed'
|
||||||
|
| 'chat.message' | 'chat.first_message' | 'chat.spike'
|
||||||
|
| 'user.arrived' | 'user.returning'
|
||||||
|
| 'follow' | 'subscribe' | 'subgift' | 'cheer'
|
||||||
|
| 'raid.in' | 'raid.out'
|
||||||
|
| 'points.redeemed' | 'shop.redeemed'
|
||||||
|
| 'poll.begin' | 'poll.end'
|
||||||
|
| 'obs.scene_changed' | 'obs.replay_saved'
|
||||||
|
| 'clip.requested' | 'clip.created' | 'clip.rendered';
|
||||||
|
|
||||||
|
export type StreamEvent = {
|
||||||
|
id: string;
|
||||||
|
ts: string; // ISO-8601, UTC
|
||||||
|
type: EventType;
|
||||||
|
streamId: string | null;
|
||||||
|
userId: string | null;
|
||||||
|
payload: unknown;
|
||||||
|
};
|
||||||
|
|
||||||
|
export type ChatMessage = {
|
||||||
|
id: string; // msg-id aus IRC
|
||||||
|
ts: string;
|
||||||
|
userId: string;
|
||||||
|
login: string;
|
||||||
|
displayName: string;
|
||||||
|
text: string;
|
||||||
|
emotes: Array<{ id: string; name: string; count: number }>;
|
||||||
|
isFirst: boolean; // first-msg Tag
|
||||||
|
isReturning: boolean; // returning-chatter Tag
|
||||||
|
isCommand: boolean;
|
||||||
|
isMod: boolean;
|
||||||
|
isSub: boolean;
|
||||||
|
isBroadcaster: boolean;
|
||||||
|
bits: number;
|
||||||
|
color: string | null;
|
||||||
|
};
|
||||||
|
|
||||||
|
export type SpikeEvent = {
|
||||||
|
score: number;
|
||||||
|
rate: number;
|
||||||
|
baseline: number;
|
||||||
|
chatters: number;
|
||||||
|
emoteHits: number;
|
||||||
|
clipMentions: number;
|
||||||
|
windowFrom: string;
|
||||||
|
windowTo: string;
|
||||||
|
overThreshold: boolean;
|
||||||
|
};
|
||||||
|
|
||||||
|
/** ULID-artige, zeitlich sortierbare ID ohne Abhaengigkeit. */
|
||||||
|
const B32 = '0123456789ABCDEFGHJKMNPQRSTVWXYZ';
|
||||||
|
export function ulid(now = Date.now()): string {
|
||||||
|
let time = '';
|
||||||
|
let t = now;
|
||||||
|
for (let i = 0; i < 10; i++) { time = B32[t % 32]! + time; t = Math.floor(t / 32); }
|
||||||
|
const rnd = new Uint8Array(16);
|
||||||
|
crypto.getRandomValues(rnd);
|
||||||
|
let tail = '';
|
||||||
|
for (const b of rnd) tail += B32[b % 32]!;
|
||||||
|
return time + tail;
|
||||||
|
}
|
||||||
@@ -1,2 +0,0 @@
|
|||||||
// @abdulive/shared — siehe docs/01-architecture.md
|
|
||||||
export {};
|
|
||||||
@@ -1,5 +0,0 @@
|
|||||||
{
|
|
||||||
"extends": "../../tsconfig.base.json",
|
|
||||||
"compilerOptions": { "outDir": "dist", "rootDir": "src" },
|
|
||||||
"include": ["src"]
|
|
||||||
}
|
|
||||||
@@ -1,12 +0,0 @@
|
|||||||
{
|
|
||||||
"name": "@abdulive/twitch",
|
|
||||||
"version": "0.1.0",
|
|
||||||
"private": true,
|
|
||||||
"type": "module",
|
|
||||||
"main": "dist/index.js",
|
|
||||||
"scripts": {
|
|
||||||
"build": "tsc -p tsconfig.json",
|
|
||||||
"typecheck": "tsc -p tsconfig.json --noEmit",
|
|
||||||
"test": "node --test"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -0,0 +1,118 @@
|
|||||||
|
// Helix-Client mit eigenem Token-Bucket. Twitch erlaubt 800 Punkte pro Minute
|
||||||
|
// je Client-ID; wir drosseln selbst, statt uns auf 429 zu verlassen, und
|
||||||
|
// werten `Ratelimit-Remaining` aus dem Antwortkopf aus.
|
||||||
|
|
||||||
|
import { logger } from '#core/logger.ts';
|
||||||
|
import { retry, sleep } from '#core/retry.ts';
|
||||||
|
import { accessToken } from './oauth.ts';
|
||||||
|
import { readSecrets, type Role } from './secrets.ts';
|
||||||
|
|
||||||
|
const log = logger('helix');
|
||||||
|
const BASE = 'https://api.twitch.tv/helix';
|
||||||
|
|
||||||
|
class Bucket {
|
||||||
|
// Keine Parameter-Properties: Node fuehrt die .ts-Datei direkt aus und
|
||||||
|
// entfernt nur Typen; `constructor(private x)` waere zur Laufzeit weg.
|
||||||
|
private tokens: number;
|
||||||
|
private last = Date.now();
|
||||||
|
private capacity: number;
|
||||||
|
private perMinute: number;
|
||||||
|
|
||||||
|
constructor(capacity = 700, perMinute = 700) {
|
||||||
|
this.capacity = capacity;
|
||||||
|
this.perMinute = perMinute;
|
||||||
|
this.tokens = capacity;
|
||||||
|
}
|
||||||
|
|
||||||
|
async take(cost = 1): Promise<void> {
|
||||||
|
for (;;) {
|
||||||
|
const now = Date.now();
|
||||||
|
this.tokens = Math.min(this.capacity, this.tokens + ((now - this.last) / 60_000) * this.perMinute);
|
||||||
|
this.last = now;
|
||||||
|
if (this.tokens >= cost) { this.tokens -= cost; return; }
|
||||||
|
await sleep(Math.ceil(((cost - this.tokens) / this.perMinute) * 60_000) + 50);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Uebernimmt den echten Stand aus dem Antwortkopf. */
|
||||||
|
sync(remaining: number | null): void {
|
||||||
|
if (remaining !== null && remaining < this.tokens) this.tokens = remaining;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
const bucket = new Bucket();
|
||||||
|
|
||||||
|
export type HelixStream = {
|
||||||
|
id: string; user_id: string; user_login: string; game_id: string; game_name: string;
|
||||||
|
title: string; viewer_count: number; started_at: string; language: string; thumbnail_url: string;
|
||||||
|
};
|
||||||
|
|
||||||
|
export type HelixUser = { id: string; login: string; display_name: string };
|
||||||
|
|
||||||
|
async function call<T>(role: Role, endpoint: string, init: RequestInit = {}): Promise<T> {
|
||||||
|
const app = readSecrets().apps[role];
|
||||||
|
if (!app?.clientId) throw new Error(`Fuer "${role}" ist keine Client-ID hinterlegt.`);
|
||||||
|
|
||||||
|
return await retry(async () => {
|
||||||
|
await bucket.take();
|
||||||
|
const token = await accessToken(role);
|
||||||
|
const res = await fetch(`${BASE}${endpoint}`, {
|
||||||
|
...init,
|
||||||
|
headers: {
|
||||||
|
'Client-Id': app.clientId,
|
||||||
|
Authorization: `Bearer ${token}`,
|
||||||
|
...(init.body ? { 'content-type': 'application/json' } : {}),
|
||||||
|
...init.headers,
|
||||||
|
},
|
||||||
|
});
|
||||||
|
|
||||||
|
const remaining = res.headers.get('ratelimit-remaining');
|
||||||
|
bucket.sync(remaining === null ? null : Number(remaining));
|
||||||
|
|
||||||
|
if (res.status === 429) {
|
||||||
|
const reset = Number(res.headers.get('ratelimit-reset') || 0) * 1000 - Date.now();
|
||||||
|
await sleep(Math.max(1000, Math.min(60_000, reset)));
|
||||||
|
throw new Error('Rate Limit erreicht');
|
||||||
|
}
|
||||||
|
if (!res.ok) {
|
||||||
|
const text = await res.text();
|
||||||
|
throw new Error(`Helix ${endpoint}: HTTP ${res.status} ${text.slice(0, 200)}`);
|
||||||
|
}
|
||||||
|
if (res.status === 204) return undefined as T;
|
||||||
|
return await res.json() as T;
|
||||||
|
}, { attempts: 4, onRetry: (err, n, ms) => log.warn(`Wiederhole ${endpoint} (${n}) in ${ms} ms`, { err: String(err) }) });
|
||||||
|
}
|
||||||
|
|
||||||
|
export const helix = {
|
||||||
|
async userByLogin(login: string): Promise<HelixUser | null> {
|
||||||
|
const r = await call<{ data: HelixUser[] }>('broadcaster', `/users?login=${encodeURIComponent(login)}`);
|
||||||
|
return r.data[0] ?? null;
|
||||||
|
},
|
||||||
|
|
||||||
|
async streamByUserId(userId: string): Promise<HelixStream | null> {
|
||||||
|
const r = await call<{ data: HelixStream[] }>('broadcaster', `/streams?user_id=${userId}`);
|
||||||
|
return r.data[0] ?? null;
|
||||||
|
},
|
||||||
|
|
||||||
|
async chatters(broadcasterId: string, moderatorId: string): Promise<string[]> {
|
||||||
|
const out: string[] = [];
|
||||||
|
let cursor = '';
|
||||||
|
do {
|
||||||
|
const q = `/chat/chatters?broadcaster_id=${broadcasterId}&moderator_id=${moderatorId}&first=1000` +
|
||||||
|
(cursor ? `&after=${cursor}` : '');
|
||||||
|
const r = await call<{ data: Array<{ user_id: string }>; pagination?: { cursor?: string } }>(
|
||||||
|
'broadcaster', q);
|
||||||
|
for (const c of r.data) out.push(c.user_id);
|
||||||
|
cursor = r.pagination?.cursor ?? '';
|
||||||
|
} while (cursor);
|
||||||
|
return out;
|
||||||
|
},
|
||||||
|
|
||||||
|
async createClip(broadcasterId: string): Promise<{ id: string; edit_url: string }> {
|
||||||
|
const r = await call<{ data: Array<{ id: string; edit_url: string }> }>(
|
||||||
|
'broadcaster', `/clips?broadcaster_id=${broadcasterId}`, { method: 'POST' });
|
||||||
|
const clip = r.data[0];
|
||||||
|
if (!clip) throw new Error('Twitch hat keinen Clip zurueckgegeben');
|
||||||
|
return clip;
|
||||||
|
},
|
||||||
|
};
|
||||||
@@ -1,2 +0,0 @@
|
|||||||
// @abdulive/twitch — siehe docs/02-twitch-integration.md
|
|
||||||
export {};
|
|
||||||
@@ -0,0 +1,59 @@
|
|||||||
|
import { test } from 'node:test';
|
||||||
|
import assert from 'node:assert/strict';
|
||||||
|
import { parseLine, toChatMessage } from './irc.ts';
|
||||||
|
|
||||||
|
const LINE =
|
||||||
|
'@badge-info=subscriber/8;badges=broadcaster/1,subscriber/6;color=#1E90FF;' +
|
||||||
|
'display-name=Nic_OOo;emotes=25:0-4/1902:6-10;first-msg=0;id=abc-123;mod=0;' +
|
||||||
|
'returning-chatter=0;subscriber=1;tmi-sent-ts=1757404800000;user-id=98765 ' +
|
||||||
|
':nic_ooo!nic_ooo@nic_ooo.tmi.twitch.tv PRIVMSG #abdulive_ :Kappa Keepo mehr Rost als Farbe';
|
||||||
|
|
||||||
|
test('zerlegt eine PRIVMSG mit Tags', () => {
|
||||||
|
const m = parseLine(LINE);
|
||||||
|
assert.ok(m);
|
||||||
|
assert.equal(m.command, 'PRIVMSG');
|
||||||
|
assert.equal(m.params[0], '#abdulive_');
|
||||||
|
assert.equal(m.params[1], 'Kappa Keepo mehr Rost als Farbe');
|
||||||
|
assert.equal(m.tags['user-id'], '98765');
|
||||||
|
assert.equal(m.tags['display-name'], 'Nic_OOo');
|
||||||
|
});
|
||||||
|
|
||||||
|
test('entschluesselt escapte Tag-Werte', () => {
|
||||||
|
const m = parseLine('@system-msg=Ein\\sTest\\szur\\sSache;id=x :tmi.twitch.tv USERNOTICE #k');
|
||||||
|
assert.equal(m?.tags['system-msg'], 'Ein Test zur Sache');
|
||||||
|
});
|
||||||
|
|
||||||
|
test('uebersetzt in eine ChatMessage', () => {
|
||||||
|
const msg = toChatMessage(parseLine(LINE)!);
|
||||||
|
assert.ok(msg);
|
||||||
|
assert.equal(msg.userId, '98765');
|
||||||
|
assert.equal(msg.login, 'nic_ooo');
|
||||||
|
assert.equal(msg.displayName, 'Nic_OOo');
|
||||||
|
assert.equal(msg.isBroadcaster, true);
|
||||||
|
assert.equal(msg.isSub, true);
|
||||||
|
assert.equal(msg.isFirst, false);
|
||||||
|
assert.equal(msg.isCommand, false);
|
||||||
|
assert.equal(msg.ts, new Date(1757404800000).toISOString());
|
||||||
|
// Emote-Namen werden aus den Positionsangaben zurueckgelesen
|
||||||
|
assert.deepEqual(msg.emotes.map((e) => e.name).sort(), ['Kappa', 'Keepo']);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('erkennt Befehle und Erstchatter', () => {
|
||||||
|
const line =
|
||||||
|
'@first-msg=1;id=y;tmi-sent-ts=1757404800000;user-id=42;display-name=Neu ' +
|
||||||
|
':neu!neu@neu.tmi.twitch.tv PRIVMSG #abdulive_ :!clip';
|
||||||
|
const msg = toChatMessage(parseLine(line)!)!;
|
||||||
|
assert.equal(msg.isFirst, true);
|
||||||
|
assert.equal(msg.isCommand, true);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('ignoriert Nachrichten ohne user-id', () => {
|
||||||
|
const line = ':tmi.twitch.tv PRIVMSG #abdulive_ :ohne Tags';
|
||||||
|
assert.equal(toChatMessage(parseLine(line)!), null);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('PING wird als solches erkannt', () => {
|
||||||
|
const m = parseLine('PING :tmi.twitch.tv');
|
||||||
|
assert.equal(m?.command, 'PING');
|
||||||
|
assert.equal(m?.params[0], 'tmi.twitch.tv');
|
||||||
|
});
|
||||||
@@ -0,0 +1,202 @@
|
|||||||
|
// Twitch-Chat ueber IRC (WebSocket). Warum IRC und nicht EventSub: die
|
||||||
|
// IRCv3-Tags enthalten first-msg, returning-chatter, msg-id, badges und
|
||||||
|
// Emote-Positionen - genau das, woran Community-Gedaechtnis und
|
||||||
|
// Spike-Erkennung haengen. Siehe ADR-0003.
|
||||||
|
//
|
||||||
|
// Ohne Zugangsdaten wird anonym verbunden (justinfan). Dann kann man mitlesen,
|
||||||
|
// aber nicht schreiben - reicht, um die Datenaufnahme zu pruefen, bevor die
|
||||||
|
// Twitch-Apps eingerichtet sind.
|
||||||
|
|
||||||
|
import { logger } from '#core/logger.ts';
|
||||||
|
import { forever } from '#core/retry.ts';
|
||||||
|
import type { ChatMessage } from '#shared/events.ts';
|
||||||
|
import { accessToken } from './oauth.ts';
|
||||||
|
import { readSecrets } from './secrets.ts';
|
||||||
|
|
||||||
|
const log = logger('irc');
|
||||||
|
const URL_WS = 'wss://irc-ws.chat.twitch.tv:443';
|
||||||
|
|
||||||
|
export type IrcMessage = { tags: Record<string, string>; prefix: string; command: string; params: string[] };
|
||||||
|
|
||||||
|
/** IRCv3-Zeile zerlegen: `@tags :prefix COMMAND params :trailing` */
|
||||||
|
export function parseLine(line: string): IrcMessage | null {
|
||||||
|
if (!line) return null;
|
||||||
|
let rest = line;
|
||||||
|
const tags: Record<string, string> = {};
|
||||||
|
|
||||||
|
if (rest.startsWith('@')) {
|
||||||
|
const sp = rest.indexOf(' ');
|
||||||
|
for (const pair of rest.slice(1, sp).split(';')) {
|
||||||
|
const eq = pair.indexOf('=');
|
||||||
|
const k = eq === -1 ? pair : pair.slice(0, eq);
|
||||||
|
const v = eq === -1 ? '' : pair.slice(eq + 1);
|
||||||
|
tags[k] = v
|
||||||
|
.replace(/\\s/g, ' ').replace(/\\n/g, '\n').replace(/\\r/g, '\r')
|
||||||
|
.replace(/\\:/g, ';').replace(/\\\\/g, '\\');
|
||||||
|
}
|
||||||
|
rest = rest.slice(sp + 1);
|
||||||
|
}
|
||||||
|
|
||||||
|
let prefix = '';
|
||||||
|
if (rest.startsWith(':')) {
|
||||||
|
const sp = rest.indexOf(' ');
|
||||||
|
prefix = rest.slice(1, sp);
|
||||||
|
rest = rest.slice(sp + 1);
|
||||||
|
}
|
||||||
|
|
||||||
|
const trailingAt = rest.indexOf(' :');
|
||||||
|
const head = trailingAt === -1 ? rest : rest.slice(0, trailingAt);
|
||||||
|
const parts = head.split(' ').filter(Boolean);
|
||||||
|
const command = parts.shift() ?? '';
|
||||||
|
if (trailingAt !== -1) parts.push(rest.slice(trailingAt + 2));
|
||||||
|
return { tags, prefix, command, params: parts };
|
||||||
|
}
|
||||||
|
|
||||||
|
function parseEmotes(tag: string | undefined, text: string): ChatMessage['emotes'] {
|
||||||
|
if (!tag) return [];
|
||||||
|
const out: ChatMessage['emotes'] = [];
|
||||||
|
const chars = [...text];
|
||||||
|
for (const part of tag.split('/')) {
|
||||||
|
const [id, ranges] = part.split(':');
|
||||||
|
if (!id || !ranges) continue;
|
||||||
|
const spans = ranges.split(',');
|
||||||
|
const first = spans[0]?.split('-');
|
||||||
|
const name = first && first[0] !== undefined && first[1] !== undefined
|
||||||
|
? chars.slice(Number(first[0]), Number(first[1]) + 1).join('')
|
||||||
|
: id;
|
||||||
|
out.push({ id, name, count: spans.length });
|
||||||
|
}
|
||||||
|
return out;
|
||||||
|
}
|
||||||
|
|
||||||
|
export function toChatMessage(m: IrcMessage): ChatMessage | null {
|
||||||
|
if (m.command !== 'PRIVMSG') return null;
|
||||||
|
const text = m.params[1] ?? '';
|
||||||
|
const login = m.prefix.split('!')[0] ?? '';
|
||||||
|
const userId = m.tags['user-id'];
|
||||||
|
if (!userId) return null;
|
||||||
|
const badges = m.tags['badges'] ?? '';
|
||||||
|
return {
|
||||||
|
id: m.tags['id'] ?? `${userId}-${Date.now()}`,
|
||||||
|
ts: new Date(Number(m.tags['tmi-sent-ts'] ?? Date.now())).toISOString(),
|
||||||
|
userId,
|
||||||
|
login,
|
||||||
|
displayName: m.tags['display-name'] || login,
|
||||||
|
text,
|
||||||
|
emotes: parseEmotes(m.tags['emotes'], text),
|
||||||
|
isFirst: m.tags['first-msg'] === '1',
|
||||||
|
isReturning: m.tags['returning-chatter'] === '1',
|
||||||
|
isCommand: text.startsWith('!'),
|
||||||
|
isMod: m.tags['mod'] === '1' || badges.includes('broadcaster'),
|
||||||
|
isSub: m.tags['subscriber'] === '1',
|
||||||
|
isBroadcaster: badges.includes('broadcaster'),
|
||||||
|
bits: Number(m.tags['bits'] ?? 0),
|
||||||
|
color: m.tags['color'] || null,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
export type IrcHandlers = {
|
||||||
|
onMessage: (msg: ChatMessage) => void;
|
||||||
|
onNotice?: (text: string) => void;
|
||||||
|
onConnected?: (anonymous: boolean) => void;
|
||||||
|
};
|
||||||
|
|
||||||
|
export type IrcClient = {
|
||||||
|
/** Haelt die Verbindung dauerhaft; kehrt erst bei `signal.abort` zurueck. */
|
||||||
|
run: (signal: AbortSignal) => Promise<void>;
|
||||||
|
/** Sendet in den Chat. Anonym verbundene Clients koennen nicht senden. */
|
||||||
|
say: (text: string) => boolean;
|
||||||
|
isAnonymous: () => boolean;
|
||||||
|
};
|
||||||
|
|
||||||
|
export function createIrc(channel: string, handlers: IrcHandlers): IrcClient {
|
||||||
|
let send: ((text: string) => boolean) | null = null;
|
||||||
|
let anonymous = true;
|
||||||
|
|
||||||
|
return {
|
||||||
|
run: (signal) => forever(
|
||||||
|
async (sig) => {
|
||||||
|
const conn = await connectOnce(channel, handlers, sig, (fn, anon) => {
|
||||||
|
send = fn; anonymous = anon;
|
||||||
|
});
|
||||||
|
void conn;
|
||||||
|
},
|
||||||
|
{
|
||||||
|
signal,
|
||||||
|
onWait: (ms, n) =>
|
||||||
|
log.warn(`Chat getrennt, neuer Versuch in ${Math.round(ms / 1000)} s (${n}.)`),
|
||||||
|
}
|
||||||
|
),
|
||||||
|
say: (text) => (send ? send(text) : false),
|
||||||
|
isAnonymous: () => anonymous,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
function connectOnce(
|
||||||
|
channel: string,
|
||||||
|
handlers: IrcHandlers,
|
||||||
|
signal: AbortSignal,
|
||||||
|
ready: (say: (text: string) => boolean, anonymous: boolean) => void
|
||||||
|
): Promise<void> {
|
||||||
|
return new Promise<void>((_resolve, reject) => {
|
||||||
|
const ws = new WebSocket(URL_WS);
|
||||||
|
let anonymous = true;
|
||||||
|
let joined = false;
|
||||||
|
let alive: ReturnType<typeof setInterval> | null = null;
|
||||||
|
|
||||||
|
const shutdown = (why: string) => {
|
||||||
|
if (alive) clearInterval(alive);
|
||||||
|
try { ws.close(); } catch { /* schon zu */ }
|
||||||
|
reject(new Error(why));
|
||||||
|
};
|
||||||
|
signal.addEventListener('abort', () => shutdown('abgebrochen'), { once: true });
|
||||||
|
|
||||||
|
ws.addEventListener('open', () => {
|
||||||
|
void (async () => {
|
||||||
|
ws.send('CAP REQ :twitch.tv/tags twitch.tv/commands');
|
||||||
|
const bot = readSecrets().tokens.bot;
|
||||||
|
if (bot) {
|
||||||
|
try {
|
||||||
|
ws.send(`PASS oauth:${await accessToken('bot')}`);
|
||||||
|
ws.send(`NICK ${bot.login}`);
|
||||||
|
anonymous = false;
|
||||||
|
} catch (err) {
|
||||||
|
log.warn(`Bot-Token nicht nutzbar, lese anonym mit: ${(err as Error).message}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if (anonymous) ws.send(`NICK justinfan${10_000 + Math.floor(Math.random() * 80_000)}`);
|
||||||
|
ws.send(`JOIN #${channel.toLowerCase()}`);
|
||||||
|
})();
|
||||||
|
});
|
||||||
|
|
||||||
|
ws.addEventListener('message', (ev) => {
|
||||||
|
for (const line of String(ev.data).split('\r\n')) {
|
||||||
|
const m = parseLine(line);
|
||||||
|
if (!m) continue;
|
||||||
|
|
||||||
|
if (m.command === 'PING') { ws.send(`PONG :${m.params[0] ?? 'tmi.twitch.tv'}`); continue; }
|
||||||
|
if (m.command === 'JOIN' && !joined) {
|
||||||
|
joined = true;
|
||||||
|
log.info(`Chat verbunden mit #${channel}${anonymous ? ' (anonym, nur lesend)' : ''}`);
|
||||||
|
ready((text) => {
|
||||||
|
if (anonymous || ws.readyState !== WebSocket.OPEN) return false;
|
||||||
|
ws.send(`PRIVMSG #${channel.toLowerCase()} :${text}`);
|
||||||
|
return true;
|
||||||
|
}, anonymous);
|
||||||
|
handlers.onConnected?.(anonymous);
|
||||||
|
// Twitch schliesst stille Verbindungen; alle 4 Minuten ein Lebenszeichen
|
||||||
|
alive = setInterval(() => { try { ws.send('PING :keepalive'); } catch { /* egal */ } }, 240_000);
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
if (m.command === 'NOTICE') { handlers.onNotice?.(m.params[1] ?? ''); continue; }
|
||||||
|
if (m.command === 'PRIVMSG') {
|
||||||
|
const msg = toChatMessage(m);
|
||||||
|
if (msg) handlers.onMessage(msg);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
ws.addEventListener('error', () => shutdown('WebSocket-Fehler'));
|
||||||
|
ws.addEventListener('close', (ev) => shutdown(`Verbindung geschlossen (${ev.code})`));
|
||||||
|
});
|
||||||
|
}
|
||||||
@@ -0,0 +1,130 @@
|
|||||||
|
// OAuth-Autorisierungscode-Flow gegen Twitch. Rueckruf laeuft auf denselben
|
||||||
|
// Server wie die Einrichtungsseite (Port 4792) - siehe ADR-0005.
|
||||||
|
|
||||||
|
import crypto from 'node:crypto';
|
||||||
|
import { logger } from '#core/logger.ts';
|
||||||
|
import { readSecrets, updateSecrets, type Role, type TokenSet } from './secrets.ts';
|
||||||
|
|
||||||
|
const log = logger('oauth');
|
||||||
|
const AUTH = 'https://id.twitch.tv/oauth2/authorize';
|
||||||
|
const TOKEN = 'https://id.twitch.tv/oauth2/token';
|
||||||
|
const VALIDATE = 'https://id.twitch.tv/oauth2/validate';
|
||||||
|
|
||||||
|
export const SCOPES: Record<Role, string[]> = {
|
||||||
|
broadcaster: [
|
||||||
|
'clips:edit',
|
||||||
|
'channel:manage:broadcast',
|
||||||
|
'channel:read:subscriptions',
|
||||||
|
'moderator:read:followers',
|
||||||
|
'moderator:read:chatters',
|
||||||
|
'channel:manage:raids',
|
||||||
|
'channel:read:redemptions',
|
||||||
|
'bits:read',
|
||||||
|
],
|
||||||
|
bot: ['chat:read', 'chat:edit'],
|
||||||
|
};
|
||||||
|
|
||||||
|
/** Kurzlebige, einmalig einloesbare `state`-Werte gegen CSRF. */
|
||||||
|
const pending = new Map<string, { role: Role; created: number }>();
|
||||||
|
|
||||||
|
export function redirectUri(port: number): string {
|
||||||
|
// Muss exakt so in der Twitch-App hinterlegt sein.
|
||||||
|
return `http://localhost:${port}/callback`;
|
||||||
|
}
|
||||||
|
|
||||||
|
export function authorizeUrl(role: Role, port: number): string {
|
||||||
|
const app = readSecrets().apps[role];
|
||||||
|
if (!app?.clientId) throw new Error(`Fuer "${role}" ist noch keine Client-ID hinterlegt.`);
|
||||||
|
const state = crypto.randomBytes(16).toString('hex');
|
||||||
|
pending.set(state, { role, created: Date.now() });
|
||||||
|
for (const [k, v] of pending) if (Date.now() - v.created > 600_000) pending.delete(k);
|
||||||
|
|
||||||
|
const u = new URL(AUTH);
|
||||||
|
u.searchParams.set('client_id', app.clientId);
|
||||||
|
u.searchParams.set('redirect_uri', redirectUri(port));
|
||||||
|
u.searchParams.set('response_type', 'code');
|
||||||
|
u.searchParams.set('scope', SCOPES[role].join(' '));
|
||||||
|
u.searchParams.set('state', state);
|
||||||
|
u.searchParams.set('force_verify', 'true');
|
||||||
|
return u.toString();
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function handleCallback(code: string, state: string, port: number): Promise<Role> {
|
||||||
|
const entry = pending.get(state);
|
||||||
|
if (!entry) throw new Error('Unbekannter oder abgelaufener state - bitte erneut verbinden.');
|
||||||
|
pending.delete(state);
|
||||||
|
const { role } = entry;
|
||||||
|
|
||||||
|
const app = readSecrets().apps[role];
|
||||||
|
if (!app) throw new Error(`Fuer "${role}" fehlen die App-Daten.`);
|
||||||
|
|
||||||
|
const res = await fetch(TOKEN, {
|
||||||
|
method: 'POST',
|
||||||
|
headers: { 'content-type': 'application/x-www-form-urlencoded' },
|
||||||
|
body: new URLSearchParams({
|
||||||
|
client_id: app.clientId,
|
||||||
|
client_secret: app.clientSecret,
|
||||||
|
code,
|
||||||
|
grant_type: 'authorization_code',
|
||||||
|
redirect_uri: redirectUri(port),
|
||||||
|
}),
|
||||||
|
});
|
||||||
|
if (!res.ok) throw new Error(`Token-Tausch fehlgeschlagen: HTTP ${res.status} ${await res.text()}`);
|
||||||
|
const body = await res.json() as { access_token: string; refresh_token: string; expires_in: number };
|
||||||
|
|
||||||
|
const who = await validate(body.access_token);
|
||||||
|
const tokens: TokenSet = {
|
||||||
|
accessToken: body.access_token,
|
||||||
|
refreshToken: body.refresh_token,
|
||||||
|
expiresAt: new Date(Date.now() + body.expires_in * 1000).toISOString(),
|
||||||
|
scopes: who.scopes,
|
||||||
|
userId: who.user_id,
|
||||||
|
login: who.login,
|
||||||
|
};
|
||||||
|
updateSecrets((s) => { s.tokens[role] = tokens; });
|
||||||
|
log.info(`${role} verbunden als ${who.login} (${who.scopes.length} Scopes)`);
|
||||||
|
return role;
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function validate(accessToken: string) {
|
||||||
|
const res = await fetch(VALIDATE, { headers: { Authorization: `OAuth ${accessToken}` } });
|
||||||
|
if (!res.ok) throw new Error(`Token ungueltig: HTTP ${res.status}`);
|
||||||
|
return await res.json() as { user_id: string; login: string; scopes: string[]; expires_in: number };
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Liefert ein gueltiges Access-Token, erneuert bei Bedarf. */
|
||||||
|
export async function accessToken(role: Role): Promise<string> {
|
||||||
|
const s = readSecrets();
|
||||||
|
const t = s.tokens[role];
|
||||||
|
if (!t) throw new Error(`"${role}" ist nicht verbunden - Einrichtung unter /setup.`);
|
||||||
|
|
||||||
|
// Fuenf Minuten Puffer, damit kein Aufruf mitten im Ablauf landet
|
||||||
|
if (Date.parse(t.expiresAt) - Date.now() > 300_000) return t.accessToken;
|
||||||
|
|
||||||
|
const app = s.apps[role];
|
||||||
|
if (!app) throw new Error(`Fuer "${role}" fehlen die App-Daten.`);
|
||||||
|
const res = await fetch(TOKEN, {
|
||||||
|
method: 'POST',
|
||||||
|
headers: { 'content-type': 'application/x-www-form-urlencoded' },
|
||||||
|
body: new URLSearchParams({
|
||||||
|
client_id: app.clientId,
|
||||||
|
client_secret: app.clientSecret,
|
||||||
|
grant_type: 'refresh_token',
|
||||||
|
refresh_token: t.refreshToken,
|
||||||
|
}),
|
||||||
|
});
|
||||||
|
if (!res.ok) {
|
||||||
|
throw new Error(`Token-Erneuerung fuer "${role}" fehlgeschlagen (HTTP ${res.status}). ` +
|
||||||
|
'Neu verbinden unter /setup.');
|
||||||
|
}
|
||||||
|
const body = await res.json() as { access_token: string; refresh_token: string; expires_in: number };
|
||||||
|
const next: TokenSet = {
|
||||||
|
...t,
|
||||||
|
accessToken: body.access_token,
|
||||||
|
refreshToken: body.refresh_token || t.refreshToken,
|
||||||
|
expiresAt: new Date(Date.now() + body.expires_in * 1000).toISOString(),
|
||||||
|
};
|
||||||
|
updateSecrets((st) => { st.tokens[role] = next; });
|
||||||
|
log.info(`Token fuer ${role} erneuert`);
|
||||||
|
return next.accessToken;
|
||||||
|
}
|
||||||
@@ -0,0 +1,122 @@
|
|||||||
|
// Verschluesselter Speicher fuer Zugangsdaten. Siehe ADR-0005.
|
||||||
|
//
|
||||||
|
// Bewusst NICHT in der SQLite-Datenbank: die wird per VACUUM INTO auf die
|
||||||
|
// Windows-Seite gesichert, wo sie in einer Cloud-Synchronisation landen kann.
|
||||||
|
//
|
||||||
|
// Der Schluessel liegt im selben Volume wie die Daten. Das schuetzt nicht
|
||||||
|
// gegen jemanden mit Zugriff auf das Volume, und tut auch nicht so. Es
|
||||||
|
// schuetzt gegen versehentliche Anzeige im Stream, gegen verirrte Kopien und
|
||||||
|
// gegen Grep-Unfaelle - und genau darum geht es hier.
|
||||||
|
|
||||||
|
import crypto from 'node:crypto';
|
||||||
|
import fs from 'node:fs';
|
||||||
|
import path from 'node:path';
|
||||||
|
import { paths } from '#core/config.ts';
|
||||||
|
import { logger } from '#core/logger.ts';
|
||||||
|
|
||||||
|
const log = logger('secrets');
|
||||||
|
const FILE = path.join(paths.db, 'secrets.json');
|
||||||
|
const KEYFILE = path.join(paths.db, '.secret-key');
|
||||||
|
|
||||||
|
export type Role = 'broadcaster' | 'bot';
|
||||||
|
|
||||||
|
export type TokenSet = {
|
||||||
|
accessToken: string;
|
||||||
|
refreshToken: string;
|
||||||
|
expiresAt: string; // ISO
|
||||||
|
scopes: string[];
|
||||||
|
userId: string;
|
||||||
|
login: string;
|
||||||
|
};
|
||||||
|
|
||||||
|
export type SecretStore = {
|
||||||
|
apps: Partial<Record<Role, { clientId: string; clientSecret: string }>>;
|
||||||
|
tokens: Partial<Record<Role, TokenSet>>;
|
||||||
|
obs?: { password: string };
|
||||||
|
discord?: { goLiveWebhook?: string; recapWebhook?: string };
|
||||||
|
bluesky?: { handle: string; appPassword: string };
|
||||||
|
};
|
||||||
|
|
||||||
|
const EMPTY: SecretStore = { apps: {}, tokens: {} };
|
||||||
|
|
||||||
|
function key(): Buffer {
|
||||||
|
fs.mkdirSync(paths.db, { recursive: true });
|
||||||
|
if (fs.existsSync(KEYFILE)) return Buffer.from(fs.readFileSync(KEYFILE, 'utf8').trim(), 'base64');
|
||||||
|
const k = crypto.randomBytes(32);
|
||||||
|
fs.writeFileSync(KEYFILE, k.toString('base64'), { mode: 0o600 });
|
||||||
|
log.info('Neuer Verschluesselungsschluessel erzeugt');
|
||||||
|
return k;
|
||||||
|
}
|
||||||
|
|
||||||
|
function encrypt(plain: string): string {
|
||||||
|
const iv = crypto.randomBytes(12);
|
||||||
|
const cipher = crypto.createCipheriv('aes-256-gcm', key(), iv);
|
||||||
|
const body = Buffer.concat([cipher.update(plain, 'utf8'), cipher.final()]);
|
||||||
|
return [iv.toString('base64'), cipher.getAuthTag().toString('base64'), body.toString('base64')]
|
||||||
|
.join('.');
|
||||||
|
}
|
||||||
|
|
||||||
|
function decrypt(blob: string): string {
|
||||||
|
const [ivB64, tagB64, bodyB64] = blob.split('.');
|
||||||
|
if (!ivB64 || !tagB64 || !bodyB64) throw new Error('secrets.json ist beschaedigt');
|
||||||
|
const decipher = crypto.createDecipheriv('aes-256-gcm', key(), Buffer.from(ivB64, 'base64'));
|
||||||
|
decipher.setAuthTag(Buffer.from(tagB64, 'base64'));
|
||||||
|
return Buffer.concat([decipher.update(Buffer.from(bodyB64, 'base64')), decipher.final()])
|
||||||
|
.toString('utf8');
|
||||||
|
}
|
||||||
|
|
||||||
|
let cache: SecretStore | null = null;
|
||||||
|
|
||||||
|
export function readSecrets(): SecretStore {
|
||||||
|
if (cache) return cache;
|
||||||
|
if (!fs.existsSync(FILE)) { cache = structuredClone(EMPTY); return cache; }
|
||||||
|
try {
|
||||||
|
cache = JSON.parse(decrypt(fs.readFileSync(FILE, 'utf8'))) as SecretStore;
|
||||||
|
} catch (err) {
|
||||||
|
log.error(`secrets.json nicht lesbar (${(err as Error).message}) - starte mit leerem Speicher`);
|
||||||
|
cache = structuredClone(EMPTY);
|
||||||
|
}
|
||||||
|
return cache;
|
||||||
|
}
|
||||||
|
|
||||||
|
export function writeSecrets(next: SecretStore): void {
|
||||||
|
fs.mkdirSync(paths.db, { recursive: true });
|
||||||
|
fs.writeFileSync(FILE, encrypt(JSON.stringify(next)), { mode: 0o600 });
|
||||||
|
cache = next;
|
||||||
|
}
|
||||||
|
|
||||||
|
export function updateSecrets(fn: (s: SecretStore) => void): SecretStore {
|
||||||
|
const s = structuredClone(readSecrets());
|
||||||
|
fn(s);
|
||||||
|
writeSecrets(s);
|
||||||
|
return s;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Zustand fuer die Einrichtungsseite - enthaelt nie einen Geheimwert. */
|
||||||
|
export type SecretStatus = {
|
||||||
|
apps: Record<Role, { clientId: string; hasSecret: boolean }>;
|
||||||
|
tokens: Record<Role, { connected: boolean; login?: string; expiresAt?: string; scopes?: number }>;
|
||||||
|
obs: boolean;
|
||||||
|
ready: boolean;
|
||||||
|
};
|
||||||
|
|
||||||
|
export function secretStatus(): SecretStatus {
|
||||||
|
const s = readSecrets();
|
||||||
|
const roles: Role[] = ['broadcaster', 'bot'];
|
||||||
|
const apps = {} as SecretStatus['apps'];
|
||||||
|
const tokens = {} as SecretStatus['tokens'];
|
||||||
|
for (const r of roles) {
|
||||||
|
// Die Client-ID ist kein Geheimnis und wird angezeigt, damit man sieht,
|
||||||
|
// welche App hinterlegt ist. Das Secret nie.
|
||||||
|
apps[r] = { clientId: s.apps[r]?.clientId ?? '', hasSecret: Boolean(s.apps[r]?.clientSecret) };
|
||||||
|
const t = s.tokens[r];
|
||||||
|
tokens[r] = t
|
||||||
|
? { connected: true, login: t.login, expiresAt: t.expiresAt, scopes: t.scopes.length }
|
||||||
|
: { connected: false };
|
||||||
|
}
|
||||||
|
return {
|
||||||
|
apps, tokens,
|
||||||
|
obs: Boolean(s.obs?.password),
|
||||||
|
ready: Boolean(s.apps.broadcaster?.clientId && s.tokens.broadcaster),
|
||||||
|
};
|
||||||
|
}
|
||||||
@@ -1,5 +0,0 @@
|
|||||||
{
|
|
||||||
"extends": "../../tsconfig.base.json",
|
|
||||||
"compilerOptions": { "outDir": "dist", "rootDir": "src" },
|
|
||||||
"include": ["src"]
|
|
||||||
}
|
|
||||||
@@ -1,3 +0,0 @@
|
|||||||
packages:
|
|
||||||
- 'packages/*'
|
|
||||||
- 'apps/*'
|
|
||||||
@@ -1,17 +0,0 @@
|
|||||||
{
|
|
||||||
"compilerOptions": {
|
|
||||||
"target": "ES2023",
|
|
||||||
"lib": ["ES2023", "DOM"],
|
|
||||||
"module": "NodeNext",
|
|
||||||
"moduleResolution": "NodeNext",
|
|
||||||
"strict": true,
|
|
||||||
"noUncheckedIndexedAccess": true,
|
|
||||||
"exactOptionalPropertyTypes": true,
|
|
||||||
"verbatimModuleSyntax": true,
|
|
||||||
"declaration": true,
|
|
||||||
"sourceMap": true,
|
|
||||||
"skipLibCheck": true,
|
|
||||||
"esModuleInterop": true,
|
|
||||||
"forceConsistentCasingInFileNames": true
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -0,0 +1,27 @@
|
|||||||
|
{
|
||||||
|
"compilerOptions": {
|
||||||
|
"target": "ES2024",
|
||||||
|
"lib": ["ES2024", "DOM"],
|
||||||
|
"module": "NodeNext",
|
||||||
|
"moduleResolution": "NodeNext",
|
||||||
|
"types": ["node"],
|
||||||
|
|
||||||
|
"strict": true,
|
||||||
|
"noUncheckedIndexedAccess": true,
|
||||||
|
"exactOptionalPropertyTypes": true,
|
||||||
|
"noImplicitOverride": true,
|
||||||
|
|
||||||
|
// Node fuehrt die .ts-Dateien direkt aus (Type Stripping). Es gibt keinen
|
||||||
|
// Build-Schritt - tsc laeuft nur zur Pruefung. Deshalb muessen alle
|
||||||
|
// Konstrukte loeschbar sein: keine enums, keine Parameter-Properties,
|
||||||
|
// keine namespaces. Siehe ADR-0001.
|
||||||
|
"erasableSyntaxOnly": true,
|
||||||
|
"verbatimModuleSyntax": true,
|
||||||
|
"allowImportingTsExtensions": true,
|
||||||
|
"noEmit": true,
|
||||||
|
|
||||||
|
"skipLibCheck": true,
|
||||||
|
"forceConsistentCasingInFileNames": true
|
||||||
|
},
|
||||||
|
"include": ["packages/**/*.ts", "apps/**/*.ts", "docker/**/*.ts"]
|
||||||
|
}
|
||||||
Reference in new issue
Block a user