Konektory do Postgresu, pristupove udaje sifrovane

Pristupove udaje konektoru se ukladaji do databaze a prezijou restart. Popis
v documentation/14-databaze.md.

Databaze je volitelna a rezimy jsou oddelene:
- postgres kdyz je DATABASE_URL i SECRETS_KEY
- memory jinak, tedy pri nasazenem mockupu a lokalnim vyvoji bez DB

Rozhodnuti je jen na jednom miste (src/data/connectorStore.ts). Nikde jinde se
nezjistuje, jestli databaze je - kdyby se to rozlezlo po kodu, jedno misto by se
zapomnelo a chovalo by se pak jinak nez zbytek.

Chybejici databaze nesmi shodit start: container, ktery nenastartuje, je pro
AppFactory nefunkcni sluzba. Misto toho se do logu napise proc a portal to ukaze
na strance Konektory. Stejne tak kdyz migrace selzou - psat do rozbiteho
schematu je horsi nez neukladat.

Databaze potrebuje oboji. Bez SECRETS_KEY by se udaje ukladaly v plaintextu
a to je horsi nez ztratit je pri restartu: tabulku vidi kazda zaloha a kazdy
dump pri ladeni.

Pridano:
- pool v src/db/pool.ts vcetne transakci a dbFor(tenantId) jako sev pro budouci
  oddelenou databazi jednoho klienta
- migrace ze src/db/migrations/*.sql pod pg_advisory_lock, jinak je pri rolling
  deployi pusti vsechny instance naraz. Jeden soubor je jedna transakce
- sifrovani AES-256-GCM s nahodnym IV a verzi klice. Nerozsifrovatelna hodnota
  nepada, chova se jako nevyplnena a zaloguje se - jeden rozbity konektor nesmi
  shodit seznam ostatnich
- /health/ready s pingem do DB. /health na databazi zamerne nezavisi, kratky
  vypadek by jinak vedl k restartovani containeru
- GET /api/dashboard/storage a hlaska v portalu o tom, ze data jsou jen v pameti
- jediny vychozi konektor na firmu a sluzbu hlida castecny unikatni index, ne jen
  kod. Dva soubezne zapisy by jinak udelaly dva vychozi

Zmeneno: cteni i zapis konektoru je asynchronni, vcetne validace stromu.

Overeno proti Postgresu 16 v kontejneru: migrace, sifrovani v tabulce, preziti
restartu, rozsifrovani spravnym klicem, degradace pri spatnem klici, PATCH bez
tajneho pole, prepnuti a smazani vychoziho konektoru, pametovy rezim bez
DATABASE_URL. Kontejner po overeni smazan.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
JiriUhlir
2026-08-12 15:08:25 +02:00
co-authored by Claude Opus 5
parent 3279dd7dac
commit 78e7f99d60
22 changed files with 1761 additions and 298 deletions
+2 -1
View File
@@ -36,7 +36,8 @@ React aplikaci ze slozky `dist/public`.
| Sprava clenstvi z portalu | chybi | memberships jdou zmenit jen v kodu |
| Bugs a wishes | chybi | vyvojarska agenda, samostatna evidence vedle ticketu |
| Beh automatizaci | chybi | ulozeny strom se nevykonava, neni runtime |
| Databaze | chybi | data jsou v pameti, restart je vrati na vychozi stav |
| Databaze pro konektory | hotovo | Postgres, udaje sifrovane. Bez DATABASE_URL jede pamet |
| Databaze pro zbytek | chybi | automatizace, rozlozeni a tickety jsou v pameti |
| Odesilani e-mailu z formulare | chybi | poptavka se zatim jen loguje |
## Znama omezeni
+3 -1
View File
@@ -9,7 +9,8 @@ Verejne:
| Metoda | Cesta | Popis |
| ------ | ------------------- | --------------------------------------- |
| GET | `/health` | health check |
| GET | `/health` | liveness, nezavisi na databazi |
| GET | `/health/ready` | readiness, 503 pri nedostupne databazi |
| GET | `/docs` | Swagger UI |
| GET | `/openapi.json` | OpenAPI definice |
| POST | `/api/auth/login` | prihlaseni, vraci JWT |
@@ -36,6 +37,7 @@ Vyzaduji `Authorization: Bearer <token>`:
| POST | `/api/dashboard/tickets/:id/status` |
| POST | `/api/dashboard/tickets/:id/comment` |
| GET | `/api/dashboard/incidents` |
| GET | `/api/dashboard/storage` |
| GET | `/api/dashboard/services` |
| GET | `/api/dashboard/connectors/services` |
| GET | `/api/dashboard/connectors` |
+184
View File
@@ -0,0 +1,184 @@
# 14 - Databaze
Naprogramovano a overeno proti Postgresu 16. Zatim se do databaze ukladaji
**konektory**, tedy pristupove udaje k sluzbam. Zbytek je v pameti procesu,
poradi dalsich kroku je na konci.
## Databaze je volitelna, ale rezimy jsou oddelene
Aplikace jede ve dvou rezimech a rozdil se resi **na jednom miste**,
v `src/data/connectorStore.ts`. Nikde jinde se nezjistuje, jestli databaze je -
kdyby se to rozlezlo po kodu, jedno misto by se zapomnelo.
| Rezim | Kdy | Prezije restart |
| ---------- | --------------------------------------- | --------------- |
| `postgres` | je `DATABASE_URL` **i** `SECRETS_KEY` | ano |
| `memory` | jinak | ne |
Chybejici databaze **nesmi shodit start**. Container, ktery nenastartuje, je pro
AppFactory nefunkcni sluzba (AGENTS.md). Misto toho se do logu napise, proc se
jede v pameti, a portal to ukaze na strance Konektory.
Databaze potrebuje **oboji**. Bez klice by se pristupove udaje ukladaly
v plaintextu, a to je horsi nez ztratit je pri restartu - tabulku vidi kazda
zaloha a kazdy dump pri ladeni.
Stejne tak: kdyz jsou migrace nastavene, ale selzou, jede se dal v pameti.
Psat do rozbiteho schematu je horsi nez neukladat.
## Promenne
| Promenna | K cemu |
| ------------------- | ---------------------------------------------------------- |
| `DATABASE_URL` | `postgres://uzivatel:heslo@host:5432/csbot` |
| `SECRETS_KEY` | klic pro sifrovani pristupovych udaju, **secret** |
| `DATABASE_POOL_MAX` | kolik spojeni si vezme jedna instance, vychozi 10 |
| `DATABASE_SSL` | `true` u spravovanych databazi, ktere vyzaduji TLS |
`SECRETS_KEY` ma byt nahodny retezec, ne heslo:
```bash
node -e "console.log(require('crypto').randomBytes(32).toString('base64url'))"
```
**Klic se nesmi ztratit ani zmenit bez prevodu dat.** Bez nej se ulozene udaje
nerozsifruji. Nic se nerozbije, jen se konektory chovaji jako nevyplnene
a v logu je napsane proc - udaje se pak zadaji znovu.
## Lokalni spusteni
```bash
docker run -d --name csbot-pg -p 5433:5432 \
-e POSTGRES_PASSWORD=devpass -e POSTGRES_DB=csbot postgres:16-alpine
export DATABASE_URL="postgres://postgres:devpass@127.0.0.1:5433/csbot"
export SECRETS_KEY="$(node -e "console.log(require('crypto').randomBytes(32).toString('base64url'))")"
npm run dev
```
Migrace se pousti samy pri startu. Kontrola, ze to jede z databaze:
```bash
curl -s http://localhost:3000/health/ready
```
Bez promennych `npm run dev` funguje dal, jen v pameti.
## Migrace
Soubory `src/db/migrations/*.sql`, v abecednim poradi, kazdy jednou.
Co uz proslo, je v tabulce `schema_migrations`.
Dve veci, na kterych to stoji:
- **Poradovy zamek.** Pri rolling deployi startuje vic instanci naraz a bez
`pg_advisory_lock` by migrace pustily vsechny.
- **Jeden soubor je jedna transakce.** Pri chybe se z nej neuplatni nic, takze
nevznikne rozdelane schema, o kterem nikdo nevi.
**Migrace se nikdy neupravuji zpetne.** Uz projely u nekoho jineho, takze zmena
souboru znamena dve rozdilna schemata se stejnym cislem. Oprava je vzdy novy
soubor.
## Sifrovani pristupovych udaju
`src/db/secretBox.ts`, AES-256-GCM.
```json
{ "v": 1, "iv": "...", "tag": "...", "data": "..." }
```
- **GCM**, ne CBC: sifruje a zaroven overuje, ze s daty nikdo nehybal.
- **Nahodne IV** pro kazdou hodnotu, aby dve stejne hodnoty nedaly stejnou sifru.
- **`v` je verze klice.** Vymena klice pak znamena precist starym a zapsat novym,
ne zahodit vsechna napojeni.
- **Nerozsifrovatelna hodnota nepada.** Jeden rozbity konektor nesmi shodit
seznam vsech ostatnich, takze se chova jako nevyplneny a zaloguje se to.
Sifruji se vsechna pole, i necitliva. Je to jednodussi nez rozhodovat u kazdeho
zvlast a nic to nestoji.
## Schema
`connectors` (migrace `001_connectors.sql`):
| Sloupec | Poznamka |
| ------------- | ------------------------------------------------- |
| `tenant_id` | povinne, index zacina jim |
| `service_id` | odkaz do katalogu v kodu, ne do tabulky |
| `secrets` | JSONB se sifrovanymi hodnotami, nikdy plaintext |
| `is_default` | jediny vychozi na firmu a sluzbu, hlida index |
**Sluzby v databazi nejsou.** Jsou to definice, ktere delame my, a repo je u nich
zdroj pravdy kvuli code review a historii v gitu. Rucne upraveny radek v produkci
nikdo za tri mesice nedohleda. Duvody jsou v [12-sluzby-a-konektory.md](12-sluzby-a-konektory.md).
Vychozi konektor hlida **castecny unikatni index**, ne jen kod:
```sql
CREATE UNIQUE INDEX connectors_one_default_idx
ON connectors (tenant_id, service_id) WHERE is_default;
```
Bez nej by dva soubezne zapisy udelaly dva vychozi a krok bez vybraneho
konektoru by si vybiral podle nahody.
## Dvere k oddelene databazi
`dbFor(tenantId)` dnes vraci vzdy tentyz pool. Je to zamerny sev: jednou prijde
klient, ktery bude chtit vlastni databazi nebo bude delat tricet procent provozu,
a presun ma byt konfigurace, ne prepisovani dotazu.
Podminka je **nikdy nespojovat dotazem dva klienty**, coz uz vynucuje povinny
argument `tenantIds` v ulozistich. Podrobnosti
v [10-runtime-a-kapacita.md](10-runtime-a-kapacita.md).
## Health
| Endpoint | Zavisi na DB | K cemu |
| --------------- | ------------ | ----------------------------------------- |
| `/health` | ne | liveness, AppFactory podle nej restartuje |
| `/health/ready` | ano | readiness, vraci 503 pri nedostupne DB |
`/health` **nesmi** na databazi zavisel. Kratky vypadek DB by jinak vedl
k restartovani containeru, coz nic nespravi. Vysledek pingu se par sekund
cachuje, aby monitoring nedelal dotaz pri kazdem pingu.
## Pool a jedno pravidlo
`DATABASE_POOL_MAX` je vychozi 10 a je to zamerne malo. **Worker nesmi drzet
spojeni po dobu volani ciziho API** - volani do iDokladu trva 300 ms a pri
stovce soubeznych kroku by to bylo sto obsazenych spojeni. Se spravnym poradim
(odeber ulohu, uvolni spojeni, volej, zapis) staci par.
Az bude instanci vic, prijde PgBouncer v transakcnim rezimu. **Pozor: v nem
nefunguje `LISTEN/NOTIFY`**, na kterem ma stat sbernice udalosti pro SSE.
Ta pak potrebuje prime spojeni mimo PgBouncer.
## Co bylo overeno
Proti Postgresu 16 v kontejneru:
| Co | Vysledek |
| ----------------------------------------------------- | -------- |
| Migrace projedou a zapisou se do `schema_migrations` | ano |
| Udaje jsou v tabulce sifrovane, plaintext nikde | ano |
| Konektor prezije restart procesu | ano |
| Se spravnym klicem se udaje rozsifruji | ano |
| Se spatnym klicem se chovaji jako nevyplnene a loguje se | ano |
| `PATCH` bez tajneho pole tajne pole nesmaze | ano |
| Prepnuti vychoziho konektoru | ano |
| Smazani vychoziho preda priznak zbylemu | ano |
| Bez `DATABASE_URL` jede pametovy rezim a rekne to | ano |
## Co chybi
| Chybi | Poznamka |
| ---------------------------- | ----------------------------------------------------- |
| Automatizace v databazi | dalsi na rade, je to to, co si clovek nastavi |
| Rozlozeni dashboardu | male a samostatne, hned po automatizacich |
| Tickety a incidenty | naposled, dnes je generuje simulace |
| Uzivatele, firmy, resitele | v prototypu je to spis konfigurace nez data |
| Sbernice udalosti pres LISTEN/NOTIFY | dnes `EventEmitter` v pameti jedne instance |
| Vymena klice (rotace) | `v` je pripravene, prevod dat napsany neni |
| Retence a partitionovani | az u tabulek behu, viz dokument 10 |
+39
View File
@@ -2,6 +2,45 @@
Nejnovejsi nahore.
## 2026-08-12 - databaze pro konektory
Konektory se ukladaji do Postgresu, pristupove udaje sifrovane.
Popis v [14-databaze.md](14-databaze.md).
### Pridano
- `pg` jako zavislost, pool v `src/db/pool.ts` vcetne transakci a `dbFor(tenantId)`
jako sev pro budouci oddelenou databazi jednoho klienta.
- Migrace ze souboru `src/db/migrations/*.sql`, pousti se pri startu pod
`pg_advisory_lock` - pri rolling deployi je jinak pusti vsechny instance naraz.
Jeden soubor je jedna transakce, takze pri chybe nevznikne rozdelane schema.
- Sifrovani pristupovych udaju (`src/db/secretBox.ts`), AES-256-GCM s nahodnym
IV a verzi klice. Nerozsifrovatelna hodnota nepada, chova se jako nevyplnena
a zaloguje se - jeden rozbity konektor nesmi shodit seznam ostatnich.
- Dve implementace uloziste konektoru za jednim rozhranim (`memory`, `postgres`).
Rozhodnuti je jen na jednom miste, v `src/data/connectorStore.ts`.
- `/health/ready` s pingem do databaze. `/health` na databazi zamerne nezavisi:
kratky vypadek DB by jinak vedl k restartovani containeru.
- `GET /api/dashboard/storage` a hlaska na strance Konektory o tom, ze data jsou
jen v pameti. Bez toho se clovek divi, kam se podely jeho konektory.
- Jediny vychozi konektor na firmu a sluzbu hlida castecny unikatni index,
ne jen kod. Dva soubezne zapisy by jinak udelaly dva vychozi.
### Zmeneno
- Cteni i zapis konektoru je asynchronni, vcetne validace stromu.
- Databaze je volitelna. Bez `DATABASE_URL` nebo `SECRETS_KEY` se jede v pameti
a rekne se to v logu i v portalu. Container, ktery nenastartuje, je pro
AppFactory nefunkcni sluzba.
- Kdyz jsou migrace nastavene a selzou, jede se dal v pameti. Psat do rozbiteho
schematu je horsi nez neukladat.
### Overeno
Proti Postgresu 16 v kontejneru: migrace, sifrovani v tabulce, preziti restartu,
rozsifrovani spravnym klicem, degradace pri spatnem klici, `PATCH` bez tajneho
pole, prepnuti a smazani vychoziho konektoru, a pametovy rezim bez `DATABASE_URL`.
## 2026-08-12 - transformace dat a oprava konektoru
Transformace dat popsana v [13-transformace-dat.md](13-transformace-dat.md).
+160
View File
@@ -12,6 +12,7 @@
"cors": "^2.8.5",
"express": "^4.21.2",
"jsonwebtoken": "^9.0.2",
"pg": "^8.23.0",
"swagger-ui-express": "^5.0.1",
"zod": "^3.24.1"
},
@@ -22,6 +23,7 @@
"@types/express": "^4.17.21",
"@types/jsonwebtoken": "^9.0.7",
"@types/node": "^22.10.5",
"@types/pg": "^8.21.0",
"@types/react": "^18.3.18",
"@types/react-dom": "^18.3.5",
"@types/swagger-ui-express": "^4.1.7",
@@ -1618,6 +1620,18 @@
"undici-types": "~6.21.0"
}
},
"node_modules/@types/pg": {
"version": "8.21.0",
"resolved": "https://registry.npmjs.org/@types/pg/-/pg-8.21.0.tgz",
"integrity": "sha512-AYdtudzabjLZgVgRZmAnU8bAnVUXzuJX2IYHeSIiIHm68olD+LgQYCGWdtcNYnP0uq9c4S4NibVG3Ni7VbKW7Q==",
"dev": true,
"license": "MIT",
"dependencies": {
"@types/node": "*",
"pg-protocol": "*",
"pg-types": "^2.2.0"
}
},
"node_modules/@types/prop-types": {
"version": "15.7.15",
"resolved": "https://registry.npmjs.org/@types/prop-types/-/prop-types-15.7.15.tgz",
@@ -3220,6 +3234,95 @@
"integrity": "sha512-A/AGNMFN3c8bOlvV9RreMdrv7jsmF9XIfDeCd87+I8RNg6s78BhJxMu69NEMHBSJFxKidViTEdruRwEk/WIKqA==",
"license": "MIT"
},
"node_modules/pg": {
"version": "8.23.0",
"resolved": "https://registry.npmjs.org/pg/-/pg-8.23.0.tgz",
"integrity": "sha512-Ip2EQCngowJLGOfCwkFhPXU7/ljlhn6Rxlmy4XYfL2Y+vyRM59+8uR2xqRWKdYmbXmxCFOAmKxBuSUCdF34qLg==",
"license": "MIT",
"dependencies": {
"pg-connection-string": "^2.14.0",
"pg-pool": "^3.14.0",
"pg-protocol": "^1.16.0",
"pg-types": "2.2.0",
"pgpass": "1.0.5"
},
"engines": {
"node": ">= 16.0.0"
},
"optionalDependencies": {
"pg-cloudflare": "^1.4.0"
},
"peerDependencies": {
"pg-native": ">=3.0.1"
},
"peerDependenciesMeta": {
"pg-native": {
"optional": true
}
}
},
"node_modules/pg-cloudflare": {
"version": "1.4.0",
"resolved": "https://registry.npmjs.org/pg-cloudflare/-/pg-cloudflare-1.4.0.tgz",
"integrity": "sha512-Vo7z/6rrQYxpNRylp4Tlob2elzbh+N/MOQbxFVWCxS7oEx6jF53GTJFxK2WWpKuBRkmiin4Mt+xofFDjx09R0A==",
"license": "MIT",
"optional": true
},
"node_modules/pg-connection-string": {
"version": "2.14.0",
"resolved": "https://registry.npmjs.org/pg-connection-string/-/pg-connection-string-2.14.0.tgz",
"integrity": "sha512-XwWDGcLRGCXAR8F/AM5bG7Q+A3Wm2s6QeEjlOKZLlH3UYcguiqCWKyWXVag5TLTIjR7oOJUY8kcADaZgWPyLeg==",
"license": "MIT"
},
"node_modules/pg-int8": {
"version": "1.0.1",
"resolved": "https://registry.npmjs.org/pg-int8/-/pg-int8-1.0.1.tgz",
"integrity": "sha512-WCtabS6t3c8SkpDBUlb1kjOs7l66xsGdKpIPZsg4wR+B3+u9UAum2odSsF9tnvxg80h4ZxLWMy4pRjOsFIqQpw==",
"license": "ISC",
"engines": {
"node": ">=4.0.0"
}
},
"node_modules/pg-pool": {
"version": "3.14.0",
"resolved": "https://registry.npmjs.org/pg-pool/-/pg-pool-3.14.0.tgz",
"integrity": "sha512-gKtPkFdQPU3DksooVLi9LsjZxrsBUZIpa+7aVx+LV5pNh0KzP4Zleud2po+ConrxbuXGBJ6Hfer6hdgpIBpBaw==",
"license": "MIT",
"peerDependencies": {
"pg": ">=8.0"
}
},
"node_modules/pg-protocol": {
"version": "1.16.0",
"resolved": "https://registry.npmjs.org/pg-protocol/-/pg-protocol-1.16.0.tgz",
"integrity": "sha512-sILXutLVjCLjcDuOmvhX5e2Z4cS5qG/6Bu3VkpFwdf/633ElGLpEh9bgmuI5I4sqKqkifQiGyiCcx1HdtrK7tg==",
"license": "MIT"
},
"node_modules/pg-types": {
"version": "2.2.0",
"resolved": "https://registry.npmjs.org/pg-types/-/pg-types-2.2.0.tgz",
"integrity": "sha512-qTAAlrEsl8s4OiEQY69wDvcMIdQN6wdz5ojQiOy6YRMuynxenON0O5oCpJI6lshc6scgAY8qvJ2On/p+CXY0GA==",
"license": "MIT",
"dependencies": {
"pg-int8": "1.0.1",
"postgres-array": "~2.0.0",
"postgres-bytea": "~1.0.0",
"postgres-date": "~1.0.4",
"postgres-interval": "^1.1.0"
},
"engines": {
"node": ">=4"
}
},
"node_modules/pgpass": {
"version": "1.0.5",
"resolved": "https://registry.npmjs.org/pgpass/-/pgpass-1.0.5.tgz",
"integrity": "sha512-FdW9r/jQZhSeohs1Z3sI1yxFQNFvMcnmfuj4WBMUTxOrAyLMaTcE1aAMBiTlbMNaXvBCQuVi0R7hd8udDSP7ug==",
"license": "MIT",
"dependencies": {
"split2": "^4.1.0"
}
},
"node_modules/picocolors": {
"version": "1.1.1",
"resolved": "https://registry.npmjs.org/picocolors/-/picocolors-1.1.1.tgz",
@@ -3269,6 +3372,45 @@
"node": "^10 || ^12 || >=14"
}
},
"node_modules/postgres-array": {
"version": "2.0.0",
"resolved": "https://registry.npmjs.org/postgres-array/-/postgres-array-2.0.0.tgz",
"integrity": "sha512-VpZrUqU5A69eQyW2c5CA1jtLecCsN2U/bD6VilrFDWq5+5UIEVO7nazS3TEcHf1zuPYO/sqGvUvW62g86RXZuA==",
"license": "MIT",
"engines": {
"node": ">=4"
}
},
"node_modules/postgres-bytea": {
"version": "1.0.1",
"resolved": "https://registry.npmjs.org/postgres-bytea/-/postgres-bytea-1.0.1.tgz",
"integrity": "sha512-5+5HqXnsZPE65IJZSMkZtURARZelel2oXUEO8rH83VS/hxH5vv1uHquPg5wZs8yMAfdv971IU+kcPUczi7NVBQ==",
"license": "MIT",
"engines": {
"node": ">=0.10.0"
}
},
"node_modules/postgres-date": {
"version": "1.0.7",
"resolved": "https://registry.npmjs.org/postgres-date/-/postgres-date-1.0.7.tgz",
"integrity": "sha512-suDmjLVQg78nMK2UZ454hAG+OAW+HQPZ6n++TNDUX+L0+uUlLywnoxJKDou51Zm+zTCjrCl0Nq6J9C5hP9vK/Q==",
"license": "MIT",
"engines": {
"node": ">=0.10.0"
}
},
"node_modules/postgres-interval": {
"version": "1.2.0",
"resolved": "https://registry.npmjs.org/postgres-interval/-/postgres-interval-1.2.0.tgz",
"integrity": "sha512-9ZhXKM/rw350N1ovuWHbGxnGh/SNJ4cnxHiM0rxE4VN41wsg8P8zWn9hv/buK00RP4WvlOyr/RBDiptyxVbkZQ==",
"license": "MIT",
"dependencies": {
"xtend": "^4.0.0"
},
"engines": {
"node": ">=0.10.0"
}
},
"node_modules/proxy-addr": {
"version": "2.0.7",
"resolved": "https://registry.npmjs.org/proxy-addr/-/proxy-addr-2.0.7.tgz",
@@ -3659,6 +3801,15 @@
"node": ">=0.10.0"
}
},
"node_modules/split2": {
"version": "4.2.0",
"resolved": "https://registry.npmjs.org/split2/-/split2-4.2.0.tgz",
"integrity": "sha512-UcjcJOWknrNkF6PLX83qcHM6KHgVKNkV62Y8a5uYDVv9ydGQVwAHMKqHdJje1VTWpljG0WYpCDhrCdAOYH4TWg==",
"license": "ISC",
"engines": {
"node": ">= 10.x"
}
},
"node_modules/statuses": {
"version": "2.0.2",
"resolved": "https://registry.npmjs.org/statuses/-/statuses-2.0.2.tgz",
@@ -4488,6 +4639,15 @@
"url": "https://github.com/chalk/wrap-ansi?sponsor=1"
}
},
"node_modules/xtend": {
"version": "4.0.2",
"resolved": "https://registry.npmjs.org/xtend/-/xtend-4.0.2.tgz",
"integrity": "sha512-LKYU1iAXJXUgAXn9URjiu+MWhyUXHsvfp7mcuYm9dSUKK0/CjtrUwFAxD82/mCWbtLsGjFIad0wIsod4zrTAEQ==",
"license": "MIT",
"engines": {
"node": ">=0.4"
}
},
"node_modules/y18n": {
"version": "5.0.8",
"resolved": "https://registry.npmjs.org/y18n/-/y18n-5.0.8.tgz",
+2
View File
@@ -19,6 +19,7 @@
"cors": "^2.8.5",
"express": "^4.21.2",
"jsonwebtoken": "^9.0.2",
"pg": "^8.23.0",
"swagger-ui-express": "^5.0.1",
"zod": "^3.24.1"
},
@@ -29,6 +30,7 @@
"@types/express": "^4.17.21",
"@types/jsonwebtoken": "^9.0.7",
"@types/node": "^22.10.5",
"@types/pg": "^8.21.0",
"@types/react": "^18.3.18",
"@types/react-dom": "^18.3.5",
"@types/swagger-ui-express": "^4.1.7",
+31
View File
@@ -75,6 +75,37 @@ export const config = {
*/
publicOrigin: (process.env.PUBLIC_ORIGIN ?? '').trim().replace(/\/+$/, ''),
// ------------------------------------------------------------------ databaze
/**
* Pripojeni do Postgresu, napr. postgres://user:pass@host:5432/csbot
*
* Prazdna hodnota je platny stav: aplikace jede v pameti procesu. Container,
* ktery nenastartuje kvuli chybejici promenne, je pro AppFactory nefunkcni
* sluzba (AGENTS.md), takze se na tom nepada.
*/
databaseUrl: (process.env.DATABASE_URL ?? '').trim(),
/**
* Klic pro sifrovani pristupovych udaju konektoru.
*
* Ma to byt nahodny retezec, ne heslo. Jak ho vygenerovat je
* v documentation/14-databaze.md.
*
* Bez nej se konektory neukladaji do databaze ani kdyz je nastavena -
* plaintext v tabulce je horsi nez ztrata dat pri restartu.
*/
secretsKey: (process.env.SECRETS_KEY ?? '').trim(),
/**
* Kolik spojeni si smi vzit jedna instance.
*
* Nizke cislo je zamer: worker nesmi drzet spojeni po dobu volani ciziho API,
* takze i pri stovce soubeznych kroku staci par spojeni. Podrobnosti
* v documentation/10-runtime-a-kapacita.md.
*/
databasePoolMax: positiveNumber(process.env.DATABASE_POOL_MAX, 10),
/** Spravovane databaze vyzaduji TLS. */
databaseSsl: process.env.DATABASE_SSL === 'true',
// ------------------------------------------------------- skripty konektoru
/**
+119 -258
View File
@@ -1,104 +1,120 @@
/**
* Konektory = napojeni jedne firmy na jednu sluzbu.
* Konektory: vyber uloziste a jedine misto, kde se rozhoduje o rezimu.
*
* Sluzba (`services.ts`) rika, co je potreba: "iDoklad chce hlavicky
* X-ClientId a X-ClientSecret". Konektor drzi hodnoty. Kazda firma ma svoje.
* Dve implementace, jedno rozhrani:
* - `postgres` kdyz je `DATABASE_URL` a `SECRETS_KEY`,
* - `memory` jinak, tedy pri mockupu a lokalnim vyvoji bez databaze.
*
* Proto **pristupove udaje nejsou v environment variables**. Kdyby byly,
* mela by je cela instance spolecne a dve firmy by fakturovaly ze stejneho
* uctu. Napojeni je vlastnost firmy, ne prostredi.
* Rozhodnuti je **jen tady**. Nikde jinde se nezjistuje, jestli databaze je -
* kdyby se to rozlezlo po kodu, jedno misto by se zapomnelo a chovalo by se
* pak jinak nez zbytek.
*
* POZOR: data jsou v pameti procesu, restart je vrati na vychozi sadu.
* Cilovy stav je Postgres se sifrovanymi hodnotami, viz
* documentation/09-navrh-rozsireni.md, bod 9.
* Databaze potrebuje **oboji**. Bez klice by se pristupove udaje ukladaly
* v plaintextu a to je horsi nez ztratit je pri restartu, proto se v tom
* pripade jede dal v pameti a rekne se to nahlas.
*/
import { randomUUID } from 'node:crypto';
import { findService, type Service, type ServiceCredentialField } from './services.js';
import { canSealSecrets } from '../db/secretBox.js';
import { isDatabaseEnabled } from '../db/pool.js';
import { memoryConnectors, seedMemoryConnectors } from './connectors/memory.js';
import { postgresConnectors } from './connectors/postgres.js';
import type {
Connector,
ConnectorRepository,
CreateConnectorInput,
UpdateConnectorInput,
} from './connectors/types.js';
export interface Connector {
id: string;
tenantId: string;
serviceId: string;
/** Firma muze mit dva ucty teze sluzby, proto vlastni nazev. */
name: string;
/** Prepis adresy sluzby. null = vychozi podle `appId` a SERVICES_BASE_URL. */
baseUrl: string | null;
/** Hodnoty poli z `Service.credentials`. Tajne se z API nikdy nevraci. */
values: Record<string, string>;
enabled: boolean;
/** untested = jeste se neoverovalo, ok = posledni overeni proslo. */
status: 'untested' | 'ok' | 'error';
lastCheckAt: string | null;
lastError: string | null;
/** Krok stromu bez vybraneho konektoru pouzije vychozi. */
isDefault: boolean;
createdAt: string;
updatedAt: string;
export type {
Connector,
ConnectorIssue,
CreateConnectorInput,
PublicConnector,
UpdateConnectorInput,
} from './connectors/types.js';
export { toPublicConnector, validateConnectorValues } from './connectors/types.js';
export type StorageMode = 'postgres' | 'memory';
export interface StorageStatus {
mode: StorageMode;
/** Proc se jede v pameti. null u databaze. */
reason: string | null;
/** true = restart procesu data smaze. */
ephemeral: boolean;
}
let repository: ConnectorRepository = memoryConnectors;
let status: StorageStatus = {
mode: 'memory',
reason: 'Databáze není nastavená.',
ephemeral: true,
};
export interface InitOptions {
/**
* false = databaze je nastavena, ale neni pouzitelna (napr. selhaly migrace).
* Pak se jede v pameti, protoze psat do rozbiteho schematu je horsi.
*/
databaseReady?: boolean;
}
/**
* Verze pro klienta. **Tajne hodnoty tady nejsou** a nikdy nesmi byt -
* secrets se nevraci z beznych endpointu (AGENTS.md).
* Vybere uloziste. Vola se jednou pri startu, po migracich.
* Volani je idempotentni, at jde zavolat i z testu.
*/
export interface PublicConnector {
id: string;
tenantId: string;
serviceId: string;
name: string;
baseUrl: string | null;
enabled: boolean;
status: Connector['status'];
lastCheckAt: string | null;
lastError: string | null;
isDefault: boolean;
createdAt: string;
updatedAt: string;
/** ID poli, ktera jsou vyplnena. Hodnoty se nevraci. */
filled: string[];
/** ID povinnych poli, ktera jeste chybi. */
missing: string[];
/** Necitliva nastaveni. Tajna pole tu nejsou vubec. */
config: Record<string, string>;
/** true = vsechna povinna pole jsou vyplnena, jde volat. */
ready: boolean;
}
export async function initConnectorStore(options: InitOptions = {}): Promise<StorageStatus> {
const databaseReady = options.databaseReady ?? true;
const connectors: Connector[] = [];
function now(): string {
return new Date().toISOString();
}
/** Povinna pole, ktera nejsou vyplnena. */
function missingFields(service: Service, values: Record<string, string>): string[] {
return service.credentials
.filter((field) => field.required && (values[field.id] ?? '').trim() === '')
.map((field) => field.id);
}
export function toPublicConnector(connector: Connector): PublicConnector {
const service = findService(connector.serviceId);
const credentials = service?.credentials ?? [];
const filled = Object.entries(connector.values)
.filter(([, value]) => value.trim() !== '')
.map(([key]) => key);
const config: Record<string, string> = {};
for (const field of credentials) {
if (field.secret) continue;
const value = connector.values[field.id];
if (value !== undefined && value.trim() !== '') config[field.id] = value;
if (isDatabaseEnabled() && !databaseReady) {
repository = memoryConnectors;
status = {
mode: 'memory',
reason: 'Databáze je nastavená, ale schéma není použitelné. Zkontrolujte log migrací.',
ephemeral: true,
};
console.error(`[storage] ${status.reason}`);
await seedMemoryConnectors();
return status;
}
const missing = service ? missingFields(service, connector.values) : [];
if (isDatabaseEnabled() && canSealSecrets()) {
repository = postgresConnectors;
status = { mode: 'postgres', reason: null, ephemeral: false };
console.info('[storage] konektory: Postgres, přístupové údaje šifrované');
return status;
}
const { values: _values, ...rest } = connector;
return { ...rest, filled, missing, config, ready: missing.length === 0 };
repository = memoryConnectors;
if (isDatabaseEnabled() && !canSealSecrets()) {
status = {
mode: 'memory',
reason:
'Databáze je nastavená, ale chybí SECRETS_KEY. Přístupové údaje by se ukládaly ' +
'v plaintextu, takže konektory zůstávají v paměti.',
ephemeral: true,
};
console.error(`[storage] ${status.reason}`);
} else {
status = {
mode: 'memory',
reason: 'Databáze není nastavená (DATABASE_URL). Restart procesu data smaže.',
ephemeral: true,
};
console.warn(`[storage] ${status.reason}`);
}
// Ukazkovy konektor jen v pametovem rezimu. V databazi by se delal pri kazdem startu.
await seedMemoryConnectors();
return status;
}
// ------------------------------------------------------------------- cteni
export function storageStatus(): StorageStatus {
return status;
}
// ------------------------------------------------- pruchod do zvoleneho uloziste
/**
* Filtr na firmu je **povinny argument**, ne volitelny. Zapomenuty filtr tak
@@ -107,18 +123,13 @@ export function toPublicConnector(connector: Connector): PublicConnector {
export function listConnectors(
tenantIds: string[],
options: { serviceId?: string } = {},
): Connector[] {
return connectors
.filter((connector) => tenantIds.includes(connector.tenantId))
.filter((connector) => !options.serviceId || connector.serviceId === options.serviceId)
.sort((a, b) => a.name.localeCompare(b.name, 'cs'));
): Promise<Connector[]> {
return repository.list(tenantIds, options);
}
/** Cizi konektor se chova jako neexistujici, tedy undefined, ne chyba prava. */
export function getConnector(id: string, tenantIds: string[]): Connector | undefined {
const connector = connectors.find((item) => item.id === id);
if (!connector) return undefined;
return tenantIds.includes(connector.tenantId) ? connector : undefined;
export function getConnector(id: string, tenantIds: string[]): Promise<Connector | undefined> {
return repository.get(id, tenantIds);
}
/**
@@ -126,161 +137,32 @@ export function getConnector(id: string, tenantIds: string[]): Connector | undef
* Krok stromu bez vybraneho konektoru pouzije tenhle, diky tomu je vzorovy
* strom prenositelny mezi firmami.
*/
export function defaultConnectorFor(tenantId: string, serviceId: string): Connector | undefined {
const forService = connectors.filter(
(connector) =>
connector.tenantId === tenantId && connector.serviceId === serviceId && connector.enabled,
);
return forService.find((connector) => connector.isDefault) ?? forService[0];
export function defaultConnectorFor(
tenantId: string,
serviceId: string,
): Promise<Connector | undefined> {
return repository.defaultFor(tenantId, serviceId);
}
/** Kolik konektoru ma firma na kterou sluzbu. Podle toho se kresli stav v katalogu. */
export function connectorCountsByService(tenantIds: string[]): Map<string, number> {
const counts = new Map<string, number>();
for (const connector of listConnectors(tenantIds)) {
counts.set(connector.serviceId, (counts.get(connector.serviceId) ?? 0) + 1);
}
return counts;
export function connectorCountsByService(tenantIds: string[]): Promise<Map<string, number>> {
return repository.countsByService(tenantIds);
}
// ------------------------------------------------------------------- validace
export interface ConnectorIssue {
field: string;
message: string;
}
/**
* Overi hodnoty proti tomu, co sluzba vyzaduje.
*
* Nevyplnene povinne pole **neni chyba** ukladani, ale nedodelek: konektor se
* ulozi a jen nepujde pouzit. Stejny rezim jako u rozdelane automatizace -
* rozdelana prace se nezahazuje.
*
* Chyba je jen pole, ktere sluzba vubec nema. To uz je rozbite napojeni.
*/
export function validateConnectorValues(
service: Service,
values: Record<string, string>,
): ConnectorIssue[] {
const known = new Map<string, ServiceCredentialField>(
service.credentials.map((field) => [field.id, field]),
);
const issues: ConnectorIssue[] = [];
for (const key of Object.keys(values)) {
if (!known.has(key)) {
issues.push({ field: key, message: `Služba ${service.name} pole „${key}" nemá.` });
}
}
return issues;
}
// -------------------------------------------------------------------- zapis
export interface CreateConnectorInput {
tenantId: string;
serviceId: string;
name: string;
baseUrl?: string | null;
values?: Record<string, string>;
isDefault?: boolean;
}
export function createConnector(input: CreateConnectorInput): Connector {
const timestamp = now();
const existing = listConnectors([input.tenantId], { serviceId: input.serviceId });
const connector: Connector = {
id: `con_${randomUUID().slice(0, 8)}`,
tenantId: input.tenantId,
serviceId: input.serviceId,
name: input.name,
baseUrl: input.baseUrl ?? null,
values: { ...(input.values ?? {}) },
enabled: true,
status: 'untested',
lastCheckAt: null,
lastError: null,
// Prvni konektor na sluzbu je vychozi, jinak by krok bez vyberu nemel co vzit.
isDefault: input.isDefault ?? existing.length === 0,
createdAt: timestamp,
updatedAt: timestamp,
};
if (connector.isDefault) clearDefaults(input.tenantId, input.serviceId);
connectors.push(connector);
console.info(`[connectors] zalozen ${connector.id} (${connector.serviceId}) pro ${connector.tenantId}`);
return connector;
}
function clearDefaults(tenantId: string, serviceId: string): void {
for (const connector of connectors) {
if (connector.tenantId === tenantId && connector.serviceId === serviceId) {
connector.isDefault = false;
}
}
}
export interface UpdateConnectorInput {
name?: string;
baseUrl?: string | null;
/**
* Jen pole, ktera se meni. Prazdny retezec hodnotu **smaze**, chybejici klic
* ji nechá. Diky tomu jde ulozit formular, ktery tajne hodnoty neposila.
*/
values?: Record<string, string>;
enabled?: boolean;
isDefault?: boolean;
export function createConnector(input: CreateConnectorInput): Promise<Connector> {
return repository.create(input);
}
export function updateConnector(
id: string,
patch: UpdateConnectorInput,
tenantIds: string[],
): Connector | undefined {
const connector = getConnector(id, tenantIds);
if (!connector) return undefined;
if (patch.name !== undefined) connector.name = patch.name;
if (patch.baseUrl !== undefined) connector.baseUrl = patch.baseUrl;
if (patch.enabled !== undefined) connector.enabled = patch.enabled;
if (patch.values) {
for (const [key, value] of Object.entries(patch.values)) {
if (value === '') delete connector.values[key];
else connector.values[key] = value;
}
// Zmena udaju znamena, ze predchozi overeni uz nic nerika.
connector.status = 'untested';
connector.lastError = null;
}
if (patch.isDefault === true) {
clearDefaults(connector.tenantId, connector.serviceId);
connector.isDefault = true;
}
connector.updatedAt = now();
return connector;
): Promise<Connector | undefined> {
return repository.update(id, patch, tenantIds);
}
export function deleteConnector(id: string, tenantIds: string[]): boolean {
const connector = getConnector(id, tenantIds);
if (!connector) return false;
const index = connectors.indexOf(connector);
connectors.splice(index, 1);
// Kdyz zmizel vychozi, prevezme to prvni zbyly - jinak by kroky bez vyberu
// prestaly fungovat, aniz by se cokoliv jineho zmenilo.
if (connector.isDefault) {
const next = listConnectors([connector.tenantId], { serviceId: connector.serviceId })[0];
if (next) next.isDefault = true;
}
console.info(`[connectors] smazan ${id}`);
return true;
export function deleteConnector(id: string, tenantIds: string[]): Promise<boolean> {
return repository.remove(id, tenantIds);
}
/** Vysledek overeni napojeni. Zapisuje ho endpoint pro test. */
@@ -289,27 +171,6 @@ export function setConnectorStatus(
status: Connector['status'],
error: string | null,
tenantIds: string[],
): Connector | undefined {
const connector = getConnector(id, tenantIds);
if (!connector) return undefined;
connector.status = status;
connector.lastError = error;
connector.lastCheckAt = now();
connector.updatedAt = connector.lastCheckAt;
return connector;
): Promise<Connector | undefined> {
return repository.setStatus(id, status, error, tenantIds);
}
// -------------------------------------------------------------- vychozi sada
/**
* Jeden ukazkovy konektor bez vyplnenych udaju.
*
* Zamerne bez nich: ukazuje presne ten stav, ve kterem konektor vznikne, tedy
* "sluzba je napojena, ale chybi pristupove udaje". Vyplnit je jde v portalu.
*/
createConnector({
tenantId: 'tnt_automia',
serviceId: 'idoklad',
name: 'iDoklad Automia',
});
+176
View File
@@ -0,0 +1,176 @@
/**
* Konektory v pameti procesu.
*
* Rezim pro mockup a pro lokalni vyvoj bez databaze. **Restart procesu data
* smaze**, vcetne pristupovych udaju. Pri vyvoji s `tsx watch` to znamena
* po kazde uprave kodu, takze je to potreba mit na pameti.
*
* Podle stejneho rozhrani jede i `postgres.ts`. Rozdil se resi jen pri vyberu
* uloziste v `../connectorStore.ts`.
*/
import { randomUUID } from 'node:crypto';
import {
missingFields,
nowIso,
type Connector,
type ConnectorRepository,
type CreateConnectorInput,
type UpdateConnectorInput,
} from './types.js';
import { findService } from '../services.js';
const rows: Connector[] = [];
function byTenant(tenantIds: string[]): Connector[] {
return rows.filter((row) => tenantIds.includes(row.tenantId));
}
function clearDefaults(tenantId: string, serviceId: string): void {
for (const row of rows) {
if (row.tenantId === tenantId && row.serviceId === serviceId) row.isDefault = false;
}
}
export const memoryConnectors: ConnectorRepository = {
kind: 'memory',
async list(tenantIds, options = {}) {
return byTenant(tenantIds)
.filter((row) => !options.serviceId || row.serviceId === options.serviceId)
.sort((a, b) => a.name.localeCompare(b.name, 'cs'))
.map((row) => ({ ...row, values: { ...row.values } }));
},
async get(id, tenantIds) {
const row = rows.find((item) => item.id === id);
// Cizi konektor se chova jako neexistujici, ne jako chyba prava.
if (!row || !tenantIds.includes(row.tenantId)) return undefined;
return { ...row, values: { ...row.values } };
},
async defaultFor(tenantId, serviceId) {
const forService = rows.filter(
(row) => row.tenantId === tenantId && row.serviceId === serviceId && row.enabled,
);
const chosen = forService.find((row) => row.isDefault) ?? forService[0];
return chosen ? { ...chosen, values: { ...chosen.values } } : undefined;
},
async countsByService(tenantIds) {
const counts = new Map<string, number>();
for (const row of byTenant(tenantIds)) {
counts.set(row.serviceId, (counts.get(row.serviceId) ?? 0) + 1);
}
return counts;
},
async create(input: CreateConnectorInput) {
const timestamp = nowIso();
const existing = rows.filter(
(row) => row.tenantId === input.tenantId && row.serviceId === input.serviceId,
);
const connector: Connector = {
id: `con_${randomUUID().slice(0, 8)}`,
tenantId: input.tenantId,
serviceId: input.serviceId,
name: input.name,
baseUrl: input.baseUrl ?? null,
values: { ...(input.values ?? {}) },
enabled: true,
status: 'untested',
lastCheckAt: null,
lastError: null,
// Prvni konektor na sluzbu je vychozi, jinak by krok bez vyberu nemel co vzit.
isDefault: input.isDefault ?? existing.length === 0,
createdAt: timestamp,
updatedAt: timestamp,
};
if (connector.isDefault) clearDefaults(input.tenantId, input.serviceId);
rows.push(connector);
return { ...connector, values: { ...connector.values } };
},
async update(id, patch: UpdateConnectorInput, tenantIds) {
const row = rows.find((item) => item.id === id);
if (!row || !tenantIds.includes(row.tenantId)) return undefined;
if (patch.name !== undefined) row.name = patch.name;
if (patch.baseUrl !== undefined) row.baseUrl = patch.baseUrl;
if (patch.enabled !== undefined) row.enabled = patch.enabled;
if (patch.values) {
for (const [key, value] of Object.entries(patch.values)) {
if (value === '') delete row.values[key];
else row.values[key] = value;
}
// Zmena udaju znamena, ze predchozi overeni uz nic nerika.
row.status = 'untested';
row.lastError = null;
}
if (patch.isDefault === true) {
clearDefaults(row.tenantId, row.serviceId);
row.isDefault = true;
}
row.updatedAt = nowIso();
return { ...row, values: { ...row.values } };
},
async remove(id, tenantIds) {
const row = rows.find((item) => item.id === id);
if (!row || !tenantIds.includes(row.tenantId)) return false;
rows.splice(rows.indexOf(row), 1);
// Kdyz zmizel vychozi, prevezme to prvni zbyly - jinak by kroky bez vyberu
// prestaly fungovat, aniz by se cokoliv jineho zmenilo.
if (row.isDefault) {
const next = rows.find(
(item) => item.tenantId === row.tenantId && item.serviceId === row.serviceId,
);
if (next) next.isDefault = true;
}
return true;
},
async setStatus(id, status, error, tenantIds) {
const row = rows.find((item) => item.id === id);
if (!row || !tenantIds.includes(row.tenantId)) return undefined;
row.status = status;
row.lastError = error;
row.lastCheckAt = nowIso();
row.updatedAt = row.lastCheckAt;
return { ...row, values: { ...row.values } };
},
};
/**
* Ukazkovy konektor bez vyplnenych udaju.
*
* Zamerne bez nich: ukazuje presne ten stav, ve kterem konektor vznikne, tedy
* "sluzba je napojena, ale chybi pristupove udaje". Zaklada se jen v pametovem
* rezimu - v databazi by se pri kazdem startu delal znovu.
*/
export async function seedMemoryConnectors(): Promise<void> {
if (rows.length > 0) return;
if (!findService('idoklad')) return;
await memoryConnectors.create({
tenantId: 'tnt_automia',
serviceId: 'idoklad',
name: 'iDoklad Automia',
});
}
/** Jen pro diagnostiku: kolik povinnych poli chybi napric vsemi konektory. */
export function memoryMissingCount(): number {
return rows.reduce((total, row) => {
const service = findService(row.serviceId);
return total + (service ? missingFields(service, row.values).length : 0);
}, 0);
}
+248
View File
@@ -0,0 +1,248 @@
/**
* Konektory v Postgresu.
*
* Rozdil proti pametovemu ulozisti je jen v tomhle souboru. Volajici nepozna,
* ktere uloziste je zapnute - obe maji stejne rozhrani.
*
* **Tajne hodnoty se sifruji.** Do tabulky nikdy nesmi plaintext, jinak by
* kazda zaloha a kazdy dump pri ladeni nesl klientske klice. Necitliva pole
* (napr. jazyk odpovedi) se sifruji taky - je to jednodussi nez rozhodovat
* u kazdeho pole a nic to nestoji.
*/
import { randomUUID } from 'node:crypto';
import { query, queryOne, transaction } from '../../db/pool.js';
import { openAll, sealAll } from '../../db/secretBox.js';
import type {
Connector,
ConnectorRepository,
CreateConnectorInput,
UpdateConnectorInput,
} from './types.js';
interface ConnectorRow {
id: string;
tenant_id: string;
service_id: string;
name: string;
base_url: string | null;
secrets: unknown;
enabled: boolean;
status: string;
last_check_at: Date | null;
last_error: string | null;
is_default: boolean;
created_at: Date;
updated_at: Date;
}
function toConnector(row: ConnectorRow): Connector {
return {
id: row.id,
tenantId: row.tenant_id,
serviceId: row.service_id,
name: row.name,
baseUrl: row.base_url,
values: openAll(row.secrets),
enabled: row.enabled,
status: row.status as Connector['status'],
lastCheckAt: row.last_check_at ? row.last_check_at.toISOString() : null,
lastError: row.last_error,
isDefault: row.is_default,
createdAt: row.created_at.toISOString(),
updatedAt: row.updated_at.toISOString(),
};
}
const COLUMNS = `
id, tenant_id, service_id, name, base_url, secrets, enabled, status,
last_check_at, last_error, is_default, created_at, updated_at
`;
export const postgresConnectors: ConnectorRepository = {
kind: 'postgres',
async list(tenantIds, options = {}) {
// Prazdny seznam firem znamena "nic", ne "vse". Bez teto vetve by
// `= ANY('{}')` sice nic nevratilo, ale spolehat se na to je past.
if (tenantIds.length === 0) return [];
const rows = await query<ConnectorRow>(
`SELECT ${COLUMNS} FROM connectors
WHERE tenant_id = ANY($1)
AND ($2::text IS NULL OR service_id = $2)
ORDER BY name`,
[tenantIds, options.serviceId ?? null],
);
return rows.map(toConnector);
},
async get(id, tenantIds) {
if (tenantIds.length === 0) return undefined;
const row = await queryOne<ConnectorRow>(
`SELECT ${COLUMNS} FROM connectors WHERE id = $1 AND tenant_id = ANY($2)`,
[id, tenantIds],
);
return row ? toConnector(row) : undefined;
},
async defaultFor(tenantId, serviceId) {
// Vychozi vyhrava, jinak prvni podle nazvu. Vypnute konektory se neberou.
const row = await queryOne<ConnectorRow>(
`SELECT ${COLUMNS} FROM connectors
WHERE tenant_id = $1 AND service_id = $2 AND enabled
ORDER BY is_default DESC, name
LIMIT 1`,
[tenantId, serviceId],
);
return row ? toConnector(row) : undefined;
},
async countsByService(tenantIds) {
if (tenantIds.length === 0) return new Map();
const rows = await query<{ service_id: string; count: string }>(
`SELECT service_id, count(*) AS count FROM connectors
WHERE tenant_id = ANY($1)
GROUP BY service_id`,
[tenantIds],
);
return new Map(rows.map((row) => [row.service_id, Number(row.count)]));
},
async create(input: CreateConnectorInput) {
return transaction(async (client) => {
const existing = await client.query<{ count: string }>(
'SELECT count(*) AS count FROM connectors WHERE tenant_id = $1 AND service_id = $2',
[input.tenantId, input.serviceId],
);
// Prvni konektor na sluzbu je vychozi, jinak by krok bez vyberu nemel co vzit.
const isDefault = input.isDefault ?? Number(existing.rows[0]?.count ?? 0) === 0;
// Jediny vychozi na dvojici firma a sluzba hlida i unikatni index.
// Uvolnit misto pred zapisem je proto nutne, ne jen uklidne.
if (isDefault) {
await client.query(
'UPDATE connectors SET is_default = false WHERE tenant_id = $1 AND service_id = $2',
[input.tenantId, input.serviceId],
);
}
const row = await client.query<ConnectorRow>(
`INSERT INTO connectors (id, tenant_id, service_id, name, base_url, secrets, is_default)
VALUES ($1, $2, $3, $4, $5, $6::jsonb, $7)
RETURNING ${COLUMNS}`,
[
`con_${randomUUID().slice(0, 8)}`,
input.tenantId,
input.serviceId,
input.name,
input.baseUrl ?? null,
JSON.stringify(sealAll(input.values ?? {})),
isDefault,
],
);
return toConnector(row.rows[0]);
});
},
async update(id, patch: UpdateConnectorInput, tenantIds) {
if (tenantIds.length === 0) return undefined;
return transaction(async (client) => {
const current = await client.query<ConnectorRow>(
`SELECT ${COLUMNS} FROM connectors WHERE id = $1 AND tenant_id = ANY($2) FOR UPDATE`,
[id, tenantIds],
);
const row = current.rows[0];
if (!row) return undefined;
// Hodnoty se skladaji ze stavajicich a novych. Prazdny retezec maze,
// chybejici klic nechava - formular tajne hodnoty neposila.
let secrets = row.secrets;
let statusReset = false;
if (patch.values) {
const merged = openAll(row.secrets);
for (const [key, value] of Object.entries(patch.values)) {
if (value === '') delete merged[key];
else merged[key] = value;
}
secrets = sealAll(merged);
statusReset = true;
}
if (patch.isDefault === true) {
await client.query(
'UPDATE connectors SET is_default = false WHERE tenant_id = $1 AND service_id = $2',
[row.tenant_id, row.service_id],
);
}
const updated = await client.query<ConnectorRow>(
`UPDATE connectors SET
name = COALESCE($2, name),
base_url = CASE WHEN $3::boolean THEN $4 ELSE base_url END,
secrets = $5::jsonb,
enabled = COALESCE($6, enabled),
is_default = CASE WHEN $7::boolean THEN true ELSE is_default END,
status = CASE WHEN $8::boolean THEN 'untested' ELSE status END,
last_error = CASE WHEN $8::boolean THEN NULL ELSE last_error END,
updated_at = now()
WHERE id = $1
RETURNING ${COLUMNS}`,
[
id,
patch.name ?? null,
patch.baseUrl !== undefined,
patch.baseUrl ?? null,
JSON.stringify(secrets),
patch.enabled ?? null,
patch.isDefault === true,
statusReset,
],
);
return toConnector(updated.rows[0]);
});
},
async remove(id, tenantIds) {
if (tenantIds.length === 0) return false;
return transaction(async (client) => {
const removed = await client.query<{ tenant_id: string; service_id: string; is_default: boolean }>(
'DELETE FROM connectors WHERE id = $1 AND tenant_id = ANY($2) RETURNING tenant_id, service_id, is_default',
[id, tenantIds],
);
const row = removed.rows[0];
if (!row) return false;
// Kdyz zmizel vychozi, prevezme to prvni zbyly - jinak by kroky bez vyberu
// prestaly fungovat, aniz by se cokoliv jineho zmenilo.
if (row.is_default) {
await client.query(
`UPDATE connectors SET is_default = true
WHERE id = (
SELECT id FROM connectors
WHERE tenant_id = $1 AND service_id = $2
ORDER BY name LIMIT 1
)`,
[row.tenant_id, row.service_id],
);
}
return true;
});
},
async setStatus(id, status, error, tenantIds) {
if (tenantIds.length === 0) return undefined;
const row = await queryOne<ConnectorRow>(
`UPDATE connectors
SET status = $3, last_error = $4, last_check_at = now(), updated_at = now()
WHERE id = $1 AND tenant_id = ANY($2)
RETURNING ${COLUMNS}`,
[id, tenantIds, status, error],
);
return row ? toConnector(row) : undefined;
},
};
+180
View File
@@ -0,0 +1,180 @@
/**
* Konektor = napojeni jedne firmy na jednu sluzbu.
*
* Sluzba (`services.ts`) rika, co je potreba: "iDoklad chce hlavicky
* X-ClientId a X-ClientSecret". Konektor drzi hodnoty. Kazda firma ma svoje.
*
* Tenhle soubor je jen model a to, co plati v obou ulozistich. Samotna
* uloziste jsou vedle: `memory.ts` a `postgres.ts`.
*/
import { findService, type Service, type ServiceCredentialField } from '../services.js';
export interface Connector {
id: string;
tenantId: string;
serviceId: string;
/** Firma muze mit dva ucty teze sluzby, proto vlastni nazev. */
name: string;
/** Prepis adresy sluzby. null = vychozi podle `appId` a SERVICES_BASE_URL. */
baseUrl: string | null;
/** Hodnoty poli z `Service.credentials`. Tajne se z API nikdy nevraci. */
values: Record<string, string>;
enabled: boolean;
/** untested = jeste se neoverovalo, ok = posledni overeni proslo. */
status: 'untested' | 'ok' | 'error';
lastCheckAt: string | null;
lastError: string | null;
/** Krok stromu bez vybraneho konektoru pouzije vychozi. */
isDefault: boolean;
createdAt: string;
updatedAt: string;
}
/**
* Verze pro klienta. **Tajne hodnoty tady nejsou** a nikdy nesmi byt -
* secrets se nevraci z beznych endpointu (AGENTS.md).
*/
export interface PublicConnector {
id: string;
tenantId: string;
serviceId: string;
name: string;
baseUrl: string | null;
enabled: boolean;
status: Connector['status'];
lastCheckAt: string | null;
lastError: string | null;
isDefault: boolean;
createdAt: string;
updatedAt: string;
/** ID poli, ktera jsou vyplnena. Hodnoty se nevraci. */
filled: string[];
/** ID povinnych poli, ktera jeste chybi. */
missing: string[];
/** Necitliva nastaveni. Tajna pole tu nejsou vubec. */
config: Record<string, string>;
/** true = vsechna povinna pole jsou vyplnena, jde volat. */
ready: boolean;
}
export interface CreateConnectorInput {
tenantId: string;
serviceId: string;
name: string;
baseUrl?: string | null;
values?: Record<string, string>;
isDefault?: boolean;
}
export interface UpdateConnectorInput {
name?: string;
baseUrl?: string | null;
/**
* Jen pole, ktera se meni. Prazdny retezec hodnotu **smaze**, chybejici klic
* ji nechava. Diky tomu jde ulozit formular, ktery tajne hodnoty neposila.
*/
values?: Record<string, string>;
enabled?: boolean;
isDefault?: boolean;
}
export interface ConnectorIssue {
field: string;
message: string;
}
/** Rozhrani uloziste. Dve implementace, vyber je na jednom miste. */
export interface ConnectorRepository {
readonly kind: 'memory' | 'postgres';
list(tenantIds: string[], options?: { serviceId?: string }): Promise<Connector[]>;
get(id: string, tenantIds: string[]): Promise<Connector | undefined>;
defaultFor(tenantId: string, serviceId: string): Promise<Connector | undefined>;
countsByService(tenantIds: string[]): Promise<Map<string, number>>;
create(input: CreateConnectorInput): Promise<Connector>;
update(
id: string,
patch: UpdateConnectorInput,
tenantIds: string[],
): Promise<Connector | undefined>;
remove(id: string, tenantIds: string[]): Promise<boolean>;
setStatus(
id: string,
status: Connector['status'],
error: string | null,
tenantIds: string[],
): Promise<Connector | undefined>;
}
// ------------------------------------------------------------------- pomocne
export function nowIso(): string {
return new Date().toISOString();
}
/** Povinna pole, ktera nejsou vyplnena. */
export function missingFields(service: Service, values: Record<string, string>): string[] {
return service.credentials
.filter((field) => field.required && (values[field.id] ?? '').trim() === '')
.map((field) => field.id);
}
/**
* Prevod na verzi pro klienta.
*
* Jedno misto pro obe uloziste. Kdyby to bylo dvakrat, jednou by se zapomnelo
* vyhodit tajne hodnoty - a to je presne ta chyba, kterou nikdo nevidi.
*/
export function toPublicConnector(connector: Connector): PublicConnector {
const service = findService(connector.serviceId);
const credentials = service?.credentials ?? [];
const filled = Object.entries(connector.values)
.filter(([, value]) => value.trim() !== '')
.map(([key]) => key);
const config: Record<string, string> = {};
for (const field of credentials) {
if (field.secret) continue;
const value = connector.values[field.id];
if (value !== undefined && value.trim() !== '') config[field.id] = value;
}
const missing = service ? missingFields(service, connector.values) : [];
const { values: _values, ...rest } = connector;
return { ...rest, filled, missing, config, ready: missing.length === 0 };
}
/**
* Overi hodnoty proti tomu, co sluzba vyzaduje.
*
* Nevyplnene povinne pole **neni chyba** ukladani, ale nedodelek: konektor se
* ulozi a jen nepujde pouzit. Stejny rezim jako u rozdelane automatizace.
*
* Chyba je jen pole, ktere sluzba vubec nema. To uz je rozbite napojeni.
*/
export function validateConnectorValues(
service: Service,
values: Record<string, string>,
): ConnectorIssue[] {
const known = new Map<string, ServiceCredentialField>(
service.credentials.map((field) => [field.id, field]),
);
const issues: ConnectorIssue[] = [];
for (const key of Object.keys(values)) {
if (!known.has(key)) {
issues.push({ field: key, message: `Služba ${service.name} pole „${key}" nemá.` });
}
}
return issues;
}
/** Ktera pole sluzby jsou tajna. Podle toho se rozhoduje o sifrovani. */
export function secretFieldIds(serviceId: string): Set<string> {
const service = findService(serviceId);
return new Set(
(service?.credentials ?? []).filter((field) => field.secret).map((field) => field.id),
);
}
+115
View File
@@ -0,0 +1,115 @@
/**
* Migrace schematu.
*
* Soubory `src/db/migrations/*.sql` se spousti v abecednim poradi, kazdy jednou.
* Co uz proslo, je v tabulce `schema_migrations`.
*
* Dve veci, na kterych to stoji:
*
* 1. **Poradovy zamek.** Pri rolling deployi startuje vic instanci naraz
* a bez zamku by migrace pustily vsechny. `pg_advisory_lock` zaridi, ze
* projede jedna a ostatni pockaji.
* 2. **Jeden soubor je jedna transakce.** Pri chybe se nic z nej neuplatni,
* takze nevznikne rozdelane schema, o kterem nikdo nevi.
*
* Migrace se **nikdy neupravuji zpetne**. Uz projely u nekoho jineho, takze
* zmena souboru znamena dve rozdilna schemata se stejnym cislem. Oprava je
* vzdy novy soubor.
*/
import fs from 'node:fs/promises';
import path from 'node:path';
import { fileURLToPath } from 'node:url';
import { db } from './pool.js';
/** Libovolne, ale stabilni cislo. Musi byt stejne ve vsech instancich. */
const LOCK_ID = 918_273_645;
const here = path.dirname(fileURLToPath(import.meta.url));
/**
* Slozka s migracemi.
*
* V nasazeni bezi zkompilovany kod z `dist/`, ale `.sql` soubory tsc nekopiruje.
* Zkousi se proto obe cesty - vedle prelozeneho souboru i v `src`.
*/
async function migrationsDir(): Promise<string> {
const candidates = [path.join(here, 'migrations'), path.resolve(here, '../../src/db/migrations')];
for (const candidate of candidates) {
try {
await fs.access(candidate);
return candidate;
} catch {
continue;
}
}
throw new Error(`Slozku s migracemi nelze najit. Zkouseno: ${candidates.join(', ')}`);
}
export interface MigrationResult {
applied: string[];
skipped: number;
}
export async function runMigrations(): Promise<MigrationResult> {
const pool = db();
const dir = await migrationsDir();
const files = (await fs.readdir(dir)).filter((name) => name.endsWith('.sql')).sort();
if (files.length === 0) {
console.warn(`[db] ve ${dir} nejsou zadne migrace`);
return { applied: [], skipped: 0 };
}
const client = await pool.connect();
const applied: string[] = [];
let skipped = 0;
try {
// Zamek drzi jedna instance, ostatni tady pockaji. Uvolni se s odpojenim.
await client.query('SELECT pg_advisory_lock($1)', [LOCK_ID]);
await client.query(`
CREATE TABLE IF NOT EXISTS schema_migrations (
name text PRIMARY KEY,
applied_at timestamptz NOT NULL DEFAULT now()
)
`);
const done = new Set(
(await client.query<{ name: string }>('SELECT name FROM schema_migrations')).rows.map(
(row) => row.name,
),
);
for (const file of files) {
if (done.has(file)) {
skipped += 1;
continue;
}
const sql = await fs.readFile(path.join(dir, file), 'utf8');
try {
await client.query('BEGIN');
await client.query(sql);
await client.query('INSERT INTO schema_migrations (name) VALUES ($1)', [file]);
await client.query('COMMIT');
} catch (err) {
await client.query('ROLLBACK').catch(() => undefined);
// Rozbita migrace nesmi projit potichu. Bez schematu nema smysl bezet.
const message = err instanceof Error ? err.message : String(err);
throw new Error(`Migrace ${file} selhala: ${message}`);
}
applied.push(file);
console.info(`[db] migrace ${file} proslá`);
}
} finally {
await client.query('SELECT pg_advisory_unlock($1)', [LOCK_ID]).catch(() => undefined);
client.release();
}
if (applied.length === 0) console.info(`[db] schema je aktualni (${skipped} migraci)`);
return { applied, skipped };
}
+41
View File
@@ -0,0 +1,41 @@
-- Konektory: napojeni jedne firmy na jednu sluzbu.
--
-- Sluzby zustavaji v katalogu v kodu (src/data/services.ts). Do databaze
-- nepatri: jsou to definice, ktere delame my, a repo je u nich zdroj pravdy
-- kvuli code review a historii v gitu. Viz documentation/12.
--
-- `tenant_id` je povinne u kazde business tabulky a indexy zacinaji jim.
-- Hodnoty pristupovych udaju jsou sifrovane, tabulka nikdy nedrzi plaintext.
CREATE TABLE IF NOT EXISTS connectors (
id text PRIMARY KEY,
tenant_id text NOT NULL,
service_id text NOT NULL,
name text NOT NULL,
base_url text,
-- Sifrovane hodnoty poli podle Service.credentials.
-- Klic je id pole, hodnota je obalka se sifrou (viz src/db/secretBox.ts).
-- JSONB, protoze se cte a zapisuje cele a nikdo se nad tim nedotazuje po polich.
secrets jsonb NOT NULL DEFAULT '{}'::jsonb,
enabled boolean NOT NULL DEFAULT true,
status text NOT NULL DEFAULT 'untested',
last_check_at timestamptz,
last_error text,
is_default boolean NOT NULL DEFAULT false,
created_at timestamptz NOT NULL DEFAULT now(),
updated_at timestamptz NOT NULL DEFAULT now(),
CONSTRAINT connectors_status_check
CHECK (status IN ('untested', 'ok', 'error'))
);
-- Seznam konektoru firmy je nejcastejsi dotaz, proto index vede tenant_id.
CREATE INDEX IF NOT EXISTS connectors_tenant_service_idx
ON connectors (tenant_id, service_id);
-- Vychozi konektor smi byt na dvojici firma a sluzba jen jeden. Vynuceno
-- databazi, ne jen kodem: bez toho by soubezne dva zapisy udelaly dva vychozi
-- a krok bez vybraneho konektoru by si vybiral podle nahody.
CREATE UNIQUE INDEX IF NOT EXISTS connectors_one_default_idx
ON connectors (tenant_id, service_id)
WHERE is_default;
+164
View File
@@ -0,0 +1,164 @@
/**
* Pripojeni do Postgresu.
*
* Databaze je **volitelna**. Kdyz `DATABASE_URL` chybi, aplikace nastartuje
* a jede v pameti procesu. Nemuze byt jinak: container, ktery nenastartuje,
* je pro AppFactory nefunkcni sluzba (AGENTS.md).
*
* Rozdil mezi obema rezimy se resi na **jednom miste**, a to pri vyberu
* implementace uloziste (`src/data/connectorStore.ts`). Nikde jinde se
* nezjistuje, jestli databaze je - jinak by se to rozlezlo po celem kodu
* a jedno misto by se zapomnelo.
*/
import { Pool, type PoolClient, type QueryResultRow } from 'pg';
import { config } from '../config.js';
let pool: Pool | null = null;
/** true = mame kam ukladat. Rozhoduje se podle toho vyber uloziste. */
export function isDatabaseEnabled(): boolean {
return config.databaseUrl !== '';
}
/**
* Pool. Vytvori se az pri prvnim pouziti, aby si aplikace bez databaze
* nezakladala spojeni, ktere nikdy nepouzije.
*/
export function db(): Pool {
if (!isDatabaseEnabled()) {
throw new Error('DATABASE_URL neni nastavena, databaze se nesmi pouzivat.');
}
if (!pool) {
pool = new Pool({
connectionString: config.databaseUrl,
max: config.databasePoolMax,
// Kratky timeout na ziskani spojeni. Radsi hlasnou chybu nez visici request.
connectionTimeoutMillis: 5_000,
idleTimeoutMillis: 30_000,
...(config.databaseSsl ? { ssl: { rejectUnauthorized: false } } : {}),
});
// Chyba na necinnem spojeni nesmi shodit proces. Pool si spojeni obnovi sam.
pool.on('error', (err) => {
console.error('[db] chyba na necinnem spojeni:', err.message);
});
console.info(`[db] pool vytvoren, max ${config.databasePoolMax} spojeni`);
}
return pool;
}
/**
* Dotaz.
*
* Zamerne tenka obalka, ne query builder. Cely projekt je psany tak, ze server
* je autorita a filtr na firmu je povinny argument - to se hlida lip nad
* viditelnym SQL nez pod nadstavbou.
*/
export async function query<T extends QueryResultRow>(
sql: string,
params: unknown[] = [],
): Promise<T[]> {
const result = await db().query<T>(sql, params);
return result.rows;
}
/** Prvni radek, nebo undefined. */
export async function queryOne<T extends QueryResultRow>(
sql: string,
params: unknown[] = [],
): Promise<T | undefined> {
const rows = await query<T>(sql, params);
return rows[0];
}
/**
* Transakce. Pri vyjimce se vraci zpatky.
*
* Az bude runtime automatizaci, bude tohle to podstatne: vysledek kroku
* a zarazeni dalsiho ukolu musi byt jedna transakce, jinak vznikne beh
* s hotovym krokem a bez pokracovani (viz documentation/10).
*/
export async function transaction<T>(work: (client: PoolClient) => Promise<T>): Promise<T> {
const client = await db().connect();
try {
await client.query('BEGIN');
const result = await work(client);
await client.query('COMMIT');
return result;
} catch (err) {
await client.query('ROLLBACK').catch(() => undefined);
throw err;
} finally {
client.release();
}
}
/**
* Spojeni pro danou firmu.
*
* Dnes vraci vzdy tentyz pool. Je to zamerny sev: jednou prijde klient, ktery
* bude chtit vlastni databazi nebo bude delat tricet procent provozu, a presun
* ma byt konfigurace, ne prepisovani dotazu. Podminka je nikdy nespojovat
* dotazem dva klienty, coz uz vynucuje povinny argument `tenantIds`
* v ulozistich. Podrobnosti v documentation/10-runtime-a-kapacita.md.
*/
export function dbFor(_tenantId: string): Pool {
return db();
}
export interface DatabaseHealth {
enabled: boolean;
ok: boolean;
latencyMs: number | null;
error: string | null;
}
let cachedHealth: { at: number; value: DatabaseHealth } | null = null;
const HEALTH_CACHE_MS = 5_000;
/**
* Stav databaze pro `/health/ready`.
*
* Vysledek se par sekund cachuje, aby monitoring nedelal dotaz pri kazdem
* pingu. `/health` na databazi zamerne nezavisi - kratky vypadek DB by jinak
* vedl k restartovani containeru, coz nic nespravi.
*/
export async function databaseHealth(): Promise<DatabaseHealth> {
if (!isDatabaseEnabled()) {
return { enabled: false, ok: true, latencyMs: null, error: null };
}
if (cachedHealth && Date.now() - cachedHealth.at < HEALTH_CACHE_MS) {
return cachedHealth.value;
}
const startedAt = Date.now();
let value: DatabaseHealth;
try {
await query('select 1');
value = { enabled: true, ok: true, latencyMs: Date.now() - startedAt, error: null };
} catch (err) {
value = {
enabled: true,
ok: false,
latencyMs: null,
error: err instanceof Error ? err.message : String(err),
};
}
cachedHealth = { at: Date.now(), value };
return value;
}
/** Zavre pool pri ukonceni procesu. */
export async function closeDatabase(): Promise<void> {
if (!pool) return;
await pool.end().catch((err: unknown) => {
console.warn('[db] pool se nepodarilo zavrit:', err);
});
pool = null;
}
+123
View File
@@ -0,0 +1,123 @@
/**
* Sifrovani pristupovych udaju konektoru.
*
* Do databaze nikdy nesmi plaintext. Kdyby ano, tak by kazda zaloha, kazdy dump
* pri ladeni a kazdy, kdo ma pristup ke cteni, mel klientske klice k iDokladu.
*
* AES-256-GCM: sifruje a zaroven overuje, ze se s daty nikdo nehral. Nahodne
* IV pro kazdou hodnotu, aby dve stejne hodnoty nedaly stejnou sifru.
*
* `v` je verze klice. Vymena klice pak znamena precist starym, zapsat novym,
* ne zahodit vsechna napojeni.
*/
import { createCipheriv, createDecipheriv, createHash, randomBytes } from 'node:crypto';
import { config } from '../config.js';
/** Obalka, ktera se uklada do JSONB. */
export interface SealedValue {
/** Verze klice. */
v: number;
/** Inicializacni vektor, base64url. */
iv: string;
/** Autentizacni tag GCM, base64url. */
tag: string;
/** Sifrovana hodnota, base64url. */
data: string;
}
const ALGORITHM = 'aes-256-gcm';
const KEY_VERSION = 1;
/**
* Klic z konfigurace, srovnany na 32 bajtu.
*
* SHA-256 z hodnoty promenne, aby fungoval jakkoliv dlouhy retezec. Neni to
* derivace hesla (na to by patril scrypt), ale `SECRETS_KEY` ma byt nahodny
* klic, ne heslo - a to je v dokumentaci napsane.
*/
function key(): Buffer {
if (config.secretsKey === '') {
throw new Error('SECRETS_KEY neni nastavena, pristupove udaje nelze sifrovat.');
}
return createHash('sha256').update(config.secretsKey).digest();
}
export function canSealSecrets(): boolean {
return config.secretsKey !== '';
}
export function seal(value: string): SealedValue {
const iv = randomBytes(12);
const cipher = createCipheriv(ALGORITHM, key(), iv);
const data = Buffer.concat([cipher.update(value, 'utf8'), cipher.final()]);
return {
v: KEY_VERSION,
iv: iv.toString('base64url'),
tag: cipher.getAuthTag().toString('base64url'),
data: data.toString('base64url'),
};
}
/**
* Rozsifruje hodnotu.
*
* Vraci `null`, kdyz to nejde - typicky po vymene klice bez prevodu dat.
* Zamerne se **nepada**: jeden nerozsifrovatelny konektor nesmi shodit seznam
* vsech ostatnich. Chybejici hodnota se pak chova jako nevyplnena, takze
* uzivatel dostane "chybi udaje" a muze je zadat znovu.
*/
export function open(sealed: unknown): string | null {
if (!isSealed(sealed)) return null;
try {
const decipher = createDecipheriv(ALGORITHM, key(), Buffer.from(sealed.iv, 'base64url'));
decipher.setAuthTag(Buffer.from(sealed.tag, 'base64url'));
const plain = Buffer.concat([
decipher.update(Buffer.from(sealed.data, 'base64url')),
decipher.final(),
]);
return plain.toString('utf8');
} catch (err) {
console.error(
`[secrets] hodnotu nelze rozsifrovat (verze klice ${sealed.v}): ` +
(err instanceof Error ? err.message : String(err)),
);
return null;
}
}
function isSealed(value: unknown): value is SealedValue {
if (value === null || typeof value !== 'object') return false;
const record = value as Record<string, unknown>;
return (
typeof record.iv === 'string' &&
typeof record.tag === 'string' &&
typeof record.data === 'string' &&
typeof record.v === 'number'
);
}
/** Zasifruje celou sadu hodnot. */
export function sealAll(values: Record<string, string>): Record<string, SealedValue> {
const result: Record<string, SealedValue> = {};
for (const [id, value] of Object.entries(values)) {
if (value === '') continue;
result[id] = seal(value);
}
return result;
}
/** Rozsifruje celou sadu. Co nejde rozsifrovat, se vynecha a zaloguje. */
export function openAll(sealed: unknown): Record<string, string> {
if (sealed === null || typeof sealed !== 'object') return {};
const result: Record<string, string> = {};
for (const [id, value] of Object.entries(sealed as Record<string, unknown>)) {
const plain = open(value);
if (plain !== null) result[id] = plain;
else console.warn(`[secrets] pole ${id} se nepodarilo rozsifrovat, chova se jako nevyplnene`);
}
return result;
}
+56
View File
@@ -16,6 +16,9 @@ import { contactRouter } from './routes/contact.js';
import { dashboardRouter } from './routes/dashboard.js';
import { simulateRouter } from './routes/simulate.js';
import { webhookRouter } from './routes/webhook.js';
import { initConnectorStore, storageStatus } from './data/connectorStore.js';
import { runMigrations } from './db/migrate.js';
import { closeDatabase, databaseHealth, isDatabaseEnabled } from './db/pool.js';
import { ensureLoaded, scriptsDir } from './scripts/registry.js';
const here = path.dirname(fileURLToPath(import.meta.url));
@@ -62,10 +65,30 @@ function isOriginAllowed(origin: string): boolean {
*/
const api = Router({ strict: true });
/**
* Liveness. Zamerne **nezavisi na databazi**: kratky vypadek DB by jinak vedl
* k restartovani containeru, coz nic nespravi (AGENTS.md).
*/
api.get('/health', (_req, res) => {
res.json({ status: 'ok', uptimeSec: Math.round(process.uptime()) });
});
/**
* Readiness. Tady uz databaze zalezi, a proto je to zvlast.
* Vysledek se par sekund cachuje, aby monitoring nedelal dotaz pri kazdem pingu.
*/
api.get('/health/ready', async (_req, res) => {
const database = await databaseHealth();
const storage = storageStatus();
const ready = !database.enabled || database.ok;
res.status(ready ? 200 : 503).json({
status: ready ? 'ok' : 'degraded',
database,
storage,
});
});
/**
* Swagger UI. Cesta bez lomitka presmerujeme na variantu s lomitkem,
* jinak by se relativni odkazy na CSS a JS skladaly o uroven vys
@@ -176,14 +199,47 @@ app.use((err: unknown, _req: Request, res: Response, _next: NextFunction) => {
*/
await ensureLoaded(true);
/**
* Databaze a volba uloziste.
*
* Chybejici nebo nedostupna databaze **nesmi shodit start**. Aplikace pak jede
* v pameti procesu a rekne to nahlas - v logu i v portalu. Container, ktery
* nenastartuje, je pro AppFactory nefunkcni sluzba.
*/
let databaseReady = true;
if (isDatabaseEnabled()) {
try {
await runMigrations();
} catch (err) {
databaseReady = false;
console.error(
'[db] migrace selhaly, aplikace pojede v pameti procesu:',
err instanceof Error ? err.message : err,
);
}
}
await initConnectorStore({ databaseReady });
// Poslouchat na vsech rozhranich containeru, ne jen na localhost (AGENTS.md).
const server = app.listen(config.port, '0.0.0.0', () => {
console.info(`[start] csbot-prototype bezi na portu ${config.port}`);
console.info(`[start] ROOT_PATH: ${config.rootPath || '(neni nastaven)'}`);
console.info(`[start] health: ${config.rootPath}/health, docs: ${config.rootPath}/docs`);
console.info(`[start] skripty konektoru: ${scriptsDir()}`);
const storage = storageStatus();
console.info(`[start] uloziste konektoru: ${storage.mode}${storage.reason ? ` (${storage.reason})` : ''}`);
});
// Pool se pri ukonceni zavre, at se spojeni neopousti otevrena.
for (const signal of ['SIGTERM', 'SIGINT'] as const) {
process.on(signal, () => {
console.info(`[stop] ${signal}, ukoncuji`);
server.close(() => {
void closeDatabase().finally(() => process.exit(0));
});
});
}
server.on('error', (err: NodeJS.ErrnoException) => {
if (err.code === 'EADDRINUSE') {
console.error(`[start] Port ${config.port} je obsazeny.`);
+40
View File
@@ -987,6 +987,46 @@ export function buildOpenApiDocument() {
responses: { '200': { description: 'Proud udalosti text/event-stream' } },
},
},
'/health/ready': {
get: {
tags: ['Provoz'],
summary: 'Readiness vcetne databaze',
description:
'Vraci 503, kdyz je databaze nastavena a nedostupna. `/health` na databazi ' +
'zamerne nezavisi - kratky vypadek DB by jinak vedl k restartovani containeru.',
responses: {
'200': { description: 'Aplikace je pripravena' },
'503': { description: 'Databaze je nastavena, ale nedostupna' },
},
},
},
'/api/dashboard/storage': {
get: {
tags: ['Dashboard'],
summary: 'Kam se uklada',
description:
'mode postgres nebo memory. `ephemeral: true` znamena, ze restart procesu ' +
'data smaze. Portal to musi umet rict nahlas.',
security: [{ bearerAuth: [] }],
responses: {
'200': {
description: 'Rezim uloziste',
content: {
'application/json': {
schema: {
type: 'object',
properties: {
mode: { type: 'string', enum: ['postgres', 'memory'] },
reason: { type: 'string', nullable: true },
ephemeral: { type: 'boolean' },
},
},
},
},
},
},
},
},
'/api/dashboard/services': {
get: {
tags: ['Sluzby'],
+19 -17
View File
@@ -91,14 +91,16 @@ function serviceOrDeny(
* pozna rozdil mezi "napojeno" a "muzete si napojit". Neni to vlastnost sluzby,
* ale te firmy, proto se to pocita tady a ne v katalogu.
*/
connectorsRouter.get('/services', (req, res) => {
connectorsRouter.get('/services', async (req, res) => {
const access = accessFor(req.user!);
const requested = typeof req.query.tenantId === 'string' ? req.query.tenantId : undefined;
const tenantId = requested ?? access.defaultTenantId;
const visible = visibleServices(req.user!, tenantId);
const withScripts = new Map(serviceCatalog().map((service) => [service.id, service]));
const counts = tenantId ? connectorCountsByService([tenantId]) : new Map<string, number>();
const counts = tenantId
? await connectorCountsByService([tenantId])
: new Map<string, number>();
const items = visible.map((service) => {
const merged = withScripts.get(service.id) ?? service;
@@ -118,12 +120,12 @@ connectorsRouter.get('/services', (req, res) => {
// ----------------------------------------------------------------- konektory
connectorsRouter.get('/', (req, res) => {
connectorsRouter.get('/', async (req, res) => {
const tenantId = tenantOrDeny(req, res);
if (!tenantId) return;
const serviceId = typeof req.query.serviceId === 'string' ? req.query.serviceId : undefined;
const items = listConnectors([tenantId], serviceId ? { serviceId } : {})
const items = (await listConnectors([tenantId], serviceId ? { serviceId } : {}))
// Konektor na sluzbu, kterou uzivatel nevidi, mu taky ukazovat nebudeme.
.filter((connector) => {
const service = findService(connector.serviceId);
@@ -134,11 +136,11 @@ connectorsRouter.get('/', (req, res) => {
return res.json({ items, tenantId });
});
connectorsRouter.get('/:id', (req, res) => {
connectorsRouter.get('/:id', async (req, res) => {
const tenantId = tenantOrDeny(req, res);
if (!tenantId) return;
const connector = getConnector(req.params.id, [tenantId]);
const connector = await getConnector(req.params.id, [tenantId]);
if (!connector) {
return res.status(404).json({ error: 'not_found', message: 'Konektor neexistuje.' });
}
@@ -154,7 +156,7 @@ const createSchema = z.object({
values: z.record(z.string()).optional(),
});
connectorsRouter.post('/', (req, res) => {
connectorsRouter.post('/', async (req, res) => {
const tenantId = tenantOrDeny(req, res);
if (!tenantId) return;
@@ -181,7 +183,7 @@ connectorsRouter.post('/', (req, res) => {
return res.status(400).json({ error: 'validation_error', message: issues[0].message, issues });
}
const connector = createConnector({
const connector = await createConnector({
tenantId,
serviceId: service.id,
name: parsed.data.name,
@@ -201,11 +203,11 @@ const updateSchema = z.object({
isDefault: z.literal(true).optional(),
});
connectorsRouter.patch('/:id', (req, res) => {
connectorsRouter.patch('/:id', async (req, res) => {
const tenantId = tenantOrDeny(req, res);
if (!tenantId) return;
const existing = getConnector(req.params.id, [tenantId]);
const existing = await getConnector(req.params.id, [tenantId]);
if (!existing) {
return res.status(404).json({ error: 'not_found', message: 'Konektor neexistuje.' });
}
@@ -228,18 +230,18 @@ connectorsRouter.patch('/:id', (req, res) => {
}
}
const updated = updateConnector(req.params.id, parsed.data, [tenantId]);
const updated = await updateConnector(req.params.id, parsed.data, [tenantId]);
if (!updated) {
return res.status(404).json({ error: 'not_found', message: 'Konektor neexistuje.' });
}
return res.json(toPublicConnector(updated));
});
connectorsRouter.delete('/:id', (req, res) => {
connectorsRouter.delete('/:id', async (req, res) => {
const tenantId = tenantOrDeny(req, res);
if (!tenantId) return;
if (!deleteConnector(req.params.id, [tenantId])) {
if (!(await deleteConnector(req.params.id, [tenantId]))) {
return res.status(404).json({ error: 'not_found', message: 'Konektor neexistuje.' });
}
return res.status(204).end();
@@ -258,7 +260,7 @@ connectorsRouter.post('/:id/test', async (req, res) => {
const tenantId = tenantOrDeny(req, res);
if (!tenantId) return;
const connector = getConnector(req.params.id, [tenantId]);
const connector = await getConnector(req.params.id, [tenantId]);
if (!connector) {
return res.status(404).json({ error: 'not_found', message: 'Konektor neexistuje.' });
}
@@ -267,7 +269,7 @@ connectorsRouter.post('/:id/test', async (req, res) => {
const target = resolveTarget(service.id, connector);
if (!target.ready) {
setConnectorStatus(connector.id, 'error', target.missing.join(', '), [tenantId]);
await setConnectorStatus(connector.id, 'error', target.missing.join(', '), [tenantId]);
return res.json({
ok: false,
checked: 'nic',
@@ -293,7 +295,7 @@ connectorsRouter.post('/:id/test', async (req, res) => {
});
const response = await http.get(path);
setConnectorStatus(connector.id, 'ok', null, [tenantId]);
await setConnectorStatus(connector.id, 'ok', null, [tenantId]);
return res.json({
ok: true,
@@ -314,7 +316,7 @@ connectorsRouter.post('/:id/test', async (req, res) => {
: undefined
: redact(truncate(String(err instanceof Error ? err.stack ?? err.message : err), config.errorDetailBytes));
setConnectorStatus(connector.id, 'error', message, [tenantId]);
await setConnectorStatus(connector.id, 'error', message, [tenantId]);
console.warn(
`[connectors] test ${connector.id} selhal: ${message}` + (detail ? `
${detail}` : ''),
+21 -10
View File
@@ -20,6 +20,7 @@ import { operatorAllowedForType, operatorsByType } from '../data/conditions.js';
import {
defaultConnectorFor,
getConnector,
storageStatus,
} from '../data/connectorStore.js';
import {
findOperation,
@@ -91,6 +92,16 @@ dashboardRouter.get('/summary', (req, res) => {
return res.json(getSummary(scope.tenantIds));
});
/**
* Kam se uklada a jestli to prezije restart.
*
* Portal to musi umet rict nahlas. Bez toho se clovek divi, kam se podely
* jeho konektory, a hleda chybu v aplikaci - presne to se stalo.
*/
dashboardRouter.get('/storage', (_req, res) => {
res.json(storageStatus());
});
dashboardRouter.get('/incidents', (_req, res) => {
res.json({ items: listIncidents() });
});
@@ -486,10 +497,10 @@ function normalizeTriggerFields(flow: z.infer<typeof flowSchema>): z.infer<typeo
* `tenantIds` je potreba kvuli konektorum: cizi konektor se musi chovat jako
* neexistujici, jinak by strom mohl volat cizim jmenem.
*/
function validateFlowReferences(
async function validateFlowReferences(
flow: z.infer<typeof flowSchema>,
tenantIds: string[],
): { problems: string[]; issues: string[] } {
): Promise<{ problems: string[]; issues: string[] }> {
const problems: string[] = [];
/** Nedodelky: strom se ulozi, jen automatizace nepujde zapnout. */
const issues: string[] = [];
@@ -516,7 +527,7 @@ function validateFlowReferences(
// jen na parametry, ktere pred ni uz vznikly.
const scopes = collectScopes(flow);
const walk = (steps: FlowStep[]) => {
const walk = async (steps: FlowStep[]): Promise<void> => {
for (const step of steps) {
if (step.kind === 'condition') {
const field = scopes.all.get(step.fieldId);
@@ -527,8 +538,8 @@ function validateFlowReferences(
`Operátor "${step.operator}" nelze použít na parametr „${field.name}" typu ${field.type}.`,
);
}
walk(step.yes);
walk(step.no);
await walk(step.yes);
await walk(step.no);
continue;
}
const action = findOperation(step.serviceId, step.operationId, 'action');
@@ -574,7 +585,7 @@ function validateFlowReferences(
// Vybrany konektor musi patrit te same firme a te same sluzbe.
// Cizi konektor je rozbity strom, ne nedodelek.
if (step.connectorId) {
const connector = getConnector(step.connectorId, tenantIds);
const connector = await getConnector(step.connectorId, tenantIds);
if (!connector) {
problems.push(`Krok odkazuje na konektor, který neexistuje (${step.connectorId}).`);
} else if (connector.serviceId !== step.serviceId) {
@@ -586,7 +597,7 @@ function validateFlowReferences(
const service = findService(step.serviceId);
// Chybejici napojeni je nedodelek, ne chyba - rozdelana prace se ulozi.
if (service && !service.general && tenantIds.length === 1) {
const fallback = defaultConnectorFor(tenantIds[0], step.serviceId);
const fallback = await defaultConnectorFor(tenantIds[0], step.serviceId);
if (!fallback) {
issues.push(
`Služba ${service.name} nemá v této firmě konektor. Vytvořte ho v Konektorech.`,
@@ -596,7 +607,7 @@ function validateFlowReferences(
}
}
};
walk(flow.steps);
await walk(flow.steps);
return { problems, issues };
}
@@ -638,7 +649,7 @@ dashboardRouter.post('/automations', (req, res) => {
return res.status(201).json(automation);
});
dashboardRouter.put('/automations/:id', (req, res) => {
dashboardRouter.put('/automations/:id', async (req, res) => {
const parsed = updateSchema.safeParse(req.body);
if (!parsed.success) {
return res.status(400).json({
@@ -653,7 +664,7 @@ dashboardRouter.put('/automations/:id', (req, res) => {
let connectorIssues: string[] = [];
if (flow) {
const { problems, issues } = validateFlowReferences(flow, writableTenants(req));
const { problems, issues } = await validateFlowReferences(flow, writableTenants(req));
if (problems.length > 0) {
console.warn(`[automations] ${req.params.id}: neplatny strom - ${problems.join(' ')}`);
return res.status(400).json({
+9 -10
View File
@@ -50,14 +50,13 @@ scriptsRouter.get('/', async (req, res) => {
access.defaultTenantId;
// Ke kazde sluzbe vychozi konektor firmy, aby portal poznal, jestli je cim volat.
const connectors = Object.fromEntries(
items
.map((item) => {
const connector = tenantId ? defaultConnectorFor(tenantId, item.serviceId) : undefined;
return connector ? [item.serviceId, toPublicConnector(connector)] : null;
})
.filter((entry): entry is [string, ReturnType<typeof toPublicConnector>] => entry !== null),
const pairs = await Promise.all(
items.map(async (item) => {
const connector = tenantId ? await defaultConnectorFor(tenantId, item.serviceId) : undefined;
return connector ? ([item.serviceId, toPublicConnector(connector)] as const) : null;
}),
);
const connectors = Object.fromEntries(pairs.filter((entry) => entry !== null));
res.json({
items,
@@ -93,7 +92,7 @@ scriptsRouter.get('/:id', async (req, res) => {
const tenantId =
(typeof req.query.tenantId === 'string' ? req.query.tenantId : undefined) ??
access.defaultTenantId;
const connector = tenantId ? defaultConnectorFor(tenantId, serviceId) : undefined;
const connector = tenantId ? await defaultConnectorFor(tenantId, serviceId) : undefined;
return res.json({
id,
@@ -182,8 +181,8 @@ scriptsRouter.post('/:id/test', requirePlatformAdmin, async (req, res) => {
// Konektor musi patrit te same firme. Cizi se chova jako neexistujici.
const connector = parsed.data.connectorId
? getConnector(parsed.data.connectorId, [tenantId])
: defaultConnectorFor(tenantId, serviceId);
? await getConnector(parsed.data.connectorId, [tenantId])
: await defaultConnectorFor(tenantId, serviceId);
if (parsed.data.connectorId && !connector) {
return res.status(404).json({ error: 'not_found', message: 'Konektor neexistuje.' });
+20 -1
View File
@@ -1,4 +1,5 @@
import {
AlertTriangle,
CheckCircle2,
Eye,
EyeOff,
@@ -26,6 +27,7 @@ import type {
Service,
ServiceCredentialField,
ServiceOverview,
StorageStatus,
} from '@/types/dashboard';
/**
@@ -46,6 +48,7 @@ export default function Connectors() {
const [services, setServices] = useState<ServiceOverview | null>(null);
const [connectors, setConnectors] = useState<Connector[] | null>(null);
const [storage, setStorage] = useState<StorageStatus | null>(null);
const [error, setError] = useState<string | null>(null);
const [loading, setLoading] = useState(true);
@@ -60,10 +63,12 @@ export default function Connectors() {
Promise.all([
apiFetch<ServiceOverview>('/api/dashboard/connectors/services'),
apiFetch<{ items: Connector[] }>('/api/dashboard/connectors'),
apiFetch<StorageStatus>('/api/dashboard/storage'),
])
.then(([overview, list]) => {
.then(([overview, list, storageStatus]) => {
setServices(overview);
setConnectors(list.items);
setStorage(storageStatus);
})
.catch((err: unknown) => {
setError(err instanceof Error ? err.message : 'Konektory se nepodařilo načíst.');
@@ -111,6 +116,20 @@ export default function Connectors() {
)}
</header>
{/*
Kam se uklada. Bez teto hlasky se clovek divi, kam se podely jeho
konektory po restartu, a hleda chybu v aplikaci.
*/}
{storage?.ephemeral && (
<p className="flex items-start gap-2 rounded-card border border-warn-400/40 bg-warn-500/8 p-3 text-sm text-warn-400">
<AlertTriangle className="mt-0.5 size-4 shrink-0" />
<span>
Přístupové údaje se ukládají jen do paměti serveru a restart je smaže.
<span className="mt-0.5 block text-white/50">{storage.reason}</span>
</span>
</p>
)}
<DataState loading={loading} error={error} onRetry={load}>
{(connectors?.length ?? 0) === 0 ? (
<p className="py-10 text-center text-sm text-white/45">
+9
View File
@@ -292,6 +292,15 @@ export interface Connector {
ready: boolean;
}
/** Kam se uklada a jestli to prezije restart. */
export interface StorageStatus {
mode: 'postgres' | 'memory';
/** Proc se jede v pameti. null u databaze. */
reason: string | null;
/** true = restart procesu data smaze. */
ephemeral: boolean;
}
export interface ConnectorTestResult {
ok: boolean;
/** Co se vlastne overilo: pristupove udaje, nebo jen dostupnost sluzby. */