244 lines
11 KiB
Markdown
244 lines
11 KiB
Markdown
# Star Citizen Inventory
|
|
|
|
A Star Citizen item catalog built from the
|
|
[scunpacked](https://github.com/richardthombs/scunpacked) game data (mirrored at
|
|
[StarCitizenWiki/scunpacked-data](https://github.com/StarCitizenWiki/scunpacked-data)).
|
|
|
|
Ships, ship equipment, FPS items, and cargo commodities (ores, refined metals,
|
|
RMC, construction materials, …) are browsable with search, manufacturer / type /
|
|
size filters, pagination, and a detail view per item.
|
|
|
|
Users can optionally create an account and build a **personal inventory**: items
|
|
from the catalog are added to their own list with a quantity and notes. The
|
|
catalog itself stays immutable read-only reference data; personal inventories
|
|
live in PostgreSQL and reference catalog items by `class_name` only, so the
|
|
dataset can be replaced wholesale without touching user data.
|
|
|
|
## Stack
|
|
|
|
| Piece | Tech | Role |
|
|
| -------- | -------------------------------------- | ---------------------------------------------------------- |
|
|
| backend | FastAPI (python:3.12-slim container) | Serves the read-only catalog and the account/inventory API |
|
|
| frontend | nginx:1.27-alpine + vanilla JS/CSS | Inventory UI; proxies `/api/` and `/healthz` |
|
|
| database | PostgreSQL 16 | Accounts, sessions, and per-user inventory rows |
|
|
| deploy | Ansible + Podman (containers.podman) | Builds images and runs the three containers |
|
|
|
|
The backend image downloads the full scunpacked dataset at build time, then
|
|
reduces `items.json` to a small `commodities.json` (one entry per commodity) and
|
|
discards the ~68 MB raw file, so the catalog needs no database — only user data
|
|
does.
|
|
|
|
## Layout
|
|
|
|
```
|
|
playbook.yml Ansible deployment (hosts: webservers)
|
|
inventory.ini Inventory (localhost + commented remote example)
|
|
trade-hub/
|
|
podman-compose.yml Local three-service stack (db + backend + frontend)
|
|
backend/ FastAPI app, tests, data fetch script, Dockerfile
|
|
frontend/ nginx config + static inventory UI
|
|
```
|
|
|
|
## Run locally with Podman Compose
|
|
|
|
```bash
|
|
cd trade-hub
|
|
podman-compose up -d --build
|
|
# or: podman compose up -d --build
|
|
```
|
|
|
|
Then open http://localhost:8080
|
|
|
|
PostgreSQL data is kept in the `pgdata` volume. Set a non-default database
|
|
password before deploying anywhere real:
|
|
|
|
```bash
|
|
POSTGRES_PASSWORD='choose-something-strong' podman-compose up -d --build
|
|
```
|
|
|
|
## Deploy with Ansible
|
|
|
|
Deploys to every host in the `webservers` group (defaults to localhost):
|
|
|
|
```bash
|
|
POSTGRES_PASSWORD='choose-something-strong' ansible-playbook playbook.yml
|
|
```
|
|
|
|
To deploy to a remote server, edit `inventory.ini`:
|
|
|
|
```ini
|
|
[webservers]
|
|
server1 ansible_host=203.0.113.10 ansible_user=deploy
|
|
```
|
|
|
|
Then run the same command from this directory. The playbook:
|
|
|
|
1. installs podman if missing,
|
|
2. removes the legacy `my-apache-server` container if present,
|
|
3. builds `sc-inventory-backend` / `sc-inventory-frontend` images (skipped when
|
|
they already exist; set `-e force_rebuild=true` to rebuild),
|
|
4. creates the `sc-inventory` podman network and the `sc-inventory-data` volume,
|
|
5. runs PostgreSQL, the backend, and the frontend containers (only the frontend
|
|
publishes a port: 8080),
|
|
6. verifies `http://localhost:8080/healthz` returns 200.
|
|
|
|
## Reset for testing (remove everything)
|
|
|
|
```bash
|
|
# preview only, changes nothing
|
|
ansible-playbook teardown.yml -e confirm_destroy=true --check
|
|
|
|
# remove containers, all project images, the network and the database volume
|
|
ansible-playbook teardown.yml -e confirm_destroy=true
|
|
```
|
|
|
|
The teardown is scoped by name (`sc-inventory*`, `sc-inv-verify*`,
|
|
`trade-hub*`) and never runs `podman system prune`, so unrelated containers,
|
|
images and volumes on the host are left alone. It removes every tag of the
|
|
project images (so stale builds cannot resurface) and deletes the
|
|
`sc-inventory-data` volume — **all accounts and inventories are destroyed**.
|
|
Run `playbook.yml` afterwards to deploy a clean install; the images rebuild
|
|
from scratch.
|
|
|
|
## Accounts and personal inventory
|
|
|
|
- **Registration is open**: anyone who can reach the site can create an account.
|
|
- A new account starts with an **empty inventory**; nothing is seeded or shared
|
|
between accounts.
|
|
- Passwords are hashed with Argon2 and never returned by the API.
|
|
- Sessions are opaque random tokens delivered in an `httpOnly`, `SameSite=Lax`
|
|
cookie; only the SHA-256 of the token is stored server-side.
|
|
- Failed logins are throttled per username and per client IP, keyed on the
|
|
`X-Real-IP` header that nginx sets from the actual peer address.
|
|
- Mutating requests are rejected when the `Origin` header does not match the
|
|
request host.
|
|
- Catalog browsing works without an account; the catalog endpoints remain
|
|
read-only.
|
|
|
|
The inventory view can be filtered by category and searched by name/class
|
|
name, and sorted by `name` (default), `category`, `quantity` (high to low) or
|
|
`updated` (most recently changed first).
|
|
|
|
Inventory rows store `(user_id, class_name, quantity, notes, visibility)`.
|
|
Adding an item that is already present increments its quantity; quantity, notes
|
|
and visibility can then be edited or the row removed. Every added item starts
|
|
`private`; the Visibility toggle switches it to `available` and back. Responses
|
|
are enriched live from the catalog. If a catalog item disappears after a
|
|
dataset refresh, the row is reported with `available: false` instead of being
|
|
silently deleted (`available` is about the catalog, `visibility` is the owner's
|
|
classification).
|
|
|
|
The **Available** tab (backed by `GET /api/v1/available`) is public, like the
|
|
catalog: it lists every row marked `available` across all accounts with the
|
|
item's catalog details, quantity, owner username and last update. Private rows
|
|
never appear there and inventory notes are never exposed.
|
|
|
|
Clicking an item name in either **My Inventory** or **Available** opens the same
|
|
catalog detail view used by the catalog grid.
|
|
|
|
### Security: read before exposing this publicly
|
|
|
|
The stack publishes plain HTTP on port 8080. Session cookies therefore travel
|
|
in cleartext, and `COOKIE_SECURE` defaults to `false` so local development works.
|
|
Before putting this on the open internet, terminate TLS in front of the
|
|
frontend (Caddy, nginx with certificates, or a load balancer) and set
|
|
`COOKIE_SECURE=true` (compose env var) or `-e cookie_secure=true` (Ansible).
|
|
Also change `POSTGRES_PASSWORD` from its development default.
|
|
|
|
## Verify
|
|
|
|
```bash
|
|
curl -fsS http://localhost:8080/healthz
|
|
curl -fsS http://localhost:8080/api/v1/stats
|
|
curl -fsS "http://localhost:8080/api/v1/items?search=carrack"
|
|
```
|
|
|
|
Register, then use the cookie jar for authenticated calls:
|
|
|
|
```bash
|
|
curl -fsS -c cookies.txt -X POST http://localhost:8080/api/v1/auth/register \
|
|
-H 'Content-Type: application/json' \
|
|
-d '{"username":"pilot","password":"correct horse battery"}'
|
|
|
|
curl -fsS -b cookies.txt http://localhost:8080/api/v1/auth/me
|
|
|
|
curl -fsS -b cookies.txt -X POST http://localhost:8080/api/v1/me/inventory \
|
|
-H 'Content-Type: application/json' \
|
|
-d '{"class_name":"ANVL_Carrack","quantity":1,"notes":"pledged"}'
|
|
|
|
curl -fsS -b cookies.txt http://localhost:8080/api/v1/me/inventory
|
|
```
|
|
|
|
## Backend development
|
|
|
|
```bash
|
|
cd trade-hub/backend
|
|
python3 -m venv .venv
|
|
.venv/bin/pip install -r requirements.txt -r requirements-dev.txt
|
|
.venv/bin/python -m pytest tests/ -q # 65 tests, no database needed
|
|
DATA_DIR=./data/samples .venv/bin/uvicorn main:app --port 8000
|
|
```
|
|
|
|
Tests run against an in-memory SQLite database configured in `tests/conftest.py`,
|
|
so no PostgreSQL server is required for the suite. Production uses PostgreSQL.
|
|
|
|
- `data/samples/` — small real-schema subset for local dev/tests
|
|
- `scripts/fetch_data.py` — downloads the full dataset (used by the Dockerfile)
|
|
|
|
Commodities are deduplicated to one catalog entry per name (the 1 SCU container
|
|
variant is canonical); quantity lives on the inventory row, so 1 SCU, 8 SCU and
|
|
32 SCU crates of the same commodity share a single entry. Refined and unrefined
|
|
materials stay separate (`Iron` vs `Iron (Ore)`).
|
|
|
|
Tables are created on startup with an idempotent `create_all` and a connection
|
|
retry loop, so a fresh volume bootstraps itself and a slow PostgreSQL start does
|
|
not crash the backend. Additive columns for existing deployments (currently
|
|
`inventory_entries.visibility`) are applied automatically on startup; changing
|
|
or dropping a column still requires Alembic.
|
|
|
|
## API
|
|
|
|
```
|
|
GET /healthz
|
|
GET /api/v1/stats
|
|
GET /api/v1/manufacturers
|
|
GET /api/v1/items?category=&search=&manufacturer=&type=&size=&page=&page_size=
|
|
GET /api/v1/items/{class_name}
|
|
|
|
POST /api/v1/auth/register {username, password} -> 201 + session cookie
|
|
POST /api/v1/auth/login {username, password} -> 200 + session cookie
|
|
POST /api/v1/auth/logout -> 204
|
|
GET /api/v1/auth/me -> {user}
|
|
|
|
GET /api/v1/me/inventory?category=&search=&sort=&page=&page_size=
|
|
GET /api/v1/me/inventory/summary
|
|
POST /api/v1/me/inventory {class_name, quantity?, notes?}
|
|
PATCH /api/v1/me/inventory/{class_name} {quantity?, notes?, visibility?}
|
|
DELETE /api/v1/me/inventory/{class_name}
|
|
|
|
GET /api/v1/available?category=&search=&sort=&page=&page_size=
|
|
```
|
|
|
|
`/api/v1/me/*` requires a valid session cookie and returns `401` otherwise.
|
|
`/api/v1/available` is public and read-only; it accepts `sort=name|category|quantity|updated`
|
|
(default `updated`). Catalog `category` values are `ships`, `ship-items`,
|
|
`fps-items` and `commodities`.
|
|
|
|
### Configuration
|
|
|
|
| Variable | Default | Purpose |
|
|
| ---------------------- | ------------------------------------------------------------------------ | ----------------------------------------------- |
|
|
| `DATA_DIR` | `backend/data/samples` | Catalog JSON location |
|
|
| `DATABASE_URL` | `postgresql+psycopg://sc_inventory:sc_inventory_dev@db:5432/sc_inventory` | SQLAlchemy database URL |
|
|
| `AUTO_INIT_DB` | `true` | Create missing tables on startup |
|
|
| `DB_WAIT_SECONDS` | `60` | Startup wait for PostgreSQL |
|
|
| `COOKIE_SECURE` | `false` | Mark the session cookie `Secure` (use with TLS) |
|
|
| `COOKIE_NAME` | `sc_session` | Session cookie name |
|
|
| `SESSION_TTL_DAYS` | `30` | Absolute session lifetime |
|
|
| `CORS_ORIGINS` | *(empty)* | Comma-separated cross-origin opt-in |
|
|
| `LOGIN_MAX_ATTEMPTS` | `5` | Failed logins before throttling |
|
|
| `LOGIN_WINDOW_SECONDS` | `900` | Throttle window |
|
|
|
|
Data source: scunpacked — unofficial Star Citizen fansite content, not
|
|
affiliated with Cloud Imperium Games.
|