300 lines
14 KiB
Markdown
300 lines
14 KiB
Markdown
# Star Citizen Inventory
|
|
|
|
A Star Citizen item catalog built from the
|
|
[scunpacked](https://github.com/richardthombs/scunpacked) game data (mirrored at
|
|
[StarCitizenWiki/scunpacked-data](https://github.com/StarCitizenWiki/scunpacked-data)).
|
|
|
|
Ships, ship equipment, FPS items, and cargo commodities (ores, refined metals,
|
|
RMC, construction materials, …) are browsable with search, manufacturer / type /
|
|
size filters, pagination, and a detail view per item.
|
|
|
|
Users can optionally create an account and build a **personal inventory**: items
|
|
from the catalog are added to their own list with a quantity and notes. Logged-in
|
|
users can also place **orders** for any catalog item; orders go onto a public
|
|
board that every visitor can see. The catalog itself stays immutable read-only
|
|
reference data; personal inventories and orders live in PostgreSQL and reference
|
|
catalog items by `class_name` only, so the dataset can be replaced wholesale
|
|
without touching user data.
|
|
|
|
## Stack
|
|
|
|
| Piece | Tech | Role |
|
|
| -------- | -------------------------------------- | ---------------------------------------------------------- |
|
|
| backend | FastAPI (python:3.12-slim container) | Serves the read-only catalog and the account/inventory API |
|
|
| frontend | nginx:1.27-alpine + vanilla JS/CSS | Inventory UI; proxies `/api/` and `/healthz` |
|
|
| database | PostgreSQL 16 | Accounts, sessions, and per-user inventory rows |
|
|
| deploy | Ansible + Podman (containers.podman) | Builds images and runs the three containers |
|
|
|
|
The backend image downloads the full scunpacked dataset at build time, then
|
|
reduces `items.json` to a small `commodities.json` (one entry per commodity) and
|
|
discards the ~68 MB raw file, so the catalog needs no database — only user data
|
|
does.
|
|
|
|
## Layout
|
|
|
|
```
|
|
playbook.yml Example play using the role (hosts: webservers)
|
|
inventory.ini Inventory (localhost + commented remote example)
|
|
teardown.yml Removes this project's containers and data
|
|
roles/trade_hub/
|
|
README.md Role usage and variables
|
|
defaults/main.yml Configurable defaults
|
|
tasks/main.yml Stage source, build images, run containers
|
|
files/trade-hub/
|
|
podman-compose.yml Local three-service stack (db + backend + frontend)
|
|
backend/ FastAPI app, tests, data fetch script, Dockerfile
|
|
frontend/ nginx config + static inventory UI
|
|
```
|
|
|
|
The Ansible deployment is a self-contained role. To deploy it from an existing
|
|
playbook, copy `roles/trade_hub` into that repository's `roles/` directory and
|
|
add the role to the play:
|
|
|
|
```yaml
|
|
- role: trade_hub
|
|
tags: [trade_hub]
|
|
```
|
|
|
|
See `roles/trade_hub/README.md` for the variables. The play-level `become: true`
|
|
used by a typical server play makes podman run as root.
|
|
|
|
## Run locally with Podman Compose
|
|
|
|
```bash
|
|
cd roles/trade_hub/files/trade-hub
|
|
podman-compose up -d --build
|
|
# or: podman compose up -d --build
|
|
```
|
|
|
|
Then open http://localhost:8080
|
|
|
|
PostgreSQL data is kept in the `pgdata` volume. Set a non-default database
|
|
password before deploying anywhere real:
|
|
|
|
```bash
|
|
POSTGRES_PASSWORD='choose-something-strong' podman-compose up -d --build
|
|
```
|
|
|
|
## Deploy with Ansible
|
|
|
|
The example play deploys the role to every host in the `webservers` group
|
|
(defaults to localhost):
|
|
|
|
```bash
|
|
POSTGRES_PASSWORD='choose-something-strong' ansible-playbook playbook.yml
|
|
```
|
|
|
|
To deploy to a remote server, edit `inventory.ini`:
|
|
|
|
```ini
|
|
[webservers]
|
|
server1 ansible_host=203.0.113.10 ansible_user=deploy
|
|
```
|
|
|
|
Then run the same command from this directory. The role:
|
|
|
|
1. installs podman if missing (only when `trade_hub_install_podman: true`),
|
|
2. optionally removes the legacy `my-apache-server` container
|
|
(`trade_hub_remove_legacy_apache: true`),
|
|
3. stages the application source in `/opt/trade-hub` and builds the
|
|
`sc-inventory-backend` / `sc-inventory-frontend` images (skipped when the
|
|
source is unchanged and the images already exist; set
|
|
`-e trade_hub_force_rebuild=true` to force it),
|
|
4. creates the `sc-inventory` podman network and the `sc-inventory-data` volume,
|
|
5. runs PostgreSQL, the backend, and the frontend containers (only the frontend
|
|
publishes a port: 8080),
|
|
6. verifies `http://127.0.0.1:8080/healthz` returns 200.
|
|
|
|
The old standalone playbook's variables (`force_rebuild`, `cookie_secure`, …)
|
|
became `trade_hub_*` variables; see `roles/trade_hub/README.md`.
|
|
|
|
## Reset for testing (remove everything)
|
|
|
|
```bash
|
|
# preview only, changes nothing
|
|
ansible-playbook teardown.yml -e confirm_destroy=true --check
|
|
|
|
# remove containers, all project images, the network and the database volume
|
|
ansible-playbook teardown.yml -e confirm_destroy=true
|
|
```
|
|
|
|
The teardown is scoped by name (`sc-inventory*`, `sc-inv-verify*`,
|
|
`trade-hub*`) and never runs `podman system prune`, so unrelated containers,
|
|
images and volumes on the host are left alone. It removes every tag of the
|
|
project images (so stale builds cannot resurface) and deletes the
|
|
`sc-inventory-data` volume — **all accounts and inventories are destroyed**.
|
|
Run `playbook.yml` afterwards to deploy a clean install; the images rebuild
|
|
from scratch.
|
|
|
|
## Accounts and personal inventory
|
|
|
|
- **Registration is open**: anyone who can reach the site can create an account.
|
|
- A new account starts with an **empty inventory**; nothing is seeded or shared
|
|
between accounts.
|
|
- Passwords are hashed with Argon2 and never returned by the API.
|
|
- Sessions are opaque random tokens delivered in an `httpOnly`, `SameSite=Lax`
|
|
cookie; only the SHA-256 of the token is stored server-side.
|
|
- Failed logins are throttled per username and per client IP, keyed on the
|
|
`X-Real-IP` header that nginx sets from the actual peer address.
|
|
- Mutating requests are rejected when the `Origin` header does not match the
|
|
request host.
|
|
- Catalog browsing works without an account; the catalog endpoints remain
|
|
read-only.
|
|
|
|
The inventory view can be filtered by category and searched by name/class
|
|
name, and sorted by `name` (default), `category`, `quantity` (high to low) or
|
|
`updated` (most recently changed first).
|
|
|
|
Inventory rows store `(user_id, class_name, quantity, notes, visibility,
|
|
quality)`. Adding an item that is already present increments its quantity;
|
|
quantity, notes, visibility and quality can then be edited or the row removed.
|
|
Every added item starts `private`; the Visibility toggle switches it to
|
|
`available` and back. Responses are enriched live from the catalog.
|
|
|
|
`quality` is an optional mining/refining value from **1 to 1000**. It is only
|
|
accepted for commodities that are actually mineable — ores, minerals and gems
|
|
(and their refined/raw/pure variants). Gases, organics, processed goods, crafted
|
|
alloys and salvage are excluded. The catalog exposes this as
|
|
`quality_applicable`; the UI shows a Quality input only for those commodities.
|
|
`null` means the owner has not recorded a quality. On `PATCH`, sending
|
|
`"quality": null` clears it, while omitting the field leaves it unchanged. Each
|
|
inventory row holds one quality (the schema keeps one row per commodity, so
|
|
different quality lots are not separated). If a catalog item disappears after a
|
|
dataset refresh, the row is reported with `available: false` instead of being
|
|
silently deleted (`available` is about the catalog, `visibility` is the owner's
|
|
classification).
|
|
|
|
The **Available** tab (backed by `GET /api/v1/available`) is public, like the
|
|
catalog: it lists every row marked `available` across all accounts with the
|
|
item's catalog details, quantity, quality (when applicable), owner username and
|
|
last update. Private rows
|
|
never appear there and inventory notes are never exposed.
|
|
|
|
The **Orders** tab (backed by `GET /api/v1/orders`) is also public. A logged-in
|
|
user clicks any item to open its detail view, then fills in the **Request this
|
|
item** form there with a quantity, optional notes, and — for mineable
|
|
commodities — an optional quality. Setting up the order publishes the request
|
|
on the public board.
|
|
Orders show the item, quantity, quality, notes, requesting username, status
|
|
(`open`/`fulfilled`) and last update. Notes on orders are public by design;
|
|
inventory notes remain private. Only the owner can edit, mark fulfilled/reopen,
|
|
or remove their own order.
|
|
|
|
Clicking an item name in **My Inventory**, **Available** or **Orders** opens the
|
|
same catalog detail view used by the catalog grid.
|
|
|
|
### Security: read before exposing this publicly
|
|
|
|
The stack publishes plain HTTP on port 8080. Session cookies therefore travel
|
|
in cleartext, and `COOKIE_SECURE` defaults to `false` so local development works.
|
|
Before putting this on the open internet, terminate TLS in front of the
|
|
frontend (Caddy, nginx with certificates, or a load balancer) and set
|
|
`COOKIE_SECURE=true` (compose env var) or
|
|
`-e trade_hub_cookie_secure=true` (Ansible).
|
|
Also change `POSTGRES_PASSWORD` from its development default.
|
|
|
|
## Verify
|
|
|
|
```bash
|
|
curl -fsS http://localhost:8080/healthz
|
|
curl -fsS http://localhost:8080/api/v1/stats
|
|
curl -fsS "http://localhost:8080/api/v1/items?search=carrack"
|
|
```
|
|
|
|
Register, then use the cookie jar for authenticated calls:
|
|
|
|
```bash
|
|
curl -fsS -c cookies.txt -X POST http://localhost:8080/api/v1/auth/register \
|
|
-H 'Content-Type: application/json' \
|
|
-d '{"username":"pilot","password":"correct horse battery"}'
|
|
|
|
curl -fsS -b cookies.txt http://localhost:8080/api/v1/auth/me
|
|
|
|
curl -fsS -b cookies.txt -X POST http://localhost:8080/api/v1/me/inventory \
|
|
-H 'Content-Type: application/json' \
|
|
-d '{"class_name":"ANVL_Carrack","quantity":1,"notes":"pledged"}'
|
|
|
|
curl -fsS -b cookies.txt http://localhost:8080/api/v1/me/inventory
|
|
```
|
|
|
|
## Backend development
|
|
|
|
```bash
|
|
cd roles/trade_hub/files/trade-hub/backend
|
|
python3 -m venv .venv
|
|
.venv/bin/pip install -r requirements.txt -r requirements-dev.txt
|
|
.venv/bin/python -m pytest tests/ -q # 83 tests, no database needed
|
|
DATA_DIR=./data/samples .venv/bin/uvicorn main:app --port 8000
|
|
```
|
|
|
|
Tests run against an in-memory SQLite database configured in `tests/conftest.py`,
|
|
so no PostgreSQL server is required for the suite. Production uses PostgreSQL.
|
|
|
|
- `data/samples/` — small real-schema subset for local dev/tests
|
|
- `scripts/fetch_data.py` — downloads the full dataset (used by the Dockerfile)
|
|
|
|
Commodities are deduplicated to one catalog entry per name (the 1 SCU container
|
|
variant is canonical); quantity lives on the inventory row, so 1 SCU, 8 SCU and
|
|
32 SCU crates of the same commodity share a single entry. Refined and unrefined
|
|
materials stay separate (`Iron` vs `Iron (Ore)`). Quality applies to both the
|
|
refined and raw/ore entries and is user-recorded per inventory row.
|
|
|
|
Tables are created on startup with an idempotent `create_all` and a connection
|
|
retry loop, so a fresh volume bootstraps itself and a slow PostgreSQL start does
|
|
not crash the backend. Additive columns for existing deployments (currently
|
|
`inventory_entries.visibility`) are applied automatically on startup; changing
|
|
or dropping a column still requires Alembic.
|
|
|
|
## API
|
|
|
|
```
|
|
GET /healthz
|
|
GET /api/v1/stats
|
|
GET /api/v1/manufacturers
|
|
GET /api/v1/items?category=&search=&manufacturer=&type=&size=&page=&page_size=
|
|
GET /api/v1/items/{class_name}
|
|
|
|
POST /api/v1/auth/register {username, password} -> 201 + session cookie
|
|
POST /api/v1/auth/login {username, password} -> 200 + session cookie
|
|
POST /api/v1/auth/logout -> 204
|
|
GET /api/v1/auth/me -> {user}
|
|
|
|
GET /api/v1/me/inventory?category=&search=&sort=&page=&page_size=
|
|
GET /api/v1/me/inventory/summary
|
|
POST /api/v1/me/inventory {class_name, quantity?, notes?, quality?}
|
|
PATCH /api/v1/me/inventory/{class_name} {quantity?, notes?, visibility?, quality?}
|
|
DELETE /api/v1/me/inventory/{class_name}
|
|
|
|
GET /api/v1/available?category=&search=&sort=&page=&page_size=
|
|
|
|
GET /api/v1/orders?category=&search=&status=&sort=&page=&page_size=
|
|
POST /api/v1/orders {class_name, quantity?, notes?, quality?}
|
|
PATCH /api/v1/orders/{id} {quantity?, notes?, status?, quality?}
|
|
DELETE /api/v1/orders/{id}
|
|
```
|
|
|
|
`/api/v1/me/*` requires a valid session cookie and returns `401` otherwise.
|
|
`/api/v1/available` and `GET /api/v1/orders` are public and read-only. Both
|
|
accept `sort=name|category|quantity|updated` (default `updated`); orders also
|
|
accept `status=open|fulfilled`. Creating or changing an order requires a
|
|
session; only its owner may update or delete it. Catalog `category` values are
|
|
`ships`, `ship-items`, `fps-items` and `commodities`.
|
|
|
|
### Configuration
|
|
|
|
| Variable | Default | Purpose |
|
|
| ---------------------- | ------------------------------------------------------------------------ | ----------------------------------------------- |
|
|
| `DATA_DIR` | `backend/data/samples` | Catalog JSON location |
|
|
| `DATABASE_URL` | `postgresql+psycopg://sc_inventory:sc_inventory_dev@db:5432/sc_inventory` | SQLAlchemy database URL |
|
|
| `AUTO_INIT_DB` | `true` | Create missing tables on startup |
|
|
| `DB_WAIT_SECONDS` | `60` | Startup wait for PostgreSQL |
|
|
| `COOKIE_SECURE` | `false` | Mark the session cookie `Secure` (use with TLS) |
|
|
| `COOKIE_NAME` | `sc_session` | Session cookie name |
|
|
| `SESSION_TTL_DAYS` | `30` | Absolute session lifetime |
|
|
| `CORS_ORIGINS` | *(empty)* | Comma-separated cross-origin opt-in |
|
|
| `LOGIN_MAX_ATTEMPTS` | `5` | Failed logins before throttling |
|
|
| `LOGIN_WINDOW_SECONDS` | `900` | Throttle window |
|
|
|
|
Data source: scunpacked — unofficial Star Citizen fansite content, not
|
|
affiliated with Cloud Imperium Games.
|