This commit is contained in:
dahyeon committed 2026-05-12 08:16:31 -07:00
1 parent 1453433b62
commit da76af3826
65 files changed
+11089 -143

No files matched your search

+544
View File
@@ -0,0 +1,544 @@
# Machine Env Service
IMDSv2-compatible local metadata daemon for homelab machines. Each machine runs one daemon, and local processes or containers can fetch machine-scoped metadata and allowlisted Infisical secrets through HTTP without holding Infisical credentials.
## API Reference
Use `BASE_URL=http://169.254.169.254` on a host with the link-local listener. For local development use `BASE_URL=http://127.0.0.1:8675`. For Docker Desktop on macOS use `BASE_URL=http://host.docker.internal:8675`.
Issue an IMDSv2-style token:
```bash
BASE_URL=http://169.254.169.254
TOKEN=$(
curl -fsS -X PUT \
-H "X-aws-ec2-metadata-token-ttl-seconds: 60" \
"$BASE_URL/latest/api/token"
)
```
Token TTL must be between `1` and `21600` seconds. All metadata, dynamic data, user-data, and secret endpoints require:
```text
X-aws-ec2-metadata-token: <token>
```
| Method | Path | Description |
| ------ | -------------------------------------------- | ---------------------------------------------------------------------------------- |
| `PUT` | `/latest/api/token` | Issue an opaque metadata token. Requires `X-aws-ec2-metadata-token-ttl-seconds`. |
| `GET` | `/latest/meta-data/` | List top-level metadata keys. |
| `GET` | `/latest/meta-data/environment` | Infisical environment configured for this machine. |
| `GET` | `/latest/meta-data/hostname` | Machine hostname. |
| `GET` | `/latest/meta-data/local-hostname` | Local hostname. |
| `GET` | `/latest/meta-data/local-ipv4` | Primary local IPv4. |
| `GET` | `/latest/meta-data/mac` | Primary MAC address. |
| `GET` | `/latest/meta-data/machine-id` | Machine ID from config or OS. |
| `GET` | `/latest/meta-data/role` | Optional machine role. Returns `404` if unset. |
| `GET` | `/latest/meta-data/tags/` | List tag namespaces. Currently `instance/`. |
| `GET` | `/latest/meta-data/tags/<KEY>` | Read a machine tag by key. |
| `GET` | `/latest/meta-data/tags/instance/` | List machine tag keys. |
| `GET` | `/latest/meta-data/tags/instance/<KEY>` | Read a machine tag by key. |
| `GET` | `/latest/meta-data/secrets/` | List allowlisted secret keys currently present in cache. |
| `GET` | `/latest/meta-data/secrets/<KEY>` | Read one allowlisted secret value as plain text. |
| `GET` | `/latest/meta-data/secrets-bundle` | Read all cached allowlisted secrets as JSON. Requires `ALLOW_SECRETS_BUNDLE=true`. |
| `GET` | `/latest/dynamic/` | List dynamic-data categories. |
| `GET` | `/latest/dynamic/instance-identity/` | List instance identity fields. |
| `GET` | `/latest/dynamic/instance-identity/document` | Machine identity document as JSON. |
| `GET` | `/latest/user-data` | Configured user-data as plain text. Returns `404` if unset. |
### Curl Examples
List top-level metadata:
```bash
curl -fsS \
-H "X-aws-ec2-metadata-token: $TOKEN" \
"$BASE_URL/latest/meta-data/"
```
Read basic machine metadata:
```bash
curl -fsS -H "X-aws-ec2-metadata-token: $TOKEN" "$BASE_URL/latest/meta-data/hostname"
curl -fsS -H "X-aws-ec2-metadata-token: $TOKEN" "$BASE_URL/latest/meta-data/local-ipv4"
curl -fsS -H "X-aws-ec2-metadata-token: $TOKEN" "$BASE_URL/latest/meta-data/mac"
curl -fsS -H "X-aws-ec2-metadata-token: $TOKEN" "$BASE_URL/latest/meta-data/machine-id"
curl -fsS -H "X-aws-ec2-metadata-token: $TOKEN" "$BASE_URL/latest/meta-data/environment"
```
Read role:
```bash
curl -i \
-H "X-aws-ec2-metadata-token: $TOKEN" \
"$BASE_URL/latest/meta-data/role"
```
List and read tags:
```bash
curl -fsS \
-H "X-aws-ec2-metadata-token: $TOKEN" \
"$BASE_URL/latest/meta-data/tags/instance/"
curl -fsS \
-H "X-aws-ec2-metadata-token: $TOKEN" \
"$BASE_URL/latest/meta-data/tags/instance/site"
```
Read the identity document:
```bash
curl -fsS \
-H "X-aws-ec2-metadata-token: $TOKEN" \
"$BASE_URL/latest/dynamic/instance-identity/document"
```
Read user-data:
```bash
curl -i \
-H "X-aws-ec2-metadata-token: $TOKEN" \
"$BASE_URL/latest/user-data"
```
List and read secrets:
```bash
curl -fsS \
-H "X-aws-ec2-metadata-token: $TOKEN" \
"$BASE_URL/latest/meta-data/secrets/"
curl -fsS \
-H "X-aws-ec2-metadata-token: $TOKEN" \
"$BASE_URL/latest/meta-data/secrets/FOO"
```
Read the optional secret bundle:
```bash
curl -fsS \
-H "X-aws-ec2-metadata-token: $TOKEN" \
"$BASE_URL/latest/meta-data/secrets-bundle"
```
Expected failure cases:
```bash
# Missing token: 401
curl -i "$BASE_URL/latest/meta-data/"
# SSRF guard: 403
curl -i \
-H "X-Forwarded-For: 1.2.3.4" \
"$BASE_URL/latest/meta-data/"
# Token endpoint with wrong method: 405
curl -i "$BASE_URL/latest/api/token"
```
## Configuration
The daemon reads its own dotenv-style config file.
Default paths:
```text
Linux: /etc/machine-env-service/env
macOS: /usr/local/etc/machine-env-service/env
```
Override with:
```bash
MACHINE_ENV_CONFIG=/path/to/env
```
Minimum config:
```env
INFISICAL_ACCESS_TOKEN=
INFISICAL_PROJECT_ID=
INFISICAL_ENVIRONMENT=prod
KNOWN_SECRET_KEYS=FOO,BAR
METADATA_PORT=8675
REFRESH_INTERVAL_SEC=60
```
Optional machine metadata:
```env
MACHINE_ID=
MACHINE_HOSTNAME=
MACHINE_LOCAL_HOSTNAME=
MACHINE_LOCAL_IPV4=
MACHINE_MAC=
MACHINE_ROLE=homelab-node
MACHINE_TAGS=site=home,tier=prod
USER_DATA=
USER_DATA_FILE=
```
If machine fields are empty, the daemon derives hostname, IPv4, MAC, and machine-id from the OS where possible.
Secret allowlist variables are checked in this order:
```text
KNOWN_SECRET_KEYS
EXPOSED_SECRET_KEYS
SECRET_ALLOWLIST
SECRET_WHITELIST
```
Values can be comma-separated or whitespace-separated.
## Local Development
Build:
```bash
npm ci
npm run build
```
Run locally without binding `169.254.169.254:80`:
```bash
NODE_ENV=dev LINK_LOCAL_LISTENER_ENABLED=false npm start
```
Request a token:
```bash
TOKEN=$(
curl -fsS -X PUT \
-H "X-aws-ec2-metadata-token-ttl-seconds: 60" \
http://127.0.0.1:8675/latest/api/token
)
```
Fetch metadata:
```bash
curl -fsS \
-H "X-aws-ec2-metadata-token: $TOKEN" \
http://127.0.0.1:8675/latest/meta-data/hostname
```
Fetch a secret:
```bash
curl -fsS \
-H "X-aws-ec2-metadata-token: $TOKEN" \
http://127.0.0.1:8675/latest/meta-data/secrets/FOO
```
Or use:
```bash
METADATA_BASE_URL=http://127.0.0.1:8675 ./examples/fetch-secret.sh FOO
```
## Ubuntu Install
The included systemd units assume the app lives at `/opt/machine-env-service`.
Install files:
```bash
sudo mkdir -p /opt/machine-env-service
sudo chown -R "$USER:$USER" /opt/machine-env-service
rsync -a \
--exclude .git \
--exclude node_modules \
--exclude dist \
--exclude env-vars \
./ /opt/machine-env-service/
cd /opt/machine-env-service
npm ci
npm run build
```
Create service user:
```bash
sudo useradd --system --home /nonexistent --shell /usr/sbin/nologin machine-env || true
```
Create config:
```bash
sudo install -d -o machine-env -g machine-env -m 0700 /etc/machine-env-service
sudo cp deploy/systemd/machine-env-service.env.example /etc/machine-env-service/env
sudo nano /etc/machine-env-service/env
sudo chown machine-env:machine-env /etc/machine-env-service/env
sudo chmod 0600 /etc/machine-env-service/env
```
Install and start systemd units:
```bash
sudo cp deploy/systemd/machine-env-setup-link-local.service /etc/systemd/system/
sudo cp deploy/systemd/machine-env-service.service /etc/systemd/system/
sudo systemctl daemon-reload
sudo systemctl enable --now machine-env-setup-link-local.service
sudo systemctl enable --now machine-env-service.service
```
Check logs:
```bash
systemctl status machine-env-service.service
journalctl -u machine-env-service.service -f
```
Test the link-local endpoint:
```bash
TOKEN=$(
curl -fsS -X PUT \
-H "X-aws-ec2-metadata-token-ttl-seconds: 60" \
http://169.254.169.254/latest/api/token
)
curl -fsS \
-H "X-aws-ec2-metadata-token: $TOKEN" \
http://169.254.169.254/latest/meta-data/
```
## macOS Install
The included launchd plists assume:
```text
App path: /opt/machine-env-service
Node path: /usr/local/bin/node
Config path: /usr/local/etc/machine-env-service/env
```
Check your Node path first:
```bash
command -v node
```
If it is not `/usr/local/bin/node`, either update `deploy/launchd/us.dahyeon.machine-env-service.plist` before installing or create a system-visible symlink:
```bash
sudo mkdir -p /usr/local/bin
sudo ln -sf "$(command -v node)" /usr/local/bin/node
```
When Node comes from `nvm`, the symlink is the quickest local setup. A Homebrew Node installed at a stable path is usually better for a long-lived daemon.
Install files:
```bash
sudo mkdir -p /opt/machine-env-service
sudo chown -R "$(id -un):admin" /opt/machine-env-service
rsync -a \
--exclude .git \
--exclude node_modules \
--exclude dist \
--exclude env-vars \
./ /opt/machine-env-service/
cd /opt/machine-env-service
npm ci
npm run build
```
Create config:
```bash
sudo install -d -m 0700 /usr/local/etc/machine-env-service
sudo cp deploy/launchd/machine-env-service.env.example /usr/local/etc/machine-env-service/env
sudo nano /usr/local/etc/machine-env-service/env
sudo chown root:wheel /usr/local/etc/machine-env-service/env
sudo chmod 0600 /usr/local/etc/machine-env-service/env
```
Install and start launchd daemons:
```bash
sudo cp deploy/launchd/us.dahyeon.machine-env-setup-link-local.plist /Library/LaunchDaemons/
sudo cp deploy/launchd/us.dahyeon.machine-env-service.plist /Library/LaunchDaemons/
sudo chown root:wheel /Library/LaunchDaemons/us.dahyeon.machine-env-*.plist
sudo chmod 0644 /Library/LaunchDaemons/us.dahyeon.machine-env-*.plist
sudo launchctl bootstrap system /Library/LaunchDaemons/us.dahyeon.machine-env-setup-link-local.plist
sudo launchctl bootstrap system /Library/LaunchDaemons/us.dahyeon.machine-env-service.plist
```
Check logs:
```bash
sudo launchctl print system/us.dahyeon.machine-env-service
tail -f /var/log/machine-env-service.log
tail -f /var/log/machine-env-service.err.log
```
Test:
```bash
TOKEN=$(
curl -fsS -X PUT \
-H "X-aws-ec2-metadata-token-ttl-seconds: 60" \
http://169.254.169.254/latest/api/token
)
curl -fsS \
-H "X-aws-ec2-metadata-token: $TOKEN" \
http://169.254.169.254/latest/meta-data/hostname
```
Docker Desktop on macOS usually cannot route container traffic to host `169.254.169.254`. Use the LAN listener instead:
```text
http://host.docker.internal:8675/latest/meta-data/...
```
## Update
Restarting the daemon invalidates issued IMDSv2 tokens. Clients should request a new token after an update.
If `/opt/machine-env-service` is a git checkout:
```bash
cd /opt/machine-env-service
git pull
npm ci
npm run build
```
If you deploy from another checkout:
```bash
rsync -a --delete \
--exclude .git \
--exclude node_modules \
--exclude dist \
--exclude env-vars \
./ /opt/machine-env-service/
cd /opt/machine-env-service
npm ci
npm run build
```
Do not overwrite the machine config file:
```text
Linux: /etc/machine-env-service/env
macOS: /usr/local/etc/machine-env-service/env
```
Restart after code-only updates:
```bash
sudo systemctl restart machine-env-service.service
```
```bash
sudo launchctl kickstart -k system/us.dahyeon.machine-env-service
```
If systemd unit files changed:
```bash
sudo cp deploy/systemd/machine-env-setup-link-local.service /etc/systemd/system/
sudo cp deploy/systemd/machine-env-service.service /etc/systemd/system/
sudo systemctl daemon-reload
sudo systemctl restart machine-env-setup-link-local.service
sudo systemctl restart machine-env-service.service
```
If launchd plist files changed:
```bash
sudo cp deploy/launchd/us.dahyeon.machine-env-setup-link-local.plist /Library/LaunchDaemons/
sudo cp deploy/launchd/us.dahyeon.machine-env-service.plist /Library/LaunchDaemons/
sudo chown root:wheel /Library/LaunchDaemons/us.dahyeon.machine-env-*.plist
sudo chmod 0644 /Library/LaunchDaemons/us.dahyeon.machine-env-*.plist
sudo launchctl bootout system /Library/LaunchDaemons/us.dahyeon.machine-env-service.plist 2>/dev/null || true
sudo launchctl bootout system /Library/LaunchDaemons/us.dahyeon.machine-env-setup-link-local.plist 2>/dev/null || true
sudo launchctl bootstrap system /Library/LaunchDaemons/us.dahyeon.machine-env-setup-link-local.plist
sudo launchctl bootstrap system /Library/LaunchDaemons/us.dahyeon.machine-env-service.plist
```
Check after updating:
```bash
systemctl status machine-env-service.service
journalctl -u machine-env-service.service -n 100
```
```bash
sudo launchctl print system/us.dahyeon.machine-env-service
tail -n 100 /var/log/machine-env-service.err.log
```
## Operations
Restart:
```bash
sudo systemctl restart machine-env-service.service
```
```bash
sudo launchctl kickstart -k system/us.dahyeon.machine-env-service
```
Stop:
```bash
sudo systemctl stop machine-env-service.service
```
```bash
sudo launchctl bootout system /Library/LaunchDaemons/us.dahyeon.machine-env-service.plist
```
## Troubleshooting
If `curl http://169.254.169.254/latest/api/token` fails with `Couldn't connect to server`, check the link-local alias and daemon separately:
```bash
ifconfig lo0 | grep 169.254.169.254
sudo launchctl print system/us.dahyeon.machine-env-service
```
If launchd shows `last exit code = 78: EX_CONFIG` and the plist points to `/usr/local/bin/node`, verify that Node exists there:
```bash
ls -l /usr/local/bin/node
command -v node
```
Fix a missing `/usr/local/bin/node` with:
```bash
sudo mkdir -p /usr/local/bin
sudo ln -sf "$(command -v node)" /usr/local/bin/node
sudo launchctl kickstart -k system/us.dahyeon.machine-env-service
```
Then retry token issuance.
## Security Notes
- Keep the config file mode at `0600`.
- Do not commit `env-vars/*.env`, production config files, or access tokens.
- Only allowlist secrets that should be readable by local machine workloads.
- Block `METADATA_PORT` from untrusted networks.
- Requests with `X-Forwarded-For` are rejected.
- Metadata requests require IMDSv2-style tokens; token TTL is `1..21600` seconds.