.
This commit is contained in:
1 parent
1453433b62
commit
da76af3826
65 files changed
+11089
-143
No files matched your search
+544
@@ -0,0 +1,544 @@
|
||||
# Machine Env Service
|
||||
|
||||
IMDSv2-compatible local metadata daemon for homelab machines. Each machine runs one daemon, and local processes or containers can fetch machine-scoped metadata and allowlisted Infisical secrets through HTTP without holding Infisical credentials.
|
||||
|
||||
## API Reference
|
||||
|
||||
Use `BASE_URL=http://169.254.169.254` on a host with the link-local listener. For local development use `BASE_URL=http://127.0.0.1:8675`. For Docker Desktop on macOS use `BASE_URL=http://host.docker.internal:8675`.
|
||||
|
||||
Issue an IMDSv2-style token:
|
||||
|
||||
```bash
|
||||
BASE_URL=http://169.254.169.254
|
||||
|
||||
TOKEN=$(
|
||||
curl -fsS -X PUT \
|
||||
-H "X-aws-ec2-metadata-token-ttl-seconds: 60" \
|
||||
"$BASE_URL/latest/api/token"
|
||||
)
|
||||
```
|
||||
|
||||
Token TTL must be between `1` and `21600` seconds. All metadata, dynamic data, user-data, and secret endpoints require:
|
||||
|
||||
```text
|
||||
X-aws-ec2-metadata-token: <token>
|
||||
```
|
||||
|
||||
| Method | Path | Description |
|
||||
| ------ | -------------------------------------------- | ---------------------------------------------------------------------------------- |
|
||||
| `PUT` | `/latest/api/token` | Issue an opaque metadata token. Requires `X-aws-ec2-metadata-token-ttl-seconds`. |
|
||||
| `GET` | `/latest/meta-data/` | List top-level metadata keys. |
|
||||
| `GET` | `/latest/meta-data/environment` | Infisical environment configured for this machine. |
|
||||
| `GET` | `/latest/meta-data/hostname` | Machine hostname. |
|
||||
| `GET` | `/latest/meta-data/local-hostname` | Local hostname. |
|
||||
| `GET` | `/latest/meta-data/local-ipv4` | Primary local IPv4. |
|
||||
| `GET` | `/latest/meta-data/mac` | Primary MAC address. |
|
||||
| `GET` | `/latest/meta-data/machine-id` | Machine ID from config or OS. |
|
||||
| `GET` | `/latest/meta-data/role` | Optional machine role. Returns `404` if unset. |
|
||||
| `GET` | `/latest/meta-data/tags/` | List tag namespaces. Currently `instance/`. |
|
||||
| `GET` | `/latest/meta-data/tags/<KEY>` | Read a machine tag by key. |
|
||||
| `GET` | `/latest/meta-data/tags/instance/` | List machine tag keys. |
|
||||
| `GET` | `/latest/meta-data/tags/instance/<KEY>` | Read a machine tag by key. |
|
||||
| `GET` | `/latest/meta-data/secrets/` | List allowlisted secret keys currently present in cache. |
|
||||
| `GET` | `/latest/meta-data/secrets/<KEY>` | Read one allowlisted secret value as plain text. |
|
||||
| `GET` | `/latest/meta-data/secrets-bundle` | Read all cached allowlisted secrets as JSON. Requires `ALLOW_SECRETS_BUNDLE=true`. |
|
||||
| `GET` | `/latest/dynamic/` | List dynamic-data categories. |
|
||||
| `GET` | `/latest/dynamic/instance-identity/` | List instance identity fields. |
|
||||
| `GET` | `/latest/dynamic/instance-identity/document` | Machine identity document as JSON. |
|
||||
| `GET` | `/latest/user-data` | Configured user-data as plain text. Returns `404` if unset. |
|
||||
|
||||
### Curl Examples
|
||||
|
||||
List top-level metadata:
|
||||
|
||||
```bash
|
||||
curl -fsS \
|
||||
-H "X-aws-ec2-metadata-token: $TOKEN" \
|
||||
"$BASE_URL/latest/meta-data/"
|
||||
```
|
||||
|
||||
Read basic machine metadata:
|
||||
|
||||
```bash
|
||||
curl -fsS -H "X-aws-ec2-metadata-token: $TOKEN" "$BASE_URL/latest/meta-data/hostname"
|
||||
curl -fsS -H "X-aws-ec2-metadata-token: $TOKEN" "$BASE_URL/latest/meta-data/local-ipv4"
|
||||
curl -fsS -H "X-aws-ec2-metadata-token: $TOKEN" "$BASE_URL/latest/meta-data/mac"
|
||||
curl -fsS -H "X-aws-ec2-metadata-token: $TOKEN" "$BASE_URL/latest/meta-data/machine-id"
|
||||
curl -fsS -H "X-aws-ec2-metadata-token: $TOKEN" "$BASE_URL/latest/meta-data/environment"
|
||||
```
|
||||
|
||||
Read role:
|
||||
|
||||
```bash
|
||||
curl -i \
|
||||
-H "X-aws-ec2-metadata-token: $TOKEN" \
|
||||
"$BASE_URL/latest/meta-data/role"
|
||||
```
|
||||
|
||||
List and read tags:
|
||||
|
||||
```bash
|
||||
curl -fsS \
|
||||
-H "X-aws-ec2-metadata-token: $TOKEN" \
|
||||
"$BASE_URL/latest/meta-data/tags/instance/"
|
||||
|
||||
curl -fsS \
|
||||
-H "X-aws-ec2-metadata-token: $TOKEN" \
|
||||
"$BASE_URL/latest/meta-data/tags/instance/site"
|
||||
```
|
||||
|
||||
Read the identity document:
|
||||
|
||||
```bash
|
||||
curl -fsS \
|
||||
-H "X-aws-ec2-metadata-token: $TOKEN" \
|
||||
"$BASE_URL/latest/dynamic/instance-identity/document"
|
||||
```
|
||||
|
||||
Read user-data:
|
||||
|
||||
```bash
|
||||
curl -i \
|
||||
-H "X-aws-ec2-metadata-token: $TOKEN" \
|
||||
"$BASE_URL/latest/user-data"
|
||||
```
|
||||
|
||||
List and read secrets:
|
||||
|
||||
```bash
|
||||
curl -fsS \
|
||||
-H "X-aws-ec2-metadata-token: $TOKEN" \
|
||||
"$BASE_URL/latest/meta-data/secrets/"
|
||||
|
||||
curl -fsS \
|
||||
-H "X-aws-ec2-metadata-token: $TOKEN" \
|
||||
"$BASE_URL/latest/meta-data/secrets/FOO"
|
||||
```
|
||||
|
||||
Read the optional secret bundle:
|
||||
|
||||
```bash
|
||||
curl -fsS \
|
||||
-H "X-aws-ec2-metadata-token: $TOKEN" \
|
||||
"$BASE_URL/latest/meta-data/secrets-bundle"
|
||||
```
|
||||
|
||||
Expected failure cases:
|
||||
|
||||
```bash
|
||||
# Missing token: 401
|
||||
curl -i "$BASE_URL/latest/meta-data/"
|
||||
|
||||
# SSRF guard: 403
|
||||
curl -i \
|
||||
-H "X-Forwarded-For: 1.2.3.4" \
|
||||
"$BASE_URL/latest/meta-data/"
|
||||
|
||||
# Token endpoint with wrong method: 405
|
||||
curl -i "$BASE_URL/latest/api/token"
|
||||
```
|
||||
|
||||
## Configuration
|
||||
|
||||
The daemon reads its own dotenv-style config file.
|
||||
|
||||
Default paths:
|
||||
|
||||
```text
|
||||
Linux: /etc/machine-env-service/env
|
||||
macOS: /usr/local/etc/machine-env-service/env
|
||||
```
|
||||
|
||||
Override with:
|
||||
|
||||
```bash
|
||||
MACHINE_ENV_CONFIG=/path/to/env
|
||||
```
|
||||
|
||||
Minimum config:
|
||||
|
||||
```env
|
||||
INFISICAL_ACCESS_TOKEN=
|
||||
INFISICAL_PROJECT_ID=
|
||||
INFISICAL_ENVIRONMENT=prod
|
||||
|
||||
KNOWN_SECRET_KEYS=FOO,BAR
|
||||
METADATA_PORT=8675
|
||||
REFRESH_INTERVAL_SEC=60
|
||||
```
|
||||
|
||||
Optional machine metadata:
|
||||
|
||||
```env
|
||||
MACHINE_ID=
|
||||
MACHINE_HOSTNAME=
|
||||
MACHINE_LOCAL_HOSTNAME=
|
||||
MACHINE_LOCAL_IPV4=
|
||||
MACHINE_MAC=
|
||||
MACHINE_ROLE=homelab-node
|
||||
MACHINE_TAGS=site=home,tier=prod
|
||||
USER_DATA=
|
||||
USER_DATA_FILE=
|
||||
```
|
||||
|
||||
If machine fields are empty, the daemon derives hostname, IPv4, MAC, and machine-id from the OS where possible.
|
||||
|
||||
Secret allowlist variables are checked in this order:
|
||||
|
||||
```text
|
||||
KNOWN_SECRET_KEYS
|
||||
EXPOSED_SECRET_KEYS
|
||||
SECRET_ALLOWLIST
|
||||
SECRET_WHITELIST
|
||||
```
|
||||
|
||||
Values can be comma-separated or whitespace-separated.
|
||||
|
||||
## Local Development
|
||||
|
||||
Build:
|
||||
|
||||
```bash
|
||||
npm ci
|
||||
npm run build
|
||||
```
|
||||
|
||||
Run locally without binding `169.254.169.254:80`:
|
||||
|
||||
```bash
|
||||
NODE_ENV=dev LINK_LOCAL_LISTENER_ENABLED=false npm start
|
||||
```
|
||||
|
||||
Request a token:
|
||||
|
||||
```bash
|
||||
TOKEN=$(
|
||||
curl -fsS -X PUT \
|
||||
-H "X-aws-ec2-metadata-token-ttl-seconds: 60" \
|
||||
http://127.0.0.1:8675/latest/api/token
|
||||
)
|
||||
```
|
||||
|
||||
Fetch metadata:
|
||||
|
||||
```bash
|
||||
curl -fsS \
|
||||
-H "X-aws-ec2-metadata-token: $TOKEN" \
|
||||
http://127.0.0.1:8675/latest/meta-data/hostname
|
||||
```
|
||||
|
||||
Fetch a secret:
|
||||
|
||||
```bash
|
||||
curl -fsS \
|
||||
-H "X-aws-ec2-metadata-token: $TOKEN" \
|
||||
http://127.0.0.1:8675/latest/meta-data/secrets/FOO
|
||||
```
|
||||
|
||||
Or use:
|
||||
|
||||
```bash
|
||||
METADATA_BASE_URL=http://127.0.0.1:8675 ./examples/fetch-secret.sh FOO
|
||||
```
|
||||
|
||||
## Ubuntu Install
|
||||
|
||||
The included systemd units assume the app lives at `/opt/machine-env-service`.
|
||||
|
||||
Install files:
|
||||
|
||||
```bash
|
||||
sudo mkdir -p /opt/machine-env-service
|
||||
sudo chown -R "$USER:$USER" /opt/machine-env-service
|
||||
|
||||
rsync -a \
|
||||
--exclude .git \
|
||||
--exclude node_modules \
|
||||
--exclude dist \
|
||||
--exclude env-vars \
|
||||
./ /opt/machine-env-service/
|
||||
|
||||
cd /opt/machine-env-service
|
||||
npm ci
|
||||
npm run build
|
||||
```
|
||||
|
||||
Create service user:
|
||||
|
||||
```bash
|
||||
sudo useradd --system --home /nonexistent --shell /usr/sbin/nologin machine-env || true
|
||||
```
|
||||
|
||||
Create config:
|
||||
|
||||
```bash
|
||||
sudo install -d -o machine-env -g machine-env -m 0700 /etc/machine-env-service
|
||||
sudo cp deploy/systemd/machine-env-service.env.example /etc/machine-env-service/env
|
||||
sudo nano /etc/machine-env-service/env
|
||||
sudo chown machine-env:machine-env /etc/machine-env-service/env
|
||||
sudo chmod 0600 /etc/machine-env-service/env
|
||||
```
|
||||
|
||||
Install and start systemd units:
|
||||
|
||||
```bash
|
||||
sudo cp deploy/systemd/machine-env-setup-link-local.service /etc/systemd/system/
|
||||
sudo cp deploy/systemd/machine-env-service.service /etc/systemd/system/
|
||||
|
||||
sudo systemctl daemon-reload
|
||||
sudo systemctl enable --now machine-env-setup-link-local.service
|
||||
sudo systemctl enable --now machine-env-service.service
|
||||
```
|
||||
|
||||
Check logs:
|
||||
|
||||
```bash
|
||||
systemctl status machine-env-service.service
|
||||
journalctl -u machine-env-service.service -f
|
||||
```
|
||||
|
||||
Test the link-local endpoint:
|
||||
|
||||
```bash
|
||||
TOKEN=$(
|
||||
curl -fsS -X PUT \
|
||||
-H "X-aws-ec2-metadata-token-ttl-seconds: 60" \
|
||||
http://169.254.169.254/latest/api/token
|
||||
)
|
||||
|
||||
curl -fsS \
|
||||
-H "X-aws-ec2-metadata-token: $TOKEN" \
|
||||
http://169.254.169.254/latest/meta-data/
|
||||
```
|
||||
|
||||
## macOS Install
|
||||
|
||||
The included launchd plists assume:
|
||||
|
||||
```text
|
||||
App path: /opt/machine-env-service
|
||||
Node path: /usr/local/bin/node
|
||||
Config path: /usr/local/etc/machine-env-service/env
|
||||
```
|
||||
|
||||
Check your Node path first:
|
||||
|
||||
```bash
|
||||
command -v node
|
||||
```
|
||||
|
||||
If it is not `/usr/local/bin/node`, either update `deploy/launchd/us.dahyeon.machine-env-service.plist` before installing or create a system-visible symlink:
|
||||
|
||||
```bash
|
||||
sudo mkdir -p /usr/local/bin
|
||||
sudo ln -sf "$(command -v node)" /usr/local/bin/node
|
||||
```
|
||||
|
||||
When Node comes from `nvm`, the symlink is the quickest local setup. A Homebrew Node installed at a stable path is usually better for a long-lived daemon.
|
||||
|
||||
Install files:
|
||||
|
||||
```bash
|
||||
sudo mkdir -p /opt/machine-env-service
|
||||
sudo chown -R "$(id -un):admin" /opt/machine-env-service
|
||||
|
||||
rsync -a \
|
||||
--exclude .git \
|
||||
--exclude node_modules \
|
||||
--exclude dist \
|
||||
--exclude env-vars \
|
||||
./ /opt/machine-env-service/
|
||||
|
||||
cd /opt/machine-env-service
|
||||
npm ci
|
||||
npm run build
|
||||
```
|
||||
|
||||
Create config:
|
||||
|
||||
```bash
|
||||
sudo install -d -m 0700 /usr/local/etc/machine-env-service
|
||||
sudo cp deploy/launchd/machine-env-service.env.example /usr/local/etc/machine-env-service/env
|
||||
sudo nano /usr/local/etc/machine-env-service/env
|
||||
sudo chown root:wheel /usr/local/etc/machine-env-service/env
|
||||
sudo chmod 0600 /usr/local/etc/machine-env-service/env
|
||||
```
|
||||
|
||||
Install and start launchd daemons:
|
||||
|
||||
```bash
|
||||
sudo cp deploy/launchd/us.dahyeon.machine-env-setup-link-local.plist /Library/LaunchDaemons/
|
||||
sudo cp deploy/launchd/us.dahyeon.machine-env-service.plist /Library/LaunchDaemons/
|
||||
|
||||
sudo chown root:wheel /Library/LaunchDaemons/us.dahyeon.machine-env-*.plist
|
||||
sudo chmod 0644 /Library/LaunchDaemons/us.dahyeon.machine-env-*.plist
|
||||
|
||||
sudo launchctl bootstrap system /Library/LaunchDaemons/us.dahyeon.machine-env-setup-link-local.plist
|
||||
sudo launchctl bootstrap system /Library/LaunchDaemons/us.dahyeon.machine-env-service.plist
|
||||
```
|
||||
|
||||
Check logs:
|
||||
|
||||
```bash
|
||||
sudo launchctl print system/us.dahyeon.machine-env-service
|
||||
tail -f /var/log/machine-env-service.log
|
||||
tail -f /var/log/machine-env-service.err.log
|
||||
```
|
||||
|
||||
Test:
|
||||
|
||||
```bash
|
||||
TOKEN=$(
|
||||
curl -fsS -X PUT \
|
||||
-H "X-aws-ec2-metadata-token-ttl-seconds: 60" \
|
||||
http://169.254.169.254/latest/api/token
|
||||
)
|
||||
|
||||
curl -fsS \
|
||||
-H "X-aws-ec2-metadata-token: $TOKEN" \
|
||||
http://169.254.169.254/latest/meta-data/hostname
|
||||
```
|
||||
|
||||
Docker Desktop on macOS usually cannot route container traffic to host `169.254.169.254`. Use the LAN listener instead:
|
||||
|
||||
```text
|
||||
http://host.docker.internal:8675/latest/meta-data/...
|
||||
```
|
||||
|
||||
## Update
|
||||
|
||||
Restarting the daemon invalidates issued IMDSv2 tokens. Clients should request a new token after an update.
|
||||
|
||||
If `/opt/machine-env-service` is a git checkout:
|
||||
|
||||
```bash
|
||||
cd /opt/machine-env-service
|
||||
git pull
|
||||
npm ci
|
||||
npm run build
|
||||
```
|
||||
|
||||
If you deploy from another checkout:
|
||||
|
||||
```bash
|
||||
rsync -a --delete \
|
||||
--exclude .git \
|
||||
--exclude node_modules \
|
||||
--exclude dist \
|
||||
--exclude env-vars \
|
||||
./ /opt/machine-env-service/
|
||||
|
||||
cd /opt/machine-env-service
|
||||
npm ci
|
||||
npm run build
|
||||
```
|
||||
|
||||
Do not overwrite the machine config file:
|
||||
|
||||
```text
|
||||
Linux: /etc/machine-env-service/env
|
||||
macOS: /usr/local/etc/machine-env-service/env
|
||||
```
|
||||
|
||||
Restart after code-only updates:
|
||||
|
||||
```bash
|
||||
sudo systemctl restart machine-env-service.service
|
||||
```
|
||||
|
||||
```bash
|
||||
sudo launchctl kickstart -k system/us.dahyeon.machine-env-service
|
||||
```
|
||||
|
||||
If systemd unit files changed:
|
||||
|
||||
```bash
|
||||
sudo cp deploy/systemd/machine-env-setup-link-local.service /etc/systemd/system/
|
||||
sudo cp deploy/systemd/machine-env-service.service /etc/systemd/system/
|
||||
sudo systemctl daemon-reload
|
||||
sudo systemctl restart machine-env-setup-link-local.service
|
||||
sudo systemctl restart machine-env-service.service
|
||||
```
|
||||
|
||||
If launchd plist files changed:
|
||||
|
||||
```bash
|
||||
sudo cp deploy/launchd/us.dahyeon.machine-env-setup-link-local.plist /Library/LaunchDaemons/
|
||||
sudo cp deploy/launchd/us.dahyeon.machine-env-service.plist /Library/LaunchDaemons/
|
||||
sudo chown root:wheel /Library/LaunchDaemons/us.dahyeon.machine-env-*.plist
|
||||
sudo chmod 0644 /Library/LaunchDaemons/us.dahyeon.machine-env-*.plist
|
||||
|
||||
sudo launchctl bootout system /Library/LaunchDaemons/us.dahyeon.machine-env-service.plist 2>/dev/null || true
|
||||
sudo launchctl bootout system /Library/LaunchDaemons/us.dahyeon.machine-env-setup-link-local.plist 2>/dev/null || true
|
||||
sudo launchctl bootstrap system /Library/LaunchDaemons/us.dahyeon.machine-env-setup-link-local.plist
|
||||
sudo launchctl bootstrap system /Library/LaunchDaemons/us.dahyeon.machine-env-service.plist
|
||||
```
|
||||
|
||||
Check after updating:
|
||||
|
||||
```bash
|
||||
systemctl status machine-env-service.service
|
||||
journalctl -u machine-env-service.service -n 100
|
||||
```
|
||||
|
||||
```bash
|
||||
sudo launchctl print system/us.dahyeon.machine-env-service
|
||||
tail -n 100 /var/log/machine-env-service.err.log
|
||||
```
|
||||
|
||||
## Operations
|
||||
|
||||
Restart:
|
||||
|
||||
```bash
|
||||
sudo systemctl restart machine-env-service.service
|
||||
```
|
||||
|
||||
```bash
|
||||
sudo launchctl kickstart -k system/us.dahyeon.machine-env-service
|
||||
```
|
||||
|
||||
Stop:
|
||||
|
||||
```bash
|
||||
sudo systemctl stop machine-env-service.service
|
||||
```
|
||||
|
||||
```bash
|
||||
sudo launchctl bootout system /Library/LaunchDaemons/us.dahyeon.machine-env-service.plist
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
If `curl http://169.254.169.254/latest/api/token` fails with `Couldn't connect to server`, check the link-local alias and daemon separately:
|
||||
|
||||
```bash
|
||||
ifconfig lo0 | grep 169.254.169.254
|
||||
sudo launchctl print system/us.dahyeon.machine-env-service
|
||||
```
|
||||
|
||||
If launchd shows `last exit code = 78: EX_CONFIG` and the plist points to `/usr/local/bin/node`, verify that Node exists there:
|
||||
|
||||
```bash
|
||||
ls -l /usr/local/bin/node
|
||||
command -v node
|
||||
```
|
||||
|
||||
Fix a missing `/usr/local/bin/node` with:
|
||||
|
||||
```bash
|
||||
sudo mkdir -p /usr/local/bin
|
||||
sudo ln -sf "$(command -v node)" /usr/local/bin/node
|
||||
sudo launchctl kickstart -k system/us.dahyeon.machine-env-service
|
||||
```
|
||||
|
||||
Then retry token issuance.
|
||||
|
||||
## Security Notes
|
||||
|
||||
- Keep the config file mode at `0600`.
|
||||
- Do not commit `env-vars/*.env`, production config files, or access tokens.
|
||||
- Only allowlist secrets that should be readable by local machine workloads.
|
||||
- Block `METADATA_PORT` from untrusted networks.
|
||||
- Requests with `X-Forwarded-For` are rejected.
|
||||
- Metadata requests require IMDSv2-style tokens; token TTL is `1..21600` seconds.
|
||||
Reference in new issue
Block a user