Probable root causes
Ranked by how much of the current breakage each one explains.
- Kubernetes API server is not reachable3 components downstream
Check the static pod containers with 'crictl ps -a | grep kube-apiserver' and read 'journalctl -u kubelet -p err -n 50' for the reason the container will not start. Until this is resolved, no deployment, scaling or self-healing can happen on this cluster.
- Broken route — nothing is listening on port 30501 component downstream
Start the service that is supposed to own port 3050, or remove the stale nginx route if the application has been retired. Check 'docker ps -a' and 'systemctl list-units --state=failed' on this host — a container or unit that exited is the usual cause.
- Broken route — nothing is listening on port 30511 component downstream
Start the service that is supposed to own port 3051, or remove the stale nginx route if the application has been retired. Check 'docker ps -a' and 'systemctl list-units --state=failed' on this host — a container or unit that exited is the usual cause.
- Broken route — nothing is listening on port 30601 component downstream
Start the service that is supposed to own port 3060, or remove the stale nginx route if the application has been retired. Check 'docker ps -a' and 'systemctl list-units --state=failed' on this host — a container or unit that exited is the usual cause.
Production — KVM8
kubernetes · systemd · nginx
- Workloads
- 12
- CPU
- 8 vCPU
- Memory free
- 27.9 of 31.3 GB
- Disk free
- 304.0 GB
- Datastores
- 8 reachable, 3 blocked
- homechore.ogmworld.com
- innovation.ogmworld.com
- investor.ogmworld.com
- ip10.ogmworld.com
- meet.ogmworld.com
- ogmlands.com
- +8 more
Stage — KVM2
kubernetes · systemd · nginx
- Workloads
- 7
- CPU
- 2 vCPU
- Memory free
- 6.6 of 7.8 GB
- Disk free
- 76.4 GB
- Kubernetes
- v1.28.15
- Datastores
- 0 reachable, 11 blocked
- stage.ogmworldwide.com
- stage.samskriti.ogmworld.com
Development — KVM4
docker · systemd · nginx
- Workloads
- 5
- CPU
- 4 vCPU
- Memory free
- 14.0 of 15.6 GB
- Disk free
- 152.2 GB
- Datastores
- 8 reachable, 3 blocked
- dev.inspector.ogmworld.com
- dev.ogmworldwide.com
- dev.samskriti.ogmworld.com
Datacenter — shared data
docker · systemd
- Workloads
- 28
- CPU
- 8 vCPU
- Memory free
- 26.3 of 31.3 GB
- Disk free
- 304.4 GB
Broken public routes
nginx forwards these hostnames to a port with no process listening — visitors get a 502.
| Environment | Backend | Reason |
|---|---|---|
| prod | 127.0.0.1:3050 | nginx forwards ip10.ogmworld.com/ here, but nothing is listening on port 3050 on this host. |
| prod | 127.0.0.1:3051 | nginx forwards investor.ogmworld.com/ here, but nothing is listening on port 3051 on this host. |
| prod | 127.0.0.1:3060 | nginx forwards homechore.ogmworld.com/_next/static/ here, but nothing is listening on port 3060 on this host. |
| prod | 127.0.0.1:3070 | nginx forwards sentinelloop.ogmworld.com/_next/static/ here, but nothing is listening on port 3070 on this host. |
| prod | 127.0.0.1:7880 | nginx forwards meet.ogmworld.com/ here, but nothing is listening on port 7880 on this host. |
| prod | 127.0.0.1:8090 | nginx forwards ip10.ogmworld.com^~ /api/ here, but nothing is listening on port 8090 on this host. |
| prod | 127.0.0.1:8091 | nginx forwards innovation.ogmworld.com/reqiq/ here, but nothing is listening on port 8091 on this host. |
| prod | 127.0.0.1:8092 | nginx forwards investor.ogmworld.com^~ /api/ here, but nothing is listening on port 8092 on this host. |
| stage | 127.0.0.1:32229 | nginx forwards stage.samskriti.ogmworld.com/ here, but nothing is listening on port 32229 on this host. |