Elasticsearch (kubernetes)
Elasticsearch is a distributed, RESTful search and analytics engine capable of addressing a growing number of use cases.
Made by
Massdriver
Official
Yes
Clouds
Tags
Kubernetes Elasticsearch
Elasticsearch is a powerful open-source search and analytics engine capable of ingesting large volumes of data and providing near real-time search capabilities. This guide helps you effectively manage Elasticsearch clusters deployed on Kubernetes.
Design Decisions
- Helm for Deployment: This module uses Helm to deploy Elasticsearch on Kubernetes, leveraging Helm's templating system for advanced configurations.
- Kubernetes Resources Configuration: It allows detailed settings for CPU, memory, and storage to tailor to different use cases.
- Scalability: Designed to easily scale by adjusting the number of replicas and other parameters.
- Security: Randomized password generation for Elasticsearch authentication, ensuring a secure setup.
- Namespace Isolation: Each Elasticsearch deployment is isolated within its Kubernetes namespace for better management and security.
Runbook
Elasticsearch Cluster Not Starting
If the Elasticsearch cluster is not coming up, check the following:
Retrieve the logs of the pod to identify any issues.
kubectl logs -n <namespace> <elasticsearch-pod>
Inspect the events in the namespace to understand resource issues or configuration errors.
kubectl get events -n <namespace>
High Memory Usage
If Elasticsearch pods are consuming a lot of memory, check the memory allocation and current usage:
kubectl top pod -n <namespace>
Examine Elasticsearch's _nodes/stats endpoint to get detailed memory usage information.
curl -u elastic:<password> -X GET "<elasticsearch-endpoint>/_nodes/stats?pretty"
Issues with Cluster Forming
If the Elasticsearch cluster is having trouble forming, look for network issues or misconfiguration:
Verify the service endpoints in your cluster.
kubectl get svc -n <namespace>
Check Elasticsearch cluster health for any node issues.
curl -u elastic:<password> -X GET "<elasticsearch-endpoint>/_cluster/health?pretty"
Password Authentication Issues
If facing issues with password authentication, ensure that the password was set correctly.
Retrieve the value of the generated secret.
kubectl get secret -n <namespace> <secret-name> -o jsonpath="{.data.elastic}"
echo "<retrieved-password>" | base64 --decode
Persistent Volume Claims (PVC) Issues
If PVCs are not being bound, check the status of PVCs in your namespace:
kubectl get pvc -n <namespace>
Describe the specific PVC to get more details.
kubectl describe pvc <pvc-name> -n <namespace>
Data Node Restart Loop
If data nodes are in a restart loop, inspect the pod events and logs for errors:
kubectl describe pod <pod-name> -n <namespace>
kubectl logs <pod-name> -n <namespace>
For deeper insights, check Elasticsearch logs within the pod.
kubectl exec -it <pod-name> -n <namespace> -- cat /usr/share/elasticsearch/logs/elasticsearch.log
Use these troubleshooting steps to resolve common issues with Elasticsearch on Kubernetes effectively.
| Variable | Type | Description |
|---|---|---|
| instance_configuration.cpu_limit | number | Unit is in CPUs. Decimal numbers are allowed (3 digits of precision). Value must be between 0.5 and 32. |
| instance_configuration.memory_limit | number | Unit is Gi. Decimal numbers are allowed. Value must be between 0.5 and 64 |
| instance_configuration.storage | integer | The size (in GiB) of the Elasticsearch instance. Must be an integer between 10 and 1000. |
| namespace | string | Choose a namespace for Elasticsearch. |
| replica_configuration.replicas | integer | Number of replicas to create. Must be an integer between 1 and 5. |
