ECK搭建ES集群

ECK简介

文档地址 https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s_learn_more_about_eck.html

ECK，也就是Elastic Cloud on Kubernetes的缩写，可以基于K8s operator在Kubernetes集群来自动化部署、管理、编排Elasticsearch、Kibana、APM Server服务。

ECK目前的features:

• Elasticsearch, Kibana 和 APM Server的部署；
• TLS证书管理；
• 安全的Elasticsearch cluster 配置和拓扑变动管理；
• 使用PV和PVC；
• 定制K8s node上的系统配置和属性；

部署

文档地址 https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-deploy-eck.html

支持的版本

• Kubernetes 1.21-1.25
• OpenShift 4.7-4.11
• Google Kubernetes Engine (GKE), Azure Kubernetes Service (AKS), and Amazon Elastic Kubernetes Service (EKS)
• Helm: 3.2.0+
• Elasticsearch, Kibana, APM Server: 6.8+, 7.1+, 8+
• Enterprise Search: 7.7+, 8+
• Beats: 7.0+, 8+
• Elastic Agent: 7.10+ (standalone), 7.14+ (Fleet), 8+
• Elastic Maps Server: 7.11+, 8+

安装ECK

安装crds

kubectl create -f https://download.elastic.co/downloads/eck/2.6.1/crds.yaml

创建如下自定义资源

customresourcedefinition.apiextensions.k8s.io/agents.agent.k8s.elastic.co created
customresourcedefinition.apiextensions.k8s.io/apmservers.apm.k8s.elastic.co created
customresourcedefinition.apiextensions.k8s.io/beats.beat.k8s.elastic.co created
customresourcedefinition.apiextensions.k8s.io/elasticmapsservers.maps.k8s.elastic.co created
customresourcedefinition.apiextensions.k8s.io/elasticsearches.elasticsearch.k8s.elastic.co created
customresourcedefinition.apiextensions.k8s.io/enterprisesearches.enterprisesearch.k8s.elastic.co created
customresourcedefinition.apiextensions.k8s.io/kibanas.kibana.k8s.elastic.co created

安装RBAC 权限

kubectl apply -f https://download.elastic.co/downloads/eck/2.6.1/operator.yaml

检查是否安装成功 查看日志

kubectl -n elastic-system logs -f statefulset.apps/elastic-operator

查看资源状态

kubectl get  statefulset elastic-operator -n elastic-system

部署ES集群

编译ECK配置文件

apiVersion: elasticsearch.k8s.elastic.co/v1
kind: Elasticsearch
metadata:
  name: eck
  namespace: elastic-system
spec:
  auth: {}
  http:
    service:
      metadata: {}
      spec: {}
    tls:
      certificate: {}
      selfSignedCertificate:
        disabled: true
  image: 's3.atai.registry/dependency/elasticsearch:7.16.3'
  monitoring:
    logs: {}
    metrics: {}
  nodeSets:
    - config:
        s3.client.minio.endpoint: 'http://172.17.20.120:9000'
      count: 3
      name: es-dev
      podTemplate:
        metadata:
          creationTimestamp: null
        spec:
          containers:
            - env:
                - name: READINESS_PROBE_TIMEOUT
                  value: '10'
                - name: ES_JAVA_OPTS
                  value: '-Xms4g -Xmx4g'
              name: elasticsearch
              readinessProbe:
                failureThreshold: 3
                initialDelaySeconds: 10
                periodSeconds: 12
                successThreshold: 1
                tcpSocket:
                  port: 9200
                timeoutSeconds: 12
              resources:
                limits:
                  cpu: '8'
                  memory: 8Gi
          imagePullSecrets:
            - name: s3-ware
          nodeSelector:
            elasticsearch: 'true'
      volumeClaimTemplates:
        - metadata:
            name: elasticsearch-data
          spec:
            accessModes:
              - ReadWriteOnce
            resources:
              requests:
                storage: 150Gi
            storageClassName: local-disk
          status: {}
  transport:
    service:
      metadata: {}
      spec: {}
    tls:
      certificate: {}
  updateStrategy:
    changeBudget: {}
  version: 7.16.3

其中nodeSets 是自定义配置文件，等同于elasticsearch.yml 配置