使用Helm v3在Kubernetes 1.15上安装Elastic Stack

随着Kubernetes的持续流行并已成为编排容器工作负载的事实上的标准，难怪在描述公司的监控堆栈时，Kubernetes和Helm现已成为标准的组合。
Elastic Stack（也称为ELK）与Kubernetes本机集成，是一种流行的开源解决方案，用于收集，存储和分析Kubernetes遥测数据。
虽然使用Kubernetes部署ELK Stack似乎是一项复杂的任务，但围绕这种场景以及Kubernetes原生解决方案的最佳实践越来越多。这些解决方案之一是使用Helm图表。
标准ELK日志记录解决方案概述

日志：确定要分析的服务器日志
Logstash：数据聚合和处理
Elasticsearch：索引编制和存储
Kibana：分析和可视化
在深入实际设置之前，让我们快速探索在Kubernetes上运行ELK堆栈的可能替代方法。
Elasticserach设置-SaaS还是自我管理？
许多云提供商都提供Elasticsearch即服务，这似乎对公司有吸引力，因为他们可以最大程度地减少构建解决方案的努力。但是，易于维护具有以下注意事项：
Elasticsearch即服务
安全性：基于云的Elasticsearch解决方案通常缺乏RBAC之类的基本ELK安全功能-最显着的是AWS Elasticsearch产品不支持X-Pack插件。因此，行使担保权的灵活性是不存在的，从长远来看，肯定会需要额外的努力。
功能：最流行的Elasticsearch云解决方案缺少分片重新平衡功能，这在大型生产环境中是至关重要的，因此，如果节点发生故障，则需要一些手动工作才能将索引移动到新节点。
SaaS选项也不支持其他插件，例如分析器插件和摄取插件。
控制：托管解决方案很少提供对Elasticsearch设置的完全控制。通常对配置更改和性能优化的支持非常有限。
维护：备份频率选项通常限制为每天一次。与Elastic的正式发布日期相比，新版本的发布时间很晚。升级通常是一个痛苦的过程，因为它们通常需要为新版本设置一个全新的集群。
可见性：监控和集群指标也非常有限。投诉，警告，GC慢日志等日志不可用
成本：托管的云服务带有针对实例类型和数量的预定义选项，与自定义解决方案相比，导致成本增加。

在Kubernetes上进行Elasticsearch
根据为ELK堆栈提供支持的公司Elastic的说法，这些只是此设置带来的一些好处：
多个Elasticsearch集群（包括Kibana）的直接部署和管理
无缝升级到Elastic Stack的新版本
简单的扩展使您可以随着用例的增长
每个群集上的默认安全性
Elastic提供了让Kubernetes上的Elasticsearch＆Kibana在Elastic Cloud托管解决方案上运行的选项，如果SaaS是您的首选选项，则可以查看其产品。
如果您愿意使用Helm 3探索Kubernetes上的Elastic设置，现在我们已经了解了Kubernetes上的Elasticsearch设置的潜在好处，那么可以继续执行本文的步骤。
以下是此设置外观的概述。

步骤1：创建Kubernetes集群
出于测试目的，您可以使用minikube-version 1.15.6，该版本应按Kubernetes Minicube文档中的说明进行安装。
或者，可以将群集部署在提供此服务的主要云提供商之一（EKS或GKE）上。我们的设置将在EKS 1.15版上运行。您可以参考官方的Cloud Provider文档以轻松进行设置。
步骤2：安装Helm 3
只需遵循官方安装指南
步骤3：部署Elasticsearch集群
为此，我们将使用Github上提供的官方Elastic Helm图表。

$ helm repo add elastic https://helm.elastic.co

1	$ helm repo add elastic https://helm.elastic.co

您可以参考资源库中提供的众多安装和设置示例。
对于我们的设置，我们将创建一个类似于以下结构的目录结构，并在每个组件的相关目录中创建values.yaml文件：

├── elasticsearch
│ └── values.yaml
├── kibana
│ └── values.yaml
└── metricbeat
└─── values.yaml

├── elasticsearch

│ └── values.yaml

├── kibana

│ └── values.yaml

└── metricbeat

└─── values.yaml

完成测试后，请特别注意CPU和内存使用情况的资源配置，并确保为生产级部署正确更新了values.yaml。
默认情况下，对于Elasticsearch，我们将为集群中的每个Pod创建容量为1Gi的PersistentVolumeClaim，以防止在意外Pod删除的情况下丢失数据。对于生产工作负载，应使用所需的存储容量和（可选）Kubernetes存储类定义卷声明模板以与持久卷关联。批量声明的名称必须始终为elasticsearch-data。

---
clusterName: "elasticsearch"
nodeGroup: "master"

# The service that non master groups will try to connect to when joining the cluster
# This should be set to clusterName + "-" + nodeGroup for your master group
masterService: ""

# Elasticsearch roles that will be applied to this nodeGroup
# These will be set as environment variables. E.g. node.master=true
roles:
  master: "true"
  ingest: "true"
  data: "true"

replicas: 3
minimumMasterNodes: 2

esMajorVersion: ""

# Allows you to add any config files in /usr/share/elasticsearch/config/
# such as elasticsearch.yml and log4j2.properties
esConfig: {}
#  elasticsearch.yml: |
#    key:
#      nestedkey: value
#  log4j2.properties: |
#    key = value

# Extra environment variables to append to this nodeGroup
# This will be appended to the current 'env:' key. You can use any of the kubernetes env
# syntax here
extraEnvs: []
#  - name: MY_ENVIRONMENT_VAR
#    value: the_value_goes_here

# A list of secrets and their paths to mount inside the pod
# This is useful for mounting certificates for security and for mounting
# the X-Pack license
secretMounts: []
#  - name: elastic-certificates
#    secretName: elastic-certificates
#    path: /usr/share/elasticsearch/config/certs

image: "docker.elastic.co/elasticsearch/elasticsearch"
imageTag: "7.6.1"
imagePullPolicy: "IfNotPresent"

podAnnotations: {}
  # iam.amazonaws.com/role: es-cluster

# additionals labels
labels: {}

esJavaOpts: "-Xmx1g -Xms1g"

resources:
  requests:
    cpu: "500m"
    memory: "1500Mi"
  limits:
    cpu: "500m"
    memory: "1500Mi"

initResources: {}
  # limits:
  #   cpu: "25m"
  #   # memory: "128Mi"
  # requests:
  #   cpu: "25m"
  #   memory: "128Mi"

sidecarResources: {}
  # limits:
  #   cpu: "25m"
  #   # memory: "128Mi"
  # requests:
  #   cpu: "25m"
  #   memory: "128Mi"

networkHost: "0.0.0.0"

volumeClaimTemplate:
  accessModes: [ "ReadWriteOnce" ]
  resources:
    requests:
      storage: 30Gi

rbac:
  create: false
  serviceAccountName: ""

podSecurityPolicy:
  create: false
  name: ""
  spec:
    privileged: true
    fsGroup:
      rule: RunAsAny
    runAsUser:
      rule: RunAsAny
    seLinux:
      rule: RunAsAny
    supplementalGroups:
      rule: RunAsAny
    volumes:
      - secret
      - configMap
      - persistentVolumeClaim

persistence:
  enabled: true
  annotations: {}

extraVolumes: ""
  # - name: extras
  #   emptyDir: {}

extraVolumeMounts: ""
  # - name: extras
  #   mountPath: /usr/share/extras
  #   readOnly: true

extraContainers: ""
  # - name: do-something
  #   image: busybox
  #   command: ['do', 'something']

extraInitContainers: ""
  # - name: do-something
  #   image: busybox
  #   command: ['do', 'something']

# This is the PriorityClass settings as defined in
# https://kubernetes.io/docs/concepts/configuration/pod-priority-preemption/#priorityclass
priorityClassName: ""

# By default this will make sure two pods don't end up on the same node
# Changing this to a region would allow you to spread pods across regions
antiAffinityTopologyKey: "kubernetes.io/hostname"

# Hard means that by default pods will only be scheduled if there are enough nodes for them
# and that they will never end up on the same node. Setting this to soft will do this "best effort"
#antiAffinity: "hard"
antiAffinity: "soft"


# This is the node affinity settings as defined in
# https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#node-affinity-beta-feature
nodeAffinity: {}

# The default is to deploy all pods serially. By setting this to parallel all pods are started at
# the same time when bootstrapping the cluster
podManagementPolicy: "Parallel"

protocol: http
httpPort: 9200
transportPort: 9300

service:
  labels: {}
  labelsHeadless: {}
  type: ClusterIP
  nodePort: ""
  annotations: {}
  httpPortName: http
  transportPortName: transport
  loadBalancerSourceRanges: []

updateStrategy: RollingUpdate

# This is the max unavailable setting for the pod disruption budget
# The default value of 1 will make sure that kubernetes won't allow more than 1
# of your pods to be unavailable during maintenance
maxUnavailable: 1

podSecurityContext:
  fsGroup: 1000
  runAsUser: 1000

# The following value is deprecated,
# please use the above podSecurityContext.fsGroup instead
fsGroup: ""

securityContext:
  capabilities:
    drop:
    - ALL
  # readOnlyRootFilesystem: true
  runAsNonRoot: true
  runAsUser: 1000

# How long to wait for elasticsearch to stop gracefully
terminationGracePeriod: 120

sysctlVmMaxMapCount: 262144

readinessProbe:
  failureThreshold: 3
  initialDelaySeconds: 60
  periodSeconds: 30
  successThreshold: 3
  timeoutSeconds: 15

# https://www.elastic.co/guide/en/elasticsearch/reference/current/cluster-health.html#request-params wait_for_status
clusterHealthCheckParams: "wait_for_status=green&timeout=1s"

## Use an alternate scheduler.
## ref: https://kubernetes.io/docs/tasks/administer-cluster/configure-multiple-schedulers/
##
schedulerName: ""

imagePullSecrets: []
nodeSelector: {}
tolerations: []

# Enabling this will publically expose your Elasticsearch instance.
# Only enable this if you have security enabled on your cluster
ingress:
  enabled: false
  annotations: {}
    # kubernetes.io/ingress.class: nginx
    # kubernetes.io/tls-acme: "true"
  path: /
  hosts:
    - chart-example.local
  tls: []
  #  - secretName: chart-example-tls
  #    hosts:
  #      - chart-example.local

nameOverride: ""
fullnameOverride: ""

# https://github.com/elastic/helm-charts/issues/63
masterTerminationFix: false

lifecycle: {}
  # preStop:
  #   exec:
  #     command: ["/bin/sh", "-c", "echo Hello from the postStart handler > /usr/share/message"]
  # postStart:
  #   exec:
  #     command:
  #       - bash
  #       - -c
  #       - |
  #         #!/bin/bash
  #         # Add a template to adjust number of shards/replicas
  #         TEMPLATE_NAME=my_template
  #         INDEX_PATTERN="logstash-*"
  #         SHARD_COUNT=8
  #         REPLICA_COUNT=1
  #         ES_URL=http://localhost:9200
  #         while [[ "$(curl -s -o /dev/null -w '%{http_code}\n' $ES_URL)" != "200" ]]; do sleep 1; done
  #         curl -XPUT "$ES_URL/_template/$TEMPLATE_NAME" -H 'Content-Type: application/json' -d'{"index_patterns":['\""$INDEX_PATTERN"\"'],"settings":{"number_of_shards":'$SHARD_COUNT',"number_of_replicas":'$REPLICA_COUNT'}}'

sysctlInitContainer:
  enabled: true

keystore: []

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

157

158

159

160

161

162

163

164

165

166

167

168

169

170

171

172

173

174

175

176

177

178

179

180

181

182

183

184

185

186

187

188

189

190

191

192

193

194

195

196

197

198

199

200

201

202

203

204

205

206

207

208

209

210

211

212

213

214

215

216

217

218

219

220

221

222

223

224

225

226

227

228

229

230

231

232

233

234

235

236

237

238

239

240

241

242

243

244

245

246

247

248

249

250

251

252

253

254

255

256

257

258

259

260

261

---

clusterName: "elasticsearch"

nodeGroup: "master"

# The service that non master groups will try to connect to when joining the cluster

# This should be set to clusterName + "-" + nodeGroup for your master group

masterService: ""

# Elasticsearch roles that will be applied to this nodeGroup

# These will be set as environment variables. E.g. node.master=true

roles:

master: "true"

ingest: "true"

data: "true"

replicas: 3

minimumMasterNodes: 2

esMajorVersion: ""

# Allows you to add any config files in /usr/share/elasticsearch/config/

# such as elasticsearch.yml and log4j2.properties

esConfig: {}

# elasticsearch.yml: |

# key:

# nestedkey: value

# log4j2.properties: |

# key = value

# Extra environment variables to append to this nodeGroup

# This will be appended to the current 'env:' key. You can use any of the kubernetes env

# syntax here

extraEnvs: []

# - name: MY_ENVIRONMENT_VAR

# value: the_value_goes_here

# A list of secrets and their paths to mount inside the pod

# This is useful for mounting certificates for security and for mounting

# the X-Pack license

secretMounts: []

# - name: elastic-certificates

# secretName: elastic-certificates

# path: /usr/share/elasticsearch/config/certs

image: "docker.elastic.co/elasticsearch/elasticsearch"

imageTag: "7.6.1"

imagePullPolicy: "IfNotPresent"

podAnnotations: {}

# iam.amazonaws.com/role: es-cluster

# additionals labels

labels: {}

esJavaOpts: "-Xmx1g -Xms1g"

resources:

requests:

cpu: "500m"

memory: "1500Mi"

limits:

cpu: "500m"

memory: "1500Mi"

initResources: {}

# limits:

# cpu: "25m"

# # memory: "128Mi"

# requests:

# cpu: "25m"

# memory: "128Mi"

sidecarResources: {}

# limits:

# cpu: "25m"

# # memory: "128Mi"

# requests:

# cpu: "25m"

# memory: "128Mi"

networkHost: "0.0.0.0"

volumeClaimTemplate:

accessModes: [ "ReadWriteOnce" ]

resources:

requests:

storage: 30Gi

rbac:

create: false

serviceAccountName: ""

podSecurityPolicy:

create: false

name: ""

spec:

privileged: true

fsGroup:

rule: RunAsAny

runAsUser:

rule: RunAsAny

seLinux:

rule: RunAsAny

supplementalGroups:

rule: RunAsAny

volumes:

- secret

- configMap

- persistentVolumeClaim

persistence:

enabled: true

annotations: {}

extraVolumes: ""

# - name: extras

# emptyDir: {}

extraVolumeMounts: ""

# - name: extras

# mountPath: /usr/share/extras

# readOnly: true

extraContainers: ""

# - name: do-something

# image: busybox

# command: ['do', 'something']

extraInitContainers: ""

# - name: do-something

# image: busybox

# command: ['do', 'something']

# This is the PriorityClass settings as defined in

# https://kubernetes.io/docs/concepts/configuration/pod-priority-preemption/#priorityclass

priorityClassName: ""

# By default this will make sure two pods don't end up on the same node

# Changing this to a region would allow you to spread pods across regions

antiAffinityTopologyKey: "kubernetes.io/hostname"

# Hard means that by default pods will only be scheduled if there are enough nodes for them

# and that they will never end up on the same node. Setting this to soft will do this "best effort"

#antiAffinity: "hard"

antiAffinity: "soft"

# This is the node affinity settings as defined in

# https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#node-affinity-beta-feature

nodeAffinity: {}

# The default is to deploy all pods serially. By setting this to parallel all pods are started at

# the same time when bootstrapping the cluster

podManagementPolicy: "Parallel"

protocol: http

httpPort: 9200

transportPort: 9300

service:

labels: {}

labelsHeadless: {}

type: ClusterIP

nodePort: ""

annotations: {}

httpPortName: http

transportPortName: transport

loadBalancerSourceRanges: []

updateStrategy: RollingUpdate

# This is the max unavailable setting for the pod disruption budget

# The default value of 1 will make sure that kubernetes won't allow more than 1

# of your pods to be unavailable during maintenance

maxUnavailable: 1

podSecurityContext:

fsGroup: 1000

runAsUser: 1000

# The following value is deprecated,

# please use the above podSecurityContext.fsGroup instead

fsGroup: ""

securityContext:

capabilities:

drop:

- ALL

# readOnlyRootFilesystem: true

runAsNonRoot: true

runAsUser: 1000

# How long to wait for elasticsearch to stop gracefully

terminationGracePeriod: 120

sysctlVmMaxMapCount: 262144

readinessProbe:

failureThreshold: 3

initialDelaySeconds: 60

periodSeconds: 30

successThreshold: 3

timeoutSeconds: 15

# https://www.elastic.co/guide/en/elasticsearch/reference/current/cluster-health.html#request-params wait_for_status

clusterHealthCheckParams: "wait_for_status=green&timeout=1s"

## Use an alternate scheduler.

## ref: https://kubernetes.io/docs/tasks/administer-cluster/configure-multiple-schedulers/

schedulerName: ""

imagePullSecrets: []

nodeSelector: {}

tolerations: []

# Enabling this will publically expose your Elasticsearch instance.

# Only enable this if you have security enabled on your cluster

ingress:

enabled: false

annotations: {}

# kubernetes.io/ingress.class: nginx

# kubernetes.io/tls-acme: "true"

path: /

hosts:

- chart-example.local

tls: []

# - secretName: chart-example-tls

# hosts:

# - chart-example.local

nameOverride: ""

fullnameOverride: ""

# https://github.com/elastic/helm-charts/issues/63

masterTerminationFix: false

lifecycle: {}

# preStop:

# exec:

# command: ["/bin/sh", "-c", "echo Hello from the postStart handler > /usr/share/message"]

# postStart:

# exec:

# command:

# - bash

# - -c

# - |

# #!/bin/bash

# # Add a template to adjust number of shards/replicas

# TEMPLATE_NAME=my_template

# INDEX_PATTERN="logstash-*"

# SHARD_COUNT=8

# REPLICA_COUNT=1

# ES_URL=http://localhost:9200

# while [[ "$(curl -s -o /dev/null -w '%{http_code}\n' $ES_URL)" != "200" ]]; do sleep 1; done

# curl -XPUT "$ES_URL/_template/$TEMPLATE_NAME" -H 'Content-Type: application/json' -d'{"index_patterns":['\""$INDEX_PATTERN"\"'],"settings":{"number_of_shards":'$SHARD_COUNT',"number_of_replicas":'$REPLICA_COUNT'}}'

sysctlInitContainer:

enabled: true

keystore: []

对于实际安装，我们将使用以下脚本。将其放置在项目的根目录中，以便可以访问3个子目录。
可以随意增强脚本或更改提供的默认值以满足您的需求，因为为了简化起见，我们尝试将其保持在最低水平：

#!/bin/bash
NAME=$2
NS=log
CHART=elastic/${NAME}
VERSION=v7.6.1
VALUES=${NAME}/values.yaml
RELEASE=$(helm ls | awk '{print $1}' | grep ${NAME})

display_usage() {
	echo -e "\nThis script must be run with 2 parameters"
	echo -e "\nUsage:\n$0 [install|update] [elasticsearch|kibana|metricbeat]"
	echo -e "\n Example:\n$0 install kibana \n"
	}

if [[  $# -le 1 ||  $# == "-h" ]]
then
  display_usage
  exit 1
fi


case $1 in
install)
		helm install ${CHART} --namespace ${NS} -f ${VALUES} --version ${VERSION} --generate-name
		#helm install --debug --dry-run --name ${NAME} --namespace ${NS} -f ${NAME}.yaml ${CHART} > ${NAME}-debug.yaml
		echo "Installed ${NAME}"
		;;
update)
		helm upgrade ${RELEASE} ${CHART} --namespace ${NS} -f ${VALUES} --version ${VERSION}
		echo "Updated ${NAME}"
		;;
*)
		display_usage
		;;
esac

#!/bin/bash

NAME=$2

NS=log

CHART=elastic/${NAME}

VERSION=v7.6.1

VALUES=${NAME}/values.yaml

RELEASE=$(helm ls | awk '{print $1}' | grep ${NAME})

display_usage() {

echo -e "\nThis script must be run with 2 parameters"

echo -e "\nUsage:\n$0 [install|update] [elasticsearch|kibana|metricbeat]"

echo -e "\n Example:\n$0 install kibana \n"

}

if [[ $# -le 1 || $# == "-h" ]]

then

display_usage

exit 1

case $1 in

install)

helm install ${CHART} --namespace ${NS} -f ${VALUES} --version ${VERSION} --generate-name

#helm install --debug --dry-run --name ${NAME} --namespace ${NS} -f ${NAME}.yaml ${CHART} > ${NAME}-debug.yaml

echo "Installed ${NAME}"

;;

update)

helm upgrade ${RELEASE} ${CHART} --namespace ${NS} -f ${VALUES} --version ${VERSION}

echo "Updated ${NAME}"

;;

display_usage

;;

esac

This script must be run with 2 parameters
Usage:
./elk-setup.sh [install|update] [elasticsearch|kibana|metricbeat|filebeat]
Example:
./elk-setup.sh install kibana

This script must be run with 2 parameters

Usage:

./elk-setup.sh [install|update] [elasticsearch|kibana|metricbeat|filebeat]

Example:

./elk-setup.sh install kibana

使用install elasticsearch选项运行上述脚本应产生以下输出：

NAME:   elasticsearch
LAST DEPLOYED: Thu Apr 1 17:28:20 2020
NAMESPACE: default
STATUS: DEPLOYED
NOTES:
1. Watch all cluster members come up.
  $ kubectl get pods --namespace=default -l app=elasticsearch-master -w
2. Test cluster health using Helm test.
  $ Helm test elasticsearch

NAME: elasticsearch

LAST DEPLOYED: Thu Apr 1 17:28:20 2020

NAMESPACE: default

STATUS: DEPLOYED

NOTES:

1. Watch all cluster members come up.

$ kubectl get pods --namespace=default -l app=elasticsearch-master -w

2. Test cluster health using Helm test.

$ Helm test elasticsearch

如上面的提示所示，您可以使用以下命令查看集群的状态：

$ kubectl get pods --namespace=default -l app=elasticsearch-master -w
$ helm test elasticsearch

1 2	$ kubectl get pods --namespace=default -l app=elasticsearch-master -w $ helm test elasticsearch

完成后，可以看看状态：

NAME                     READY     STATUS     RESTARTS   AGE
elasticsearch-master-0   1/1       Running   0         1m
elasticsearch-master-2   1/1       Running   0         1m
elasticsearch-master-1   1/1       Running   0         1m

NAME READY STATUS RESTARTS AGE

elasticsearch-master-0 1/1 Running 0 1m

elasticsearch-master-2 1/1 Running 0 1m

elasticsearch-master-1 1/1 Running 0 1m

过在本地进行测试来完成设置。您可以通过执行转发到本地计算机的端口来访问群集。

$ kubectl port-forward svc/elasticsearch-master 9200

1	$ kubectl port-forward svc/elasticsearch-master 9200

步骤4：部署Kibana
对于Kibana部署，我们将采用完全相同的方法，只更新Kibana的相关图表和values.yaml：

---

elasticsearchURL: "" # "http://elasticsearch-master:9200"
elasticsearchHosts: "http://elasticsearch-master:9200"

replicas: 1

# Extra environment variables to append to this nodeGroup
# This will be appended to the current 'env:' key. You can use any of the kubernetes env
# syntax here
extraEnvs: []
#  - name: MY_ENVIRONMENT_VAR
#    value: the_value_goes_here

# A list of secrets and their paths to mount inside the pod
# This is useful for mounting certificates for security and for mounting
# the X-Pack license
secretMounts: []
#  - name: kibana-keystore
#    secretName: kibana-keystore
#    path: /usr/share/kibana/data/kibana.keystore
#    subPath: kibana.keystore # optional

image: "docker.elastic.co/kibana/kibana"
imageTag: "7.6.1"
imagePullPolicy: "IfNotPresent"

# additionals labels
labels: {}

podAnnotations: {}
  # iam.amazonaws.com/role: es-cluster

resources:
  requests:
    cpu: "100m"
    memory: "500Mi"
  limits:
    cpu: "1000m"
    memory: "1Gi"

protocol: http

serverHost: "0.0.0.0"

healthCheckPath: "/app/kibana"

# Allows you to add any config files in /usr/share/kibana/config/
# such as kibana.yml
kibanaConfig: {}
#   kibana.yml: |
#     key:
#       nestedkey: value

# If Pod Security Policy in use it may be required to specify security context as well as service account

podSecurityContext:
  fsGroup: 1000

securityContext:
  capabilities:
    drop:
    - ALL
  # readOnlyRootFilesystem: true
  runAsNonRoot: true
  runAsUser: 1000

serviceAccount: ""

# This is the PriorityClass settings as defined in
# https://kubernetes.io/docs/concepts/configuration/pod-priority-preemption/#priorityclass
priorityClassName: ""

httpPort: 5601

extraContainers: ""
# - name: dummy-init
#   image: busybox
#   command: ['echo', 'hey']

extraInitContainers: ""
# - name: dummy-init
#   image: busybox
#   command: ['echo', 'hey']

updateStrategy:
  type: "Recreate"

service:
  type: ClusterIP
  port: 5601
  nodePort: "" 
  labels: {}
  annotations: {}
    # cloud.google.com/load-balancer-type: "Internal"
    # service.beta.kubernetes.io/aws-load-balancer-internal: 0.0.0.0/0
    # service.beta.kubernetes.io/azure-load-balancer-internal: "true"
    # service.beta.kubernetes.io/openstack-internal-load-balancer: "true"
    # service.beta.kubernetes.io/cce-load-balancer-internal-vpc: "true"
  loadBalancerSourceRanges: []
    # 0.0.0.0/0

ingress:
  enabled: false
  annotations: {}
    # kubernetes.io/ingress.class: nginx
    # kubernetes.io/tls-acme: "true"
  path: /
  hosts:
    - chart-example.local
  tls: []
  #  - secretName: chart-example-tls
  #    hosts:
  #      - chart-example.local

readinessProbe:
  failureThreshold: 3
  initialDelaySeconds: 10
  periodSeconds: 10
  successThreshold: 3
  timeoutSeconds: 5

imagePullSecrets: []
nodeSelector: {}
tolerations: []
affinity: {}

nameOverride: ""
fullnameOverride: ""

lifecycle: {}
  # preStop:
  #   exec:
  #     command: ["/bin/sh", "-c", "echo Hello from the postStart handler > /usr/share/message"]
  # postStart:
  #   exec:
  #     command: ["/bin/sh", "-c", "echo Hello from the postStart handler > /usr/share/message"]

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

---

elasticsearchURL: "" # "http://elasticsearch-master:9200"

elasticsearchHosts: "http://elasticsearch-master:9200"

replicas: 1

# Extra environment variables to append to this nodeGroup

# This will be appended to the current 'env:' key. You can use any of the kubernetes env

# syntax here

extraEnvs: []

# - name: MY_ENVIRONMENT_VAR

# value: the_value_goes_here

# A list of secrets and their paths to mount inside the pod

# This is useful for mounting certificates for security and for mounting

# the X-Pack license

secretMounts: []

# - name: kibana-keystore

# secretName: kibana-keystore

# path: /usr/share/kibana/data/kibana.keystore

# subPath: kibana.keystore # optional

image: "docker.elastic.co/kibana/kibana"

imageTag: "7.6.1"

imagePullPolicy: "IfNotPresent"

# additionals labels

labels: {}

podAnnotations: {}

# iam.amazonaws.com/role: es-cluster

resources:

requests:

cpu: "100m"

memory: "500Mi"

limits:

cpu: "1000m"

memory: "1Gi"

protocol: http

serverHost: "0.0.0.0"

healthCheckPath: "/app/kibana"

# Allows you to add any config files in /usr/share/kibana/config/

# such as kibana.yml

kibanaConfig: {}

# kibana.yml: |

# key:

# nestedkey: value

# If Pod Security Policy in use it may be required to specify security context as well as service account

podSecurityContext:

fsGroup: 1000

securityContext:

capabilities:

drop:

- ALL

# readOnlyRootFilesystem: true

runAsNonRoot: true

runAsUser: 1000

serviceAccount: ""

# This is the PriorityClass settings as defined in

# https://kubernetes.io/docs/concepts/configuration/pod-priority-preemption/#priorityclass

priorityClassName: ""

httpPort: 5601

extraContainers: ""

# - name: dummy-init

# image: busybox

# command: ['echo', 'hey']

extraInitContainers: ""

# - name: dummy-init

# image: busybox

# command: ['echo', 'hey']

updateStrategy:

type: "Recreate"

service:

type: ClusterIP

port: 5601

nodePort: ""

labels: {}

annotations: {}

# cloud.google.com/load-balancer-type: "Internal"

# service.beta.kubernetes.io/aws-load-balancer-internal: 0.0.0.0/0

# service.beta.kubernetes.io/azure-load-balancer-internal: "true"

# service.beta.kubernetes.io/openstack-internal-load-balancer: "true"

# service.beta.kubernetes.io/cce-load-balancer-internal-vpc: "true"

loadBalancerSourceRanges: []

# 0.0.0.0/0

ingress:

enabled: false

annotations: {}

# kubernetes.io/ingress.class: nginx

# kubernetes.io/tls-acme: "true"

path: /

hosts:

- chart-example.local

tls: []

# - secretName: chart-example-tls

# hosts:

# - chart-example.local

readinessProbe:

failureThreshold: 3

initialDelaySeconds: 10

periodSeconds: 10

successThreshold: 3

timeoutSeconds: 5

imagePullSecrets: []

nodeSelector: {}

tolerations: []

affinity: {}

nameOverride: ""

fullnameOverride: ""

lifecycle: {}

# preStop:

# exec:

# command: ["/bin/sh", "-c", "echo Hello from the postStart handler > /usr/share/message"]

# postStart:

# exec:

# command: ["/bin/sh", "-c", "echo Hello from the postStart handler > /usr/share/message"]

运行脚本：

$ ./elk-setup install elasticsearch
NAME:   kibana
LAST DEPLOYED: Thu Apr 1 09:52:21 2020
NAMESPACE: default
STATUS: DEPLOYED

$ ./elk-setup install elasticsearch

NAME: kibana

LAST DEPLOYED: Thu Apr 1 09:52:21 2020

NAMESPACE: default

STATUS: DEPLOYED

查看状态：

kubectl get pods
NAME                             READY     STATUS    RESTARTS   AGE
elasticsearch-master-0           1/1       Running   0          15m
elasticsearch-master-1           1/1       Running   0          15m
elasticsearch-master-2           1/1       Running   0          15m
kibana-kibana-6d7466b9b9-fbmsz   1/1       Running   0          2m

kubectl get pods

NAME READY STATUS RESTARTS AGE

elasticsearch-master-0 1/1 Running 0 15m

elasticsearch-master-1 1/1 Running 0 15m

elasticsearch-master-2 1/1 Running 0 15m

kibana-kibana-6d7466b9b9-fbmsz 1/1 Running 0 2m

端口转发：

$ kubectl port-forward deployment/kibana-kibana 5601

1	$ kubectl port-forward deployment/kibana-kibana 5601

访问测试： http://localhost:5601

步骤5：部署metricbeat
下一步将是部署metricbeat。
它是Elastic提供的常用节拍之一，负责发送指标数据。它可以将数据直接发送到Elasticsearch或通过Logstash发送，您可以在此处进一步处理和增强数据，然后再在Kibana中对其进行可视化。
以下是Elastic提供的所有节拍的完整列表。他们每个人都负责收集某种数据并将其运送到Elasticsearch集群。请花一点时间熟悉Beats文档，因为在本文中我们将不对它们进行太多详细介绍。

$ ./elk-setup install metricbeat

1	$ ./elk-setup install metricbeat

---
# Allows you to add any config files in /usr/share/metricbeat
# such as metricbeat.yml
metricbeatConfig:
  metricbeat.yml: |
    metricbeat.modules:
    - module: kubernetes
      metricsets:
        - container
        - node
        - pod
        - system
        - volume
      period: 10s
      host: "${NODE_NAME}"
      hosts: ["${NODE_NAME}:10255"]
      processors:
      - add_kubernetes_metadata:
          in_cluster: true
    - module: kubernetes
      enabled: true
      metricsets:
        - event
    - module: system
      period: 10s
      metricsets:
        - cpu
        - load
        - memory
        - network
        - process
        - process_summary
      processes: ['.*']
      process.include_top_n:
        by_cpu: 5
        by_memory: 5
    - module: system
      period: 1m
      metricsets:
        - filesystem
        - fsstat
      processors:
      - drop_event.when.regexp:
          system.filesystem.mount_point: '^/(sys|cgroup|proc|dev|etc|host|lib)($|/)'
    output.elasticsearch:
      hosts: '${ELASTICSEARCH_HOSTS:elasticsearch-master:9200}'
  kube-state-metrics-metricbeat.yml: |
    metricbeat.modules:
    - module: kubernetes
      enabled: true
      metricsets:
        - state_node
        - state_deployment
        - state_replicaset
        - state_pod
        - state_container
      period: 10s
      hosts: ["${KUBE_STATE_METRICS_HOSTS}"]
    output.elasticsearch:
      hosts: '${ELASTICSEARCH_HOSTS:elasticsearch-master:9200}'
# Replicas being used for the kube-state-metrics metricbeat deployment

replicas: 1

extraContainers: ""
# - name: dummy-init
#   image: busybox
#   command: ['echo', 'hey']

extraInitContainers: ""
# - name: dummy-init
#   image: busybox
#   command: ['echo', 'hey']

# Extra environment variables to append to the DaemonSet pod spec.
# This will be appended to the current 'env:' key. You can use any of the kubernetes env
# syntax here
extraEnvs: []
#  - name: MY_ENVIRONMENT_VAR
#    value: the_value_goes_here

extraVolumeMounts: []
  # - name: extras
  #   mountPath: /usr/share/extras
  #   readOnly: true

extraVolumes: []
  # - name: extras
  #   emptyDir: {}

envFrom: []
  # - configMapRef:
  #     name: config-secret

# Root directory where metricbeat will write data to in order to persist registry data across pod restarts (file position and other metadata).
hostPathRoot: /var/lib

image: "docker.elastic.co/beats/metricbeat"
imageTag: "7.6.2"
imagePullPolicy: "IfNotPresent"
imagePullSecrets: []

livenessProbe:
  exec:
    command:
      - sh
      - -c
      - |
        #!/usr/bin/env bash -e
        curl --fail 127.0.0.1:5066
  failureThreshold: 3
  initialDelaySeconds: 10
  periodSeconds: 10
  timeoutSeconds: 5

readinessProbe:
  exec:
    command:
      - sh
      - -c
      - |
        #!/usr/bin/env bash -e
        metricbeat test output
  failureThreshold: 3
  initialDelaySeconds: 10
  periodSeconds: 10
  timeoutSeconds: 5

# additionals labels
labels: {}

# Whether this chart should self-manage its service account, role, and associated role binding.
managedServiceAccount: true

clusterRoleRules:
  - apiGroups:
    - "extensions"
    - "apps"
    - ""
    resources:
    - namespaces
    - pods
    - events
    - deployments
    - nodes
    - replicasets
    verbs:
    - get
    - list
    - watch

podAnnotations: {}
  # iam.amazonaws.com/role: es-cluster

# Various pod security context settings. Bear in mind that many of these have an impact on metricbeat functioning properly.
#
# - Filesystem group for the metricbeat user. The official elastic docker images always have an id of 1000.
# - User that the container will execute as. Typically necessary to run as root (0) in order to properly collect host container logs.
# - Whether to execute the metricbeat containers as privileged containers. Typically not necessarily unless running within environments such as OpenShift.
podSecurityContext:
  runAsUser: 0
  privileged: false

resources:
  requests:
    cpu: "100m"
    memory: "100Mi"
  limits:
    cpu: "1000m"
    memory: "200Mi"

# Custom service account override that the pod will use
serviceAccount: ""

# A list of secrets and their paths to mount inside the pod
# This is useful for mounting certificates for security other sensitive values
secretMounts: []
#  - name: metricbeat-certificates
#    secretName: metricbeat-certificates
#    path: /usr/share/metricbeat/certs

# How long to wait for metricbeat pods to stop gracefully
terminationGracePeriod: 30

tolerations: []

nodeSelector: {}

affinity: {}

# This is the PriorityClass settings as defined in
# https://kubernetes.io/docs/concepts/configuration/pod-priority-preemption/#priorityclass
priorityClassName: ""

updateStrategy: RollingUpdate

# Override various naming aspects of this chart
# Only edit these if you know what you're doing
nameOverride: ""
fullnameOverride: ""

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

157

158

159

160

161

162

163

164

165

166

167

168

169

170

171

172

173

174

175

176

177

178

179

180

181

182

183

184

185

186

187

188

189

190

191

192

193

194

195

196

197

198

199

200

---

# Allows you to add any config files in /usr/share/metricbeat

# such as metricbeat.yml

metricbeatConfig:

metricbeat.yml: |

metricbeat.modules:

- module: kubernetes

metricsets:

- container

- node

- pod

- system

- volume

period: 10s

host: "${NODE_NAME}"

hosts: ["${NODE_NAME}:10255"]

processors:

- add_kubernetes_metadata:

in_cluster: true

- module: kubernetes

enabled: true

metricsets:

- event

- module: system

period: 10s

metricsets:

- cpu

- load

- memory

- network

- process

- process_summary

processes: ['.*']

process.include_top_n:

by_cpu: 5

by_memory: 5

- module: system

period: 1m

metricsets:

- filesystem

- fsstat

processors:

- drop_event.when.regexp:

system.filesystem.mount_point: '^/(sys|cgroup|proc|dev|etc|host|lib)($|/)'

output.elasticsearch:

hosts: '${ELASTICSEARCH_HOSTS:elasticsearch-master:9200}'

kube-state-metrics-metricbeat.yml: |

metricbeat.modules:

- module: kubernetes

enabled: true

metricsets:

- state_node

- state_deployment

- state_replicaset

- state_pod

- state_container

period: 10s

hosts: ["${KUBE_STATE_METRICS_HOSTS}"]

output.elasticsearch:

hosts: '${ELASTICSEARCH_HOSTS:elasticsearch-master:9200}'

# Replicas being used for the kube-state-metrics metricbeat deployment

replicas: 1

extraContainers: ""

# - name: dummy-init

# image: busybox

# command: ['echo', 'hey']

extraInitContainers: ""

# - name: dummy-init

# image: busybox

# command: ['echo', 'hey']

# Extra environment variables to append to the DaemonSet pod spec.

# This will be appended to the current 'env:' key. You can use any of the kubernetes env

# syntax here

extraEnvs: []

# - name: MY_ENVIRONMENT_VAR

# value: the_value_goes_here

extraVolumeMounts: []

# - name: extras

# mountPath: /usr/share/extras

# readOnly: true

extraVolumes: []

# - name: extras

# emptyDir: {}

envFrom: []

# - configMapRef:

# name: config-secret

# Root directory where metricbeat will write data to in order to persist registry data across pod restarts (file position and other metadata).

hostPathRoot: /var/lib

image: "docker.elastic.co/beats/metricbeat"

imageTag: "7.6.2"

imagePullPolicy: "IfNotPresent"

imagePullSecrets: []

livenessProbe:

exec:

command:

- sh

- -c

- |

#!/usr/bin/env bash -e

curl --fail 127.0.0.1:5066

failureThreshold: 3

initialDelaySeconds: 10

periodSeconds: 10

timeoutSeconds: 5

readinessProbe:

exec:

command:

- sh

- -c

- |

#!/usr/bin/env bash -e

metricbeat test output

failureThreshold: 3

initialDelaySeconds: 10

periodSeconds: 10

timeoutSeconds: 5

# additionals labels

labels: {}

# Whether this chart should self-manage its service account, role, and associated role binding.

managedServiceAccount: true

clusterRoleRules:

- apiGroups:

- "extensions"

- "apps"

- ""

resources:

- namespaces

- pods

- events

- deployments

- nodes

- replicasets

verbs:

- get

- list

- watch

podAnnotations: {}

# iam.amazonaws.com/role: es-cluster

# Various pod security context settings. Bear in mind that many of these have an impact on metricbeat functioning properly.

# - Filesystem group for the metricbeat user. The official elastic docker images always have an id of 1000.

# - User that the container will execute as. Typically necessary to run as root (0) in order to properly collect host container logs.

# - Whether to execute the metricbeat containers as privileged containers. Typically not necessarily unless running within environments such as OpenShift.

podSecurityContext:

runAsUser: 0

privileged: false

resources:

requests:

cpu: "100m"

memory: "100Mi"

limits:

cpu: "1000m"

memory: "200Mi"

# Custom service account override that the pod will use

serviceAccount: ""

# A list of secrets and their paths to mount inside the pod

# This is useful for mounting certificates for security other sensitive values

secretMounts: []

# - name: metricbeat-certificates

# secretName: metricbeat-certificates

# path: /usr/share/metricbeat/certs

# How long to wait for metricbeat pods to stop gracefully

terminationGracePeriod: 30

tolerations: []

nodeSelector: {}

affinity: {}

# This is the PriorityClass settings as defined in

# https://kubernetes.io/docs/concepts/configuration/pod-priority-preemption/#priorityclass

priorityClassName: ""

updateStrategy: RollingUpdate

# Override various naming aspects of this chart

# Only edit these if you know what you're doing

nameOverride: ""

fullnameOverride: ""

如果我们仔细检查metricbeat-values.yaml文件，我们会注意到它配置了各种度量收集模块（kubernetes，系统等）。 Metricbeat模块定义了要从服务中收集哪些特定指标，定义了收集这些指标的频率以及如何连接到该指标。模块由一个或多个指标集组成-实际上，这是一组要收集和运送的相关指标。
模块分解：
metricbeat.yml文件，提供Metricbeat的常规配置（包含从每个节点上的kubelet收集的系统和kubernetes模块）
kube-state-metrics-metricbeat.yml — kube-state-metrics是一项简单的服务，它侦听Kubernetes API服务器并生成有关Kubernetes内部各种对象（如部署，节点和Pod）的状态和运行状况的度量。
我们将把Metricbeat作为DaemonSet部署在k8s集群的每个节点上。通过将metricbeat部署为DaemonSet，我们确保在集群的每个节点上都获得一个正在运行的metricbeat守护程序。
请注意，在配置中，所有带有state_前缀的指标集的requirehosts字段均指向集群中的kube-state-metrics服务，而其余指标应指向kubelet服务。
让我们测试一下设置：

$ curl localhost:9200/_cat/indices

Handling connection for 9200
green open .kibana_task_manager_1    1 1     2 2     55.5kb  27.6kb
green open .apm-agent-configuration  1 1     0 0       566b    283b
green open ilm-history-1-000001      1 1    18 0     50.9kb  25.4kb
green open .kibana_1                 1 1    77 6      2.2mb   1.1mb
green open metricbeat-7.6.1          1 1    1512060 0 1.7gb 910.5mb

$ curl localhost:9200/_cat/indices

Handling connection for 9200

green open .kibana_task_manager_1 1 1 2 2 55.5kb 27.6kb

green open .apm-agent-configuration 1 1 0 0 566b 283b

green open ilm-history-1-000001 1 1 18 0 50.9kb 25.4kb

green open .kibana_1 1 1 77 6 2.2mb 1.1mb

green open metricbeat-7.6.1 1 1 1512060 0 1.7gb 910.5mb

默认情况下，用于Metricbeat的Prometheus模块公开/ metrics端点，这意味着这些度量也可以由现有Prometheus设置收集。
步骤6：部署Filebeat

$ ./elk-setup install filebeat
Release "filebeat-1586246704" has been upgraded. Happy Helming!
NAME: filebeat-1586246704
LAST DEPLOYED: Wed Apr  8 12:48:24 2020
NAMESPACE: log
STATUS: deployed
REVISION: 4
TEST SUITE: None
NOTES:
1. Watch all containers come up.
  $ kubectl get pods --namespace=log -l app=filebeat-1586246704-filebeat -w
Updated filebeat

$ ./elk-setup install filebeat

Release "filebeat-1586246704" has been upgraded. Happy Helming!

NAME: filebeat-1586246704

LAST DEPLOYED: Wed Apr 8 12:48:24 2020

NAMESPACE: log

STATUS: deployed

REVISION: 4

TEST SUITE: None

NOTES:

1. Watch all containers come up.

$ kubectl get pods --namespace=log -l app=filebeat-1586246704-filebeat -w

Updated filebeat

步骤7：将所有内容放在一起
如果检查创建的设置，则可以检查Helm的发布状态（为简单起见已删除了某些字段）：

$ helm ls
NAME          NAMESPACE REVISION    STATUS    CHART   APP VERSION                            
elasticsearch log      1   deployed elasticsearch-7.6.1 7.6.1      
kibana        log      1   deployed kibana-7.6.1        7.6.1      
metricbeat    log      1   deployed metricbeat-7.6.1    7.6.1
filebeat      log      1   deployed filebeat-7.6.1      7.6.1

$ helm ls

NAME NAMESPACE REVISION STATUS CHART APP VERSION

elasticsearch log 1 deployed elasticsearch-7.6.1 7.6.1

kibana log 1 deployed kibana-7.6.1 7.6.1

metricbeat log 1 deployed metricbeat-7.6.1 7.6.1

filebeat log 1 deployed filebeat-7.6.1 7.6.1

要查看有关创建的Kubernetes资源的详细信息，可以运行以下命令：

$ helm get manifest <RELEASE-NAME> | kubectl get -f -
$ helm get manifest kibana-1585817443 | kubectl get -f -
NAME                               TYPE        CLUSTER-IP    EXTERNAL-IP   PORT(S)    AGE
service/kibana-1585817443-kibana   ClusterIP   10.20.5.20   <none>        5601/TCP   25h
NAME                                       READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/kibana-1585817443-kibana   1/1     1            1           25h

$ helm get manifest <RELEASE-NAME> | kubectl get -f -

$ helm get manifest kibana-1585817443 | kubectl get -f -

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE

service/kibana-1585817443-kibana ClusterIP 10.20.5.20 <none> 5601/TCP 25h

NAME READY UP-TO-DATE AVAILABLE AGE

deployment.apps/kibana-1585817443-kibana 1/1 1 1 25h

步骤9：在Kibana中分析数据
现在我们已经部署了所有组件，在Kibana中，我们可以转到管理→Kibana→索引模式页面，然后单击创建索引模式。 Kibana将自动识别并显示Metricbeat索引。
输入“ metricbeat- *”，然后在下一步中选择@timestamp字段，以最终在Kibana中创建索引模式。
跳至“发现”页面。您会看到显示了Metricbeat从Kubernetes集群收集的所有指标：

如果您在Kibana UI中导航，则可以浏览其提供的各个字段，仪表板和统计信息。
跳到机器学习选项卡并选择metricbeat- *索引以可视化我们之前配置的大量Kubernetes和System指标。

http：// localhost：5601 / app / infra＃/ infrastructure / inventory
检查基础结构清单并选择感兴趣的资源，以浏览其指标并查看其日志文件。

关于Kibana仪表板的一句话：
Kubernetes的Kibana仪表板非常有用，因为它们为我们提供了Kubernetes集群及其组件的良好概述。 Elastic Demo Page上提供了一个非常不错的Dashboard Explorer。
此时，Elastic Helm图表默认情况下不启用Filebeat和Metricbeat仪表板。官方文档中提供了如何实现此目标的功能，并且可能需要对默认的values.yaml文件进行一些补充：
Metricbeat仪表板设置
Filebeat仪表板设置
最后的话
您已经获得了一个简单的设置，可以使用官方的Elastic Helm Charts在Kubernetes上部署Elastic Stack（Elasticsearch，Kibana，Metricbeat，Filebeat）。
现在，我们已经在群集上启动并运行了所有这些组件，因此存在无限可能的增强潜力：高可用性和高性能集群配置，自定义保留策略，安全性增强，用户可访问性改进等等。您可以自由探索无尽的选项，并以相同的方式进行试验，就像这是在本地或云中托管的VM上的传统部署一样。

https://github.com/elastic/helm-charts

原文：https://itnext.io/deploy-elastic-stack-on-kubernetes-1-15-using-helm-v3-9105653c7c8

头脑的思考

头脑的思考

使用Helm v3在Kubernetes 1.15上安装Elastic Stack

2024年四月
M	T	W	T	F	S	S
« Jun
1	2	3	4	5	6	7
8	9	10	11	12	13	14
15	16	17	18	19	20	21
22	23	24	25	26	27	28
29	30