elasticsearch

安装

优化

• 关闭swap

  swapoff -a
  # 注释 /etc/fstab swap
  sysctl -p
  

• unlimit调整

  sysctl -w vm.max_map_count=262144
  echo 'vm.max_map_count=262144' >> /etc/sysctl.conf
  

• 使用ssd

重启

curl -X PUT "localhost:9200/_cluster/settings" -H 'Content-Type: application/json' -d'
{
  "persistent": {
    "cluster.routing.allocation.enable": "none"
  }
}
'
curl -X POST "localhost:9200/_flush/synced"
supervisorctl restart elasticsearch
curl -X GET "localhost:9200/_cat/health"
curl -X PUT "localhost:9200/_cluster/settings" -H 'Content-Type: application/json' -d'
{
  "persistent": {
    "cluster.routing.allocation.enable": null
  }
}
'

踩坑

• 滚动重启

  重启前

  curl -XPUT http://192.168.67.14:9200/_cluster/settings -d '{
      "transient" : {
          "cluster.routing.allocation.enable" : "none"
      }
  }'
  

  重启后

  curl -XPUT http://192.168.67.14:9200/_cluster/settings -d '{
      "transient" : {
          "cluster.routing.allocation.enable" : "all"
      }
  }'
  

• 修改index.number_of_shards
• 磁盘损坏

  curl -XPOST 'http://192.168.60.32:9200/_cluster/reroute' -d '{
          "commands" : [ {
                "allocate" : {
                    "index" : ".marvel-2018.10.10",
                    "shard" : 0,
                    "node" : "1.7.5-192.168.67.14",
                    "allow_primary" : true
                }
              }
          ]
      }'
  
  

• unassigned shards

  curl http://127.0.0.1:9200/_cat/shards?v 2>&1 | grep UNASSIGNED
  

  curl http://127.0.0.1:9200/_nodes/process?pretty
  

  curl -XPOST 'http://192.168.60.32:9200/_cluster/reroute' -d '{
          "commands" : [ {
                  "allocate" : {
                      "index" : "logstash-xxx-up-2019.01.29",
                      "shard" : 9,
                      "node" : "kzI0751OSJBcx_wmLQ",
                      "allow_primary" : true
                  }
              }
          ]
      }'
  

• 修改@timestamp类型

  curl -XPUT "http://127.0.0.1:9200/_template/uplog" -H 'Content-Type: application/json' -H 'Host: elasticsearch.morgans' -d'
  {
    "index_patterns": ["uplog-*"],
    "settings": {
      "index.number_of_replicas": 0
    },
    "mappings": {
      "_default_": {
          "properties": {
              "@timestamp": {
                  "type": "date",
                  "doc_values": true
              }
          }
      }
    }
  }'
  

kibana

安装

优化

踩坑

• 保存时kibana报错 Visualization Editor: blocked by: [FORBIDDEN/12/index read-only

  参考 https://github.com/elastic/kibana/issues/13685

  curl -XPUT -H "Content-Type: application/json" http://127.0.0.1:9200/.kibana/_settings -d '{"index.blocks.read_only_allow_delete": null}'
  

• Timelion每次查询得到的数据不一样

  参考 https://github.com/elastic/kibana/issues/23428 和 https://discuss.elastic.co/t/timelion-gives-different-result-when-it-auto-refreshes/155182 修改kibana.yml配置

  elasticsearch.shardTimeout = 30000