蓝易云

Elasticsearch 搜索和数据分析引擎基本安装和使用教程

396次阅读
没有评论

共计 23677 个字符,预计需要花费 60 分钟才能阅读完成。

一、简介和安装

ElasticSearch 是一个基于 Lucene 的搜索服务器。它提供了一个分布式多用户能力的全文搜索引擎,基于 RESTful web 接口。

这里使用 Docker 安装 Elasticsearch:

docker run -d \
--name elasticsearch \
-p 9200:9200 \
-p 9300:9300 \
-e "discovery.type=single-node" \
-v /home/docker/es/plugins:/usr/share/elasticsearch/plugins \
-v /home/docker/es/data:/usr/share/elasticsearch/data \
--restart always \
elasticsearch:7.14.2

如果启动报错,执行 chmod 777 /home/docker/es/data

访问 9200 端口,能够看见如下信息则表示搭建成功。

Elasticsearch 搜索和数据分析引擎基本安装和使用教程

二、相关概念

  • 索引(Index): ES将数据存储于一个或多个索引中,索引是具有类似特性的文档的集合。类比传统的关系型数据库领域来说,索引相当于SQL中的一个数据库,或者一个数据存储方案(schema)。索引由其名称(必须为全小写字符)进行标识,并通过引用此名称完成文档的创建、搜索、更新及删除操作。一个ES集群中可以按需创建任意数目的索引。

  • 类型(Type):类型是索引内部的逻辑分区(category/partition),然而其意义完全取决于用户需求。因此,一个索引内部可定义一个或多个类型(type)。一般来说,类型就是为那些拥有相同的域的文档做的预定义。例如,在索引中,可以定义一个用于存储用户数据的类型,一个存储日志数据的类型,以及一个存储评论数据的类型。类比传统的关系型数据库领域来说,类型相当于“表”。

  • 文档(Document):文档是Lucene索引和搜索的原子单位,它是包含了一个或多个域的容器,基于JSON格式进行表示。文档由一个或多个域组成,每个域拥有一个名字及一个或多个值,有多个值的域通常称为“多值域”。每个文档可以存储不同的域集,但同一类型下的文档至应该有某种程度上的相似之处。

  • 映射(Mapping): ES中,所有的文档在存储之前都要首先进行分析。用户可根据需要定义如何将文本分割成token、哪些token应该被过滤掉,以及哪些文本需要进行额外处理等等。另外,ES还提供了额外功能,例如将域中的内容按需排序。事实上,ES也能自动根据其值确定域的类型。

  • 节点(Node):运行了单个实例的ES主机称为节点,它是集群的一个成员,可以存储数据、参与集群索引及搜索操作。类似于集群,节点靠其名称进行标识,默认为启动时自动生成的随机Marvel字符名称。用户可以按需要自定义任何希望使用的名称,但出于管理的目的,此名称应该尽可能有较好的识别性。节点通过为其配置的ES集群名称确定其所要加入的集群。

  • 分片(Shard)和副本(Replica):ES的“分片(shard)”机制可将一个索引内部的数据分布地存储于多个节点,它通过将一个索引切分为多个底层物理的Lucene索引完成索引数据的分割存储功能,这每一个物理的Lucene索引称为一个分片(shard)。每个分片其内部都是一个全功能且独立的索引,因此可由集群中的任何主机存储。创建索引时,用户可指定其分片的数量,默认数量为5个。

三、基本使用

1、基础

1.1 索引

1.1.1 创建索引
PUT http://127.0.0.1:9200/sj
{
    "acknowledged": true,
    "shards_acknowledged": true,
    "index": "sj"
}
1.1.2 查看所有索引
GET http://127.0.0.1:9200/_cat/indices
green  open .geoip_databases                69SccWg_Qy-WcYTahh-x7g 1 0   46    7  48.2mb  48.2mb
yellow open bank                            p1ZJWKzLSL2yPe_NfhED5w 1 1 1000    0 372.7kb 372.7kb
green  open .apm-custom-link                sfwTjQZjTMy7QZgQfR-0Zg 1 0    0    0    208b    208b
yellow open my_index                        a_wq-ITURwCCxxa5TCQgAA 1 1    0    0    208b    208b
green  open .kibana_task_manager_7.14.2_001 zi4N4a88S5uJC3JDjRvu7A 1 0   14 2758 474.9kb 474.9kb
yellow open sj                              Frku6z9qQ-ih4cNV-CcyiQ 1 1    1    2  17.7kb  17.7kb
green  open .apm-agent-configuration        cv7NcBeERa-ZwoRuXEgbgQ 1 0    0    0    208b    208b
green  open .kibana_7.14.2_001              2qIDo8JJSGuJj7DdBOYzAg 1 0   44    1   2.3mb   2.3mb
yellow open 2sj                             xAc5JA7WSD-YSGei6RhDaQ 1 1    0    0    208b    208b
yellow open index                           WvnJ6OIRQ2GGewXderW0ew 1 1    4    0    16kb    16kb
green  open .tasks                          sn8q8od2QNC9rujgg51RGQ 1 0    8    0  36.6kb  36.6kb
green  open .kibana-event-log-7.14.2-000001 7_Wq_AiMRjemTW3nOOSD2g 1 0    5    0  27.3kb  27.3kb
1.1.3 删除索引
DELETE http://127.0.0.1:9200/sj
{
    "acknowledged": true
}

1.2 文档

1.2.1 创建文档
POST http://127.0.0.1:9200/sj/sj/1001
{
    "title": "小米手机",
    "catagory": "xiaomi",
    "images": "www.baidu.com",
    "price": 8999
}
{
    "_index": "sj",
    "_type": "sj",
    "_id": "1001",
    "_version": 2,
    "result": "updated",
    "_shards": {
        "total": 2,
        "successful": 1,
        "failed": 0
    },
    "_seq_no": 1,
    "_primary_term": 9
}
1.2.2 查询单个文档
GET http://127.0.0.1:9200/sj/sj/1001
{
    "_index": "sj",
    "_type": "sj",
    "_id": "1001",
    "_version": 2,
    "_seq_no": 1,
    "_primary_term": 9,
    "found": true,
    "_source": {
        "title": "小米手机",
        "catagory": "xiaomi",
        "images": "www.baidu.com",
        "price": 8999
    }
}
1.2.3 查询单个索引下所有文档
GET http://127.0.0.1:9200/sj/_search
{
    "took": 2,
    "timed_out": false,
    "_shards": {
        "total": 1,
        "successful": 1,
        "skipped": 0,
        "failed": 0
    },
    "hits": {
        "total": {
            "value": 1,
            "relation": "eq"
        },
        "max_score": 1.0,
        "hits": [
            {
                "_index": "sj",
                "_type": "sj",
                "_id": "1001",
                "_score": 1.0,
                "_source": {
                    "title": "小米手机",
                    "catagory": "xiaomi",
                    "images": "www.baidu.com",
                    "price": 8999
                }
            }
        ]
    }
}
1.2.4 局部更新文档
POST http://127.0.0.1:9200/sj/sj/1001/_update
{
    "doc": {
        "title": "华为手机"
    }
}
{
    "_index": "sj",
    "_type": "sj",
    "_id": "1001",
    "_version": 3,
    "result": "updated",
    "_shards": {
        "total": 2,
        "successful": 1,
        "failed": 0
    },
    "_seq_no": 2,
    "_primary_term": 9
}

再次查询结果:

{
    "_index": "sj",
    "_type": "sj",
    "_id": "1001",
    "_version": 3,
    "_seq_no": 2,
    "_primary_term": 9,
    "found": true,
    "_source": {
        "title": "华为手机",
        "catagory": "xiaomi",
        "images": "www.baidu.com",
        "price": 8999
    }
}
1.2.5 完全覆盖更新
POST http://127.0.0.1:9200/sj/sj/1001
{
    "title": "锤子手机"
}
{
    "_index": "sj",
    "_type": "sj",
    "_id": "1001",
    "_version": 4,
    "result": "updated",
    "_shards": {
        "total": 2,
        "successful": 1,
        "failed": 0
    },
    "_seq_no": 3,
    "_primary_term": 9
}

再次查询结果:

{
    "_index": "sj",
    "_type": "sj",
    "_id": "1001",
    "_version": 4,
    "_seq_no": 3,
    "_primary_term": 9,
    "found": true,
    "_source": {
        "title": "锤子手机"
    }
}
1.2.6 删除文档
DELETE http://127.0.0.1:9200/sj/sj/1001
{
    "_index": "sj",
    "_type": "sj",
    "_id": "1001",
    "_version": 5,
    "result": "deleted",
    "_shards": {
        "total": 2,
        "successful": 1,
        "failed": 0
    },
    "_seq_no": 4,
    "_primary_term": 9
}

2、进阶

2.1 批量操作

2.1.1 批量创建
POST http://127.0.0.1:9200/sj/sj/_bulk
{"index":{"_id":"11"}}
{"price":10,"productID":"华为"}
{"index":{"_id":"12"}}
{"price":20,"productID":"小米"}
{"index":{"_id":"13"}}
{"price":30,"productID":"oppo"}
{"index":{"_id":"14"}}
{"price":40,"productID":"vivo"}
{
    "took": 654,
    "errors": false,
    "items": [
        {
            "index": {
                "_index": "sj",
                "_type": "sj",
                "_id": "11",
                "_version": 1,
                "result": "created",
                "_shards": {
                    "total": 2,
                    "successful": 1,
                    "failed": 0
                },
                "_seq_no": 5,
                "_primary_term": 9,
                "status": 201
            }
        },
        {
            "index": {
                "_index": "sj",
                "_type": "sj",
                "_id": "12",
                "_version": 1,
                "result": "created",
                "_shards": {
                    "total": 2,
                    "successful": 1,
                    "failed": 0
                },
                "_seq_no": 6,
                "_primary_term": 9,
                "status": 201
            }
        },
        {
            "index": {
                "_index": "sj",
                "_type": "sj",
                "_id": "13",
                "_version": 1,
                "result": "created",
                "_shards": {
                    "total": 2,
                    "successful": 1,
                    "failed": 0
                },
                "_seq_no": 7,
                "_primary_term": 9,
                "status": 201
            }
        },
        {
            "index": {
                "_index": "sj",
                "_type": "sj",
                "_id": "14",
                "_version": 1,
                "result": "created",
                "_shards": {
                    "total": 2,
                    "successful": 1,
                    "failed": 0
                },
                "_seq_no": 8,
                "_primary_term": 9,
                "status": 201
            }
        }
    ]
}
2.1.2 批量增删改
POST http://127.0.0.1:9200/sj/sj/_bulk
{"delete":{"_index":"sj","_type":"sj","_id":"11"}}
{"create":{"_index":"sj","_type":"sj","_id":"10"}}
{"price":50,"productID":"1115"}
{"index":{"_index":"sj","_type":"sj","_id":"16"}}
{"price":60,"productID":"1116"}
{"update":{"_index":"sj","_type":"sj","_id":"12"}}
{"doc": {"price":120,"productID":"11120"}}
{
    "took": 141,
    "errors": false,
    "items": [
        {
            "index": {
                "_index": "sj",
                "_type": "sj",
                "_id": "11",
                "_version": 2,
                "result": "updated",
                "_shards": {
                    "total": 2,
                    "successful": 1,
                    "failed": 0
                },
                "_seq_no": 9,
                "_primary_term": 9,
                "status": 200
            }
        },
        {
            "index": {
                "_index": "sj",
                "_type": "sj",
                "_id": "12",
                "_version": 2,
                "result": "updated",
                "_shards": {
                    "total": 2,
                    "successful": 1,
                    "failed": 0
                },
                "_seq_no": 10,
                "_primary_term": 9,
                "status": 200
            }
        },
        {
            "index": {
                "_index": "sj",
                "_type": "sj",
                "_id": "13",
                "_version": 2,
                "result": "updated",
                "_shards": {
                    "total": 2,
                    "successful": 1,
                    "failed": 0
                },
                "_seq_no": 11,
                "_primary_term": 9,
                "status": 200
            }
        },
        {
            "index": {
                "_index": "sj",
                "_type": "sj",
                "_id": "14",
                "_version": 2,
                "result": "updated",
                "_shards": {
                    "total": 2,
                    "successful": 1,
                    "failed": 0
                },
                "_seq_no": 12,
                "_primary_term": 9,
                "status": 200
            }
        }
    ]
}

2.2 普通查询

2.2.1 数据导入
POST http://127.0.0.1:9200/bank/account/_bulk
... (es 官方测试数据,可以 github 中找到)
{
    "took": 2450,
    "errors": false,
    ...
}
2.2.2 分页查询
GET http://127.0.0.1:9200/bank/_search
{
    "query":{
        "match_all": {}
    },
    "from": 0,
    "size": 2
}
{
    "took": 1,
    "timed_out": false,
    "_shards": {
        "total": 1,
        "successful": 1,
        "skipped": 0,
        "failed": 0
    },
    "hits": {
        "total": {
            "value": 1000,
            "relation": "eq"
        },
        "max_score": 1.0,
        "hits": [
            {
                "_index": "bank",
                "_type": "account",
                "_id": "1",
                "_score": 1.0,
                "_source": {
                    "account_number": 1,
                    "balance": 39225,
                    "firstname": "Amber",
                    "lastname": "Duke",
                    "age": 32,
                    "gender": "M",
                    "address": "880 Holmes Lane",
                    "employer": "Pyrami",
                    "email": "amberduke@pyrami.com",
                    "city": "Brogan",
                    "state": "IL"
                }
            },
            {
                "_index": "bank",
                "_type": "account",
                "_id": "6",
                "_score": 1.0,
                "_source": {
                    "account_number": 6,
                    "balance": 5686,
                    "firstname": "Hattie",
                    "lastname": "Bond",
                    "age": 36,
                    "gender": "M",
                    "address": "671 Bristol Street",
                    "employer": "Netagy",
                    "email": "hattiebond@netagy.com",
                    "city": "Dante",
                    "state": "TN"
                }
            }
        ]
    }
}
2.2.3 返回部分字段
POST http://127.0.0.1:9200/bank/_search
{
    "query":{
        "match_all":{}
    },
    "sort":[
        {
          "balance": {
            "order": "desc"
          }  
        }
    ],
    "from": 0,
    "size": 2,
    "_source": [
        "firstname",
        "lastname",
        "city",
        "age"
    ]
}
{
    "took": 2,
    "timed_out": false,
    "_shards": {
        "total": 1,
        "successful": 1,
        "skipped": 0,
        "failed": 0
    },
    "hits": {
        "total": {
            "value": 1000,
            "relation": "eq"
        },
        "max_score": null,
        "hits": [
            {
                "_index": "bank",
                "_type": "account",
                "_id": "248",
                "_score": null,
                "_source": {
                    "firstname": "West",
                    "city": "Maury",
                    "age": 36,
                    "lastname": "England"
                },
                "sort": [
                    49989
                ]
            },
            {
                "_index": "bank",
                "_type": "account",
                "_id": "854",
                "_score": null,
                "_source": {
                    "firstname": "Jimenez",
                    "city": "Moscow",
                    "age": 25,
                    "lastname": "Barry"
                },
                "sort": [
                    49795
                ]
            }
        ]
    }
}
2.2.4 数值等值匹配

对于数值,只查询等值匹配的数据

GET http://127.0.0.1:9200/bank/_search
{
    "query":{
        "match":{
            "account_number": 20
        }
    }
}
{
    "took": 0,
    "timed_out": false,
    "_shards": {
        "total": 1,
        "successful": 1,
        "skipped": 0,
        "failed": 0
    },
    "hits": {
        "total": {
            "value": 1,
            "relation": "eq"
        },
        "max_score": 1.0,
        "hits": [
            {
                "_index": "bank",
                "_type": "account",
                "_id": "20",
                "_score": 1.0,
                "_source": {
                    "account_number": 20,
                    "balance": 16418,
                    "firstname": "Elinor",
                    "lastname": "Ratliff",
                    "age": 36,
                    "gender": "M",
                    "address": "282 Kings Place",
                    "employer": "Scentric",
                    "email": "elinorratliff@scentric.com",
                    "city": "Ribera",
                    "state": "WA"
                }
            }
        ]
    }
}
2.2.5 字符串模糊匹配

将匹配类似的数据

GET http://127.0.0.1:9200/bank/_search
{
    "query":{
        "match":{
            "address": "mill lane foajfoaofjja"
        }
    }
}
{
    "took": 1,
    "timed_out": false,
    "_shards": {
        "total": 1,
        "successful": 1,
        "skipped": 0,
        "failed": 0
    },
    ...
}
2.2.6 短语匹配

不拆分关键字,当作一个整体

GET http://127.0.0.1:9200/bank/_search
{
    "query":{
        "match_phrase":{
            "address": "mill lane"
        }
    }
}
{
    "took": 6,
    "timed_out": false,
    "_shards": {
        "total": 1,
        "successful": 1,
        "skipped": 0,
        "failed": 0
    },
    "hits": {
        "total": {
            "value": 1,
            "relation": "eq"
        },
        "max_score": 9.507477,
        "hits": [
            {
                "_index": "bank",
                "_type": "account",
                "_id": "136",
                "_score": 9.507477,
                "_source": {
                    "account_number": 136,
                    "balance": 45801,
                    "firstname": "Winnie",
                    "lastname": "Holland",
                    "age": 38,
                    "gender": "M",
                    "address": "198 Mill Lane",
                    "employer": "Neteria",
                    "email": "winnieholland@neteria.com",
                    "city": "Urie",
                    "state": "IL"
                }
            }
        ]
    }
}
2.2.7 多字段匹配
GET http://127.0.0.1:9200/bank/_search
{
    "query":{
        "multi_match":{
            "query": "mill Movico",
            "fields": ["address", "city"]
        }
    }
}
{
    "took": 5,
    "timed_out": false,
    "_shards": {
        "total": 1,
        "successful": 1,
        "skipped": 0,
        "failed": 0
    },
    "hits": {
        "total": {
            "value": 4,
            "relation": "eq"
        },
        "max_score": 6.5059485,
        "hits": [
            {
                "_index": "bank",
                "_type": "account",
                "_id": "472",
                "_score": 6.5059485,
                "_source": {
                    "account_number": 472,
                    "balance": 25571,
                    "firstname": "Lee",
                    "lastname": "Long",
                    "age": 32,
                    "gender": "F",
                    "address": "288 Mill Street",
                    "employer": "Comverges",
                    "email": "leelong@comverges.com",
                    "city": "Movico",
                    "state": "MT"
                }
            },
            {
                "_index": "bank",
                "_type": "account",
                "_id": "970",
                "_score": 5.4032025,
                "_source": {
                    "account_number": 970,
                    "balance": 19648,
                    "firstname": "Forbes",
                    "lastname": "Wallace",
                    "age": 28,
                    "gender": "M",
                    "address": "990 Mill Road",
                    "employer": "Pheast",
                    "email": "forbeswallace@pheast.com",
                    "city": "Lopezo",
                    "state": "AK"
                }
            },
            {
                "_index": "bank",
                "_type": "account",
                "_id": "136",
                "_score": 5.4032025,
                "_source": {
                    "account_number": 136,
                    "balance": 45801,
                    "firstname": "Winnie",
                    "lastname": "Holland",
                    "age": 38,
                    "gender": "M",
                    "address": "198 Mill Lane",
                    "employer": "Neteria",
                    "email": "winnieholland@neteria.com",
                    "city": "Urie",
                    "state": "IL"
                }
            },
            {
                "_index": "bank",
                "_type": "account",
                "_id": "345",
                "_score": 5.4032025,
                "_source": {
                    "account_number": 345,
                    "balance": 9812,
                    "firstname": "Parker",
                    "lastname": "Hines",
                    "age": 38,
                    "gender": "M",
                    "address": "715 Mill Avenue",
                    "employer": "Baluba",
                    "email": "parkerhines@baluba.com",
                    "city": "Blackgum",
                    "state": "KY"
                }
            }
        ]
    }
}
2.2.8 复合查询
GET http://127.0.0.1:9200/bank/_search
{
    "query":{
        "bool":{
            "must":[
                {"match": {
                    "gender": "M"
                }},
                {"match": {
                    "address": "mill"
                }}
            ],
            "must_not":[
               {
                   "match":{
                       "age": 28
                   }
               }
            ],
            "should":[
                {"match": {
                    "lastname": "Wallace"
                }}
            ]
        }
    }
}
{
    "took": 6,
    "timed_out": false,
    "_shards": {
        "total": 1,
        "successful": 1,
        "skipped": 0,
        "failed": 0
    },
    "hits": {
        "total": {
            "value": 2,
            "relation": "eq"
        },
        "max_score": 6.0824604,
        "hits": [
            {
                "_index": "bank",
                "_type": "account",
                "_id": "136",
                "_score": 6.0824604,
                "_source": {
                    "account_number": 136,
                    "balance": 45801,
                    "firstname": "Winnie",
                    "lastname": "Holland",
                    "age": 38,
                    "gender": "M",
                    "address": "198 Mill Lane",
                    "employer": "Neteria",
                    "email": "winnieholland@neteria.com",
                    "city": "Urie",
                    "state": "IL"
                }
            },
            {
                "_index": "bank",
                "_type": "account",
                "_id": "345",
                "_score": 6.0824604,
                "_source": {
                    "account_number": 345,
                    "balance": 9812,
                    "firstname": "Parker",
                    "lastname": "Hines",
                    "age": 38,
                    "gender": "M",
                    "address": "715 Mill Avenue",
                    "employer": "Baluba",
                    "email": "parkerhines@baluba.com",
                    "city": "Blackgum",
                    "state": "KY"
                }
            }
        ]
    }
}
2.2.9 过滤

must should会贡献相关性得分,filter 不参与

POST http://127.0.0.1:9200/bank/_search
{
    "query":{
        "bool":{
            "must":[
                {"match": {
                    "gender": "M"
                }},
                {"match": {
                    "address": "mill"
                }}
            ],
            "must_not":[
               {
                   "match":{
                       "age": 28
                   }
               }
            ],
            "should":[
                {"match": {
                    "lastname": "Wallace"
                }}
            ],
            "filter":[
                {"range":{
                    "age":{
                        "gte": 18,
                        "lte": 40
                    }
                }}
            ]
        }
    }
}
{
    "took": 2,
    "timed_out": false,
    "_shards": {
        "total": 1,
        "successful": 1,
        "skipped": 0,
        "failed": 0
    },
    "hits": {
        "total": {
            "value": 2,
            "relation": "eq"
        },
        "max_score": 6.0824604,
        "hits": [
            {
                "_index": "bank",
                "_type": "account",
                "_id": "136",
                "_score": 6.0824604,
                "_source": {
                    "account_number": 136,
                    "balance": 45801,
                    "firstname": "Winnie",
                    "lastname": "Holland",
                    "age": 38,
                    "gender": "M",
                    "address": "198 Mill Lane",
                    "employer": "Neteria",
                    "email": "winnieholland@neteria.com",
                    "city": "Urie",
                    "state": "IL"
                }
            },
            {
                "_index": "bank",
                "_type": "account",
                "_id": "345",
                "_score": 6.0824604,
                "_source": {
                    "account_number": 345,
                    "balance": 9812,
                    "firstname": "Parker",
                    "lastname": "Hines",
                    "age": 38,
                    "gender": "M",
                    "address": "715 Mill Avenue",
                    "employer": "Baluba",
                    "email": "parkerhines@baluba.com",
                    "city": "Blackgum",
                    "state": "KY"
                }
            }
        ]
    }
}
2.2.10 term

和match类似,推荐:找精确字段使用term,找文本字段使用match

GET http://127.0.0.1:9200/bank/_search
{
    "query":{
        "term":{
            "account_number": 20
        }
    }
}

2.3 聚合查询

2.3.1 聚合一

搜索 address中包含mill的所有人的年龄分布以及平均年龄

GET http://127.0.0.1:9200/bank/_search
{
    "query":{
        "match":{
            "address": "mill"
        }
    },
    "aggs": {
        "ageAgg": {
            "terms":{
                "field": "age",
                "size": 10
            }
        },
        "ageAvg": {
             "avg": {
                 "field": "age"
             }
        },
        "balanceAvg": {
            "avg": {
                "field": "balance"
            }
        }
    },
    "size": 0
}
{
    "took": 2,
    "timed_out": false,
    "_shards": {
        "total": 1,
        "successful": 1,
        "skipped": 0,
        "failed": 0
    },
    "hits": {
        "total": {
            "value": 4,
            "relation": "eq"
        },
        "max_score": null,
        "hits": []
    },
    "aggregations": {
        "ageAgg": {
            "doc_count_error_upper_bound": 0,
            "sum_other_doc_count": 0,
            "buckets": [
                {
                    "key": 38,
                    "doc_count": 2
                },
                {
                    "key": 28,
                    "doc_count": 1
                },
                {
                    "key": 32,
                    "doc_count": 1
                }
            ]
        },
        "ageAvg": {
            "value": 34.0
        },
        "balanceAvg": {
            "value": 25208.0
        }
    }
}
2.3.2 聚合二

按照年龄聚合,并且请求这些年龄段的这些人的平均薪资

GET http://127.0.0.1:9200/bank/_search
{
    "query":{
        "match_all": {}
    },
    "aggs": {
        "ageAgg": {
            "terms": {
                "field": "age",
                "size": 100
            },
            "aggs": {
                "ageAvg": {
                    "avg": {
                        "field": "balance"
                    }
                }
            }
        }
    },
    "size": 0
}
{
    "query":{
        "match_all": {}
    },
    "aggs": {
        "ageAgg": {
            "terms": {
                "field": "age",
                "size": 100
            },
            "aggs": {
                "ageAvg": {
                    "avg": {
                        "field": "balance"
                    }
                }
            }
        }
    },
    "size": 0
}
2.3.3 聚合三

查出所有年龄分布,并且这些年龄段中M的平均薪资和F的平均薪资以及这个年龄段的总体平均薪资

GET http://127.0.0.1:9200/bank/_search
{
    "query":{
        "match_all": {}
    },
    "aggs": {
        "ageAgg": {
            "terms": {
                "field": "age",
                "size": 100
            },
            "aggs": {
                "genderAgg": {
                    "terms": {
                        "field": "gender.keyword"
                    },
                    "aggs": {
                        "balanceAvg": {
                            "avg": {
                                "field": "balance"
                            }
                        }
                    }
                },
                "balanceAvg": {
                    "avg": {
                        "field": "balance"
                    }
                }
            }
        }
    },
    "size": 0

}
{
    "took": 7,
    "timed_out": false,
    "_shards": {
        "total": 1,
        "successful": 1,
        "skipped": 0,
        "failed": 0
    },
    "hits": {
        "total": {
            "value": 1000,
            "relation": "eq"
        },
        "max_score": null,
        "hits": []
    },
    "aggregations": {
        "ageAgg": {
            "doc_count_error_upper_bound": 0,
            "sum_other_doc_count": 0,
            "buckets": [
                {
                    "key": 31,
                    "doc_count": 61,
                    "genderAgg": {
                        "doc_count_error_upper_bound": 0,
                        "sum_other_doc_count": 0,
                        "buckets": [
                            {
                                "key": "M",
                                "doc_count": 35,
                                "balanceAvg": {
                                    "value": 29565.628571428573
                                }
                            },
                            {
                                "key": "F",
                                "doc_count": 26,
                                "balanceAvg": {
                                    "value": 26626.576923076922
                                }
                            }
                        ]
                    },
                    "balanceAvg": {
                        "value": 28312.918032786885
                    }
                },
                {
                    "key": 39,
                    "doc_count": 60,
                    "genderAgg": {
                        "doc_count_error_upper_bound": 0,
                        "sum_other_doc_count": 0,
                        "buckets": [
                            {
                                "key": "F",
                                "doc_count": 38,
                                "balanceAvg": {
                                    "value": 26348.684210526317
                                }
                            },
                            {
                                "key": "M",
                                "doc_count": 22,
                                "balanceAvg": {
                                    "value": 23405.68181818182
                                }
                            }
                        ]
                    },
                    "balanceAvg": {
                        "value": 25269.583333333332
                    }
                },
                {
                    "key": 26,
                    "doc_count": 59,
                    "genderAgg": {
                        "doc_count_error_upper_bound": 0,
                        "sum_other_doc_count": 0,
                        "buckets": [
                            {
                                "key": "M",
                                "doc_count": 32,
                                "balanceAvg": {
                                    "value": 25094.78125
                                }
                            },
                            {
                                "key": "F",
                                "doc_count": 27,
                                "balanceAvg": {
                                    "value": 20943.0
                                }
                            }
                        ]
                    },
                    "balanceAvg": {
                        "value": 23194.813559322032
                    }
                },
                {
                    "key": 32,
                    "doc_count": 52,
                    "genderAgg": {
                        "doc_count_error_upper_bound": 0,
                        "sum_other_doc_count": 0,
                        "buckets": [
                            {
                                "key": "M",
                                "doc_count": 28,
                                "balanceAvg": {
                                    "value": 22941.964285714286
                                }
                            },
                            {
                                "key": "F",
                                "doc_count": 24,
                                "balanceAvg": {
                                    "value": 25128.958333333332
                                }
                            }
                        ]
                    },
                    "balanceAvg": {
                        "value": 23951.346153846152
                    }
                },
                {
                    "key": 35,
                    "doc_count": 52,
                    "genderAgg": {
                        "doc_count_error_upper_bound": 0,
                        "sum_other_doc_count": 0,
                        "buckets": [
                            {
                                "key": "M",
                                "doc_count": 28,
                                "balanceAvg": {
                                    "value": 24226.321428571428
                                }
                            },
                            {
                                "key": "F",
                                "doc_count": 24,
                                "balanceAvg": {
                                    "value": 19698.791666666668
                                }
                            }
                        ]
                    },
                    "balanceAvg": {
                        "value": 22136.69230769231
                    }
                },
                {
                    "key": 36,
                    "doc_count": 52,
                    "genderAgg": {
                        "doc_count_error_upper_bound": 0,
                        "sum_other_doc_count": 0,
                        "buckets": [
                            {
                                "key": "M",
                                "doc_count": 31,
                                "balanceAvg": {
                                    "value": 20884.677419354837
                                }
                            },
                            {
                                "key": "F",
                                "doc_count": 21,
                                "balanceAvg": {
                                    "value": 24079.04761904762
                                }
                            }
                        ]
                    },
                    "balanceAvg": {
                        "value": 22174.71153846154
                    }
                },
                {
                    "key": 22,
                    "doc_count": 51,
                    "genderAgg": {
                        "doc_count_error_upper_bound": 0,
                        "sum_other_doc_count": 0,
                        "buckets": [
                            {
                                "key": "F",
                                "doc_count": 27,
                                "balanceAvg": {
                                    "value": 22152.74074074074
                                }
                            },
                            {
                                "key": "M",
                                "doc_count": 24,
                                "balanceAvg": {
                                    "value": 27631.708333333332
                                }
                            }
                        ]
                    },
                    "balanceAvg": {
                        "value": 24731.07843137255
                    }
                },
                {
                    "key": 28,
                    "doc_count": 51,
                    "genderAgg": {
                        "doc_count_error_upper_bound": 0,
                        "sum_other_doc_count": 0,
                        "buckets": [
                            {
                                "key": "F",
                                "doc_count": 31,
                                "balanceAvg": {
                                    "value": 27076.8064516129
                                }
                            },
                            {
                                "key": "M",
                                "doc_count": 20,
                                "balanceAvg": {
                                    "value": 30129.35
                                }
                            }
                        ]
                    },
                    "balanceAvg": {
                        "value": 28273.882352941175
                    }
                },
                {
                    "key": 33,
                    "doc_count": 50,
                    "genderAgg": {
                        "doc_count_error_upper_bound": 0,
                        "sum_other_doc_count": 0,
                        "buckets": [
                            {
                                "key": "F",
                                "doc_count": 26,
                                "balanceAvg": {
                                    "value": 26437.615384615383
                                }
                            },
                            {
                                "key": "M",
                                "doc_count": 24,
                                "balanceAvg": {
                                    "value": 23638.291666666668
                                }
                            }
                        ]
                    },
                    "balanceAvg": {
                        "value": 25093.94
                    }
                },
                {
                    "key": 34,
                    "doc_count": 49,
                    "genderAgg": {
                        "doc_count_error_upper_bound": 0,
                        "sum_other_doc_count": 0,
                        "buckets": [
                            {
                                "key": "F",
                                "doc_count": 30,
                                "balanceAvg": {
                                    "value": 26039.166666666668
                                }
                            },
                            {
                                "key": "M",
                                "doc_count": 19,
                                "balanceAvg": {
                                    "value": 28027.0
                                }
                            }
                        ]
                    },
                    "balanceAvg": {
                        "value": 26809.95918367347
                    }
                },
                {
                    "key": 30,
                    "doc_count": 47,
                    "genderAgg": {
                        "doc_count_error_upper_bound": 0,
                        "sum_other_doc_count": 0,
                        "buckets": [
                            {
                                "key": "F",
                                "doc_count": 25,
                                "balanceAvg": {
                                    "value": 25316.16
                                }
                            },
                            {
                                "key": "M",
                                "doc_count": 22,
                                "balanceAvg": {
                                    "value": 20028.545454545456
                                }
                            }
                        ]
                    },
                    "balanceAvg": {
                        "value": 22841.106382978724
                    }
                },
                {
                    "key": 21,
                    "doc_count": 46,
                    "genderAgg": {
                        "doc_count_error_upper_bound": 0,
                        "sum_other_doc_count": 0,
                        "buckets": [
                            {
                                "key": "F",
                                "doc_count": 24,
                                "balanceAvg": {
                                    "value": 28210.916666666668
                                }
                            },
                            {
                                "key": "M",
                                "doc_count": 22,
                                "balanceAvg": {
                                    "value": 25640.18181818182
                                }
                            }
                        ]
                    },
                    "balanceAvg": {
                        "value": 26981.434782608696
                    }
                },
                {
                    "key": 40,
                    "doc_count": 45,
                    "genderAgg": {
                        "doc_count_error_upper_bound": 0,
                        "sum_other_doc_count": 0,
                        "buckets": [
                            {
                                "key": "M",
                                "doc_count": 24,
                                "balanceAvg": {
                                    "value": 26474.958333333332
                                }
                            },
                            {
                                "key": "F",
                                "doc_count": 21,
                                "balanceAvg": {
                                    "value": 27992.571428571428
                                }
                            }
                        ]
                    },
                    "balanceAvg": {
                        "value": 27183.17777777778
                    }
                },
                {
                    "key": 20,
                    "doc_count": 44,
                    "genderAgg": {
                        "doc_count_error_upper_bound": 0,
                        "sum_other_doc_count": 0,
                        "buckets": [
                            {
                                "key": "M",
                                "doc_count": 27,
                                "balanceAvg": {
                                    "value": 29047.444444444445
                                }
                            },
                            {
                                "key": "F",
                                "doc_count": 17,
                                "balanceAvg": {
                                    "value": 25666.647058823528
                                }
                            }
                        ]
                    },
                    "balanceAvg": {
                        "value": 27741.227272727272
                    }
                },
                {
                    "key": 23,
                    "doc_count": 42,
                    "genderAgg": {
                        "doc_count_error_upper_bound": 0,
                        "sum_other_doc_count": 0,
                        "buckets": [
                            {
                                "key": "M",
                                "doc_count": 24,
                                "balanceAvg": {
                                    "value": 27730.75
                                }
                            },
                            {
                                "key": "F",
                                "doc_count": 18,
                                "balanceAvg": {
                                    "value": 26758.833333333332
                                }
                            }
                        ]
                    },
                    "balanceAvg": {
                        "value": 27314.214285714286
                    }
                },
                {
                    "key": 24,
                    "doc_count": 42,
                    "genderAgg": {
                        "doc_count_error_upper_bound": 0,
                        "sum_other_doc_count": 0,
                        "buckets": [
                            {
                                "key": "F",
                                "doc_count": 23,
                                "balanceAvg": {
                                    "value": 29414.521739130436
                                }
                            },
                            {
                                "key": "M",
                                "doc_count": 19,
                                "balanceAvg": {
                                    "value": 27435.052631578947
                                }
                            }
                        ]
                    },
                    "balanceAvg": {
                        "value": 28519.04761904762
                    }
                },
                {
                    "key": 25,
                    "doc_count": 42,
                    "genderAgg": {
                        "doc_count_error_upper_bound": 0,
                        "sum_other_doc_count": 0,
                        "buckets": [
                            {
                                "key": "M",
                                "doc_count": 23,
                                "balanceAvg": {
                                    "value": 29336.08695652174
                                }
                            },
                            {
                                "key": "F",
                                "doc_count": 19,
                                "balanceAvg": {
                                    "value": 25156.263157894737
                                }
                            }
                        ]
                    },
                    "balanceAvg": {
                        "value": 27445.214285714286
                    }
                },
                {
                    "key": 37,
                    "doc_count": 42,
                    "genderAgg": {
                        "doc_count_error_upper_bound": 0,
                        "sum_other_doc_count": 0,
                        "buckets": [
                            {
                                "key": "M",
                                "doc_count": 23,
                                "balanceAvg": {
                                    "value": 25015.739130434784
                                }
                            },
                            {
                                "key": "F",
                                "doc_count": 19,
                                "balanceAvg": {
                                    "value": 29451.21052631579
                                }
                            }
                        ]
                    },
                    "balanceAvg": {
                        "value": 27022.261904761905
                    }
                },
                {
                    "key": 27,
                    "doc_count": 39,
                    "genderAgg": {
                        "doc_count_error_upper_bound": 0,
                        "sum_other_doc_count": 0,
                        "buckets": [
                            {
                                "key": "F",
                                "doc_count": 21,
                                "balanceAvg": {
                                    "value": 21618.85714285714
                                }
                            },
                            {
                                "key": "M",
                                "doc_count": 18,
                                "balanceAvg": {
                                    "value": 21300.38888888889
                                }
                            }
                        ]
                    },
                    "balanceAvg": {
                        "value": 21471.871794871793
                    }
                },
                {
                    "key": 38,
                    "doc_count": 39,
                    "genderAgg": {
                        "doc_count_error_upper_bound": 0,
                        "sum_other_doc_count": 0,
                        "buckets": [
                            {
                                "key": "F",
                                "doc_count": 20,
                                "balanceAvg": {
                                    "value": 27931.65
                                }
                            },
                            {
                                "key": "M",
                                "doc_count": 19,
                                "balanceAvg": {
                                    "value": 24350.894736842107
                                }
                            }
                        ]
                    },
                    "balanceAvg": {
                        "value": 26187.17948717949
                    }
                },
                {
                    "key": 29,
                    "doc_count": 35,
                    "genderAgg": {
                        "doc_count_error_upper_bound": 0,
                        "sum_other_doc_count": 0,
                        "buckets": [
                            {
                                "key": "M",
                                "doc_count": 23,
                                "balanceAvg": {
                                    "value": 29943.17391304348
                                }
                            },
                            {
                                "key": "F",
                                "doc_count": 12,
                                "balanceAvg": {
                                    "value": 28601.416666666668
                                }
                            }
                        ]
                    },
                    "balanceAvg": {
                        "value": 29483.14285714286
                    }
                }
            ]
        }
    }
}

2.4 映射

2.4.1 查看映射关系
GET http://127.0.0.1:9200/bank/_mapping
{
    "bank": {
        "mappings": {
            "properties": {
                "account_number": {
                    "type": "long"
                },
                "address": {
                    "type": "text",
                    "fields": {
                        "keyword": {
                            "type": "keyword",
                            "ignore_above": 256
                        }
                    }
                },
                "age": {
                    "type": "long"
                },
                "balance": {
                    "type": "long"
                },
                "city": {
                    "type": "text",
                    "fields": {
                        "keyword": {
                            "type": "keyword",
                            "ignore_above": 256
                        }
                    }
                },
                "email": {
                    "type": "text",
                    "fields": {
                        "keyword": {
                            "type": "keyword",
                            "ignore_above": 256
                        }
                    }
                },
                "employer": {
                    "type": "text",
                    "fields": {
                        "keyword": {
                            "type": "keyword",
                            "ignore_above": 256
                        }
                    }
                },
                "firstname": {
                    "type": "text",
                    "fields": {
                        "keyword": {
                            "type": "keyword",
                            "ignore_above": 256
                        }
                    }
                },
                "gender": {
                    "type": "text",
                    "fields": {
                        "keyword": {
                            "type": "keyword",
                            "ignore_above": 256
                        }
                    }
                },
                "lastname": {
                    "type": "text",
                    "fields": {
                        "keyword": {
                            "type": "keyword",
                            "ignore_above": 256
                        }
                    }
                },
                "state": {
                    "type": "text",
                    "fields": {
                        "keyword": {
                            "type": "keyword",
                            "ignore_above": 256
                        }
                    }
                }
            }
        }
    }
}
2.4.2 创建时指定映射
PUT http://127.0.0.1:9200/my_index
{
  "mappings": {
    "properties": {
      "age": {"type": "integer"},
      "email": {"type": "keyword"},
      "name": {"type": "text"}
    }
  }
}
{
    "acknowledged": true,
    "shards_acknowledged": true,
    "index": "my_index"
}
2.4.3 添加新字段
PUT http://127.0.0.1:9200/my_index/_mapping
{
  "properties": {
    "employee-id": {
      "type": "keyword",
      "index": false
    }
  }
}
{
    "acknowledged": true
}
2.4.4 修改映射

本质:创建一个新的索引,然后进行数据迁移

PUT http://127.0.0.1:9200/my_index2
{
  "mappings": {
      "properties": {
        "account_number": {
          "type": "long"
        },
        "address": {
          "type": "text"
        },
        "age": {
          "type": "integer"
        },
        "balance": {
          "type": "long"
        },
        "city": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword",
              "ignore_above": 256
            }
          }
        },
        "email": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword",
              "ignore_above": 256
            }
          }
        },
        "employer": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword",
              "ignore_above": 256
            }
          }
        },
        "firstname": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword",
              "ignore_above": 256
            }
          }
        },
        "gender": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword",
              "ignore_above": 256
            }
          }
        },
        "lastname": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword",
              "ignore_above": 256
            }
          }
        },
        "state": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword",
              "ignore_above": 256
            }
          }
        }
      }
    }
}
{
    "acknowledged": true,
    "shards_acknowledged": true,
    "index": "my_index2"
}
2.4.5 数据迁移
POST http://127.0.0.1:9200/_reindex
{
  "source": {
    "index": "my_index",
    "type": "table"
  },
  "dest": {
    "index": "my_index3"
  }
}
{
    "took": 1,
    "timed_out": false,
    "total": 0,
    "updated": 0,
    "created": 0,
    "deleted": 0,
    "batches": 0,
    "version_conflicts": 0,
    "noops": 0,
    "retries": {
        "bulk": 0,
        "search": 0
    },
    "throttled_millis": 0,
    "requests_per_second": -1.0,
    "throttled_until_millis": 0,
    "failures": []
}

2.5 分词器

2.5.1 测试分词器
POST http://127.0.0.1:9200/_analyze
{
  "analyzer": "standard",
  "text": "I am huangzhen,come from China"
}
{
    "tokens": [
        {
            "token": "i",
            "start_offset": 0,
            "end_offset": 1,
            "type": "<ALPHANUM>",
            "position": 0
        },
        {
            "token": "am",
            "start_offset": 2,
            "end_offset": 4,
            "type": "<ALPHANUM>",
            "position": 1
        },
        {
            "token": "huangzhen",
            "start_offset": 5,
            "end_offset": 14,
            "type": "<ALPHANUM>",
            "position": 2
        },
        {
            "token": "come",
            "start_offset": 15,
            "end_offset": 19,
            "type": "<ALPHANUM>",
            "position": 3
        },
        {
            "token": "from",
            "start_offset": 20,
            "end_offset": 24,
            "type": "<ALPHANUM>",
            "position": 4
        },
        {
            "token": "china",
            "start_offset": 25,
            "end_offset": 30,
            "type": "<ALPHANUM>",
            "position": 5
        }
    ]
}

2.5.2 中文分词器

需要安装ik分词器

POST http://127.0.0.1:9200/_analyze
{
  "analyzer": "ik_max_word",
  "text": "我是中国人"
}
{
    "tokens": [
        {
            "token": "我",
            "start_offset": 0,
            "end_offset": 1,
            "type": "CN_CHAR",
            "position": 0
        },
        {
            "token": "是",
            "start_offset": 1,
            "end_offset": 2,
            "type": "CN_CHAR",
            "position": 1
        },
        {
            "token": "中国人",
            "start_offset": 2,
            "end_offset": 5,
            "type": "CN_WORD",
            "position": 2
        },
        {
            "token": "中国",
            "start_offset": 2,
            "end_offset": 4,
            "type": "CN_WORD",
            "position": 3
        },
        {
            "token": "国人",
            "start_offset": 3,
            "end_offset": 5,
            "type": "CN_WORD",
            "position": 4
        }
    ]
}

提醒:本文发布于612天前,文中所关联的信息可能已发生改变,请知悉!

AD:【腾讯云服务器大降价】2核4G 222元/3年 1核2G 38元/年
正文完
 0
阿蛮君
版权声明:本站原创文章,由 阿蛮君 于2023-04-10发表,共计23677字。
转载说明:除特殊说明外本站文章皆由CC-4.0协议发布,转载请注明出处。
评论(没有评论)
Copyright © 2022-2024 阿蛮君博客 湘ICP备2023001393号
本网站由 亿信互联 提供云计算服务 | 蓝易云CDN 提供安全防护和加速服务
Powered by Wordpress  Theme by Puock