Search This Blog

Friday, September 4, 2020

Elasticsearch with Geopoint

Install Elasticsearch on Mac

Install the tap

brew tap elastic/tap

Install Elasticsearch

brew install elastic/tap/elasticsearch-full

Start Elasticsearch

elasticsearch

Confirm the installation works

curl http://localhost:9200

Something similar to the following should be shown:

{
  "name" : "MACC02Y753HJGH5",
  "cluster_name" : "elasticsearch_zhentao.li",
  "cluster_uuid" : "mkwsHWW8SoCcPzXg_KOyuQ",
  "version" : {
    "number" : "7.9.1",
    "build_flavor" : "default",
    "build_type" : "tar",
    "build_hash" : "083627f112ba94dffc1232e8b42b73492789ef91",
    "build_date" : "2020-09-01T21:22:21.964974Z",
    "build_snapshot" : false,
    "lucene_version" : "8.6.2",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}

Search by Geo Distance

Define the index for Geo Type

put http://localhost:9200/regions
{
"mappings": {
"properties": {
"name": {
type: "text"
},
"location": {
"type": "geo_point"
}
}
}
}

With the following response:

{
"acknowledged": true,
"shards_acknowledged": true,
"index": "regions"
}

Download Geo data file

curl https://restcountries.eu/rest/v1/all | jq -c '.[] | {"index": {"_index": "regions", "_id": .alpha3Code}}, {name: .name, location: [.latlng[1], .latlng[0]] }' > regions

or download it from here.

Import the file

curl -H 'Content-Type: application/json'  -s -XPUT localhost:9200/_bulk --data-binary "@regions"

List all regions

http://localhost:9200/regions/_search?size=100
GET /regions/_search
{
"size" : 100,
"query": {
"match_all": {}
}
}

Filter by Geo distance

{
"size": 50,
"query": {
"bool": {
"must": {
"match_all": {}
},
"filter": {
"geo_distance": {
"distance": "3000km",
"location": {
"lat": 35,
"lon": 105
}
}
}
}
},
"sort": [
{
"_geo_distance": {
"location": {
"lat": 35,
"lon": 105
},
"order": "asc",
"unit": "km",
"distance_type": "arc"
}
}
]
}

The result is like below

{
"took": 15,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 19,
"relation": "eq"
},
"max_score": null,
"hits": [
{
"_index": "regions",
"_type": "_doc",
"_id": "CHN",
"_score": null,
"_source": {
"name": "China",
"location": [
105,
35
]
},
"sort": [
0.0
]
},
{
"_index": "regions",
"_type": "_doc",
"_id": "MNG",
"_score": null,
"_source": {
"name": "Mongolia",
"location": [
105,
46
]
},
"sort": [
1223.1458751104453
]
},
{
"_index": "regions",
"_type": "_doc",
"_id": "MMR",
"_score": null,
"_source": {
"name": "Myanmar",
"location": [
98,
22
]
},
"sort": [
1597.9864780876323
]
},
{
"_index": "regions",
"_type": "_doc",
"_id": "BTN",
"_score": null,
"_source": {
"name": "Bhutan",
"location": [
90.5,
27.5
]
},
"sort": [
1608.420695556179
]
},
{
"_index": "regions",
"_type": "_doc",
"_id": "MAC",
"_score": null,
"_source": {
"name": "Macau",
"location": [
113.55,
22.16666666
]
},
"sort": [
1651.5104987054583
]
},
{
"_index": "regions",
"_type": "_doc",
"_id": "HKG",
"_score": null,
"_source": {
"name": "Hong Kong",
"location": [
114.16666666,
22.25
]
},
"sort": [
1674.4580484734988
]
},
{
"_index": "regions",
"_type": "_doc",
"_id": "LAO",
"_score": null,
"_source": {
"name": "Laos",
"location": [
105,
18
]
},
"sort": [
1890.3163582803475
]
},
{
"_index": "regions",
"_type": "_doc",
"_id": "BGD",
"_score": null,
"_source": {
"name": "Bangladesh",
"location": [
90,
24
]
},
"sort": [
1894.1562153610287
]
},
{
"_index": "regions",
"_type": "_doc",
"_id": "TWN",
"_score": null,
"_source": {
"name": "Taiwan",
"location": [
121,
23.5
]
},
"sort": [
2006.2988706499348
]
},
{
"_index": "regions",
"_type": "_doc",
"_id": "PRK",
"_score": null,
"_source": {
"name": "North Korea",
"location": [
127,
40
]
},
"sort": [
2012.9111514604588
]
},
{
"_index": "regions",
"_type": "_doc",
"_id": "KOR",
"_score": null,
"_source": {
"name": "South Korea",
"location": [
127.5,
37
]
},
"sort": [
2031.4778900496392
]
},
{
"_index": "regions",
"_type": "_doc",
"_id": "VNM",
"_score": null,
"_source": {
"name": "Vietnam",
"location": [
107.83333333,
16.16666666
]
},
"sort": [
2113.0731526216714
]
},
{
"_index": "regions",
"_type": "_doc",
"_id": "NPL",
"_score": null,
"_source": {
"name": "Nepal",
"location": [
84,
28
]
},
"sort": [
2132.4138804176705
]
},
{
"_index": "regions",
"_type": "_doc",
"_id": "THA",
"_score": null,
"_source": {
"name": "Thailand",
"location": [
100,
15
]
},
"sort": [
2279.3258680765903
]
},
{
"_index": "regions",
"_type": "_doc",
"_id": "KHM",
"_score": null,
"_source": {
"name": "Cambodia",
"location": [
105,
13
]
},
"sort": [
2446.291756434398
]
},
{
"_index": "regions",
"_type": "_doc",
"_id": "KGZ",
"_score": null,
"_source": {
"name": "Kyrgyzstan",
"location": [
75,
41
]
},
"sort": [
2697.507030320444
]
},
{
"_index": "regions",
"_type": "_doc",
"_id": "RUS",
"_score": null,
"_source": {
"name": "Russia",
"location": [
100,
60
]
},
"sort": [
2803.2803033336695
]
},
{
"_index": "regions",
"_type": "_doc",
"_id": "JPN",
"_score": null,
"_source": {
"name": "Japan",
"location": [
138,
36
]
},
"sort": [
2975.112941971521
]
},
{
"_index": "regions",
"_type": "_doc",
"_id": "PHL",
"_score": null,
"_source": {
"name": "Philippines",
"location": [
122,
13
]
},
"sort": [
2983.948851295192
]
}
]
}
}

When you search by distance, Elasticsearch won’t return the distance itself as it isn’t a real field. If you need sorting, Elasticsearch advise to use the sort function to return the value.