使用ESM(Elasticsearch Migration)简单且快速的迁移ElasticSearch数据
超快速的 Elasticsearch 跨版本数据迁移工具。支持并行传输、兼容性强,已验证支持 1.x 到 7.x 的跨版本迁移。
官方地址
可选参数
# Options
-s, --source= source elasticsearch instance
-d, --dest= destination elasticsearch instance
-q, --query= query against source elasticsearch instance, filter data before migrate, ie: name:medcl
-m, --source_auth basic auth of source elasticsearch instance, ie: user:pass
-n, --dest_auth basic auth of target elasticsearch instance, ie: user:pass
-c, --count= number of documents at a time: ie "size" in the scroll request (10000)
--sliced_scroll_size= size of sliced scroll, to make it work, the size should be > 1, default:"1"
-t, --time= scroll time (1m)
--shards= set a number of shards on newly created indexes
--copy_settings copy index settings from source
--copy_mappings copy mappings mappings from source
-f, --force delete destination index before copying, default:false
-x, --src_indexes= list of indexes to copy, comma separated (_all), support wildcard match(*)
-y, --dest_index= indexes name to save, allow only one indexname, original indexname will be used if not specified
-a, --all copy indexes starting with . and _ (false)
-w, --workers= concurrency number for bulk workers, default is: "1"
-b --bulk_size bulk size in MB" default:5
-v --log setting log level,options:trace,debug,info,warn,error
-i --input_file indexing from local dump file, file format: {"_id":"xxx","_index":"xxx","_source":{"xxx":"xxx"},"_type":"xxx" }
-o --output_file output documents of source index into local file, file format same as input_file.
--source_proxy set proxy to source http connections, ie: http://127.0.0.1:8080
--dest_proxy set proxy to destination http connections, ie: http://127.0.0.1:8080
--refresh refresh after migration finished
快速开始
示例命令
esm-windows-amd64 -s http://es-host-source:9200 -m elastic:passwd -d http://es-host-dest:9200 -n elastic:passwd -x v1-price-bill -y v1-price-bill -w=5 -b=1000
查看目标 ElasticSearch 的索引。
curl http://es-host-dest:9200/_cat/indices