Elasticsearch backup using snapshot

The snapshot and restore module allows to create snapshots of individual indices or an entire cluster into a remote repository like shared file system.These snapshots are great for backups because they can be restored relatively quickly but they are not archival because they can only be restored to versions of Elasticsearch that can read the index.Its very easy to take backup of your elasticsearch index.you can just make a snapshot, rsync/NFS export the files to another host and restore them from those files.

Snapshot repository : Register a snapshot repository in Elasticsearch.

curl -XPUT 'http://localhost:9200/_snapshot/backup' -d '{
  "type": "fs",
  "settings": {
    "location": "/home/wantcode/Documents/es/backup",
    "compress": true
  }
}'

Note : If path.repo is not set in your elasticsearch.yml(/etc/elasticsearch/elasticsearch.yml) configuration please set it.

path.repo = /home/wantcode/Documents/es/backup

Also set the permission for /home/wantcode/Documents/es/backup directory.

chown elasticsearch:elasticsearch /home/wantcode/Documents/es/backup.

Now restart your elastics server.(root user command otherwise add sudo)

service elasticsearch restart

After register the snapshot . Make a script for taking snapshot using script (takesnapshot.sh).

#!/bin/bash
SNAPSHOT=`date +%Y%m%d-%H%M%S`
curl -XPUT "localhost:9200/_snapshot/backup/$SNAPSHOT?wait_for_completion=true"

It’s very easy to set up this backup, there is currently no way included to remove old snapshots.so you can write a small script that keeps the last 30 snapshots and deletes anything older.

#!/bin/bash

# The amount of snapshots we want to keep.
LIMIT=30

# Name of our snapshot repository
REPO=backup

# Get a list of snapshots that we want to delete
SNAPSHOTS=`curl -s -XGET "localhost:9200/_snapshot/$REPO/_all" \
  | jq -r ".snapshots[:-${LIMIT}][].snapshot"`

# Loop over the results and delete each snapshot
for SNAPSHOT in $SNAPSHOTS
do
 echo "Deleting snapshot: $SNAPSHOT"
 curl -s -XDELETE "localhost:9200/_snapshot/$REPO/$SNAPSHOT?pretty"
done
echo "Done!"

Now you can check your /home/wantcode/Documents/es/backup folder all index folders are created so your backup is created over the folder. Now you can try to delete the some index and restore it using the script.

Get the list of snapshot using command.

curl -s -XGET "localhost:9200/_snapshot/backup/_all?pretty"

Output like :

{
  "snapshots" : [ {
    "snapshot" : "20170117-131953",
    "version_id" : 2010099,
    "version" : "2.1.0",
    "indices" : ["jdbc", "index_2111", "river", "catalog" ],
    "state" : "SUCCESS",
    "start_time" : "2017-01-17T07:49:53.817Z",
    "start_time_in_millis" : 1484639393817,
    "end_time" : "2017-01-17T07:51:05.626Z",
    "end_time_in_millis" : 1484639465626,
    "duration_in_millis" : 71809,
    "failures" : [ ],
    "shards" : {
      "total" : 25,
      "failed" : 0,
      "successful" : 25
    }
  } ]
}

Now make a script to restore a particular index (snapshotrestore.sh).

#!/bin/bash
#
# Restore a snapshot from our repository directory
SNAPSHOT=20170117-131953

# We need to close the index first
curl -XPOST "localhost:9200/catalog/_close"

# Restore the snapshot we want
curl -XPOST "http://localhost:9200/_snapshot/backup/$SNAPSHOT/_restore" -d '{
 "indices": "catalog"
}'

# Re-open the index
curl -XPOST 'localhost:9200/catalog/_open'

After run this script it will restore the catalog index in your elasticsearch server.please comment for any further help.

International PHP Conference 2017

The International PHP Conference is the world's first PHP conference and stands since more than a decade for top-notch pragmatic expertise in PHP and web technologies. At the IPC, internationally renowned experts from the PHP industry meet up with PHP users and developers from large and small companies.....Read More

Highlights : 

Expo with exiting exhibitors

Cool PHP & the Gang: 60+ international top speakers

90+ best practice sessions

Official certificate for attendees

Conference Combo: Visit the webinale For Free

All inclusive: Changing buffets, snacks & refreshing drinks

Save the event