ELK Configure (with Docker)

ELK??

 

• ELK는 분석 및 저장 기능을 담당하는 ElasticSearch, 수집 기능을 하는 Logstash, 이를 시각화하는 도구인 Kibana의 앞글자만 딴 단어이다. ELK는 접근성과 용이성이 좋아 최근 가장 핫한 Log 및 데이터 분석 도구이다

1) ElasticSearch
- ElasticSearch는 Lucene 기반으로 개발한 분산 검색엔진으로, Logstash를 통해 수신된 데이터를 저장소에 저장하는 역할을 담당
- 정형, 비정형, 위치정보, 메트릭 등 원하는 방법으로 다양한 유형의 검색을 수행하고 결합할 수 있다.

2) Logstash
- 오픈소스 서버측 데이터 처리 파이프라인으로, 다양한 소스에서 동시에 데이터를 수집하고 변환하여 stash 보관소로 보낸다.
- 수집할 로그를 선정해서, 지정된 대상 서버(ElasticSearch)에 인덱싱하여 전송하는 역할을 담당하는 소프트웨어

3) Kibana
- 데이터를 시각적으로 탐색하고 실시간으로 분석 할 수 있다.

 

 

[사전필요사항]

- CentOS 7

- docker
- docker-compose
- git

 

[Docker Install]

$ yum -y install docker

$ service docker start

$ systemctl enable docker

$ docker --version

$ Docker version 1.13.1, build cccb291/1.13.1

 

[이전버전제거]
$ sudo yum remove docker \
                  docker-client \
                  docker-client-latest \
                  docker-common \
                  docker-latest \
                  docker-latest-logrotate \
                  docker-logrotate \
                  docker-engine

 

$ sudo yum install -y yum-utils

$ sudo yum-config-manager \
    --add-repo \
    https://download.docker.com/linux/centos/docker-ce.repo

 

$ sudo yum install docker-ce docker-ce-cli containerd.io

 

$ docker --version

$ Docker version 19.03.8, build afacb8b

 

$ service docker start

$ systemctl enable docker

 

 

 

[Docker Composer 설치 (버전확인필요)]

$ sudo curl -L "https://github.com/docker/compose/releases/download/1.25.5/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose

$ sudo chmod +x /usr/local/bin/docker-compose

$ docker-compose --version

$ docker-compose version 1.25.4, build 8d51620a

 

[빌드 & 설치]

[root@image-centos7 docker-elk]# docker-compose build && docker-compose up -d

Building elasticsearch

Step 1/2 : ARG ELK_VERSION

Step 2/2 : FROM docker.elastic.co/elasticsearch/elasticsearch:${ELK_VERSION}

7.6.2: Pulling from elasticsearch/elasticsearch

c808caf183b6: Pull complete

d6caf8e15a64: Pull complete

b0ba5f324e82: Pull complete

d7e8c1e99b9a: Pull complete

85c4d6c81438: Pull complete

3119218fac98: Pull complete

914accf214bb: Pull complete

Digest: sha256:59342c577e2b7082b819654d119f42514ddf47f0699c8b54dc1f0150250ce7aa

Status: Downloaded newer image for docker.elastic.co/elasticsearch/elasticsearch:7.6.2

 ---> f29a1ee41030

Successfully built f29a1ee41030

Successfully tagged docker-elk_elasticsearch:latest

Building logstash

Step 1/2 : ARG ELK_VERSION

Step 2/2 : FROM docker.elastic.co/logstash/logstash:${ELK_VERSION}

7.6.2: Pulling from logstash/logstash

c808caf183b6: Already exists

9706c92d19d9: Pull complete

1152fbc1420a: Pull complete

990e64a3de7c: Pull complete

79ed4d44f5cf: Pull complete

7a799895b8de: Pull complete

6634773ec0c9: Pull complete

7029aebbf168: Pull complete

f8918e25aa8e: Pull complete

9e30f4b0c65d: Pull complete

36d677a6c019: Pull complete

Digest: sha256:baed5f5bf04299994ea41881afb4d4985cb0f33427a2aef39223c75975bab60e

Status: Downloaded newer image for docker.elastic.co/logstash/logstash:7.6.2

 ---> fa5b3b1e9757

Successfully built fa5b3b1e9757

Successfully tagged docker-elk_logstash:latest

Building kibana

Step 1/2 : ARG ELK_VERSION

Step 2/2 : FROM docker.elastic.co/kibana/kibana:${ELK_VERSION}

7.6.2: Pulling from kibana/kibana

c808caf183b6: Already exists

2626d0d3032a: Pull complete

a5df092b79ca: Pull complete

37e516b76124: Pull complete

34f8e164abea: Pull complete

cab1ef4ac629: Pull complete

378eadcda05a: Pull complete

427475b87dd6: Pull complete

08d3e5897257: Pull complete

c84b8d77ec94: Pull complete

Digest: sha256:e8f3743e404462709663422056db2d5076a7a6bd6024f64aea1599b3014c63be

Status: Downloaded newer image for docker.elastic.co/kibana/kibana:7.6.2

 ---> f70986bc5191

Successfully built f70986bc5191

Successfully tagged docker-elk_kibana:latest

Creating network "docker-elk_elk" with driver "bridge"

Creating volume "docker-elk_elasticsearch" with default driver

Creating docker-elk_elasticsearch_1 ... done

Creating docker-elk_kibana_1        ... done

Creating docker-elk_logstash_1      ... done

 

 

# docker ps -a

CONTAINER ID        IMAGE                      COMMAND                  CREATED             STATUS              PORTS                                                                              NAMES

564b8952ba9e        docker-elk_logstash        "/usr/local/bin/dock…"   16 hours ago        Up 16 hours         0.0.0.0:5000->5000/tcp, 0.0.0.0:9600->9600/tcp, 0.0.0.0:5000->5000/udp, 5044/tcp   docker-elk_logstash_1

f3ea1739a537        docker-elk_kibana          "/usr/local/bin/dumb…"   16 hours ago        Up 16 hours         0.0.0.0:5601->5601/tcp                                                             docker-elk_kibana_1

efd6e2dd77bf        docker-elk_elasticsearch   "/usr/local/bin/dock…"   16 hours ago        Up 16 hours         0.0.0.0:9200->9200/tcp, 0.0.0.0:9300->9300/tcp                                     docker-elk_elasticsearch_1

 

[Kibana Login 확인]