Mengelola management LOG aplikasi menggunakan LOKI & GRAFANA

Introduction

Mengapa Log management itu penting?

Monitoring real-time aplikasi
Debugging dan troubleshooting
Audit trail dan compliance
Performance analysis

Kenapa Loki + Grafana?

Loki: Log aggregation system yang efisien (seperti Prometheus untuk logs)
Grafana: Visualization dan alerting yang powerful
Cost-effective dibanding ELK Stack
Native integration dengan ecosystem Grafana

Flow

Arhchitecture Overview

Prerequisites:

Docker & docker compose
Port 3000 ( Grafana) dan 31000 (Loki)
Loki sebagai log storage
Promtail sebagai log shipper
Grafana untuk vizualization
S3 sebagai tempati penyimpanan

Step by Step Implementation

Step 1: Setup Docker Compose

Buat docker-compose.yml dengan service Loki, Grafana, Promtail

version: '3.8'

services:
  loki:
    image: grafana/loki:2.9.0
    container_name: loki
    ports:
      - "3100:3100"
    volumes:
      - ./loki-config.yml:/etc/loki/local-config.yaml
      - loki-data:/loki
    command: -config.file=/etc/loki/local-config.yaml
    networks:
      - loki-network

  grafana:
    image: grafana/grafana:10.1.0
    container_name: grafana
    ports:
      - "3000:3000"
    environment:
      - GF_SECURITY_ADMIN_PASSWORD=admin123
    volumes:
      - grafana-data:/var/lib/grafana
      - ./grafana/provisioning:/etc/grafana/provisioning
    networks:
      - loki-network

  promtail:
    image: grafana/promtail:2.9.0
    container_name: promtail
    volumes:
      - ./promtail-config.yml:/etc/promtail/config.yml
      - /var/log:/var/log:ro
      - ./logs:/app/logs:ro
    command: -config.file=/etc/promtail/config.yml
    networks:
      - loki-network
    depends_on:
      - loki

  # Sample app untuk generate logs
  app01:
    image: igunawan-alpina:01
    container_name: sample-app
    ports:
      - "8080:80"
    volumes:
      - ./logs:/var/logs/docker/app01
    networks:
      - loki-network

volumes:
  loki-data:
  grafana-data:

networks:
  loki-network:
    driver: bridge

Step 2: konfigurasi Loki

Konfigurasi Loki , loki-local-config.yaml

auth_enabled: false

server:
  http_listen_port: 3100
  grpc_listen_port: 9096

common:
  path_prefix: /etc/loki
  replication_factor: 1
  ring:
    instance_addr: 0.0.0.0
    kvstore:
      store: inmemory

storage_config:
  aws:
    bucketnames:igunawan-loki-dev
    region: ap-southeast-3
    s3forcepathstyle: true

  boltdb_shipper:
    active_index_directory: /etc/loki/index
    cache_location: /etc/loki/index_cache
    cache_ttl: 24h
    shared_store: s3

schema_config:
  configs:
    - from: 2020-07-01
      store: boltdb-shipper
      object_store: s3
      schema: v11
      index:
        prefix: index_
        period: 24h

compactor:
  working_directory: /etc/loki/boltdb-shipper-compactor
  shared_store: s3

table_manager:
  retention_deletes_enabled: true
  retention_period: 120d

chunk_store_config:
  max_look_back_period: 120d

Step 3: Konfigurasi Promtail

Nama file contoh promtail-config.yaml
Define log sources dan labels

server:
  http_listen_port: 9080
  grpc_listen_port: 0

positions:
  filename: /etc/promtail/positions.yaml

clients:
  - url: http://localhost:3100/loki/api/v1/push

scrape_configs:
- job_name: app01
  static_configs:
  - targets:
      - localhost
    labels:
      job: app01
      __path__: /var/logs/docker/app01