A Web Socket Log Tailing Tool

原创
2020/12/28 00:00
阅读数 215

A web socket log tailing tool

Dec 28, 2020 2 min read

A web socket log tailing tool

logtail

logtail is a web socket log tailing tool.

1. Features

  • tailing command output
  • support multiple command tailing
  • support webhook notice (include dingtalk)
  • support websocket tailing
  • support log matching filter

2. Architecture

logtail

3. Usage

3.1. install logtail

go get -u github.com/vogo/logtail/cmd/[email protected]

3.2. start logtail server

usage: logtail -port=<port> -cmd="<cmd>"

examples:

# tailing file logs
logtail -port=54321 -cmd="tailf /home/my/logs/myapp.log"

# tailing kubectl logs
logtail -port=54322 -cmd="kubectl logs --tail 10 -f \$(kubectl get pods --selector=app=myapp -o jsonpath='{.items[*].metadata.name}')"

# using a config file
logtail -file=/Users/wongoo/logtail-config.json

config file sample:

{
"port": 54321,
"default_format":{
    "prefix": "!!!!-!!-!!"
},
"global_routers": [
    {
      "matchers": [],
      "transfers": [
        {"type": "console"}
      ]
    }
],
"default_routers": [
    {
      "matchers": [],
      "transfers": [
        {"type": "console"}
      ]
    }
],
"servers": [
    {
      "id": "app1",
      "command": "tail -f /Users/wongoo/app/app1.log",
      "routers": [
        {
          "matchers": [
            {"match_contains": "ERROR"}
          ],
          "transfers": [
            {
              "type": "ding",
              "ding_url": "https://oapi.dingtalk.com/robot/send?access_token=<token>"
            },
            {
              "type": "webhook",
              "webhook_url": "http://127.0.0.1:9000"
            }
          ]
        }
      ]
    },
    {
      "id": "app2",
      "command": "tail -f /Users/wongoo/app/app2.log"
    }
]
}

3.3. tailing logs

browse http://<server-ip>:<port> to list all tailing logs.

4. log format

You can config log format globally, or config it for a server.

The config prefix of the format is the wildcard of the prefix of a new log record,

logtail will check whether a new line is the start of a new log record, or one of the following lines.

The wildcard does NOT support ‘*’ for none or many chars, it supports the following tag:

  • ‘?’ as one byte char;
  • ‘~’ as one alphabet char;
  • ‘!’ as one number char;
  • other chars must exact match.

example:

{
    "default_format":{
      "prefix": "!!!!-!!-!!"  # matches 2020-12-12
    },
    "servers": [
        {
          "id": "app1",
          "command": "tail -f /Users/wongoo/app/app1.log",
          "format":{
            "prefix": "!!!!-!!-!!"
          }
        }
    ]
}

5. command examples

# tail log file
tail -f /usr/local/myapp/myapp.log

# k8s: find and tail logs for the single pod of myapp
kubectl logs --tail 10 -f $(kubectl get pods --selector=app=myapp -o jsonpath='{.items[*].metadata.name}')

# k8s: find and tail logs for the myapp deployment (multiple pods)
kubectl logs --tail 10 -f deployment/$(kubectl get deployments --selector=project-name=myapp -o jsonpath='{.items[*].metadata.name}')

# k8s: find and tail logs for the latest version of the myapp deployment
s=$(kubectl get deployments --selector=project-name=myapp -o jsonpath='{.items[*].metadata.name}');s=${s##* };kubectl logs --tail 10 -f deployment/$s

GitHub

Logging

Go development plugin for VimPrevious Post

Go development plugin for Vim

Comments

You might also like…

Realtime log viewer for docker containers

Logging

Realtime log viewer for docker containers

Dozzle is a simple, lightweight application that provides you with a web based interface to monitor your Docker container logs live.

17 December 2020

A concise, functional, flexible and extensible logger for go

Logging

A concise, functional, flexible and extensible logger for go

Gxlog is short for Go eXtensible LOGger. It is concise, functional, flexible and extensible.

17 August 2019

A distributed and coördination-free log management system

Logging

A distributed and coördination-free log management system

OK Log is a distributed and coördination-free log management system for big ol' clusters.

10 May 2019

Subscribe to Golang Example

Get the latest posts delivered right to your inbox

Subscribe

Tags

Command Line69

Tools55

Networking38

Framework28

Miscellaneous23

Images20

HTTP15

Database tool15

Authentication13

Proxy12

Text Processing12

Games11

Data Structures11

Security10

Distributed Systems9

Server8

Files8

Logging7

Code Analysis7

JSON6

Machine Learning6

Job Scheduler6

GUI5

Forms5

Charts5

Date5

Web Scraping4

Time4

Video4

PDF3

Recent Posts

A web socket log tailing tool

Dec 28, 2020

Go development plugin for Vim

Dec 22, 2020

Go tool to modify struct field tags

Dec 22, 2020

Command-line DNS Client for Humans Written in Golang

Dec 21, 2020

A cross-platform strong encryption and not detectable proxy

Dec 21, 2020

Powered by Golang Example

本文同步分享在 博客“大地缸”(other)。
如有侵权,请联系 support@oschina.cn 删除。
本文参与“OSC源创计划”,欢迎正在阅读的你也加入,一起分享。

展开阅读全文
打赏
0
0 收藏
分享
加载中
更多评论
打赏
0 评论
0 收藏
0
分享
返回顶部
顶部