Overview


Alerts

More

Alerts (4)

State
Severity
Alert
Content
firing
none
firing
warning
firing
warning
container=ingester endpoint=http instance=10.1.66.159:8080 job=kube-state-metrics namespace=logging pod=loki-ingester-0 reason=CrashLoopBackOff service=kube-state-metrics uid=e96520e6-fd04-4525-a095-748a2b97486d
firing
warning
container=kube-state-metrics endpoint=http instance=10.1.66.159:8080 job=kube-state-metrics namespace=logging pod=kube-state-metrics-5d87776b47-9czmn service=kube-state-metrics statefulset=loki-ingester

Logs


Logs

Date
Count (lines)
2026-04-12T00:00:00Z
4300252
2026-04-11T00:00:00Z
9998071
2026-04-10T00:00:00Z
10019727
2026-04-09T00:00:00Z
9924044

Logs (by level)

Type
Count (lines)
8794820
1069866
103954
8640

Error Logs (by pod)

Panic logs

2026-04-09T07:42:51Z {log_app="webapi",log_type="std",pod_name="webapi-789d8bbdb5-vsdmv",pod_namespace="cloud-platform"} @feilshu: true _msg: context canceled; sql: transaction has already been committed or rolled back _stream: {log_app="webapi",log_type="std",pod_name="webapi-789d8bbdb5-vsdmv",pod_namespace="cloud-platform"} _time: 2026-04-09T07:42:51.805845468Z app_name: webapi cluster_name: global-silver level: error log_app: webapi log_type: std msg: panic-log pod_name: webapi-789d8bbdb5-vsdmv pod_namespace: cloud-platform remoteAddr: 162.158.179.29 requestID: 67cc7f2a7c250e9a4e87e461ec8a3800 requestMethod: POST requestURI: /api/v1/account/renew-token stack: goroutine 9113 [running]: runtime/debug.Stack() /usr/local/go/src/runtime/debug/stack.go:26 +0x5e databendcloud/webapi/server.(*App).Serve.PanicLog.func9.1() /build/webapi/middleware/paniclog.go:59 +0x6fb panic({0x2ad07a0?, 0xc0000781c0?}) /usr/local/go/src/runtime/panic.go:783 +0x132 github.com/getsentry/sentry-go/gin.(*handler).recoverWithSentry(0xc000f944d0, 0xc001ef0c90, 0xc001b8ba40) /go/pkg/mod/github.com/getsentry/sentry-go/gin@v0.31.1/sentrygin.go:121 +0xfa panic({0x2ad07a0?, 0xc0000781c0?}) /usr/local/go/src/runtime/panic.go:783 +0x132 databendcloud/webapi/dao.(*DAO).MustRotateAccountRefreshToken(0xc000f9c540, {0x37859d8, 0xc00206f300}, 0x1f, {0xc0013fc3af, 0x24}, {0xc0017d4cc0?, 0x2c82000?}) /build/webapi/dao/account_refresh_token_dao.go:68 +0x26e databendcloud/webapi/logic/account_logic.(*AccountLogicImpl).refreshAccessToken(0xc000000000, {0x37859d8, 0xc00206f300}, 0x1f, 0x1, {0xc0013fc3af, 0x24}) /build/webapi/logic/account_logic/account_logic.go:639 +0x40f databendcloud/webapi/logic/account_logic.(*AccountLogicImpl).RenewToken(0xc000000000, {0x37859d8, 0xc00206f300}, 0xc002501640) /build/webapi/logic/account_logic/account_logic.go:967 +0x105 databendcloud/webapi/controllers/auth.(*AccountController).HandleRenewToken(0xc000f8f560, 0xc00206f300) /build/webapi/controllers/auth/account_controller.go:570 +0xf8 github.com/gin-gonic/gin.(*Context).Next(...) /go/pkg/mod/github.com/gin-gonic/gin@v1.10.0/context.go:185 databendcloud/webapi/controllers/auth.(*Authenticator).PingcapWriteGuard(0xc00206f398?, 0xc00206f300) /build/webapi/controllers/auth/pingcap_guard.go:24 +0x258 github.com/gin-gonic/gin.(*Context).Next(...) /go/pkg/mod/github.com/gin-gonic/gin@v1.10.0/context.go:185 github.com/getsentry/sentry-go/gin.(*handler).handle(0xc000f944d0, 0xc00206f300) /go/pkg/mod/github.com/getsentry/sentry-go/gin@v0.31.1/sentrygin.go:106 +0x5de github.com/gin-gonic/gin.(*Context).Next(0xc00206f300) /go/pkg/mod/github.com/gin-gonic/gin@v1.10.0/context.go:185 +0x2b databendcloud/webapi/server.(*App).Serve.PanicLog.func9(0xc001033088?) /build/webapi/middleware/paniclog.go:72 +0x3f github.com/gin-gonic/gin.(*Context).Next(0xc00206f300) /go/pkg/mod/github.com/gin-gonic/gin@v1.10.0/context.go:185 +0x2b databendcloud/webapi/server.(*App).Serve.WithApiMetrics.func8(0xc00206f300) /build/webapi/middleware/api_metrics.go:18 +0x5d github.com/gin-gonic/gin.(*Context).Next(0xc00206f300) /go/pkg/mod/github.com/gin-gonic/gin@v1.10.0/context.go:185 +0x2b databendcloud/webapi/server.(*App).Serve.AccessLog.func7(0xc00206f300) /build/webapi/middleware/accesslog.go:76 +0x5aa github.com/gin-gonic/gin.(*Context).Next(0xc00206f300) /go/pkg/mod/github.com/gin-gonic/gin@v1.10.0/context.go:185 +0x2b databendcloud/webapi/server.(*App).Serve.withAllowDatabendCrossOrigin.func6(0xc00206f300) /build/webapi/server/util.go:53 +0x1b0 github.com/gin-gonic/gin.(*Context).Next(0xc00206f300) /go/pkg/mod/github.com/gin-gonic/gin@v1.10.0/context.go:185 +0x2b databendcloud/webapi/server.(*App).Serve.withTrackClientInfo.func5(0xc00206f300) /build/webapi/server/util.go:87 +0x569 github.com/gin-gonic/gin.(*Context).Next(...) /go/pkg/mod/github.com/gin-gonic/gin@v1.10.0/context.go:185 github.com/gin-contrib/requestid.New.func2(0xc00206f300) /go/pkg/mod/github.com/gin-contrib/requestid@v1.0.5/requestid.go:48 +0x10c github.com/gin-gonic/gin.(*Context).Next(...) /go/pkg/mod/github.com/gin-gonic/gin@v1.10.0/context.go:185 github.com/gin-gonic/gin.CustomRecoveryWithWriter.func1(0xc00206f300) /go/pkg/mod/github.com/gin-gonic/gin@v1.10.0/recovery.go:102 +0x6f github.com/gin-gonic/gin.(*Context).Next(...) /go/pkg/mod/github.com/gin-gonic/gin@v1.10.0/context.go:185 github.com/gin-gonic/gin.(*Engine).handleHTTPRequest(0xc001295380, 0xc00206f300) /go/pkg/mod/github.com/gin-gonic/gin@v1.10.0/gin.go:633 +0x94b github.com/gin-gonic/gin.(*Engine).ServeHTTP(0xc001295380, {0x3781490, 0xc001d39950}, 0xc001b8b680) /go/pkg/mod/github.com/gin-gonic/gin@v1.10.0/gin.go:589 +0x1ad net/http.serverHandler.ServeHTTP({0x376dd38?}, {0x3781490?, 0xc001d39950?}, 0x6?) /usr/local/go/src/net/http/server.go:3340 +0x8e net/http.(*conn).serve(0xc0014bcab0, {0x3785688, 0xc0006b3bc0}) /usr/local/go/src/net/http/server.go:2109 +0x665 created by net/http.(*Server).Serve in goroutine 32 /usr/local/go/src/net/http/server.go:3493 +0x485 stream: stdout time: 2026-04-09T07:42:51Z
2026-04-09T07:42:51Z {log_app="webapi",log_type="std",pod_name="webapi-789d8bbdb5-vsdmv",pod_namespace="cloud-platform"} @feilshu: true _msg: context canceled; sql: transaction has already been committed or rolled back _stream: {log_app="webapi",log_type="std",pod_name="webapi-789d8bbdb5-vsdmv",pod_namespace="cloud-platform"} _time: 2026-04-09T07:42:51.805845468Z app_name: webapi cluster_name: global-silver level: error log_app: webapi log_type: std msg: panic-log pod_name: webapi-789d8bbdb5-vsdmv pod_namespace: cloud-platform remoteAddr: 162.158.179.29 requestID: 67cc7f2a7c250e9a4e87e461ec8a3800 requestMethod: POST requestURI: /api/v1/account/renew-token stack: goroutine 9113 [running]: runtime/debug.Stack() /usr/local/go/src/runtime/debug/stack.go:26 +0x5e databendcloud/webapi/server.(*App).Serve.PanicLog.func9.1() /build/webapi/middleware/paniclog.go:59 +0x6fb panic({0x2ad07a0?, 0xc0000781c0?}) /usr/local/go/src/runtime/panic.go:783 +0x132 github.com/getsentry/sentry-go/gin.(*handler).recoverWithSentry(0xc000f944d0, 0xc001ef0c90, 0xc001b8ba40) /go/pkg/mod/github.com/getsentry/sentry-go/gin@v0.31.1/sentrygin.go:121 +0xfa panic({0x2ad07a0?, 0xc0000781c0?}) /usr/local/go/src/runtime/panic.go:783 +0x132 databendcloud/webapi/dao.(*DAO).MustRotateAccountRefreshToken(0xc000f9c540, {0x37859d8, 0xc00206f300}, 0x1f, {0xc0013fc3af, 0x24}, {0xc0017d4cc0?, 0x2c82000?}) /build/webapi/dao/account_refresh_token_dao.go:68 +0x26e databendcloud/webapi/logic/account_logic.(*AccountLogicImpl).refreshAccessToken(0xc000000000, {0x37859d8, 0xc00206f300}, 0x1f, 0x1, {0xc0013fc3af, 0x24}) /build/webapi/logic/account_logic/account_logic.go:639 +0x40f databendcloud/webapi/logic/account_logic.(*AccountLogicImpl).RenewToken(0xc000000000, {0x37859d8, 0xc00206f300}, 0xc002501640) /build/webapi/logic/account_logic/account_logic.go:967 +0x105 databendcloud/webapi/controllers/auth.(*AccountController).HandleRenewToken(0xc000f8f560, 0xc00206f300) /build/webapi/controllers/auth/account_controller.go:570 +0xf8 github.com/gin-gonic/gin.(*Context).Next(...) /go/pkg/mod/github.com/gin-gonic/gin@v1.10.0/context.go:185 databendcloud/webapi/controllers/auth.(*Authenticator).PingcapWriteGuard(0xc00206f398?, 0xc00206f300) /build/webapi/controllers/auth/pingcap_guard.go:24 +0x258 github.com/gin-gonic/gin.(*Context).Next(...) /go/pkg/mod/github.com/gin-gonic/gin@v1.10.0/context.go:185 github.com/getsentry/sentry-go/gin.(*handler).handle(0xc000f944d0, 0xc00206f300) /go/pkg/mod/github.com/getsentry/sentry-go/gin@v0.31.1/sentrygin.go:106 +0x5de github.com/gin-gonic/gin.(*Context).Next(0xc00206f300) /go/pkg/mod/github.com/gin-gonic/gin@v1.10.0/context.go:185 +0x2b databendcloud/webapi/server.(*App).Serve.PanicLog.func9(0xc001033088?) /build/webapi/middleware/paniclog.go:72 +0x3f github.com/gin-gonic/gin.(*Context).Next(0xc00206f300) /go/pkg/mod/github.com/gin-gonic/gin@v1.10.0/context.go:185 +0x2b databendcloud/webapi/server.(*App).Serve.WithApiMetrics.func8(0xc00206f300) /build/webapi/middleware/api_metrics.go:18 +0x5d github.com/gin-gonic/gin.(*Context).Next(0xc00206f300) /go/pkg/mod/github.com/gin-gonic/gin@v1.10.0/context.go:185 +0x2b databendcloud/webapi/server.(*App).Serve.AccessLog.func7(0xc00206f300) /build/webapi/middleware/accesslog.go:76 +0x5aa github.com/gin-gonic/gin.(*Context).Next(0xc00206f300) /go/pkg/mod/github.com/gin-gonic/gin@v1.10.0/context.go:185 +0x2b databendcloud/webapi/server.(*App).Serve.withAllowDatabendCrossOrigin.func6(0xc00206f300) /build/webapi/server/util.go:53 +0x1b0 github.com/gin-gonic/gin.(*Context).Next(0xc00206f300) /go/pkg/mod/github.com/gin-gonic/gin@v1.10.0/context.go:185 +0x2b databendcloud/webapi/server.(*App).Serve.withTrackClientInfo.func5(0xc00206f300) /build/webapi/server/util.go:87 +0x569 github.com/gin-gonic/gin.(*Context).Next(...) /go/pkg/mod/github.com/gin-gonic/gin@v1.10.0/context.go:185 github.com/gin-contrib/requestid.New.func2(0xc00206f300) /go/pkg/mod/github.com/gin-contrib/requestid@v1.0.5/requestid.go:48 +0x10c github.com/gin-gonic/gin.(*Context).Next(...) /go/pkg/mod/github.com/gin-gonic/gin@v1.10.0/context.go:185 github.com/gin-gonic/gin.CustomRecoveryWithWriter.func1(0xc00206f300) /go/pkg/mod/github.com/gin-gonic/gin@v1.10.0/recovery.go:102 +0x6f github.com/gin-gonic/gin.(*Context).Next(...) /go/pkg/mod/github.com/gin-gonic/gin@v1.10.0/context.go:185 github.com/gin-gonic/gin.(*Engine).handleHTTPRequest(0xc001295380, 0xc00206f300) /go/pkg/mod/github.com/gin-gonic/gin@v1.10.0/gin.go:633 +0x94b github.com/gin-gonic/gin.(*Engine).ServeHTTP(0xc001295380, {0x3781490, 0xc001d39950}, 0xc001b8b680) /go/pkg/mod/github.com/gin-gonic/gin@v1.10.0/gin.go:589 +0x1ad net/http.serverHandler.ServeHTTP({0x376dd38?}, {0x3781490?, 0xc001d39950?}, 0x6?) /usr/local/go/src/net/http/server.go:3340 +0x8e net/http.(*conn).serve(0xc0014bcab0, {0x3785688, 0xc0006b3bc0}) /usr/local/go/src/net/http/server.go:2109 +0x665 created by net/http.(*Server).Serve in goroutine 32 /usr/local/go/src/net/http/server.go:3493 +0x485 stream: stdout time: 2026-04-09T07:42:51Z
2026-04-09T07:38:57Z {log_app="webapi",log_type="std",pod_name="webapi-789d8bbdb5-vsdmv",pod_namespace="cloud-platform"} _msg: context canceled (client disconnected or timed out) _stream: {log_app="webapi",log_type="std",pod_name="webapi-789d8bbdb5-vsdmv",pod_namespace="cloud-platform"} _time: 2026-04-09T07:38:57.147594734Z app_name: webapi cluster_name: global-silver level: error log_app: webapi log_type: std msg: panic-log pod_name: webapi-789d8bbdb5-vsdmv pod_namespace: cloud-platform remoteAddr: 10.1.158.166 requestID: 1d4269ec14a64f6536cee9789872897e requestMethod: GET requestURI: /api/v1/worksheets/2955 stream: stdout time: 2026-04-09T07:38:57Z
2026-04-09T07:38:57Z {log_app="webapi",log_type="std",pod_name="webapi-789d8bbdb5-vsdmv",pod_namespace="cloud-platform"} _msg: context canceled (client disconnected or timed out) _stream: {log_app="webapi",log_type="std",pod_name="webapi-789d8bbdb5-vsdmv",pod_namespace="cloud-platform"} _time: 2026-04-09T07:38:57.147594734Z app_name: webapi cluster_name: global-silver level: error log_app: webapi log_type: std msg: panic-log pod_name: webapi-789d8bbdb5-vsdmv pod_namespace: cloud-platform remoteAddr: 10.1.158.166 requestID: 1d4269ec14a64f6536cee9789872897e requestMethod: GET requestURI: /api/v1/worksheets/2955 stream: stdout time: 2026-04-09T07:38:57Z
2026-04-09T07:27:19Z {log_app="webapi",log_type="std",pod_name="webapi-789d8bbdb5-vsdmv",pod_namespace="cloud-platform"} _msg: context canceled (client disconnected or timed out) _stream: {log_app="webapi",log_type="std",pod_name="webapi-789d8bbdb5-vsdmv",pod_namespace="cloud-platform"} _time: 2026-04-09T07:27:19.901861448Z accountEmail: zhangsenquan@datafuselabs.com accountID: 2 app_name: webapi cluster_name: global-silver level: error log_app: webapi log_type: std msg: panic-log pod_name: webapi-789d8bbdb5-vsdmv pod_namespace: cloud-platform remoteAddr: 10.1.160.61 requestID: d91e6542a31e415d7034fa135a95e491 requestMethod: GET requestURI: /api/v1/my/orgs stream: stdout time: 2026-04-09T07:27:19Z
2026-04-09T07:27:19Z {log_app="webapi",log_type="std",pod_name="webapi-789d8bbdb5-vsdmv",pod_namespace="cloud-platform"} _msg: context canceled (client disconnected or timed out) _stream: {log_app="webapi",log_type="std",pod_name="webapi-789d8bbdb5-vsdmv",pod_namespace="cloud-platform"} _time: 2026-04-09T07:27:19.901861448Z accountEmail: zhangsenquan@datafuselabs.com accountID: 2 app_name: webapi cluster_name: global-silver level: error log_app: webapi log_type: std msg: panic-log pod_name: webapi-789d8bbdb5-vsdmv pod_namespace: cloud-platform remoteAddr: 10.1.160.61 requestID: d91e6542a31e415d7034fa135a95e491 requestMethod: GET requestURI: /api/v1/my/orgs stream: stdout time: 2026-04-09T07:27:19Z

Service


S3

Name
Total
Status
Failed
GetObject
2
🟢
0
ExistObject
4
🔴
PutObject
4
🟢
0
DeleteObject
0
🟢
0

Databend-gateway

Name
Total
Status
Failed
ProxyQuery
12
🟢
0

HTTP request


Requests (3762)

Code
Status
Count
HTTP_200
🟢
HTTP_404
🔴
HTTP_401
🔴

Top visited request

URL
Count
3762

Slow request (by latency>=1s)

More
Time
Latency
URL
Account
2026-04-12T08:24:16Z
37.55s

Account & Ticket


Account

More
Total: 166 users

Org

More
Total: 176 orgs

Tickets

More
Total: 173 tickets