Overview
Alerts
MoreAlerts (4)
State
Severity
Alert
Content
firing
warning
container=ingester
endpoint=http
instance=10.1.66.159:8080
job=kube-state-metrics
namespace=logging
pod=loki-ingester-0
reason=CrashLoopBackOff
service=kube-state-metrics
uid=e96520e6-fd04-4525-a095-748a2b97486d
firing
warning
container=kube-state-metrics
endpoint=http
instance=10.1.66.159:8080
job=kube-state-metrics
namespace=logging
pod=kube-state-metrics-5d87776b47-9czmn
service=kube-state-metrics
statefulset=loki-ingester
Logs
Logs
Date
Count (lines)
2026-04-12T00:00:00Z
4300252
2026-04-11T00:00:00Z
9998071
2026-04-10T00:00:00Z
10019727
2026-04-09T00:00:00Z
9924044
Logs (by level)
Type
Count (lines)
8794820
1069866
103954
8640
284
Error Logs (by pod)
Pod
Count (lines)
103684
Panic logs
2026-04-09T07:42:51Z
{log_app="webapi",log_type="std",pod_name="webapi-789d8bbdb5-vsdmv",pod_namespace="cloud-platform"}
@feilshu: true
_msg: context canceled; sql: transaction has already been committed or rolled back
_stream: {log_app="webapi",log_type="std",pod_name="webapi-789d8bbdb5-vsdmv",pod_namespace="cloud-platform"}
_time: 2026-04-09T07:42:51.805845468Z
app_name: webapi
cluster_name: global-silver
level: error
log_app: webapi
log_type: std
msg: panic-log
pod_name: webapi-789d8bbdb5-vsdmv
pod_namespace: cloud-platform
remoteAddr: 162.158.179.29
requestID: 67cc7f2a7c250e9a4e87e461ec8a3800
requestMethod: POST
requestURI: /api/v1/account/renew-token
stack: goroutine 9113 [running]:
runtime/debug.Stack()
/usr/local/go/src/runtime/debug/stack.go:26 +0x5e
databendcloud/webapi/server.(*App).Serve.PanicLog.func9.1()
/build/webapi/middleware/paniclog.go:59 +0x6fb
panic({0x2ad07a0?, 0xc0000781c0?})
/usr/local/go/src/runtime/panic.go:783 +0x132
github.com/getsentry/sentry-go/gin.(*handler).recoverWithSentry(0xc000f944d0, 0xc001ef0c90, 0xc001b8ba40)
/go/pkg/mod/github.com/getsentry/sentry-go/gin@v0.31.1/sentrygin.go:121 +0xfa
panic({0x2ad07a0?, 0xc0000781c0?})
/usr/local/go/src/runtime/panic.go:783 +0x132
databendcloud/webapi/dao.(*DAO).MustRotateAccountRefreshToken(0xc000f9c540, {0x37859d8, 0xc00206f300}, 0x1f, {0xc0013fc3af, 0x24}, {0xc0017d4cc0?, 0x2c82000?})
/build/webapi/dao/account_refresh_token_dao.go:68 +0x26e
databendcloud/webapi/logic/account_logic.(*AccountLogicImpl).refreshAccessToken(0xc000000000, {0x37859d8, 0xc00206f300}, 0x1f, 0x1, {0xc0013fc3af, 0x24})
/build/webapi/logic/account_logic/account_logic.go:639 +0x40f
databendcloud/webapi/logic/account_logic.(*AccountLogicImpl).RenewToken(0xc000000000, {0x37859d8, 0xc00206f300}, 0xc002501640)
/build/webapi/logic/account_logic/account_logic.go:967 +0x105
databendcloud/webapi/controllers/auth.(*AccountController).HandleRenewToken(0xc000f8f560, 0xc00206f300)
/build/webapi/controllers/auth/account_controller.go:570 +0xf8
github.com/gin-gonic/gin.(*Context).Next(...)
/go/pkg/mod/github.com/gin-gonic/gin@v1.10.0/context.go:185
databendcloud/webapi/controllers/auth.(*Authenticator).PingcapWriteGuard(0xc00206f398?, 0xc00206f300)
/build/webapi/controllers/auth/pingcap_guard.go:24 +0x258
github.com/gin-gonic/gin.(*Context).Next(...)
/go/pkg/mod/github.com/gin-gonic/gin@v1.10.0/context.go:185
github.com/getsentry/sentry-go/gin.(*handler).handle(0xc000f944d0, 0xc00206f300)
/go/pkg/mod/github.com/getsentry/sentry-go/gin@v0.31.1/sentrygin.go:106 +0x5de
github.com/gin-gonic/gin.(*Context).Next(0xc00206f300)
/go/pkg/mod/github.com/gin-gonic/gin@v1.10.0/context.go:185 +0x2b
databendcloud/webapi/server.(*App).Serve.PanicLog.func9(0xc001033088?)
/build/webapi/middleware/paniclog.go:72 +0x3f
github.com/gin-gonic/gin.(*Context).Next(0xc00206f300)
/go/pkg/mod/github.com/gin-gonic/gin@v1.10.0/context.go:185 +0x2b
databendcloud/webapi/server.(*App).Serve.WithApiMetrics.func8(0xc00206f300)
/build/webapi/middleware/api_metrics.go:18 +0x5d
github.com/gin-gonic/gin.(*Context).Next(0xc00206f300)
/go/pkg/mod/github.com/gin-gonic/gin@v1.10.0/context.go:185 +0x2b
databendcloud/webapi/server.(*App).Serve.AccessLog.func7(0xc00206f300)
/build/webapi/middleware/accesslog.go:76 +0x5aa
github.com/gin-gonic/gin.(*Context).Next(0xc00206f300)
/go/pkg/mod/github.com/gin-gonic/gin@v1.10.0/context.go:185 +0x2b
databendcloud/webapi/server.(*App).Serve.withAllowDatabendCrossOrigin.func6(0xc00206f300)
/build/webapi/server/util.go:53 +0x1b0
github.com/gin-gonic/gin.(*Context).Next(0xc00206f300)
/go/pkg/mod/github.com/gin-gonic/gin@v1.10.0/context.go:185 +0x2b
databendcloud/webapi/server.(*App).Serve.withTrackClientInfo.func5(0xc00206f300)
/build/webapi/server/util.go:87 +0x569
github.com/gin-gonic/gin.(*Context).Next(...)
/go/pkg/mod/github.com/gin-gonic/gin@v1.10.0/context.go:185
github.com/gin-contrib/requestid.New.func2(0xc00206f300)
/go/pkg/mod/github.com/gin-contrib/requestid@v1.0.5/requestid.go:48 +0x10c
github.com/gin-gonic/gin.(*Context).Next(...)
/go/pkg/mod/github.com/gin-gonic/gin@v1.10.0/context.go:185
github.com/gin-gonic/gin.CustomRecoveryWithWriter.func1(0xc00206f300)
/go/pkg/mod/github.com/gin-gonic/gin@v1.10.0/recovery.go:102 +0x6f
github.com/gin-gonic/gin.(*Context).Next(...)
/go/pkg/mod/github.com/gin-gonic/gin@v1.10.0/context.go:185
github.com/gin-gonic/gin.(*Engine).handleHTTPRequest(0xc001295380, 0xc00206f300)
/go/pkg/mod/github.com/gin-gonic/gin@v1.10.0/gin.go:633 +0x94b
github.com/gin-gonic/gin.(*Engine).ServeHTTP(0xc001295380, {0x3781490, 0xc001d39950}, 0xc001b8b680)
/go/pkg/mod/github.com/gin-gonic/gin@v1.10.0/gin.go:589 +0x1ad
net/http.serverHandler.ServeHTTP({0x376dd38?}, {0x3781490?, 0xc001d39950?}, 0x6?)
/usr/local/go/src/net/http/server.go:3340 +0x8e
net/http.(*conn).serve(0xc0014bcab0, {0x3785688, 0xc0006b3bc0})
/usr/local/go/src/net/http/server.go:2109 +0x665
created by net/http.(*Server).Serve in goroutine 32
/usr/local/go/src/net/http/server.go:3493 +0x485
stream: stdout
time: 2026-04-09T07:42:51Z
2026-04-09T07:42:51Z
{log_app="webapi",log_type="std",pod_name="webapi-789d8bbdb5-vsdmv",pod_namespace="cloud-platform"}
@feilshu: true
_msg: context canceled; sql: transaction has already been committed or rolled back
_stream: {log_app="webapi",log_type="std",pod_name="webapi-789d8bbdb5-vsdmv",pod_namespace="cloud-platform"}
_time: 2026-04-09T07:42:51.805845468Z
app_name: webapi
cluster_name: global-silver
level: error
log_app: webapi
log_type: std
msg: panic-log
pod_name: webapi-789d8bbdb5-vsdmv
pod_namespace: cloud-platform
remoteAddr: 162.158.179.29
requestID: 67cc7f2a7c250e9a4e87e461ec8a3800
requestMethod: POST
requestURI: /api/v1/account/renew-token
stack: goroutine 9113 [running]:
runtime/debug.Stack()
/usr/local/go/src/runtime/debug/stack.go:26 +0x5e
databendcloud/webapi/server.(*App).Serve.PanicLog.func9.1()
/build/webapi/middleware/paniclog.go:59 +0x6fb
panic({0x2ad07a0?, 0xc0000781c0?})
/usr/local/go/src/runtime/panic.go:783 +0x132
github.com/getsentry/sentry-go/gin.(*handler).recoverWithSentry(0xc000f944d0, 0xc001ef0c90, 0xc001b8ba40)
/go/pkg/mod/github.com/getsentry/sentry-go/gin@v0.31.1/sentrygin.go:121 +0xfa
panic({0x2ad07a0?, 0xc0000781c0?})
/usr/local/go/src/runtime/panic.go:783 +0x132
databendcloud/webapi/dao.(*DAO).MustRotateAccountRefreshToken(0xc000f9c540, {0x37859d8, 0xc00206f300}, 0x1f, {0xc0013fc3af, 0x24}, {0xc0017d4cc0?, 0x2c82000?})
/build/webapi/dao/account_refresh_token_dao.go:68 +0x26e
databendcloud/webapi/logic/account_logic.(*AccountLogicImpl).refreshAccessToken(0xc000000000, {0x37859d8, 0xc00206f300}, 0x1f, 0x1, {0xc0013fc3af, 0x24})
/build/webapi/logic/account_logic/account_logic.go:639 +0x40f
databendcloud/webapi/logic/account_logic.(*AccountLogicImpl).RenewToken(0xc000000000, {0x37859d8, 0xc00206f300}, 0xc002501640)
/build/webapi/logic/account_logic/account_logic.go:967 +0x105
databendcloud/webapi/controllers/auth.(*AccountController).HandleRenewToken(0xc000f8f560, 0xc00206f300)
/build/webapi/controllers/auth/account_controller.go:570 +0xf8
github.com/gin-gonic/gin.(*Context).Next(...)
/go/pkg/mod/github.com/gin-gonic/gin@v1.10.0/context.go:185
databendcloud/webapi/controllers/auth.(*Authenticator).PingcapWriteGuard(0xc00206f398?, 0xc00206f300)
/build/webapi/controllers/auth/pingcap_guard.go:24 +0x258
github.com/gin-gonic/gin.(*Context).Next(...)
/go/pkg/mod/github.com/gin-gonic/gin@v1.10.0/context.go:185
github.com/getsentry/sentry-go/gin.(*handler).handle(0xc000f944d0, 0xc00206f300)
/go/pkg/mod/github.com/getsentry/sentry-go/gin@v0.31.1/sentrygin.go:106 +0x5de
github.com/gin-gonic/gin.(*Context).Next(0xc00206f300)
/go/pkg/mod/github.com/gin-gonic/gin@v1.10.0/context.go:185 +0x2b
databendcloud/webapi/server.(*App).Serve.PanicLog.func9(0xc001033088?)
/build/webapi/middleware/paniclog.go:72 +0x3f
github.com/gin-gonic/gin.(*Context).Next(0xc00206f300)
/go/pkg/mod/github.com/gin-gonic/gin@v1.10.0/context.go:185 +0x2b
databendcloud/webapi/server.(*App).Serve.WithApiMetrics.func8(0xc00206f300)
/build/webapi/middleware/api_metrics.go:18 +0x5d
github.com/gin-gonic/gin.(*Context).Next(0xc00206f300)
/go/pkg/mod/github.com/gin-gonic/gin@v1.10.0/context.go:185 +0x2b
databendcloud/webapi/server.(*App).Serve.AccessLog.func7(0xc00206f300)
/build/webapi/middleware/accesslog.go:76 +0x5aa
github.com/gin-gonic/gin.(*Context).Next(0xc00206f300)
/go/pkg/mod/github.com/gin-gonic/gin@v1.10.0/context.go:185 +0x2b
databendcloud/webapi/server.(*App).Serve.withAllowDatabendCrossOrigin.func6(0xc00206f300)
/build/webapi/server/util.go:53 +0x1b0
github.com/gin-gonic/gin.(*Context).Next(0xc00206f300)
/go/pkg/mod/github.com/gin-gonic/gin@v1.10.0/context.go:185 +0x2b
databendcloud/webapi/server.(*App).Serve.withTrackClientInfo.func5(0xc00206f300)
/build/webapi/server/util.go:87 +0x569
github.com/gin-gonic/gin.(*Context).Next(...)
/go/pkg/mod/github.com/gin-gonic/gin@v1.10.0/context.go:185
github.com/gin-contrib/requestid.New.func2(0xc00206f300)
/go/pkg/mod/github.com/gin-contrib/requestid@v1.0.5/requestid.go:48 +0x10c
github.com/gin-gonic/gin.(*Context).Next(...)
/go/pkg/mod/github.com/gin-gonic/gin@v1.10.0/context.go:185
github.com/gin-gonic/gin.CustomRecoveryWithWriter.func1(0xc00206f300)
/go/pkg/mod/github.com/gin-gonic/gin@v1.10.0/recovery.go:102 +0x6f
github.com/gin-gonic/gin.(*Context).Next(...)
/go/pkg/mod/github.com/gin-gonic/gin@v1.10.0/context.go:185
github.com/gin-gonic/gin.(*Engine).handleHTTPRequest(0xc001295380, 0xc00206f300)
/go/pkg/mod/github.com/gin-gonic/gin@v1.10.0/gin.go:633 +0x94b
github.com/gin-gonic/gin.(*Engine).ServeHTTP(0xc001295380, {0x3781490, 0xc001d39950}, 0xc001b8b680)
/go/pkg/mod/github.com/gin-gonic/gin@v1.10.0/gin.go:589 +0x1ad
net/http.serverHandler.ServeHTTP({0x376dd38?}, {0x3781490?, 0xc001d39950?}, 0x6?)
/usr/local/go/src/net/http/server.go:3340 +0x8e
net/http.(*conn).serve(0xc0014bcab0, {0x3785688, 0xc0006b3bc0})
/usr/local/go/src/net/http/server.go:2109 +0x665
created by net/http.(*Server).Serve in goroutine 32
/usr/local/go/src/net/http/server.go:3493 +0x485
stream: stdout
time: 2026-04-09T07:42:51Z
2026-04-09T07:38:57Z
{log_app="webapi",log_type="std",pod_name="webapi-789d8bbdb5-vsdmv",pod_namespace="cloud-platform"}
_msg: context canceled (client disconnected or timed out)
_stream: {log_app="webapi",log_type="std",pod_name="webapi-789d8bbdb5-vsdmv",pod_namespace="cloud-platform"}
_time: 2026-04-09T07:38:57.147594734Z
app_name: webapi
cluster_name: global-silver
level: error
log_app: webapi
log_type: std
msg: panic-log
pod_name: webapi-789d8bbdb5-vsdmv
pod_namespace: cloud-platform
remoteAddr: 10.1.158.166
requestID: 1d4269ec14a64f6536cee9789872897e
requestMethod: GET
requestURI: /api/v1/worksheets/2955
stream: stdout
time: 2026-04-09T07:38:57Z
2026-04-09T07:38:57Z
{log_app="webapi",log_type="std",pod_name="webapi-789d8bbdb5-vsdmv",pod_namespace="cloud-platform"}
_msg: context canceled (client disconnected or timed out)
_stream: {log_app="webapi",log_type="std",pod_name="webapi-789d8bbdb5-vsdmv",pod_namespace="cloud-platform"}
_time: 2026-04-09T07:38:57.147594734Z
app_name: webapi
cluster_name: global-silver
level: error
log_app: webapi
log_type: std
msg: panic-log
pod_name: webapi-789d8bbdb5-vsdmv
pod_namespace: cloud-platform
remoteAddr: 10.1.158.166
requestID: 1d4269ec14a64f6536cee9789872897e
requestMethod: GET
requestURI: /api/v1/worksheets/2955
stream: stdout
time: 2026-04-09T07:38:57Z
2026-04-09T07:27:19Z
{log_app="webapi",log_type="std",pod_name="webapi-789d8bbdb5-vsdmv",pod_namespace="cloud-platform"}
_msg: context canceled (client disconnected or timed out)
_stream: {log_app="webapi",log_type="std",pod_name="webapi-789d8bbdb5-vsdmv",pod_namespace="cloud-platform"}
_time: 2026-04-09T07:27:19.901861448Z
accountEmail: zhangsenquan@datafuselabs.com
accountID: 2
app_name: webapi
cluster_name: global-silver
level: error
log_app: webapi
log_type: std
msg: panic-log
pod_name: webapi-789d8bbdb5-vsdmv
pod_namespace: cloud-platform
remoteAddr: 10.1.160.61
requestID: d91e6542a31e415d7034fa135a95e491
requestMethod: GET
requestURI: /api/v1/my/orgs
stream: stdout
time: 2026-04-09T07:27:19Z
2026-04-09T07:27:19Z
{log_app="webapi",log_type="std",pod_name="webapi-789d8bbdb5-vsdmv",pod_namespace="cloud-platform"}
_msg: context canceled (client disconnected or timed out)
_stream: {log_app="webapi",log_type="std",pod_name="webapi-789d8bbdb5-vsdmv",pod_namespace="cloud-platform"}
_time: 2026-04-09T07:27:19.901861448Z
accountEmail: zhangsenquan@datafuselabs.com
accountID: 2
app_name: webapi
cluster_name: global-silver
level: error
log_app: webapi
log_type: std
msg: panic-log
pod_name: webapi-789d8bbdb5-vsdmv
pod_namespace: cloud-platform
remoteAddr: 10.1.160.61
requestID: d91e6542a31e415d7034fa135a95e491
requestMethod: GET
requestURI: /api/v1/my/orgs
stream: stdout
time: 2026-04-09T07:27:19Z
Service
S3
Databend-gateway
Name
Total
Status
Failed
ProxyQuery
12
🟢
0
HTTP request
Top visited request
URL
Count
3762