books

《后端架构设计·演进与治理》6.1 日志、指标与追踪的统一语义约定

从 arch-lab 订单系统一次错误率告警查不到数据的事故切入，说明日志、指标与追踪为什么需要同一套属性 schema，给出按名字、类型、基数与敏感度校验语义约定的 Go 实现，并用 4 条事件和 8 类违规跑出本机可复现的拒绝率与稳定排序。

Leeting Yan 7 分钟阅读 3319 字

一次查不到数的错误率告警

2026 年 10 月大促前，arch-lab 订单服务的错误率面板突然空了。值班同学先怀疑采集链路，查了十分钟才发现：过去一周有两次发布，第一次把 span 上的 order_id 改成了 order.id，第二次把 http.status_code 从整数改成了字符串 "500"。面板的查询同时按旧键和新键过滤，两个条件都不匹配，于是返回 0 条；告警规则算不出分母，安静地没有触发。

更隐蔽的问题是有人在指标上加了 order.id 标签。订单号是每单唯一的值，指标的时间序列数一夜之间从几百涨到十几万，采集端开始丢点，而应用日志看起来一切正常。三份遥测各自都能“打出来”，却无法回答同一个问题：订单可观测性契约 里约定的字段，在导出时已经各自变形成了另一套。

语义约定是 schema，不是命名建议

很多团队把语义约定当成“命名规范”，写在 wiki 上，靠评审提醒。真正决定它能否约束系统的，是它有没有像数据库 schema 一样被机器执行：字段名、类型、是否必填、是否允许出现在某类信号里，都要有唯一答案。只要一个字段在日志里叫 order.id、在指标里叫 order_id，跨信号关联就退化成人工猜测。

约束层规则违规示例直接后果
名字小写、点分、每段语义稳定Order.ID、order_id查询与看板匹配不到
类型同一属性在所有信号中同型状态码写成 "500"聚合、比较、告警表达式失败
基数指标禁止订单级、用户级键指标带 order.id时间序列爆炸、采集丢点
敏感度邮箱、卡号导出前必须脱敏span 带 user.email合规风险与日志泄露

这四层里，名字最容易被发现，基数和敏感度最容易被忽略。名字错了通常当天就有人报“看板没数”；基数错了要等到流量上来才暴露；敏感度错了往往在审计时才暴露。校验器必须把四层都变成结构化 finding，而不是只打印一句 warning。

校验要区分信号，而不是一刀切

日志与追踪天然允许高基数：一条日志带订单号是合理的，span 带 order.id 才能从慢请求追到具体订单。指标不行，指标标签会变成时间序列维度，订单号这种无界值必须拒绝。因此同一个 order.id 在 span 上是合法属性，在 metric 上就是违规，这个差异只能由“信号类型 + 属性规格”共同决定。

必填项也随信号变化。span 需要 http.route、http.response.status_code 和 order.status 才能还原一次支付调用；日志需要 log.severity 和 log.message；指标只需要服务名、版本和环境。把这些规则写进注册表后，校验器就不需要在每个埋点处复制判断，新增字段也只改一处。

可运行实现

下面的包只依赖标准库。NewOrderValidator 固定 arch-lab 的注册表，Validate 先按属性名排序，再输出名字、类型、基数、敏感度和枚举五类问题，最后补齐必填项。排序是为了让同一批事件在任何机器上产生完全相同的报告，便于做门禁和 diff。

// file: internal/platform/obs/conventions.go
// tier: L1
// verify: go test ./internal/platform/obs/ -count=1
// Package obs holds chapter 6's observability building blocks: semantic
// convention checks, trace propagation and sampling, and incident correlation.
package obs

import (
	"fmt"
	"regexp"
	"slices"
	"sort"
)

// AttrKind is the wire type of one telemetry attribute.
type AttrKind int

const (
	StringKind AttrKind = iota
	IntKind
)

func (k AttrKind) String() string {
	if k == IntKind {
		return "int"
	}
	return "string"
}

// Value keeps the kind explicit so "500" cannot pass as an int status code.
type Value struct {
	Kind AttrKind
	Str  string
	Int  int64
}

func StringValue(v string) Value { return Value{Kind: StringKind, Str: v} }
func IntValue(v int64) Value     { return Value{Kind: IntKind, Int: v} }

// SignalKind decides cardinality rules: a metric cannot carry per-order keys.
type SignalKind string

const (
	SignalMetric SignalKind = "metric"
	SignalLog    SignalKind = "log"
	SignalSpan   SignalKind = "span"
)

// AttrSpec is one registry entry; RequiredFor differs per signal.
type AttrSpec struct {
	Name            string
	Kind            AttrKind
	RequiredFor     []SignalKind
	HighCardinality bool
	Sensitive       bool
	Enum            []string
}

// Finding is one violation with a stable code that can be counted and alerted.
type Finding struct {
	Attribute string
	Code      string
	Detail    string
}

type Report struct {
	Signal   SignalKind
	Checked  int
	Findings []Finding
}

func (r Report) OK() bool { return len(r.Findings) == 0 }

func (r Report) Codes() map[string]int {
	out := make(map[string]int, len(r.Findings))
	for _, f := range r.Findings {
		out[f.Code]++
	}
	return out
}

const maxAttrNameLen = 64

var attrNameRE = regexp.MustCompile(`^[a-z][a-z0-9_]*(\.[a-z][a-z0-9_]*)*$`)

// Validator is immutable after construction and safe to share.
type Validator struct {
	specs    map[string]AttrSpec
	required []string
}

// NewOrderValidator returns the arch-lab order system's conventions.
func NewOrderValidator() *Validator {
	specs := []AttrSpec{
		{Name: "service.name", Kind: StringKind, RequiredFor: []SignalKind{SignalMetric, SignalLog, SignalSpan}},
		{Name: "service.version", Kind: StringKind, RequiredFor: []SignalKind{SignalMetric, SignalLog, SignalSpan}},
		{Name: "deployment.environment", Kind: StringKind, RequiredFor: []SignalKind{SignalMetric, SignalLog, SignalSpan}, Enum: []string{"dev", "staging", "prod"}},
		{Name: "order.id", Kind: StringKind, RequiredFor: []SignalKind{SignalLog, SignalSpan}, HighCardinality: true},
		{Name: "order.status", Kind: StringKind, RequiredFor: []SignalKind{SignalSpan}, Enum: []string{"CREATED", "INVENTORY_RESERVED", "PAID", "FULFILLED", "CANCELLED", "REFUNDED"}},
		{Name: "http.request.method", Kind: StringKind, RequiredFor: []SignalKind{SignalSpan}, Enum: []string{"GET", "POST", "PUT", "PATCH", "DELETE"}},
		{Name: "http.route", Kind: StringKind, RequiredFor: []SignalKind{SignalSpan}},
		{Name: "http.response.status_code", Kind: IntKind, RequiredFor: []SignalKind{SignalSpan}},
		{Name: "log.severity", Kind: StringKind, RequiredFor: []SignalKind{SignalLog}, Enum: []string{"INFO", "WARN", "ERROR"}},
		{Name: "log.message", Kind: StringKind, RequiredFor: []SignalKind{SignalLog}},
		{Name: "payment.provider", Kind: StringKind, RequiredFor: []SignalKind{SignalSpan}},
		{Name: "user.email", Kind: StringKind, HighCardinality: true, Sensitive: true},
		{Name: "payment.card_number", Kind: StringKind, HighCardinality: true, Sensitive: true},
	}
	v := &Validator{specs: make(map[string]AttrSpec, len(specs))}
	for _, s := range specs {
		v.specs[s.Name] = s
		v.required = append(v.required, s.Name)
	}
	sort.Strings(v.required)
	return v
}

// Validate returns findings in deterministic order: attributes sorted by name,
// then missing-required entries sorted by name.
func (v *Validator) Validate(sig SignalKind, attrs map[string]Value) Report {
	rep := Report{Signal: sig, Checked: len(attrs)}
	names := make([]string, 0, len(attrs))
	for n := range attrs {
		names = append(names, n)
	}
	sort.Strings(names)
	for _, name := range names {
		val := attrs[name]
		if len(name) > maxAttrNameLen {
			rep.Findings = append(rep.Findings, Finding{name, "name_too_long", fmt.Sprintf("%d bytes", len(name))})
			continue
		}
		if !attrNameRE.MatchString(name) {
			rep.Findings = append(rep.Findings, Finding{name, "invalid_name", "lower-case dot-separated segments required"})
			continue
		}
		spec, ok := v.specs[name]
		if !ok {
			rep.Findings = append(rep.Findings, Finding{name, "unknown_attribute", "not in registry"})
			continue
		}
		if spec.Kind != val.Kind {
			rep.Findings = append(rep.Findings, Finding{name, "type_mismatch", fmt.Sprintf("want %s got %s", spec.Kind, val.Kind)})
			continue
		}
		if spec.Sensitive {
			rep.Findings = append(rep.Findings, Finding{name, "sensitive_attribute", "redact before export"})
		}
		if spec.HighCardinality && sig == SignalMetric {
			rep.Findings = append(rep.Findings, Finding{name, "high_cardinality_on_metric", "per-order key on a metric"})
		}
		if len(spec.Enum) > 0 && !slices.Contains(spec.Enum, val.Str) {
			rep.Findings = append(rep.Findings, Finding{name, "invalid_enum_value", fmt.Sprintf("%q not allowed", val.Str)})
		}
	}
	for _, name := range v.required {
		required := slices.Contains(v.specs[name].RequiredFor, sig)
		if !required {
			continue
		}
		if _, ok := attrs[name]; !ok {
			rep.Findings = append(rep.Findings, Finding{name, "missing_required", "required for " + string(sig)})
		}
	}
	return rep
}

测试覆盖八类违规、指标基数与敏感度的叠加、批量门禁和确定性排序。批量门禁故意混入一条合法 span、一条拼错键的 span、一条把状态码写成字符串的 span，以及一条带 order.id 的指标，预期拒绝率 75%。

// file: internal/platform/obs/conventions_test.go
// tier: L1
// verify: go test ./internal/platform/obs/ -count=1
package obs

import (
	"reflect"
	"strings"
	"testing"
)

func validSpanAttrs() map[string]Value {
	return map[string]Value{
		"service.name":              StringValue("order-service"),
		"service.version":           StringValue("v42"),
		"deployment.environment":    StringValue("prod"),
		"order.id":                  StringValue("ord-10086"),
		"order.status":              StringValue("PAID"),
		"http.request.method":       StringValue("POST"),
		"http.route":                StringValue("/orders/{id}/pay"),
		"http.response.status_code": IntValue(200),
		"payment.provider":          StringValue("mockpay"),
	}
}

func hasCode(rep Report, code string) bool {
	for _, f := range rep.Findings {
		if f.Code == code {
			return true
		}
	}
	return false
}

func TestValidateFindings(t *testing.T) {
	v := NewOrderValidator()
	cases := []struct {
		name   string
		signal SignalKind
		mutate func(map[string]Value)
		want   string
	}{
		{"typo key", SignalSpan, func(m map[string]Value) { m["order_id"] = StringValue("o-1") }, "unknown_attribute"},
		{"wrong type", SignalSpan, func(m map[string]Value) { m["http.response.status_code"] = StringValue("200") }, "type_mismatch"},
		{"metric cardinality", SignalMetric, func(m map[string]Value) { m["order.id"] = StringValue("o-1") }, "high_cardinality_on_metric"},
		{"sensitive", SignalSpan, func(m map[string]Value) { m["user.email"] = StringValue("a@b.c") }, "sensitive_attribute"},
		{"bad enum", SignalSpan, func(m map[string]Value) { m["order.status"] = StringValue("paid") }, "invalid_enum_value"},
		{"missing", SignalSpan, func(m map[string]Value) { delete(m, "order.id") }, "missing_required"},
		{"bad name", SignalSpan, func(m map[string]Value) { m["Order.ID"] = StringValue("o-1") }, "invalid_name"},
		{"long name", SignalSpan, func(m map[string]Value) { m["a."+strings.Repeat("b", 63)] = StringValue("x") }, "name_too_long"},
	}
	for _, tc := range cases {
		attrs := validSpanAttrs()
		if tc.signal == SignalMetric {
			for _, k := range []string{"order.status", "http.request.method", "http.route", "http.response.status_code", "payment.provider"} {
				delete(attrs, k)
			}
		}
		tc.mutate(attrs)
		if rep := v.Validate(tc.signal, attrs); !hasCode(rep, tc.want) {
			t.Fatalf("%s: want %s, got %+v", tc.name, tc.want, rep.Findings)
		}
	}
}

func TestMetricRejectsBoth(t *testing.T) {
	rep := NewOrderValidator().Validate(SignalMetric, map[string]Value{
		"service.name": StringValue("order-service"), "service.version": StringValue("v42"),
		"deployment.environment": StringValue("prod"), "order.id": StringValue("o-1"),
		"user.email": StringValue("a@b.c"),
	})
	codes := rep.Codes()
	if codes["high_cardinality_on_metric"] != 2 || codes["sensitive_attribute"] != 1 {
		t.Fatalf("codes = %v", codes)
	}
	t.Logf("codes=%v", codes)
}

func TestBatchGate(t *testing.T) {
	v := NewOrderValidator()
	events := []struct {
		name   string
		signal SignalKind
		attrs  map[string]Value
	}{
		{"good", SignalSpan, validSpanAttrs()},
		{"typo", SignalSpan, with(validSpanAttrs(), "order_id", StringValue("o-1"))},
		{"string status", SignalSpan, with(validSpanAttrs(), "http.response.status_code", StringValue("500"))},
		{"metric order id", SignalMetric, map[string]Value{
			"service.name": StringValue("order-service"), "service.version": StringValue("v42"),
			"deployment.environment": StringValue("prod"), "order.id": StringValue("o-1"),
		}},
	}
	rejected := 0
	for _, e := range events {
		if rep := v.Validate(e.signal, e.attrs); !rep.OK() {
			rejected++
			t.Logf("event=%q codes=%v", e.name, rep.Codes())
		}
	}
	t.Logf("events=%d rejected=%d rate=%.2f", len(events), rejected, float64(rejected)/float64(len(events)))
	if rejected != 3 {
		t.Fatalf("rejected = %d", rejected)
	}
}

func with(base map[string]Value, key string, val Value) map[string]Value {
	base[key] = val
	return base
}

func TestDeterministicFindings(t *testing.T) {
	v := NewOrderValidator()
	attrs := validSpanAttrs()
	attrs["z.bad"], attrs["a.bad"] = StringValue("x"), StringValue("x")
	attrs["service.name"] = IntValue(7)
	first, second := v.Validate(SignalSpan, attrs), v.Validate(SignalSpan, attrs)
	if !reflect.DeepEqual(first.Findings, second.Findings) || first.Findings[0].Attribute != "a.bad" {
		t.Fatalf("not deterministic: %+v", first.Findings)
	}
	t.Logf("findings=%d first=%s", len(first.Findings), first.Findings[0].Attribute)
}

本机运行输出

在 .local/checks/be-arch-gov-ch06 的 archlab module 中执行。环境是 macOS darwin/arm64、Go 1.26.0；gofmt -l . 和 go vet ./... 均无输出，下面只保留本篇相关测试。

$ go version
go version go1.26.0 darwin/arm64
$ gofmt -l .
$ go vet ./...
$ go test ./internal/platform/obs/ -count=1 -v -run 'TestValidateFindings|TestMetricRejectsBoth|TestBatchGate|TestDeterministicFindings'
=== RUN   TestValidateFindings
--- PASS: TestValidateFindings (0.00s)
=== RUN   TestMetricRejectsBoth
    conventions_test.go:76: codes=map[high_cardinality_on_metric:2 sensitive_attribute:1]
--- PASS: TestMetricRejectsBoth (0.00s)
=== RUN   TestBatchGate
    conventions_test.go:98: event="typo" codes=map[unknown_attribute:1]
    conventions_test.go:98: event="string status" codes=map[type_mismatch:1]
    conventions_test.go:98: event="metric order id" codes=map[high_cardinality_on_metric:1]
    conventions_test.go:101: events=4 rejected=3 rate=0.75
--- PASS: TestBatchGate (0.00s)
=== RUN   TestDeterministicFindings
    conventions_test.go:121: findings=3 first=a.bad
--- PASS: TestDeterministicFindings (0.00s)
PASS
ok  	archlab/internal/platform/obs	0.247s

读数分三组。违规组，八种输入分别命中 unknown_attribute、type_mismatch、high_cardinality_on_metric、sensitive_attribute、invalid_enum_value、missing_required、invalid_name、name_too_long，说明校验没有把“键不存在”和“键存在但值不对”混成同一个错误。叠加组，一条指标同时带 order.id 与 user.email，得到 2 个基数问题和 1 个敏感度问题，说明规则是并列执行而不是命中第一条就返回。门禁组，4 条事件拒绝 3 条，拒绝率 0.75，与预期一致；合法 span 没有被误杀，是这条门禁能否上线的关键。

边界要说明：这是单进程、内存注册表、固定输入的测试，不覆盖真实 SDK 的批量导出、属性截断和采样后的缺字段。0.247 秒只代表本机这组输入，不代表生产校验吞吐；线上应在采集端或 gateway 做抽样校验，避免每条遥测都走一遍完整注册表。

常见错误与边界

  • 只在代码评审里约定名字。评审无法覆盖第三方库、历史埋点和异步任务；必须把注册表做成可执行门禁。
  • 把所有高基数字段一刀切禁掉。order.id 在 span 和日志上必须保留，否则无法从慢请求定位订单；禁止的是它出现在指标标签里。
  • 忽略同一版本内的类型漂移。同一个 http.response.status_code 这次是 int、下次是 string，聚合会静默失败；类型必须和名字一起校验。
  • 把敏感字段交给下游脱敏。遥测一旦离开进程就可能被多处复制；应在导出前拒绝，而不是指望日志平台统一擦除。

小结

语义约定的价值不在于统一写法，而在于让“这个字段能不能出现在这类信号里”变成机器可判定的规则。名字、类型、基数、敏感度四层要一起校验，且必须按信号区分：span 允许订单号，metric 不允许；span 要求路由与状态码，日志要求级别与消息。

本节用 166 行的 obs 校验器把注册表、finding 和确定性排序固定下来，4 条事件的批量门禁拒绝了 3 条。它是后续两篇的基础：只有字段在同一套语义下可对齐，追踪采样和跨信号排障才不会建立在各自命名的沙地上。

阅读导航：上一节：5.3 动态配置一致性：推送、轮询与回滚 · 下一节：6.2 分布式追踪：采样、上下文传播与成本 。

继续阅读

探索更多技术文章

浏览归档，发现更多关于系统设计、工具链和工程实践的内容。

全部文章 返回首页

「books」更多文章

  1. 《互联网产品的 AI 架构·平台化与治理》附录 C 平台化评审清单
  2. 《互联网产品的 AI 架构·平台化与治理》附录 B 能力目录与配额字段速查
  3. 《互联网产品的 AI 架构·平台化与治理》附录 A 治理角色与职责速查