16-EFT.WP.Methods.Cleaning v1.0 | 附录C 清单模板与样例

目录／文档-技术白皮书／ 16-EFT.WP.Methods.Cleaning v1.0

附录C 清单模板与样例

一句话目标：提供发布级 manifest 的最小键集、字段口径与可直接装配的模板与样例，支撑“可追溯、可审计、可回退”的清洗发布流程。

I. 范围与对象

适用对象
批处理数据集发布、事件流窗口发布、在线快照发布。
输出形态
manifest.json（机器可读）与 manifest.sig（签名块），必要时附人读版 manifest.yaml。
基本约束
内部计算在 tau_mono，对外发布在 ts；两口径产出 T_arr 与 delta_form；量纲守恒通过 check_dim(expr)。

II. 结构与键规范

顶层键建议（命名稳定）

meta：版本、生成器与时间戳
lineage：来源、作业、提交与 TraceID
schema：模式引用与哈希
units_dim：单位系统与量纲校核摘要
timing：offset/skew/J 与时基映射
path_arrival：gamma(ell)、两口径与 delta_form
missing_impute：m ∈ {0,1} 覆盖与插补记录
quality：q_score、覆盖率与 P99 指标
outlier_drift：离群率与漂移度量
integrity：唯一性、参照与去重结果
env_correction：RefCond 与 corr_env(x; RefCond)
contracts：assert_contract 测试清单与结果
release：冻结信息、签名与回退锚点
artifacts：导出物与校验和
audit：执行日志与链式哈希

III. 最小键集（强制）

meta.version = "1.0"
meta.generated_at = ts
lineage.TraceID
schema.ref 与 schema.hash
units_dim.system 与 units_dim.pass
timing.offset, timing.skew, timing.J
path_arrival.c_ref, path_arrival.T_arr_form1, path_arrival.T_arr_form2, path_arrival.delta_form, path_arrival.tol_Tarr
missing_impute.missing_ratio, missing_impute.methods[]
integrity.unique_pass, integrity.fk_pass
contracts.tests[] 与 contracts.pass
release.tag, release.freeze_at, release.hash_sha256, release.signature

IV. 字段口径说明（摘选）

T_arr 两口径
- T_arr_form1 = ( 1 / c_ref ) * ( ∫_{gamma(ell)} n_eff d ell )
- T_arr_form2 = ( ∫_{gamma(ell)} ( n_eff / c_ref ) d ell )
- delta_form = | T_arr_form1 - T_arr_form2 |，需断言 delta_form ≤ tol_Tarr。
时基指标
offset（平均相位偏移），skew（频偏，单位 ppm），J（抖动 P95 或 P99）。
量纲与单位
unit(t_arr)="s", dim(t_arr)="[T]"；所有字段经 check_dim 校核通过方可发布。
路径与长度
non_decreasing(ell)；L_gamma = ( ∫_gamma 1 d ell ) 记录于 path_arrival。
缺失与插补
缺失掩码 m 的覆盖率与插补方法及 RefCond 必须落盘。
签名与追溯
release.hash_sha256(blob) 与 release.signature；审计链 audit.prev_hash 串联。

V. 模板 A（批处理发布 manifest.json）

JSON json

{
  "meta": {
    "version": "1.0",
    "title": "D_clean daily snapshot",
    "generated_at": "<ts>",
    "generator": "EFT.cleaning.freeze_release/1.3.2"
  },
  "lineage": {
    "source_uris": [ "s3://bucket/raw/2025-08-30/" ],
    "job_id": "<job-uuid>",
    "commit": "<git-sha>",
    "TraceID": "<trace-id>"
  },
  "schema": { "ref": "EFT.WP.Core.DataSpec v1.0:SRef", "hash": "<sha256-of-schema-json>" },
  "units_dim": {
    "system": "SI",
    "checks": [
      { "expr": "t_arr", "dim": "[T]", "pass": true },
      { "expr": "n_eff", "dim": "[]", "pass": true }
    ],
    "pass": true
  },
  "timing": {
    "timebase_in": "tau_mono",
    "timebase_out": "ts",
    "offset_ms": "<float>",
    "skew_ppm": "<float>",
    "J_ms_p99": "<float>"
  },
  "path_arrival": {
    "gamma_param": "ell",
    "L_gamma": "<float>",
    "c_ref": "<float>",
    "T_arr_form1_s": "<float>",
    "T_arr_form2_s": "<float>",
    "delta_form_s": "<float>",
    "tol_Tarr_s": "<float>",
    "p99_delta_form_s": "<float>"
  },
  "missing_impute": {
    "missing_ratio": "<0..1>",
    "mask_field": "m",
    "methods": [ { "field": "Xi", "method": "linear", "RefCond": "T=293K,P=1atm" } ]
  },
  "quality": {
    "q_score_mean": "<float>",
    "q_score_p99": "<float>",
    "coverage": { "records": "<int>", "fields": "<int>" }
  },
  "outlier_drift": { "outlier_rate": "<float>", "drift_metric": "PSI", "drift_value": "<float>", "window": "7d" },
  "integrity": {
    "unique_keys": [ "pk" ],
    "unique_pass": true,
    "fk_checks": [ { "child": "pid", "parent": "pid_ref", "pass": true, "orphan": 0 } ],
    "dedup": { "conflicts_resolved": "<int>", "residual_conflicts": 0 }
  },
  "env_correction": {
    "RefCond": "T=293K,P=1atm",
    "fields": [ "T_arr" ],
    "corr": "corr_env(T_arr; RefCond)",
    "uncertainty_U": "<float>"
  },
  "contracts": {
    "tests": [
      "UNIQUE(pk)",
      "DIM(\"t_arr\",\"[T]\")",
      "ARRIVAL_FORMS(c_ref=<float>, tol=<float>, tolP99=<float>)",
      "MANIFEST_SIGNED()"
    ],
    "pass": true,
    "failed": [  ]
  },
  "release": {
    "tag": "clean-2025-08-30",
    "freeze_at": "<ts>",
    "hash_sha256": "<sha256-of-payload>",
    "signature": "<base64-signature>",
    "public_key_id": "<kid>",
    "prev_hash": "<sha256-prev-release>"
  },
  "artifacts": [ { "name": "D_clean.parquet", "uri": "s3://bucket/clean/...", "sha256": "<hash>" } ],
  "audit": {
    "operator": "batch-runner",
    "events": [ { "ts": "<ts>", "action": "assert_contract", "result": "pass" } ],
    "prev_hash": "<sha256-prev-audit>"
  }
}

VI. 模板 B（事件流窗口发布 manifest.window.json）

JSON json

{
  "meta": { "version": "1.0", "generated_at": "<ts>", "generator": "EFT.stream.release/0.9" },
  "lineage": { "topic": "topic.clean", "partition": 7, "offset_range": "12345..12890", "TraceID": "<trace>" },
  "window": { "start_ts": "<ts>", "end_ts": "<ts>", "Delta_t": "15m" },
  "timing": { "offset_ms": "<float>", "skew_ppm": "<float>", "J_ms_p95": "<float>" },
  "path_arrival": { "c_ref": "<float>", "p99_delta_form_s": "<float>", "tolP99_Tarr_s": "<float>" },
  "missing_impute": { "missing_ratio": "<float>", "mask_field": "m" },
  "outlier_drift": { "outlier_rate": "<float>", "drift_metric": "PSI", "drift_value": "<float>" },
  "contracts": {
    "tests": [ "MONOTONE(\"tau_mono\")", "SYNC_BOUNDS(...)", "ARRIVAL_FORMS(...)" ],
    "pass": true
  },
  "release": {
    "tag": "stream-2025-08-30T12:15Z",
    "hash_sha256": "<hash>",
    "signature": "<sig>",
    "prev_hash": "<hash>"
  }
}

VII. 模板 C（在线快照发布 manifest.api.json）

JSON json

{
  "meta": { "version": "1.0", "generated_at": "<ts>", "generator": "EFT.api.snapshot/2.1" },
  "lineage": { "service": "read-api", "build": "<git-sha>", "TraceID": "<trace>" },
  "timing": { "offset_ms": "<float>", "skew_ppm": "<float>", "J_ms_p99": "<float>" },
  "quality": { "fresh_age_s": "<float>", "q_score_mean": "<float>" },
  "contracts": { "tests": [ "SYNC_BOUNDS(...)", "UNIT_SYSTEM(\"SI\")" ], "pass": true },
  "release": { "tag": "api-<ts>", "hash_sha256": "<hash>", "signature": "<sig>" }
}

VIII. 样例（已填充，批处理发布）

{

"meta": {

"version": "1.0",

"title": "D_clean daily snapshot",

"generated_at": "2025-08-30T02:10:45Z",

"generator": "EFT.cleaning.freeze_release/1.3.2"

"lineage": {

"source_uris": ["s3://lab/raw/2025-08-29/"],

"job_id": "7d8c1f1e-1c3a-4bd7-91d2-7c2b3d1e0a77",

"commit": "a9b3c4d",

"TraceID": "tr-01HZY2P6Z9"

"schema": {

"ref": "EFT.WP.Core.DataSpec v1.0:SRef",

"hash": "3f1c0f7b1b7a2c...e9d"

"units_dim": {

"system": "SI",

"checks": [

{"expr": "t_arr", "dim": "[T]", "pass": true},

{"expr": "n_eff", "dim": "[]", "pass": true}

"pass": true

"timing": {

"timebase_in": "tau_mono",

"timebase_out": "ts",

"offset_ms": 1.8,

"skew_ppm": 27.0,

"J_ms_p99": 2.6

"path_arrival": {

"gamma_param": "ell",

"L_gamma": 1243.7,

"c_ref": 2.99792458e8,

"T_arr_form1_s": 4.150002e-06,

"T_arr_form2_s": 4.150006e-06,

"delta_form_s": 4.0e-12,

"tol_Tarr_s": 5.0e-06,

"p99_delta_form_s": 6.0e-12

"missing_impute": {

"missing_ratio": 0.032,

"mask_field": "m",

"methods": [

{"field": "Xi", "method": "linear", "RefCond": "T=293K,P=1atm"}

]

"quality": {

"q_score_mean": 0.982,

"q_score_p99": 0.998,

"coverage": {"records": 18423321, "fields": 57}

"outlier_drift": {

"outlier_rate": 0.008,

"drift_metric": "PSI",

"drift_value": 0.06,

"window": "7d"

"integrity": {

"unique_keys": ["pk"],

"unique_pass": true,

"fk_checks": [

{"child": "pid", "parent": "pid_ref", "pass": true, "orphan": 0}

"dedup": {"conflicts_resolved": 271, "residual_conflicts": 0}

"env_correction": {

"RefCond": "T=293K,P=1atm",

"fields": ["T_arr"],

"corr": "corr_env(T_arr; RefCond)",

"uncertainty_U": 1.4e-07

"contracts": {

"tests": [

"UNIQUE(pk)",

"DIM(\"t_arr\",\"[T]\")",

"ARRIVAL_FORMS(c_ref=2.99792458e8, tol=5e-6, tolP99=1e-5)",

"MANIFEST_SIGNED()"

"pass": true,

"failed": []

"release": {

"tag": "clean-2025-08-30",

"freeze_at": "2025-08-30T02:12:11Z",

"hash_sha256": "b7c1f6c4...aa12",

"signature": "MEYCIQDZ...AB",

"public_key_id": "kid-ops-2025Q3",

"prev_hash": "8aa9d7...01fe"

"artifacts": [

{"name": "D_clean.parquet", "uri": "s3://lab/clean/2025-08-30/D_clean.parquet", "sha256": "1d77...9e"}

"audit": {

"operator": "batch-runner",

"events": [

{"ts": "2025-08-30T02:10:50Z", "action": "assert_contract", "result": "pass"},

{"ts": "2025-08-30T02:12:10Z", "action": "sign", "result": "pass"}

"prev_hash": "a1f2...ccd"

}

IX. 生成与校验流程（对接 I10-*）

准备输入
ds 经 standardize_names 与 repair_units 完成模式与量纲就绪。
计算与标注
align_timebase 产出 offset/skew/J；enforce_arrival_time_convention 同步计算两口径与 delta_form；handle_missing 落盘 m 与 RefCond；detect_outlier 与去重结果写入统计。
契约执行
调用 assert_contract(ds, tests)，聚合通过/失败与 sev，写入 contracts。
冻结与签名
freeze_release(ds, tag) 生成 artifacts、计算 hash_sha256(blob)，使用 KMS 签名写入 signature，串联 prev_hash。
发布与审计
清单与导出物写入对象存储；写入 audit.events 与链式哈希。

X. 校验要点与回退锚点

量纲守恒
断言 check_dim( t_arr - ( 1 / c_ref ) * ( ∫ n_eff d ell ) ) = 0。
两口径差
断言 delta_form ≤ tol_Tarr 且 Q_0.99(delta_form) ≤ tolP99_Tarr。
单调与参照
断言 non_decreasing(ts) 与 foreign_key 全通过；residual_conflicts = 0。
回退
使用 release.prev_hash 快速定位最近一次健康版本，执行回滚或重放。

小结
本附录给出统一结构、最小键集与三类发布场景的 manifest 模板，并配套已填充样例。按本模板落盘，可实现跨卷一致的到达时两口径、时基、量纲、参照与签名追溯，支撑 assert_contract、freeze_release 与审计回退的端到端闭环。

版权与许可（CC BY 4.0）

版权声明：除另有说明外，《能量丝理论》（含文本、图表、插图、符号与公式）的著作权由作者（“屠广林”先生）享有。
许可方式：本作品采用 Creative Commons 署名 4.0 国际许可协议（CC BY 4.0）进行许可；在注明作者与来源的前提下，允许为商业或非商业目的进行复制、转载、节选、改编与再分发。
署名格式（建议）：作者：“屠广林”；作品：《能量丝理论》；来源：energyfilament.org；许可证：CC BY 4.0。

首次发布： 2025-11-11｜当前版本：v5.1
协议链接：https://creativecommons.org/licenses/by/4.0/