json - 如何将 bash 命令输出添加到漂亮格式的 json git log

Question

我对 bash 很陌生，并且坚持使用这种格式。我正在运行一个 bash 脚本来循环一些提交 url（一个 txt 文件包含这些，并且变量repo_url在循环期间存储一个 url）并以 json 格式获取日志数据保存到 file json.txt。

我能够以正确的格式获取我需要的所有数据，但我还想包含 repo url 以了解特定提交的来源。

repo url 采用 variable 的形式$repo_url。

如何将 bash 命令输出包含到漂亮格式的 json 中？

这是我目前拥有的：

git log -n 1\
--pretty=format:'{"commit": "%H", "author": "%aN <%aE>", "date": "%ai", "message": """%B""", "notes": """%N""" },' \
$@  | awk 'BEGIN { print("[") } { print($0) } END { print("]") }' | python -u -c \
'import ast,json,sys; print(json.dumps(ast.literal_eval(sys.stdin.read())))'

但我也想包括repo: "$repo_url"现在没有发生的所有条目。如果我尝试下面的代码，而不是实际的 url，我会"$repo_url"在输出中得到字符串。

git log -n 1\
--pretty=format:'{"repo": "$repo_url", "commit": "%H", "author": "%aN <%aE>", "date": "%ai", "message": """%B""", "notes": """%N""" },' \
$@  | awk 'BEGIN { print("[") } { print($0) } END { print("]") }' | python -u -c \
'import ast,json,sys; print(json.dumps(ast.literal_eval(sys.stdin.read())))'

如果我应该分享任何其他信息，请告诉我。

标签： jsonbashgit

Answer 1

@Reino，我还需要我无法通过 API 获得的每个提交的描述/消息。

你实际上可以。在我看来， JSON 解析器xidel是完成这项任务的理想命令行工具。我不知道您要处理哪些存储库，因此我将使用列出来自FFmpeg存储库的提交的API url。它返回一个数组，其中每个项目如下所示：

$ xidel -s https://api.github.com/repos/FFmpeg/FFmpeg/commits -e '
  $json()[sha="74e43824421bac9fa0816dce8b069eeab6f4cd07"]
'
{
  "sha": "74e43824421bac9fa0816dce8b069eeab6f4cd07",
  "node_id": "MDY6Q29tbWl0MTYxNDQxMDo3NGU0MzgyNDQyMWJhYzlmYTA4MTZkY2U4YjA2OWVlYWI2ZjRjZDA3",
  "commit": {
    "author": {
      "name": "Pierre-Anthony Lemieux",
      "email": "[e-mail]",
      "date": "2021-07-18T18:07:43Z"
    },
    "committer": {
      "name": "Marton Balint",
      "email": "[e-mail]",
      "date": "2021-07-29T20:38:02Z"
    },
    "message": "avformat/mxfdec: fix frame wrapping detection for J2K essence container\n\nFor JPEG 2000 essence, the MXF input format module currently uses the value of\nbyte 14 of the essence container UL to determine whether the J2K essence is\nclip- (byte 14 is 0x02) or frame-wrapped (byte 14 is 0x01). Otherwise it\nassumes an unknown wrapping.\n\nAdditional wrappings are documented in SMPTE ST422:2019:\n\n0x03: Interlaced Frame, 1 field/KLV\n0x04: Interlaced Frame, 2 fields/KLV\n0x05: Field-wrapped Picture Element\n0x06: Frame-wrapped Picture Element\n\nAnd these should also be handled as frame wrapped content.\n\nSigned-off-by: Pierre-Anthony Lemieux <[e-mail]>\nSigned-off-by: Marton Balint <[e-mail]>",
    [...]
  },
  [...]
}

要处理最新的 3 次提交，您可以执行以下操作：

$ xidel -s https://api.github.com/repos/FFmpeg/FFmpeg/commits -e '
  array{
    $json()[position() = 1 to 3]/{
      "repo":"https://github.com/FFmpeg/FFmpeg.git",
      "commit":sha,
      "author":commit/author/x"{name} <{email}>",
      "date":commit/author/date,
      "message":tokenize(commit/message,"\n\n")[1],
      "notes":extract(commit/message,"\n\n(.+)",1,"s")[.]
    }
  }
'
[
  {
    "repo": "https://github.com/FFmpeg/FFmpeg.git",
    "commit": "b9176dbfb7c209f2adf1f420df74b91df56c1fb3",
    "author": "Gyan Doshi <[e-mail]>",
    "date": "2021-07-30T11:38:53Z",
    "message": "avcodec/noise_bsf: restore dropamount for backwards compatibility",
    "notes": null
  },
  {
    "repo": "https://github.com/FFmpeg/FFmpeg.git",
    "commit": "ae94868a576c901f313c619950ec7d8a5f6ca615",
    "author": "Paul B Mahol <[e-mail]>",
    "date": "2021-07-30T07:24:49Z",
    "message": "avfilter/avf_showfreqs: switch to TX FFT from avutil",
    "notes": null
  },
  {
    "repo": "https://github.com/FFmpeg/FFmpeg.git",
    "commit": "74e43824421bac9fa0816dce8b069eeab6f4cd07",
    "author": "Pierre-Anthony Lemieux <[e-mail]>",
    "date": "2021-07-18T18:07:43Z",
    "message": "avformat/mxfdec: fix frame wrapping detection for J2K essence container",
    "notes": "For JPEG 2000 essence, the MXF input format module currently uses the value of\nbyte 14 of the essence container UL to determine whether the J2K essence is\nclip- (byte 14 is 0x02) or frame-wrapped (byte 14 is 0x01). Otherwise it\nassumes an unknown wrapping.\n\nAdditional wrappings are documented in SMPTE ST422:2019:\n\n0x03: Interlaced Frame, 1 field/KLV\n0x04: Interlaced Frame, 2 fields/KLV\n0x05: Field-wrapped Picture Element\n0x06: Frame-wrapped Picture Element\n\nAnd these should also be handled as frame wrapped content.\n\nSigned-off-by: Pierre-Anthony Lemieux <[e-mail]>\nSigned-off-by: Marton Balint <[e-mail]>"
  }
]

---

要处理 repo-url 来自文本文件的 repo 的最新提交：

$ xidel -s urls.txt -e '$raw'
#or
$ xidel -se 'file:read-text("urls.txt")'
https://github.com/FFmpeg/FFmpeg.git
https://github.com/rg3/youtube-dl.git
https://github.com/benibela/xidel.git

$ xidel -se '
  for $url in file:read-text-lines("urls.txt") return
  replace($url,".+/(.+/.+)\.git","https://api.github.com/repos/$1/commits")
'
https://api.github.com/repos/FFmpeg/FFmpeg/commits
https://api.github.com/repos/rg3/youtube-dl/commits
https://api.github.com/repos/benibela/xidel/commits

$ xidel -se '
  array{
    for $url in file:read-text-lines("urls.txt") return
    json-doc(
      replace($url,".+/(.+/.+)\.git","https://api.github.com/repos/$1/commits")
    )(1)/{
      "repo":$url,
      "commit":sha,
      "author":commit/author/x"{name} <{email}>",
      "date":commit/author/date,
      "message":tokenize(commit/message,"\n\n")[1],
      "notes":extract(commit/message,"\n\n(.+)",1,"s")[.]
    }
  }
'
[
  {
    "repo": "https://github.com/FFmpeg/FFmpeg.git",
    "commit": "b9176dbfb7c209f2adf1f420df74b91df56c1fb3",
    "author": "Gyan Doshi <[e-mail]>",
    "date": "2021-07-30T11:38:53Z",
    "message": "avcodec/noise_bsf: restore dropamount for backwards compatibility",
    "notes": null
  },
  {
    "repo": "https://github.com/rg3/youtube-dl.git",
    "commit": "a8035827177d6b59aca03bd717acb6a9bdd75ada",
    "author": "bopol <[e-mail]>",
    "date": "2021-07-01T06:53:22Z",
    "message": "[peertube] only call description endpoint if necessary (#29383)",
    "notes": null
  },
  {
    "repo": "https://github.com/benibela/xidel.git",
    "commit": "6d3b34408a0b0f0ed691753ea37860e98bcc29e6",
    "author": "benibela <[e-mail]>",
    "date": "2021-07-20T14:57:15Z",
    "message": "perhaps remove recursive follow",
    "notes": null
  }
]