时间戳

首页 > 解决方案 > 使用 shell 脚本将 YAML 数据转换为 CSV

bash - 使用 shell 脚本将 YAML 数据转换为 CSV

问题描述

我有一个文本文件，其中包含一个YAML对象，该对象具有通过访问远程服务器生成的重复键。

---
response:
  data:
    configElement:
      elementType: steering-pool
      attribute:
        name: ip-address
        value: 10.10.10.1
      attribute:
        name: start-port
        value: 100
      attribute:
        name: end-port
        value: 9000
      attribute:
        name: realm-id
        value: TSTore
      attribute:
        name: network-interface
        value: M10:150
      attribute:
        name: last-modified-by
        value: admin@10.10.10.10
      attribute:
        name: last-modified-date
        value: 2018-01-23 10:11
    configElement:
      elementType: steering-pool
      attribute:
        name: ip-address
        value: 10.10.10.2
      attribute:
        name: start-port
        value: 1000
      attribute:
        name: end-port
        value: 6535
      attribute:
        name: realm-id
        value: TSTLAN
      attribute:
        name: network-interface
        value: M10:278
      attribute:
        name: last-modified-by
        value: admin@10.10.10.10
      attribute:
        name: last-modified-date
        value: 2010-01-07 11:28
    configElement:
      elementType: steering-pool
      attribute:
        name: ip-address
        value: 10.10.10.3
      attribute:
        name: start-port
        value: 8000
      attribute:
        name: end-port
        value: 9535
      attribute:
        name: realm-id
        value: TestLab2
      attribute:
        name: network-interface
        value: M10:332
      attribute:
        name: last-modified-by
        value: admin@10.10.10.10
      attribute:
        name: last-modified-date
        value: 2009-02-10 12:14
    configElement:
      elementType: steering-pool
      attribute:
        name: ip-address
        value: 10.10.10.5
      attribute:
        name: start-port
        value: 100
      attribute:
        name: end-port
        value: 435
      attribute:
        name: realm-id
        value: NEWLA15
      attribute:
        name: network-interface
        value: M10:253
      attribute:
        name: last-modified-by
        value: admin@10.10.10.10
      attribute:
        name: last-modified-date
        value: 2016-02-22 17:32
    configElement:
      elementType: steering-pool
      attribute:
        name: ip-address
        value: 10.10.10.4
      attribute:
        name: start-port
        value: 100
      attribute:
        name: end-port
        value: 655
      attribute:
        name: realm-id
        value: LAN325
      attribute:
        name: network-interface
        value: M10:2153
      attribute:
        name: last-modified-by
        value: admin@10.10.10.10
      attribute:
        name: last-modified-date
        value: 2019-04-20 13:12
  messages:
  links:

我正在努力根据其结构将数据放入行中。它在一个标签下有多个元素configElement。

使用 shell 脚本，我想将数据写入行，每行包含configElement.

我已经尝试过https://stackoverflow.com/a/21189044/8763301上提出的解决方案，但它不适合我的文本数据中使用的格式。

预期的输出如下所示，将所有值放在一起configElement：

attribute: 10.10.10.1,100,9000,TSTore,M10:150,admin@10.10.10.10,"2018-01-23 10:11"
.
.
.
attribute: 10.10.10.2,1000,6535,TSTLAN,M10:278,admin@10.10.10.10,"2010-01-07 11:28"

请帮忙。

提前致谢。

标签： bashshellcsvyaml

解决方案

如果你跑

<input grep -E '(name:|value:)' | \
sed -r 's/^ +//g;s/name: ip-address/~\nname: ip-address/g;s/(name: |value: )//g' | \
perl -pe 's/\n/,/g;s/~,/\n/g;' | \
perl -pe 's/,$//g' | \
grep -vE '^$' | \
mlr --ocsv -N altkv >output.csv

你将会有

+------------+------------+----------+----------+-------------------+-------------------+--------------------+
| ip-address | start-port | end-port | realm-id | network-interface | last-modified-by  | last-modified-date |
+------------+------------+----------+----------+-------------------+-------------------+--------------------+
| 10.10.10.1 | 100        | 9000     | TSTore   | M10:150           | admin@10.10.10.10 | 2018-01-23 10:11   |
| 10.10.10.2 | 1000       | 6535     | TSTLAN   | M10:278           | admin@10.10.10.10 | 2010-01-07 11:28   |
| 10.10.10.3 | 8000       | 9535     | TestLab2 | M10:332           | admin@10.10.10.10 | 2009-02-10 12:14   |
| 10.10.10.5 | 100        | 435      | NEWLA15  | M10:253           | admin@10.10.10.10 | 2016-02-22 17:32   |
| 10.10.10.4 | 100        | 655      | LAN325   | M10:2153          | admin@10.10.10.10 | 2019-04-20 13:12   |
+------------+------------+----------+----------+-------------------+-------------------+--------------------+

最后一个实用程序 - mlr - 是 Miller ( https://github.com/johnkerl/miller )

推荐阅读