时间戳

首页 > 解决方案 > Shell - 删除重复的段落

shell - Shell - 删除重复的段落

问题描述

我有一个 .sh 文件，里面有很多重复的释义。现在我需要删除这些重复项，只保留一个

例如

**startkeyword**
  today is good
  how are you
  apple
**endKeyword**
**startkeyword**
  today is good
  how are you
  apple
**endKeyword**
**startkeyword**
 do you like apple
 I like apple
**endKeyword**
**startkeyword**
  today is good
  how are you
**endKeyword**
**startkeyword**
  today is good
  how are you
  apple
**endKeyword**

结果应该是：

**startkeyword**
  today is good
  how are you
  apple
**endKeyword**
**startkeyword**
 do you like apple
 I like apple
**endKeyword**
**startkeyword**
  today is good
  how are you
**endKeyword**

如何在 .sh 文件中执行此操作？免费使用 sed、awk 或 tr

标签： shellawk

解决方案

如果您不介意删除星号**

awk -v RS='[*][*]\n[*][*]' '{gsub(/[*]|\n$/,""); if(!visited[$0]++) print $0}' file

输出

startkeyword
  today is good
  how are you
  apple
endKeyword
startkeyword
 do you like apple
 I like apple
endKeyword
startkeyword
  today is good
  how are you
endKeyword

推荐阅读