时间戳

首页 > 解决方案 > 使用 xargs -P 时如何将代码提取到函数中？

shell - 使用 xargs -P 时如何将代码提取到函数中？

问题描述

首先，我已经编写了代码，并且运行良好。

# version1
all_num=10
thread_num=5 
a=$(date +%H%M%S) 
seq 1 ${all_num} | xargs -n 1 -I {} -P ${thread_num} sh -c 'echo abc{}' 
b=$(date +%H%M%S) 
echo -e "startTime:\t$a"
echo -e "endTime:\t$b"

现在我想将代码提取到一个函数中，但它是错误的，如何修复它？

get_file(i){
echo "abc"+i
 } 
all_num=10
thread_num=5 
a=$(date +%H%M%S) 
seq 1 ${all_num} | xargs -n 1 -I {} -P ${thread_num} sh -c "$(get_file {})"
b=$(date +%H%M%S) 
echo -e "startTime:\t$a"
echo -e "endTime:\t$b"

标签： shellshxargs

解决方案

Because /bin/sh isn't guaranteed to have support for either printing text that when evaluates defines your function, or exporting functions through the environment, we need to do this the hard way, just duplicating the text of the function inside the copy of sh started by xargs.

Other questions already exist in this site describing how to accomplish this with bash, which is quite considerably easier. See f/e How can I use xargs to run a function in a command substitution for each match?

#!/bin/sh

all_num=10
thread_num=5 
batch_size=1  # but with a larger all_num, turn this up to start fewer copies of sh

a=$(date +%H%M%S) # warning: this is really inefficient
seq 1 ${all_num} | xargs -n "${batch_size}" -P "${thread_num}" sh -c '
  get_file() { i=$1; echo "abc ${i}"; }
  for arg do
    get_file "$arg"
  done
' _
b=$(date +%H%M%S)

printf 'startTime:\t%s\n' "$a"
printf 'endTime:\t%s\n' "$b"

Note:

  • echo -e is not guaranteed to work with /bin/sh. Moreover, for a shell to be truly compliant, echo -e is required to write -e to its output. See Why is printf better than echo? on UNIX & Linux Stack Exchange, and the APPLICATION USAGE section of the POSIX echo specification.
  • Putting {} in a sh -c '...{}...' position is a Really Bad Idea. Consider the case where you're passed in a filename that contains $(rm -rf ~)'$(rm -rf ~)' -- it can't be safely inserted in an unquoted context, or a double-quoted context, or a single-quoted context, or a heredoc.
  • Note that seq is also nonstandard and not guaranteed to be present on all POSIX-compliant systems. i=0; while [ "$i" -lt "$all_num" ]; do echo "$i"; i=$((i + 1)); done is an alternative that will work on all POSIX systems.

推荐阅读