时间戳

首页 > 解决方案 > 为什么 R 坚持进口羽毛？

r - 为什么 R 坚持进口羽毛？

问题描述

我有一个 R 脚本，当我在循环中运行它时，它会在可变数量的迭代后粘住。没有错误消息 - 只是挂起。

通过 MRE：

library(arrow)

## clear environment
rm(list=ls())

### read in models
x <- 1
while (x < 50)
{
  print(x)
  matches_for_prediction <- read_feather(<file_path>)
  x = x + 1
  Sys.sleep(2)
}

我想知道这是否可能是某种打开文件冲突，但我尝试过延长睡眠时间但无济于事。

完全坦白......我主要使用Python并且对R不太了解。

标签： r

解决方案

As the code runs perfectly on small files, it could be a memory problem.

You could try the following, which will give:

  1. the size of the file you're trying to read
  2. the size of the new object in R
  3. the total memory used by R (memory.size() only works on Windows)
  4. perform a garbage collection so that unused objects are eliminated
x <- 1
while (x < 50) {
  print(paste('Reading file #',x,":",format(structure(file.size(path), class="object_size"),units='auto')))
  matches_for_prediction <- read_feather(path)
  print(paste('Object size',format(object.size(matches_for_prediction),'auto'),'- Total Memory used :',memory.size()))
  gc()
  x = x + 1
}

[1] "Reading file # 1 : 108.5 Mb"
[1] "Object size 669.9 Mb - Total Memory used : 1258.53"
...

The loop will only finish if it fits in memory!

推荐阅读