时间戳

首页 > 解决方案 > data.table 未按组返回正确的 splinefun

r - data.table 未按组返回正确的 splinefun

问题描述

我们最近更新了data.tablefrom 版本1.12.0to1.12.8和 R from 3.5.3to 3.6.3。该示例在 Windows 操作系统上。

我们有一个data.table地方，我们正在循环一个类别列并创建一个splinefun稍后使用的对象。我们将此splinefun函数输出存储到列list中的 a 中data.table。它在我们的旧规范上按预期工作，splinefun根据分段数据为每个类别级别生成一个唯一的。但是，现在看起来它只是保留最终类别的值并将其解析为所有条目。

设置数据

创建一些虚假数据来显示问题

# R version: 3.6.3 (2020-02-29)
library(data.table) # data.table_1.12.8
library(ggplot2)
library(stats) 

# mimic our data in simpler format
set.seed(1)
dt <- data.table(cat = rep(letters[1:3], each = 10),
                 x = 1:10)
dt[, y := x^0.5 * rnorm(.N, mean=runif(1, 1, 100), sd=runif(1, 1, 10)), by=cat]

# can see that each line is different
pl0 <- ggplot(data=dt, aes(x=x, y=y, col=cat)) + geom_line()
pl0

拟合样条线

通过我们当前的方法拟合样条曲线并lapply用于比较。lapply按预期工作，data.table没有。

# fit spline, segment the data by category
mod_splines <- dt[, .(Spline = list(splinefun(x=x, y=y, method = "natural"))),
                  by = c("cat")]

# splinefun works such that you provide new values of x and it gives an output
# y from a spline fitted to y~x
# Can see they are all the same, which seems unlikely
mod_splines$Spline[[1]](5)
mod_splines$Spline[[2]](5)
mod_splines$Spline[[3]](5)

# alternative approach
alt_splines <-  lapply(unique(dt$cat), function(x_cat){
  splinefun(x=dt[cat==x_cat, ]$x, 
            y=dt[cat==x_cat, ]$y, 
            method = "natural")
})

# looks more realistic
alt_splines[[1]](5)
alt_splines[[2]](5)
alt_splines[[3]](5) # Matches the mod_splines one!

检查是否splinefun合身

当我们从 data.table 循环中打印出来时，数据和输出splinefun看起来是正确的，但它没有正确存储。

# check the data is segmenting
mod_splines2 <- dt[, .(Spline = list(splinefun(x=x, y=y, method = "natural")),
                      x=x, y=y),
                  by = c("cat")]
mod_splines2[] # the data is definitely segmenting ok

# try catching and printing the data
splinefun_withmorefun <- function(x, y){

  writeLines(paste(x, collapse =", "))
  writeLines(paste(round(y, 0), collapse =", "))

  foo <- splinefun(x=x, 
            y=y, 
            method = "natural")
  writeLines(paste(foo(5), collapse =", "))
  writeLines("")
  return(foo)
}

# looks like its in the function ok, as it prints out different results 
mod_splines3 <- dt[, .(Spline = list(splinefun_withmorefun(x=x, y=y))),
                   by = c("cat")]

# but not coming through in to the listed function
mod_splines3$Spline[[1]](5)
mod_splines3$Spline[[2]](5)
mod_splines3$Spline[[3]](5)

任何想法为什么这在更新后会成为一个问题会很棒！我们担心可能会有其他案例使用类似的data.table方法，现在可能会像这个案例一样被无声地破解。

谢谢你，乔尼

标签： rdata.tablespline

解决方案

正如我在https://github.com/Rdatatable/data.table/issues/4298#issuecomment-597737776copy()中回答的那样，添加x和y变量将解决此问题。

原因是它splinefun()会尝试存储 和 的x值y。但是，内部对象data.table总是通过引用传递（为了速度）......在这种情况下，您可能必须显式地copy()指定变量才能获得预期的答案。

简而言之，改变

mod_splines <- dt[, .(Spline = list(splinefun(x=x, y=y, method = "natural"))),
                  by = c("cat")]

至

mod_splines <- dt[, .(Spline = list(splinefun(x=copy(x), y=copy(y), method = "natural"))),
                  by = c("cat")]

或者这个（你可以忽略这个，但它可能会让你更好地理解）

mod_splines <- dt[, .(Spline = list(splinefun(x=x+0, y=y+0, method = "natural"))),
                  by = cat]

足够的。

推荐阅读