scala - 从另一个数据框列值过滤数据框

Question

我们有两个数据框，我们需要用另一个数据框列中的数据过滤一个数据框中的数据

df1
-------------------------------
name paid_amount date_paid
-------------------------------
aaa   10          2017-10-10
aba   10          2017-01-10
aac   10          2017-10-10
daa   10          2017-16-10

df2
-----------------------------
start_date end_date
-----------------------------
2017-01-01 2018-01-01
------------------------------

we need to create third dataframe by checking  
(date_paid) field in df1 falls in between df2(start_date) & df2(end_date)

df1.where($date_paid).isin(df2.start_date && df2.end_date)

标签： scalaapache-sparkapache-spark-sql

Answer 1

应该：

df1.crossJoin(df2).where($"date_paid".between($"start_date", $"end_date"))