apache-spark - java spark withColumn绑定属性错误

Question

在 java 中调用 spark udf 时出现“TreeNodeException：绑定属性，树：visit_datetime#161”错误

火花版本：'2.3.0' java 版本：'1.8.0'

java伪代码

udfs.put(function.name + "3", udf((String a, Boolean b, Boolean c) -> "hello", DataTypes.StringType));
udfs.put(function.name + "5", udf((String a, String b, String c, Boolean d, Boolean e) -> "world", DataTypes.StringType));

UserDefinedFunction udf = (UserDefinedFunction)udfs.get(function.name + seq.length());
if (udf != null) {
   tableDataSet = tableDataSet.withColumn(column + "_", udf.apply(seq));
}

tableDataSet.explain(true);
tableDataSet.show(5);

错误信息 ：

引起：java.lang.RuntimeException：在 [outpatient_visit_id#0,outpatient_no#1,patient_id#2,dom_hospital_no#3,outpatient_visit_count#4,cdt_visit_doctor#5,cdt_visit_doctor_id#6,cdt_visit_department#7 中找不到 visit_datetime#161， cdt_visit_dept_speciality#8,cdt_visit_dept_code#9,rstd_visit_dept_name#10,rstd_visit_dept_code#11,rstd_visit_speciality#12,cdt_visit_type#13,visit_datetime#14,clinic_document_id#15,患者年龄#16,cdt_fee_type#17,cdt_fee_type#17,cdt_19,status 20,registered_dept_code#21,registered_speciality#22,rstd_regist_dept_name#23,rstd_regist_dept_code#24,rstd_regist_speciality#25,selected_doctor#26,selected_doctor_id#27,time_of_register#28,triage_time#29,tirage_nurse#30,triage_du_mapping_id32,#source yidu_register_status#33,yidu_visit_type#34,yidu_data_credibility#35,first_or_further_visit#36,visit_dept_code#37,clinic_type#38]

== 物理计划 ==

!Project [trim(outpatient_visit_id#0, None) AS outpatient_visit_id#361, trim(outpatient_no#1, None) AS outpatient_no#401, trim(patient_id#2, None) AS patient_id#441, trim(dom_hospital_no#3, None) AS dom_hospital_no#481, trim(outpatient_visit_count#4, None) AS outpatient_visit_count#521, trim(cdt_visit_doctor#5, None) AS cdt_visit_doctor#561, trim(cdt_visit_doctor_id#6, None) AS cdt_visit_doctor_id#601, trim(cdt_visit_department#7,无）作为 cdt_visit_department#641，修剪（cdt_visit_dept_speciality#8，无）作为 cdt_visit_dept_speciality#681，修剪（cdt_visit_dept_code#9，无）作为 cdt_visit_dept_code#721，修剪（rstd_visit_dept_name#10，无）作为 rstd_visit_dept_name_761，修剪11，无）AS rstd_visit_dept_code#801，修剪（rstd_visit_speciality#12，无）AS rstd_visit_speciality#841，修剪（cdt_visit_type#13，无）AS cdt_visit_type#881，trim(UDF(visit_datetime#14, false, true), None) AS visit_datetime#921, trim(clinic_document_id#15, None) AS Clinic_document_id#961, trim(UDF(patient_age#16, visit_datetime#161,birth_date#201, false , true), None) AS patient_age#1001, trim(cdt_fee_type#17, None) AS cdt_fee_type#1041, trim(cdt_register_status#18, None) AS cdt_register_status#1081, trim(UDF(birth_date#19, false, true),无）作为出生日期#1121，修剪（注册部门#20，无）作为注册部门#1161，修剪（注册部门代码#21，无）作为注册部门代码#1201，修剪（注册专业#22，无）作为注册专业#1241，修剪（rstd_regist_dept_name# 23，无）AS rstd_regist_dept_name#1281，... 15 个更多字段]visit_datetime#161,birth_date#201, false, true), None) AS patient_age#1001, trim(cdt_fee_type#17, None) AS cdt_fee_type#1041, trim(cdt_register_status#18, None) AS cdt_register_status#1081, trim(UDF( birth_date#19, false, true), None) ASbirth_date#1121, trim(registered_dept#20, None) AS registered_dept#1161, trim(registered_dept_code#21, None) AS registered_dept_code#1201, trim(registered_speciality#22, None) AS registered_speciality#1241, trim(rstd_regist_dept_name#23, None) AS rstd_regist_dept_name#1281, ... 还有 15 个字段]visit_datetime#161,birth_date#201, false, true), None) AS patient_age#1001, trim(cdt_fee_type#17, None) AS cdt_fee_type#1041, trim(cdt_register_status#18, None) AS cdt_register_status#1081, trim(UDF( birth_date#19, false, true), None) ASbirth_date#1121, trim(registered_dept#20, None) AS registered_dept#1161, trim(registered_dept_code#21, None) AS registered_dept_code#1201, trim(registered_speciality#22, None) AS registered_speciality#1241, trim(rstd_regist_dept_name#23, None) AS rstd_regist_dept_name#1281, ... 还有 15 个字段]无）AS registered_dept_code#1201, trim(registered_speciality#22, None) AS registered_speciality#1241, trim(rstd_regist_dept_name#23, None) AS rstd_regist_dept_name#1281, ... 还有 15 个字段]无）AS registered_dept_code#1201, trim(registered_speciality#22, None) AS registered_speciality#1241, trim(rstd_regist_dept_name#23, None) AS rstd_regist_dept_name#1281, ... 还有 15 个字段]

标签： apache-spark