时间戳

首页 > 解决方案 > 无法选择表中的列，因为列名之一是限制

pyspark - 无法选择表中的列，因为列名之一是限制

问题描述

The above code is resulting in issues as it has a column name as keyword

限制。如果我从选择列表中删除列“限制”，则脚本工作正常。表 A 有以下内容 \**** 表 A *******\\\\

There are two tables A , B Table A as follows
 ID    Day      Name  Description  limit  
 1   2016-09-01  Sam   Retail       100
 2   2016-01-28  Chris Retail       200
 3   2016-02-06  ChrisTY Retail      50
 4   2016-02-26  Christa Retail      10
 3   2016-12-06  ChrisTu Retail     200
 4   2016-12-31  Christi Retail     500

Table B has following contents
\\\**** Table B *****\\\\\\\
 Table B

ID SkEY
1  1.1
2  1.2
3  1.3

上面的代码导致问题，因为它有一个列名作为关键字限制。如果我从选择列表中删除列“限制”，则脚本工作正常。

\\\**** Tried Code *****\\\\\\\
 from pyspark.sql import sparksession
 from pyspark.sql import functions as F
 from pyspark.sql.functions import lit
 from pyspark import HiveContext
 hiveContext= HiveContext(sc)
 ABC2 = spark.sql(
"select * From A where day ='{0}'".format(i[0])
 )
Join = ABC2.join(
 Tab2,
 (
    ABC2.ID == Tab2.ID
)
)\
.select(
Tab2.skey,
ABC2.Day,
ABC2.Name,
ABC2.limit,)
withColumn('newcol1, lit('')),
withColumn('newcol2, lit('A'))
ABC2 .show()

ABC=spark.sql("select distinct day from A where day='2016-01-01'")

\\\**** Expected Result *****\\\\\\\
How can we amend the code so that the limit is also selected

标签： pyspark

解决方案

It worked this wasy. not sure functional reason but is successful, Renaming   
the limit as alias before and there after getting it back
\\**** Tried Code *****\\\\\\\ 
from pyspark.sql import sparksession 
from pyspark.sql import functions as F 
from pyspark.sql.functions import lit 
from pyspark import HiveContext 
hiveContext= HiveContext(sc) 
ABC2 = spark.sql( "select Day,Name,Description,limit as liu From A where day
 ='{0}'".format(i[0]) ) 
 Join = ABC2.join( Tab2, ( ABC2.ID == Tab2.ID ) )\ 
.selectexpr( "skey as skey", 
            "Day as Day", 
            "Name as Day", 
            "liu as limit",) 
 withColumn('newcol1, lit('')), 
 withColumn('newcol2, lit('A')) 
 ABC2 .show()

推荐阅读