python - UnicodeEncodeError: 'ascii' codec can't encode character u'\xa0' in position 10 error while creating data flow template

Question

有人可以帮我解决在本地运行管道以创建数据流模板时遇到的错误。

我正在尝试从 CSV 文件中的大查询表加载数据，并且在本地运行管道时需要使用Python 2.7.

RuntimeError: UnicodeEncodeError: 'ascii' codec can't encode character u'\xa0' in position 10: ordinal not in range(128) [while running 'CSV format']

我尝试了诸如在我的python脚本中添加以下选项

import sys
reload(sys)
sys.setdefaultencoding('utf8')

同样在下面运行python脚本创建数据流模板之前，我也在下面运行。

from __future__ import absolute_import
import apache_beam as beam
import argparse
import pickle
import logging
#import sys
#reload(sys)
#sys.setdefaultencoding('utf8')
from apache_beam.options.pipeline_options import PipelineOptions
from apache_beam.io.gcp.internal.clients import bigquery
from datetime import date
today = date.today()
current_date = today.strftime("%Y%m%d")
def createFile(argv=None):
    parser = argparse.ArgumentParser()
    known_args, pipeline_args = parser.parse_known_args(argv)
    p = beam.Pipeline(options=PipelineOptions(pipeline_args))
    (p | 'ReadTable' >> beam.io.Read(beam.io.BigQuerySource(query="SELECT CUST_ROLE,CUST_ROLE_TXT,CUST_C4C_CUSTID,CUST_FNAME,CUST_LNAME,CUST_EMAIL,CUST_MOBILE,DL_BILL_TO_ADD,DL_SHIP_TO_ADD,CUST_STATE,CUST_GENDER_TXT,CUST_LOCALITY,CUST_GROUP,CUST_GROUP_TXT,CUST_DOB,CUST_SHIP_TO_EMAIL,CUST_CITY,CUST_WEDDING_ANNIVERSARY,CUST_PINCODE,CUST_UUID,CUST_ENTITY_ID,CONSENT_EMAIL,CONSENT_WHATSAPP,CONSENT_MOBILE FROM `whr-asia-datalake-nonprod.WHR_DATALAKE.CONSUMER_EXTRACT`",use_standard_sql=True))
       | 'read values' >> beam.Map(lambda x: x.values())
       | 'CSV format' >> beam.Map(lambda row:'|'.join(['"'+ str(column) +'"' for column in row]))
       | 'Write_to_GCS' >> beam.io.WriteToText('gs://whr-asia-datalake-dev-standard/outbound/Adobe/Customer_Master'+''+ str(current_date),file_name_suffix='.csv',header='CUST_ROLE|CUST_ROLE_TXT|CUST_C4C_CUSTID|CUST_FNAME|CUST_LNAME|CUST_EMAIL|CUST_MOBILE|DL_BILL_TO_ADD|DL_SHIP_TO_ADD|CUST_STATE|CUST_GENDER_TXT|CUST_LOCALITY|CUST_GROUP|CUST_GROUP_TXT|CUST_DOB|CUST_SHIP_TO_EMAIL|CUST_CITY|CUST_WEDDING_ANNIVERSARY|CUST_PINCODE|CUST_UUID|CUST_ENTITY_ID|CONSENT_EMAIL|CONSENT_WHATSAPP|CONSENT_MOBILE'))
    p.run().wait_until_finish()
if __name__ == '__main__':
    logging.getLogger().setLevel(logging.INFO)
    createFile()

请问有什么建议吗？导出 PYTHONIOENCODING=utf8

我得到的错误如下

"/home/radhika_sharma_ibm/Consumer_Extract.py", line 20, in <lambda> RuntimeError: UnicodeEncodeError: 'ascii' codec can't encode character u'\xc3' in position 8: ordinal not in range(128) [while running 'CSV format']

我的python脚本有以下代码

标签： pythongoogle-cloud-dataflowapache-beamdataflow