python-3.x - 将数据从 csv 文件插入到 mongodb

Question

在烧瓶中制作了一条从 csv 文件中读取数据并插入 mongodb 的路线。这是我第一次编写 python 代码，所以我尝试了一些我想在我的项目中做的事情。

@app.route('/adddata', methods=["GET"])
def add_data():
data = []
with open(csvfile) as file:
    filereader = csv.DictReader(file)
    for row in filereader:
        data.append(row)

usersList = mongo.db.users
usersList.insert_many(data)
return jsonify(data)



ERROR in app: Exception on /adddata [GET]
Traceback (most recent call last):
File "/Users/aditya/Work/python/flaskMongo/venv/lib/python3.8/site-packages/flask/app.py", 
line 2446, in wsgi_app
response = self.full_dispatch_request()
File "/Users/aditya/Work/python/flaskMongo/venv/lib/python3.8/site-packages/flask/app.py", 
line 1951, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/Users/aditya/Work/python/flaskMongo/venv/lib/python3.8/site-packages/flask/app.py", 
line 1820, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/Users/aditya/Work/python/flaskMongo/venv/lib/python3.8/site- 
packages/flask/_compat.py", line 39, in reraise
raise value
File "/Users/aditya/Work/python/flaskMongo/venv/lib/python3.8/site-packages/flask/app.py", 
line 1949, in full_dispatch_request
rv = self.dispatch_request()
File "/Users/aditya/Work/python/flaskMongo/venv/lib/python3.8/site-packages/flask/app.py", 
line 1935, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "index.py", line 39, in add_data
return jsonify(d)
File "/Users/aditya/Work/python/flaskMongo/venv/lib/python3.8/site- 
packages/flask/json/__init__.py", line 370, in jsonify
dumps(data, indent=indent, separators=separators) + "\n",
File "/Users/aditya/Work/python/flaskMongo/venv/lib/python3.8/site- 
packages/flask/json/__init__.py", line 211, in dumps
rv = _json.dumps(obj, **kwargs)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/json/__init__.py", line 
234, in dumps
return cls(
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/json/encoder.py", line 
199, in encode
chunks = self.iterencode(o, _one_shot=True)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/json/encoder.py", line 
257, in iterencode
return _iterencode(o, 0)     File 
"/Users/aditya/Work/python/flaskMongo/venv/lib/python3.8/site- 
packages/flask/json/__init__.py", 
line 100, in default
return _json.JSONEncoder.default(self, o)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/json/encoder.py", line 
179, in default
raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type ObjectId is not JSON serializable
127.0.0.1 - - [12/Jan/2020 01:19:44] "GET /adddata HTTP/1.1" 500 -

数据正在插入 mongodb，但我没有在浏览器中得到响应。当我删除插入代码块时，我可以在浏览器中看到响应。

usersList = mongo.db.users
usersList.insert_many(data)

从 mockaroo 生成 1000 个随机数据

id,first_name,last_name,email,gender,ip_address
1,Boonie,Caldero,[e-mail],Male,61.232.233.148
2,Tillie,Speck,[e-mail],Female,214.179.103.117
3,Lotti,Mulqueen,[e-mail],Female,28.90.88.221
4,Hilda,Arlett,[e-mail],Female,81.29.176.212
5,Philbert,Dwelley,[e-mail],Male,183.54.107.176
6,Anastasia,Faucett,[e-mail],Female,79.121.189.231
7,Trev,Hakes,[e-mail],Male,132.112.216.92
8,Kiersten,Siss,[e-mail],Female,34.204.92.1
9,Scot,Donoghue,[e-mail],Male,227.46.24.115
10,Anatollo,Urien,[e-mail],Male,198.96.237.73

标签： python-3.xmongodbpymongopymongo-3.xflask-pymongo

Answer 1

您已经大部分回答了自己的问题，但为了解决您的问题，pymongo 驱动程序将始终_id在插入数据时添加一个字段。

插入文档时，如果文档尚未包含“_id”键，则会自动添加特殊键“_id”。 参考

看这个简单的例子。如果您不想要该_id字段，只需在插入后弹出即可；或者，您可以在插入数据之前将其复制到新变量中。

import pymongo

db = pymongo.MongoClient()['mycollections']
records = [{'a': 1}, {'b': 2}]
usersList = db.testcollection.insert_many(records)
print(f'Before pop() : {records}')
for record in records:
    record.pop('_id')
print(f'After pop()  : {records}')

给出：

Before pop() : [{'a': 1, '_id': ObjectId('5e1b055b1eaf204578766fc6')}, {'b': 2, '_id': ObjectId('5e1b055b1eaf204578766fc7')}]
After pop()  : [{'a': 1}, {'b': 2}]