python - How to use UPDATE/DELETE WHERE CURRENT OF in psycopg2?
问题描述
I'm using server-side cursor in PostgreSQL with psycopg2, based on this well-explained answer.
with conn.cursor(name='name_of_cursor') as cursor:
query = "SELECT * FROM tbl FOR UPDATE"
cursor.execute(query)
for row in cursor:
# process row
In processing each row, I'd like to update a few fields in the row using PostgreSQL's UPDATE tbl SET ... WHERE CURRENT OF name_of_cursor
(docs), but it seems that, when the for loop enters and row
is set, the position of the server-side cursor is in a different record, so while I can run the command, the wrong record is updated.
How can I make sure the result iterator is in the same position as the cursor? (also preferably in a way that won't make the loop slower than updating using an ID)
解决方案
The reason why a different record was being updated was because internally psycopg2 does a FETCH FORWARD 1000
(or whatever the default chunk size is), positioning the cursor at the end of the block. You can override this by fetching one record at a time:
updcursor = conn.cursor()
with conn.cursor(name='name_of_cursor') as cursor:
cursor.itersize = 1 # to make server-side cursor be in the same position as the iterator
cursor.execute('SELECT * FROM tbl FOR UPDATE')
for row in cursor:
# process row...
updcursor.execute('UPDATE tbl SET fld1 = %s WHERE CURRENT OF name_of_cursor', [val])
The snippet above will update the correct record. Note that you cannot use the same cursor for selecting and updating, they must be different cursors.
Performance
Reducing the FETCH size to 1 reduces the retrieval performance by a lot. I definitely wouldn't recommend using this technique if you're iterating a large dataset (which is probably the case you're searching for server-side cursors) from a different host than the PostgreSQL server.
I ended up using a combination of exporting records to CSV, then importing them later using COPY FROM
(with the function copy_expert).
推荐阅读
- node.js - 如何以编程方式将 excel/CSV 数据导入 MongoDB 集合/文档(MERN)
- apache-kafka - 如何检查 Kerberos 身份验证作为活性探测的一部分?
- javascript - 如何使用 Javascript 格式化电子邮件中的 HTML?
- javascript - 使用 JS 下载 PNG 二进制文件
- html - Hugo 静态生成器网站上未触发单页模板
- haskell - 交互如何在多行字符串上与 Haskell 一起工作?
- javascript - 定义事件侦听器时无法调用命名函数
- c# - Inconsistent SSL error when accessing different sites
- r - 一次更改 R 中的多个矢量类
- r - Loop to apply lm() over rows of a matrix and extract significant models