amazon-dynamodb-streams - dynamoDBStreams getRecords() returning empty records when ShardIteratorType is set to LATEST
问题描述
I was trying to retrieve last minute's changes in DynamoDB via dynamodb streams with AWS-SDK for nodeJS. I thought of its implementation by using ShardIteratorType as "LATEST". Every time I call getRecords() with the Iterator that I got getShardIterator() it is returning empty records whereas changing ShardIteratorType to "TRIM_HORIZON" returns the objects in that Shard.
I am unable to understand this behaviour. If anyone can help me figure it out, it would be appreciated.
Here is the pseudo code for the implementation.
let getShardIterator = async (shardID) => {
let params = {
ShardId: shardID,
ShardIteratorType: "LATEST", //using TRIM_HORIZON is working fine
StreamArn: streamInfo.StreamArn
};
return new Promise((resolve, reject) => {
dynamoStreams.getShardIterator(params, function (err, data) {
if (err) reject(err) // an error occurred
else resolve(data);
});
})
}
let getDataFromIterator = async (params) => {
return new Promise((resolve, reject) => {
dynamoStreams.getRecords(params, function (err, data) {
if (err) reject(err, err.stack);
else resolve(data);
});
})
}
let shardIterObj = await getShardIterator(ShardId);
let data = await getDataFromIterator(shardIterObj);
Thanks.
解决方案
As per the docs, LATEST
will return just after most recent record, this option normally is used if you are have worker/thread
running in background, so that any subsequent events can be picked up.
TRIM_HORIZON - Start reading at the last untrimmed record in the shard in the system, which is the oldest data record in the shard.
LATEST - Start reading just after the most recent record in the shard, so that you always read the most recent data in the shard.
推荐阅读
- android - android清单文件中的哪个权限触发了这个权限?
- python - 在 Linux 中使用 OpenCv Python 使用来自网络的视频
- python - Python 3.6:旋转矩阵
- javascript - 字符串与数字不兼容(默认为流类型可选属性)
- c# - 检查 ICollection 是否包含基于属性的元素
- sql - SQL 对过去三周的星期日到星期六之间的值求和
- javascript - 在Angular2底部滚动的div
- javascript - 使用 JS 显示和隐藏 div
- java - Spring Security - 如果缺少角色,则阻止登录
- docker - Docker Swarm 模式路由网格不适用于wireguard VPN