apache-flink - Apache flink: How to forward elements in timeWinow to process function when window closes?
问题描述
My Flink pseudo-code
timeWindow(Time.seconds(10))
.trigger(onElement -> if total size of elements in window > 100KB: FIRE_AND_PURGE or CONTINUE)
.process(print(input_collection))
I have timewindow of 10 seconds. I have trigger that sends (FIRE_AND_PURGE) contents of window(when total size of all elements in window reaches 100KB) to window process function But after 10 seconds when time window is closed , if I have 80KB of data in window then that residual data is not sent to window process function. I am loosing that 80KB of data … Whats right approach for it so that I dont loose that 80KB of remaining elements in timeWindow when timeWindow is closed at 10th second (edited)
For example, if I generate 230KB of data in 10 seconds then first 2 100KBs are sent to process function(by trigger by FIRE_AND_PURGE) but when timeWindow closes at 10th second, remaining 30KB is lost (edited)
Whats right approach to forward that 30KB to process function when time window closes ? (edited)
解决方案
When the event or processing time window expires, don't you get a call to your Trigger's onEventTime()
or onProcessingTime()
method that you could use to purge contents, the same as what you're doing in your onElement()
method?
推荐阅读
- javascript - NodeJS防止在长请求上重定向到主路由
- java - java - 如何通过UDP连接从java中的数据包中读取序列号?
- linux - udev 规则在连接 USB 重新启动时创建额外的符号链接
- javascript - 等待 fetch 获取数据
- vb.net - vb.net 设置 windows.old 文件夹权限
- android - WindowController 的切换状态栏文本颜色/setAppearanceLightStatusBars 未按预期工作
- angular - 在 Angular HTML 中显示来自对象的图像 blob
- youtube - Youtube 广告替代您自己的平台/创建一个类似 Youtube 的平台
- database - 在 SQLAlchemy 中使用 add_entity
- arrays - 如何在 VBA 中批量切片数组