Gal Stainfeld08/21/2023, 8:40 PM
and inside it again by
date. Is there a way to tell the stream to only look for new data each interval from the
partition that suits today and yesterday? Not sure how exactly how it knows from where on the table to take the new events from the last checkpoint so maybe by utilizing the table’s partitions can help it. Thanks.
Nick Karpov08/22/2023, 4:47 PM
Gal Stainfeld08/22/2023, 4:53 PM
So the question is if putting such filter on the
will help the the stream in any way (because it will focus on reading two partitions only when getting the new data for the batch) or it won’t matter at all.
Nick Karpov08/22/2023, 5:03 PM