Martin04/13/2023, 5:07 PM
) in order to keep the Delta table healthy. We are currently leaning towards (1). What are your thoughts?
Nick Karpov04/13/2023, 5:15 PM
Kashyap Bhatt04/13/2023, 5:26 PM
• tie a new Azure Function to consume from this pipe and write one file per minute (if there are any messages).It's been a while and things change pretty quickly. But IIRC you might be able to define the trigger for this new Function based on time or quantity (of messages waiting in Q) or both. So you might end up with even less than 1440 files per day. Only driver to consume from pipe more often than less often would be your application's latency requirements.
Nick Karpov04/13/2023, 5:31 PM
Kashyap Bhatt04/13/2023, 5:37 PM
Martin04/13/2023, 7:39 PM