Jalil Alchy04/07/2023, 8:45 PM
Scott Sandre (Delta Lake)04/08/2023, 3:03 AM
Krzysztof Chmielewski04/08/2023, 12:07 PM
Do you mean a row kind other than RowData?@Scott Sandre (Delta Lake) I think what @Jalil Alchy was refering to by RowKind is Flink's RowKind. The Flink's RowData::getRowKind() return a reference to RowKind. Currently Delta Source And Sink do not use this field so it will have no effect on the processing. As small side note, somewhat relevant to this is that for SQL/Table API that Im working on, both Delta Table Sink and Table Source will not work with CDC connectors that produces RowKind other than Insert. Flink table planner for every query verifies if ChangelogMode for source and Sink in the pipeline is compatible. I think that you could use CDC connectors with Delta connector in DataStreamAPI, where you could easily filter out RowData that has different RowKind than Insert. Flink SQL does not provide such a feature.