Alec Ventura09/18/2023, 9:38 PM
This log is from the daily job that have a
Files were added to partition [event_date=2023-09-18] by a concurrent update
and I am sure that there is no 2023-09-18 data on that dataframe. So my idea now is use upsert on hourly run and continue with replaceWhere on daily one... but I don't know if daily will continue to fail since it will be an insert and an update operation at the same time.... Any thoughts? Thank you
replaceWhere = event_date between '2023-08-18' and '2023-09-17'
rtyler09/18/2023, 9:39 PM
Alec Ventura09/18/2023, 9:45 PM
Lori09/18/2023, 11:09 PM