Hey when running a daily Optimize job on a Delta table without adding a where clause (meaning on the entire table) i would think it will optimize only the non optimized files, but it seems like it go over all the partitions again even if they weren’t touched at all from the last run is that correct?

Hi I had the same expectation, but I fear you are right. I made a feature request last year: https://github.com/delta-io/delta/issues/1260

On Databricks it will only optimize files/partitions that are not optimized yet. not sure about the OSS implementation