https://delta.io logo
Powered by
j

Jakub Filipek

05/03/2023, 5:33 AM
Hi, for 
delta-rs
when appending to a table it seems to go through every commit in history before writing (see thread for log). Is there a way to tell the update/append operation to start from a specific commit? Also, will this have performance impact if there are potentially tens of thousands of commits (assuming no debug logs are printed)?
Copy code
2023-05-02T22:29:28.442964721-07:00 DEBUG deltalake::delta - update without checkpoint
2023-05-02T22:29:28.443066165-07:00 DEBUG deltalake::delta - incremental update with version(-1) and max_version(None)
2023-05-02T22:29:28.443543361-07:00 DEBUG deltalake::delta - parsing commit with version 0...
2023-05-02T22:29:28.444063922-07:00 DEBUG deltalake::delta - merging table state with version: 0
2023-05-02T22:29:28.445715387-07:00 DEBUG deltalake::delta - parsing commit with version 1...
2023-05-02T22:29:28.479244435-07:00 DEBUG deltalake::delta - merging table state with version: 1
2023-05-02T22:29:28.484886002-07:00 DEBUG deltalake::delta - parsing commit with version 2...
2023-05-02T22:29:28.512041105-07:00 DEBUG deltalake::delta - merging table state with version: 2
Powered by