https://delta.io logo
r

rtyler

07/20/2023, 2:37 PM
@Robert @Will Jones I have been fighting with GitHub actions and this periodic issue we see on link
Copy code
= note: /usr/bin/ld: final link failed: No space left on device
I don't know why this keeps happening, I nuked all our caches in Actions, but it still shows up šŸ˜ž I'm only seeing it on Linux but also experience it locally. This seems like a difficult to reproduce bug in Cargo unfortunately. I'm still trying to figure out what the heck is going on here, but if your travels have pointed out anything interesting, please let me know
w

Will Jones

07/20/2023, 2:50 PM
Just a random thought: might be good to check if we have more duplicate dependencies right now, which might be taking up more space
r

rtyler

07/20/2023, 2:50 PM
I am periodically running
cargo +nightly udeps
fwiw
šŸ‘ 1
I'll check for dupes now
o

Omkar

07/20/2023, 3:07 PM
Hey @rtyler, in case removing duplicate dependencies also doesn't help, you can probably try these as well: ā€¢ Disable incremental compilation in Cargo (config property here) - Incremental compilation caches additional info to disk during compilation to speed up builds. Disabling it might help save disk space, but we may need to check it's implications on compilation time. ā€¢ Add this Github Action called free-disk-space which is configurable and frees disk space on Ubuntu based runners.
r

rtyler

07/20/2023, 3:07 PM
I actually enabled incremental compilation earlier this week and it unfortunately didn't help šŸ˜•
There's something about how our integration tests are running (
cargo test -p deltalake --no-default-features --features integration_test,s3-native-tls,datafusion
) which I think is related. With a clean
target/
, that command alone generates 6GB of data šŸ˜¬
o

Omkar

07/20/2023, 3:20 PM
Is it possible to try disabling incremental compilation? And yes you're right, something looks to be happening during the test runs, I just tried compiling and running tests on an Ubuntu instance and the
target
folder is showing up as 7.3GB šŸ˜…
r

rtyler

07/20/2023, 3:21 PM
@Omkar in CI CARGO_INCREMENTAL=0 is set already šŸ˜•
o

Omkar

07/20/2023, 3:26 PM
Ohh got it okay, I'll check further and get back
w

Will Jones

07/20/2023, 3:28 PM
That's large, but what is the size if you set
RUSTFLAGS: "-C debuginfo=1"
(which we do in CI)
šŸ‘€ 1
šŸ‘šŸ¼ 1
r

rtyler

07/20/2023, 3:29 PM
i suspect that this at least has something to do with the sheer number of integration tests we're using, since each of those must be fully linked binaries. On my local machine when testing, the integration test binaries are each over 700M
šŸ‘ 1
šŸ‘šŸ¼ 1
w

Will Jones

07/20/2023, 3:29 PM
Yeah we should consolidate those into single binaries
For example, all the command_* can be a single binary
r

Robert

07/20/2023, 3:30 PM
I also seem to remember thinking that some of these could / should live in the crate itself ..
there are some fairly low level tests for some commit details in their own file...
r

rtyler

07/20/2023, 3:34 PM
@Will Jones RUSTFLAGS does chomp down to 3.3GB šŸ˜› Each of these test binaries are now only 4-500MB!
I'm about to drop into meetings. I will be able to look into this later today, I might have a pull request consolidating the
tests/
directory a bit coming soon
šŸ‘šŸ¼ 1
šŸ‘ 1
o

Omkar

07/21/2023, 9:49 AM
Thanks @rtyler, I just tried your PR. The tests worked like a charm, dropped a comment here. Cheers!
šŸ‘ 1
r

rtyler

07/21/2023, 4:14 PM
I added a
Cargo.toml
change to make development easier as well. Once this is merged I think there will be a lot more that will be unblocked šŸ¤ž
šŸ‘šŸ¼ 1
c

Cole MacKenzie

07/28/2023, 8:43 PM
My
delta-rs
workspace was 30 GB of rust compile targets šŸ¤Æ
šŸ™€ 2