krishna prasad

09/14/2023, 9:50 AM
Hi, I’m trying to clone an existing delta table locally using delta-rs. I’m using `DeltaTable.get_schema().unwrap().clone()`; to get the schema and I create a new table with
let mut config = DeltaOps(table).create().with_columns(schema.get_fields().clone());
I use datafusion a dataframe by `let df = self.ctx.sql(sql).await.unwrap()`; . When I try to write the reecordbatch from this dataframe by `record_batch_writer.write(batch).await?`; I see that the recordbatchs schema has a change. I see that the timestamp is data_type:
Timestamp(Nanosecond, None)
but the table schema has data_type:
Timestamp(Microsecond, None)
. Is there a config to fix this behaviour. Any help is appreciated.

Matthew Powers

09/14/2023, 7:37 PM

krishna prasad

09/15/2023, 6:55 AM
I’m sorry I’m a bit of novice, so I don’t think I understood your reply fully. From what I can make out, you are implying that clone (or the alternative in the conversation could do the job of what I’m trying). I will check that , But what I’m also faced with is that I have a lot of existing tables form which I’m trying to build new tables and not always are they clones. but in the case I still see the TimeStamp datatpe mismatch.