Pascal Bellerose on 07 Oct 2021 19:31:27
I can understand why a simple table merge of one-to-one relationship having less than 10 000 records is taking more than a minute to process!!.
I could run the same script in 'R' and it would take less then 10 seconds to run it.
Honestly, I don't get why it has to go back to the source every single time. It should cache the dataset and just apply the transformations I select when developing a model.
Simples things like filtering, grouping or table merging should be able to use cached data when the dataset is less than a million records.
Yet my laptop's memory is through the roof and my CPU is constantly at 100% doing stupid background refreshes.
Even when doing it using Dataflows it kills your cloud resources.
Let me know if you need examples. I have plenty!