Daniel Bartley on 20 Jan 2017 19:48:45
I have a source system with lots of uniqueidentifiers. I do ETL to strategically flatten the data model and change the data type of relationship keys and bitint to int64 (database is third party and overzealous for our purposes.
Could SSAS engine detect that some fields are unique identifier / hash and skip the dictionary step to reduce bloat? Ie. the processing engine encodes the hash as int64 and throws away the gibberish or dumps it in a redundant temp file.
- Comments (2)
RE: Data type optimisation for Uniqueidentifier/hash
If you use Tabular Editor 3, you can edit the encoding hints of your columns so that it becomes more likely that they will be encoded using value encoding (i.e. no dictionary) instead of hash encoding (i.e. with dictionary). However, as far as I know, editing the encoding hints is not supported by Microsoft.
RE: Data type optimisation for Uniqueidentifier/hash
Something like the link below would be great. Not too hard to implement, but would be nice if it came out the boxhttps://community.powerbi.com/t5/Desktop/Privacy-Hashing-of-keys/m-p/110574