Andrew Chisholm on 16 Apr 2024 06:58:02
We have large log files stored in JSONL format, where each line is a separate JSON document, which makes storage and compression of these files efficient. Sadly the only way to get Power BI to process files like this quickly is to pre-split them into different files.
Or use a splitting script as per article titled - Solved: Change JSON Blob data during import - Microsoft Fabric Community
As noted in the example it takes 10mins to read 250,000 lines of this JSON.
A simple C# script can parse 250,000 JSON files in 203ms. I understand Power BI has more work to do than just deserialize, but this seems incredibly slow.