JSON Parsing Issue: Missing Commas Between Objects

Problem Introduction

When processing the returned JSON today, I encountered the following error:

When troubleshooting this issue, the first step is to check the format of the current string. For example, after inspection, I found mine looked like this:

I noticed there were no commas between the JSON objects—meaning the JSON format was invalid. Checking in byte mode revealed only a single newline character separating them:

Solution

The method I found uses the Pandas library, which has a lines mode specifically designed to handle this scenario and parse the data into JSON dictionaries:

import pandas as pd
df = pd.read_json(ret.text, lines=True)

This was the only solution that worked for me. The built-in json module couldn’t resolve the issue—whether I tried writing the string to a file and reading it back, or replacing newline characters in the string before parsing, nothing worked. I tested multiple approaches, and only Pandas succeeded.

But why is that?

I’m not entirely sure yet. If you’re familiar with Python and know the reason, or have any guesses, please feel free to let me know. After all, understanding the “why” is how we progress.

I hope these will help someone in need~