A JSON Lines ( or ) file format is a simple, plain-text format used for storing structured data where each line is a valid, self-contained JSON object. It's often referred to as Newline-Delimited JSON () and has the or file extension.
Unlike a standard JSON file, which typically contains a single JSON array encompassing all objects, a JSON Lines file avoids wrapping the data in an array. This structure offers several key advantages, particularly when processing large datasets:
Ease of Processing: Data can be streamed and processed line-by-line.
Convenience: Adding new data is simple; you just append a new JSON object to the end of the file on a new line.
Flexibility: Each line is independent, meaning if one line is corrupted, the rest of the data remains accessible.
JSON Lines is a preferred format for logging, transferring data between systems, and in big data applications.
| Example Structure |
Copy Code
|
|---|---|
{"name": "Alice", "age": 30}
{"name": "Bob", "age": 25}
{"name": "Charlie", "age": 35}
|
|
The built in JSON Lines editor makes it posisble to open huge JSON Lines files. Find, view and edit individual entries working on a formatted readable representation of the entry.
The editor is split into 2 views
The top view is the whole file. This is displayed in the large file editor, which can open files far larger that the physical memory on the machine (Terrabyte files open instantly), however this does come at the cost of limited editing functionality.
You can use JSON Lines in data mapper transforms using the "JSON Lines Reader" and "JSON Lines Writer". They behave exactly the same as the "JSON Source" and "JSON Target"