8base freezes when trying to scroll in a large JSON dataset field

If you have a very large set of JSON data in your row, edit that row, and try to scroll in the JSON field 8base freezes stops responding.

You have to kill the page to recover.

In my case, this is happening with an array of ~9,800 objects with 5 keys each

I can confirm, the entire JSON value is displayed in a . That could get out of control with even moderate size JSON in every row. The same happens for large text fields.

This bring up an interesting point – how can this be done in 8base? Simply store another field with the truncated data? Use left() on a table view? Add a custom resolver?

1 Like

Can you guys share a screenshot of what’s happening?

1 Like

Each row loads the entire contents of each field, no matter how large the data. With JSON and large text fields, that could slow down the page and cause a lot of bandwidth. Getting the first 100 characters or so would probably be fine. The api call for only those 2 rows was around 285K.

2 Likes

@mike_ekim1024 That’s the problem I tried to explain in this post: [BIG performance issue] Table never loads with multiple large JSON fields

The one is when you open try to edit a row and scroll down the page just freezes (nothing works, you can’t exit out) and after a while Chrome gives you an alert saying the page has stopped responding and you get an option to “Kill” it.

1 Like

Thank you both so much for sharing this! We 100% need to figure out a solution quickly.

Some type of lazy loading or only loading if the user opens a viewing module would probably work well for now.

1 Like

@sebastian.scholl, I wouldn’t say these 2 performance tickets are top priority.

They’re not blocking anything, I just thought I would highlight them to make 8base better.

There are other tickets for missing functionality that’s blocking such as updateMany, deleteMany & createMany w/ relationships (at least from my point of view)

The performance stuff you can get around this by just using queries until it’s fixed.

1 Like

Data is still needed for each row to display the first N characters of a field, so I’d assume using SQL left() to only get 100 characters or so. I think lazy loading would only work if you didn’t see any in the rows first.

Editing huge JSON could be a browser problem – it’s just a simple textarea. I added a huge amount of text and scrolling did start getting choppy in Firefox. What would be another way to store the JSON, as a File? Split into separate, multiple JSON fields?

1 Like

I’m picturing that we simply show a generic label in the Data Viewer field that when clicked opens a modal. When the modal opens, it queries the individual field from the records and displays it in a json viewer.

This way, JSON data is only loaded 1 record at a time if the user indicates they want to view the JSON data.

Good? Bad? Thoughts?

1 Like

Not being able to see the JSON on the row is probably ok, since a small piece of data doesn’t help much anyway, but this also applies to large TEXT fields—the exact same thing happens—it loads the entire text per row. Why can’t it use SQL “left(data, 100)” to only retrieve the first 100 characters?

I just noticed, one of my main tables (44 rows, 23 fields, 15 of them table type) is sluggish when I scroll up/down and left/right in the data viewer. If I keep adding more data, it will become unusable.

The second, separate point, is about editing huge JSON or TEXT. Maybe if the data is > 100,000 characters it prompts to download it and an option for uploading? It’s not feasible/appropriate to edit huge JSON/TEXT in a little textarea.

2 Likes

Yeah I agree.

Ideally, it would show the first e.g. 20-100 characters of the JSON.

If that’s not feasible, I’d rather not see any JSON at all, than I would be having the table be sluggish or slow to load. But yeah, as @mike_ekim1024 said it also applies to large text fields.

But there definitely are times where I would love to know if a json field stands out from the rest by e.g. being an empty object or an empty array that’s supposed to have something in it or the beginning of the json is different from all the other rows. Not showing the data at all would remove the ease of catching these unless you’re looking for it.

also, love @mike_ekim1024’s idea of downloading/uploading JSON/TEXT if it’s more than X characters in size That’s a nice solution for the EDIT modal.

1 Like

We pushed a fix out for this one. How do you guys feel about it? It doesn’t load the JSON in the records list, but instead loads the JSON once the file eye-con is clicked and displays it all pretty.

3 Likes

Thanks 8base team! That’s a big improvement to my workflow :grinning:

1 Like