[FEATURE] Idea for hard delete vs. soft delete

1

We’re trying to figure out the best way to handle deleted records. I thought it’s worth discussing.

My proposed solution is that at a table should have a togglable “soft delete” configuration. If soft delete is selected, that means that the record gets hidden, but could later be restored, if necessary. The record would STILL affect unique fields. However, the admin could always hop in, look up the “deleted” records, and then permanently delete any given record.

This could then further be extended to the API, with a soft: true/false flag when calling the delete operation.

The other option would be to tread all deletes permanently. Once a record is deleted, it’s fully removed and unrecoverable. Since it’s gone, any unique field values would be again available.

1 Like
2

Yeah I’m happy with either option. I know some of my tables are low priority so easiest to hard delete for those, but would be great to have the option to soft delete on some more important records.

1 Like
3

My 2 ¢ – what are the scenarios? Like a trash bin that can be emptied? Or an undo or version history like Google docs? As I understand it, many companies never delete records–they are marked for archive (moved to a separate db with matching schemas?). The soft true/false seems like it could add complexity and/or possibly make some operations ambiguous. If you do it wrong, records can be deleted when you thought they wouldn’t be.

Another approach, just thinking out loud, maybe each table supports a virtual “hidden” field, but is independent from a deletion operation. When querying, you can specify if you want to see all records, only visible records, or only hidden ones. Developers can use this to later delete, archive, or do something else with it.

When you say “The record would STILL affect unique fields”, what does that mean?
When you say “the admin could always hop in”, which admin are you talking about?

Mike

1 Like
4

My vote here would be to give us the option of hard-deleting data. Not being able to hard-delete data causes some unintended consequences (see issues #59 and #45), for example, wanting to mark fields as unique or mandatory, but being unable to do so because of pre-existing data that’s either the same or empty.

1 Like
5

Makes sense @mike_ekim1024 - I think of it more along the lines of, imagine that you’re building an app and you (the 8base user), wants to allow users to “delete their account.” However, if the user decides to sign back a few months later, their account could be restored. The “soft” delete would make that very easy, and were you to have unique usernames, the “deleted” user’s username would still be considered unavailable - until the record was manually hard-deleted.

Also, by the “admin” could hop in, I mean that the 8base developer could go into their console and have a way of reviewing soft-deleted records to manually restore them / permanently delete them.

Does that make sense?

6

Yeah @heymartinadams this happened to me the other day and was super annoying.

7

To be honest, the deleted account scenario sounds more like a user-level filter “account-active” field, rather than a system level “soft-delete”. What about banned accounts? Unpaid accounts? I think of those as user level, also.

If an account is not active, they simply can’t log in, so it’s not very complicated. Where I see potential for soft delete is where you want it to be treated as deleted (completely), but want to be able to restore it. @heymartinadams mentioned issues with just hiding it, so maybe it would be moved to a different table/workspace. Doing that of course is not trivial :slight_smile:

8

I get it! Yeah the account example is just an example, same thing would apply for any deleted records - by specifying its a soft-delete on the table, it hides it rather than permanently deletes it.

9

I agree, this seems unneccesary at the system level. If I want to build a feature that allows the user to restore something, I will build that at the field level into my api (active, published, etc).

Calling a delete mutation that removes the item from the data tab on the console while still fully functional and able to be queried makes no sense. I expect it to be DELETED, not ‘hidden’. It has already caused me multiple headaches and now I simply have to work around it, which sucks. I cannot have a mandatory unique field on any of my Users because the existing DELETED users don’t have any value for that field.

I would say a hard delete should be the default with an option to add a soft option if that’s what I what. It is completely counterintuitive to have soft delete the default AND give no option to permanently delete things. It makes it extraordinarily hard to build up anything without already knowing exactly all the fields you need ahead of time.

10

Completely agree, just hit this myself.

Would make more sense if records were soft-deleted but I could at least have an option to hard delete records from admin console, but I can’t set a field to mandatory or add a new field because of these phantom rows in the background.

@sebastian.scholl are there any plans to at least let us hard delete these records? A simple toggle on a table to hard delete all soft-deleted records wouuld solve my issue right now

11

Boom! After much (not too much…) deliberation we’ve come up with a plan that will be working on soon.

We’re going to add a tablenameDestroy mutation. The current delete mutation will behave as is, while the new destroy one will permanently delete the record.

Then, we are going to update the data viewer to allow for showing deleted records, where they can then be restored or permanently deleted manually.

Thoughts?

1 Like
12

Yeah definitely happy with this, I’ll just swap all my mutations to use destroy instead - leaves the option to soft-delete easily in the future too

1 Like
13

Can you remind us where this currently happens? Is this only for the Users table?

14

No no this will be for every table @mike_ekim1024. Any table you create will have the option of Deleting vs. Destroying a record.

15

What is as is for the API explorer? When I delete a record, I know that it’s permanently deleted because I can add a new row with the same unique value.

16

Even if it’s a field that enforces no duplicates?

17

Correct, in Data Builder, on a table I created. I have a status title (String) and value (Int), and I set it to 1000, delete it, create another one with 1000, ad infinitum.

So I don’t understand what the change will do if it applies to all tables.

18

This feature doesn’t seem to be out yet because deleting records from the Data Builder still seems to soft delete. I can’t find a switch anywhere that lets me choose to hard delete instead, or to view soft deleted records and permanently delete them.

This is quite annoying, since I’ve already soft deleted records (without realizing it - I thought Delete meant truly deleting something) and now I can’t disallow duplicate values in my database because the soft deleted records have duplicate values.

Is there a way for me to use the API Explorer to view the records that have been soft deleted and then permanently delete them?

19

Hey Serena! We working on this right now. In our original attempt there were some edge cases that prevented us from pushing out the “original” solution. Sorry that it’s giving you trouble :frowning_face:

Here is some info about some of the strategies we’re considering. Happy to recieve input on how you, and everyone else, would like to see this work.

SoftDelete vs HardDelete

(Strategy 1) - Table Level Settings

A table has a setting for Hard Delete Vs Soft Delete. At any point, the setting can be toggled on or off, however, changing the setting will result in irrecoverable data loss (destroy all deleted records).​

We would restrict this to ONLY the Administrator role, as well as add some custom documentation about how the Administrator has specific special abilities… like destroying records.

If a table is set to soft delete, there will be a toggle in the Data Viewer that says “See Deleted Records”. ANY ONE THAT HAS ACCESS TO THE ADMIN DATA VIEWER WILL HAVE ACCESS TO DESTROYING RECORDS ON A SOFT DELETE TABLE. However, this functionality will NOT be available via the API, for now.

(Strategy 2) - Delete vs. Destroy Mutations

​We add a recordDelete and recordDestroy mutation to the API. Delete soft deletes the record, Destroy eliminates the record. In the roles and permissions section, we add a drop down to the Delete action column instead of a checkbox with the following options.

Delete:

  • Off
  • Soft Delete
  • Hard Delete

Anyone with the permission, app user or team member, will be able to manage records accordingly via the API.

There will be a toggle in the Data Viewer that says “See Deleted Records”. Only users with Hard Delete permission will see this toggle, and then be able to restore records. However, this functionality will NOT be available via the API, for now. Additionally, ANY MANDATORY RELATIONSHIPS WILL CASCADE THE DELETE/DESTROY OPERATION RUN ON THE PARENT WHEN THE FORCE FLAG IS SPECIFIED.

(Strategy 3) - Destroy flag

Add a mutation argument top the recordDelete operation that is false by default, though when true runs a destroy operation against the record and cascades to any mandatory relationships.

There will be a toggle in the Data Viewer that says “See Deleted Records”. Only Administrators with the table Delete permission will be able to view and hard delete records.

mutation {
  somethingDelete(id: "123456789", destroy: true) {
     success
  }
}
1 Like
20

I need this to be available via the API (and I imagine most others have the same requirement).
Strategies 2 and 3 are basically the same in my view, just a different mutation vs passing a flag.
So keen!