Mobile Bytes #1: Additive and Destructive Schema Changes

henna.s · 2022-01-19T09:18:59.092Z

Note: additive and destructive schema changes are now referred to as non-breaking and breaking changes but the concepts are identical.

Hello Everybody

I hope you are having a good time getting information on the forums. As you are aware, I started as Community Manager for Realm in Dec and I have been going through some questions on the forum and I found Schema Changes to be frequently asked questions and hence the birth of the idea Realm Bytes

Every Week, I would be selecting anyone such topic and along with your collaboration, we would be talking more on that.

Schema Changes can be additive (backwards compatible) or destructive (not backwards compatible) and care should be taken when making changes to Schema for a running application.

A diagram is presented below to help identify whether a schema change is additive or destructive:

image1712×663 74.8 KB

Additive Changes: These changes do not trigger a re-sync on the server and your SDK data models should be able to start using the new schema seamlessly. Your client-code (schema on mobile) can be a subset of your Cloud UI (schema on the server) Schema. Learn more

Note: Additive Changes can only Remove Fields if they are optional, removing Required fields will be a destructive change.

When you make an additive schema change there will be a brief update that takes place on the Realm Sync backend and changes may be delayed in propagating to MongoDB Atlas for a short period of time.

Destructive Changes: These changes will require a re-sync i.e Terminate and Re-enable Sync to map the new schema in place, and we strongly recommend making sure you have client reset logic in place first. These changes are preferably made directly in the cloud UI. Learn more

Note: Destructive changes are not allowed to be done via the Realm CLI or Code Deployment.

I hope the provided information is helpful.

I would love to hear from your experience, what worked and what didn’t. If there is a topic you want me to cover next week, please feel free to reach out.

Happy Realming!

Cheers,
Henna

henna.s · 2022-04-23T21:16:36.342Z

G’Day, Folks,

I would love to hear your experience with Additive and Destructive Schema changes I would like to share some more insights today that I discovered while experimenting with my restaurant app.

When you have development mode on, you can only make additive changes. If you try to make a destructive change, it will throw an error like this:

An exception has been thrown: The following changes cannot be made in additive-only schema mode:
    - Property 'User._id' has been changed from 'object id' to 'string'.

I wanted to change the type of the _id field in my User Schema and this was a destructive change and I was not able to do it from within the application.

I changed the schema directly on the cloud UI and it prompted me to resync my data I had to do that. Now when I try to sync my mobile client app, the server logs show the below error:

client file not found. The server has forgotten about the client-side file presented
 by the client. This is likely due to using a synchronized realm after terminating 
and re-enabling sync. Please wipe the file on the client to resume 
synchronization. { sessionIdent: 1, clientFileIdent: 25 } (ProtocolErrorCode=208)

This is client-reset message. I have not implemented client-reset in my code as this was a test app, so I uninstalled the application and synced again and I am able to run the app without any error.

This is not a friendly method for production apps, so the recommendation is to have client-reset implementation in the application.

Cheers,

lHengl · 2023-03-29T00:11:16.181Z

Hi @henna.s,

I’ve been having this issue on destructive change during development mode on and off and it seems the only way to resolve it has been to uninstall the app.

My issue is described here: How to update breaking-change schema of a Synced Realm during development?

In your example you mentioned that you didn’t have a client reset code, so you had to uninstall the app. It was the same for me, so I wanted to write the client reset code to handle the reset. But it seems that Realm only detects a single client reset, and subsequent resets no longer invokes the clientResetHandler. Which means i will have to uninstall the app anyway.

My question is: if we fail to handle the initial client reset, is the only option is to reinstall the app as per your example?

Brock · 2023-03-29T19:14:21.520Z

Oooohhh Did somebody say destructive schema change?

Oh boy!!! I got two years of stories!!!

But, unfortunately a lot of them may be NDA protected, so I’ll go with what my own findings are in my own tests:

Changing required fields to optional
Changing optional to required
Deleting a field
Changing the name of a field (including misspellings)

Additive changes are always best, unless you plan an outage to then make all of your breaking changes at the same time, and then reinitiate sync.

Brock · 2023-03-29T19:36:37.786Z

Client Reset Logic needs to seriously be in the tutorial get started app materials for Realm/Device Sync, even on outside forums and discord servers etc.

Next to additive vs destructive schema issues, it’s the lack of knowledge Client Reset Logic even exists, and is not automatic is still something nobody ever knows about until they break something and they don’t understand why a term and resync isn’t working right.

I was on a discord call last night as a matter of fact for a food delivery application company out of the UK using Realm, because someone wanted to change a spelling of a field and launched it. Brought the entire app down, they terminated and resync’d the app to find nothing fixed.

Lo and behold, they had no client reset logic, I had to walk them through it. Of course I didn’t mention a lot of things I would have liked to, staying professional and all, but 2 years, going on 3 years, it’s STILL a problem.

mongodb.com

Client Resets — Atlas App Services

henna.s · 2023-03-29T19:52:54.784Z

Hello @lHengl ,

Thank you for raising your concern. Please allow me some time to talk to the team and I will be back with feedback.

Appreciate your patience in the meantime

Cheers,
Henna

Tyler_Kaye · 2023-03-30T17:19:05.649Z

Hi @lHengl,

I am an engineer on the Device Sync team. I hear you and we agree that this is not an ideal experience for our developers. We have just begun a project to allow clients in Development Mode to make breaking schema changes that will automatically handle terminating/re-enabling sync and discarding the local realm file.

As for the best procedure to do this in the meantime, we agree that what you are doing is unfortunately the way to do it. Assuming you are testing in an emulator, the best way to go about making breaking schema changes is to:

Terminate sync
Remove the schema from the App Services UI
Enable Sync
Restart the emulator you are developing on
Connect to sync and it will sync the now additive changes to the schema (because you removed it)

We are very excited to fix this behavior and are actively working on it! I will try to reach back out here when this work is completed.

Best,
Tyler

Brock · 2023-03-30T17:35:30.089Z

@Tyler_Kaye you’re awesome!!! Can it also please be looked into for automated Client Reset Logic?

lHengl · 2023-04-03T22:00:19.878Z

Thanks @Tyler_Kaye, @Brock , @henna.s, I appreciate and look forward to the improvements. Is there some notifications I can subscribe to for such upcoming changes. E.g. is there an issue tracker?

Caleb_Thompson · 2023-04-27T20:50:22.495Z

Hi All - Caleb here, an engineer on the docs and tutorial projects. I’d love feedback on how we can improve this page on client resets: https://www.mongodb.com/docs/atlas/app-services/sync/error-handling/client-resets/.

lHengl · 2023-05-11T05:50:19.255Z

Hi @Caleb_Thompson ,

Here are my thoughts on the documentation…

Client Reset

The Client Resets page should link the reader back the Make Breaking Schema page. to avoid making the mistake in the first place.

Making Breaking Schema Changes

This page needs to be updated to take the reader through a step by step guide on how to correctly perform breaking changes in multiple ways:

One way is through the development mode. What steps should be taken when relying on the client SDK to write the schema for you.
The second way is through the Cloud Atlas App Services UI. What steps need to be taken in order for the change to take affect on all client. E.g. make the schema changes on Cloud Atlas, force manual resets by disabling and enabling the device sync in cloud atlas.
The third way is through the partner collection (which is existing). The partner collection I feel is a more advanced topic. People who are reading the documentation are probably the ones like me who are just starting out, using the development mode. So it should be left to last.

Create a Data Model

This page explains how to create models from existing data and from realm objects created by the SDK. Because of this, new developers like me would be relying heavily on this method to generate the models. But what if this fails? How does one manage this? The following sections should be introduced:

‘Manually update App Services Schema’ section on or similar should be introduced to help developers rectify and modify their schema when things go awry like breaking changes.
A note on ways to rectify the schema discrepancies. E.g. A simple way would be to delete the schema to force the client SDK to rewrite the schema during development.

That’s all for now. Hope to see some updates soon. Thanks!