How to prepopulate a synced realm?


#1

How do I correctly bundle a prepopulated Realm file into an app and then open and sync future changes for each distinct app user and/or device?

TLDR;

I am struggling to figure out how to prepopulate a synced realm. I’ve followed all of the suggestions I’ve seen online, but whenever I try to open the realm file with a sync configuration (full or query-based), I get the informative error message “Incompatible histories. Expected an empty Realm or a Realm written by Realm Mobile Platform.”

This will be a long post; I apologize. Some additional information, here is my tech stack:

React Native: 0.57.8
realm: 2.21.1
Realm Cloud

Steps I’ve followed up to this point:

  1. I developed the app with a single admin user; this admin has permissions to write data to the Realm Cloud instance and is also the sync user used during development
  2. I then integrated Auth0 and setup JWT authentication which I have verified is working and I see new Auth0 users in the Users table in my Realm Cloud instance
  3. Following advice I’ve found online, I made a copy of the Realm file to bundle with the app (1 opened the Realm file in a node process, 2 used writeCopyTo to generate a new file, 3 add that as an asset in XCode and use copyBundledRealmFiles during app startup)
  4. I can open the copied realm file in Realm studio without issue and I can open the Realm file in the app, without a sync configuration without issue which is what I do for new app users

When a new user registers:

  1. App starts and opens the bundled Realm file without sync
  2. User initiates signup and is directed to the Auth0 registration page
  3. If successful, Auth0 returns the access, refresh and JWT tokens
  4. Create a sync user using the Auth0 JWT
  5. If successful, do a writeCopyTo of the initially opened Realm file, create a sync configuration for the newly created (using user.createConfiguration) and specifying the path as the newly copied realm file
  6. Receive the histories error

I have exhausted my research into the histories error and how to bundle a prepopulated realm file. I have put a lot of time and investment into Realm, but if I can’t solve this seemingly simple problem I’m not sure I can continue using it.

The only solution that I have found to work is what I started with during development: use a single admin user across all app sessions and devices which does not seem viable in a production release.

Please help.


#2

Having the same issue. Did you find a solution?


#3

Nope. I’ve had no luck solving this.


#4

We populate realms from json files on first run. Works well for us. Also allows us to populate with different data depending on the user region or whatever.


#5

I’ve thought about doing this but what happens when the content in Realm is “newer” than the JSON files shipped with the client.

Is your workflow the following:

  1. Authenticate user
  2. Open synced global Realm (I assume you do this synchronously?)
  3. Write JSON data to Realm while it is downloading…?

Step 3 is what confuses me. If you’re writing data to Realm while it is syncing from the server, do you ever overwrite newer changes on the server unintentionally or does the server always win?

I’m probably over thinking this, but I’ve been struggling with this issue for too long.


#6

The writeCopyTo method copies the contents of the Realm file without the sync history. As an analogy, you could think of this operation as copying files on disk from out of a git repository without the git metadata, the data is intact, but you can no longer perform git operations (sync). A realm file produced with writeCopyTo can be opened locally, but cannot be opened with a sync configuration anymore.

What problem are you trying to solve by trying to prepackage a sync’d Realm? If you expect the data to change, you should probably just request a fresh copy using sync as normal without trying to prepackage the data. If you do not expect the data to change, then packaging a prepopulated Realm file for offline use is fine; no need to try to use sync. You can also do both with different Realm files if you want, one using sync and one not.


#7

The explanation of writeCopyTo makes sense; thank you for the clarification.

The problem that I am trying to solve is that I want my app to startup “instantly” with an initial catalog of data. The size of my compacted global Realm file is around 70 MB. However, on a fresh sync to a new device, Realm downloads almost 10 GB of data. I can only assume this is the change history of the catalog since the initial creation of my database? Because of this, I want to ship a pre-populated Realm file with my app so that when opened, the downloaded changeset is minimal thus keeping my initial app startup after a fresh install as optimal as possible.

The catalog today is about 50k records in size. Changes will ebb and flow in a 12-month period, but in general, the catalog will grow by 1500 - 2000 records per year and, at a guess, about 50% of the catalog receives updates of some kind in that same timespan. I chose Realm as I thought it would be the easiest way to aggregate and maintain my catalog server-side and automatically keep my user’s devices in sync.

Also, I am trying to follow the recommendations found here regarding how to choose between full and query-based sync and I have chosen full sync as a result which I assume is contributing to the 10 GB download size mentioned earlier.


#8

I’ve managed to solve the issue - will describe the solution in a couple of days


#9

Awesome! I look forward to seeing what you’ve done.


#10

After lots of investigation, I’ve came up to using two realms - offline and online.

My app always use offline one, and I manually transfer changes from online synced realm to the offline one.