Assume you have 1 000 000 records to synchronize. If you want to do this one time, with one user - easy.
Let’s make our case more complicated. In our scenario, we will have 100 users, we will trace changes and send only differences, and split data to make sure, that only data that the user is able to see will be synchronized to the device.
In the simplest scenario, you can synchronize the whole table to a device and limit visible records on a device. It sounds easy but it’s not. For example, if you have a document table with 1million records and another table where you will store relations between users and documents it will give you at least 2 million records to synchronize.
In our sample case, I will be using MySQL 8 as a master database and .NET C# client.
MySQL 8 master database schema:
Used resources:
MySQL 8: phpMyAdmin (https://one-million-demo.ampliapps.com/phpmyadmin/, user: demo, password: zPWvb7aSFyZJWgeF).
Synchronization URL: https://one-million-demo.ampliapps.com/sync/API3
Demo client: .NET C# sample application and source
Remember, this is a public demo and other users can work on it simultaneously.
Test scenario users:
[subscriber id] - [documents attached]
u90 - 100
u89 - 10 000
u88 - 100 000
u87 - 1 000 000
Subscriber (u87) with 1 million documents attached (without any filters):
- Frist (full) synchronization time, 1 million records downloaded to a client: 3 minutes and 5,88 seconds,
- Second and next synchronizations (differential, only changes): more or less 30 seconds.
Subscriber (u89) with 10 000 documents attached (without any filters):
- Frist (full) synchronization time, 10 000 records downloaded to a client: 0 minutes and 27,02 seconds,
- Second and next synchronizations (differential, only changes): more or less 4 seconds.
Subscriber (u88) with 100 000 documents attached (without any filters):
- Frist (full) synchronization time, 100 000 records downloaded to a client: 0 minutes and 16,88 seconds,
- Second and next synchronizations (differential, only changes): more or less 2 seconds.
Subscriber (u90) with 100 documents attached (without any filters):
- Frist (full) synchronization time, 100 records downloaded to a client: 0 minutes and 16,54 seconds,
- Second and next synchronizations (differential, only changes): more or less 2 seconds.
As you can see the amount of records has a huge impact on synchronization times. From my experience, there are not too many situations when you need all the records on a device. When you planing your solution it is very important to design the synchronization filter to limit the amount of data synchronizing between devices. Imagin your system is running for 10 years. The amount of data will be huge. With synchronization filters, you can limit data that is flying over (based on date or any other criteria).
Filters are also useful when you want to limit data available for each device. You can design filters in a way where each device will get data that only belongs to a user who uses a device.
Documentation how you can create filters you can find here: https://projects.ampliapps.com/projects/ampli-sync/wiki#Data-filtering
In this demo I used two filters to limit data that is sent to devices:
Use the credentials provided above to log in.