The vast majority of computer systems today are distributed. Sharing data in a such a system is problematic, because, as proved by the CAP Theorem, it cannot be at the same time available and strongly consistent. For decades, research and industry alike had been searching for a solution to this quandary.
In 2009–2011, Marc Shapiro and Nuno Preguiça developed a novel approach, called Conflict-Free Replicated Data Types (CRDTs), whose properties mathematically ensure both availability and convergence. CRDTs were quickly taken up by both research and industry, for instance being used in the Riak, Redis and CosmosDB databases. The recent research of our team members has focused on making CRDTs practical in highly-available applications, on guaranteeing application-level correctness properties (invariants), and on building an open-source CRDT ecosystem (the AntidoteDB geo-distributed database and associated tools) following a Just-Right Consistency approach.
The Concordant start-up plan represents a new, ambitious step in the development of highly-available, highly-scalable distributed systems, enabling sharing between massive numbers of edge and mobile devices, and extending CRDTs and Just-Right Consistency seamlessly across the whole cloud-to-far-edge spectrum. It leverages the experience and contributions of Shapiro et al. and builds upon the open-source artefacts they developed.
Despite improved network infrastructure, users still experience expensive roaming charges, high latencies, and extended periods of disconnection; furthermore a user or group may decide to “branch” and work independently for a while. This poses important challenges for the development of mobile applications, as users expect their mobile applications to work at any time and location. According to the offline-first approach, mobile applications should be designed to mask disconnection, providing users with a uniform experience independent of connectivity. This constitutes an improvement over the more traditional approach of many BaaS platforms, where mobile applications depend heavily on the cloud for their operation.
We go further, and advocate an edge-first approach: design mobile applications for very fast response, by having data stored locally on the mobile device and by collaborating directly between nearby clients, at all times (whether online or offline). Consider for instance supporting the field agents of a utility company: the edge-first design enables its users to share information, even when in locations with poor network connectivity, such as basements. Another example: In a mobile game application, fast interaction among nearby users is key to improved interactivity. This approach would also support disaster teams operating their own ad-hoc infrastructure when the public telco system is out of service.
Although highly desirable, the edge-first approach would be very hard for application developers themselves. They would have to replicate the right data on each user device, and propagate updates between replicas. They would have to ensure that propagation does not overwrite any local change, but instead merges all changes. They would have to implement direct peer-to-peer sharing connections between nearby devices. They would have to worry about keeping replicas consistent, and about concurrent updates violating business rules undetectably. Instead, they can outsource these difficult issues to the Concordant platform.