Decentralized NoSQL DB on Arweave

WeaveDB is a decentralized Firestore on smart contracts, which brings web2-like smooth UX, complexity, and scalability to web3 dapps. Developers can build complex logic for full-stack dapps without writing a smart contract.

You would be thinking... is that even possible?

Well, we firmly believe this is the endgame for web3 mass adoption. And it's already here with all the beautiful tools and infrastructures in the Arweave ecosystem!

TL;DR

• WeaveDB is a new paradigm for dapp development bringing web2-like UX both to users and developers with Firestore-like NoSQL DB on smart contracts.

• Storage-based Consensus Paradigm of Arweave makes it all possible with Warp SmartWeave contracts and Bundlr instant finality network.

• WeaveDB Features: Cross-chain authentication, Auto-signing, JSON-like simple APIs, Data indexing, JSON-based functional programming over data validation and access control, Built-in scheduled cron jobs, REPL, Web console, gRPC node, Light client

• For developers, one-line instant contract deployment, building complex logic without writing a smart contract, building full-stack scalable dapps that weren’t possible before, using Javascript/Typescript, Rust, or Go for development

• For users, instant finality, no signing with a crypto wallet, no gas, web2-like UX and versatile features, biometric authentication, truly owning your own data

SCP - Storage-based Consensus Paradigm

The Arweave blockchain is decentralized permanent storage, which is immutable once the transactions are finalized. This immutable permanency allows computation to be deterministic and totally separate from storage. If initial states, a function, and a series of input values to execute the function sequentially with to evolve the initial states, are all stored permanently and immutably on Arweave, whoever calculates the final states should come to the same conclusion wherever they execute the computation. In other words, with immutable permanent storage, calculations can always be deterministic. This is what’s called Storage-based Consensus/Computation Paradigm (SCP).

What's more, when computation is taken off-chain, it virtually costs nothing and makes unlimited scalability possible. With the current blockchain paradigm, the two most significant bottlenecks for scalability are storage and computation since these are what consume gas and expensive. Arweave can remove these bottlenecks for scalability with the super cheap immutable permanent storage and the unlimited off-chain computation separate from the storage.

You can read more about SCP by @outprog_eth, founder of everPay.

• A Storage-based Computation Paradigm Enabled by Arweave

• Storage-based Consensus Paradigm

Warp - SmartWeave Contracts

SmartWeave was the first uniform specification to realize smart contracts with the idea of SCP. Then Redstone took it further to build a solid infrastructure for Arweave smart contracts and called it Warp. WeaveDB is a decentralized NoSQL database written as a Warp smart contract. Warp aims to let developers use any language to write smart contracts, such as JavaScript/TypeScript, Rust, and Go. They have a sequencer, a gateway indexer, a transaction explorer, and Warp SDK to bring smart contract computation off-chain to the client side. At the same time, they are building a decentralized network of nodes called DEN to remove computation from the client side.

You can learn more about Warp Smart Contracts at Warp Academy.

Bundlr - Instant Finality and Meta Transaction

Bundlr is a decentralized network on top of Arweave to make transactions simpler and faster.

3 key improvements Bundlr brings in

• Bundlr lets the transaction fees be paid in advance with tokens from different chains.

• Bundlr processes transactions instantly with a simple API and guarantees later finality.

• Bundlr makes the data instantly accessible via arweave.net.

The warp sequencer, which processes smart contract transactions, uses Bundler to make instant finality. Technically speaking, you wouldn't say it's finality for the data storage on the blockchain, but at the sequencer, it is instant finality for smart contract processing. It shares some analogies with how L2 rollups work with a sequencer and later finality on L1 (with security assumptions put aside).

Dapp users don't have to pay to make transactions with $AR token, but the protocol or the dapp developers can pay for them in advance, similar to how meta transactions remove the need for the native token from end users.

WeaveDB - Firestore on Smart Contracts

With all the god-blessing advantages the Arweave ecosystem brings in, would a decentralized Firestore to achieve web2-like UX be possible?

• Super cheap immutable permanent storage

• Unlimited off-chain computation

• Instant finality and data accessibility

• No gas fees for users (prepaid by dapps)

• Any programming language to write smart contracts

Of course, it's possible! We've already built everything a dev needs.

Cross-Chain Crypto Authentication

Since Arweave smart contracts have unlimited computation with any programming language, Arweave can be a hub of cross-chain authentication. We have integrated the cryptography from EVM-based accounts (secp256k1), DFINITY Internet Identity (ed25519), and Arweave (RSA-PSS) into the authentication mechanism.

Internet Identity brings biometric authentication on any device without needing browser extensions and mobile apps, which is ideal for web2-like UX.

Auto-Signing

With WeaveDB, making transactions doesn't require a signature using a crypto wallet. When signing into a dapp, users create and store a disposal key pair in a secure space in local indexedDB, link the disposal address to the original address in WeaveDB, and then transactions thereafter will be auto-signed by the disposal private key.

There is no need for a disposal key pair to auto-sign with Internet Identity; it already works like that.

Firestore-like APIs with Syntactic Sugar

Once upon a time, I personally wasn't fond of how Firebase SDK was building a query by chaining functions. It felt so redundant, verbose, and JS-dependent, so I created syntactic sugar called Firestore Sweet. It expresses all the queries as just JSON arrays and objects, making them much shorter and simpler. We are bringing the same spirit to WeaveDB, but WeaveDB can do most of the things Firestore can do and much more with cryptography.

And this JSON-like query syntax plays a crucial part in the powerful logic building explained in the later section.

Data Indexing

The biggest bottleneck for building a full-stack dapp is data indexing. It's just not feasible with regular blockchains. That's why many protocols are trying to solve indexing, such as The Graph and API3.

But we made data indexing built-in. WeaveDB auto-indexes the stored data with single fields just like Firestore does, and you can add multi-field indexes just like you would with Firestore.

Data Schemas & Access Control Rules

A decentralized database is very different from a centralized counterpart, even if they seem to function similarly. With a centralized DB, there usually is only one gateway authentication for a few authorized users with the editor role. You don't have to define detailed rules on data and access as long as you can ensure admin users won't mess up the data with programming malfunctions.

On the other hand, a decentralized DB can be accessed permissionlessly, and anyone can attempt to write any malformed data to it. So you need to set up total control over data schemes and access control rules.

To solve this issue with a simple solution, we invented a JSON-based functional programming language called FPJSON. It's just a JSON array, but you can write complex logic in functional programming. It is language-agnostic, simple yet powerful, with no overhead. This enables complex algorithms to be stored as a smart contract state because everything is just a JSON data object. It's packed with about 250 Ramda functions and also combined with JSONLogic to build hyper-complex logic over data validation and access control.

The FPJSON website comes with a quick learning system for developers.

Scheduled Cron Jobs

The powerful combination of FPJSON and the JSON-like simple query APIs with the deterministic nature of Arweave smart contracts makes it possible to run scheduled cron jobs without periodically sending a transaction to execute them.

What!? How is it even possible?

First, the DB queries and any logic to execute can be expressed as a JSON object and stored as a smart contract state.

Secondly, computation with SCP is always deterministic, so once cron jobs are defined and stored, WeaveDB can figure out where to insert cron calculations and auto-execute them up to the current timestamp before executing the latest query being requested.

It's basically like Lisp (but just JSON), and you can define Macros that auto-execute periodically on smart contracts without transactions.

For instance, our social bookmarking demo dapp uses a cron job to auto-calculate which articles are trending in the past 2 weeks. But it doesn’t require any transactions. Only the cron job was defined once and it continues to periodically rank articles forever.

REPL & Web Console

WeaveDB comes with command line REPL and Web Console for local development. With the web console, developers can build complex logic for dapps without writing a smart contract. All you have to do is to configure the DB instance on the web UI.

WeaveDB is smart contract 2.0 and a new paradigm for dapp development.

gRPC Node & Light Client

To provide web2-like UX, DB queries should be processed extremely fast. SmartWeave brings computation to the client side, but this means each client needs to build the entire cache of the contract, and computation becomes a burden to the client side.

Warp is working on a delegated evaluation network (DEN) to solve this problem, but we are building yet another computation layer to achieve better performance for dapp users.

You can set up a gRPC node and connect to it with WeaveDB Light Client to remove computation and cache from the client side. This reduces the read query execution time to under 1 second and the write query execution time to under 3 seconds.

We are also planning to implement real-time streaming communication using gRPC node.

Pros for Users and Devs

For developers

1. You can use a near real-time NoSQL database just like Firestore but all on smart contracts. This has never been possible with any other blockchain but Arweave.

2. Contract deployment is one simple command and instant, and the DB instance can evolve (be upgraded) in the same way.

3. You don’t even have to write smart contracts for most logic but just configure the DB on a web UI with JSON-based simple logic building.

4. Building dapps with web2-like data complexities and scalability is possible.

5. You can use Javascript/Typescript, Rust, or Go for further development of Warp smart contracts

For Users

1. The integration with Dfinity Internet Identity enables biometric authentication on any device and obsolete private key management. You can sign in with MetaMask and ArConnect as well, and it creates a disposal key pair for auto-signing transactions during a session.

2. With Arweave and Bundlr, dapps can cover your transaction gas, so your actions are free.

3. Thanks to Bundlr, transactions are processed instantly and guaranteed later finality.

4. Dapps will be very responsive and have versatile features both on the web and mobile devices, just like web2 apps.

5. Only you will own and control your own data, and dapps will be permissionless, trustless, and censorship-resistant if configured so.

What could be Built with WeaveDB

We believe WeaveDB will play a massive role in bringing web2 to web3 adoption since it benefits both the developers and end users to a great extent.

Things that seemed impossible to build as a full-stack dapp before are starting to become possibilities with WeaveDB.

• Decentralized Blogging

• Decentralized Wikipedia

• Decentralized Twitter

• Decentralized Forum

• Decentralized GitHub

• Decentralized Reddit

You name it!

Resources

Everything you need to build and launch dapps is already here. But please note that WeaveDB is still in its infancy and things need lots of improvements.

GitHub Monorepo
https://github.com/weavedb/weavedb

Website
https://weavedb.dev

Docs
https://docs.weavedb.dev

FPJSON
https://fpjson.asteroid.ac/

Sourceable Smart Contract
https://sonar.warp.cc/#/app/source/UR6tFLrjEdiujqQBTGEnyNYB7YUv3IkIJvNIiCiLCic

Start Building

You can start building a dapp by following the todo dapp tutorial. Everything you need is there!
https://docs.weavedb.dev/docs/examples/todos

or Try the Demo Dapps

The Wall 2 on Internet Computer
https://xdiw4-hiaaa-aaaai-qnnbq-cai.ic0.app
(The entire dapp is WeaveDB.)

Social Bookmarking with Mirror.xyz scan
https://asteroid.ac
(WeaveDB is partially used for the bookmarking feature.)

Gitcoin GR15

We may be a bit late for the party, but WeaveDB has just been approved for Gitcoin GR15!

Any contribution will be much appreciated! Let’s make web3 mass adoption happen!
https://gitcoin.co/grants/7716/weavedb-decentralized-nosql-database