El Pepe Notes
Data Fetching & Persistence

Photo by K. Mitch Hodge on Unsplash

Data Fetching & Persistence

Eko Julianto Salim's photo
Eko Julianto Salim
·

6 min read

Table of contents

Fetching and Storing Data are common tasks often encountered in many different types of programs and applications. Let's take a look at a case-study on how an application (Ajaib DeX) utilizes existing data fetching and persistence methods.

Ajaib DeX

image.png

Ajaib DeX is a React (Native) mobile application that serves as a 'hub' for decentralized finance with functionalities for holding a 'crypto' wallet, sending transactions and trading through DeFi platforms.

To enable data fetching & persistence, Ajaib DeX utilizes 3 different libraries: SWR, @react-native-async-storage/async-storage, and expo-secure-store.

SWR

The name “SWR” is derived from stale-while-revalidate, a HTTP cache invalidation strategy popularized by HTTP RFC 5861. SWR is a strategy to first return the data from cache (stale), then send the fetch request (revalidate), and finally come with the up-to-date data.

SWR is a React library developed by Vercel that provides hooks for data fetching. As a library, it provides the following features:

  • Fast, lightweight and reusable data fetching
  • Built-in cache and request deduplication
  • Real-time experience
  • Transport and protocol agnostic
  • Fast page navigation
  • Polling on interval
  • Data dependency
  • Revalidation on focus
  • Revalidation on network recovery
  • Local mutation (Optimistic UI)
  • Smart error retry
  • Pagination and scroll position recovery

AsyncStorage and SecureStore

Since Ajaib DeX is a financial application that needs to be secure for transactional use, we utilizes 2 different types of persistence method, namely a simple, fast and unencrypted AsyncStorage to store preferences and miscellaneous data and a secure, relatively slow and costly SecureStorage to store wallet and transactional data. Here are more details about the methods used:

AsyncStorage

@react-native-async-storage/async-storage provides asynchronous, unencrypted, persistent, key-value storage system for React Native.

SecureStore

expo-secure-store provides a way to encrypt and securely store key–value pairs locally on the device. Different storage services/methods are used for different device types:

  • iOS: Values are stored using the keychain services as kSecClassGenericPassword. iOS has the additional option of being able to set the value's kSecAttrAccessible attribute, which controls when the value is available to be fetched.

  • Android: Values are stored in SharedPreferences, encrypted with Android's Keystore system.

Implementation

Fetching

In addition to SWR, we use a wrapper library @nandorojo/swr-react-native to integrate SWR with React Native. With this library, we are able to automatically refetch data on screen focus. Here are the code and abstractions we used to define the urls and hooks used to communicate with the API.

import useSWRNative from "@nandorojo/swr-react-native";
import "react-native";

export const API_BASE_URL = "https://staging.ajaib.me";

export const fetcher = (url: string) => fetch(url, { mode: "cors" }).then((r) => r.json());

export const useConditionalAjaibAPI = (path: string) => {
  return useSWRNative(path ? `${API_BASE_URL}/${path}` : null, fetcher);
};

export const useAjaibAPI = (path: string) => {
  return useSWRNative(`${API_BASE_URL}/${path}`, fetcher);
};

export const useWalletAPI = (address: string, method: string) => {
  return useAjaibAPI(`wallet/${address}/${method}`);
};

export const useConditionalWalletAPI = (address: string, method: string) => {
  return useConditionalAjaibAPI(address ? `wallet/${address}/${method}` : null);
};

export const walletAPIURL = (address: string, method: string) => {
  return `${API_BASE_URL}/wallet/${address}/${method}`;
};

export const infoAPIURL = (method: string) => {
  return `${API_BASE_URL}/info/${method}`;
};

export const fetchAjaibHistory = (address: string) => {
  return fetcher(walletAPIURL(address, "history"));
};

export const fetchAjaibAmount = (address: string) => {
  return fetcher(walletAPIURL(address, "amount"));
};

export const fetchAjaibDailyPrice = () => {
  return fetcher(infoAPIURL("price"));
};

As can be seen above, there are several different varations for different url and endpoints. In addition, we provided a custom fetcher in order to use the native fetch provided by React Native. The usage of the hooks useConditionalWalletAPI is demonstrated below:

const PortfolioHeader = () => {
  const wallet = useWallet();
  const { data } = useConditionalWalletAPI(wallet?.address, "returns");
  const [holdings, setHoldings] = useState<number>(-1);
  const getTotalInvestment = useTotalInvestment();
  const isFocused = useIsFocused();
  useEffect(() => {
    const fetchInvestments = async () => {
      await getTotalInvestment().then((holdings) => {
        setHoldings(holdings);
      });
    };
    if (isFocused) {
      void fetchInvestments();
    } else setHoldings(-1);
  }, [isFocused]);
  const returns = data?.data?.returns ?? "-";
  const percentage = holdings > 0 ? (returns / holdings).toFixed(2) : 0;
  return (
    <BlueHeaderBox percentage={0.3} space={3}>
      <Text color="white" bold fontSize="2xl">
        PORTFOLIO
      </Text>
      <VStack px={7} space={2} alignSelf="flex-start" width="full">
        <VStack>
          <Text bold color="blue.200" fontSize="sm">
            TOTAL INVESTMENTS
          </Text>
          <Text bold color="white" fontSize="2xl">
            {holdings === -1 ? <Spinner color="white" /> : CurrencyFormat(holdings)}
          </Text>
        </VStack>
        <HStack justifyContent="space-between">
          <VStack>
            <Text bold color="blue.200" fontSize="sm">
              PROFIT
            </Text>
            <Text color="white">
              {returns === "-" ? <Spinner color="white" /> : CurrencyFormat(returns)}
            </Text>
          </VStack>
          <VStack>
            <Text bold color="blue.200" fontSize="sm">
              PROFIT PERCENTAGE
            </Text>
            <Text color={percentage > 0 ? "green.300" : percentage === 0 ? "white" : "red.600"}>
              {returns === "-" || holdings === -1 ? <Spinner color="white" /> : `${percentage}%`}
            </Text>
          </VStack>
        </HStack>
      </VStack>
    </BlueHeaderBox>
  );
};

As can be seen above, the data from the hooks can be instantly used (automatically JSON-parsed) and there is no need for explicit focus hooks. Without using the wrapper library, we would have to resort to using the focus hook useFocusEffect as shown in the example below(taken from this docs):

import { useFocusEffect } from '@react-navigation/native';

function Profile({ userId }) {
  const [user, setUser] = React.useState(null);

  useFocusEffect(
    React.useCallback(() => {
      const unsubscribe = API.subscribe(userId, user => setUser(user));

      return () => unsubscribe();
    }, [userId])
  );

  return <ProfileContent user={user} />;
}

Persistence

Persistence in Ajaib DeX is mainly used to store wallet data securely. However, for developmental purposes and to ensure compatibility for the web version, wallet data can also be stored using unencrypted storage (though the wallet itself is encrypted by a user password). Therefore, persistence methods in Ajaib DeX first utilize SecureStore then fallbacks to AsyncStorage if SecureStore is not available. Here are the code used to define store and load with fallbacks:

import * as SecureStore from "expo-secure-store";
import AsyncStorage from "@react-native-async-storage/async-storage";

const WALLET_RECORD_KEY = "WALLET-ADRESSES";

// alias for load & store (AsyncStorage)
const store = AsyncStorage.setItem;
const load = AsyncStorage.getItem;

// custom secure store & load function
// default to SecureStore and fallback to AsyncStorage
const secureStore = async (key, value) => {
  if (await SecureStore.isAvailableAsync()) {
    await SecureStore.setItemAsync(key, value);
  } else {
    await store(key, value);
  }
};

const secureLoad = async (key) => {
  if (await SecureStore.isAvailableAsync()) {
    return await SecureStore.getItemAsync(key);
  } else {
    return load(key);
  }
};

// records
type WalletRecord = Record<string, string>;
export type WatchlistRecord = string[];

const storeRecord = async (value: WalletRecord) => {
  return await store(WALLET_RECORD_KEY, JSON.stringify(value));
};

const loadRecord = async (): Promise<WalletRecord> => {
  const storedRecord = await load(WALLET_RECORD_KEY);
  if (!storedRecord) return {}; // default to empty
  return JSON.parse(storedRecord) as WalletRecord;
};

const getStoreKeyFromWalletAddress = (address: string): string => `wallet-${address}`;

const addNewAddresstoRecord = async (address: string, name = "DEFAULT"): Promise<WalletRecord> => {
  const walletRecord = await loadRecord();
  walletRecord[name] = address;
  await storeRecord(walletRecord);
  return walletRecord;
};

export const storeWallet = async (wallet: ethers.Wallet, password: string) => {
  const encryptedJSON = await wallet.encrypt(password, ENCRYPTION_OPTION);
  await secureStore(getStoreKeyFromWalletAddress(wallet.address), encryptedJSON);
};

export const storeDefaultWallet = async (wallet: ethers.Wallet, password: string) => {
  await storeWallet(wallet, password);
  await addNewAddresstoRecord(wallet.address);
};

const loadWallet = async (storeKey: string, password: string): Promise<ethers.Wallet> => {
  const storeValue = await secureLoad(storeKey);

  if (!storeValue) throw new Error("No such wallet was found!");

  return await ethers.Wallet.fromEncryptedJson(storeValue, password);
};

export const loadDefaultWallet = async (password: string): Promise<ethers.Wallet> => {
  const record = await loadRecord();
  const address = record["DEFAULT"] ?? Object.values(record)[0]!;
  return await loadWallet(getStoreKeyFromWalletAddress(address), password);
};

export const isDefaultWalletExist = async () => {
  const record = await loadRecord();
  return Object.keys(record).length > 0;
};