Want more from your frontend framework! Re-thinking web dev experience
An extended transcript of my talk at DevFest Norway 2020 (slides here).
Do you also love creating useful (web)apps and get easily frustrated by any friction in the development process? I will compare Redux + REST with a full-stack, component-centric solution based on a graph API (think GraphQL) that I came to love. You might not be able to use the same framework - Fulcro - but you can still look for similar, more developer-friendly solutions that implement some of the same ideas and provide some of the same functionality. We will discuss REST vs. Graph APIs, networking, error handling, and more. (You should have an idea about React, Redux, and GraphQL to gain most out of this.)
Introduction
I want to talk to you about the pain of web development. Especially related to getting data from a backend displayed on the screen and to understanding the code of a UI you haven’t written.
I’ve worked on three React apps where I had to make what felt like an insane amount of changes across multiple files to display a new kind of data. Until I said “enough, there must be a better way!” And there is, at least one.
The message
The key point of this talk is this:
Don’t settle for the mainstream, for an “industrial best practice”, for what everyone else is doing. Keep searching for better tools, less friction in your development flow, higher productivity, for better developer experience, perhaps better languages, for what is best for your case.
Redux & REST vs. Fulcro & Graph API
I am going to compare and contrast one widespread, mainstream solution - Redux & REST - with the innovative approach provided by Fulcro & Graph API.
What is Fulcro (and its sister library Pathom)?
Full-stack, batteries included ClojureScript framework built on top of React
Based on a few simple ideas…
… and adaptability as a key feature
The ideas (explained below):
Graph API (∋ GraphQL)
Client declares its data needs
Co-location & fat components
(Normalized data)
Batteries included:
Status tracking (loading / failed)
Load sync/async/on-demand
UISM, routing, forms, & more
The idea
Graph API (∋ GraphQL)
The first idea is to use a Graph API (provided by Pathom) instead of REST. (The talk Data Navigation with Pathom 3 by Wilker Lucio is a great explanation of the problem with REST and multiple "clients" with varying data needs (such as UI components) and why an attribute-centric approach - such as implemented by Pathom - is a better solution. You will see Pathom in action and learn about some of its super-powers. All of this in just 45 minutes.)
While with REST you have multiple endpoints, each pushing you all the data it has, a Graph API - the best known example being GraphQL - provides you with a single endpoint that can serve all (or most) of your data needs. And it is the client who decides what data it gets by explicitly querying for it.
It is similar to SQL - you send a query to your database and get data back. But while in the case of SQL the data is in the form of a table, in the case of a Graph API it is a graph, most typically a tree. And that’s a perfect match for the UI because it is a tree of components. Imagine that you want to display a list of all female players, and for each player you want her name and home address, and for each home address you only want the city. This is a tree in the UI and in data.
A single endpoint means less configuration, UI-shaped data means no need for crazy client-side data transformations.
Co-location & fat components
The second idea is co-location.
Let’s take a step back: Why do we create web applications? To enable users to interact with our systems. So they are, in essence, user interfaces. And the most important things in UIs are UI components.
To understand what a component does and why it exists, I don’t want to search through four different files. I want everything right next to the body of the component, or at most a click away*. And that is the idea of co-location: include all the key information with the component. What data it needs. Which route is it under. What is the ID attribute of the associated data entity. How it wants to handle errors.
Actually, Fulcro goes even a step further. Its (stateful) UI components are expected to map to and to describe your (UI-centric view of) data entities, such as Person, PersonList, Address. So they are ultimate "source of truth" in your application.
*) Literally one click away - Fulcro is also optimized for developer experience, in particular for navigability. The important pieces look like standard functions/references so your editor’s built-in navigation (control/command-click) works on them.
Normalized data
The last key idea is to store the data at the client not as it comes, as a graph, but normalized, deduplicated, in a map from entity ID to entity properties (whose values can be references to other entities).
Thus, if I have two players living at the same home address and I update it, both will show the new, updated value.
This prevents a whole class of defects due to data inconsistency and makes updating any data trivial.
Code time!
Let’s see how that looks in practice. Imagine you have a webshop and have been asked to show a list of "hot deals" - and the data should only be loaded when the component is displayed.
(I will only show the most important parts of the code, it is not complete.)
Solution 1: Redux + REST
This is a standard Redux + REST solution based on what I have seen in production projects.
We start with the UI itself.
export default HotDeals = Redux.connect(
(state) => _.pick(state, ["deals", "dealsError", "dealsLoading"]), (2)
{ loadHotDeals } // defined below
)(function HotDeals({deals, dealsError, dealsLoading, loadHotDeals}){ (1)
React.useEffect(() => loadHotDeals(), []) // on mount
if (!deals || dealsLoading) return <p>Loading....</p>
if (dealsError) return <p>Something went wrong</p>
return <ul>{deals.map(deal => <Deal {...deal}/>)}</ul>
})| 1 | We define our React component that asks for the data to be loaded and displays "Loading…", an error, or the list of deals |
| 2 | We connect the component to the Redux-managed client state, asking it to pluck three properties from the state and send them as props to the component |
Now onto our next file, actions.js. (It is an overkill to split this tiny example into multiple files but it actually makes sense for bigger applications and it is what is used in practice.)
Here we define the action creator producing the action called "LOAD_HOT_DEALS", which will eventually contain the hot deals data, and that we dispatch from our component.
export function loadHotDeals() {
return {
type: "LOAD_HOT_DEALS",
promise: fetchHotDeals()
}
}Our next file, backend-client.js, encapsulates the low-level details of the retrieval of the data, i.e. which URL and transforming the response text into JavaScript data structures.
export function fetchHotDeals() {
return fetch('https://backend/hot-deals')
.then(res => res.json())
}Our fourth and final file contains the reducer, i.e. the function that processes actions to incorporate new and changed data into the global state. (This separation decouples the action and data processing logic decoupled from the components and makes it possible to test it much more easily.)
I cheat a little and use the redux-pack that makes it somewhat less verbose. I couldn’t bring myself to write it all manually (as we have done in the actual projects) even for a presentation. Pack provides me with the function handle that will trigger three out of the four possible events - start when we dispatch the action, finish when it is done, failure if it failed, or success if it succeeded.
import { handle } from 'redux-pack'; // 1 Promise action -> 4 events
export function reducer(state = myInitialState, action) {
const { type, payload } = action;
switch (type) {
case "LOAD_HOT_DEALS":
// The UI expects deals, dealsLoading, dealsError:
return handle(state, action, {
start: prevState => ({
...prevState,
dealsLoading: true, dealsError: null (1)
}),
finish: prevState =>
({ ...prevState, dealsLoading: false }), (2)
failure: prevState =>
({ ...prevState, dealsError: payload }), (3)
success: prevState =>
({ ...prevState, deals: payload }) (4)
});
// ... repeat ∀ data sources ...
}
}| 1 | Started, set dealsLoading for the component’s property to true |
| 2 | Not loading anymore so reset it |
| 3 | Upon failure, we store dealsError for the component |
| 4 | Upon success, we store the actual deals for the component |
The backend is trivial - the business function that somehow produces the data and little bit of plumbing to expose it at particular URL as JSON (which must match the backend-client.js).
// ################################################### BACKEND
// BACKEND - BUSINESS: webshop.js
async function hotDeals(env) { return ...; }
// BACKEND - PLUMBING: controller.js
router.get('/hot-deals', async(req, res) =>
res.json(await webshop.hotDeals(req.env)));Solution 2: Fulcro + Graph API
Let’s see how it looks with Fulcro and Graph API. See the frontend code below - and notice how much shorter it is.
First we defsc - define the stateful component - HotDeals, a React component that mirrors the one we saw above. It will get the self-reference this and props, which will include deals.
There is one major difference however - aside of the body itself, we also provide additional metadata, especially the query: :query [{:deals (fcomp/get-query Deal)} …]. It means "I want the deals and for each deal whatever the Deal component needs." There are a few observations to make: 1) nobody but the component itself needs to know what data in wants; this information is not spread across multiple places as in Redux (i.e. Redux.connect and the reducer); 2) the queries do compose - HotDeals includes the query of its child Deal (without needing to know anything about its details) - so that the query of the root element will include all queries for its descendants and will thus produce data for the whole application.
See the callouts below the code for details.
;; Syntax: [1, 2, ...] = "array", {:key "value", ...} = map, (something ...) =
;; invoke something (a function, ...)
;; (Here, all Fulcro library function calls start with f, as in `fhooks/...`.)
(defsc HotDeals [this props]
{:query [{:deals (fcomp/get-query Deal)} [ffetch/marker-table :deals-marker]] ; (1)
:use-hooks? true}
(fhooks/use-effect
(fn [] (ffetch/load! ; (2)
this :deals Deal
{:marker :deals-marker})) ; (3)
[])
(let [marker (get props [ffetch/marker-table :deals-marker])]
(cond
(ffetch/loading? marker) (p "Loading...") ; (4)
(ffetch/failed? marker) (p "Something went wrong...") ; (5)
:else
(ul (map (fcomp/factory Deal) (get props :deals)))))) ; (6) (7)| 1 | The query declares what data the component wants; it asks for deals, which we
get from the props at to render the list |
| 2 | As before, we ask for the data to be loaded - but thanks to the uniformity of the Graph API
and to having declared query on the component, we can use the generic, Fulcro-provided
load! function instead of writing our own load/fetchHotDeals
(telling it "load the deals and for each whatever Deal wants") |
| 3 | …and we also ask it to plug into the framework-provided loading/result tracking
and give us the "status marker" called :deals-marker so that we can question it later |
| 4 | We leverage the marker to check whether the loading is in progress |
| 5 | We leverage the marker to check whether the loading has failed |
| 6 | As before, we display a list of the individual deals |
| 7 | Note: the same :deals is referred to in 1, 2, and 6 |
Highlights: 1) declarative data needs; 2) built-in load!; 3) built-in tracking of loading/failed.
The backend is similarly simple as in solution 1, only this time we do not expose the data as a REST endpoint but as a graph API resolver.
;; BACKEND - BUSINESS: webshop.clj
(defn hot-deals [env] ...)
;; BACKEND - PLUMBING: graph-api.clj
(pc/defresolver hot-deals [env _]
{::pc/input #{}
::pc/output [{:deals [:deal/id :deal/title ...]}]} ; (1)
{:deals (hot-deals env)}) ; (2)
;; NOTE: The output key :deals, (4) matches the key frontend queries for
;; In config:
... (pc/connect-plugin {::pc/register [hot-deals ...]}) ...| 1 | We declare that this resolver can produce deals and for each deal an id, title, etc. (This is optional but useful so that we can explore, browse and play with the data in developer tooling) |
| 2 | We return the promised data. |
I don’t want to…
In the first, Redux and REST solution, we have seen a number of things I don’t want to:
I don’t want to have to coordinate a change across 2, 3, 4 different files and places.
I don’t want to manage failure tracking manually (the dealsError prop).
I don’t want to manage loading status manually (the dealsLoading prop).
And I certainly don’t want to do this again and again, for each single endpoint.
I don’t want to write data fetching for each endpoint (the fetchHotDeals function).
I know I am always getting back JSON and if there is any error, I want the UI component to decide how that should be handled.
I don’t want to coordinate loading data from a number of (possibly inter-dependent) endpoints. Graph API can figure this out for me and just give me the data I want, no matter which sources they came and how they depend on each other.
I don’t want to manually maintain the consistency of my, possibly duplicated data.
I want
The second approach, using Fulcro & Graph API, offers a number of things that I want.
I want the minimal friction when getting (new) data from a backend to the UI. In Fulcro I just needed to define the resolver that exposes the data in the backend and then just query for it and use it in my component. You cannot ask for less! (Well, you can. With Fulcro Rapid Application Development you get the resolvers generated for you.)
I want built-in request status (loading/error) tracking.
I want built-in built-in data fetching and caching. With the uniform Graph API backend and the composable data needs declaration - queries - in the frontend, I can use a generic, framework provided load! function.
This is a big one - I want the ability to easily switch loading modes - load all data at once, when the application starts vs. start loading all data at once but only wait for the essential data before displaying the UI, while displaying the secondary data when it arrives vs. loading data on-demand (on click, when a component mounts, …). I don’t want to wait for the slowest data source before displaying anything useful to the users.
I want a framework that:
... is full-stack and integrated, i.e. where the backend and frontend have been made to work together and where there is minimal friction and boilerplate in getting data from the one to the other. A framework that provides a complete, well-integrated solution to all common needs of non-trivial applications.
I want a framework that has "batteries included" (see below).
I want a framework that is adaptable, where the maintainers are not (presumably) omniscient, making all the decisions for me. I have been burnt repeatedly by running into the walls created by such decisions that were contrary to the needs of a particular project. Fulcro provides "hooks" that allow me to extend or override its key behaviors - and a lot of deep design thinking went into that - so that I can truly adapt it to the unique needs of my project, as long as I am aligned with its overall philosophy.
I want a framework that provides:
A graph API so that I can simply get the data I need, in the form that suits my UI, and so that only the place - the component - that uses the data need to know about what data it needs. (Though, obviously, the backend must be able to provide it.) I want to be able to declare data needs and compose them into the complete query.
Co-location & fat components so that everything important to understand (and create) a component is contained within the component.
Normalized data so that I don’t need to worry about data duplication and data out-of-sync problems.
A framework that has batteries included:
Error handling and tacking, “loading…” status
💪 Load data synchronously / asynchronously / on-demand
😍 And more goodies that Fulcro offers such as UI State Machines (indispensable when you have any more complicated interaction flow), routing (which SPA doesn’t need one?), forms support, & more
Stuck in the JS land?
Not everybody is as lucky as I am and gets to work with ClojureScript and Fulcro. If you are stuck with JavaScript, have a look at Facebook Relay and GraphQL. Obviously I think they are inferior to the technologies I use but they are still a great improvement over Redux and REST.
Homework
Read/watch to learn about why/when a Graph API makes sense (compared to REST) - the aforementioned talk Data Navigation with Pathom 3 is great for that
Have a look at Relay (or perhaps Apollo?)
If interested in the technologies I have used:
Watch Why Build Solutions with Fulcro - Tom Toor
Beware: Don’t try to learn Clojure, ClojureScript, and Fulcro all at once! Start simple.
Read Gene Kim’s Love Letter To Clojure (Part 1)
Bonus: SWR
SWR is a neat hooks-based library for no-boilerplate data fetching and caching, with built-in tracking of the "loading" and "failed" status, with a support for a universal data fetching function (no more fetchHotDeals). So if you are stuck with a REST API, it might be best. However it is very non-functional, with potentially remote calls spread all throughout your code.