Moving tech forward with Gomix, Express, and Google Spreadsheets

I'm a frontend and PHP developer and a left-leaning moderate. I don't write Node and I don't get involved in political causes. At least, not until now. This is the brief story of how I was inspired to create Tech Forward and how I learned Gomix and Express to do it.

The backstory (skip if you just want to read about the tech)

I've spent the last five to ten years trying to make small changes for the good of the world by working in the relationships I already have, in person and online, to help White Americans become more engaged and interested in working toward justice. A little bit at a time, over coffee or surprisingly decent Facebook comment threads.

On November 9th, my wife turned to me and said: "Matt, it's time for you to stop trying to change individual people on Facebook and go do something real." Ouch. But she was right.

Right around that time DeRay Mckesson put out a call to programmers who wanted to help work for social change. I responded, as did quite a few others, and I met DeRay and Sam and Aditi and a few other incredible individuals really making a difference. Over the span of a few weeks I had the chance to work on The Resistance Manual and a few other great projects.

During this time I've had no less than a dozen friends in tech ask me, "How can I as a technologist contribute to social progress?" I wanted to make that question as easy to answer as possible, and I knew there are far more projects out there than just those we were working on at StayWoke. So I decided to catalog them all in one space.

The tech stack

The first version of the site was a static site, hosted using GitHub pages, pulling its data using JavaScript from locally-hosted JSON files. The idea here was to make it easy for folks to contribute: make a GitHub pull request updating the JSON and we'll handle the rest.

The problem is, JSON isn't that user-friendly, and pull requests aren't either. I wanted to keep the same spirit of GitHub pages–simple, easy to spin up, editable by anyone–but on a dynamic server. Turns out, the answer is Gomix (formerly Hyperdev).

Gomix is a platform that makes it absurdly easy to spin up a new app (static HTML or Node) and see it online instantly. You can also invite your friends to collaborate, and the moment you make a change in the editor, your site updates. So, at this point I'm using Gomix and Node, and Express is an easy pick.

I strongly considered using Firebase for data storage, but the Gomix team linked me to this Gomix site using Google Spreadsheets as the backing data source and I really wanted to try it out.

So we've now settled: I'll take my old HTML and JavaScript, but instead of the JavaScript loading its data from JSON files, I'll run an Express app on Gomix pulling the data from Google Spreadsheets and output its data in a JSON format. No big deal.

Starting the project on Gomix

Gomix treats your code as a shareable document which can be collaborated on and "remixed," or copied into a new project–like GitHub's forking, but with no ties to the original app. Hit "remix" on any public Gomix project and it'll copy all its code into a new project that you own with a randomly generated name.

So the first thing I did was "remix" that data dashboard app. Why start from scratch, especially as someone who's literally never written Node code in my life? The code for accessing Google Spreadsheets looks like this:

const GoogleSpreadsheets = require('google-spreadsheets')

GoogleSpreadsheets({
  key: 'google spreadsheet id here'
}, function(err, spreadsheet) {
  spreadsheet.worksheets[0].cells({
    range: 'R1C1:R20C9'
  }, function(err, result) {
    // result is the entire sheet within the provided range
  })
})

And, if you've never worked with Express before, you teach the server how to run it using the start script key in package.json:

{
  ...,
  "scripts": {
    "start": "node server.js"
  },
  ...
}

Now, we just edit server.js (simplified version here to give the gist of it):

const express = require('express')
const app = express()

app.get('/', function (request, response) {
  response.sendFile(__dirname + '/views/index.html')
})

// listen for requests :)
const listener = app.listen(3030, function () {
  console.log('Your app is listening on port ' + listener.address().port)
})

If you have your dependencies set up right, the above app can viewed at localhost:3030 by simply running npm install && npm start on your command line. It's brilliantly simple.

Getting our data working

So we have a working Express app. It's running on Gomix, so literally every time I edit the files, Gomix updates the server and it's completely accessible at my staging URL. I know how to pull data from Google Spreadsheets.

All that remained was getting my data into Google Spreadsheets and running some transformations over the returned data to structure it like JSON so my pre-existing JavaScript could consume it. It looked a little bit like this:

// server.js
const json = require('./controllers/json')

app.get('/data/orgs.json', json.orgs)
// ... repeat for tools, projects, resources, data sources

// controllers/json.js
const orgs = (req, res) => {
    // get all the data from the "Organizations" sheet in our Google Spreadsheet
    // transform all the data
    // return it as JSON
}
// ...repeat

export {
    orgs
}

And here's what my spreadsheet is shaped like (this is the organizations sheet):

We now have tech-forward-2.gomix.me/data/orgs.json returning JSON cobbled together from our Google Spreadsheets data. It was easy after that to set up a few Google Forms allowing people to suggest additions to the app (and it was intermediately difficult to set up a custom domain, but they tell me that will be easier very soon).

Final steps

So, I launched the project and everything worked great. However, I heard from a few folks that the way I had implemented the JavaScript loading left screen reader users out in the cold, so yesterday I re-wrote the app to pass the Google Spreadsheets data directly to the view, dropping the AJAX entirely.

It made the app load much slower, but it was surprisingly easy to implement; I just set Express up to use Handlebars as its templating engine (which I had already been using on the frontend, so I could copy exactly the same templates with no changes) and passed the data directly into the views.

A little bit later, I googled "node express cache" and landed on Simple server-side cache for Express.js, and about 20 minutes later I had a 1-minute cache set up on the Google Spreadsheets calls.

You can see all of the code on Gomix, remix it yourself, or see the backup on GitHub.

That's it. It took me a few late-night coding sessions, a bit of Googling, and I have my first production Express app, consuming and caching data from Google Spreadsheets, hosted on Gomix. Beautiful.

Important thanks: I learned everything I know about Express by reading code from my friend Pascal who learned Express about a week before I did. Also, thanks to DeRay and StayWoke for bringing me in and thanks to the entire Gomix team for being awesome.