8 min read
I’ve been building Sugg for quite a while now, and have seen some imaginable problem and roadblock, I figured it’d be nice to share some interesting technical information: sugg.xyz.
Firebase Realtime Database
Firebase Realtime Database is a cloud-hosted, NoSQL database that stores and sync data between users and devices in real-time. It integrates with Firebase Authentication and provides declarative security model allowing access based on user identity or pattern matching with stored data.
In my opinion, Firebase declarative security model is fascinating, especially since anyone can have access to your database credentials but with security rules, only an authenticated user can read and write and at best they’ll be affecting only their own data and not the entire database.
Although Firebase supports local cache on devices to serve and store changes when users go offline and synchronize when devices come online, Sugg doesn’t implement this feature yet.
Firebase provides authentication with social network like Facebook, Twitter, Google, etc. and makes managing users simple and secure. Using this method of authentication made me focus on providing seamless sign-in and UI for users and Firebase handles complex cases.
Angular.js was initially released October 2010 which makes it among the very first Single Page Application framework, I decided to use Angular.js since it’s maintained by Google, provided extensive documentation, community support, and as of Sugg initial release Angular.js powered some Google projects. Plus I’m was super familiar with how it works and was just the obvious choice.
Although, Google has released several version of Angular 2, 4, 6 as it’s successor and the major changes:
- “scope” or controllers to components as its primary architecture.
- syntax expression, “[ ]” for property binding, and “( )” for event binding.
- Microsoft TypeScript language.
- Modularity, moving core functionality to modules — it’s about time.
- Asynchronous template compilation
- Dynamic loading
I’ll be using yaml to represent Sugg application structure, which uses Gulp, Jade, Sass, and Angular.js. I’ll be explaining just the core but you can view the project on GitHub for better understanding.
sugg: - bin: - auth.js - www - client: - public - src: - app: - controllers: - *.controller.js - directives: - *.directive.js - filters: - *.filter.js - partials: - *.partial.jade - services: - *.service.js - application.js - config.js - scripts: - **/*.js - *.js - shared: - *.jade - styles: - **/*.sass - *.sass - 404.jade - index.jade - config: - config.js - middleware.js - routes.js - worker: - *.js - index.js - gulpfile.js - package.json - yarn.lock - .travis.yml
auth.jsfor generating API Firebase access token, verifying and authenticating with Firebase server.
wwwinitializes and starts the Node.js server.
configapplication wide configuration files
middleware.jsmiddleware used for every API routes defined.
routes.jsAPI, static and Web Application routes.
config.jstest, development and production environment variables configurations.
index.jsglobal middleware, request headers, logger and error-handlers.
gulpfile.jsconfigures tasks for deploying, copying, bundling and watching.
.travis.ymlconfigures CI/CD to Heroku.
package.jsondependencies, scripts and project information.
yarn.lockwell, it locks dependencies. :)
srcstyles, root index and 404, Angular.js controllers, directives, services, filters, and config.
src/app/application.jsrequires dependencies, configure client routes and state.
publicgulp generated directory with bundled files and linked dependencies.
workerscheduled scripts that perform actions like removing deleted account.
I’ll be using yaml to represent Sugg current database structure:
root: notes: note_id: content: "<div><!--block-->hi there</div>" lang: "html" settings: color: "lavender" is_public: false # if `is_public` is true, anyone with link can view share_with: # if `share_with` is defined, only specific people can view, userA and userB user_id: read: true share_id: "-LN5jJzDwEbATlGWjGXc" share_at: 1537711363113 write: true users: user_id: access_token: "Gl0hBu_NOz3b7Wy-wHB8o6K51LVkjyF" created: 1490962741849 suspended: 1522574098167 email: "email@example.com" id: "18474522088343517" image_url: "https://www.akinjide.me/static/images/akinjide-avatar.png" is_active: true is_new: false metadata: metadata_id: created: 1537447335290 is_public: false # if `is_public` is true, anyone with link can view note_id: "-LMr-8DcC7szafF9aH7T" share_with: # if `share_with` is defined, only specific people can view, userA user_id: read: true share_id: "-LN5jJzDwEbATlGWjGXc" share_at: 1537711363114 write: true title: "Sugg" updated: 1537711783943 share_with_me: # if `share_with_me` is defined, only specific people can view, userB share_id: metadata_id: "-LMqmY5WGsM2jxwcLq4v" note_id: "-LMqmY1WMu76rBjjV-X8" read: true shared_at: 1537711362781 shared_by: "google:447335290" write: true name: "Akinjide Bankole" provider: "google" updated: 1537682964670 tags: tag_id: created: 1525895724452 title: "Personal" updated: 1525895724452 settings: user_id: created: 1525884100175 default_layout: "list" default_note_color: "pink" updated: 1537528578663
Sugg has four main nodes users, notes, tags, and settings. I’ll describe and explain the decisions behind each node.
rootevery node supported by Sugg and is an entry to the database.
usersauthenticated users and note sharing information.
notesauthenticated users notes and note sharing information.
settingsauthenticated users and default settings for personalization.
tagsallows authenticated users group notes by tagging each note created, thereby enabling sorting and filtering (i.e. Filter notes by Travel tag, every note with Travel tag will be fetched and displayed). Although tags aren’t currently in use.
id, user_id, tag_id, metadata_id, note_id, share_idunique identifiers that enable querying and persisting individual records in the database.
user_id and idare extracted from a successful social network authentication response.
*_idare generated by Firebase on persisting to the database.
created, suspended, updated, shared_at, share_attimestamps representing when persisted to the database.
read, write, is_publicset permissions for shared note.
shared_byauthenticated user identifier of note owner.
access_token, email, image_url, name, providerare extracted from a successful social network authentication response.
is_activeenabled when a user deactivates Sugg.
is_newenabled for the newly registered user, switched to false on second sign-in.
default_layout, default_note_colorset by Sugg for the newly registered user, but changes based on user personalization.
title, color, lang, contentdefines each note created, all attributes are store within node
Sugg database structure will ever evolve but the structure above serves the current purpose. I’ve personally identified minor flaws with the current database structure but this structure will scale without problems.
Heroku is a Platform-as-a-Service (PaaS) where the developer is freed up from managing the underlying server infrastructure and concentrate on the functionality of the application. This guided my decision to use Heroku for hosting Sugg, plus Heroku offers reasonable pricing which is awesome and it currently serves its API and Web application using 1 Web/1 Worker with 512MB RAM and is capable of handling quite a bit of concurrent traffic, ~1 000 requests per second.
Sugg uses Heroku Pipeline for deploying to Heroku which is essentially a group of Heroku applications that share the same codebase. Each application in a pipeline will represent one of the development and production stages, this is extremely useful and allows me to manually trigger push to the production stage.
# GitHub (Master) # └─> Travis (continuous integration/continuous deployment (CI/CD)) # └─> Heroku Pipeline //> (Development (dev.sugg.xyz)) //> (Production (sugg.xyz))
Users can share notes either privately using an email address which sets:
share_withnode on the sender notes > settings node and users > metadata node,
share_with_menode on the receiver users node,
or publicly which sets:
is_publicflag on notes > settings node,
is_publicflag on users > metadata node,
Sugg generates URL which follows the pattern
note_idnote unique identifier as path param (i.e.
uiduser unique identifier as query param (i.e.
meta_iduser metadata unique identifier as query param (i.e.
sharedboolean value indicating public or private (i.e.
Coloring notes was quite straightforward, Sugg has a defined color list registered as CSS class selector. When a user picks a color, the color title is saved on the notes > settings node and the note renders using the color title as CSS class.
A user can perform CRUD operations but the major difficulty was deleting user account alongside all records. When an account is deleted Sugg sets:
suspendedfield with current timestamp as value,
is_activeto false preventing users from login.
A worker runs monthly to fetch and delete accounts flagged as inactive alongside records attached. Notes shared publicly or privately are deleted as well to uphold data integrity.
NOTE: Current thought, process stated below might change.
Currently in progress but the idea is to prevent preying eyes from reading notes when logged in. User can enable lock on note which will require a unique pin.
Encrypting a note will use combination:
- sugg private key.
- user unique identifier.
- generated unique identifier.
- user unique pin.
Sugg will persist the encrypted note with a flag
is_secure alongside the generated unique identifier.
Decrypting a note will use the same process as encrypting but reversed. Sugg will not persist the decrypted note unless the user disables lock which resets
is_secure flag and deletes the generated unique identifier. The generated unique identifier can’t be fetched by anyone, just the note owner.
My hope is that my writing here convinces you to take a look at Sugg architecture the next time you build an application and consider following its steps or even better as reference. Anyhow, if you share the same opinion as do I, give sugg.xyz a try and mail me if you have any questions? Would be happy to help.
NOTE: I’m not in any way affiliated with any services mentioned, these thoughts are completely my own.
PS: If you read this far, you might want to follow me on github, or subscribe below for updates (I'll email you new essays when I publish them).