Introduction
Many customers have asked our team to provide a “best practices” example of how we’d like to see a mobile application developed to utilize the power of Stream APIs. Since a mobile app would also need a backend API service with which to communicate, we also needed to develop a server-side application which we have open-sourced today. Our backend API service was written using Go using several other open-source libraries, and shows our best practices for a simple photo sharing application. The user flow is quite simple and we’ll describe how our feeds are used along the way. Users who are logged in can upload a photo, like photos from other users, and follow other users so their timeline feed is up to date with photos from those other users. You can also see your own feed of photos and see statistics such as how many photos you’ve submitted and how many followers you have. Important notes: This backend service was produced to show our best practices about building on Stream’s API for activity feeds and social timelines only. We took some liberties along the way and this application is NOT meant to highlight industry best practices around things like user authentication, database indexing, API authentication from other client software, complete error handling, monitoring and metrics, and so on.
Database Setup
We chose a simple MySQL database schema (included in the GitHub repo) to track some simple resources for our service. The ORM we used (described below) automatically created fields for a unique auto-incrementing ID for each row, and a datetime field for when each record was created, last updated, or marked deleted. Our ‘users’ table allows individuals to select a unique username and email address and assigns a UUID. The table for tracking photos, aptly named ‘photos’, has a UserID field to determine which user took the photo, a unique UUID, a count of likes the photo has received, and a string to hold the URL of where the image is hosted online. Users who ‘like’ a photo will have an entry in the ‘likes’ table which tracks the UserID who liked the photo and the PhotoID of the photo itself. A ‘sum’ operation on a PhotoID or a UserID would provide quick, useful statistics. Finally, users who follow each other have an entry in the ‘follows’ table, which holds two UserID values where the first UserID is the user who wishes to follow another user, and the second UserID is the target to follow.
Open-Source Used
This application also gave us a means to stress-test our own Go SDK for Stream which we open-sourced in September 2016. (You can read more about it here) We used the Gin framework to build our API endpoints, which allowed for rapid development of the endpoints, and Gin's built in request/response contexts were quite easy to use for JSON responses and HTTP status codes. Call us crazy, but we chose two ORMs to work together to build this project. We chose GORM for handling our struct models but found it’s syntax difficult to learn for rapid development, so we also implemented gorp for handling our queries and data management. Since we needed a scalable place to make our photos available to mobile apps, we implemented Amazon’s AWS Go SDK to upload our images to a S3 bucket. Finally, we used other open source libraries for handling UUIDs, connecting to MySQL and for image manipulation.
Implementation
The endpoints we’d need were pretty clear: a means to register and log in, fetch a list of users, fetch one of several feed types, as well as a way to pull some statistics for the user’s profile screen. Users also needed some endpoints to follow/unfollow users and to like/unlike their photos. The primary goal of the application was to make the mobile app as light as possible for data management, so the heavy lifting was placed on Go. For example, we could have sent the raw Stream feed JSON to the mobile client and let it parse out what it needed, but mobile apps need snappy response times so we needed to minimize that as much as possible. Read more about our feed setup for this photo sharing app in Part 1 of this series. We'll publish our Android app very soon!
Route Setup
Gin feels a lot like Python’s Flask microframework for setting up routes. Just as Flask can manage minimal written code for “a user requesting this URL path should be sent to this method”, Gin allows a very similar mechanism for the router to define the HTTP verb (GET, POST, PUT, etc) and a handler which can either be an in-line closure, or the name of a function defined elsewhere in your code. Our application shows a mix of both. The main difference between Flask and Gin is that Flask allows for multiple HTTP verbs to route to a single function where Gin would need each verb to be defined separately. For example, we have a route for the base URL of our API showing an OK status:
As well as an example of a route calling a function as its handler:
While we do technically mix some GET requests in the creation and destruction of content (adding/deleting likes and following/unfollowing other users) which isn’t truly RESTful, this was more to allow for rapid development of the mobile app we’ll be announcing soon.
Notes about some of the endpoints
We also wanted to describe behavior in some of our endpoints to clarify best practices around tracking data in your own database or calling Stream for data.
The Users endpoint
This endpoint simply pulls a list of all users out of the database to present in the app, but we needed to do some extra processing to determine whether a user followed others. When a user requests the follow/unfollow endpoints, we add or remove entries in the ‘follows’ table. We could have performed an API call to Stream to determine which feeds your timeline feed follows, but this would be far more processing than a simple table in the backend database. This reduces load on our servers and is much quicker if indexed in your database properly.
Notification Feeds
The ‘object’ field is optional for activities posted to Stream, and contain a reference that’s only relevant to your application. For our photo sharing app, if a user ‘likes’ a photo, we use the ‘object’ field to hold a reference to the photo such as its UUID. As an example, if Tommaso liked a photo by Alessandro, the activity’s “actor” would be Tommaso’s user feed (“user:tommaso”), the “verb” is “like”, and posted to Alessandro’s notification feed (“notification:alessandro”) with the “object” field containing the photo’s UUID such as “photo:9cf34d34-a042-4231...” which would be our backend's internal reference to that photo. Now, when we retrieve that notification feed item, we can use that “object” data to enrich the feed item with more information about the photo such as its URL, number of likes, and so on.
Aggregated Feeds
Our backend retrieves timeline_aggregated:user_uuid when calling the feed endpoint of /feed/timeline_aggregated/UUID. Stream’s JSON payload will group things by verb but to present this in a meaningful way to our Android client that makes processing faster, we manipulate the data heavily before passing to the mobile device. For “follow” verbs this was very easy as there would be a single aggregated verb of “follow” which contained a list of other activity actors which we could enrich from our database. This wasn’t so easy for the “photo” verb, because the list of photos presented in Stream’s JSON payload was listed in a chronological order, not grouped further by actor. Our backend, then, needed to aggregate the data further internally before building a new JSON payload for the mobile app to consume so that the mobile app didn’t have to perform this processing.
Image Manipulation
The photo upload backend shouldn't make the user wait for processing and upload to S3, so we utilize Go's built-in concurrency and implemented a "goroutine". Once the photo is received, we immediately send a JSON response to the user, and call our goroutine. The image manipulation resizes images to a maximum resolution of 1024x768 to keep the mobile application very responsive at the expense of image quality. The best practice would be to upload the full sized image as well as a smaller version, track both URLs in the database, and let the mobile app determine which image to show.
We’d Love Your Feedback
As with all of our open-source projects and examples, we would love to hear your questions, ideas for improvements and so on by posting a new issue on GitHub.