Dashboard

In this post I am going to share an experience about refactoring a project that decreased the load time of a website.

Brief History

A few years ago, I was working on a project where I was tasked to improve the performance of a certain dashboard. Before I tell you the problem with the dashboard, let me tell you what it looked like. Mainly it was a mash-up of multiple charts to show some data points. When the page loaded, it used to show a loading screen and in the background, it used to make one single request to fetch those data. The backend would then query the database couple of times to generate around seven different data points (yes, because that’s how many charts were there). Once all the querying is completed it would return back the results to the front. It used to take around 10-12 seconds before we could see anything on the website. To put the cherry on top, it was the first page for an admin after the login. So even if the user wants to go to a different page for different work they had to wait before the entire page loads.

Although I am using the word frontend, it is not like a detached frontend. It’s a legacy application where each page was served by the server. However, in a few parts, the project was using JQuery to fetch a few data from API. (I did not know why some parts of the application use APIs and on the other hand some serve HTTP pages… I was not there when they made this decision).

Do we make it look faster or actually faster?

People used to complain about this dashboard a lot especially the slowness of that specific page and something had to be done. Eventually, we reached a conclusion to make that part faster!! Now here is the thing, you can either make things faster or you can make people feel that the site is getting faster. And we needed both. This requires a lot of refactoring.

A frontend framework for the reactive dashboard

To further improve the performance of the dashboard, we decided to use Vue.js to render the dashboard. Using Vue.js, we can take advantage of its reactive components to render the page quickly and provide an improved user experience. Additionally, Vue.js allows us to use virtual DOM, which helps to reduce the time taken to render the page. This allows us to update the page faster, providing users with a more responsive experience. Furthermore, Vue.js provides a number of features that can be used to further improve the performance of the dashboard, such as server-side rendering and code-splitting.

By using Vue.js to render the dashboard, we can ensure that the page is rendered quickly and efficiently, giving users a better experience. Now a lot of you might say, we could’ve used React or even Svelt instead of VueJS, but at that time, some of our devs knew vueJS a little bit more than any other frameworks. Which is why we choose that. Plus their way of making any page a vue app just by adding the CDN link was extra beneficial. That’s why we choose that.

the power of refactoring for faster dashboard

Splitting APIs

To further enhance the performance of the dashboard, we have employed a strategy of splitting the data into multiple APIs as opposed to serving it all together. This approach has allowed us to retrieve only the data that is necessary for the user, thus reducing the load time of the page. Moreover, the division of the data into multiple APIs has enabled us to reduce the processing time of the data, again leading to improved performance of the page. At the same time, we have been able to take advantage of caching, which allows us to store the data and serve it quickly, thereby reducing the time taken to render the page. In this way, we are able to ensure that users have access to the data in a rapid and efficient manner.

Change in UI to make it faster

The first approach was to serve an empty dashboard page when the user was logged into the site. We decided to use vueJS to fetch the data. Since we implemented 7 new API endpoints to get seven different data points, we made 7 parallel API requests. While waiting for a response, we only rendered 7 boxes with loading indicators. Whenever one of the data is returned through the API, the vueJS would update the graph and show it in the dashboard. Using this approach may have increased the number of network calls but we did not have to wait for the whole data to be fetched from the backend.

Since the empty dashboard was loaded at first, if the user wants to go to a different page they can go to that page from the navbar. These changes made the UI a little more responsive than before. Once that is done we moved to the next stage, which is using caching to serve data faster.

Implement caching to make response faster

Another strategy we have adopted to further improve the performance of the dashboard is to use Redis to cache the results. This means that the data is stored in Redis, which is a fast and lightweight database so that it can be retrieved quickly. Since Redis is in-memory data storage, it is able to store a large amount of data without consuming too much time, thus making it ideal for this purpose. Furthermore, since Redis is highly scalable, it is able to handle large volumes of data, making it an ideal solution for caching search results.

Redis is an open-source, in-memory data structure store that is used to cache data for faster retrieval. It helps to reduce the load time of search results by storing the result in memory, allowing users to access the data quickly. We decided to use Redis to cache the search results in order to improve the performance and reduce the time it took to query the database. Using Redis helped us to reduce the load on the database, saving time and resources.

In our case, the data in the dashboard did not have to be real-time updated. Instead, the requirement was like, as long as the data in the dashboard does not show any data that is more than 30 minutes old, they are ok with it. This made our work a little bit easier. We configured the redis in such a way so that the data expires within 30 minutes.

Here is an example code snippet using Node.js to set a value in Redis with an expiry of 30 minutes:

const redis = require('redis');
const client = redis.createClient();

client.set('myKey', 'myValue', 'EX', 1800, (err, reply) => {
  if (err) throw err;
  console.log(reply);
});

client.quit();

In this example, myKey is the key that we want to set in Redis and myValue is the value that we want to associate with the key. The third argument 'EX' indicates that we want to set an expiry time for the key, and the fourth argument 1800 sets the expiry time to 30 minutes (since the expiry time is measured in seconds).

The set method also takes a callback function that is executed when the operation is complete. If an error occurs, the error is thrown; otherwise, the reply from Redis is logged into the console.

Finally, we close the Redis client connection using client.quit().

So every time the frontend calls the APIs, we would check if the redis has any data about that, if there is we would just return that data otherwise, we would query the database and once we get the data, we would save the data in the redis with 30 min expiry and return that data as API response.

Conclusion

It’s not that hard to make the service faster, it was just a lot of rewrites. That is why I always tell people to think, is there a better way to complete a feature, before jumping into implementing a feature? Even if it may take more time right now to make it right, if we do not fix that, it will only consume more time to fix that later on.

Thank you for reading! What topic would you like me to write about next? Let me know and I’ll do my best to create a helpful blog post for you.