👋

Hey ,

Every time you hear the term big data, it usually comes along with various terms like AI (Artificial Intelligence), Machine Learning, Neural Network and blackbox algorithm, for example. All these terms sound a lot like mambo jambo and make your head spin.

Take a deep breath, keep calm and let your head gradually stop spinning. The steps to get your company’s (or your own paperwork/filing) in order is simpler than all these hyperbolic terms suggest. And by the end of this post, you can easily get started on your Machine Learning optimized AI black box algorithm.  

Ignore those last few buzzwords and let’s get cracking.

Four Simple Steps to Get Your Data Organized

Ok, now that you’re ready, where do you start?

1. Define your goal: The wise Lawrence J. PetIf says that “if you don’t know where you are going, you will probably end up somewhere else."

There is some truth in that, both in life and managing your data. You can have goals to organize your company’s financial, operating or personnel data, track future data and generate insightful trends to further optimize your company’s performance. 

Or the goal can simply be to get organized by getting all the data under one roof, to make the data easy to use and accessed by relevant folks in the company.

In a nutshell, having a goal is important for your data journey. Otherwise, there will be lots of data to gather, metrics to track like nobody’s business. At the end, you might feel overwhelmed and raise the white flag.

2. Gather and/or acquire data: It is tempting to skip the data you already have and think that you need to get new additional data to get meaningful insights. It is often a mistake. Start with all the data that you already have. You have more than you think. Think about all those receipts, invoices, bank statements, payroll and other data you/your employees keep on their machines and email attachments.

Once you have gathered all the existing troves of data in your company, then determine if you need to acquire “funky” external data sources such as social media feeds, web scraping or real-time geo-location data. Acquire all these additional data if they help you achieve the goal(s) in step #1.

3. Organize and cleanse the data: Now that you have dusted the data dusts, it’s time to put them onto a (preferably online) database, so that it will be easy to access, view and update.

You need to understand how each type of information is presented. For example, a typical invoice would have the following details a) invoice number; b) invoice date; c) vendor’s name and contact details; d) details of products/services rendered; e) total amount due; f) payment details and g) late payment information, if any.

Afterwards, you may want to retain all or some of this information onto the “Invoice” database by creating fields and adding each individual invoice’s information to the rows. To start, simply use readily available tools like Excel or Google Sheet. No need for any fancier tools than that.

Cleaning the data on the database is the next crucial step. If you skip this step, you won’t be able to analyze and visualize your insights meaningfully. No amount of beautiful visualization charts will salvage bad data.

Just like you would wash and cleanse your face before applying a face mask to give the youthful result you aim for.

You need to remove duplicates and any incorrect entries. Plus to standardize common terms across the database. For instance:

Pvt. Ltd. or Private Limited

Dept. or Department

Sdn. Bhd. or Sendirian Berhad

SHAMELESS PLUG 😬: I clean data for a living. Check out Timbadata’s website for more info. Or ping me if you want to know more, happy to chat.

4. Analyze the data to generate meaningful insights: Now comes the fun and funky step. When people talk about big data, they usually skip the first three steps and jump right onto this stage. You are better than them, and you will get better outcomes as a result.

Alright, with the data you have gathered, organized and cleansed, it’s time to crunch the data and get useful insights. There are simple functions you can use in Excel to get started. Or once you are comfortable, you can graduate to more functions or regression analysis. There’s even an external tool called Idea that literally gives you ideas on what sort of analysis you can do on the data you have.

Your analysis might uncover overwhelmed staff that have been working diligently from home this past year without taking vacation days. Red flag 🚩. It’s high time to nudge them to take a break and recharge.

5. Present the analysis result through visualization: Usually, step #4 and #5 are done together. Once you have analyzed the data, it’s time to present it in a visual way. Visual presentation often makes the data and analysis easier to view, digest, comprehend and ultimately take action on.

You can display data in one of two common ways - tables or charts. Tables are good to present the underlying data. Charts on the other hand would be a better choice to illustrate trends or patterns. Excel (and Google Sheet) does provide excellent tools for you to create simple or more advanced charts. If you’re ambitious, you can also create a dashboard to display various data points/analytics at a glance.

Notwithstanding the various tables and charts you can choose to display your analysis, it is important to identify the “story” you want to convey. This is why Step #1 is important. Revisit the goal you set earlier in this data journey to align the “story” you would like to tell, aided by the data and analysis you just undertook.  

Next, think about the audience that you are going to share the story with. Are you going to share the findings with your teammates, the Big Boss or Board of Directors? The data and story need to make sense to them and more importantly act upon it. Based on what they know (or don’t know) you need to create the right context, choose the tables or charts to display. A great data story is lost if your audience can’t understand it.

If you notice in the diagram above, I mentioned data security, governance, provenance architecture, integration and interoperability and store and backups. These are more advanced concepts in data management.  

I’d say, when you’re starting out, keep it simple. You don’t need fancy governance or provenance. Two things you might need are security (to keep away prying eyes) and storage and backing up your data. Even then, most online storage providers will provide these features as part of their service. The rest are just bells and whistles.

I hope that these five simple steps to managing your data provide a good glimpse to get you on board on the big data train. Cut the noise. Stop reading about all those funky algorithms than run on quantum computing. Keep it simple. More often than not, simple is beautiful. 

Be great,

Reez Nordin

If you like what you read, do share this link with folks in your network for them to sign up at

https://sendfox.com/spicyvc