The fish 🐟 in our home aquarium aren’t very communicative. I don’t necessarily need them to communicate their deepest emotions, but it would be helpful if they would at least tell me when they are hungry. Believe it or not, they don’t. Alas, when I peer inside the aquarium, I don’t know if they are hungry or if someone else in the family has already fed them breakfast🍩 or dinner🍕. IoT to the rescue!
I am in the process of creating a system that enables our family to log when the fish have been fed. The feeding times can be logged by pressing a push-button on a circuit board connected to a Raspberry Pi, pressing an Amazon Dash button, or clicking a button through a web interface. The resulting log file looks like this:
2018-4-15 19:06:48|circuit board
2018-4-15 10:11:22|dash button
2018-4-14 11:46:54|webI’d like to occasionally crunch through the log file and count the number of times each type of button has been pressed. How can this be accomplished?
In this article, we will learn how to count the number of unique items in a JavaScript array. Rather than counting something mundane like integers, we will learn about counting in the context of an IoT project to count the number of button presses that occur as a result of pressing a circuit board button, Amazon dash button, or web button.
Article contents
Preparing the Way
Here’s the sample fish feeding log file we’ll be using:
button-presses.log
2018-4-16 20:03:49|dash button
2018-4-16 09:23:19|circuit board
2018-4-15 19:06:48|circuit board
2018-4-15 10:11:22|dash button
2018-4-14 11:46:54|web
2018-4-14 08:26:27|dash button
2018-4-13 18:21:00|circuit board
2018-4-13 09:53:08|web
2018-4-12 19:31:56|dash button
2018-4-12 12:51:01|dash button
2018-4-11 18:38:36|dash button
2018-4-11 09:41:51|dash button
2018-4-10 21:17:39|dash button
2018-4-10 11:53:10|dash button
2018-4-9 19:29:49|dash button
2018-4-9 11:03:04|web
2018-4-8 19:14:10|dash button
2018-3-31 19:13:25|dash button
2018-3-31 19:11:11|circuit board
2018-3-30 19:02:00|circuit boardI’ve chosen a subset of the log file for brevity, but you can imagine that the full log file will contain many more entries.
To count the number of each button source (circuit board push button, Amazon dash button, web button), let’s first build a function to read the log file and create a JavaScript array that we can use:
function getButtonSources(filePath) {
const lines = fs.readFileSync(filePath, 'utf8').split('\n');
const buttons = [];
lines.forEach(line => {
// Destructure into an array, throwing away the first element
const [, button] = line.split('|');
buttons.push(button);
});
return buttons;
}In the function above, we read the log file and split each line of the log file into an array called lines. We next walk through each line, splitting each line into a two-element array of strings using “|” as a separator. Finally, we use JavaScript destructuring to retrieve the button source to the right of the | character (the second element of our array).
Invoking this function yields the following array:
[ 'dash button',
'circuit board',
'circuit board',
'dash button',
'web',
'dash button',
'circuit board',
'web',
'dash button',
'dash button',
'dash button',
'dash button',
'dash button',
'dash button',
'dash button',
'web',
'dash button',
'dash button',
'circuit board',
'circuit board' ]Excellent! We are ready to process our JavaScript array.
Get Distinct Elements from a JavaScript Array
Let’s start by retrieving the number of distinct elements in the array.
const buttons = getButtonSources(logFilePath);
// returns something like this:
// [ 'dash button',
// 'circuit board',
// 'web',
// 'dash button' ]
// Get unique buttons
// const seen = {};
// The above line is not recommended for objects-as-dictionaries since
// you can run into problems with inherited properties and __proto__
// so use the next line instead per Axel Rauschmayer:
const seen = Object.create(null);
buttons.forEach(btn => {
seen[btn] = true;
});We create a seen object and then walk through the buttons array using the Array.prototype.forEach() function. This approach is a little bit naive since we overwrite a given key/value pair for the seen object several times if the key appears more than once in the array; nonetheless, it keeps the code simpler rather than going the extra step of checking if the key already exists.
Note: Axel Rauschmayer (@rauschma) was kind enough to leave me a recommendation through Twitter to use
Object.create(null)instread of{}to initialize theseenobject since it is an object-as-a-dictionary object and this avoids potential issues with inherited properties. See Axel’s article (The dict pattern: objects without prototypes are better maps) for additional insights.
Here’s the full code sample for this approach:
const fs = require('fs');
const logFilePath = './button-presses.log';
function getButtonSources(filePath) {
const lines = fs.readFileSync(filePath, 'utf8').split('\n');
const buttons = [];
lines.forEach(line => {
const [, button] = line.split('|');
buttons.push(button);
});
return buttons;
}
function printTable(title, obj) {
console.log(`${title}\n------------`);
console.log(obj.join('\n'));
}
const buttons = getButtonSources(logFilePath);
// Get unique buttons
const seen = Object.create(null);
buttons.forEach(btn => {
seen[btn] = true;
});
const uniqueButtons = Object.keys(seen);
printTable('buttons', uniqueButtons);We use Object.keys(seen) to return the array of keys that were added to the seen object.
We also include a printTable helper function to format the result. Our program produces a list of distinct button sources in our array:
buttons
------------
dash button
circuit board
webLooking good!
If we want to be more clever, we can use the Set object to produce a list of distinct array elements:
const uniqueButtons = [...new Set(buttons)];The Set object constructor accepts our buttons array and returns a Set object containing the distinct elements. We use the spread syntax ... to transform the Set into an array containing the distinct elements.
Count Distinct Items in a JavaScript Array
Let’s move on and focus on our key objective: counting the number of distinct elements in a JavaScript array. We’ll consider two ways of accomplishing the first goal starting with the Array.prototype.forEach() method:
const counts = Object.create(null);
buttons.forEach(btn => {
counts[btn] = counts[btn] ? counts[btn] + 1 : 1;
});We create an object called counts and then walk through each element of the array using forEach. If this is the first time we are seeing a given btn key, we initialize the value in the key/value pair to 1; otherwise, we increment it.
Here’s the full program context:
const fs = require('fs');
const Table = require('easy-table');
const logFilePath = './button-presses.log';
function getButtonSources(filePath) {
const lines = fs.readFileSync(filePath, 'utf8').split('\n');
const buttons = [];
lines.forEach(line => {
const [, button] = line.split('|');
buttons.push(button);
});
return buttons;
}
function printTable(titles, obj) {
const entries = Object.entries(obj);
entries.sort((a, b) => b[1] - a[1]);
const t = new Table();
entries.forEach(row => {
t.cell('button', row[0]);
t.cell('count', row[1]);
t.newRow();
});
console.log(t.toString());
}
const buttons = getButtonSources(logFilePath);
// Count buttons
const counts = Object.create(null);
buttons.forEach(btn => {
counts[btn] = counts[btn] ? counts[btn] + 1 : 1;
});
printTable(['button', 'count'], counts);The above code includes a printTable utility function to sort the counts in descending order and print the contents of the counts object we have created. We use the easy-table npm package to print a nicely formatted table containing the results:
button count
------------- -----
dash button 12
circuit board 5
web 3We’re doing awesome!😎
As a second approach, we can also use the Array.prototype.reduce() method:
const counts = buttons.reduce((acc, btn) => {
acc[btn] = acc[btn] ? acc[btn] + 1 : 1;
return acc;
}, Object.create(null));The very powerful and versatile reduce method employs this syntax:
arr.reduce(callback[, initialValue])In this context, we supply an initial value for our “accumulator” (in “reduce” parlance) to an empty object: Object.create(null). We walk through our array and create btn key/value pairs for our acc (accumulator” object).
Using the array reduce method returns the same results.
Accomplishing through Linux/*BSD commands
We’re increasing our knowledge of JavaScript arrays, but achieving a unique count of array elements can also be accomplished in the Linux/*BSD world in several different ways. We explore two methods:
Method 1: cut/sort/uniq
$ cat button-presses.log | cut -d "|" -f 2 | sort | uniq -cWe use the cut command with a delimiter of “|” to retrieve field number 2. These results are then sorted using the sort command so they can be passed to the uniq command to produce a count of the unique items in the array. Here is the output:
5 circuit board
12 dash button
3 webMethod 2: awk
We can also use the venerable and versatile awk to achieve the goal:
$ awk -F\| '{ a[$2]++ } END { for (n in a) print n, a[n] }' button-presses.logThis command-line syntax uses a delimiter (-F) of “|” and creates an associative array called “a” to count the occurrences of each button source. After processing the log file, we iterate through a to produce the following results:
dash button 12
circuit board 5
web 3Count Distinct Array Elements by Month
Let’s take our array counting one step further and produce a count of button presses from each button source within a given month. Here’s the full source code:
const fs = require('fs');
const Table = require('easy-table');
const logFilePath = './button-presses.log';
function printTable(titles, obj) {
const entries = Object.entries(obj);
entries.sort((a, b) => b[1] - a[1]);
const t = new Table();
entries.forEach(row => {
t.cell('button', row[0]);
t.cell('count', row[1]);
t.newRow();
});
console.log(t.toString());
}
const lines = fs.readFileSync(logFilePath, 'utf8').split('\n');
const re = /(\d+-\d+)/;
const counts = Object.create(null);
for (const line of lines) {
const [timestamp, button] = line.split('|');
const m = re.exec(timestamp);
if (m.length > 1) {
const ym = m[1]; // ym = year-month (e.g. 2018-4)
if (!counts[ym]) counts[ym] = Object.create(null);
counts[ym][button] = counts[ym][button] ? counts[ym][button] + 1 : 1;
}
}
for (const yearMonth of Object.keys(counts)) {
console.log(yearMonth);
printTable(['button', 'count'], counts[yearMonth]);
}We extract the year-month (ym) from the log file timestamp entry (e.g.”2018-4″) and create a JavaScript object called counts containing ym as the key and a nested object containing keys for each button source. The resulting counts object looks like this:
{ '2018-4': { 'dash button': 11, 'circuit board': 3, web: 3 },
'2018-3': { 'dash button': 1, 'circuit board': 2 } }Very powerful! We print the results to the console in a more tabular fashion using the printTable function:
2018-4
button count
------------- -----
dash button 11
circuit board 3
web 3
2018-3
button count
------------- -----
circuit board 2
dash button 1Mission accomplished! 🚀
Can we achieve the same goal from the *nix command line using awk or other standard tools? It’s not so easy. Let me know in the comments if you arrive at a solution. Of course, our main goal here has been to learn about JavaScript arrays for a variety of contexts, but it’s always good to have multiple tools we can leverage for solving problems.
Conclusion
Yes, we can count the number of distinct elements in a JavaScript array. JavaScript is extremely flexible and provides numerous methods to accomplish the goal. It looks like most of my family members log the completion of fish feeding using the Amazon dash button. Maybe I should write an article about that sometime.😉
Would you accomplish the goal differently? Is so, how? Please let me know in the comments so I can learn and all of my readers can learn too!
Follow @thisDaveJ (Dave Johnson) on Twitter to stay up to date with the latest tutorials and tech articles.
Additional articles
Using TOML Config Files in Your Node.js Applications
Guide to Using Redis with Node.js
Making Interactive Node.js Console Apps That Listen for Keypress Events
Guide to Installing Node.js on a Raspberry Pi
Last updated Apr 17 2018
Why are you using Array.map when you are only interested in side effects? We have Array.forEach for that purpose.
Ed, I agree with you completely. It makes more sense to use Array.forEach in this context since there are no side effects. Thanks for taking the time to point out this improvement! I updated the article accordingly.
It seems like you’re basically creating your own implementation of a set data structure. Why not just use the set object added with es2015?
function countUnique(iter) {
return new Set(iter).size;
}