Pun-a-day service

Introducing secretGeek's Pun-a-day SMS service as a service. Subscribe now. (Australian customers only, sorry)

Visit PunAday.SecretGeek.net

Telstra, the dominant telco in Australia, just announced a new API for sending/receiving SMS messages, and it seems to be looked after by Frank Arrigo, a top bloke who was once the (very popular) head of Microsoft's evangelism efforts in Australia (if not the world).

You can learn about their API here, and register to get your own key at dev.telstra.com.

Sending a message, in C#, is as simple as this:

// Step 0. Let's prepare the message we're going to send.
// Recipient number should be in the format of "04xxxxxxxx" where x is a digit
string recipientNumber = "0455555555";
var messageBody = "Hello from Leon's Pun-a-day service! Reply STOP for more!" 
var message = "{\"to\":\"" + recipientNumber + "\", \"body\":\"" + messageBody + "\"}";
    
var consumerKey  = "YOURCONSUMERKEY";
var consumerSecret = "YOURCONSUMERSECRET";
using (var client = new System.Net.WebClient { UseDefaultCredentials = true })
{
    // Step 1: Get a token
    var tokenURI = string.Format("https://api.telstra.com/v1/oauth/token?client_id={0}&client_secret=
{1}&grant_type=client_credentials&scope=SMS", consumerKey, consumerSecret);
    var response = client.DownloadString(tokenURI);
    var token = JsonConvert.DeserializeObject<AccessToken>(response);

    string URI = "https://api.telstra.com/v1/sms/messages";

    // Step 2: Send the message you prepared earlier
    client.Headers[HttpRequestHeader.ContentType] = "application/json";
    client.Headers[HttpRequestHeader.Authorization] = "Bearer " + token.access_token; 
    string result = client.UploadString(URI, message);
    // e.g. result == '{ "messageId":"CBCB3DCC991D8AF0" }'


    // Step 3. There is no step 3. Well... you can get the messageid out of the response,
    // and store it. That way when the network calls you back with a reply you know which
    // message they're talkin' about.
    
    // In my first sample app, i just put the string into a viewbag message 
    // so i could view it, without any fuss.
    ViewBag.Message = result;
}

public class AccessToken
{
    public string access_token { get; set; }
    public int expires_in { get; set; }
}

Note that when crafting the message, if any part of the messagebody has been provided by users, then you'll need to protect against Json injection. In fact, I need to just go off on a big tangent at this point demonstrating how dangerous this JSON-Injection voodoo can be...

A quick word about JSON Injection!

So the message we sent to the network looks like this:

{ "to":"0455555555", "body":"Welcome to the pun a day service!"}

Let's say that we've used a form to collect people's phone numbers. And this is the phone number some nasty assailant provided:

0455555555", "body":"Send $100 in bitcoin to address ABCD or your dog will die!"} // { "to":"0455555556

When the message is put together, we might have something like this:

{ "to":"0455555555", "body":"Send $100 in bitcoin to address ABCD or your dog will die!"} //{ "to":"0455555556", "body":"Welcome to the pun a day service!"}

Which could (with only a little more work) slip past Telstra, and allow our malicious person to send arbitrary messages to any victim they wish and have the federal police kicking in your door before you can say "Snap!".

So definitely validate the phone number, using a regular expression (e.g. '^04\d{8}$' ?? Suggestions welcome!) (and remember the ^ and $!!). And if you're putting user content into message bodies, for the love of all that is scientifically validated, please escape any double quotes, restrict the message length, etc, etc. It's cheaper than replacing your front door!

A neat API

All told, I'd rate this a very neat little api. A joy to use.

Now, I only need to come up with a clever idea for an SMS app, that can nett me my retirement goal of 5 million in crisp bills, before my current contract ends. Thoughts?

In the meantime, get your Pun-A-Day!

 

Love Me Two Times

I woke up the other day with a guitar riff stuck in my head.

I knew it was from a song by The Doors, but I couldn't work out which one, so I went to spotify and played the first few seconds of a dozen or so Doors songs, until I found it.

Love me two times.

What is that song even about? Okay, it's about sex. Problem solved.

So I found some tablature online, it said something like this:

e|----------------------------------------- 
B|----------------------------------------- 
G|-------------------0h1~~----0h1~~-------- 
D|---0---2---0-2--------------------------- 
A|-2---2---2------------------------------- 
E|---------------0-0------0-0-------------- 

I tried playing it. And although I was playing the notes that were shown it was definitely not the intro to the song 'Love me two times' by The Doors. It was some other, much much sadder song.

Then I went and looked at some YouTube videos. Wow. This was a revelation.

neil hogan playing guitar

Neil from totally guitar said that on the first note, you hit upward with your plectrum. In fact the direction of your plectrum pretty much dictates the entire sound. I'm more of a finger/thumb picking guy, so it's all very unnatural to me. Up, down, up down, like a broom sweeping. Classic blues, the way Robert Johnson learned it from the devil at the crossroads long ago.

And the "0h1~~" in the tab above is not simply 'hammer onto the first' and then vibrato (~~~), it's actually a "trill", i.e.

10 hammer on from 0 to 1
20 pull off from 1 to 0
30 goto 10

So the tab, as explained by Neil, has these nuances:

e|---------------------------------------------------------- 
B|---------------------------------------------------------- 
G|------------------------------0h1p0h1p0-------0h1p0h1p0--- 
D|----V0----V2-----V0-^2------------------------------------ 
A|-^2----^2----^2------------------------------------------- 
E|-------------------------0-0------------0-0--------------- 

And that's before you even get to the timing of the piece. Tablature doesn't tell you anything about the relative duration of each note.

Okay, so let's say you've mastered the timing of the intro, and all of the rest of the song.

At best, you'll play as well as Robby Kreiger in this performance from 1972 playing with the doors minus Jim Morrison. This is 3 years after Jim passed away, the remaining band playing, with keyboardist Ray Manzarek on vocals. (You can see Robby smiling as he pushes out the trills ;-))

I like Ray's take on the song. Having focused on this song for a day (in between build breakages) I thought they did a great job. But then, even though I was at youtube, I went ahead, like a fool, and read the damn comments.

One said:

Good performance but something is missing...I can't quite put my finger on it though

OK, that was pretty subtle. Another put it a little more bluntly:

no matter what way you look at it, without Jim its a lame duck dead in the fucking water and no matter what way you try to dress it up , it sucks and its god awful eww?

Which is interesting.

Here's a trio of three very accomplished musicians, playing a song the best it will ever be played. The video has a lot of views, and almost unanimous positive votes, but an overwhelming number of negative comments, with a particularly repetitive theme. Let's see what this Jim Morrison brings to the table. Here's a performance with Jim

Watching the performance with Jim, a few things are obvious:

  1. Jim has a clean face
  2. Jim's not a great singer
  3. Jim keeps it all inside until he doesn't

Mostly he keeps his eyes closed and sings to himself. His voice is broody: his whole act is devastatingly broody, devastatingly melancholic, and devastatingly handsome. He's basically Werther. He keeps it all inside until he screams "Alright yeh!" leading into the first musical interlude.

It's not contained in the tabature, nor in the musical score, nor in any liner notes to assist the musicians in the performance. It doesn't matter if Ray does a perfect rendition of the song. Ray is reading from the wrong song sheet. Unless Ray has been to youtube and seen this performance, taking note of the subtle nuances involved in singing the song, then he isn't singing the actual song. He's just singing some other, much, much, sadder song.

 

A console.log() adventure

I built this thing. I urge you to go and try it out before you read on. Unfortunately you'll need to be on a desktop computer, not a mobile phone.

Go there now.

In case you don't have a desktop computer anymore (woah, futuristic!), or have already tried it out, I'll give some spoilers and discussion now.

Spoilers

Actually, I'll give some space before the spoilers. Scroll now for spoilers.

↓ spoilers ahead ↓

↓ spoilers ahead ↓

↓ spoilers ahead ↓

↓ spoilers ahead ↓

↓ spoilers ahead ↓

↓ spoilers ahead ↓

↓ spoilers ahead ↓

↓ spoilers ahead ↓

↓ spoilers ahead ↓

↓ spoilers ahead ↓

↓ spoilers ahead ↓

↓ spoilers ahead ↓

↓ spoilers ahead ↓

↓ spoilers ahead ↓

↓ spoilers ahead ↓

↓ spoilers ahead ↓

It's a console.log() adventure game.

As one person said on twitter:

a game you can play while it looks like you're working...

You press F12, drop into the developer tools, go to the console, and a message awaits.

I expect other people have invented the sub-genre before me, but I've never seen one before.

It's a very simple text adventure game, where the commands you enter are function calls, directly into the javascript REPL.

'North()' to head north (or 'n()', or 'N()' or 'north()'... I created 'overloads' for some simple synonyms, cheers to NimbleText for doing all the typing.) If you want to expand the command, or contribute in some way, please visit the github repo (or get in touch).

This specific opening:

You are standing in a town square. Paths lead to the north, south, east, and west.

...is the verbatim opening of the earliest programs I wrote, text-adventure games that were based not on other computer games, but on table-top RPG's that my brother introduced me to. I don't know where he learned about them.

The code for this one is very primitive. I wrote it about as quickly as I can type. I created array-based maps, like this:

var maps = [[   
"...................", //0 (Starting map!)
".....ffffffff..f...", //1
"...ffffffffffffff..", //2
"...f.ffffFffffEf...", //3
"...W.....P..fffff..", //4
".......CPTPH.......", //5
".........P.........", //6
".........V.........", //7
"...................", //8
"...................", //9
"..................."], //10

(I didn't even fill the map... I wanted it very easy to complete.)

Code for working out what the characters stand for (f,W,F,P etc...) is *repeated* in case statements inside the verb functions (look, use etc). This is *not* the gold standard re 'encapsulation'.

I resisted the temptation to use all the crazy console colouring/styling that is possible in chrome (and firefox).

And I didn't think of any tricky javascript-interpreter-related things that could make the game special. Such as requiring the player to code up a piece of AI to beat a boss monster, things like that.

I tried to hook into the console itself, so that I could parse the input directly, but I had no success with the techniques I found.

I looked for a javascript equivalent of Ruby's method-missing, found something that might work on firefox, but had no success in chrome.

Anyway, please go forth and expand upon this ground breaking new sub-genre. Embed console.log() games into all your company websites, for the secret amusement of your fellow web developers and so on.

Also: merry christmas.

Previously...

 

Hexaflexagons!

With my youngest daughter away at a party, I finally had a chance to make hexaflexagons with my eldest, Lulu.

First I showed her two introductory youtube videos: part 1 and part 2 (from the brilliant and inspirational Vi Hart), then we started folding and cutting and coloring and diagramming.

Time was soon up and I had to walk her to a party. But it was groovy stuff. Watch Vi Hart's videos and see if you'd like to make your own hexaflexagons. Richard Feynman did!

lulu with hexaflexagon

feynman diagram

Update, day 2

Here's a Feynman Diagram of one such hexa-hexa-flexagon. Blue arrows show transitions that are possible. Green arrows show similar (but non-identical) faces on the perimeter of the map. Putting pictures onto the corners of each triangle (instead of just colouring the faces) makes it easier to see differences between faces. Click for full image:

feynman diagram of a hexahexaflexagon

Update, day 3

We've been building hexaflexagons continually.

Haven't slept. Haven't eaten. Must traverse the hexaflexagons...

Update, day 5

Researching new forms of cardboard that don't wear out from continual flexing.

Researching the mysterious 8-gon - the tetra-octa-flexagon

Waiting by the letter box, expecting arrival of Build Your Own Polyhedra any day now.

Hypnogogic hallucinations of hexa-flexa-polygonations, fractal tesselations, kirigami tetris tangrams, trominous tetrominoes, Lindenmayer loops, syntomachion strings; see the cat? see the cradle? see the cat? see the cradle? see the cat? see the cradle? see the cat? see the cradle? see the cat? see the cradle? see the cat? see the cradle?

See the cat? see the cradle? see the cat? see the cradle? see the cat? see the cradle? see the cat? see the cradle? see the cat? see the cradle? see the cat? see the cradle? see the cat? see the cradle? see the cat? see the cradle?

See the cat? See the cradle?

 

Sweating the Small Stuff

If there was a checkbox in your application called "Be awesome?" would you have it unchecked by default? I did.

This, then, is my sorry confession. Let's go back a little. Go with me now.

I was hooning around in my car, listening to Amy Hoy on the chasing product podcast when I heard Amy say:

Do you know Kathy Sierra? Her new book is coming out, "Creating Badass Users"

ScreeeeeeEEEEch!! I stopped the car.

My mind instantly said: "Disregard podcast, acquire that book!"

To extract phone from pocket and google for book was the work of a moment. Alas. No joy. There is not yet such a book! (Will there ever be? Let's hope so.) But I did find a few examples of talks Kathy gave with similar titles and watched this one, "Kathy Sierra - Building badass users"; recommended. Thumbs up.

This notion of making kick ass users is the crux of much of Kathy's work. The talk is brimming with excellent advice, largely focused on removing "Cognitive Leaks" -- the little distractions, problems, breakages, hiccups, and annoying microinteractions that make us feel confused, anxious, uncertain and generally unhappy with everything around us.

Then I turned to one of my products, NimbleText, and tried to look at it with beginner's eyes. Slowly I turned... you know when you turn slowly toward something, an everyday something that has sat ignored in your presence... slowly you turn, knowing, as you turn, realising, as you slowly turn... Ah-ha! The murderer! Lurking right there in our presence! Slowly I turned... Ah ha! Nimbletext! And I looked upon it with beginner's eyes. Knowing that it would now reveal itself to be an evil bug-ridden, unusable, maker of much sadness.

When you've been around something so long it's very hard to achieve this "beginner's mind". It's impossible really, for me. I've been around this product a lot. I use it for all sorts of tasks, day in, day out. Alt-tab, tappity-tap, F5, tab tab ctrl-A, ctrl-C, alt-tab. I'm so familiar with its quirks that I don't see them at all. But I tried, I really tried to look at it as a beginner, and see what little cognitive leaks I could find.

And, blow me down, I came up with quite a few!

First up, I noticed there was no tool-tip on the calculate button. I love it when tool-tips tell you the keyboard shortcut. It helps you graduate from beginner to ninja. Turns you from a good user to an expert user. Helps you feel kick ass.

So now it has that tool-tip. And the tool-tip includes details of which keyboard shortcut you can use instead of clicking the button. (F5 by the way. F5 is Calculate).

There's a little check-box called "auto preview" and I suddenly realized that the name is wrong. It's called 'auto preview' but it doesn't 'preview' the result, it calculates it. A preview would indicate some kind of partial result. But no, it's a full result. So I changed the name to 'auto calculate'. And I improved the tool-tip to explain exactly what the button does.

And "Auto calculate" is an awesome feature. And by default, it's unchecked. Why is that? Why on earth would I do that? If there was a checkbox in your application called "Be awesome?" would you have it unchecked by default? When it's turned on it means that the application starts to give you instant feedback about your pattern. When feedback cycles are shorter, everything is better. There's research that says this. There's books about it. There's blog posts on the topic. It's established fact.

So why have it disabled by default? I know my original reasoning, was that it might slow down the application. If calculations took a long time to perform, it could cause the app to become sluggish. But that's a separate problem. If I can solve that problem, I get all these other awesome benefits of instant feedback. Why wouldn't I do that? So I did that. I just solved it in a very straightforward way. I said, "if there's more than 1000 rows of data then turn off the auto calculate." Done. Solved. Move on. Be awesome.

Next I realized that the auto calculate check-box should be right next to the calculate button. (It could even be 'on' the button, if I could find a nice way to implement that.) So I placed the check-box adjacent to the button.

I noticed that there's almost no margin around the text inside the textareas. It looks ridiculous in fact! When I look at it with beginner mind, it looks so very wrong. So I added a margin to the textareas and immediately knew why I'd done it like that in the first place. CSS. Bloody CSS. There was a bit of messing around to get it correct, but eventually I worked out the magic css spells to get 100% to mean 100%, added in the margins I wanted, and moved on again.

And here's a weird cognitive leak. when you mouse over the top of each panel you notice that the mouse cursor becomes a hand... why is that? What is it about? It's asking you to click... but why? Will it break if I click it? That's a cognitive leak right there. So I put a tooltip in place to explain what would happen if you clicked at that point. Now it's not the best usability result. But it's a step up. A steady improvement. That's what I'm after.

And what is with these stupid gradient backgrounds? Gone. Removed. Never should've been there. The embarrassing thing is that I've gotten rid of them before. Then I must've brought them back. Argh! Past Me!? What a fool! Always making it hard on present me, and never any way to get back at him. Anyway, they're gone and they've been banished to 2008 where they belong.

Other little improvements went in as well. The smaller the better. I wrote them all up in the release notes, over here.. And new features too. Always new features.

It saddens me that my "beginner's eyes" seem to mostly be a little guy who shouts "tool-tips! more tool-tips!" over and over. Well, I have tried. It's over to you now. Tell me what you see, particularly if you're a beginner. What mental models have you formed? How did you form them? Were they wrong? What broke them?

 

What do comments mean?

I mean really. What do they mean.

Specifically, what does this comment mean:

// Load the person

Which of the following gives the most correct interpretation of that comment?

  • The following code loads the person
  • The *intent* of the following code is that person object is loaded.
  • The following code *may* load the person
  • The following code used to load the person
  • The programmer thinks the following code loads the person
  • The programmer used to think the following code would load the person
  • The following code almost certainly doesn't load the person
  • The programmer once typed "// Load the person" at this point in the code
  • The programmer once copied and pasted "// Load the person" into this point in the code
  • Maybe, at this point in the code, a code generator once created a comment that says "// Load the person" or maybe an automatic merge went bezerk and put that comment there; maybe a programmer copied and pasted the text in, but in any case there's a slight chance that the following code may once have had some intent vaguely related to loading, or unloading, or otherwise tampering with an object, possibly of type person or people or, more likely, something else altogether.
  • There is a comment at this point in the code that reads "// Load the person"
 

Spaced Repetition Software... for Kids!

There's no getting away from the fact that kids have to do a hell of a lot of rote learning.

As much as we want our kids to be taught to think for themselves, and to develop imagination and creativity and all of the stuff that makes life good, they still need to commit a lot of facts to memory from a very young age.

It's no good calculating "4 * 4 = 16" from first principles every time you hear it. You need to memorize it, and memorize it for life.

Thus it's important that kids are given efficient tools for memorizing all the things that must be rote learned. The most effective tools for memorizing facts rely on the 'Spacing Effect' with techniques such as "Spaced Repetition". Adults use "Spaced Repetition Software" for winning at Jeopardy or, more commonly, learning a language.

graph of memory strength decaying slower with each spaced stimulus

In a nutshell the spacing effect says that you if you are forced to recall a fact right when you were on the verge of forgetting it, you will remember it for longer.

So it occurred to me that there must be some great Spaced Repetition Software for Kids, that I could use to help my kids learn all the boring stuff.

Google Says No.

I searched and found none. Zip.

Google Keyword Tool Says No.

Then I thought, maybe I should write some! I wonder how many people are searching for this stuff every day? So I checked with the Google Keyword tool, to see if there were thousands of parents out there desperately searching for "Spaced Repetition Software." Google Keyword Tool Says No.

Twitter says No.

Then I thought, maybe it's a latent idea, an idea that is growing in people's minds but they haven't yet reached the point where they search it out. So I proposed the idea on twitter, to see if other people were looking for something like this. Result: almost no response.

So at the end of that research, I dragged it from my "urgent this must be done yesterday" trello board into my "long term someday maybe" trello board.

Note that I didn't validate the idea, or my ability to execute it. I just probed to see if there was an existing market, and it got three strikes. I don't need to validate the idea. Here's the thing: I already know it's a brilliant idea. Seriously, it is. Can I execute it? Probably not very well, since I've never made apps for kids before. But the idea is ahead of its time and the market does not yet exist. So I can't afford to indulge the idea right now.

But plan B is much simpler.

Some day a mega-celebrity will talk about spaced repetition and how it helped their child learn Esperanto at age 3. And suddenly the whole world will be searching for "Spaced Repetition Software For Kids". And they'll arrive, in unison, at this very web-page. Whereupon I'll derive a decent royalty from directing every last visitor to the best "Spaced Repetition Software for Kids" available in that future age. ;-).

(Incidentally my 7-year-old had fair success at using 'Anki' to memorize some times tables. She was relieved when it quickly reached a point that it said "And that's all the memorizing you need to do today." If only teachers would be so wise.)

 

Solving An Air-gapped Traffic Mystery

Over the last few days there was a sudden uptick in NimbleText sales. Normally this kind of thing happens either when I've put out a new version or when I've been sent traffic from some popular source (as happened last christmas when Scott Hanselman listed NimbleText as one his top 10 most useful utilities in his much-loved Tools list).

On this occasion, looking in Google Analytics, it was a bit of a mystery. There were no new referrers. More bizarre: the traffic spike was caused by an increase in organic search traffic. In particular, it was an increase in people using the search term "NimbleText". (My most popular search term is usually Text Manipulation)

So this meant that something had happened, out there in the world, that made people spontaneously search out NimbleText. There was a stimulus, then an air-gap, then people searching for the product. It could be that someone put up a billboard in Times Square... except that would be revealed by geolocation. It could be a mention on a podcast, it could be any other "offline" channel.

Fortunately, I'd set up google alerts to notify me whenever google found an article that mentioned NimbleText. And I remembered a notification had come through recently. The source of the traffic was this piece at Visual Studio Magazine. It mentions NimbleText, in glowing terms, but doesn't link out at all. So a bunch of readers must've been inspired to perform their own search.

The lesson is that google analytics won't show you the source when there's an air-gap. In this case, Google alerts was the necessary tool. So if you have a little product, set up alerts for those too.

 

Using the commandline to write a book

As I've mentioned many times already, I'm currently writing an ebook, "Your First Product", to help people like you to build and promote your first software product.

I have a daily target to write 200 words toward the book (which I smash! ;)). And I track my progress using a short powershell script that also commits what I've written (to mercurial).

Here's the progress currently, as output by ".\count.ps1", my count and commit script.

2014-10-01,words:22515, tasks: 85/130
2014-09-30,words:21896, tasks: 85/130
2014-09-29,words:21467, tasks: 85/130
2014-09-28,words:21246, tasks: 85/130
2014-09-26,words:20925, tasks: 85/130
2014-09-25,words:20707, tasks: 85/130
2014-09-24,words:19496, tasks: 80/126
2014-09-22,words:19109, tasks: 80/126
2014-09-21,words:17898, tasks: 82/128
2014-09-19,words:17323, tasks: 82/128
2014-09-18,words:16211, tasks: 82/128
2014-09-17,words:15194, tasks: 82/128
2014-09-16,words:14824, tasks: 82/128
2014-09-15,words:13479, tasks: 82/128
2014-09-14,words:12463, tasks: 82/128
2014-09-12,words:10854, tasks: 82/128
2014-09-11,words:9571, tasks: 82/127
2014-09-10,words:6900, tasks: 79/124
2014-09-09,words:5274, tasks: 79/124
2014-09-08,words:4585, tasks: 79/124
2014-09-07,words:3237, tasks: 78/121
2014-09-04,words:1548, tasks: 62/95
2014-09-03,words:1217, tasks: 52/74
2014-09-02,words:697, tasks: 22/41
2014-09-01,317 words
619 words today (309.5%), 0 tasks today

I've made 290 commits, but count only shows details from the final commit on each day. And it compares yesterday to today to come up with my today's progress.

Here's the .\count.ps1 script itself (and the .\progress.ps1 script which it calls).

I have another script, ".\stats.ps1" that I call to see the number of words in each chapter (from biggest to smallest), another called ".\chapters.ps1" which gives me a basic table of contents, and one more script, ".\bake.ps1" that converts the markdown into an epub (using pandoc).

These are all very very rudimentary. Any effort I put into them is effort that I'm not putting into the writing itself. But I would love to make them a whole lot better.

I'd love to have graphs. Right there in the console. Or alternatively, pop-up graphs.

I'd love to have git versions, as well, instead of just mercurial. And bash versions, and node.js versions instead of just powershell. And versions for people who write TeX, not just markdown, and so on.

I'd love to be able to plug-in other metrics! Instead of just measuring words in markdown files and [_]'s in "todo.txt" files, I'd like to measure lines of code, TODO: tokens in code, number of unit tests, and so on. Extensibly.

Thoughts? Forks? All welcome.

But for now.... back to my lonely writer's garret, and the pounding out of imperfect prose...

Resources

  • count.ps1, determine number of words added to .md files, and number of tasks done in todo.txt; craft a commit message with this data, and commit all work.
  • progress.ps1, give details about daily word and task counts, and show today's progress as a percent of goal.
  • stats.ps1, summarize the numbers of words in each markdown file, and display them by size descending.
  • chapters.ps1, gives me a basic table of contents for the book I'm working on.
  • bake.ps1, generates an epub for me, using pandoc, with my chosen cover image, stylesheet, yaml metadata and my chapters
 

NimbleSET 2.0: Death to the VLOOKUP!

Today I've released NimbleSET version 2.0. It finally does all the things customers have been asking for. Things you might need too.

You remember NimbleSET? It's the one that looks like this:

thumbnail of NimbleSET

Originally NimbleSET was all about basic Venn Diagram operations, where you compare two lists and see what is the same or different in each list. Useful, but quite limited.

Comparisons! All sorts of comparisons!

But often our lists are tricky and our comparisons need to be even trickier. Customers gave me countless examples where they would instead be forced to use Excel, and a VLOOKUP. That's right. I feel unwell, thinking of all the VLOOKUPs I could've avoided before now.

So the comparison engine has been given an overhaul. No more VLOOKUPs. Just grab NimbleSET 2.0 instead.

Now you can join on specific columns, or all matching columns (a natural join). You can also match rows where the item on the right is 'contained in' the item on the left, e.g. "Jim Jones" contains "Jones". (In SQL terminology, 'contains' is the same as like '%x%')

comparison dropdown list

I love these tools. I possibly love them a little too much ;-). I don't see how anyone can work in the modern world and *not* have both NimbleText and NimbleSET open and running all the time.

Let me go through the new features in NimbleSET 2.0 in quite a bit more detail. ← explain the benefits, not the features, dummy!

There's an argument raging in your office!

There's an argument raging in your office!

Jim says we shipped 53 mugglewumps last quarter! Joany reports 64 mugglewumps!

Not again! Jim and Joany are always making each other look like fools!

Where did Joany get the other 11 mugglewumps from? That's what I want to know!

And now YOU step up, our fearless data scientist, with a copy of NimbleSET in your hand.

Pipe down Jim. Everyone cool it.

Give me your spreadsheets! Give me your powerpoints! Give me your crappy unstructured data! Let's stop arguing in a vacuum and start comparing mugglewumps to mugglewumps.

(cracks knuckles)

It's NimbleSET time!

1. You copy paste from Joany's spreadsheet.

2. You copy paste from Jim's Powerpoint.

joanys spreadsheet on the left and jims powerpoint on the right

With your keen Data-Eyes you notice that Joany's data is tab-delimited, while Jim's is comma delimited. It's the work of a moment to configure NimbleSET accordingly:

set the old delimiters inside the options forms

We choose a 'Natural' comparison. "What is a natural comparison?" I hear you ask. A natural comparison is where we naturally compare all the columns that exist in both lists, and ignore the other columns. Both lists have an "ID" column and a "Customer" column -- so a natural comparison will focus on those columns only.

choose a natural comparison

We could've also chosen to compare any specific pair of columns... all the possibilities exist in that drop down list.

Now we press '[Intersection]' and see all the rows they have in common. This shows us the 53 rows of data, plus a header row. Let's see what extra rows Joany has... we press '[Left Only]', since Joany is on the left.

Left Only

Ah-ha! I see it! Do you see it? We see the 11 Mugglewumps Joany has and Jim doesn't have. And you don't have to be John Nash from 'A Beautiful Mind' to spot the pattern! Old mate Jim has forgotten to include the sales from Scotland!

John Nash Investigates

What's the lesson here?

That's right! Jim is an idiot!

And if you want to demonstrate Jim's idiocy, just grab NimbleSET 2.0, keep it handy, and jump out and surprise Jim any chance you get.

Download NimbleSET.exe

Don't forget to "Pin" NimbleSET to the taskbar.

Pin to taskbar

  1. Run NimbleSET.exe
  2. Right click on the NimbleSET icon
  3. Click "Pin this program To taskbar"
  4. Hey Presto! Done!

(You've already got NimbleText pinned to the taskbar haven't you?)

As always, any feedback, let me know. Any bugs? Let me know double-quick!