# Where's Waldo? Using Machine Learning to Find All the Waldos By [Minh Tran](/content/blog/author/minh-tran/index.html) Where’s Waldo is a game wherein one looks for Waldo (a bespectacled, dead-eyed human candy cane looking guy) hidden in chaotic, busy scenes of whimsy. In other words, it’s the perfect data set to challenge our [**visual recognition API**](/content/blog/test-visual-recognition-models-with-clarifais-v2-api-javascript-starter/index.html) with using Custom Training! At [**Clarifai**](/content/site-root.html), we have an internal Hack Day every month where everyone works on a pet project they don’t normally have time for. My Hack Day project this month was to attempt to answer a question that bored kids everywhere have asked when they found themselves stuck indoors, be it from either from a rainy day, a doctor’s visit, or a cross country road trip in the back of a minivan … Where’s Waldo? With scenes filled with animals, aliens, mermaids, and people of all sorts (including several Waldo imposters or “friends” as they call them), I wanted to see just how well our [Custom Training model](/content/products/enlight-model-training?__hstc=233546881.1d3d76c4cf8d6fe6fa94ddfd58087e7e.1606227685129.1606227685129.1606227685129.1&__hssc=233546881.1.1606227685130&__hsfp=223677638/index.html) could perform on such a challenging and popular data set. Here’s how I did it! **Step one: Find the data** Luckily for me, some kind and curious soul had already [created and shared a Where’s Waldo dataset online](https://github.com/vc1492a/Hey-Waldo), where he took 19 Where’s Waldo maps, split them into grids/tiles, then labeled the tiles accordingly (‘waldo’ or ‘not waldo’). I decided to follow that method and do the same with other Waldo maps not included in his set. Maps were split up in several different grids (4×5, 4×6, 5×8, etc) to increase the sample sizes. These tiles were then uploaded into our app and labelled accordingly. **Step two: Train, test, more data** After the initial training, I tested the model against a map not in the training set, featuring Waldo on a moon colony, naturally. Against the entire map, our model found … nothing. Well.. a 1% chance of Waldo being present. Not an unexpected result, since so much of the image simply isn’t Waldo – that’s what makes playing these so fun/frustrating. I decided to test it again using the same grid system as the training. Going from tile to tile, results started making more sense. 1-3% chance of Waldo on relatively empty tiles. 10-20% chance on busier ones. 30-40% chance on Waldo imposters (shakes fist). then, a tile with 50% chance of Waldo! I scanned the tile anxiously. Boom. Waldo, hiding behind a crowd of people on the 2nd floor of a biodome. **Step three: More data, less imposters** My next goal was to lower the percentages of the Waldo lookalikes being flagged as Waldo. I went back to the original maps and manually cropped out several (somewhere between 500-1000) people; some lookalikes, others with similar [**color schemes**](/content/blog/create-digital-art-with-p5-js-and-clarifais-color-recognition-model/index.html), others just random people. All to tell the model who wasn’t Waldo. **Results** Higher percentages on Waldo positive tiles, lower percentages on Waldo negative tiles, less false-positives on lookalikes. Overall though, the numbers still aren’t great. The busier tiles still throw the model for a loop, Waldo predictions still aren’t high enough, and false-positives are still present. Oddly enough, it isn’t the characters that would normally trick human eyes (the lookalikes, characters wearing stripes or glasses, etc) that are throwing the false-positives, but something else that I haven’t been able to discern yet. As I revisit this project, I believe my next steps would be to find/increase the Waldo-positive examples, to try and counteract the implicit unbalanced nature of the data. That’s the beauty of [**machine learning**](/content/blog/what-is-machine-learning-a-four-minute-rundown/index.html)– the more examples you show, the better the results. If you’re interested in trying this for yourself, [sign up for a free API account](https://portal.clarifai.com/signup) and get started with [Custom Training](/content/products/enlight-model-training?__hstc=233546881.1d3d76c4cf8d6fe6fa94ddfd58087e7e.1606227685129.1606227685129.1606227685129.1&__hssc=233546881.1.1606227685130&__hsfp=223677638/index.html)!