Mr. Robot

Dave Touretzky, Research Professor at Carnegie Mellon University.

Professor Dave Touretzky is something of a superstar in the Cozmo development community. As one of the earliest SDK users (he was actually one of the first people to use the SDK outside of Anki), he’s provided others with an immeasureable amount of insight, feedback, and tools to help them with their endeavors.

But Dave is more than just an active Cozmo SDK user. Among other things, he teaches a cognitive robotics course at Carnegie Mellon University, conducts impactful outreach work with other educators, and is building an exciting new robot intelligence framework—all using Cozmo and the SDK.

While many of you might already be familiar with him, we thought it’d be great to formally introduce Dave and his work to all of you.

Anki: Thanks for taking the time to chat with us, Dave. For those who might be unfamiliar with you or your work, can you tell us a little about your personal background?

Dave Touretzky: I’m a Research Professor in the Computer Science Department and the Center for the Neural Basis of Cognition at Carnegie Mellon University in Pittsburgh (CMU). I’ve done work in artificial intelligence (AI), computational neuroscience, robotics, and computer science education. I also like to cook and eat spicy food, am an active pilot, and live with an Amazon parrot named Ignatz.

You clearly have a passion for robotics. Can you tell us what it is about the field that you love?

The short answer is representations. Robots, people, and animals all need representations of the world in their heads in order to achieve their goals. All of my work, from AI to computational neuroscience to cognitive robotics, is about investigating these representations.

As a computer science professor whose work integrates with robotics, what is your view on the relationship between the two fields?

Robots are computers. More specifically, they’re computers with sensors and motors attached. That doesn’t mean every roboticist has to be a computer scientist. For instance, we need mechanical engineers to make good robot bodies, and electrical engineers to make good sensors and controllers. But if you want the robot to show some intelligence, that falls in the realm of computer science.

Did your interest in robotics and computer science inevitably lead you to becoming an educator, or was teaching always something you wanted to do?

I taught my first computer class when I was 15. (My mother had to write a note to my high school explaining why I was missing school that day.) Teaching is always something I’ve enjoyed. But like most faculty at research universities, I think of myself primarily as a researcher.

You also work with various outreach programs. Can you talk a little about some of the organizations you work with and why?

Since 2005 I’ve been working with colleagues at various Historically Black Colleges and Universities (HBCUs) to help establish robotics courses there and encourage students to go on to graduate training in computer science. Some of this work was funded by the National Science Foundation (NSF) through its Broadening Participation in Computing program.

Andrew Williams, then at Spelman College, and I formed a group called the Advancing Robotics Technology for Societal Impact (ARTSI) Alliance. Andrew was later succeeded as head of ARTSI by Chutima Boonthum-Denecke of Hampton University. Our group consisted of 17 HBCUs, including major ones like Norfolk State, Florida A&M, Winston-Salem State, Howard, and the University of the District of Columbia. We also had prominent African American roboticists at major research universities, such as Monica Anderson at the University of Alabama, Chad Jenkins at Brown (now at Michigan), Ayanna Howard of Georgia Tech, and CJ Taylor at Penn.

Although our NSF funding has ended, we all still keep in touch. In fact, I’ve started sending Cozmo robots to these HBCUs and we’re looking to see how they can use Cozmo in their activities, including their university coursework and K-12 outreach.

The “robot graveyard” in Professor Touretzky’s lab, with retired Chiara, Calliope2SP, and Calliope5KP robots.

Calliope3 robots in Professor Touretzky’s lab at Carnegie Mellon. The Calliope3 was the last robot built in the lab before switching to Cozmo.

You have a history of building your own robots which, by any standard, is nontrivial work. Why did you take the bespoke route?

Previously I was using Sony’s AIBO robot dog, but in 2006, it was discontinued. Unfortunately no one else stepped in to fill the market need for consumer robotics. And because we just couldn’t buy an affordable mobile robot with vision and serious computing power, I and a lot of other people started making our own.

One of my first robots, the Chiara hexapod, was programmed to play chess on a real chessboard. Though wheeled robots are certainly more practical, I have a particular fondness for hexapods.

This past spring, you taught a cognitive robotics course using Cozmo for the first time. Can you explain what cognitive robotics is and what your students learn in your class?

Cognitive robotics means different things to different people. I use the term to describe robot programming techniques that draw inspiration from cognitive science. But it can also mean applying artificial intelligence ideas to robotics, such as planning algorithms, or advanced perception, to make robots intelligent.

My course at CMU covers concepts such as computer vision, landmark-based localization, and path planning algorithms for navigation. And, of course, there’s a lot in there about representations, both of the world and of the robot’s own body.

As soon as I saw that Cozmo was a vision-based robot and was user-programmable, I knew that he was something special.

How did you first hear about Cozmo? Were there specific features about him that made you to take special notice?

Initially I saw an early announcement of Cozmo on a robotics news site. But what really motivated me to look into Cozmo was the recommendation of someone here at CMU who knew about him. As soon as I saw that Cozmo was a vision-based robot and was user-programmable, I knew that he was something special. Vision is a critical aspect of robotics, and at present, Cozmo is the only consumer robot on the market that can see.

Cozmo locates and captures a red ball via custom software a student wrote.

How did the experience in your cognitive robotics course differ for you and for your students when compared to previous years when you used your own hardware?

Cozmo is both cheaper and more reliable than the robots I could build myself. But what surprised me most was the benefits from him being so small. He can happily run around on a standard desktop. This meant we could have ten teams of students in one lab, all working at the same time, and no one felt crowded.

Over the years I developed a robotics application framework called Tekkotsu that was implemented primarily in C++. When the class switched to Cozmo, I wanted to provide the students with same kinds of tools, so I built an open source package called cozmo-tools that sits on top of the Cozmo SDK and provides many of the best features of Tekkotsu, such as a hierarchical, parallel state machine formalism, and a variety of visualization tools.

A look at some add-on accessories made for Professor Touretzky’s class.

A student project where two Cozmos played tic-tac-toe against one another. The Cozmos kept track of the chips, and also knew when a win state was reached.

More custom accessories including a 3D-printed ball hoop and colored chips.

Cozmo navigates a maze using optic flow, IR reflectance, and edge detection.

Custom cubes with magnets.

Can you talk about some of the projects your students made using Cozmo last spring?

There was a nice mix of projects last spring. Some people developed code for manipulating new kinds of objects, such as little poker chips, or cardboard cubes with magnets inside. Others worked on computer vision problems, such as using deep neural nets for object recognition, or using optic flow information to detect doorways and obstacles. Other people developed applications. For example, we had two different tic-tac-toe programs, and one that allowed you to create a simple drawing on the screen which Cozmo would then reproduce on paper using a magic marker attached to his lift.

Watch the trailer for Calypso.

Outside of your cognitive robotics course, how else are you using Cozmo?

The most exciting thing I’m developing for Cozmo right now is Calypso, a kid’s programming language inspired by Microsoft’s Kodu Game Lab. Calypso is a pattern-matching rule-based language, so it looks very different from procedural languages like Python. It also lets you use a game controller to interact with Cozmo. We’ve been testing it with kids and they’ve loved it.

Calypso is currently in a private beta, but expect to hear much more about it in the coming weeks.

In closing, why do you think it’s important for people both young and old to learn robotics?

I was a boyhood ham radio operator, which required arcane technical knowledge and a federal license. Most people couldn’t be bothered, but for those with a passion for technology, it was incredibly exciting.

Today, everyone’s walking around with a sophisticated radio transceiver in their pocket, and that radio is also a web browser and a camera and a satellite navigation system; we call it a smartphone. While it’s still possible to live your life without a smartphone, it’s a serious handicap if you can’t use one at all.

Robots today have the same potential that radio had 40 years ago. They are going to become indispensable parts of all our lives, as smartphones have, and in ways we can’t even imagine yet. That means people learning about robotics today are not only getting a glimpse of our collective future, but they’re preparing themselves with the knowledge and understanding of what is quickly becoming a ubiquitous technology.

• Visit Dave’s GitHub for access to his Cozmo programming tools and unofficial Cozmopedia.

• Get more info on the Calypso project at the official website.

• Ready to try the Cozmo SDK yourself? Read this post to quickly get up and running.

• Want to see what other people are creating with the SDK? Make sure to visit the project showcase.