Hello
So, I have finally decided to build a robot. Not just any old robot, but something I can use to experiment all my ideas on AI (artificial intelligence), and robotics in general with.I have been preparing for this for a few months now, looking around at various technology stacks, API, SDKs etc. So firstly I'll list off how I plan to do things with a 10,000 foot view.
I have to say it - don't be put off all true friends
First though a word to the sagacious reader. This is my blog. That is, I'm writing it for my enjoyment, and also to plan things, and as a record of what I've been doing. So I'll write it the way I want, using the style I want, and choosing the content I want. If you don't like anything here, I don't care at all, be on your way, and seek out people who want to listen to your opinions, I don't at all.If you enjoy/learn something from all this, then great, if you want to tell the world how wonderful, clever, erudite you are, go do it someplace else, we are all full up with narcissists here, there's no more room not even for one more. Sorry.
So, what have I been looking at ?
Firstly, the brains of the operation - like mine, will be British. The Raspberry Pi has been a friend to me for about a year now. We've done many things together (not all have involved MAME), I've played around setting up various things on it, like :- Machine On - A VNC type program, allows you to remote log in to Linux from any other OS (including the dreaded Windows). Once logged in to the Pi you can do anything you like. So with this there is no need for a keyboard/mouse./monitor to connect up to the Pi. I tried using tightVNC and others but they just don't play ball with every OS, or I get weird keyboard outputs when using Qt Creator etc.
- Linux - Various flavours - I'm going for a cut down Debian version for my final Linux version on each production Pi, but for now I'm using Raspbian (the official OS for the Pi). So I have various versions at the moment, all of which are the full GUI versions. This is just while I develop things, once the robot becomes production ready, I'll be stripping down the Linux OS to a bare minimum number of processes, to free up as many resources as poss.
- OpenCV - An SDK (open source - free) - basically a free set of software libraries I can use to do things with images/pictures. I've delved into some face detection and recognition (more of this later). But this will be used to detect who the robot is talking to, or at least who is in the room.
- Qt - Another free SDK - this baby works on most OS's, and its for developing software that you write once, and run on any OS. However unlike things like Java, it needs compiling (as it's C++) using a native compiler. So if you are going to run your app on Linux, you need to compile the app on Linux. This will be for any GUI apps I need, for example collecting faces to train the face recognition brain with. This way I can write the code once, run it on either Windows/Android/IOS/Linux etc.
- HTK - Another free open source SDK, this one is for voice recognition. Needs to be trained, uses Hidden Markov Models (more of this lot later). Once trained, it will be able to understand what the hell garbage I'm shouting at it to do, and also understand I'm in charge, and to do my bidding.
- Visual Studio - community edition (it's Microsoft contribution to all this madness), I've set up various free plugins - git repository, Arduino libs etc. It's an IDE that does everything and it's free. Yes there are others, but setting up Eclipse (I used to write plugins for it) is a pain, so I've opted for the behemoth that is Visual Studio Community Edition. It beats having to use Arduino IDE as well, which is crap for debugging (although having said that my first job was using xdb on HP UNIX back in the late 80s). But yes I still use vi as an editor.
- Arduino - The ubiquitous generic electronics board. Using this family of boards (Uno and I've just got a pre-prod MKR1000 - it's go built in Wifi so nanananana), it's very easy to program firmware (software that runs on electronics/embedded devices). I'll be using the Arduino (various types) to capture sensor inputs, and drive outputs like LEDS/Screens/Motors etc. This will be the muscle and the senses of the operation.
- Raspberry Pi 3 - I'll be using 5 of these in parallel to control things. These will be the brains. There will be a layer of IPC - inter process communication between them, and I will probably be using some kind of TCP across Wifi, so they can talk to each other. For comms to the robot from a manual transmitter (a remote control joystick type thing - so I can ram the robot into the wall when drunk, I'm using some cheap wireless transceivers from Amazon), and to talk to the robot via Wifi using my PC I'm using UDP - who cares if you lose a few packets heh ?). More of how I will split the robot's tasking systems later.
- The Agent 390 - Servo City purchase robot. The friendly looking (not yet !) tank, the robot itself, this will become my best friend over time, as I will be programming it (to only like me, and think I am great - ah perfect)
- Various Electronic Sensors from Amazon.ca. I've bought lots of in-expensive sensors, so my robot can read its environment, and adapt itself to the outside world. Examples are proximity sensors, temperature, laser detectors, fire, water, gas, sound etc.
- Other electronics - I've salvaged a few broken tablets (I'll be using the screens off these, maybe with some touch input if they still work). I couldn't seem to get a decent version of Android on them though, by decent I mean a version I could program with. I've written some Ant Simulator thing based on some swarm algorithms to help with some of the AI later, that I wrote on Android in Java, so I might migrate this over soon to Linux. I won't be using these old tablets for any Brains, but just for a kind of entertainment system, like "Robot look up on Wikipedia how many people are reading the entry for me, under the title clevererist person who ever did live" etc. Or put on the stone roses latest album, or play my favourite film and output it on a projector. Which reminds me, I've got a mini projector to beam stuff on walls - cool.
What kind of things do I want this robot to do ?
Good question, thank you, that's ok, no problem. Ok Bye.Yes I want it to talk, listen and understand, recognise faces, point to the person it's talking to,
remember things about people it's met, be able to look up info on the web, be able to control an entertainment system, play games with me, tell me it loves me...hold on forget that bit. Yes, while we are it it, I'm not doing all this for any salubrious reasons, I'm not into virtual girlfriends etc, but if all this works and I can make a million then maybe...just maybe...
Beyond Good and Evil
There will be five main systems, and therefore each system will be running on it's own Raspberry Pi 3. Here is a list of them, and what each will be responsible for :- The Master Brain - This will contain a semantic network knowledge representation system ,with an inference engine (I did my PhD in this - so this bit should be easy - gulp). This is basically a way of organising information, and being able to extract that information using natural language. Context is the enemy here, so I will start off with non-generic very specific definitions of words in our dictionary. And of course the only language I'm communicating and representing the knowledge in, will be English., This model will also be handling the communication between other Pis, and the main commands between systems. So for example all data comms will be routed to the brain, the brain will decide where those instructions/data should go, and route them accordingly.
- The Eyes - This Pi went to market, this pi has roast beef, and this pi had EYES ! Yes I will be using 2 or 4 USB cameras for eyes. I've written most of this system using Qt and OpenCV in C++, so my face detection and recognition system will be covered later in more detail. But for now the only thing the eyes can do is detect and recognise faces, and of course capture images of people it doesn't know, so it can train itself as to who these strangers are. I will be adding object recognition, and shape detection etc later.
- The Ears - Hear no evil, see no evil, etc. Using HTK I will be running a live speech recognition system on this one. It will extract out words (mostly commands at first), and pass them to the brain.
- The Entertainment system - Dance you bitch dance ! This one will be playing games, music, films, browsing the web etc. And generally filling in for me having no friends etc.
- The mobility system - This system will handle moving the monster about, sneaking up behind kids and screaming in their ears etc. So this will have movement, navigation, avoidance etc.
And that's about it. Note that I will also be utilising a few programming tricks such as :
- Adaptive systems - for learning things like faces
- Heuristics rule based AI
- Neural Networks - Googles TensorFlow - for training pattern recognition
- Genetic Programming - for getting the robot to learn how to move, creating new things from old things via mutation and inheritance
- Semantic Networks - Storing knowledge using English words
- Inference Engine - Extracting knowledge from the Sem Net (often using inheritance/composition - same as OO)
- Various SQL DBs - For storing relational data - free ones mainly MySQL I love you.
- Web API -RESTful (pragmatic REST) - splitting up the backend resources so they become URL resources, with the 4 HTTP verbs -keep it simple stupid, I mean CRUD - create, retrieve, update and the dreaded delete - reminds my of my days at Ubisoft ahh Montreal). This will allow me to interact with the robot through a web portal, probably using IIS as a server.
One other thing, I write software in any language that I think will do the job. So here's a general guide about what 'll be using :
- Java - For any Android Stuff - like my ant simulator - runs on mobiles/tablets as well
- C - Arduino and other electronic firmware ie RoboClaw Motor Shield etc
- C++ - Any apps I want to execute quickly but also give me the sanity that comes with object orientation, so any projects bigger than a few files I will be switching from C to C++ The OpenCV face detection stuff for example. Also any GUI apps that are cross platform/OS via Qt
- HTML - Any static web pages - ie web admin portals, for monitoring the robot IO via RESTful API on IIS
- SQL - All relational DB queries
- JSON/XML - Any data sent across the wire that needs to be slightly better than bytes, i.e. DB results, REST results, this lot will be tucked inside HTTP headers etc
- C# - Any research things I normally use this, as its super quick for prototyping, fast, and simple to write. I love C# (even though all it's ideas were stolen from JAVA !)
- Bash shell - programming scheduled jobs for linux will be done using scripts, I'll list them all here
- Python - for more complex scripting I'll use this hackers delight
All my source code will be available (except where licenses prevent), and if I decide not to show it due to it being very clever. I might offer some form of paywall in the future for commercial usage etc.
I've been writing software for 30 years now, from my first Atari 400 BASIC program to the latest C# back end web services for EA's UFC free to play, I've seen it all.
Don't Forget
Yes I'd like to thank my Mum, for having me, etc.Thank you to all those who worked so hard before I did to put down this great fabric of SDKs, APIs, tool kits, apps etc in both Open Source and Commercial markets.
Yes, what Mankind (not just men !!), has achieved is amazing in terms of software development, and I am stealing all of it from everywhere I can. So be aware - just like all those great big American companies, I too will be stealing ideas, and software to further my goals. My goals are not money and world dominance though, and I won't be lying to everyone telling them I did it. No, I will be simply trying to inspire people to get involved with technology in a hands on, anyone can do it way.
You see Apple with their iPhone didn't invent all that tech, they bought it together in a very clever way. Anyone can do that, and I am going to show you how. So try something yourself today, become a robot engineer !
