Encoder Front Page
SRS Home | Front Page | Monthly Issue | Index
Google
Search WWW Search seattlerobotics.org

The BallBot Project

by Dominic Ford


Section I: Introduction

1.1 Abstract

The problem with hitting a tennis ball away from you is that you often have to go and get it back afterwards! This causes long delays when tennis players are in training - if they practice serving 20 balls, they've then got 20 balls to go and pick up. This time could be much more productively spent if the players could consider their performance, and how it could be improved.

The delay also causes long, annoying, and tiring breaks in tennis matches. This can easily break a player's concentration, and certainly increases the time that the match takes (which in turn makes the players more tired).

There are of course, many existing solutions to this problem, such as:

1. Employing ballboys/girls.

All very well if you happen to be playing at a professional tournament, but most players don't have anyone who they can persuade to act as a ballboy for them.

2. Getting your dog to fetch the balls.

This means training your dog, and even then the most obedient of dogs can still become a nuisance if they get bored. Also, what about people who don't have dogs?

1.2 The BallBot Project

BallBot is a robot which aims to solve these problems by acting as an assistant to the players in a tennis match. It will go and get any stray balls and return them to the players, who can concentrate on the more important matter of their tennis. And unlike a dog, BallBot doesn't need training, and is never disobedient!

BallBot (Front View)

 1.3 How does BallBot work?

Unlike many similar robots, BallBot doesn't even need a human controller to drive it. The microcontroller which is at its heart captures frames from a camera on board the robot, and analyses them. It then locates any objects in these images which look like tennis balls, or which could be the net. It uses the location of the net to navigate around the court, and uses the location of tennis balls to know where it is aiming for.

When BallBot has finished collecting tennis balls from the court - either when it is full or when it cannot see any more - it dumps all the balls that it has collected in a neat pile, and moves away from the court surface. This ensures that it does not get in the way of the match.

Section II: The Development of the project

2.1 The initial idea

As it happens, a tennis court is an ideal environment for a robot to work in. It is one of the few cases where relatively controlled conditions are founded outside a laboratory. The court will always be flat; it will have the same layout and dimensions; and it will be for the most part uncluttered.

Firstly, I began mapping a general outline of how the robot could work. We would need a rover to run around the court collecting balls. In order to find the balls, though, it needs to be able to detect them. Therefore the rover will need a sensor capable of locating tennis balls. We will need a computer on board to read the sensors and use the information to control the rover's movements. And to do this task effectively, the computer will need a substantial amount of programming. Remember - there's a net across the middle of the court that cannot be crossed without ending up in a tangled mess. But the user will want BallBot to collect all balls, regardless of which side of the net they happen to be on, so it must be programmed to work it's way around the edge of the net when necessary.

I found that the way to cope with the huge job ahead was to split it down into the smallest blocks possible and take it bit by bit. It is also a good idea to do the hardest work first. That way, if you get stuck and have to give up or radically reshape your plans, you haven't wasted too much of your time.

I figured the most challenging part would be the computerised navigation and ball sensing, so I started work here. I also figured that development time on tennis courts would probably be quite limited, so the more work I could do at home, the better. For this purpose, I developed a computer simulation of a tennis court. My PC emulated a robot on a tennis court surrounded by tennis balls, and a separate navigation module had to move the robot around the court picking up the tennis balls. The navigation code was only allowed to access selected data from the simulation - similar to the data it would be able to collect in reality - making the simulation as true to life as possible. This did not help me with the tennis ball sensing routines, but it did mean that I could work on the navigation algorithms in the comfort of my own home.

This simulation is available to download with instructions. The simulation is programmed with the AI algorithms, which means that you can watch BallBot in action on the virtual court. Requires MS Windows 3.1 or later. File size: 26K (zipped). DOWNLOAD

2.2 The Artificial Intelligence Algorithms

I started out with the simplest case. If there's a ball straight ahead, go and get it. If there's a ball to the left or right, then turn towards it and then head for it. Then there's the case where there aren't any balls present in our 30 degree field of vision. In that case, turn either left or right, and carry on turning until a ball comes into sight. If the robot turns through 360 degrees without seeing any tennis balls, we have finished, because there are no balls in any direction. This is illustrated in detail below:

  1. BallBot rotates until a ball comes into view straight ahead.
  2. BallBot moves forward, and so aims towards that ball in order to collect it.
  3. Continue straight ahead until the ball has been collected.
  4. Repeat this and collect any more balls.
  • If BallBot rotates through 360o without seeing any balls ahead of it, and which are in front of the net, there are no balls left on this side. There are failure cases such as when a ball is behind one of the players or in the centre of BallBot's circular path, as shown.

So far we have a simple method by which we can be sure of collecting all tennis balls providing that there are no obstacles and that we can see all tennis balls within our 30 degree field of vision. Next, we must take into account the one important obstacle that we will meet on a tennis court: the net. BallBot has to collect all tennis balls on both sides of the net, but never actually collide with it. I figured that the easiest way of achieving this was to split the task into several phases:

  1. Get all the tennis balls on the same side of the net as the robot (as detailed above).
  2. Steer round the net onto the other side of the court.
  3. Get all the tennis balls on this side of the net (i.e. repeat the process detailed above all over again).
  4. Drive off the court (to avoid getting in the way of the game).
  5. Finished!

Phases 1 and 3 are simply achieved by the method detailed in the first paragraph. Rotate until you see a tennis ball, and when you do, go and get it. When you manage to steer through 360 degrees without seeing a ball, move into the next phase of the process. To avoid colliding with the net, any balls which are seen but which have the net in front of them must be ignored. That is, BallBot only responds to tennis balls so long as they appear in front of the net.

Phase 2 is achieved as follows:

  1. BallBot rotates until a net post comes into view just to the right of centre (i.e. it is heading for the net post, but will just pass to the left of it).
  2. BallBot heads straight forwards, towards the end of the net.
  3. When the net gets very close, BallBot manoeuvres around the net post, and is then ready to repeat the search process on the other side of the net.

As it happens, when phase 2 completes itself, the robot is invariably just off the side of the court. So it is in fact also a suitable method of achieving phase 4 as well.

That's the net dealt with. There will, of course, be other obstacles present, such as players and any kit that they leave lying around. I decided that these could probably be ignored - BallBot will not head forwards and collide with anything unless it can see something straight ahead to go for. If there's a player or a bag in the way, this will block the view, and BallBot won't see anything worth going for. Therefore it won't go ahead and won't collide with the obstacle. So, given that we can see tennis balls and the net within our field of vision, we have a fairly complete method of collecting tennis balls from the court surface.

I would suggest at this point that if you haven't already done so, you DOWNLOAD the simulation. Here you can see the algorithms that I have just described, in action.

2.3 Seeing Tennis Balls

The Principle of ultrasonic detection

With the AI hurdle out of the way, I moved onto how we would actually 'see' tennis balls and the net. Ultrasound seemed to be a popular method for achieving robotic vision, so I decided to give it a go. The principle here is that you point both an ultrasonic transmitter and a receiver at the given target, and then send a pulse from the transmitter. This is reflected off the target and picked up by the receiver. The time delay gives your distance; the strength and waveform the type of target hit.

I bought a standard 40kHz ultrasonic transmitter/receiver set, and wired them up. I wired the transmitter to a 40kHz wave generator, with a computer controlled switch. The receiver I wired into an oscilloscope. The computer created a 1ms pulse to the transmitter, and the oscilloscope simultaneously scanned for a response. To start with, I pointed the sensor at a wall about a metre away. The response I got was along the lines of:

 

This looked very promising. The echo from the wall gave a measurable response at 4 foot, and would probably continue to do so at the kind of distance we would be looking at using across a tennis court. The next step was to replace the wall with a genuine tennis ball. I immediately hit trouble here, as the response was much weaker. Still, with a bit of amplification, I got the following response at 2 feet:

At 4 foot, though, the response was so weak that it could not be distinguished from the background noise. I had a word with a sound engineer about this - I was wondering if I could increase the power of ultrasound transmitted to get a better response. He pointed out that tennis balls were in fact about the worst targets I could have chosen for sonar:

All in all, sonar wasn't looking promising, so I started looking around for other ways of detecting tennis balls.

2.4 Visual Analysis

I decided that I would have to make the ambitious move of trying to write a visual analysis routine to actually detect tennis balls given data from a camera. The use of a visual input would also make the hardware requirements much more difficult to meet. We would need a video digitiser to interface the analogue video signal from the camera to the computer. First of all, I went out on a tennis court, and took a few photos to work from. Here's a good (i.e. very tough to deal with) example:

There's a lot of irrelevant data in the image: cars and buildings in the background and stones in the foreground. I won't go into detail as to how I managed to distinguish tennis balls from the rest of the image, because that's enough for an Encoder article in it's own right (an article which you will find on this issue, as it happens). Essentially, tennis balls appear as light coloured circles, and these are what the analysis routines look for.

2.5 The Hardware

The program algorithm was now completed, so the next step was to select a microcontroller to run the program on. The algorithm would then need translating into a language which the microcontroller could interpret. The specifications I required were as follows:

I had a careful look at all of the microcontrollers on the market, and even considered ripping the inside of a Commodore 64 out and using that (with the program code on an EPROM). Eventually I got in touch with a microcontroller expert who recommended Triangle Digital Services (www.triangledigital.com). They are rather pricey in places, but I found that what I got was well worth the cost. The controller was designed for use with a wide range of batteries, and draws a mere 15mA. It also has ample input/output lines which seem fairly durable (I know from the various wiring accidents I subjected them to!)  

The TDS9090 microcontroller

The motor controller I built myself - I used one output line from the microcontroller to switch the left/right drive motors, with a third to control forwards/reverse:

Above: How to link a relay to a microcontroller output. Right: This circuit built onto a board in BallBot.

 The video digitiser posed a little more of a problem. Triangle Digital didn't sell anything that quite matched the job, and I couldn't find any digitisers elsewhere which could easily be adapted to interface with a microcontroller of any description. Triangle Digital were, however, interested by the idea of a video digitiser compatible with their cards, and they agreed that if I could develop one myself and market it, they would help to promote it. So, I set about the ambitious task of developing my own microcontroller video digitiser. I got together with a couple of electrical engineers and we got to work sketching block diagrams and transferring them into CAD software. Six months later, and we had a working circuit diagram which we have produced, and which will shortly be going on the market. More details on this are again in the visual techniques article on this Encoder.

The diagram to the right shows the overall plan BallBot's computer system. The three cards are built separately and interconnected via a motherboard to allow them to be detached and tested separately when necessary. The motherboard simply consists of three inter-connected card sockets and a serial socket/power supply linked to the relevant pins of the card sockets.

 

2.6 Building the rover

At this stage, I had a computer and programming which could, in theory, act as the tennis ball collector that I had initially thought up. Now all I needed was a buggy that it could drive around on! As a prototype I used a Lego buggy (pictured to the right). I could continue to develop the robot, and test my conversion of the program code to the microcontroller without needing a tennis court. Before long, I had a reduced scale model of BallBot running on the floor, which headed towards tennis balls (it couldn't collect the things because they were about the same size as the rover!)

The Lego Prototype of BallBot (the computer wasn't mounted on board at this stage).

So I moved into the final stage - building a large rover which could house the computer and drive around a tennis court at a fair speed. I was lucky enough to know a mechanical engineer who recommended the exact part numbers for the motors I would be wanting. For those who are interested, they are (from RS components):

255-9598 12V Steel geared motor, capable of providing an excellent torque.
595-053 12V 3.5Ah Rechargeable Lead-Acid cell.
387-969 Rubber drive wheels (125mm diameter) giving a good grip.

From here onwards it was fairly simple. I welded together a small cage for holding collected tennis balls (large enough to hold about eight), and used a couple of brass bushes to mount the drive wheels/axles onto the front. I screwed a caster onto the back to act as a third slave wheel. This is shown more clearly in the design plans below.

There was some dispute between the various people who were helping work on the BallBot as to whether or not we needed any special system for collecting the tennis balls. Some people were keen on the idea of building a robotic arm to literally pick up the ball, while others were more keen on a flap on the front of the robot. BallBot would drive straight into the ball and then trap it behind this flap in the ball collection cage. Personally I was after simplicity, so I decided to try having no such mechanism at all. I just left a hole in the front of the rover leading into the collection cage. It does mean that if BallBot does an 'emergency stop' the balls will fly out the front, but that should never happen normally. The only such rapid stop that the robot will ever do is when it's finished, and we want the balls to come out then. In fact, I programmed BallBot to reverse when it's finished to make sure that they come out!

Section III: Design Plans

Below are design drawings for the frame of BallBot. All measurements given in mm:

3.1 Plan View

NB: Dark Grey = Axle mounting bush; Light Grey = Wheel (125mm diameter)

The white item mounted on the axle is a gear wheel.

3.2 Side View

3.3 Motor mounting diagram

The motor mounting (from above).

3.4 Physical Specifications

Physical Dimensions:

Length: 350mm; Width: 400mm; Height: 300mm.

Weight:

10kg

Maximum Speed:

0.8 m/s

Power Supply:

1 x PP3 (for electronics)

1 x 12V Lead/Acid (main motor power supply)

Section IV: Future Developments

The BallBot rover unit which we have built is very versatile. Its behaviour is entirely governed by how the microcontroller is programmed. This means that it can be reprogrammed easily to apply different visual analysis, or to respond differently to the objects that it detects. We are currently looking into whether there are any other tasks that our robot could be programmed to do, both as enhancements as a tennis assistant, and in completely new areas.

For example, it could be programmed with several tennis assisting functions as well as just collecting balls, and do each as and when it is requested by the user. It could even be programmed to perform a task completely unrelated to tennis.


Dominic Ford, August 1999.

Website: www.ballbot.mcmail.com

Comments? Email me on: dominic.ford@cwcom.net