2.1. Hardware of Vision System

The omnidirectional vision system of our robot consists of a camera (FLIR, Flea3), a varifocal lens (Vstone), and a hyperboloidal mirror (Vstone). We developed the vision system, shown in Figure 2, for the RoboCup MSL robot by combining the above elements. The image captured by this vision system is shown in Figure 3a. The image size and frame rate are 512 × 512 pixels and 30 frame per millisecond (fps), respectively.

Figure 2

Hardware of vision system.

Figure 3

Process of creating search model. (a) Original image from camera. (b) White detection image. (c) Orthogonalized image. (d) Search model.

2.2. Searching Model

Figure 3 shows the process of creating the search model for the proposed method. First, the detection image of the white line is needed to make the searching model. We obtain the white detection image by applying the converting method of color space from RGB to HSV and to YUV as in Figure 3b. Then we generate the field information by orthogonalizing the white line information, shown in Figure 3c. We determine the searching model by thinning down the field information based on the white lines as in Figure 3d. Finally, we use the thinned model as the searching model for self-localization.

2.3. Model-based Matching

The proposed self-localization method generates the search space by model-based matching between geometric information of the white lines on the MSL field and the above-mentioned searching model shown in Figure 4. The evaluation function  F(ϕ˜) {ϕ˜=[x˜, y˜, θ˜]} of the model-based matching is given as follows:

Figure 4

Model-based matching.

Here, the movement of the searching model S in the matching area is expressed as  S(ϕ˜). And, if the pixel value of field image corresponding to the area of the moving model is expressed as p(r˜){r˜=(x˜, y˜), r˜∈S(ϕ˜)}. The fitness function F(ϕ˜) obtains the maximum value when the position of the searching model corresponds to the correct position of the robot on the MSL field. Then, the task of detecting robot position is converted to a search task of ϕ˜  such that F(ϕ˜) is maximized.

2.4. Genetic Algorithm

In the proposed self-localization method, we implement a GA to determine the maximum value of the fitness function F(ϕ˜) . A GA is an example of an artificial intelligence program and is well known as a parallel search and optimization process that mimics natural selection and evolution. The proposed method utilizes an elitist model of a GA that preserves the best individual in the population at every generation. The method also utilizes genetic coding using gray code, roulette selection, and one-point crossover. The parameters of the GA process were determined in previous experiments.

2.5. Verification Experiment

We performed a self-localization experiment to determine the error between the correct position and detected position on the MSL field to verify the effectiveness of the proposed method. In the verification experiment, the resulting maximum error was 36.1 cm and the average error was 12.8 cm. These errors are small compared to the field size, indicating that the self-localization could be detected with sufficiently high accuracy.

We conducted an additional verification experiment to confirm the smoothness of the recognized position in an actual environment, in which the robot moved along a test path on the field shown on the left side of Figure 5. The results of the self-localization are shown on the right side of the figure, where the black dots represent the detected position at each camera frame. We also recorded the x, y-coordinates of the detected position at each frame number on the test path, shown in Figure 6. The results show that the self-localization result did not deviate significantly, and the robot’s coordinates of its detected position are continuous.

Figure 5

Result of self-localization on test path.

Figure 6

Coordinates of detected position at each frame number.

3.1. Experiment

In a real environment, a variety of noises may appear in the image, so qualitative experiments cannot be performed. Here we perform verification experiments by adding artificial noise.

We conducted the experiments at the seven locations on the MSL field indicated in Figure 7. These seven locations are characteristic points on the field. Taking the center of the panoramic image as the center of the circle, we set the fan-shaped area with a center angle of 30° as noise, shown in Figure 8. By changing the position and amount of noise, we can simulate different noise conditions and verify the results of self-localization. In the field of research using omnidirectional cameras, the verification method using fan shape noise as described above is often used [10].

Figure 7

Experiment location.

Figure 8

Varying amounts of noise.

Figure 8 shows the changes in the amount of noise at the position D in Figure 7, and the changes in angle are shown in Figure 9. The image in the left of Figure 8 is the original image without any noise. We verified the error in the self-localization results measured at the angle and number of various noise at each coordinate location on the MSL field. The results of each measurement position are summarized in Table 1. For the position deviation caused by the noise at different angles, we took the average value combined with the deviation value caused by the amount of noise. The bold text in the table indicates that the error value of the self-localization exceeds the width of a robot (50 cm), meaning that the robot has lost its exact position.

Figure 9

Noise at different angles.

3.2. Experiment Results

As shown in the results in Table 1, when the amount of noise increases, the self-localization error also increases. Generally, when the number of noises is greater than 7, it will cause large errors except for special locations, i.e., where there are more than two features in the image obtained by the robot (Locations B and F) and when the robot is in the center circle (Location C), shown in Figures 10 and 11, respectively. Locations E and G also have errors when the noise is low. This may be because the robot was on the vertical white line, and the noise obscured all of the white line information, shown in Figure 12. Overall, the results show that when the number of noise is less than 3, the self-localization error is very small, and when the noise increases to 4, the maximum self-localization error is 120 cm. In a real game environment, the probability of more than four noises existing at the same time is very low.

Figure 10

Eight noise patterns at position B.

Figure 11

10 noise patterns at position C.

Figure 12

Five noise patterns at position E.

4.1. Inheritance of Genetic Information

An example of the output fitness value from the general GA search is shown in Figure 13, where the vertical line represents the timing of frame-to-frame switching. In this figure, the fitness value drops significantly because the best individual is reset with every frame change and the search restarts from a very low initial fitness value. The proposed method uses the genetic information of the previous frame as the initial genetic information of the next frame at every frame change. The resulting output of the proposed method is shown in Figure 14. As seen in the figure, a high fitness value is maintained by inheriting the individual information.

Figure 13

Output of fitness value from general GA search.

Figure 14

Output of fitness value using proposed method.

4.2. Experiment Results

To verify the method’s robustness to moving speed, we conducted experiments in which we measured the robot’s fitness value and self-localization accuracy at different robot speeds. The robot moves forward on test paths A–C, shown in Figure 15, at a speed of 1–3 m/s.

Figure 15

Test paths A–C.

The result of test path A is shown in Figures 16 and 17. In Figure 16, the vertical axis represents the fitness value and the horizontal axis represents the distance from the starting point. To verify the relationship between accuracy and moving speed, we picked three areas surrounded by rectangles in Figure 16, which are rearranged in Figure 17. In Figure 17, I–III are the respective positions and frames at each speed.

Figure 16

Results of fitness value at each moving speed (test path A).

Figure 17

Self-localization error at each moving speed (test path A).

Here, the position error is represented with a bar graph. As shown in Figure 17, the self-localization error does not change greatly as moving speed changes. The maximum error is about 13 cm, which is small enough compared to the size of the MSL field. Test paths B and C yielded similar results to test path A; the maximum errors were 9.9 and 10.12 cm for B and C, respectively. This shows that the proposed method is robust against changes in the robot’s moving speed within 1–3 m/s.